2024-05-09T05:14:48.290 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T05:14:48.290 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120 --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} -- /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/orig.config.yaml 2024-05-09T05:14:48.351 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T05:14:48.446 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120 branch: wip-pdonnell-testing-20240508.183908-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: pdonnell@redhat.com first_in_suite: false job_id: '7699120' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240508.183908-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.183908-debug sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1722 sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.183908-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwLRy3NJu0QMR2Wrvxm2Ky4rCTroTs+v6A6dIq8ejOghgc2RO1q4cepqkmnVWhKWBTGFdIZVygQClYIjKB8Io4= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJmAfhVVJZsHL0RPD8MpyMI6DMOLMj4lsfWG9XVvbDFiRL68GpNLWjpCsRb0e5M9Q6xdVb3oDsnNWq2E3mNEX6A= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGYmgDvZ4wD6omeOjOM/UDHcKW+kBeWAQjQOII2hHZxS3AvuiuUyzfR8Ex3IsV3pCRmdEZXVcgc+njbCh7f23BQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_22:06:20 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T05:14:48.446 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa; will attempt to use it 2024-05-09T05:14:48.447 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks 2024-05-09T05:14:48.447 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T05:14:48.449 INFO:teuthology.task.internal:Checking packages... 2024-05-09T05:14:48.472 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c9da4049f638c0f84bdd9b4e33c42e86046ac554' 2024-05-09T05:14:48.472 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T05:14:48.472 INFO:teuthology.packaging:ref: None 2024-05-09T05:14:48.472 INFO:teuthology.packaging:tag: None 2024-05-09T05:14:48.472 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.183908-debug 2024-05-09T05:14:48.472 INFO:teuthology.packaging:sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:14:48.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240508.183908-debug 2024-05-09T05:14:48.619 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3594.gc9da4049 2024-05-09T05:14:48.621 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T05:14:48.627 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T05:14:48.627 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T05:14:48.633 INFO:teuthology.task.internal:Saving configuration 2024-05-09T05:14:48.654 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T05:14:48.660 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T05:14:48.684 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 05:09:43.333263', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwLRy3NJu0QMR2Wrvxm2Ky4rCTroTs+v6A6dIq8ejOghgc2RO1q4cepqkmnVWhKWBTGFdIZVygQClYIjKB8Io4='} 2024-05-09T05:14:48.709 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 05:09:43.331219', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJmAfhVVJZsHL0RPD8MpyMI6DMOLMj4lsfWG9XVvbDFiRL68GpNLWjpCsRb0e5M9Q6xdVb3oDsnNWq2E3mNEX6A='} 2024-05-09T05:14:48.733 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 05:09:43.332511', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGYmgDvZ4wD6omeOjOM/UDHcKW+kBeWAQjQOII2hHZxS3AvuiuUyzfR8Ex3IsV3pCRmdEZXVcgc+njbCh7f23BQ='} 2024-05-09T05:14:48.733 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T05:14:48.739 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-09T05:14:48.739 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-09T05:14:48.739 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-09T05:14:48.739 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T05:14:48.843 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4b122e90d0>, signals=[15]) 2024-05-09T05:14:48.843 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T05:14:48.848 INFO:teuthology.task.internal:Opening connections... 2024-05-09T05:14:48.849 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T05:14:48.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:14:48.926 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-05-09T05:14:48.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:14:49.001 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-05-09T05:14:49.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:14:49.077 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T05:14:49.083 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-05-09T05:14:49.101 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-05-09T05:14:49.101 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-05-09T05:14:49.158 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-05-09T05:14:49.158 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-05-09T05:14:49.158 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-05-09T05:14:49.158 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T05:14:49.159 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T05:14:49.160 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-05-09T05:14:49.180 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-05-09T05:14:49.197 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-05-09T05:14:49.198 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-05-09T05:14:49.255 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T05:14:49.256 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-05-09T05:14:49.256 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T05:14:49.256 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T05:14:49.256 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T05:14:49.256 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-05-09T05:14:49.279 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-05-09T05:14:49.297 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-05-09T05:14:49.298 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-05-09T05:14:49.354 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2024-05-09T05:14:49.354 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9" 2024-05-09T05:14:49.354 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2024-05-09T05:14:49.354 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2024-05-09T05:14:49.354 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9" 2024-05-09T05:14:49.355 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9" 2024-05-09T05:14:49.355 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T05:14:49.355 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2024-05-09T05:14:49.355 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon" 2024-05-09T05:14:49.355 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T05:14:49.355 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2024-05-09T05:14:49.355 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T05:14:49.355 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T05:14:49.355 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T05:14:49.355 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-05-09T05:14:49.379 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T05:14:49.389 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T05:14:49.395 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T05:14:49.395 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-05-09T05:14:49.397 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-05-09T05:14:49.400 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-05-09T05:14:49.415 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T05:14:49.421 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T05:14:49.421 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-05-09T05:14:49.454 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-05-09T05:14:49.456 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-05-09T05:14:49.589 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T05:14:49.664 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T05:14:49.681 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-09T05:14:49.682 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-09T05:14:49.682 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T05:14:49.683 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T05:14:49.683 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T05:14:49.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:14:49.700 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-05-09T05:14:49.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:14:49.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:14:49.702 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-05-09T05:14:49.702 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-05-09T05:14:49.757 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-437.el9.x86_64 2024-05-09T05:14:49.757 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.14.0-437.el9.x86_64 2024-05-09T05:14:49.758 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2024-05-09T05:14:49.760 INFO:teuthology.orchestra.run.smithi161.stdout:5.14.0-437.el9.x86_64 2024-05-09T05:14:49.760 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-437.el9.x86_64 2024-05-09T05:14:49.760 INFO:teuthology.task.kernel:Running kernel on smithi161: 5.14.0-437.el9.x86_64 2024-05-09T05:14:49.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum install -y kernel 2024-05-09T05:14:49.760 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.14.0-437.el9.x86_64 2024-05-09T05:14:49.760 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum install -y kernel 2024-05-09T05:14:52.112 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-09T05:14:53.229 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.1 MB 00:01 2024-05-09T05:14:53.587 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.1 MB 00:02 2024-05-09T05:14:54.971 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-09T05:14:55.546 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 19 MB 00:02 2024-05-09T05:14:55.718 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-05-09T05:14:59.780 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.2 MB 00:01 2024-05-09T05:15:00.044 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-05-09T05:15:00.772 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-05-09T05:15:01.570 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 16 kB 00:00 2024-05-09T05:15:01.598 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-09T05:15:02.122 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 16 kB 00:00 2024-05-09T05:15:04.326 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-09T05:15:04.771 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 21 MB 00:03 2024-05-09T05:15:07.390 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 3.7 MB/s | 21 MB 00:05 2024-05-09T05:15:09.464 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-05-09T05:15:09.921 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-09T05:15:11.504 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T05:15:11.504 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T05:15:11.601 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-09T05:15:11.609 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-09T05:15:11.609 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-05-09T05:15:11.609 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-09T05:15:11.609 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-05-09T05:15:11.610 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:15:11.611 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 78 M 2024-05-09T05:15:11.611 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 123 M 2024-05-09T05:15:11.611 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-09T05:15:11.944 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T05:15:11.944 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T05:15:12.050 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout: Package Architecture Version Repository Size 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-05-09T05:15:12.063 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-05-09T05:15:12.064 INFO:teuthology.orchestra.run.smithi195.stdout:Install 4 Packages 2024-05-09T05:15:12.064 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:15:12.064 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 78 M 2024-05-09T05:15:12.064 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 123 M 2024-05-09T05:15:12.064 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-05-09T05:15:12.249 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 806 kB/s | 484 kB 00:00 2024-05-09T05:15:12.305 INFO:teuthology.orchestra.run.smithi195.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 4.9 MB/s | 484 kB 00:00 2024-05-09T05:15:12.540 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-09T05:15:12.898 INFO:teuthology.orchestra.run.smithi195.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-05-09T05:15:13.372 INFO:teuthology.orchestra.run.smithi195.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 29 MB/s | 34 MB 00:01 2024-05-09T05:15:13.673 INFO:teuthology.orchestra.run.smithi195.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 20 MB/s | 28 MB 00:01 2024-05-09T05:15:13.673 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-05-09T05:15:13.673 INFO:teuthology.orchestra.run.smithi195.stdout:Total 48 MB/s | 78 MB 00:01 2024-05-09T05:15:14.357 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-05-09T05:15:14.579 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T05:15:14.579 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T05:15:14.673 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T05:15:14.691 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:15:14.692 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-05-09T05:15:14.692 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-05-09T05:15:14.692 INFO:teuthology.orchestra.run.smithi161.stdout:Install 4 Packages 2024-05-09T05:15:14.692 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:15:14.692 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 78 M 2024-05-09T05:15:14.692 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 123 M 2024-05-09T05:15:14.692 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-05-09T05:15:14.719 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-05-09T05:15:14.719 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-05-09T05:15:15.024 INFO:teuthology.orchestra.run.smithi161.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.6 MB/s | 484 kB 00:00 2024-05-09T05:15:15.645 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-05-09T05:15:15.645 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-05-09T05:15:15.850 INFO:teuthology.orchestra.run.smithi161.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 15 MB/s | 15 MB 00:01 2024-05-09T05:15:16.342 INFO:teuthology.orchestra.run.smithi161.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 21 MB/s | 28 MB 00:01 2024-05-09T05:15:17.908 INFO:teuthology.orchestra.run.smithi161.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 11 MB/s | 34 MB 00:03 2024-05-09T05:15:17.909 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-05-09T05:15:17.909 INFO:teuthology.orchestra.run.smithi161.stdout:Total 24 MB/s | 78 MB 00:03 2024-05-09T05:15:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-05-09T05:15:18.589 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-05-09T05:15:18.693 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T05:15:18.773 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T05:15:18.937 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-05-09T05:15:18.937 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-05-09T05:15:19.206 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 2.0 MB/s | 15 MB 00:07 2024-05-09T05:15:19.870 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-05-09T05:15:19.871 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-05-09T05:15:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T05:15:21.560 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T05:15:22.291 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-05-09T05:15:23.012 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T05:15:23.431 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T05:15:25.789 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T05:15:25.912 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T05:15:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T05:15:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:15:30.640 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T05:15:30.781 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:15:31.082 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:15:35.589 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:15:41.435 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 981 kB/s | 28 MB 00:29 2024-05-09T05:15:44.460 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 1.0 MB/s | 34 MB 00:32 2024-05-09T05:15:44.460 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-09T05:15:44.460 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.4 MB/s | 78 MB 00:32 2024-05-09T05:15:45.141 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-09T05:15:45.532 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-09T05:15:45.532 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-09T05:15:46.434 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-09T05:15:46.435 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-09T05:15:48.904 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-09T05:15:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T05:15:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T05:15:52.336 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T05:15:52.472 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T05:15:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T05:15:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:00.512 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:00.551 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:00.889 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:00.890 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T05:16:00.890 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T05:16:00.890 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T05:16:01.686 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:01.687 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:16:01.687 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-05-09T05:16:01.687 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T05:16:01.687 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T05:16:01.687 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T05:16:01.687 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T05:16:01.687 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:16:01.687 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-05-09T05:16:02.039 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T05:16:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:02.367 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T05:16:02.368 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T05:16:02.368 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T05:16:02.368 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T05:16:02.368 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T05:16:02.368 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-05-09T05:16:02.368 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T05:16:02.369 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T05:16:02.419 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T05:16:02.419 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T05:16:02.419 DEBUG:teuthology.orchestra.run.smithi195:> sudo rpm -qi grub2-tools 2024-05-09T05:16:02.449 INFO:teuthology.orchestra.run.smithi195.stdout:Name : grub2-tools 2024-05-09T05:16:02.449 INFO:teuthology.orchestra.run.smithi195.stdout:Epoch : 1 2024-05-09T05:16:02.449 INFO:teuthology.orchestra.run.smithi195.stdout:Version : 2.06 2024-05-09T05:16:02.449 INFO:teuthology.orchestra.run.smithi195.stdout:Release : 68.el9 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: x86_64 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Group : Unspecified 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Size : 8264254 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:License : GPLv3+ 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Packager : builder@centos.org 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Vendor : CentOS 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Summary : Support tools for GRUB. 2024-05-09T05:16:02.450 INFO:teuthology.orchestra.run.smithi195.stdout:Description : 2024-05-09T05:16:02.451 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:16:02.451 INFO:teuthology.orchestra.run.smithi195.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T05:16:02.451 INFO:teuthology.orchestra.run.smithi195.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T05:16:02.451 INFO:teuthology.orchestra.run.smithi195.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T05:16:02.451 INFO:teuthology.orchestra.run.smithi195.stdout:hardware devices. 2024-05-09T05:16:02.451 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:16:02.451 INFO:teuthology.orchestra.run.smithi195.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T05:16:02.452 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T05:16:02.452 INFO:teuthology.task.kernel:Updating grub on smithi195 to boot 5.14.0-444.el9.x86_64 2024-05-09T05:16:02.452 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T05:16:03.164 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-05-09T05:16:04.559 INFO:teuthology.orchestra.run.smithi195.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T05:16:04.582 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-05-09T05:16:04.584 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-05-09T05:16:04.605 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.HHufn0Jcry 2024-05-09T05:16:04.605 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HHufn0Jcry 2024-05-09T05:16:04.679 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.HHufn0Jcry 2024-05-09T05:16:04.817 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.HHufn0Jcry is 6KB 2024-05-09T05:16:04.866 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.HHufn0Jcry 2024-05-09T05:16:04.881 DEBUG:teuthology.orchestra.run.smithi195:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T05:16:04.935 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:04.942 INFO:teuthology.orchestra.run.smithi195.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T05:16:04.942 INFO:teuthology.orchestra.run.smithi195.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T05:16:04.942 INFO:teuthology.orchestra.run.smithi195.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T05:16:04.943 INFO:teuthology.orchestra.run.smithi195.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T05:16:04.944 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T05:16:04.973 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:05.075 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-05-09T05:16:05.337 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:05.337 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T05:16:05.337 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T05:16:05.337 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T05:16:06.251 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:06.251 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:16:06.251 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-05-09T05:16:06.251 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T05:16:06.251 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T05:16:06.251 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T05:16:06.251 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T05:16:06.251 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:16:06.251 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-05-09T05:16:06.596 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T05:16:06.674 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T05:16:06.674 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T05:16:06.674 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T05:16:06.674 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T05:16:06.674 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T05:16:06.674 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-09T05:16:06.674 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T05:16:06.674 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T05:16:06.724 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T05:16:06.724 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T05:16:06.724 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -qi grub2-tools 2024-05-09T05:16:06.757 INFO:teuthology.orchestra.run.smithi161.stdout:Name : grub2-tools 2024-05-09T05:16:06.757 INFO:teuthology.orchestra.run.smithi161.stdout:Epoch : 1 2024-05-09T05:16:06.757 INFO:teuthology.orchestra.run.smithi161.stdout:Version : 2.06 2024-05-09T05:16:06.757 INFO:teuthology.orchestra.run.smithi161.stdout:Release : 68.el9 2024-05-09T05:16:06.757 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: x86_64 2024-05-09T05:16:06.757 INFO:teuthology.orchestra.run.smithi161.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Group : Unspecified 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Size : 8264254 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:License : GPLv3+ 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Packager : builder@centos.org 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Vendor : CentOS 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Summary : Support tools for GRUB. 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Description : 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:16:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T05:16:06.759 INFO:teuthology.orchestra.run.smithi161.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T05:16:06.759 INFO:teuthology.orchestra.run.smithi161.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T05:16:06.759 INFO:teuthology.orchestra.run.smithi161.stdout:hardware devices. 2024-05-09T05:16:06.759 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:16:06.759 INFO:teuthology.orchestra.run.smithi161.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T05:16:06.760 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T05:16:06.760 INFO:teuthology.task.kernel:Updating grub on smithi161 to boot 5.14.0-444.el9.x86_64 2024-05-09T05:16:06.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T05:16:07.339 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2024-05-09T05:16:08.692 INFO:teuthology.orchestra.run.smithi161.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T05:16:08.710 INFO:teuthology.orchestra.run.smithi161.stderr:done 2024-05-09T05:16:08.713 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-05-09T05:16:08.734 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.8plsjZfu5I 2024-05-09T05:16:08.734 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8plsjZfu5I 2024-05-09T05:16:08.800 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.8plsjZfu5I 2024-05-09T05:16:08.934 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.8plsjZfu5I is 6KB 2024-05-09T05:16:08.982 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.8plsjZfu5I 2024-05-09T05:16:08.997 DEBUG:teuthology.orchestra.run.smithi161:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T05:16:09.058 INFO:teuthology.orchestra.run.smithi161.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T05:16:09.058 INFO:teuthology.orchestra.run.smithi161.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T05:16:09.058 INFO:teuthology.orchestra.run.smithi161.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T05:16:09.058 INFO:teuthology.orchestra.run.smithi161.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T05:16:09.059 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T05:16:09.209 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2024-05-09T05:16:31.347 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:31.385 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T05:16:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T05:16:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T05:16:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T05:16:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:16:32.528 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-09T05:16:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T05:16:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T05:16:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T05:16:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T05:16:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:16:32.529 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-09T05:16:32.899 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T05:16:32.990 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T05:16:32.990 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T05:16:32.991 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T05:16:32.991 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T05:16:32.991 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T05:16:32.991 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-09T05:16:32.991 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T05:16:32.991 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T05:16:33.040 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T05:16:33.041 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T05:16:33.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2024-05-09T05:16:33.071 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2024-05-09T05:16:33.071 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 68.el9 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8264254 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2024-05-09T05:16:33.072 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2024-05-09T05:16:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:16:33.073 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T05:16:33.073 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T05:16:33.073 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T05:16:33.073 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2024-05-09T05:16:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:16:33.073 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T05:16:33.074 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T05:16:33.074 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 5.14.0-444.el9.x86_64 2024-05-09T05:16:33.074 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T05:16:33.741 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-05-09T05:16:35.079 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-09T05:16:35.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-09T05:16:35.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:16:35.140 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T05:16:35.171 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-05-09T05:16:35.173 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-05-09T05:16:35.188 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.fpXQfVu9Np 2024-05-09T05:16:35.188 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fpXQfVu9Np 2024-05-09T05:16:35.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.fpXQfVu9Np 2024-05-09T05:16:35.391 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.fpXQfVu9Np is 6KB 2024-05-09T05:16:35.440 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.fpXQfVu9Np 2024-05-09T05:16:35.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T05:16:35.525 INFO:teuthology.orchestra.run.smithi115.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T05:16:35.525 INFO:teuthology.orchestra.run.smithi115.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T05:16:35.525 INFO:teuthology.orchestra.run.smithi115.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T05:16:35.525 INFO:teuthology.orchestra.run.smithi115.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T05:16:35.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T05:16:35.683 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-05-09T05:16:39.214 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-09T05:16:39.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-09T05:16:39.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:16:53.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-05-09T05:16:57.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-05-09T05:17:02.412 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-09T05:17:02.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:17:05.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-05-09T05:17:05.686 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-09T05:17:05.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-09T05:17:05.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:17:06.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-09T05:17:06.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:17:09.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-05-09T05:17:17.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-09T05:17:17.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:17:21.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-09T05:17:21.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:17:24.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-05-09T05:17:33.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-09T05:17:33.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:17:36.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-05-09T05:17:40.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-05-09T05:17:48.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-09T05:17:48.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:17:49.314 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-05-09T05:17:49.680 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-09T05:17:49.680 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-444.el9.x86_64"... 2024-05-09T05:17:49.680 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-05-09T05:17:49.695 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-444.el9.x86_64 2024-05-09T05:17:49.696 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T05:17:49.696 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T05:17:49.696 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T05:17:50.696 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-09T05:17:50.697 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T05:17:50.768 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2024-05-09T05:17:50.805 DEBUG:teuthology.parallel:result is None 2024-05-09T05:17:55.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-09T05:17:55.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:17:55.754 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-05-09T05:17:56.158 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-09T05:17:56.159 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-09T05:17:56.159 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-05-09T05:17:56.173 INFO:teuthology.orchestra.run.smithi161.stdout:5.14.0-444.el9.x86_64 2024-05-09T05:17:56.174 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T05:17:56.174 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T05:17:56.174 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T05:17:57.175 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-09T05:17:57.175 DEBUG:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T05:17:57.252 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2024-05-09T05:17:57.287 DEBUG:teuthology.parallel:result is None 2024-05-09T05:18:06.626 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-05-09T05:18:21.628 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-09T05:18:21.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:18:21.959 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-05-09T05:18:22.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-09T05:18:22.457 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-09T05:18:22.457 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-05-09T05:18:22.472 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-444.el9.x86_64 2024-05-09T05:18:22.473 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T05:18:22.473 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T05:18:22.473 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T05:18:23.473 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-09T05:18:23.474 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T05:18:23.551 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-05-09T05:18:23.585 DEBUG:teuthology.parallel:result is None 2024-05-09T05:18:23.586 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T05:18:23.592 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T05:18:23.593 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T05:18:23.595 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T05:18:23.598 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T05:18:23.618 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T05:18:23.625 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T05:18:23.634 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T05:18:23.634 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T05:18:23.650 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T05:18:23.659 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T05:18:23.780 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T05:18:23.787 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T05:18:23.787 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T05:18:23.789 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T05:18:23.792 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T05:18:23.821 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T05:18:23.822 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T05:18:23.829 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T05:18:23.833 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T05:18:23.833 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T05:18:23.840 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T05:18:23.841 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T05:18:23.848 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T05:18:23.848 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T05:18:23.877 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T05:18:23.879 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T05:18:23.909 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T05:18:23.918 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T05:18:23.919 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T05:18:23.943 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T05:18:23.947 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T05:18:23.967 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T05:18:24.039 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T05:18:24.086 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T05:18:24.160 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T05:18:24.208 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:18:24.208 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T05:18:24.281 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T05:18:24.321 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T05:18:24.379 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T05:18:24.463 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T05:18:24.512 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:18:24.513 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T05:18:24.580 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T05:18:24.621 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T05:18:24.681 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T05:18:24.760 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T05:18:24.812 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:18:24.812 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T05:18:24.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-05-09T05:18:24.883 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-05-09T05:18:24.886 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-05-09T05:18:24.921 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T05:18:24.925 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T05:18:24.965 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T05:18:25.416 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T05:18:25.424 INFO:teuthology.task.internal:Starting timer... 2024-05-09T05:18:25.424 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T05:18:25.446 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T05:18:25.457 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T05:18:25.457 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-05-09T05:18:25.508 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-05-09T05:18:25.510 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-05-09T05:18:25.564 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-05-09T05:18:25.566 DEBUG:teuthology.orchestra.run.smithi195:> sudo service auditd rotate 2024-05-09T05:18:25.640 INFO:teuthology.orchestra.run.smithi195.stdout:Rotating logs: 2024-05-09T05:18:25.643 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T05:18:25.645 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-05-09T05:18:25.669 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-05-09T05:18:25.669 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-05-09T05:18:25.705 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-05-09T05:18:25.705 DEBUG:teuthology.orchestra.run.smithi195:> /usr/sbin/getenforce 2024-05-09T05:18:25.758 INFO:teuthology.orchestra.run.smithi195.stdout:Permissive 2024-05-09T05:18:25.758 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi195.front.sepia.ceph.com': 'permissive'} 2024-05-09T05:18:25.758 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T05:18:25.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:18:25.783 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T05:18:25.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:18:25.813 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T05:18:25.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:18:25.843 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T05:18:25.843 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-05-09T05:18:25.869 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-05-09T05:18:25.897 DEBUG:teuthology.orchestra.run.smithi195:> sudo /usr/sbin/setenforce permissive 2024-05-09T05:18:25.926 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T05:18:25.936 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T05:18:25.954 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T05:18:25.956 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi115.front.sepia.ceph.com,smithi161.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T05:22:31.756 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-05-09T05:22:31.757 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-09T05:22:31.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:22:31.846 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-05-09T05:22:31.929 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-09T05:22:31.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-09T05:22:31.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:22:32.003 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-05-09T05:22:32.042 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-09T05:22:32.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-09T05:22:32.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T05:22:32.122 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-05-09T05:22:32.203 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-05-09T05:22:32.203 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T05:22:32.214 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T05:22:32.214 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T05:22:32.214 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T05:22:32.217 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T05:22:32.218 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T05:22:32.220 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T05:22:32.220 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T05:22:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T05:22:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T05:22:32.260 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T05:22:32.260 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T05:22:32.274 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T05:22:32.284 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-05-09T05:22:32.284 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-05-09T05:22:32.288 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T05:22:32.295 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-05-09T05:22:32.295 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-05-09T05:22:32.309 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T05:22:32.310 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T05:22:32.313 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: ntpd: command not found 2024-05-09T05:22:32.323 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T05:22:32.323 INFO:teuthology.orchestra.run.smithi195.stdout:506 Cannot talk to daemon 2024-05-09T05:22:32.324 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T05:22:32.337 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T05:22:32.351 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T05:22:32.379 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-05-09T05:22:32.380 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-05-09T05:22:32.382 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T05:22:32.382 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-05-09T05:22:32.383 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.383 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.383 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.383 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.384 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T05:22:32.384 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-05-09T05:22:32.384 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.385 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.385 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.385 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.406 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-05-09T05:22:32.409 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T05:22:32.409 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-05-09T05:22:32.409 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.409 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.410 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.410 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:22:32.411 INFO:teuthology.run_tasks:Running task install... 2024-05-09T05:22:32.419 DEBUG:teuthology.task.install:project ceph 2024-05-09T05:22:32.419 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'}} 2024-05-09T05:22:32.419 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'} 2024-05-09T05:22:32.419 INFO:teuthology.task.install:Using flavor: default 2024-05-09T05:22:32.425 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T05:22:32.425 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T05:22:32.426 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T05:22:32.426 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:22:32.428 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T05:22:32.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:22:32.429 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T05:22:32.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:22:32.593 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T05:22:32.593 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T05:22:32.669 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T05:22:32.669 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T05:22:32.678 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T05:22:32.678 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T05:22:32.712 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T05:22:32.712 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:22:32.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T05:22:32.739 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T05:22:32.739 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T05:22:32.789 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T05:22:32.790 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:22:32.790 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T05:22:32.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T05:22:32.815 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T05:22:32.816 DEBUG:teuthology.orchestra.run.smithi195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T05:22:32.881 DEBUG:teuthology.orchestra.run.smithi195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T05:22:32.882 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T05:22:32.946 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-05-09T05:22:32.948 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-05-09T05:22:32.956 DEBUG:teuthology.orchestra.run.smithi195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T05:22:33.020 INFO:teuthology.orchestra.run.smithi195.stdout:check_obsoletes = 1 2024-05-09T05:22:33.022 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum clean all 2024-05-09T05:22:33.184 INFO:teuthology.orchestra.run.smithi115.stdout:54 files removed 2024-05-09T05:22:33.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T05:22:33.252 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T05:22:33.252 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:22:33.252 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T05:22:33.261 INFO:teuthology.orchestra.run.smithi195.stdout:54 files removed 2024-05-09T05:22:33.283 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T05:22:33.283 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T05:22:33.291 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T05:22:33.352 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T05:22:33.432 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T05:22:33.498 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-05-09T05:22:33.500 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-05-09T05:22:33.741 INFO:teuthology.orchestra.run.smithi161.stdout:54 files removed 2024-05-09T05:22:33.768 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T05:22:34.038 INFO:teuthology.orchestra.run.smithi195.stdout:ceph packages for x86_64 176 kB/s | 84 kB 00:00 2024-05-09T05:22:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 142 kB/s | 84 kB 00:00 2024-05-09T05:22:34.422 INFO:teuthology.orchestra.run.smithi195.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-05-09T05:22:34.491 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 187 kB/s | 84 kB 00:00 2024-05-09T05:22:34.761 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-05-09T05:22:34.858 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-05-09T05:22:35.031 INFO:teuthology.orchestra.run.smithi195.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-05-09T05:22:35.162 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-05-09T05:22:35.526 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-05-09T05:22:35.931 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.1 MB 00:00 2024-05-09T05:22:36.970 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-09T05:22:37.626 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-05-09T05:22:38.066 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-05-09T05:22:38.846 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-09T05:22:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-09T05:22:44.512 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-05-09T05:22:44.959 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.2 MB 00:03 2024-05-09T05:22:45.030 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.2 MB 00:02 2024-05-09T05:22:46.071 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 16 kB 00:00 2024-05-09T05:22:46.309 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 16 kB 00:00 2024-05-09T05:22:46.472 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-05-09T05:22:47.425 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-09T05:22:49.061 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 21 MB 00:02 2024-05-09T05:22:52.673 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-09T05:22:54.178 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-09T05:22:54.999 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T05:22:55.000 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T05:22:55.010 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T05:22:55.011 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T05:22:55.012 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T05:22:55.013 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T05:22:55.014 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T05:22:55.017 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T05:22:55.017 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T05:22:55.080 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-05-09T05:22:55.090 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-05-09T05:22:55.090 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-05-09T05:22:55.090 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-05-09T05:22:55.090 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-05-09T05:22:55.090 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T05:22:55.090 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T05:22:55.091 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T05:22:55.092 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T05:22:55.093 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T05:22:55.094 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T05:22:55.095 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T05:22:55.096 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T05:22:55.097 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T05:22:55.098 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T05:22:55.099 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T05:22:55.100 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T05:22:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout:Install 148 Packages 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 8 Packages 2024-05-09T05:22:55.102 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:22:55.103 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 219 M 2024-05-09T05:22:55.103 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-05-09T05:22:56.074 INFO:teuthology.orchestra.run.smithi161.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-09T05:22:56.283 INFO:teuthology.orchestra.run.smithi161.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 4.1 MB/s | 863 kB 00:00 2024-05-09T05:22:56.358 INFO:teuthology.orchestra.run.smithi161.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 1.9 MB/s | 147 kB 00:00 2024-05-09T05:22:56.467 INFO:teuthology.orchestra.run.smithi161.stdout:(4/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 9.6 MB/s | 5.4 MB 00:00 2024-05-09T05:22:56.520 INFO:teuthology.orchestra.run.smithi195.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T05:22:56.521 INFO:teuthology.orchestra.run.smithi195.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T05:22:56.532 INFO:teuthology.orchestra.run.smithi195.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T05:22:56.532 INFO:teuthology.orchestra.run.smithi195.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T05:22:56.533 INFO:teuthology.orchestra.run.smithi195.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T05:22:56.534 INFO:teuthology.orchestra.run.smithi195.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T05:22:56.536 INFO:teuthology.orchestra.run.smithi195.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T05:22:56.538 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T05:22:56.539 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T05:22:56.599 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-05-09T05:22:56.600 INFO:teuthology.orchestra.run.smithi161.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 11 MB/s | 1.5 MB 00:00 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T05:22:56.617 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T05:22:56.618 INFO:teuthology.orchestra.run.smithi195.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T05:22:56.619 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrading: 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T05:22:56.620 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T05:22:56.621 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T05:22:56.622 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T05:22:56.623 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T05:22:56.624 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T05:22:56.625 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T05:22:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T05:22:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T05:22:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout:Installing weak dependencies: 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout:Install 148 Packages 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrade 8 Packages 2024-05-09T05:22:56.629 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:22:56.630 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 219 M 2024-05-09T05:22:56.630 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-05-09T05:22:56.642 INFO:teuthology.orchestra.run.smithi161.stdout:(6/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 8.0 MB/s | 2.3 MB 00:00 2024-05-09T05:22:56.993 INFO:teuthology.orchestra.run.smithi161.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 12 MB/s | 4.9 MB 00:00 2024-05-09T05:22:57.738 INFO:teuthology.orchestra.run.smithi195.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-09T05:22:57.943 INFO:teuthology.orchestra.run.smithi161.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 11 MB/s | 10 MB 00:00 2024-05-09T05:22:58.026 INFO:teuthology.orchestra.run.smithi161.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 303 kB/s | 25 kB 00:00 2024-05-09T05:22:58.396 INFO:teuthology.orchestra.run.smithi195.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 1.3 MB/s | 863 kB 00:00 2024-05-09T05:22:58.572 INFO:teuthology.orchestra.run.smithi195.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 838 kB/s | 147 kB 00:00 2024-05-09T05:22:58.843 INFO:teuthology.orchestra.run.smithi161.stdout:(10/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 8.1 MB/s | 18 MB 00:02 2024-05-09T05:22:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 2.6 MB/s | 224 kB 00:00 2024-05-09T05:22:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 324 kB/s | 27 kB 00:00 2024-05-09T05:22:59.094 INFO:teuthology.orchestra.run.smithi161.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 381 kB/s | 32 kB 00:00 2024-05-09T05:22:59.328 INFO:teuthology.orchestra.run.smithi161.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 3.2 MB/s | 756 kB 00:00 2024-05-09T05:22:59.428 INFO:teuthology.orchestra.run.smithi161.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.5 MB/s | 158 kB 00:00 2024-05-09T05:22:59.520 INFO:teuthology.orchestra.run.smithi161.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.4 MB/s | 127 kB 00:00 2024-05-09T05:22:59.955 INFO:teuthology.orchestra.run.smithi195.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 1.6 MB/s | 2.3 MB 00:01 2024-05-09T05:22:59.970 INFO:teuthology.orchestra.run.smithi161.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 1.1 MB/s | 497 kB 00:00 2024-05-09T05:23:00.822 INFO:teuthology.orchestra.run.smithi195.stdout:(5/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-05-09T05:23:01.147 INFO:teuthology.orchestra.run.smithi195.stdout:(6/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 1.3 MB/s | 1.5 MB 00:01 2024-05-09T05:23:01.921 INFO:teuthology.orchestra.run.smithi161.stdout:(18/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 2.6 MB/s | 5.2 MB 00:01 2024-05-09T05:23:02.021 INFO:teuthology.orchestra.run.smithi161.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 452 kB/s | 45 kB 00:00 2024-05-09T05:23:02.121 INFO:teuthology.orchestra.run.smithi161.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.3 MB/s | 135 kB 00:00 2024-05-09T05:23:02.222 INFO:teuthology.orchestra.run.smithi161.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.6 MB/s | 162 kB 00:00 2024-05-09T05:23:02.405 INFO:teuthology.orchestra.run.smithi161.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 1.7 MB/s | 321 kB 00:00 2024-05-09T05:23:02.580 INFO:teuthology.orchestra.run.smithi161.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 1.7 MB/s | 301 kB 00:00 2024-05-09T05:23:02.664 INFO:teuthology.orchestra.run.smithi161.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.2 MB/s | 99 kB 00:00 2024-05-09T05:23:02.756 INFO:teuthology.orchestra.run.smithi161.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 935 kB/s | 86 kB 00:00 2024-05-09T05:23:03.664 INFO:teuthology.orchestra.run.smithi195.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 1.7 MB/s | 4.9 MB 00:02 2024-05-09T05:23:03.829 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 1.2 MB/s | 21 MB 00:17 2024-05-09T05:23:04.131 INFO:teuthology.orchestra.run.smithi161.stdout:(26/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 2.2 MB/s | 3.0 MB 00:01 2024-05-09T05:23:04.332 INFO:teuthology.orchestra.run.smithi161.stdout:(27/156): ceph-common-19.0.0-3594.gc9da4049.el9 2.3 MB/s | 20 MB 00:08 2024-05-09T05:23:04.357 INFO:teuthology.orchestra.run.smithi161.stdout:(28/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 750 kB/s | 169 kB 00:00 2024-05-09T05:23:04.440 INFO:teuthology.orchestra.run.smithi161.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 1.7 MB/s | 142 kB 00:00 2024-05-09T05:23:04.657 INFO:teuthology.orchestra.run.smithi161.stdout:(30/156): ceph-grafana-dashboards-19.0.0-3594.g 105 kB/s | 34 kB 00:00 2024-05-09T05:23:05.557 INFO:teuthology.orchestra.run.smithi161.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 3.2 MB/s | 3.5 MB 00:01 2024-05-09T05:23:05.724 INFO:teuthology.orchestra.run.smithi161.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 1.6 MB/s | 274 kB 00:00 2024-05-09T05:23:05.808 INFO:teuthology.orchestra.run.smithi161.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 592 kB/s | 49 kB 00:00 2024-05-09T05:23:05.908 INFO:teuthology.orchestra.run.smithi161.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 164 kB/s | 16 kB 00:00 2024-05-09T05:23:06.350 INFO:teuthology.orchestra.run.smithi161.stdout:(35/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 5.2 MB/s | 43 MB 00:08 2024-05-09T05:23:06.375 INFO:teuthology.orchestra.run.smithi161.stdout:(36/156): ceph-volume-19.0.0-3594.gc9da4049.el9 585 kB/s | 273 kB 00:00 2024-05-09T05:23:06.467 INFO:teuthology.orchestra.run.smithi161.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 199 kB/s | 18 kB 00:00 2024-05-09T05:23:06.543 INFO:teuthology.orchestra.run.smithi161.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 80 kB 00:00 2024-05-09T05:23:06.585 INFO:teuthology.orchestra.run.smithi161.stdout:(39/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 3.2 MB/s | 763 kB 00:00 2024-05-09T05:23:06.694 INFO:teuthology.orchestra.run.smithi161.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-05-09T05:23:06.835 INFO:teuthology.orchestra.run.smithi161.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-05-09T05:23:06.877 INFO:teuthology.orchestra.run.smithi161.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-05-09T05:23:06.969 INFO:teuthology.orchestra.run.smithi161.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-05-09T05:23:07.053 INFO:teuthology.orchestra.run.smithi161.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-09T05:23:07.145 INFO:teuthology.orchestra.run.smithi161.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 4.0 MB/s | 1.2 MB 00:00 2024-05-09T05:23:07.170 INFO:teuthology.orchestra.run.smithi161.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-05-09T05:23:07.262 INFO:teuthology.orchestra.run.smithi161.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 7.4 MB/s | 689 kB 00:00 2024-05-09T05:23:07.287 INFO:teuthology.orchestra.run.smithi161.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-05-09T05:23:07.346 INFO:teuthology.orchestra.run.smithi161.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-09T05:23:07.380 INFO:teuthology.orchestra.run.smithi161.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-05-09T05:23:07.413 INFO:teuthology.orchestra.run.smithi161.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-05-09T05:23:07.455 INFO:teuthology.orchestra.run.smithi161.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 785 kB/s | 33 kB 00:00 2024-05-09T05:23:07.481 INFO:teuthology.orchestra.run.smithi161.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-05-09T05:23:07.531 INFO:teuthology.orchestra.run.smithi161.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-09T05:23:07.573 INFO:teuthology.orchestra.run.smithi161.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-05-09T05:23:07.648 INFO:teuthology.orchestra.run.smithi161.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-05-09T05:23:07.690 INFO:teuthology.orchestra.run.smithi161.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-09T05:23:07.715 INFO:teuthology.orchestra.run.smithi161.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 678 kB/s | 45 kB 00:00 2024-05-09T05:23:07.807 INFO:teuthology.orchestra.run.smithi161.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-05-09T05:23:07.841 INFO:teuthology.orchestra.run.smithi161.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-05-09T05:23:07.883 INFO:teuthology.orchestra.run.smithi161.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 337 kB/s | 25 kB 00:00 2024-05-09T05:23:07.908 INFO:teuthology.orchestra.run.smithi161.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-05-09T05:23:07.938 INFO:teuthology.orchestra.run.smithi195.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 2.4 MB/s | 10 MB 00:04 2024-05-09T05:23:07.975 INFO:teuthology.orchestra.run.smithi161.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-09T05:23:08.000 INFO:teuthology.orchestra.run.smithi161.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-09T05:23:08.014 INFO:teuthology.orchestra.run.smithi195.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 336 kB/s | 25 kB 00:00 2024-05-09T05:23:08.042 INFO:teuthology.orchestra.run.smithi161.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 528 kB/s | 35 kB 00:00 2024-05-09T05:23:08.284 INFO:teuthology.orchestra.run.smithi161.stdout:(66/156): ceph-mgr-diskprediction-local-19.0.0- 2.0 MB/s | 7.4 MB 00:03 2024-05-09T05:23:08.385 INFO:teuthology.orchestra.run.smithi161.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-05-09T05:23:08.452 INFO:teuthology.orchestra.run.smithi161.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-09T05:23:08.519 INFO:teuthology.orchestra.run.smithi161.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-09T05:23:08.560 INFO:teuthology.orchestra.run.smithi161.stdout:(70/156): postgresql-13.14-1.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-05-09T05:23:08.603 INFO:teuthology.orchestra.run.smithi161.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 4.1 MB/s | 911 kB 00:00 2024-05-09T05:23:08.661 INFO:teuthology.orchestra.run.smithi161.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-09T05:23:08.711 INFO:teuthology.orchestra.run.smithi161.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-05-09T05:23:08.803 INFO:teuthology.orchestra.run.smithi161.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-09T05:23:08.895 INFO:teuthology.orchestra.run.smithi161.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-05-09T05:23:08.912 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-05-09T05:23:08.979 INFO:teuthology.orchestra.run.smithi161.stdout:(76/156): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-05-09T05:23:09.013 INFO:teuthology.orchestra.run.smithi161.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-05-09T05:23:09.063 INFO:teuthology.orchestra.run.smithi161.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-05-09T05:23:09.189 INFO:teuthology.orchestra.run.smithi161.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-05-09T05:23:09.222 INFO:teuthology.orchestra.run.smithi161.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 822 kB/s | 172 kB 00:00 2024-05-09T05:23:09.239 INFO:teuthology.orchestra.run.smithi161.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 197 kB/s | 35 kB 00:00 2024-05-09T05:23:09.306 INFO:teuthology.orchestra.run.smithi161.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-05-09T05:23:09.332 INFO:teuthology.orchestra.run.smithi161.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-09T05:23:09.382 INFO:teuthology.orchestra.run.smithi161.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-05-09T05:23:09.399 INFO:teuthology.orchestra.run.smithi161.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-05-09T05:23:09.507 INFO:teuthology.orchestra.run.smithi161.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-05-09T05:23:09.533 INFO:teuthology.orchestra.run.smithi161.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-05-09T05:23:09.558 INFO:teuthology.orchestra.run.smithi161.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-09T05:23:09.608 INFO:teuthology.orchestra.run.smithi161.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-05-09T05:23:09.633 INFO:teuthology.orchestra.run.smithi161.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-05-09T05:23:09.659 INFO:teuthology.orchestra.run.smithi161.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 533 kB/s | 54 kB 00:00 2024-05-09T05:23:09.709 INFO:teuthology.orchestra.run.smithi161.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-05-09T05:23:09.734 INFO:teuthology.orchestra.run.smithi161.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-05-09T05:23:09.785 INFO:teuthology.orchestra.run.smithi161.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-05-09T05:23:09.810 INFO:teuthology.orchestra.run.smithi161.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-09T05:23:09.860 INFO:teuthology.orchestra.run.smithi161.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-05-09T05:23:09.910 INFO:teuthology.orchestra.run.smithi161.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-09T05:23:09.936 INFO:teuthology.orchestra.run.smithi161.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 766 kB/s | 58 kB 00:00 2024-05-09T05:23:09.981 INFO:teuthology.orchestra.run.smithi195.stdout:(10/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 2.0 MB/s | 18 MB 00:08 2024-05-09T05:23:09.995 INFO:teuthology.orchestra.run.smithi161.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-05-09T05:23:10.020 INFO:teuthology.orchestra.run.smithi161.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-05-09T05:23:10.078 INFO:teuthology.orchestra.run.smithi161.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-05-09T05:23:10.106 INFO:teuthology.orchestra.run.smithi195.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 1.8 MB/s | 224 kB 00:00 2024-05-09T05:23:10.290 INFO:teuthology.orchestra.run.smithi195.stdout:(12/156): ceph-common-19.0.0-3594.gc9da4049.el9 1.5 MB/s | 20 MB 00:12 2024-05-09T05:23:10.315 INFO:teuthology.orchestra.run.smithi195.stdout:(13/156): cephfs-shell-19.0.0-3594.gc9da4049.el 129 kB/s | 27 kB 00:00 2024-05-09T05:23:10.345 INFO:teuthology.orchestra.run.smithi161.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-09T05:23:10.365 INFO:teuthology.orchestra.run.smithi195.stdout:(14/156): libcephfs-devel-19.0.0-3594.gc9da4049 424 kB/s | 32 kB 00:00 2024-05-09T05:23:10.371 INFO:teuthology.orchestra.run.smithi161.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 64 kB/s | 22 kB 00:00 2024-05-09T05:23:10.474 INFO:teuthology.orchestra.run.smithi195.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.4 MB/s | 158 kB 00:00 2024-05-09T05:23:10.499 INFO:teuthology.orchestra.run.smithi195.stdout:(16/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 4.0 MB/s | 756 kB 00:00 2024-05-09T05:23:10.599 INFO:teuthology.orchestra.run.smithi195.stdout:(17/156): librados-devel-19.0.0-3594.gc9da4049. 1.0 MB/s | 127 kB 00:00 2024-05-09T05:23:10.621 INFO:teuthology.orchestra.run.smithi161.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-09T05:23:10.655 INFO:teuthology.orchestra.run.smithi161.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-05-09T05:23:10.680 INFO:teuthology.orchestra.run.smithi161.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-05-09T05:23:10.683 INFO:teuthology.orchestra.run.smithi195.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 2.6 MB/s | 497 kB 00:00 2024-05-09T05:23:10.739 INFO:teuthology.orchestra.run.smithi161.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-05-09T05:23:10.778 INFO:teuthology.orchestra.run.smithi195.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 478 kB/s | 45 kB 00:00 2024-05-09T05:23:10.797 INFO:teuthology.orchestra.run.smithi161.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-05-09T05:23:10.881 INFO:teuthology.orchestra.run.smithi161.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-09T05:23:10.898 INFO:teuthology.orchestra.run.smithi161.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-05-09T05:23:10.940 INFO:teuthology.orchestra.run.smithi161.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-09T05:23:10.999 INFO:teuthology.orchestra.run.smithi161.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 938 kB/s | 863 kB 00:00 2024-05-09T05:23:11.032 INFO:teuthology.orchestra.run.smithi161.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-09T05:23:11.057 INFO:teuthology.orchestra.run.smithi161.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-05-09T05:23:11.082 INFO:teuthology.orchestra.run.smithi161.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-05-09T05:23:11.108 INFO:teuthology.orchestra.run.smithi161.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-05-09T05:23:11.141 INFO:teuthology.orchestra.run.smithi161.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-09T05:23:11.150 INFO:teuthology.orchestra.run.smithi195.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 364 kB/s | 135 kB 00:00 2024-05-09T05:23:11.158 INFO:teuthology.orchestra.run.smithi161.stdout:(118/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 272 kB/s | 43 kB 00:00 2024-05-09T05:23:11.192 INFO:teuthology.orchestra.run.smithi161.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-09T05:23:11.213 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T05:23:11.214 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T05:23:11.217 INFO:teuthology.orchestra.run.smithi161.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-05-09T05:23:11.225 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T05:23:11.225 INFO:teuthology.orchestra.run.smithi195.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 2.1 MB/s | 162 kB 00:00 2024-05-09T05:23:11.226 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T05:23:11.226 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T05:23:11.227 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T05:23:11.229 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T05:23:11.231 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T05:23:11.232 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T05:23:11.234 INFO:teuthology.orchestra.run.smithi161.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 467 kB/s | 35 kB 00:00 2024-05-09T05:23:11.268 INFO:teuthology.orchestra.run.smithi161.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T05:23:11.294 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-09T05:23:11.309 INFO:teuthology.orchestra.run.smithi195.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 3.8 MB/s | 321 kB 00:00 2024-05-09T05:23:11.318 INFO:teuthology.orchestra.run.smithi161.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T05:23:11.319 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T05:23:11.320 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T05:23:11.321 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T05:23:11.322 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T05:23:11.323 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T05:23:11.324 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T05:23:11.325 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T05:23:11.326 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T05:23:11.327 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T05:23:11.328 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T05:23:11.329 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T05:23:11.330 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:23:11.331 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-09T05:23:11.331 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-05-09T05:23:11.331 INFO:teuthology.orchestra.run.smithi115.stdout:Install 148 Packages 2024-05-09T05:23:11.331 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 8 Packages 2024-05-09T05:23:11.331 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:23:11.331 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 219 M 2024-05-09T05:23:11.331 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-09T05:23:11.360 INFO:teuthology.orchestra.run.smithi161.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-09T05:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 3.9 MB/s | 301 kB 00:00 2024-05-09T05:23:11.385 INFO:teuthology.orchestra.run.smithi161.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 958 kB/s | 144 kB 00:00 2024-05-09T05:23:11.402 INFO:teuthology.orchestra.run.smithi161.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-05-09T05:23:11.419 INFO:teuthology.orchestra.run.smithi161.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-05-09T05:23:11.426 INFO:teuthology.orchestra.run.smithi195.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 2.3 MB/s | 99 kB 00:00 2024-05-09T05:23:11.461 INFO:teuthology.orchestra.run.smithi161.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2024-05-09T05:23:11.476 INFO:teuthology.orchestra.run.smithi195.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.7 MB/s | 86 kB 00:00 2024-05-09T05:23:11.486 INFO:teuthology.orchestra.run.smithi161.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-05-09T05:23:11.520 INFO:teuthology.orchestra.run.smithi161.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-05-09T05:23:11.545 INFO:teuthology.orchestra.run.smithi161.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 552 kB/s | 46 kB 00:00 2024-05-09T05:23:11.571 INFO:teuthology.orchestra.run.smithi161.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-05-09T05:23:11.596 INFO:teuthology.orchestra.run.smithi161.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 762 kB/s | 58 kB 00:00 2024-05-09T05:23:11.621 INFO:teuthology.orchestra.run.smithi161.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-09T05:23:11.647 INFO:teuthology.orchestra.run.smithi161.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-05-09T05:23:11.663 INFO:teuthology.orchestra.run.smithi161.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-09T05:23:11.709 INFO:teuthology.orchestra.run.smithi161.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 324 kB/s | 28 kB 00:00 2024-05-09T05:23:11.739 INFO:teuthology.orchestra.run.smithi161.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-05-09T05:23:11.764 INFO:teuthology.orchestra.run.smithi161.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-09T05:23:11.790 INFO:teuthology.orchestra.run.smithi161.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 735 kB/s | 59 kB 00:00 2024-05-09T05:23:11.815 INFO:teuthology.orchestra.run.smithi161.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-05-09T05:23:11.840 INFO:teuthology.orchestra.run.smithi161.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-05-09T05:23:11.865 INFO:teuthology.orchestra.run.smithi161.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-09T05:23:11.888 INFO:teuthology.orchestra.run.smithi115.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-05-09T05:23:11.891 INFO:teuthology.orchestra.run.smithi161.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-09T05:23:11.924 INFO:teuthology.orchestra.run.smithi161.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-09T05:23:11.950 INFO:teuthology.orchestra.run.smithi161.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-09T05:23:11.975 INFO:teuthology.orchestra.run.smithi161.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-09T05:23:12.017 INFO:teuthology.orchestra.run.smithi161.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-09T05:23:12.384 INFO:teuthology.orchestra.run.smithi161.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 6.3 MB/s | 2.3 MB 00:00 2024-05-09T05:23:12.542 INFO:teuthology.orchestra.run.smithi195.stdout:(26/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 2.9 MB/s | 3.0 MB 00:01 2024-05-09T05:23:12.576 INFO:teuthology.orchestra.run.smithi161.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 9.5 MB/s | 1.8 MB 00:00 2024-05-09T05:23:12.627 INFO:teuthology.orchestra.run.smithi195.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 1.8 MB/s | 169 kB 00:00 2024-05-09T05:23:12.660 INFO:teuthology.orchestra.run.smithi161.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 502 kB/s | 42 kB 00:00 2024-05-09T05:23:12.712 INFO:teuthology.orchestra.run.smithi195.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 411 kB/s | 34 kB 00:00 2024-05-09T05:23:12.776 INFO:teuthology.orchestra.run.smithi161.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-05-09T05:23:12.789 INFO:teuthology.orchestra.run.smithi115.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 959 kB/s | 863 kB 00:00 2024-05-09T05:23:12.795 INFO:teuthology.orchestra.run.smithi195.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 1.7 MB/s | 142 kB 00:00 2024-05-09T05:23:13.014 INFO:teuthology.orchestra.run.smithi115.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 652 kB/s | 147 kB 00:00 2024-05-09T05:23:13.045 INFO:teuthology.orchestra.run.smithi195.stdout:(30/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 2.1 MB/s | 5.2 MB 00:02 2024-05-09T05:23:13.127 INFO:teuthology.orchestra.run.smithi161.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 1.3 MB 00:00 2024-05-09T05:23:13.177 INFO:teuthology.orchestra.run.smithi161.stdout:(154/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-05-09T05:23:13.285 INFO:teuthology.orchestra.run.smithi161.stdout:(155/156): librados2-19.0.0-3594.gc9da4049.el9. 2.5 MB/s | 3.3 MB 00:01 2024-05-09T05:23:13.369 INFO:teuthology.orchestra.run.smithi161.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.9 MB/s | 2.4 MB 00:00 2024-05-09T05:23:13.372 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-05-09T05:23:13.372 INFO:teuthology.orchestra.run.smithi161.stdout:Total 12 MB/s | 219 MB 00:18 2024-05-09T05:23:13.928 INFO:teuthology.orchestra.run.smithi195.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 3.1 MB/s | 3.5 MB 00:01 2024-05-09T05:23:14.062 INFO:teuthology.orchestra.run.smithi195.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 2.0 MB/s | 274 kB 00:00 2024-05-09T05:23:14.196 INFO:teuthology.orchestra.run.smithi195.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 369 kB/s | 49 kB 00:00 2024-05-09T05:23:14.237 INFO:teuthology.orchestra.run.smithi195.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 397 kB/s | 16 kB 00:00 2024-05-09T05:23:14.346 INFO:teuthology.orchestra.run.smithi195.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 2.5 MB/s | 273 kB 00:00 2024-05-09T05:23:14.497 INFO:teuthology.orchestra.run.smithi115.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 1.5 MB/s | 2.3 MB 00:01 2024-05-09T05:23:14.561 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-05-09T05:23:14.588 INFO:teuthology.orchestra.run.smithi195.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 3.1 MB/s | 763 kB 00:00 2024-05-09T05:23:14.672 INFO:teuthology.orchestra.run.smithi195.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 219 kB/s | 18 kB 00:00 2024-05-09T05:23:14.710 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-05-09T05:23:14.711 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-05-09T05:23:14.814 INFO:teuthology.orchestra.run.smithi195.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 569 kB/s | 80 kB 00:00 2024-05-09T05:23:15.047 INFO:teuthology.orchestra.run.smithi115.stdout:(5/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-05-09T05:23:15.064 INFO:teuthology.orchestra.run.smithi195.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.1 MB/s | 803 kB 00:00 2024-05-09T05:23:15.148 INFO:teuthology.orchestra.run.smithi195.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-09T05:23:15.231 INFO:teuthology.orchestra.run.smithi195.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-09T05:23:15.464 INFO:teuthology.orchestra.run.smithi115.stdout:(6/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-05-09T05:23:15.523 INFO:teuthology.orchestra.run.smithi195.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-05-09T05:23:15.574 INFO:teuthology.orchestra.run.smithi195.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-09T05:23:15.632 INFO:teuthology.orchestra.run.smithi195.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-09T05:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-09T05:23:15.774 INFO:teuthology.orchestra.run.smithi195.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-05-09T05:23:15.941 INFO:teuthology.orchestra.run.smithi195.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-05-09T05:23:16.008 INFO:teuthology.orchestra.run.smithi195.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-05-09T05:23:16.092 INFO:teuthology.orchestra.run.smithi195.stdout:(49/156): ceph-mgr-diskprediction-local-19.0.0- 2.4 MB/s | 7.4 MB 00:03 2024-05-09T05:23:16.117 INFO:teuthology.orchestra.run.smithi195.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 985 kB/s | 107 kB 00:00 2024-05-09T05:23:16.168 INFO:teuthology.orchestra.run.smithi195.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 313 kB 00:00 2024-05-09T05:23:16.193 INFO:teuthology.orchestra.run.smithi195.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-05-09T05:23:16.218 INFO:teuthology.orchestra.run.smithi195.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-05-09T05:23:16.269 INFO:teuthology.orchestra.run.smithi195.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-09T05:23:16.310 INFO:teuthology.orchestra.run.smithi195.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-05-09T05:23:16.353 INFO:teuthology.orchestra.run.smithi195.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-05-09T05:23:16.394 INFO:teuthology.orchestra.run.smithi195.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-09T05:23:16.486 INFO:teuthology.orchestra.run.smithi195.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-05-09T05:23:16.537 INFO:teuthology.orchestra.run.smithi195.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-05-09T05:23:16.587 INFO:teuthology.orchestra.run.smithi195.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.6 MB/s | 3.0 MB 00:00 2024-05-09T05:23:16.612 INFO:teuthology.orchestra.run.smithi195.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-05-09T05:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-09T05:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-05-09T05:23:16.738 INFO:teuthology.orchestra.run.smithi195.stdout:(64/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 314 kB/s | 47 kB 00:00 2024-05-09T05:23:16.763 INFO:teuthology.orchestra.run.smithi195.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-05-09T05:23:16.843 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-05-09T05:23:16.843 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-05-09T05:23:16.872 INFO:teuthology.orchestra.run.smithi195.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-05-09T05:23:16.889 INFO:teuthology.orchestra.run.smithi115.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 2.6 MB/s | 4.9 MB 00:01 2024-05-09T05:23:16.939 INFO:teuthology.orchestra.run.smithi195.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-05-09T05:23:16.998 INFO:teuthology.orchestra.run.smithi195.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-05-09T05:23:17.065 INFO:teuthology.orchestra.run.smithi195.stdout:(69/156): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-05-09T05:23:17.090 INFO:teuthology.orchestra.run.smithi195.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-09T05:23:17.199 INFO:teuthology.orchestra.run.smithi195.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-09T05:23:17.291 INFO:teuthology.orchestra.run.smithi195.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-05-09T05:23:17.316 INFO:teuthology.orchestra.run.smithi195.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 751 kB/s | 87 kB 00:00 2024-05-09T05:23:17.383 INFO:teuthology.orchestra.run.smithi195.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-05-09T05:23:17.467 INFO:teuthology.orchestra.run.smithi195.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-05-09T05:23:17.500 INFO:teuthology.orchestra.run.smithi195.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-05-09T05:23:17.525 INFO:teuthology.orchestra.run.smithi195.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-05-09T05:23:17.551 INFO:teuthology.orchestra.run.smithi195.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-09T05:23:17.584 INFO:teuthology.orchestra.run.smithi195.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-09T05:23:17.610 INFO:teuthology.orchestra.run.smithi195.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-05-09T05:23:17.660 INFO:teuthology.orchestra.run.smithi195.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-09T05:23:17.735 INFO:teuthology.orchestra.run.smithi195.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-05-09T05:23:17.761 INFO:teuthology.orchestra.run.smithi195.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-09T05:23:17.786 INFO:teuthology.orchestra.run.smithi195.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-09T05:23:17.811 INFO:teuthology.orchestra.run.smithi195.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-05-09T05:23:17.836 INFO:teuthology.orchestra.run.smithi195.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-09T05:23:17.862 INFO:teuthology.orchestra.run.smithi195.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-09T05:23:17.887 INFO:teuthology.orchestra.run.smithi195.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-09T05:23:17.912 INFO:teuthology.orchestra.run.smithi195.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-09T05:23:17.938 INFO:teuthology.orchestra.run.smithi195.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-09T05:23:17.980 INFO:teuthology.orchestra.run.smithi195.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-09T05:23:18.022 INFO:teuthology.orchestra.run.smithi195.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-09T05:23:18.072 INFO:teuthology.orchestra.run.smithi195.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-09T05:23:18.114 INFO:teuthology.orchestra.run.smithi195.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 863 kB/s | 36 kB 00:00 2024-05-09T05:23:18.289 INFO:teuthology.orchestra.run.smithi195.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-05-09T05:23:18.314 INFO:teuthology.orchestra.run.smithi195.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 237 kB/s | 47 kB 00:00 2024-05-09T05:23:18.341 INFO:teuthology.orchestra.run.smithi195.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-09T05:23:18.365 INFO:teuthology.orchestra.run.smithi195.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-09T05:23:18.390 INFO:teuthology.orchestra.run.smithi195.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-09T05:23:18.415 INFO:teuthology.orchestra.run.smithi195.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T05:23:18.441 INFO:teuthology.orchestra.run.smithi195.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-05-09T05:23:18.683 INFO:teuthology.orchestra.run.smithi195.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-05-09T05:23:19.150 INFO:teuthology.orchestra.run.smithi195.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-05-09T05:23:19.250 INFO:teuthology.orchestra.run.smithi195.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-05-09T05:23:19.275 INFO:teuthology.orchestra.run.smithi195.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-05-09T05:23:19.501 INFO:teuthology.orchestra.run.smithi195.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-05-09T05:23:19.567 INFO:teuthology.orchestra.run.smithi195.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-05-09T05:23:19.593 INFO:teuthology.orchestra.run.smithi195.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-05-09T05:23:19.635 INFO:teuthology.orchestra.run.smithi195.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 723 kB/s | 49 kB 00:00 2024-05-09T05:23:19.719 INFO:teuthology.orchestra.run.smithi195.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-09T05:23:19.744 INFO:teuthology.orchestra.run.smithi195.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 446 kB/s | 67 kB 00:00 2024-05-09T05:23:19.802 INFO:teuthology.orchestra.run.smithi195.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-05-09T05:23:19.870 INFO:teuthology.orchestra.run.smithi195.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-05-09T05:23:19.889 INFO:teuthology.orchestra.run.smithi115.stdout:(8/156): ceph-common-19.0.0-3594.gc9da4049.el9. 2.4 MB/s | 20 MB 00:08 2024-05-09T05:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 484 kB/s | 32 kB 00:00 2024-05-09T05:23:19.939 INFO:teuthology.orchestra.run.smithi115.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 507 kB/s | 25 kB 00:00 2024-05-09T05:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-05-09T05:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-09T05:23:20.054 INFO:teuthology.orchestra.run.smithi195.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-09T05:23:20.088 INFO:teuthology.orchestra.run.smithi195.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-05-09T05:23:20.146 INFO:teuthology.orchestra.run.smithi195.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-09T05:23:20.172 INFO:teuthology.orchestra.run.smithi195.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 423 kB/s | 35 kB 00:00 2024-05-09T05:23:20.230 INFO:teuthology.orchestra.run.smithi195.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-09T05:23:20.281 INFO:teuthology.orchestra.run.smithi195.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-05-09T05:23:20.340 INFO:teuthology.orchestra.run.smithi115.stdout:(10/156): ceph-radosgw-19.0.0-3594.gc9da4049.el 2.9 MB/s | 10 MB 00:03 2024-05-09T05:23:20.389 INFO:teuthology.orchestra.run.smithi195.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-05-09T05:23:20.415 INFO:teuthology.orchestra.run.smithi195.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 786 kB/s | 144 kB 00:00 2024-05-09T05:23:20.465 INFO:teuthology.orchestra.run.smithi195.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-05-09T05:23:20.490 INFO:teuthology.orchestra.run.smithi195.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-05-09T05:23:20.540 INFO:teuthology.orchestra.run.smithi195.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-09T05:23:20.548 INFO:teuthology.orchestra.run.smithi115.stdout:(11/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 3.5 MB/s | 18 MB 00:05 2024-05-09T05:23:20.566 INFO:teuthology.orchestra.run.smithi195.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-05-09T05:23:20.573 INFO:teuthology.orchestra.run.smithi115.stdout:(12/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 960 kB/s | 224 kB 00:00 2024-05-09T05:23:20.607 INFO:teuthology.orchestra.run.smithi115.stdout:(13/156): cephfs-shell-19.0.0-3594.gc9da4049.el 463 kB/s | 27 kB 00:00 2024-05-09T05:23:20.649 INFO:teuthology.orchestra.run.smithi115.stdout:(14/156): libcephfs-devel-19.0.0-3594.gc9da4049 422 kB/s | 32 kB 00:00 2024-05-09T05:23:20.741 INFO:teuthology.orchestra.run.smithi115.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.7 MB/s | 158 kB 00:00 2024-05-09T05:23:20.809 INFO:teuthology.orchestra.run.smithi115.stdout:(16/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 3.7 MB/s | 756 kB 00:00 2024-05-09T05:23:20.833 INFO:teuthology.orchestra.run.smithi115.stdout:(17/156): librados-devel-19.0.0-3594.gc9da4049. 1.3 MB/s | 127 kB 00:00 2024-05-09T05:23:20.983 INFO:teuthology.orchestra.run.smithi195.stdout:(129/156): ceph-test-19.0.0-3594.gc9da4049.el9. 3.3 MB/s | 43 MB 00:12 2024-05-09T05:23:21.008 INFO:teuthology.orchestra.run.smithi195.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 105 kB/s | 46 kB 00:00 2024-05-09T05:23:21.008 INFO:teuthology.orchestra.run.smithi115.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 2.4 MB/s | 497 kB 00:00 2024-05-09T05:23:21.041 INFO:teuthology.orchestra.run.smithi195.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 2.0 MB/s | 1.0 MB 00:00 2024-05-09T05:23:21.067 INFO:teuthology.orchestra.run.smithi115.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 769 kB/s | 45 kB 00:00 2024-05-09T05:23:21.092 INFO:teuthology.orchestra.run.smithi195.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-05-09T05:23:21.142 INFO:teuthology.orchestra.run.smithi195.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-05-09T05:23:21.159 INFO:teuthology.orchestra.run.smithi115.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.4 MB/s | 135 kB 00:00 2024-05-09T05:23:21.167 INFO:teuthology.orchestra.run.smithi195.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-09T05:23:21.218 INFO:teuthology.orchestra.run.smithi195.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-09T05:23:21.234 INFO:teuthology.orchestra.run.smithi115.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 2.1 MB/s | 162 kB 00:00 2024-05-09T05:23:21.243 INFO:teuthology.orchestra.run.smithi195.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-05-09T05:23:21.293 INFO:teuthology.orchestra.run.smithi195.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-09T05:23:21.319 INFO:teuthology.orchestra.run.smithi195.stdout:(138/156): python3-more-itertools-8.12.0-2.el9. 235 kB/s | 79 kB 00:00 2024-05-09T05:23:21.344 INFO:teuthology.orchestra.run.smithi195.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-09T05:23:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 2.5 MB/s | 321 kB 00:00 2024-05-09T05:23:21.369 INFO:teuthology.orchestra.run.smithi195.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-05-09T05:23:21.411 INFO:teuthology.orchestra.run.smithi195.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-05-09T05:23:21.436 INFO:teuthology.orchestra.run.smithi195.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-05-09T05:23:21.462 INFO:teuthology.orchestra.run.smithi195.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-09T05:23:21.477 INFO:teuthology.orchestra.run.smithi115.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 2.5 MB/s | 301 kB 00:00 2024-05-09T05:23:21.543 INFO:teuthology.orchestra.run.smithi115.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.5 MB/s | 99 kB 00:00 2024-05-09T05:23:21.554 INFO:teuthology.orchestra.run.smithi195.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 631 kB/s | 90 kB 00:00 2024-05-09T05:23:21.579 INFO:teuthology.orchestra.run.smithi195.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-05-09T05:23:21.612 INFO:teuthology.orchestra.run.smithi195.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-05-09T05:23:21.627 INFO:teuthology.orchestra.run.smithi115.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-09T05:23:21.680 INFO:teuthology.orchestra.run.smithi195.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-09T05:23:21.721 INFO:teuthology.orchestra.run.smithi195.stdout:(148/156): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-05-09T05:23:22.194 INFO:teuthology.orchestra.run.smithi115.stdout:(26/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 5.4 MB/s | 3.0 MB 00:00 2024-05-09T05:23:22.244 INFO:teuthology.orchestra.run.smithi115.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 3.3 MB/s | 169 kB 00:00 2024-05-09T05:23:22.263 INFO:teuthology.orchestra.run.smithi195.stdout:(149/156): librados2-19.0.0-3594.gc9da4049.el9. 5.1 MB/s | 3.3 MB 00:00 2024-05-09T05:23:22.295 INFO:teuthology.orchestra.run.smithi115.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 685 kB/s | 34 kB 00:00 2024-05-09T05:23:22.305 INFO:teuthology.orchestra.run.smithi195.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 4.0 MB/s | 2.3 MB 00:00 2024-05-09T05:23:22.345 INFO:teuthology.orchestra.run.smithi115.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 2.8 MB/s | 142 kB 00:00 2024-05-09T05:23:22.356 INFO:teuthology.orchestra.run.smithi195.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 837 kB/s | 42 kB 00:00 2024-05-09T05:23:22.414 INFO:teuthology.orchestra.run.smithi195.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-05-09T05:23:22.598 INFO:teuthology.orchestra.run.smithi195.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-05-09T05:23:22.629 INFO:teuthology.orchestra.run.smithi115.stdout:(30/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 2.9 MB/s | 5.2 MB 00:01 2024-05-09T05:23:22.806 INFO:teuthology.orchestra.run.smithi195.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 3.4 MB/s | 1.8 MB 00:00 2024-05-09T05:23:22.812 INFO:teuthology.orchestra.run.smithi115.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 7.6 MB/s | 3.5 MB 00:00 2024-05-09T05:23:22.879 INFO:teuthology.orchestra.run.smithi115.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 4.0 MB/s | 274 kB 00:00 2024-05-09T05:23:22.915 INFO:teuthology.orchestra.run.smithi195.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.6 MB/s | 2.4 MB 00:00 2024-05-09T05:23:22.921 INFO:teuthology.orchestra.run.smithi115.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 1.2 MB/s | 49 kB 00:00 2024-05-09T05:23:22.971 INFO:teuthology.orchestra.run.smithi115.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 330 kB/s | 16 kB 00:00 2024-05-09T05:23:23.016 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-05-09T05:23:23.047 INFO:teuthology.orchestra.run.smithi115.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 3.6 MB/s | 273 kB 00:00 2024-05-09T05:23:23.099 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T05:23:23.137 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T05:23:23.201 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T05:23:23.241 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T05:23:23.266 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T05:23:23.330 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T05:23:23.414 INFO:teuthology.orchestra.run.smithi115.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 2.0 MB/s | 763 kB 00:00 2024-05-09T05:23:23.432 INFO:teuthology.orchestra.run.smithi195.stdout:(156/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 1.8 MB/s | 3.1 MB 00:01 2024-05-09T05:23:23.438 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-05-09T05:23:23.438 INFO:teuthology.orchestra.run.smithi195.stdout:Total 8.2 MB/s | 219 MB 00:26 2024-05-09T05:23:23.464 INFO:teuthology.orchestra.run.smithi115.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 365 kB/s | 18 kB 00:00 2024-05-09T05:23:23.531 INFO:teuthology.orchestra.run.smithi115.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-05-09T05:23:23.623 INFO:teuthology.orchestra.run.smithi115.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-05-09T05:23:23.662 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T05:23:23.690 INFO:teuthology.orchestra.run.smithi115.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-09T05:23:23.773 INFO:teuthology.orchestra.run.smithi115.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-09T05:23:23.775 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T05:23:23.899 INFO:teuthology.orchestra.run.smithi115.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-05-09T05:23:23.919 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T05:23:23.944 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T05:23:23.965 INFO:teuthology.orchestra.run.smithi115.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-09T05:23:24.033 INFO:teuthology.orchestra.run.smithi115.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-09T05:23:24.035 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T05:23:24.073 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T05:23:24.100 INFO:teuthology.orchestra.run.smithi115.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-09T05:23:24.175 INFO:teuthology.orchestra.run.smithi115.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-09T05:23:24.242 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T05:23:24.296 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T05:23:24.331 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T05:23:24.375 INFO:teuthology.orchestra.run.smithi115.stdout:(47/156): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-05-09T05:23:24.438 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T05:23:24.459 INFO:teuthology.orchestra.run.smithi115.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-05-09T05:23:24.466 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T05:23:24.501 INFO:teuthology.orchestra.run.smithi115.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-09T05:23:24.512 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T05:23:24.541 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T05:23:24.543 INFO:teuthology.orchestra.run.smithi115.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-05-09T05:23:24.564 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T05:23:24.585 INFO:teuthology.orchestra.run.smithi115.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-05-09T05:23:24.610 INFO:teuthology.orchestra.run.smithi115.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-05-09T05:23:24.626 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-05-09T05:23:24.636 INFO:teuthology.orchestra.run.smithi115.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-05-09T05:23:24.677 INFO:teuthology.orchestra.run.smithi115.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 440 kB/s | 18 kB 00:00 2024-05-09T05:23:24.719 INFO:teuthology.orchestra.run.smithi115.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-05-09T05:23:24.751 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T05:23:24.774 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-05-09T05:23:24.774 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-05-09T05:23:24.778 INFO:teuthology.orchestra.run.smithi115.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-05-09T05:23:24.803 INFO:teuthology.orchestra.run.smithi115.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-05-09T05:23:24.829 INFO:teuthology.orchestra.run.smithi115.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-05-09T05:23:24.862 INFO:teuthology.orchestra.run.smithi115.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-09T05:23:24.896 INFO:teuthology.orchestra.run.smithi115.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-09T05:23:24.916 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T05:23:24.971 INFO:teuthology.orchestra.run.smithi115.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-05-09T05:23:24.996 INFO:teuthology.orchestra.run.smithi115.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-09T05:23:25.022 INFO:teuthology.orchestra.run.smithi115.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-09T05:23:25.030 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T05:23:25.047 INFO:teuthology.orchestra.run.smithi115.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-09T05:23:25.072 INFO:teuthology.orchestra.run.smithi115.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-05-09T05:23:25.074 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T05:23:25.177 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T05:23:25.189 INFO:teuthology.orchestra.run.smithi115.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-05-09T05:23:25.258 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T05:23:25.265 INFO:teuthology.orchestra.run.smithi115.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-05-09T05:23:25.332 INFO:teuthology.orchestra.run.smithi115.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-09T05:23:25.365 INFO:teuthology.orchestra.run.smithi115.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-05-09T05:23:25.390 INFO:teuthology.orchestra.run.smithi115.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-09T05:23:25.457 INFO:teuthology.orchestra.run.smithi115.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-09T05:23:25.466 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T05:23:25.533 INFO:teuthology.orchestra.run.smithi115.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-05-09T05:23:25.558 INFO:teuthology.orchestra.run.smithi115.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 871 kB/s | 87 kB 00:00 2024-05-09T05:23:25.585 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T05:23:25.617 INFO:teuthology.orchestra.run.smithi115.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-05-09T05:23:25.621 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T05:23:25.664 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T05:23:25.700 INFO:teuthology.orchestra.run.smithi115.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-05-09T05:23:25.726 INFO:teuthology.orchestra.run.smithi115.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-09T05:23:25.729 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T05:23:25.742 INFO:teuthology.orchestra.run.smithi115.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 611 kB/s | 26 kB 00:00 2024-05-09T05:23:25.759 INFO:teuthology.orchestra.run.smithi115.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 570 kB/s | 19 kB 00:00 2024-05-09T05:23:25.773 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T05:23:25.785 INFO:teuthology.orchestra.run.smithi115.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-09T05:23:25.802 INFO:teuthology.orchestra.run.smithi115.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 829 kB/s | 35 kB 00:00 2024-05-09T05:23:25.816 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T05:23:25.849 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T05:23:25.885 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T05:23:25.953 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T05:23:26.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T05:23:26.038 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T05:23:26.082 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T05:23:26.159 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T05:23:26.177 INFO:teuthology.orchestra.run.smithi115.stdout:(81/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 6.9 MB/s | 43 MB 00:06 2024-05-09T05:23:26.192 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T05:23:26.202 INFO:teuthology.orchestra.run.smithi115.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 626 kB/s | 250 kB 00:00 2024-05-09T05:23:26.237 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T05:23:26.244 INFO:teuthology.orchestra.run.smithi115.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-09T05:23:26.264 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T05:23:26.320 INFO:teuthology.orchestra.run.smithi115.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.5 MB/s | 5.1 MB 00:00 2024-05-09T05:23:26.336 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T05:23:26.345 INFO:teuthology.orchestra.run.smithi115.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-05-09T05:23:26.370 INFO:teuthology.orchestra.run.smithi115.stdout:(86/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-05-09T05:23:26.389 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T05:23:26.395 INFO:teuthology.orchestra.run.smithi115.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-05-09T05:23:26.421 INFO:teuthology.orchestra.run.smithi115.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-09T05:23:26.446 INFO:teuthology.orchestra.run.smithi115.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-09T05:23:26.464 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T05:23:26.471 INFO:teuthology.orchestra.run.smithi115.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-09T05:23:26.496 INFO:teuthology.orchestra.run.smithi115.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-05-09T05:23:26.522 INFO:teuthology.orchestra.run.smithi115.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-05-09T05:23:26.547 INFO:teuthology.orchestra.run.smithi115.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 877 kB/s | 44 kB 00:00 2024-05-09T05:23:26.572 INFO:teuthology.orchestra.run.smithi115.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-09T05:23:26.598 INFO:teuthology.orchestra.run.smithi115.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-05-09T05:23:26.602 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T05:23:26.623 INFO:teuthology.orchestra.run.smithi115.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-05-09T05:23:26.648 INFO:teuthology.orchestra.run.smithi115.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-09T05:23:26.657 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T05:23:26.674 INFO:teuthology.orchestra.run.smithi115.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-09T05:23:26.699 INFO:teuthology.orchestra.run.smithi115.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-09T05:23:26.724 INFO:teuthology.orchestra.run.smithi115.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T05:23:26.750 INFO:teuthology.orchestra.run.smithi115.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-05-09T05:23:26.879 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-05-09T05:23:26.879 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-05-09T05:23:27.033 INFO:teuthology.orchestra.run.smithi115.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-05-09T05:23:27.193 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T05:23:27.278 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T05:23:27.307 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T05:23:27.351 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T05:23:27.375 INFO:teuthology.orchestra.run.smithi115.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-09T05:23:27.396 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T05:23:27.434 INFO:teuthology.orchestra.run.smithi115.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-05-09T05:23:27.438 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T05:23:27.467 INFO:teuthology.orchestra.run.smithi115.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 30 kB/s | 22 kB 00:00 2024-05-09T05:23:27.492 INFO:teuthology.orchestra.run.smithi115.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 336 kB/s | 19 kB 00:00 2024-05-09T05:23:27.551 INFO:teuthology.orchestra.run.smithi115.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-05-09T05:23:27.592 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T05:23:27.610 INFO:teuthology.orchestra.run.smithi115.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-05-09T05:23:27.649 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T05:23:27.669 INFO:teuthology.orchestra.run.smithi115.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-09T05:23:27.727 INFO:teuthology.orchestra.run.smithi115.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-09T05:23:27.739 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T05:23:27.804 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T05:23:27.844 INFO:teuthology.orchestra.run.smithi115.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-09T05:23:27.877 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T05:23:27.903 INFO:teuthology.orchestra.run.smithi115.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-05-09T05:23:27.908 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T05:23:27.936 INFO:teuthology.orchestra.run.smithi115.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 728 kB/s | 863 kB 00:01 2024-05-09T05:23:27.944 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T05:23:27.961 INFO:teuthology.orchestra.run.smithi115.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-05-09T05:23:27.992 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T05:23:28.020 INFO:teuthology.orchestra.run.smithi115.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-05-09T05:23:28.028 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T05:23:28.071 INFO:teuthology.orchestra.run.smithi115.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-05-09T05:23:28.071 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T05:23:28.113 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T05:23:28.121 INFO:teuthology.orchestra.run.smithi115.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 175 kB/s | 32 kB 00:00 2024-05-09T05:23:28.146 INFO:teuthology.orchestra.run.smithi115.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-09T05:23:28.205 INFO:teuthology.orchestra.run.smithi115.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-05-09T05:23:28.248 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T05:23:28.272 INFO:teuthology.orchestra.run.smithi115.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-05-09T05:23:28.293 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T05:23:28.297 INFO:teuthology.orchestra.run.smithi115.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-05-09T05:23:28.323 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T05:23:28.347 INFO:teuthology.orchestra.run.smithi115.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-09T05:23:28.365 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T05:23:28.406 INFO:teuthology.orchestra.run.smithi115.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-05-09T05:23:28.458 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T05:23:28.473 INFO:teuthology.orchestra.run.smithi115.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-05-09T05:23:28.490 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T05:23:28.498 INFO:teuthology.orchestra.run.smithi115.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-09T05:23:28.532 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T05:23:28.548 INFO:teuthology.orchestra.run.smithi115.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-05-09T05:23:28.565 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T05:23:28.574 INFO:teuthology.orchestra.run.smithi115.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-05-09T05:23:28.607 INFO:teuthology.orchestra.run.smithi115.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-05-09T05:23:28.624 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T05:23:28.666 INFO:teuthology.orchestra.run.smithi115.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-05-09T05:23:28.700 INFO:teuthology.orchestra.run.smithi115.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-05-09T05:23:28.717 INFO:teuthology.orchestra.run.smithi115.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-09T05:23:28.759 INFO:teuthology.orchestra.run.smithi115.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 984 kB/s | 58 kB 00:00 2024-05-09T05:23:28.784 INFO:teuthology.orchestra.run.smithi115.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-05-09T05:23:28.809 INFO:teuthology.orchestra.run.smithi115.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-09T05:23:28.843 INFO:teuthology.orchestra.run.smithi115.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-09T05:23:28.860 INFO:teuthology.orchestra.run.smithi115.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 559 kB/s | 28 kB 00:00 2024-05-09T05:23:28.902 INFO:teuthology.orchestra.run.smithi115.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 523 kB/s | 31 kB 00:00 2024-05-09T05:23:28.918 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T05:23:28.952 INFO:teuthology.orchestra.run.smithi115.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-09T05:23:28.969 INFO:teuthology.orchestra.run.smithi115.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-05-09T05:23:29.003 INFO:teuthology.orchestra.run.smithi115.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-05-09T05:23:29.020 INFO:teuthology.orchestra.run.smithi115.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-09T05:23:29.041 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T05:23:29.062 INFO:teuthology.orchestra.run.smithi115.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-09T05:23:29.087 INFO:teuthology.orchestra.run.smithi115.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-09T05:23:29.097 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T05:23:29.125 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T05:23:29.129 INFO:teuthology.orchestra.run.smithi115.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-05-09T05:23:29.154 INFO:teuthology.orchestra.run.smithi115.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-05-09T05:23:29.159 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T05:23:29.198 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T05:23:29.221 INFO:teuthology.orchestra.run.smithi115.stdout:(146/156): libarrow-9.0.0-11.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-05-09T05:23:29.230 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T05:23:29.246 INFO:teuthology.orchestra.run.smithi115.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-05-09T05:23:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T05:23:29.294 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T05:23:29.328 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T05:23:29.330 INFO:teuthology.orchestra.run.smithi115.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-05-09T05:23:29.369 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T05:23:29.404 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T05:23:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T05:23:29.505 INFO:teuthology.orchestra.run.smithi115.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-05-09T05:23:29.534 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T05:23:29.566 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T05:23:29.604 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T05:23:29.689 INFO:teuthology.orchestra.run.smithi115.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 10 MB/s | 1.8 MB 00:00 2024-05-09T05:23:29.701 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T05:23:29.751 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T05:23:29.755 INFO:teuthology.orchestra.run.smithi115.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 629 kB/s | 42 kB 00:00 2024-05-09T05:23:29.864 INFO:teuthology.orchestra.run.smithi115.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-05-09T05:23:30.007 INFO:teuthology.orchestra.run.smithi115.stdout:(153/156): librados2-19.0.0-3594.gc9da4049.el9. 4.2 MB/s | 3.3 MB 00:00 2024-05-09T05:23:30.107 INFO:teuthology.orchestra.run.smithi115.stdout:(154/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 3.6 MB/s | 3.1 MB 00:00 2024-05-09T05:23:30.190 INFO:teuthology.orchestra.run.smithi115.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 4.1 MB/s | 1.3 MB 00:00 2024-05-09T05:23:30.399 INFO:teuthology.orchestra.run.smithi115.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.1 MB/s | 2.4 MB 00:00 2024-05-09T05:23:30.406 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-09T05:23:30.406 INFO:teuthology.orchestra.run.smithi115.stdout:Total 11 MB/s | 219 MB 00:19 2024-05-09T05:23:30.467 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T05:23:30.514 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T05:23:30.548 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T05:23:30.647 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T05:23:30.705 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T05:23:30.794 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T05:23:30.834 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T05:23:31.011 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T05:23:31.039 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T05:23:31.129 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T05:23:31.595 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-09T05:23:31.745 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-09T05:23:31.745 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-09T05:23:31.790 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T05:23:31.911 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T05:23:32.896 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-05-09T05:23:32.995 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T05:23:33.033 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T05:23:33.088 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T05:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T05:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T05:23:33.175 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T05:23:33.222 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T05:23:33.277 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T05:23:33.305 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T05:23:33.338 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T05:23:33.616 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T05:23:33.655 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T05:23:33.680 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T05:23:33.700 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T05:23:33.775 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T05:23:33.805 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T05:23:33.806 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T05:23:33.840 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T05:23:33.848 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T05:23:33.872 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-09T05:23:33.872 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-09T05:23:33.931 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T05:23:33.961 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T05:23:34.129 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T05:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T05:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T05:23:34.328 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T05:23:34.330 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T05:23:34.353 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T05:23:34.362 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T05:23:34.400 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T05:23:34.429 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T05:23:34.433 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T05:23:34.449 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T05:23:34.472 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T05:23:34.875 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T05:23:35.049 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T05:23:35.202 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T05:23:35.245 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T05:23:35.369 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T05:23:35.438 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T05:23:35.638 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T05:23:35.745 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T05:23:35.786 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T05:23:35.827 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T05:23:35.900 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T05:23:35.936 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T05:23:35.978 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T05:23:36.011 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T05:23:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T05:23:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T05:23:36.161 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T05:23:36.201 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T05:23:36.244 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T05:23:36.340 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T05:23:36.380 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T05:23:36.423 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T05:23:36.452 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T05:23:36.520 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T05:23:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T05:23:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T05:23:36.749 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T05:23:36.790 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T05:23:36.808 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T05:23:36.844 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T05:23:37.365 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T05:23:37.390 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T05:23:37.430 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T05:23:37.456 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T05:23:37.494 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T05:23:37.521 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T05:23:37.521 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T05:23:37.521 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T05:23:37.521 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:37.543 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T05:23:37.565 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T05:23:37.583 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T05:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T05:23:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T05:23:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T05:23:37.952 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T05:23:38.016 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T05:23:38.090 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T05:23:38.121 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T05:23:38.158 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T05:23:38.206 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T05:23:38.241 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T05:23:38.284 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T05:23:38.326 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T05:23:38.461 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T05:23:38.507 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T05:23:38.537 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T05:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T05:23:38.674 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T05:23:38.712 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T05:23:38.753 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T05:23:38.786 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T05:23:38.810 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T05:23:38.913 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T05:23:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T05:23:39.085 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T05:23:39.113 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T05:23:39.147 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T05:23:39.186 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T05:23:39.218 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T05:23:39.248 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T05:23:39.282 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T05:23:39.316 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T05:23:39.357 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T05:23:39.392 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T05:23:39.470 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T05:23:39.514 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T05:23:39.545 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T05:23:39.592 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T05:23:39.690 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T05:23:39.738 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T05:23:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-09T05:23:40.095 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T05:23:40.133 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T05:23:40.189 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T05:23:40.228 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T05:23:40.254 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T05:23:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T05:23:40.450 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T05:23:40.494 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T05:23:40.527 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T05:23:40.626 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T05:23:40.677 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T05:23:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T05:23:40.763 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T05:23:40.764 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T05:23:40.805 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T05:23:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T05:23:40.939 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T05:23:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T05:23:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T05:23:41.031 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T05:23:41.060 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T05:23:41.101 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T05:23:41.236 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T05:23:41.291 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T05:23:41.318 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T05:23:41.428 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T05:23:41.453 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T05:23:41.516 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T05:23:41.537 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T05:23:41.563 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T05:23:41.705 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T05:23:41.759 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T05:23:41.876 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T05:23:41.881 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T05:23:42.001 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T05:23:42.053 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T05:23:42.181 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T05:23:42.254 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T05:23:42.459 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T05:23:42.567 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T05:23:42.603 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T05:23:42.643 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T05:23:42.707 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T05:23:42.743 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T05:23:42.786 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T05:23:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T05:23:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T05:23:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T05:23:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T05:23:42.991 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T05:23:43.035 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T05:23:43.110 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T05:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T05:23:43.145 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T05:23:43.189 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T05:23:43.207 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T05:23:43.218 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T05:23:43.234 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T05:23:43.268 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T05:23:43.287 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T05:23:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T05:23:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T05:23:43.540 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T05:23:43.584 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T05:23:43.592 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T05:23:43.631 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T05:23:43.721 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T05:23:43.752 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T05:23:43.788 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T05:23:44.110 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T05:23:44.196 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T05:23:44.226 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T05:23:44.261 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T05:23:44.278 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T05:23:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T05:23:44.309 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T05:23:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T05:23:44.380 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T05:23:44.402 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T05:23:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T05:23:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T05:23:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T05:23:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T05:23:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T05:23:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T05:23:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T05:23:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T05:23:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T05:23:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T05:23:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T05:23:45.169 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T05:23:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T05:23:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T05:23:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T05:23:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T05:23:45.418 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T05:23:45.460 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T05:23:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T05:23:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T05:23:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T05:23:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T05:23:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T05:23:45.806 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T05:23:45.836 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T05:23:45.875 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T05:23:45.907 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T05:23:45.936 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T05:23:45.971 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T05:23:46.004 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T05:23:46.046 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T05:23:46.080 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T05:23:46.159 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T05:23:46.202 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T05:23:46.233 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T05:23:46.280 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T05:23:46.379 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T05:23:46.426 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T05:23:46.582 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T05:23:46.623 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T05:23:47.139 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T05:23:47.184 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T05:23:47.190 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T05:23:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T05:23:47.242 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T05:23:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T05:23:47.341 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T05:23:47.342 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T05:23:47.342 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T05:23:47.342 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T05:23:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T05:23:47.469 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T05:23:47.509 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T05:23:47.685 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T05:23:47.723 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T05:23:47.801 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T05:23:48.454 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T05:23:48.570 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T05:23:49.458 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T05:23:49.458 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-05-09T05:23:49.458 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-05-09T05:23:49.458 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-05-09T05:23:49.458 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-05-09T05:23:49.458 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-05-09T05:23:49.458 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-05-09T05:23:49.458 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-05-09T05:23:49.458 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:49.648 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T05:23:49.664 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T05:23:49.664 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:49.664 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T05:23:49.664 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T05:23:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T05:23:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T05:23:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T05:23:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T05:23:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T05:23:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T05:23:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T05:23:50.475 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T05:23:50.986 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T05:23:51.020 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T05:23:51.091 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T05:23:51.113 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T05:23:51.284 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T05:23:51.310 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T05:23:51.310 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:51.310 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T05:23:51.311 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T05:23:51.311 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T05:23:51.311 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:51.534 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T05:23:51.557 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T05:23:51.558 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:51.558 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T05:23:51.558 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T05:23:51.558 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T05:23:51.558 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:52.018 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T05:23:52.042 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T05:23:52.042 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:52.042 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T05:23:52.042 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T05:23:52.042 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T05:23:52.042 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:52.148 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T05:23:52.205 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T05:23:52.269 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T05:23:52.453 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T05:23:52.510 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T05:23:52.588 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T05:23:52.676 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T05:23:52.816 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T05:23:52.863 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T05:23:52.902 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T05:23:52.928 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T05:23:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T05:23:53.382 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T05:23:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T05:23:53.410 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T05:23:53.442 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T05:23:53.504 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T05:23:53.533 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T05:23:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T05:23:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T05:23:54.101 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T05:23:54.101 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T05:23:54.101 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T05:23:54.101 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:23:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T05:23:54.580 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T05:23:54.618 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T05:23:55.586 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T05:23:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T05:23:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T05:23:55.903 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T05:23:55.929 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T05:23:55.953 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T05:23:55.953 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:55.953 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T05:23:55.953 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T05:23:55.953 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T05:23:55.953 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:56.004 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T05:23:56.016 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T05:23:56.113 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T05:23:56.167 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T05:23:57.104 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T05:23:57.132 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T05:23:57.156 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T05:23:57.156 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:57.156 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T05:23:57.156 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T05:23:57.156 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T05:23:57.156 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:57.199 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T05:23:57.221 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T05:23:57.222 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:57.222 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T05:23:57.222 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:57.274 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T05:23:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T05:23:57.298 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:57.298 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T05:23:57.298 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T05:23:57.298 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T05:23:57.298 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:57.617 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T05:23:57.640 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T05:23:57.640 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:57.640 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T05:23:57.640 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T05:23:57.641 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T05:23:57.641 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:23:59.490 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T05:23:59.490 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /sys 2024-05-09T05:23:59.490 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /proc 2024-05-09T05:23:59.491 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /mnt 2024-05-09T05:23:59.491 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /var/tmp 2024-05-09T05:23:59.491 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /home 2024-05-09T05:23:59.491 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /root 2024-05-09T05:23:59.491 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /tmp 2024-05-09T05:23:59.491 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T05:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T05:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T05:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:01.272 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T05:24:01.299 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T05:24:01.299 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:01.299 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T05:24:01.299 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T05:24:01.299 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T05:24:01.299 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:01.523 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T05:24:01.547 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T05:24:01.547 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:01.547 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T05:24:01.547 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T05:24:01.547 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T05:24:01.547 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:01.912 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T05:24:01.953 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T05:24:01.994 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T05:24:02.007 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T05:24:02.026 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T05:24:02.030 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T05:24:02.030 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:02.030 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T05:24:02.031 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T05:24:02.031 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T05:24:02.031 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:02.053 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T05:24:02.128 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T05:24:02.185 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T05:24:02.241 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T05:24:02.252 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T05:24:02.299 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T05:24:02.329 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T05:24:02.359 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T05:24:02.422 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T05:24:02.423 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T05:24:02.466 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T05:24:02.543 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T05:24:02.640 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T05:24:02.779 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T05:24:02.828 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T05:24:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T05:24:02.892 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T05:24:03.005 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T05:24:03.044 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T05:24:03.065 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T05:24:03.355 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T05:24:03.383 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T05:24:03.415 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T05:24:03.475 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T05:24:03.506 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T05:24:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T05:24:04.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T05:24:04.058 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T05:24:04.186 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T05:24:04.218 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T05:24:04.244 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T05:24:04.244 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T05:24:04.336 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T05:24:04.369 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T05:24:04.370 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T05:24:04.454 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T05:24:04.479 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T05:24:04.493 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T05:24:04.493 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T05:24:04.544 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T05:24:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T05:24:04.689 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T05:24:04.689 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T05:24:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T05:24:04.839 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T05:24:05.544 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T05:24:05.574 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T05:24:05.684 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T05:24:05.785 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T05:24:05.785 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-05-09T05:24:05.785 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-05-09T05:24:05.785 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-05-09T05:24:05.785 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-05-09T05:24:05.785 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-05-09T05:24:05.785 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-05-09T05:24:05.785 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-05-09T05:24:05.785 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:05.850 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T05:24:05.885 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T05:24:05.909 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T05:24:05.909 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:05.909 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T05:24:05.909 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T05:24:05.909 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T05:24:05.910 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:05.961 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T05:24:05.973 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T05:24:05.978 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T05:24:05.994 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T05:24:05.994 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:05.994 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T05:24:05.994 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:06.070 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T05:24:06.144 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T05:24:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T05:24:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T05:24:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T05:24:07.121 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:07.121 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T05:24:07.121 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T05:24:07.121 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T05:24:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:07.164 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T05:24:07.187 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T05:24:07.188 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:07.188 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T05:24:07.188 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:07.240 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T05:24:07.263 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T05:24:07.263 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:07.263 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T05:24:07.264 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T05:24:07.264 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T05:24:07.264 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:07.582 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T05:24:07.591 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T05:24:07.605 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T05:24:07.605 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:07.605 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T05:24:07.605 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T05:24:07.605 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T05:24:07.605 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:07.618 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T05:24:07.619 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:07.619 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T05:24:07.619 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T05:24:07.619 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T05:24:07.619 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T05:24:07.874 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T05:24:07.874 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:07.874 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T05:24:07.874 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T05:24:07.875 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T05:24:07.875 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T05:24:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T05:24:08.358 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:08.358 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T05:24:08.358 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T05:24:08.358 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T05:24:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:08.456 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T05:24:08.513 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T05:24:08.568 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T05:24:08.763 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T05:24:08.810 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T05:24:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T05:24:08.975 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T05:24:09.115 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T05:24:09.169 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T05:24:09.200 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T05:24:09.227 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T05:24:09.707 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T05:24:09.734 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T05:24:09.767 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T05:24:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T05:24:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T05:24:10.367 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T05:24:10.368 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T05:24:10.369 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T05:24:10.370 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T05:24:10.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T05:24:10.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T05:24:10.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T05:24:10.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T05:24:10.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T05:24:10.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T05:24:10.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T05:24:10.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T05:24:10.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T05:24:10.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T05:24:10.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T05:24:10.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T05:24:10.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T05:24:10.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T05:24:10.390 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T05:24:10.391 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T05:24:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T05:24:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T05:24:11.869 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T05:24:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T05:24:11.910 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T05:24:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T05:24:11.951 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T05:24:11.983 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T05:24:12.010 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T05:24:12.027 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T05:24:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T05:24:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T05:24:12.236 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T05:24:12.248 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T05:24:12.261 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T05:24:12.261 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:12.261 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T05:24:12.261 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T05:24:12.261 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T05:24:12.261 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T05:24:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T05:24:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T05:24:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T05:24:12.354 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T05:24:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T05:24:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T05:24:12.954 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T05:24:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T05:24:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T05:24:13.280 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T05:24:13.280 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:24:13.280 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-05-09T05:24:13.280 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:13.280 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:13.280 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.281 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.282 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T05:24:13.283 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T05:24:13.284 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T05:24:13.285 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.286 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T05:24:13.287 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T05:24:13.288 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T05:24:13.288 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T05:24:13.288 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T05:24:13.288 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T05:24:13.288 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T05:24:13.288 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T05:24:13.288 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T05:24:13.289 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T05:24:13.290 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T05:24:13.290 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T05:24:13.290 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T05:24:13.290 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T05:24:13.290 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T05:24:13.290 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T05:24:13.290 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T05:24:13.290 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T05:24:13.290 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T05:24:13.291 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:13.292 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T05:24:13.293 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T05:24:13.293 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T05:24:13.293 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T05:24:13.293 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T05:24:13.293 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T05:24:13.293 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T05:24:13.294 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-09T05:24:13.294 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-05-09T05:24:13.416 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T05:24:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T05:24:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T05:24:13.472 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:13.472 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T05:24:13.472 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T05:24:13.473 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T05:24:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T05:24:13.538 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T05:24:13.538 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:13.538 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T05:24:13.538 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:13.591 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T05:24:13.614 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T05:24:13.614 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:13.614 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T05:24:13.614 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T05:24:13.614 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T05:24:13.614 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:13.715 DEBUG:teuthology.parallel:result is None 2024-05-09T05:24:13.847 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T05:24:13.932 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T05:24:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T05:24:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T05:24:13.965 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T05:24:13.965 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T05:24:13.966 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T05:24:13.966 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T05:24:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:13.983 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T05:24:14.080 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T05:24:14.118 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T05:24:14.144 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T05:24:14.144 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T05:24:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T05:24:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T05:24:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T05:24:14.363 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T05:24:14.389 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T05:24:14.404 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T05:24:14.404 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T05:24:14.590 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T05:24:14.590 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T05:24:14.674 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T05:24:14.738 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T05:24:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T05:24:18.303 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T05:24:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T05:24:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T05:24:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T05:24:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T05:24:18.649 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T05:24:18.670 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T05:24:18.700 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T05:24:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T05:24:19.339 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T05:24:19.377 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T05:24:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T05:24:20.225 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T05:24:20.315 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T05:24:20.363 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T05:24:20.438 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T05:24:20.438 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T05:24:20.438 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T05:24:20.438 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T05:24:20.439 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T05:24:20.440 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T05:24:20.441 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T05:24:20.441 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T05:24:20.441 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T05:24:20.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T05:24:20.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T05:24:20.456 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T05:24:20.457 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T05:24:20.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T05:24:20.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T05:24:20.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T05:24:20.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T05:24:20.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T05:24:20.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T05:24:20.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T05:24:20.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T05:24:20.458 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T05:24:20.459 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T05:24:20.460 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T05:24:20.461 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T05:24:20.462 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T05:24:20.463 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T05:24:20.464 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T05:24:20.465 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T05:24:20.466 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T05:24:20.467 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T05:24:20.468 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T05:24:20.493 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T05:24:20.519 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T05:24:20.519 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T05:24:20.603 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T05:24:20.628 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T05:24:20.628 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T05:24:20.704 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T05:24:20.729 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T05:24:20.750 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T05:24:20.750 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T05:24:20.931 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T05:24:20.931 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T05:24:20.998 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T05:24:21.063 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout:Upgraded: 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T05:24:23.240 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.241 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.242 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.242 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.242 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.242 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.242 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.242 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.242 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.242 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.243 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.243 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.243 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.243 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.243 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.243 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.243 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.243 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.243 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.244 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.244 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:23.244 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T05:24:23.244 INFO:teuthology.orchestra.run.smithi195.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T05:24:23.244 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T05:24:23.244 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T05:24:23.244 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T05:24:23.244 INFO:teuthology.orchestra.run.smithi195.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T05:24:23.244 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.245 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T05:24:23.246 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.247 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T05:24:23.248 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T05:24:23.249 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T05:24:23.250 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T05:24:23.250 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T05:24:23.250 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T05:24:23.250 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T05:24:23.250 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T05:24:23.250 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T05:24:23.250 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T05:24:23.251 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T05:24:23.251 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T05:24:23.251 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T05:24:23.251 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T05:24:23.251 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T05:24:23.251 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T05:24:23.251 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T05:24:23.252 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T05:24:23.253 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T05:24:23.254 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T05:24:23.254 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T05:24:23.254 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:24:23.254 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-05-09T05:24:23.640 DEBUG:teuthology.parallel:result is None 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T05:24:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T05:24:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T05:24:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T05:24:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T05:24:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T05:24:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T05:24:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T05:24:26.799 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T05:24:26.800 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T05:24:26.801 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T05:24:26.802 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T05:24:26.803 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T05:24:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T05:24:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T05:24:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T05:24:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T05:24:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T05:24:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T05:24:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T05:24:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T05:24:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T05:24:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T05:24:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T05:24:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T05:24:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T05:24:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T05:24:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T05:24:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T05:24:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T05:24:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T05:24:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T05:24:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T05:24:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T05:24:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T05:24:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T05:24:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T05:24:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T05:24:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T05:24:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T05:24:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T05:24:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T05:24:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T05:24:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T05:24:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T05:24:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T05:24:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T05:24:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T05:24:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T05:24:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T05:24:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T05:24:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T05:24:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:24:29.198 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-09T05:24:29.599 DEBUG:teuthology.parallel:result is None 2024-05-09T05:24:29.600 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:24:29.835 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T05:24:29.857 INFO:teuthology.orchestra.run.smithi115.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T05:24:29.858 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T05:24:29.858 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T05:24:29.860 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:24:30.015 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T05:24:30.042 INFO:teuthology.orchestra.run.smithi161.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T05:24:30.043 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T05:24:30.043 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T05:24:30.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:24:30.189 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T05:24:30.215 INFO:teuthology.orchestra.run.smithi195.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T05:24:30.215 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T05:24:30.215 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T05:24:30.217 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T05:24:30.217 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:24:30.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T05:24:30.246 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:24:30.246 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T05:24:30.278 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:24:30.278 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T05:24:30.309 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T05:24:30.309 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:24:30.309 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T05:24:30.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T05:24:30.401 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:24:30.401 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T05:24:30.430 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T05:24:30.497 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:24:30.498 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T05:24:30.526 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T05:24:30.596 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T05:24:30.596 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:24:30.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T05:24:30.622 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T05:24:30.686 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:24:30.687 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T05:24:30.712 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T05:24:30.779 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:24:30.779 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T05:24:30.807 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T05:24:30.875 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T05:24:30.875 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:24:30.875 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T05:24:30.901 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T05:24:30.965 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:24:30.965 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T05:24:30.990 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T05:24:31.056 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:24:31.056 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T05:24:31.085 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T05:24:31.154 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T05:24:31.271 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-09T05:24:31.271 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:24:31.271 INFO:tasks.cephadm:Cluster fsid is 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:24:31.272 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T05:24:31.272 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.115', 'mon.b': '172.21.15.161', 'mon.c': '172.21.15.195'} 2024-05-09T05:24:31.272 INFO:tasks.cephadm:First mon is mon.a on smithi115 2024-05-09T05:24:31.272 INFO:tasks.cephadm:First mgr is x 2024-05-09T05:24:31.272 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T05:24:31.272 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2024-05-09T05:24:31.297 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-05-09T05:24:31.322 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname $(hostname -s) 2024-05-09T05:24:31.347 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-09T05:24:31.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:24:31.515 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79226/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 00:57:42.125322', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79224/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:52:58.989794', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-09T05:24:31.665 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240508.183908-debug, sha1 c9da4049f638c0f84bdd9b4e33c42e86046ac554 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:24:31.667 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm 2024-05-09T05:24:31.667 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T05:24:32.141 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 05:24 /home/ubuntu/cephtest/cephadm 2024-05-09T05:24:32.141 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T05:24:32.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 05:24 /home/ubuntu/cephtest/cephadm 2024-05-09T05:24:32.594 DEBUG:teuthology.orchestra.run.smithi195:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T05:24:33.190 INFO:teuthology.orchestra.run.smithi195.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 05:24 /home/ubuntu/cephtest/cephadm 2024-05-09T05:24:33.254 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T05:24:33.278 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T05:24:33.306 DEBUG:teuthology.orchestra.run.smithi195:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T05:24:33.355 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 on all hosts... 2024-05-09T05:24:33.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T05:24:33.359 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T05:24:33.363 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T05:24:34.934 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T05:24:34.952 INFO:teuthology.orchestra.run.smithi195.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T05:24:35.057 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T05:25:11.393 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-09T05:25:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T05:25:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T05:25:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "repo_digests": [ 2024-05-09T05:25:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T05:25:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-05-09T05:25:11.394 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-09T05:25:23.478 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:25:23.479 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T05:25:23.479 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T05:25:23.479 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2024-05-09T05:25:23.479 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T05:25:23.479 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T05:25:23.479 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:25:24.994 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-09T05:25:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T05:25:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T05:25:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-05-09T05:25:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T05:25:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-09T05:25:24.994 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-09T05:25:25.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2024-05-09T05:25:25.053 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-05-09T05:25:25.080 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph 2024-05-09T05:25:25.114 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2024-05-09T05:25:25.142 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-05-09T05:25:25.167 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /etc/ceph 2024-05-09T05:25:25.195 INFO:tasks.cephadm:Writing seed config... 2024-05-09T05:25:25.196 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T05:25:25.196 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T05:25:25.196 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T05:25:25.196 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T05:25:25.196 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-09T05:25:25.197 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-09T05:25:25.198 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-09T05:25:25.199 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-09T05:25:25.200 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:25:25.200 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T05:25:25.217 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 69de8a38-0dc4-11ef-bc97-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T05:25:25.217 DEBUG:teuthology.orchestra.run.smithi115:mon.a> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.a.service 2024-05-09T05:25:25.260 DEBUG:teuthology.orchestra.run.smithi115:mgr.x> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.x.service 2024-05-09T05:25:25.302 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T05:25:25.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 -v bootstrap --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.115 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:25:25.495 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-09T05:25:25.496 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554', '-v', 'bootstrap', '--fsid', '69de8a38-0dc4-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.115', '--skip-admin-label'] 2024-05-09T05:25:25.496 INFO:teuthology.orchestra.run.smithi115.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T05:25:25.496 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying podman|docker is present... 2024-05-09T05:25:25.523 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T05:25:25.523 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying lvm2 is present... 2024-05-09T05:25:25.523 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying time synchronization is in place... 2024-05-09T05:25:25.531 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T05:25:25.531 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T05:25:25.538 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T05:25:25.538 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-05-09T05:25:25.544 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout enabled 2024-05-09T05:25:25.550 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout active 2024-05-09T05:25:25.550 INFO:teuthology.orchestra.run.smithi115.stdout:Unit chronyd.service is enabled and running 2024-05-09T05:25:25.551 INFO:teuthology.orchestra.run.smithi115.stdout:Repeating the final host check... 2024-05-09T05:25:25.574 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T05:25:25.574 INFO:teuthology.orchestra.run.smithi115.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T05:25:25.574 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl is present 2024-05-09T05:25:25.574 INFO:teuthology.orchestra.run.smithi115.stdout:lvcreate is present 2024-05-09T05:25:25.580 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T05:25:25.580 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T05:25:25.586 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T05:25:25.587 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-05-09T05:25:25.593 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout enabled 2024-05-09T05:25:25.600 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout active 2024-05-09T05:25:25.600 INFO:teuthology.orchestra.run.smithi115.stdout:Unit chronyd.service is enabled and running 2024-05-09T05:25:25.600 INFO:teuthology.orchestra.run.smithi115.stdout:Host looks OK 2024-05-09T05:25:25.600 INFO:teuthology.orchestra.run.smithi115.stdout:Cluster fsid: 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:25:25.600 INFO:teuthology.orchestra.run.smithi115.stdout:Acquiring lock 139984026611136 on /run/cephadm/69de8a38-0dc4-11ef-bc97-c7b262605968.lock 2024-05-09T05:25:25.600 INFO:teuthology.orchestra.run.smithi115.stdout:Lock 139984026611136 acquired on /run/cephadm/69de8a38-0dc4-11ef-bc97-c7b262605968.lock 2024-05-09T05:25:25.600 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying IP 172.21.15.115 port 3300 ... 2024-05-09T05:25:25.601 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying IP 172.21.15.115 port 6789 ... 2024-05-09T05:25:25.601 INFO:teuthology.orchestra.run.smithi115.stdout:Base mon IP(s) is [172.21.15.115:3300, 172.21.15.115:6789], mon addrv is [v2:172.21.15.115:3300,v1:172.21.15.115:6789] 2024-05-09T05:25:25.604 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.115 metric 100 2024-05-09T05:25:25.604 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.115 metric 100 2024-05-09T05:25:25.607 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T05:25:25.607 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-09T05:25:25.607 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2024-05-09T05:25:25.609 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T05:25:25.609 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T05:25:25.610 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T05:25:25.610 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-09T05:25:25.610 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2024-05-09T05:25:25.610 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T05:25:25.612 INFO:teuthology.orchestra.run.smithi115.stdout:Mon IP `172.21.15.115` is in CIDR network `172.21.0.0/20` 2024-05-09T05:25:25.612 INFO:teuthology.orchestra.run.smithi115.stdout:Mon IP `172.21.15.115` is in CIDR network `172.21.0.0/20` 2024-05-09T05:25:25.612 INFO:teuthology.orchestra.run.smithi115.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T05:25:25.612 INFO:teuthology.orchestra.run.smithi115.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T05:25:25.612 INFO:teuthology.orchestra.run.smithi115.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T05:25:26.048 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T05:25:26.048 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T05:25:26.049 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T05:25:26.049 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying blob sha256:0411e0fd7a5d45f8ab31b23cec89f97c70494de39461f644c7cdc77de3b9cbca 2024-05-09T05:25:26.049 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-09T05:25:26.049 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying config sha256:db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T05:25:26.049 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T05:25:26.931 INFO:teuthology.orchestra.run.smithi115.stdout:ceph: stdout ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:25:26.931 INFO:teuthology.orchestra.run.smithi115.stdout:Ceph version: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:25:26.931 INFO:teuthology.orchestra.run.smithi115.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T05:25:27.848 INFO:teuthology.orchestra.run.smithi115.stdout:stat: stdout 167 167 2024-05-09T05:25:27.848 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial keys... 2024-05-09T05:25:28.778 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQBIXjxmrvNNChAAKXbq9D/PXnLWtHBmhOoatg== 2024-05-09T05:25:29.642 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQBJXjxmF4orBhAAv/VDlKz0b1kpllYKj0iJAg== 2024-05-09T05:25:30.540 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQBJXjxm0RSdORAAjX3jPNrDVermxN6OTJc5hw== 2024-05-09T05:25:30.540 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial monmap... 2024-05-09T05:25:31.422 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T05:25:31.422 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T05:25:31.423 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:25:31.423 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T05:25:31.423 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool for a [v2:172.21.15.115:3300,v1:172.21.15.115:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T05:25:31.423 INFO:teuthology.orchestra.run.smithi115.stdout:setting min_mon_release = quincy 2024-05-09T05:25:31.423 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: set fsid to 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:25:31.423 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T05:25:31.423 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:25:31.423 INFO:teuthology.orchestra.run.smithi115.stdout:Creating mon... 2024-05-09T05:25:32.353 INFO:teuthology.orchestra.run.smithi115.stdout:create mon.a on 2024-05-09T05:25:32.552 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T05:25:32.692 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T05:25:32.857 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968.target. 2024-05-09T05:25:32.857 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968.target. 2024-05-09T05:25:33.101 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.a 2024-05-09T05:25:33.101 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to reset failed state of unit ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-09T05:25:33.261 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968.target.wants/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@.service. 2024-05-09T05:25:33.810 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T05:25:33.811 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-05-09T05:25:33.818 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T05:25:33.818 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-05-09T05:25:33.818 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-05-09T05:25:33.818 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T05:25:33.818 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mon to start... 2024-05-09T05:25:33.818 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mon... 2024-05-09T05:25:34.839 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T05:25:34.845 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout id: 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:25:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T05:25:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout services: 2024-05-09T05:25:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.584323s) 2024-05-09T05:25:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T05:25:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T05:25:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout data: 2024-05-09T05:25:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T05:25:34.847 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T05:25:34.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T05:25:34.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T05:25:34.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:34.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.279+0000 7f9f47ddd700 1 Processor -- start 2024-05-09T05:25:34.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.280+0000 7f9f47ddd700 1 -- start start 2024-05-09T05:25:34.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.280+0000 7f9f47ddd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f40108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:34.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.280+0000 7f9f47ddd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9f401093d0 con 0x7f9f401089d0 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.280+0000 7f9f45b79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f40108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.280+0000 7f9f45b79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f40108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51360/0 (socket says 172.21.15.115:51360) 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.280+0000 7f9f45b79700 1 -- 172.21.15.115:0/3138401974 learned_addr learned my addr 172.21.15.115:0/3138401974 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.280+0000 7f9f45b79700 1 -- 172.21.15.115:0/3138401974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f40109ba0 con 0x7f9f401089d0 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.281+0000 7f9f45b79700 1 --2- 172.21.15.115:0/3138401974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f40108de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f340098a0 tx=0x7f9f34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ca330fce95b4b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.281+0000 7f9f44b77700 1 -- 172.21.15.115:0/3138401974 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f34004160 con 0x7f9f401089d0 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.281+0000 7f9f44b77700 1 -- 172.21.15.115:0/3138401974 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f34004300 con 0x7f9f401089d0 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.281+0000 7f9f44b77700 1 -- 172.21.15.115:0/3138401974 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f340036a0 con 0x7f9f401089d0 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.281+0000 7f9f47ddd700 1 -- 172.21.15.115:0/3138401974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 msgr2=0x7f9f40108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.281+0000 7f9f47ddd700 1 --2- 172.21.15.115:0/3138401974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f40108de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f340098a0 tx=0x7f9f34009bb0 comp rx=0 tx=0).stop 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.282+0000 7f9f47ddd700 1 -- 172.21.15.115:0/3138401974 shutdown_connections 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.282+0000 7f9f47ddd700 1 --2- 172.21.15.115:0/3138401974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f40108de0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:34.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.282+0000 7f9f47ddd700 1 -- 172.21.15.115:0/3138401974 >> 172.21.15.115:0/3138401974 conn(0x7f9f401040b0 msgr2=0x7f9f40106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.282+0000 7f9f47ddd700 1 -- 172.21.15.115:0/3138401974 shutdown_connections 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.282+0000 7f9f47ddd700 1 -- 172.21.15.115:0/3138401974 wait complete. 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.282+0000 7f9f47ddd700 1 Processor -- start 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.282+0000 7f9f47ddd700 1 -- start start 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f47ddd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f4007d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f47ddd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4010a0d0 con 0x7f9f401089d0 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f45b79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f4007d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f45b79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f4007d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51372/0 (socket says 172.21.15.115:51372) 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f45b79700 1 -- 172.21.15.115:0/971867434 learned_addr learned my addr 172.21.15.115:0/971867434 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f45b79700 1 -- 172.21.15.115:0/971867434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4007d710 con 0x7f9f401089d0 2024-05-09T05:25:34.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f45b79700 1 --2- 172.21.15.115:0/971867434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f4007d1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9f34000c00 tx=0x7f9f34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:34.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f32ffd700 1 -- 172.21.15.115:0/971867434 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f34003db0 con 0x7f9f401089d0 2024-05-09T05:25:34.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f32ffd700 1 -- 172.21.15.115:0/971867434 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f34011630 con 0x7f9f401089d0 2024-05-09T05:25:34.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f32ffd700 1 -- 172.21.15.115:0/971867434 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f34011950 con 0x7f9f401089d0 2024-05-09T05:25:34.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f47ddd700 1 -- 172.21.15.115:0/971867434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f401a0340 con 0x7f9f401089d0 2024-05-09T05:25:34.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.283+0000 7f9f47ddd700 1 -- 172.21.15.115:0/971867434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f401a07c0 con 0x7f9f401089d0 2024-05-09T05:25:34.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.284+0000 7f9f32ffd700 1 -- 172.21.15.115:0/971867434 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f3401a460 con 0x7f9f401089d0 2024-05-09T05:25:34.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.284+0000 7f9f32ffd700 1 -- 172.21.15.115:0/971867434 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f3401b780 con 0x7f9f401089d0 2024-05-09T05:25:34.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.284+0000 7f9f47ddd700 1 -- 172.21.15.115:0/971867434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f40064e20 con 0x7f9f401089d0 2024-05-09T05:25:34.852 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.287+0000 7f9f32ffd700 1 -- 172.21.15.115:0/971867434 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9f3401b980 con 0x7f9f401089d0 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.357+0000 7f9f47ddd700 1 -- 172.21.15.115:0/971867434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9f400643d0 con 0x7f9f401089d0 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.357+0000 7f9f32ffd700 1 -- 172.21.15.115:0/971867434 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f9f3401bb60 con 0x7f9f401089d0 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.358+0000 7f9f47ddd700 1 -- 172.21.15.115:0/971867434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 msgr2=0x7f9f4007d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.358+0000 7f9f47ddd700 1 --2- 172.21.15.115:0/971867434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f4007d1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9f34000c00 tx=0x7f9f34004750 comp rx=0 tx=0).stop 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.358+0000 7f9f47ddd700 1 -- 172.21.15.115:0/971867434 shutdown_connections 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.358+0000 7f9f47ddd700 1 --2- 172.21.15.115:0/971867434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f401089d0 0x7f9f4007d1d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.358+0000 7f9f47ddd700 1 -- 172.21.15.115:0/971867434 >> 172.21.15.115:0/971867434 conn(0x7f9f401040b0 msgr2=0x7f9f40192bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.358+0000 7f9f47ddd700 1 -- 172.21.15.115:0/971867434 shutdown_connections 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:34.358+0000 7f9f47ddd700 1 -- 172.21.15.115:0/971867434 wait complete. 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:mon is available 2024-05-09T05:25:34.853 INFO:teuthology.orchestra.run.smithi115.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T05:25:35.935 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:35 smithi115 ceph-mon[29970]: from='client.? 172.21.15.115:0/1957254598' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout fsid = 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.115:3300,v1:172.21.15.115:6789] 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T05:25:35.945 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.277+0000 7efdb44b1700 1 Processor -- start 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.277+0000 7efdb44b1700 1 -- start start 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.277+0000 7efdb44b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac10ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:35.946 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.277+0000 7efdb44b1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efdac07d6a0 con 0x7efdac10a610 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.277+0000 7efdb224d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac10ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.277+0000 7efdb224d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac10ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51376/0 (socket says 172.21.15.115:51376) 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.277+0000 7efdb224d700 1 -- 172.21.15.115:0/2988345030 learned_addr learned my addr 172.21.15.115:0/2988345030 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.278+0000 7efdb224d700 1 -- 172.21.15.115:0/2988345030 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac10cf70 con 0x7efdac10a610 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.278+0000 7efdb224d700 1 --2- 172.21.15.115:0/2988345030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac10ca30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efda80098a0 tx=0x7efda8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddd34c053f12806 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.278+0000 7efdb124b700 1 -- 172.21.15.115:0/2988345030 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efda8004160 con 0x7efdac10a610 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.278+0000 7efdb124b700 1 -- 172.21.15.115:0/2988345030 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda8004300 con 0x7efdac10a610 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.278+0000 7efdb124b700 1 -- 172.21.15.115:0/2988345030 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efda80036a0 con 0x7efdac10a610 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.279+0000 7efdb44b1700 1 -- 172.21.15.115:0/2988345030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 msgr2=0x7efdac10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.279+0000 7efdb44b1700 1 --2- 172.21.15.115:0/2988345030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac10ca30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efda80098a0 tx=0x7efda8009bb0 comp rx=0 tx=0).stop 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.279+0000 7efdb44b1700 1 -- 172.21.15.115:0/2988345030 shutdown_connections 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.279+0000 7efdb44b1700 1 --2- 172.21.15.115:0/2988345030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac10ca30 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.279+0000 7efdb44b1700 1 -- 172.21.15.115:0/2988345030 >> 172.21.15.115:0/2988345030 conn(0x7efdac1040b0 msgr2=0x7efdac106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:35.947 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.279+0000 7efdb44b1700 1 -- 172.21.15.115:0/2988345030 shutdown_connections 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.279+0000 7efdb44b1700 1 -- 172.21.15.115:0/2988345030 wait complete. 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.279+0000 7efdb44b1700 1 Processor -- start 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.279+0000 7efdb44b1700 1 -- start start 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efdb44b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac109d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efdb44b1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10d380 con 0x7efdac10a610 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efdb224d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efdb224d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51390/0 (socket says 172.21.15.115:51390) 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efdb224d700 1 -- 172.21.15.115:0/1957254598 learned_addr learned my addr 172.21.15.115:0/1957254598 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efdb224d700 1 -- 172.21.15.115:0/1957254598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac10a2d0 con 0x7efdac10a610 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efdb224d700 1 --2- 172.21.15.115:0/1957254598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac109d90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7efda8000c00 tx=0x7efda8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efda37fe700 1 -- 172.21.15.115:0/1957254598 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efda8003db0 con 0x7efdac10a610 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efda37fe700 1 -- 172.21.15.115:0/1957254598 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda8011630 con 0x7efdac10a610 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efdb44b1700 1 -- 172.21.15.115:0/1957254598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac108520 con 0x7efdac10a610 2024-05-09T05:25:35.948 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efda37fe700 1 -- 172.21.15.115:0/1957254598 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efda8011950 con 0x7efdac10a610 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.280+0000 7efdb44b1700 1 -- 172.21.15.115:0/1957254598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac108a00 con 0x7efdac10a610 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.281+0000 7efda37fe700 1 -- 172.21.15.115:0/1957254598 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efda801a460 con 0x7efdac10a610 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.281+0000 7efda37fe700 1 -- 172.21.15.115:0/1957254598 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efda801b780 con 0x7efdac10a610 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.281+0000 7efdb44b1700 1 -- 172.21.15.115:0/1957254598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdac064e20 con 0x7efdac10a610 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.284+0000 7efda37fe700 1 -- 172.21.15.115:0/1957254598 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7efda8031020 con 0x7efdac10a610 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.350+0000 7efdb44b1700 1 -- 172.21.15.115:0/1957254598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7efdac196b60 con 0x7efdac10a610 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.353+0000 7efda37fe700 1 -- 172.21.15.115:0/1957254598 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7efda801b9d0 con 0x7efdac10a610 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.354+0000 7efdb44b1700 1 -- 172.21.15.115:0/1957254598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 msgr2=0x7efdac109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.354+0000 7efda37fe700 1 -- 172.21.15.115:0/1957254598 <== mon.0 v2:172.21.15.115:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7efda801ad00 con 0x7efdac10a610 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.354+0000 7efdb44b1700 1 --2- 172.21.15.115:0/1957254598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac109d90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7efda8000c00 tx=0x7efda8004750 comp rx=0 tx=0).stop 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.354+0000 7efdb44b1700 1 -- 172.21.15.115:0/1957254598 shutdown_connections 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.354+0000 7efdb44b1700 1 --2- 172.21.15.115:0/1957254598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdac10a610 0x7efdac109d90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:35.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.354+0000 7efdb44b1700 1 -- 172.21.15.115:0/1957254598 >> 172.21.15.115:0/1957254598 conn(0x7efdac1040b0 msgr2=0x7efdac105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:35.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.355+0000 7efdb44b1700 1 -- 172.21.15.115:0/1957254598 shutdown_connections 2024-05-09T05:25:35.950 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:35.355+0000 7efdb44b1700 1 -- 172.21.15.115:0/1957254598 wait complete. 2024-05-09T05:25:35.950 INFO:teuthology.orchestra.run.smithi115.stdout:Generating new minimal ceph.conf... 2024-05-09T05:25:36.965 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.392+0000 7f3a273a6700 1 Processor -- start 2024-05-09T05:25:36.965 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.392+0000 7f3a273a6700 1 -- start start 2024-05-09T05:25:36.965 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.392+0000 7f3a273a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2010a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:36.965 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.392+0000 7f3a273a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2010a9b0 con 0x7f3a20109fd0 2024-05-09T05:25:36.965 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.392+0000 7f3a25142700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2010a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:36.965 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.392+0000 7f3a25142700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2010a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51404/0 (socket says 172.21.15.115:51404) 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.392+0000 7f3a25142700 1 -- 172.21.15.115:0/3717653126 learned_addr learned my addr 172.21.15.115:0/3717653126 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.392+0000 7f3a25142700 1 -- 172.21.15.115:0/3717653126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a20108c50 con 0x7f3a20109fd0 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.393+0000 7f3a25142700 1 --2- 172.21.15.115:0/3717653126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2010a3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000bb40 tx=0x7f3a1000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f541a7c37f62c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.393+0000 7f3a17fff700 1 -- 172.21.15.115:0/3717653126 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a1000dbe0 con 0x7f3a20109fd0 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.393+0000 7f3a17fff700 1 -- 172.21.15.115:0/3717653126 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3a100044a0 con 0x7f3a20109fd0 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.393+0000 7f3a17fff700 1 -- 172.21.15.115:0/3717653126 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a10003e30 con 0x7f3a20109fd0 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.393+0000 7f3a273a6700 1 -- 172.21.15.115:0/3717653126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 msgr2=0x7f3a2010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.393+0000 7f3a273a6700 1 --2- 172.21.15.115:0/3717653126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2010a3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000bb40 tx=0x7f3a1000be50 comp rx=0 tx=0).stop 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.394+0000 7f3a273a6700 1 -- 172.21.15.115:0/3717653126 shutdown_connections 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.394+0000 7f3a273a6700 1 --2- 172.21.15.115:0/3717653126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2010a3e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.394+0000 7f3a273a6700 1 -- 172.21.15.115:0/3717653126 >> 172.21.15.115:0/3717653126 conn(0x7f3a20103810 msgr2=0x7f3a20105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.394+0000 7f3a273a6700 1 -- 172.21.15.115:0/3717653126 shutdown_connections 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.394+0000 7f3a273a6700 1 -- 172.21.15.115:0/3717653126 wait complete. 2024-05-09T05:25:36.966 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.394+0000 7f3a273a6700 1 Processor -- start 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.394+0000 7f3a273a6700 1 -- start start 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a273a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a273a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20107b40 con 0x7f3a20109fd0 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a25142700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a25142700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2019bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51408/0 (socket says 172.21.15.115:51408) 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a25142700 1 -- 172.21.15.115:0/2127978712 learned_addr learned my addr 172.21.15.115:0/2127978712 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a25142700 1 -- 172.21.15.115:0/2127978712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2019c0f0 con 0x7f3a20109fd0 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a25142700 1 --2- 172.21.15.115:0/2127978712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2019bbb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000b340 tx=0x7f3a10003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a167fc700 1 -- 172.21.15.115:0/2127978712 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a10027070 con 0x7f3a20109fd0 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a167fc700 1 -- 172.21.15.115:0/2127978712 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3a100109f0 con 0x7f3a20109fd0 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a273a6700 1 -- 172.21.15.115:0/2127978712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2019c380 con 0x7f3a20109fd0 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a167fc700 1 -- 172.21.15.115:0/2127978712 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a10022980 con 0x7f3a20109fd0 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.395+0000 7f3a273a6700 1 -- 172.21.15.115:0/2127978712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2019ef90 con 0x7f3a20109fd0 2024-05-09T05:25:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.396+0000 7f3a167fc700 1 -- 172.21.15.115:0/2127978712 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a1002a070 con 0x7f3a20109fd0 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.396+0000 7f3a167fc700 1 -- 172.21.15.115:0/2127978712 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a1001f070 con 0x7f3a20109fd0 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.396+0000 7f3a273a6700 1 -- 172.21.15.115:0/2127978712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a20064e20 con 0x7f3a20109fd0 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.399+0000 7f3a167fc700 1 -- 172.21.15.115:0/2127978712 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3a10031070 con 0x7f3a20109fd0 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.466+0000 7f3a273a6700 1 -- 172.21.15.115:0/2127978712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"} v 0) -- 0x7f3a2019f160 con 0x7f3a20109fd0 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.467+0000 7f3a167fc700 1 -- 172.21.15.115:0/2127978712 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]=0 v2) ==== 126+0+177 (secure 0 0 0) 0x7f3a10015070 con 0x7f3a20109fd0 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.468+0000 7f3a273a6700 1 -- 172.21.15.115:0/2127978712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 msgr2=0x7f3a2019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.468+0000 7f3a273a6700 1 --2- 172.21.15.115:0/2127978712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2019bbb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000b340 tx=0x7f3a10003f80 comp rx=0 tx=0).stop 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.468+0000 7f3a273a6700 1 -- 172.21.15.115:0/2127978712 shutdown_connections 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.468+0000 7f3a273a6700 1 --2- 172.21.15.115:0/2127978712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a20109fd0 0x7f3a2019bbb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.468+0000 7f3a273a6700 1 -- 172.21.15.115:0/2127978712 >> 172.21.15.115:0/2127978712 conn(0x7f3a20103810 msgr2=0x7f3a20191e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.468+0000 7f3a273a6700 1 -- 172.21.15.115:0/2127978712 shutdown_connections 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:36.468+0000 7f3a273a6700 1 -- 172.21.15.115:0/2127978712 wait complete. 2024-05-09T05:25:36.968 INFO:teuthology.orchestra.run.smithi115.stdout:Restarting the monitor... 2024-05-09T05:25:37.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:36 smithi115 ceph-mon[29970]: from='client.? 172.21.15.115:0/2127978712' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]: dispatch 2024-05-09T05:25:37.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:36 smithi115 systemd[1]: Stopping Ceph mon.a for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:25:37.553 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:37 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a[29966]: 2024-05-09T05:25:37.249+0000 7f637c3a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:25:37.553 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:37 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a[29966]: 2024-05-09T05:25:37.249+0000 7f637c3a6700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T05:25:37.553 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:37 smithi115 podman[30216]: 2024-05-09 05:25:37.291121216 +0000 UTC m=+0.160684016 container died 09b58308a9458465b631804b7517e5d043e7b4480cc857a544160389f1add636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T05:25:37.907 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:37 smithi115 podman[30216]: 2024-05-09 05:25:37.658265844 +0000 UTC m=+0.527828695 container cleanup 09b58308a9458465b631804b7517e5d043e7b4480cc857a544160389f1add636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:25:37.907 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:37 smithi115 bash[30216]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a 2024-05-09T05:25:38.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:37 smithi115 podman[30230]: 2024-05-09 05:25:37.900654908 +0000 UTC m=+0.604598281 container remove 09b58308a9458465b631804b7517e5d043e7b4480cc857a544160389f1add636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T05:25:38.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T05:25:38.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 systemd[1]: Stopped Ceph mon.a for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:25:38.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 systemd[1]: Starting Ceph mon.a for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:25:38.499 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 podman[30330]: 2024-05-09 05:25:38.234976248 +0000 UTC m=+0.103871073 container create 8cbc383f681f2c98febd5d219163594e98e58ef270fde82cf6cf750efb543c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 podman[30330]: 2024-05-09 05:25:38.149195498 +0000 UTC m=+0.018090295 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 podman[30330]: 2024-05-09 05:25:38.437291611 +0000 UTC m=+0.306186398 container init 8cbc383f681f2c98febd5d219163594e98e58ef270fde82cf6cf750efb543c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 podman[30330]: 2024-05-09 05:25:38.44037822 +0000 UTC m=+0.309273000 container start 8cbc383f681f2c98febd5d219163594e98e58ef270fde82cf6cf750efb543c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: pidfile_write: ignore empty --pid-file 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: load: jerasure load: lrc 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: RocksDB version: 7.9.2 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Git sha 0 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: DB SUMMARY 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: DB Session ID: AFU8FUC96A6KCY4TTXIM 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: CURRENT file: CURRENT 2024-05-09T05:25:38.500 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104664 ; 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.error_if_exists: 0 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.create_if_missing: 0 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.paranoid_checks: 1 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.env: 0x55a85057c500 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.info_log: 0x55a851f62020 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.statistics: (nil) 2024-05-09T05:25:38.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.use_fsync: 0 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_log_file_size: 0 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.allow_fallocate: 1 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.use_direct_reads: 0 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T05:25:38.502 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.db_log_dir: 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.wal_dir: 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T05:25:38.503 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.write_buffer_manager: 0x55a851e2c8c0 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.unordered_write: 0 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.row_cache: None 2024-05-09T05:25:38.504 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.wal_filter: None 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.two_write_queues: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.wal_compression: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.atomic_flush: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.log_readahead_size: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T05:25:38.505 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_background_jobs: 2 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_background_compactions: -1 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_subcompactions: 1 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_open_files: -1 2024-05-09T05:25:38.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T05:25:38.507 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T05:25:38.507 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T05:25:38.507 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T05:25:38.507 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_background_flushes: -1 2024-05-09T05:25:38.507 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Compression algorithms supported: 2024-05-09T05:25:38.507 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: kZSTD supported: 0 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: kXpressCompression supported: 0 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: kZlibCompression supported: 1 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: kSnappyCompression supported: 1 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: kLZ4Compression supported: 1 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: kBZip2Compression supported: 0 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T05:25:38.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.merge_operator: 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_filter: None 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a851f621e0) 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout: cache_index_and_filter_blocks: 1 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout: pin_top_level_index_and_filter: 1 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout: index_type: 0 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout: data_block_index_type: 0 2024-05-09T05:25:38.509 INFO:journalctl@ceph.mon.a.smithi115.stdout: index_shortening: 1 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: checksum: 4 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: no_block_cache: 0 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache: 0x55a851e5f1f0 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache_name: BinnedLRUCache 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache_options: 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: capacity : 536870912 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: num_shard_bits : 4 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: strict_capacity_limit : 0 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: high_pri_pool_ratio: 0.000 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache_compressed: (nil) 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: persistent_cache: (nil) 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_size: 4096 2024-05-09T05:25:38.510 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_size_deviation: 10 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_restart_interval: 16 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: index_block_restart_interval: 1 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: metadata_block_size: 4096 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: partition_filters: 0 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: use_delta_encoding: 1 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: filter_policy: bloomfilter 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: whole_key_filtering: 1 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: verify_compression: 0 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: read_amp_bytes_per_bit: 0 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: format_version: 5 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: enable_index_compression: 1 2024-05-09T05:25:38.511 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_align: 0 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout: max_auto_readahead_size: 262144 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout: prepopulate_block_cache: 0 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout: initial_auto_readahead_size: 8192 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression: NoCompression 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.num_levels: 7 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T05:25:38.512 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T05:25:38.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T05:25:38.514 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T05:25:38.514 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T05:25:38.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T05:25:38.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.inplace_update_support: 0 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T05:25:38.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T05:25:38.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T05:25:38.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.bloom_locality: 0 2024-05-09T05:25:38.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.max_successive_merges: 0 2024-05-09T05:25:38.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T05:25:38.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T05:25:38.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T05:25:38.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T05:25:38.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.ttl: 2592000 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.enable_blob_files: false 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.min_blob_size: 0 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T05:25:38.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f97360d0-1c59-4dc5-a955-e4097e361130 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715232338480027, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715232338481083, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99919, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97801, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14010, "raw_average_key_size": 49, "raw_value_size": 90149, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715232338, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f97360d0-1c59-4dc5-a955-e4097e361130", "db_session_id": "AFU8FUC96A6KCY4TTXIM", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715232338481170, "job": 1, "event": "recovery_finished"} 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a851e43c00 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: DB pointer 0x55a856c36000 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** DB Stats ** 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T05:25:38.520 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: L0 2/0 99.44 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: Sum 2/0 99.44 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-05-09T05:25:38.521 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 153.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative compaction: 0.00 GB write, 28.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T05:25:38.522 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval compaction: 0.00 GB write, 28.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout: Block cache BinnedLRUCache@0x55a851e5f1f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: starting mon.a rank 0 at public addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] at bind addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mon.a@-1(???) e1 preinit fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mon.a@-1(???).mds e1 new map 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mon.a@-1(???).mds e1 print_map 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout: e1 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T05:25:38.523 INFO:journalctl@ceph.mon.a.smithi115.stdout: legacy client fscid: -1 2024-05-09T05:25:38.524 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-05-09T05:25:38.524 INFO:journalctl@ceph.mon.a.smithi115.stdout: No filesystems configured 2024-05-09T05:25:38.524 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T05:25:38.524 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T05:25:38.524 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T05:25:38.524 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T05:25:38.524 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T05:25:38.524 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 bash[30330]: 8cbc383f681f2c98febd5d219163594e98e58ef270fde82cf6cf750efb543c68 2024-05-09T05:25:38.524 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 systemd[1]: Started Ceph mon.a for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:25:38.524 INFO:teuthology.orchestra.run.smithi115.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T05:25:38.751 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T05:25:38.751 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: monmap epoch 1 2024-05-09T05:25:38.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:25:38.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: last_changed 2024-05-09T05:25:30.870570+0000 2024-05-09T05:25:38.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:25:38.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: min_mon_release 19 (squid) 2024-05-09T05:25:38.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: election_strategy: 1 2024-05-09T05:25:38.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:25:38.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: fsmap 2024-05-09T05:25:38.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T05:25:38.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:38 smithi115 ceph-mon[30349]: mgrmap e1: no daemons active 2024-05-09T05:25:39.613 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.991+0000 7f17ffddf700 1 Processor -- start 2024-05-09T05:25:39.613 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.991+0000 7f17ffddf700 1 -- start start 2024-05-09T05:25:39.614 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.991+0000 7f17ffddf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f810c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:39.614 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.991+0000 7f17ffddf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f17f807d700 con 0x7f17f810a5b0 2024-05-09T05:25:39.614 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.992+0000 7f17fdb7b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f810c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:39.614 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.992+0000 7f17fdb7b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f810c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51418/0 (socket says 172.21.15.115:51418) 2024-05-09T05:25:39.614 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.992+0000 7f17fdb7b700 1 -- 172.21.15.115:0/170718233 learned_addr learned my addr 172.21.15.115:0/170718233 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:39.614 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.992+0000 7f17fdb7b700 1 -- 172.21.15.115:0/170718233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f810cf10 con 0x7f17f810a5b0 2024-05-09T05:25:39.614 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.992+0000 7f17fdb7b700 1 --2- 172.21.15.115:0/170718233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f810c9d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009e80 tx=0x7f17f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1af52cab44babe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:39.614 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17fcb79700 1 -- 172.21.15.115:0/170718233 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f4017070 con 0x7f17f810a5b0 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17fcb79700 1 -- 172.21.15.115:0/170718233 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f17f400b8c0 con 0x7f17f810a5b0 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17fcb79700 1 -- 172.21.15.115:0/170718233 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f4004420 con 0x7f17f810a5b0 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17ffddf700 1 -- 172.21.15.115:0/170718233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 msgr2=0x7f17f810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17ffddf700 1 --2- 172.21.15.115:0/170718233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f810c9d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009e80 tx=0x7f17f400b330 comp rx=0 tx=0).stop 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17ffddf700 1 -- 172.21.15.115:0/170718233 shutdown_connections 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17ffddf700 1 --2- 172.21.15.115:0/170718233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f810c9d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17ffddf700 1 -- 172.21.15.115:0/170718233 >> 172.21.15.115:0/170718233 conn(0x7f17f81040b0 msgr2=0x7f17f8106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17ffddf700 1 -- 172.21.15.115:0/170718233 shutdown_connections 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.993+0000 7f17ffddf700 1 -- 172.21.15.115:0/170718233 wait complete. 2024-05-09T05:25:39.615 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.994+0000 7f17ffddf700 1 Processor -- start 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.994+0000 7f17ffddf700 1 -- start start 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.994+0000 7f17ffddf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.994+0000 7f17ffddf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f17f810de70 con 0x7f17f810a5b0 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.994+0000 7f17fdb7b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.994+0000 7f17fdb7b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51420/0 (socket says 172.21.15.115:51420) 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.994+0000 7f17fdb7b700 1 -- 172.21.15.115:0/4020212328 learned_addr learned my addr 172.21.15.115:0/4020212328 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.995+0000 7f17fdb7b700 1 -- 172.21.15.115:0/4020212328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f819e320 con 0x7f17f810a5b0 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.995+0000 7f17fdb7b700 1 --2- 172.21.15.115:0/4020212328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f819dde0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009e50 tx=0x7f17f400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.995+0000 7f17eeffd700 1 -- 172.21.15.115:0/4020212328 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f4020020 con 0x7f17f810a5b0 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.995+0000 7f17eeffd700 1 -- 172.21.15.115:0/4020212328 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f17f4012430 con 0x7f17f810a5b0 2024-05-09T05:25:39.616 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.995+0000 7f17ffddf700 1 -- 172.21.15.115:0/4020212328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f819e5b0 con 0x7f17f810a5b0 2024-05-09T05:25:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.995+0000 7f17eeffd700 1 -- 172.21.15.115:0/4020212328 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f4017040 con 0x7f17f810a5b0 2024-05-09T05:25:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.995+0000 7f17ffddf700 1 -- 172.21.15.115:0/4020212328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f81a1180 con 0x7f17f810a5b0 2024-05-09T05:25:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.995+0000 7f17eeffd700 1 -- 172.21.15.115:0/4020212328 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f17f4023020 con 0x7f17f810a5b0 2024-05-09T05:25:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.995+0000 7f17eeffd700 1 -- 172.21.15.115:0/4020212328 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f17f400f050 con 0x7f17f810a5b0 2024-05-09T05:25:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.996+0000 7f17ffddf700 1 -- 172.21.15.115:0/4020212328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f8064e20 con 0x7f17f810a5b0 2024-05-09T05:25:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:38.999+0000 7f17eeffd700 1 -- 172.21.15.115:0/4020212328 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f17f402a020 con 0x7f17f810a5b0 2024-05-09T05:25:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:39.065+0000 7f17ffddf700 1 -- 172.21.15.115:0/4020212328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f17f81a13b0 con 0x7f17f810a5b0 2024-05-09T05:25:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:39.068+0000 7f17eeffd700 1 -- 172.21.15.115:0/4020212328 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f17f4025400 con 0x7f17f810a5b0 2024-05-09T05:25:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:39.068+0000 7f17ffddf700 1 -- 172.21.15.115:0/4020212328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 msgr2=0x7f17f819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:39.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:39.068+0000 7f17ffddf700 1 --2- 172.21.15.115:0/4020212328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f819dde0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009e50 tx=0x7f17f400bd50 comp rx=0 tx=0).stop 2024-05-09T05:25:39.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:39.069+0000 7f17ffddf700 1 -- 172.21.15.115:0/4020212328 shutdown_connections 2024-05-09T05:25:39.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:39.069+0000 7f17ffddf700 1 --2- 172.21.15.115:0/4020212328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17f810a5b0 0x7f17f819dde0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:39.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:39.069+0000 7f17ffddf700 1 -- 172.21.15.115:0/4020212328 >> 172.21.15.115:0/4020212328 conn(0x7f17f81040b0 msgr2=0x7f17f81059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:39.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:39.069+0000 7f17ffddf700 1 -- 172.21.15.115:0/4020212328 shutdown_connections 2024-05-09T05:25:39.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:39.069+0000 7f17ffddf700 1 -- 172.21.15.115:0/4020212328 wait complete. 2024-05-09T05:25:39.618 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T05:25:39.629 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:25:39.630 INFO:teuthology.orchestra.run.smithi115.stdout:Creating mgr... 2024-05-09T05:25:39.630 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T05:25:39.630 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T05:25:39.850 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.x 2024-05-09T05:25:39.850 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to reset failed state of unit ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-09T05:25:39.999 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968.target.wants/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@.service. 2024-05-09T05:25:40.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:40 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4020212328' entity='client.admin' 2024-05-09T05:25:40.476 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T05:25:40.476 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-05-09T05:25:40.484 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T05:25:40.484 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-05-09T05:25:40.484 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-05-09T05:25:40.484 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T05:25:40.495 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T05:25:40.495 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-05-09T05:25:40.503 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T05:25:40.503 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-05-09T05:25:40.503 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-05-09T05:25:40.503 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T05:25:40.503 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr to start... 2024-05-09T05:25:40.504 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr... 2024-05-09T05:25:40.564 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:40 smithi115 systemd[1]: Started Ceph mgr.x for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:25:40.859 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:40 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:40.569+0000 7f815265d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T05:25:40.859 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:40 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:40.759+0000 7f815265d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:25:40.859 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:40 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:40.864+0000 7f815265d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T05:25:41.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:41 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/494847557' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T05:25:41.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:41 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:41.192+0000 7f815265d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T05:25:41.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:41 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:41.277+0000 7f815265d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T05:25:41.604 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:41.604 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-05-09T05:25:41.604 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:25:41.604 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:41.605 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T05:25:41.606 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T05:25:41.606 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:41.606 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T05:25:41.606 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:41.606 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T05:25:41.606 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T05:25:41.606 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T05:25:41.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T05:25:41.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T05:25:41.617 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T05:25:41.618 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T05:25:41.619 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T05:25:33.776061+0000", 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.997+0000 7f89a77a2700 1 Processor -- start 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.997+0000 7f89a77a2700 1 -- start start 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.997+0000 7f89a77a2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a010a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.997+0000 7f89a77a2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89a010a940 con 0x7f89a0109f60 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.997+0000 7f89a553e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a010a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.997+0000 7f89a553e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a010a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51452/0 (socket says 172.21.15.115:51452) 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.997+0000 7f89a553e700 1 -- 172.21.15.115:0/2276046142 learned_addr learned my addr 172.21.15.115:0/2276046142 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.998+0000 7f89a553e700 1 -- 172.21.15.115:0/2276046142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a0108be0 con 0x7f89a0109f60 2024-05-09T05:25:41.620 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.998+0000 7f89a553e700 1 --2- 172.21.15.115:0/2276046142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a010a370 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f899c00bb40 tx=0x7f899c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8087740ef78d4712 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.998+0000 7f8997fff700 1 -- 172.21.15.115:0/2276046142 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f899c00dbe0 con 0x7f89a0109f60 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.998+0000 7f8997fff700 1 -- 172.21.15.115:0/2276046142 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f899c0044a0 con 0x7f89a0109f60 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.998+0000 7f8997fff700 1 -- 172.21.15.115:0/2276046142 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f899c003e30 con 0x7f89a0109f60 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.999+0000 7f89a77a2700 1 -- 172.21.15.115:0/2276046142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 msgr2=0x7f89a010a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.999+0000 7f89a77a2700 1 --2- 172.21.15.115:0/2276046142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a010a370 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f899c00bb40 tx=0x7f899c00be50 comp rx=0 tx=0).stop 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.999+0000 7f89a77a2700 1 -- 172.21.15.115:0/2276046142 shutdown_connections 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.999+0000 7f89a77a2700 1 --2- 172.21.15.115:0/2276046142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a010a370 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.999+0000 7f89a77a2700 1 -- 172.21.15.115:0/2276046142 >> 172.21.15.115:0/2276046142 conn(0x7f89a0103780 msgr2=0x7f89a0105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.999+0000 7f89a77a2700 1 -- 172.21.15.115:0/2276046142 shutdown_connections 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:40.999+0000 7f89a77a2700 1 -- 172.21.15.115:0/2276046142 wait complete. 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.000+0000 7f89a77a2700 1 Processor -- start 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.000+0000 7f89a77a2700 1 -- start start 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.000+0000 7f89a77a2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:41.621 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.000+0000 7f89a77a2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89a0107ad0 con 0x7f89a0109f60 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.000+0000 7f89a553e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.000+0000 7f89a553e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a019bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51464/0 (socket says 172.21.15.115:51464) 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.000+0000 7f89a553e700 1 -- 172.21.15.115:0/494847557 learned_addr learned my addr 172.21.15.115:0/494847557 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.001+0000 7f89a553e700 1 -- 172.21.15.115:0/494847557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a019c0c0 con 0x7f89a0109f60 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.001+0000 7f89a553e700 1 --2- 172.21.15.115:0/494847557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a019bb80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f899c00b340 tx=0x7f899c003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.001+0000 7f89967fc700 1 -- 172.21.15.115:0/494847557 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f899c027070 con 0x7f89a0109f60 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.001+0000 7f89a77a2700 1 -- 172.21.15.115:0/494847557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a019c350 con 0x7f89a0109f60 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.001+0000 7f89967fc700 1 -- 172.21.15.115:0/494847557 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f899c0109f0 con 0x7f89a0109f60 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.001+0000 7f89967fc700 1 -- 172.21.15.115:0/494847557 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f899c022980 con 0x7f89a0109f60 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.001+0000 7f89a77a2700 1 -- 172.21.15.115:0/494847557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a019ef60 con 0x7f89a0109f60 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.001+0000 7f89967fc700 1 -- 172.21.15.115:0/494847557 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f899c02a070 con 0x7f89a0109f60 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.001+0000 7f89967fc700 1 -- 172.21.15.115:0/494847557 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f899c01f070 con 0x7f89a0109f60 2024-05-09T05:25:41.622 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.002+0000 7f89a77a2700 1 -- 172.21.15.115:0/494847557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a0064e20 con 0x7f89a0109f60 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.004+0000 7f89967fc700 1 -- 172.21.15.115:0/494847557 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f899c031070 con 0x7f89a0109f60 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.077+0000 7f89a77a2700 1 -- 172.21.15.115:0/494847557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f89a0194db0 con 0x7f89a0109f60 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.078+0000 7f89967fc700 1 -- 172.21.15.115:0/494847557 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f899c015070 con 0x7f89a0109f60 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.079+0000 7f89a77a2700 1 -- 172.21.15.115:0/494847557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 msgr2=0x7f89a019bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.079+0000 7f89a77a2700 1 --2- 172.21.15.115:0/494847557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a019bb80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f899c00b340 tx=0x7f899c003f80 comp rx=0 tx=0).stop 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.079+0000 7f89a77a2700 1 -- 172.21.15.115:0/494847557 shutdown_connections 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.079+0000 7f89a77a2700 1 --2- 172.21.15.115:0/494847557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89a0109f60 0x7f89a019bb80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.079+0000 7f89a77a2700 1 -- 172.21.15.115:0/494847557 >> 172.21.15.115:0/494847557 conn(0x7f89a0103780 msgr2=0x7f89a0191e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.079+0000 7f89a77a2700 1 -- 172.21.15.115:0/494847557 shutdown_connections 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:41.080+0000 7f89a77a2700 1 -- 172.21.15.115:0/494847557 wait complete. 2024-05-09T05:25:41.623 INFO:teuthology.orchestra.run.smithi115.stdout:mgr not available, waiting (1/15)... 2024-05-09T05:25:42.802 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:42 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:42.461+0000 7f815265d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T05:25:42.803 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:42 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:42.637+0000 7f815265d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T05:25:42.803 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:42 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:42.807+0000 7f815265d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T05:25:43.372 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:43 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:43.088+0000 7f815265d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T05:25:43.372 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:43 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:43.179+0000 7f815265d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T05:25:43.372 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:43 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:43.377+0000 7f815265d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T05:25:43.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:43 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:43.566+0000 7f815265d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:25:44.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:44 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3379835401' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T05:25:44.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:44 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:44.190+0000 7f815265d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T05:25:44.664 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:44 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:44.411+0000 7f815265d200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T05:25:44.678 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:44.678 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-05-09T05:25:44.678 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-05-09T05:25:44.679 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T05:25:44.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T05:25:44.681 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T05:25:33.776061+0000", 2024-05-09T05:25:44.682 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.019+0000 7fb79933b700 1 Processor -- start 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.019+0000 7fb79933b700 1 -- start start 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.019+0000 7fb79933b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79410c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.019+0000 7fb79933b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb79407ce30 con 0x7fb79410a580 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.019+0000 7fb792ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79410c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.019+0000 7fb792ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79410c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34734/0 (socket says 172.21.15.115:34734) 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.019+0000 7fb792ffd700 1 -- 172.21.15.115:0/448768196 learned_addr learned my addr 172.21.15.115:0/448768196 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.019+0000 7fb792ffd700 1 -- 172.21.15.115:0/448768196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79410cee0 con 0x7fb79410a580 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.020+0000 7fb792ffd700 1 --2- 172.21.15.115:0/448768196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79410c9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7840098a0 tx=0x7fb784009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62422fac45893dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.020+0000 7fb791ffb700 1 -- 172.21.15.115:0/448768196 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb784004160 con 0x7fb79410a580 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.020+0000 7fb791ffb700 1 -- 172.21.15.115:0/448768196 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb784004300 con 0x7fb79410a580 2024-05-09T05:25:44.683 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.020+0000 7fb791ffb700 1 -- 172.21.15.115:0/448768196 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7840119e0 con 0x7fb79410a580 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.021+0000 7fb79933b700 1 -- 172.21.15.115:0/448768196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 msgr2=0x7fb79410c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.021+0000 7fb79933b700 1 --2- 172.21.15.115:0/448768196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79410c9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7840098a0 tx=0x7fb784009bb0 comp rx=0 tx=0).stop 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.021+0000 7fb79933b700 1 -- 172.21.15.115:0/448768196 shutdown_connections 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.021+0000 7fb79933b700 1 --2- 172.21.15.115:0/448768196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79410c9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.021+0000 7fb79933b700 1 -- 172.21.15.115:0/448768196 >> 172.21.15.115:0/448768196 conn(0x7fb794103fe0 msgr2=0x7fb794106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.021+0000 7fb79933b700 1 -- 172.21.15.115:0/448768196 shutdown_connections 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.021+0000 7fb79933b700 1 -- 172.21.15.115:0/448768196 wait complete. 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.022+0000 7fb79933b700 1 Processor -- start 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.022+0000 7fb79933b700 1 -- start start 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.022+0000 7fb79933b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.022+0000 7fb79933b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb79410dde0 con 0x7fb79410a580 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.022+0000 7fb792ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.022+0000 7fb792ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79419dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34736/0 (socket says 172.21.15.115:34736) 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.022+0000 7fb792ffd700 1 -- 172.21.15.115:0/3379835401 learned_addr learned my addr 172.21.15.115:0/3379835401 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:44.684 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.023+0000 7fb792ffd700 1 -- 172.21.15.115:0/3379835401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79419e290 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.023+0000 7fb792ffd700 1 --2- 172.21.15.115:0/3379835401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79419dd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb7840119b0 tx=0x7fb784011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.023+0000 7fb77bfff700 1 -- 172.21.15.115:0/3379835401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb784003af0 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.023+0000 7fb77bfff700 1 -- 172.21.15.115:0/3379835401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb78401a8a0 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.023+0000 7fb79933b700 1 -- 172.21.15.115:0/3379835401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79419e520 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.023+0000 7fb77bfff700 1 -- 172.21.15.115:0/3379835401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb784024830 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.023+0000 7fb79933b700 1 -- 172.21.15.115:0/3379835401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7941a1120 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.023+0000 7fb77bfff700 1 -- 172.21.15.115:0/3379835401 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb784022020 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.023+0000 7fb77bfff700 1 -- 172.21.15.115:0/3379835401 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb7840193c0 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.024+0000 7fb79933b700 1 -- 172.21.15.115:0/3379835401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb794064e20 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.028+0000 7fb77bfff700 1 -- 172.21.15.115:0/3379835401 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb78402ddd0 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.099+0000 7fb79933b700 1 -- 172.21.15.115:0/3379835401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb7941a12f0 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.100+0000 7fb77bfff700 1 -- 172.21.15.115:0/3379835401 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb784032030 con 0x7fb79410a580 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.101+0000 7fb79933b700 1 -- 172.21.15.115:0/3379835401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 msgr2=0x7fb79419dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.101+0000 7fb79933b700 1 --2- 172.21.15.115:0/3379835401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79419dd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb7840119b0 tx=0x7fb784011a90 comp rx=0 tx=0).stop 2024-05-09T05:25:44.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.101+0000 7fb79933b700 1 -- 172.21.15.115:0/3379835401 shutdown_connections 2024-05-09T05:25:44.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.101+0000 7fb79933b700 1 --2- 172.21.15.115:0/3379835401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79410a580 0x7fb79419dd50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:44.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.101+0000 7fb79933b700 1 -- 172.21.15.115:0/3379835401 >> 172.21.15.115:0/3379835401 conn(0x7fb794103fe0 msgr2=0x7fb7941048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:44.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.101+0000 7fb79933b700 1 -- 172.21.15.115:0/3379835401 shutdown_connections 2024-05-09T05:25:44.686 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:44.101+0000 7fb79933b700 1 -- 172.21.15.115:0/3379835401 wait complete. 2024-05-09T05:25:44.686 INFO:teuthology.orchestra.run.smithi115.stdout:mgr not available, waiting (2/15)... 2024-05-09T05:25:44.930 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:44 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:44.838+0000 7f815265d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T05:25:44.930 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:44 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:44.935+0000 7f815265d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T05:25:45.647 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:45 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:45.301+0000 7f815265d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T05:25:45.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:45 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:45.653+0000 7f815265d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T05:25:45.897 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:45 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:45.746+0000 7f815265d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T05:25:46.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:45 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:45.987+0000 7f815265d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T05:25:46.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:46 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:46.120+0000 7f815265d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T05:25:46.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:46 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:46.216+0000 7f815265d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T05:25:46.796 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:46 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:46.541+0000 7f815265d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T05:25:46.796 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:46 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:46.632+0000 7f815265d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T05:25:47.510 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:47 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:47.242+0000 7f815265d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T05:25:47.510 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:47 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:47.460+0000 7f815265d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: Activating manager daemon x 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: mgrmap e2: x(active, starting, since 0.00387186s) 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: Manager daemon x is now available 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T05:25:47.883 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' 2024-05-09T05:25:47.884 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' 2024-05-09T05:25:47.884 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:47 smithi115 ceph-mon[30349]: from='mgr.14100 172.21.15.115:0/4272494304' entity='mgr.x' 2024-05-09T05:25:49.713 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:49 smithi115 ceph-mon[30349]: mgrmap e3: x(active, since 1.00825s) 2024-05-09T05:25:50.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:50 smithi115 ceph-mon[30349]: mgrmap e4: x(active, since 2s) 2024-05-09T05:25:50.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:50 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1462139182' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T05:25:56.181 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T05:25:56.182 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T05:25:56.183 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T05:25:56.184 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T05:25:33.776061+0000", 2024-05-09T05:25:56.185 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.707+0000 7f359460c700 1 Processor -- start 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.707+0000 7f359460c700 1 -- start start 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f359460c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c10c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f359460c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f358c07ce30 con 0x7f358c10a580 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f35923a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c10c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f35923a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c10c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34802/0 (socket says 172.21.15.115:34802) 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f35923a8700 1 -- 172.21.15.115:0/1028528211 learned_addr learned my addr 172.21.15.115:0/1028528211 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f35923a8700 1 -- 172.21.15.115:0/1028528211 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c10cee0 con 0x7f358c10a580 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f35923a8700 1 --2- 172.21.15.115:0/1028528211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c10c9a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f35880098a0 tx=0x7f3588009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56988af617356c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:56.186 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f35913a6700 1 -- 172.21.15.115:0/1028528211 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3588004160 con 0x7f358c10a580 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f35913a6700 1 -- 172.21.15.115:0/1028528211 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3588004300 con 0x7f358c10a580 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.708+0000 7f35913a6700 1 -- 172.21.15.115:0/1028528211 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35880119e0 con 0x7f358c10a580 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.709+0000 7f359460c700 1 -- 172.21.15.115:0/1028528211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 msgr2=0x7f358c10c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.709+0000 7f359460c700 1 --2- 172.21.15.115:0/1028528211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c10c9a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f35880098a0 tx=0x7f3588009bb0 comp rx=0 tx=0).stop 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.709+0000 7f359460c700 1 -- 172.21.15.115:0/1028528211 shutdown_connections 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.709+0000 7f359460c700 1 --2- 172.21.15.115:0/1028528211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c10c9a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.709+0000 7f359460c700 1 -- 172.21.15.115:0/1028528211 >> 172.21.15.115:0/1028528211 conn(0x7f358c103fe0 msgr2=0x7f358c106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.709+0000 7f359460c700 1 -- 172.21.15.115:0/1028528211 shutdown_connections 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.709+0000 7f359460c700 1 -- 172.21.15.115:0/1028528211 wait complete. 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.710+0000 7f359460c700 1 Processor -- start 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.710+0000 7f359460c700 1 -- start start 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.710+0000 7f359460c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.710+0000 7f359460c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f358c10dde0 con 0x7f358c10a580 2024-05-09T05:25:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.710+0000 7f35923a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.710+0000 7f35923a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34814/0 (socket says 172.21.15.115:34814) 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.710+0000 7f35923a8700 1 -- 172.21.15.115:0/1462139182 learned_addr learned my addr 172.21.15.115:0/1462139182 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.711+0000 7f35923a8700 1 -- 172.21.15.115:0/1462139182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c19e290 con 0x7f358c10a580 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.711+0000 7f35923a8700 1 --2- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c19dd50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f35880119b0 tx=0x7f3588011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.711+0000 7f35837fe700 1 -- 172.21.15.115:0/1462139182 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3588003af0 con 0x7f358c10a580 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.711+0000 7f35837fe700 1 -- 172.21.15.115:0/1462139182 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f358801a8a0 con 0x7f358c10a580 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.711+0000 7f35837fe700 1 -- 172.21.15.115:0/1462139182 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3588024830 con 0x7f358c10a580 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.711+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c19e520 con 0x7f358c10a580 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.711+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c1a1120 con 0x7f358c10a580 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.712+0000 7f35837fe700 1 -- 172.21.15.115:0/1462139182 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f3588022020 con 0x7f358c10a580 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.712+0000 7f35837fe700 1 --2- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f357803e530 0x7f3578040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.712+0000 7f35837fe700 1 -- 172.21.15.115:0/1462139182 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3588054360 con 0x7f358c10a580 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.712+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c064e20 con 0x7f358c10a580 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.712+0000 7f3591ba7700 1 --2- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f357803e530 0x7f3578040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:56.188 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.713+0000 7f3591ba7700 1 --2- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f357803e530 0x7f3578040a40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f357c006fd0 tx=0x7f357c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:49.718+0000 7f35837fe700 1 -- 172.21.15.115:0/1462139182 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f358800b3e0 con 0x7f358c10a580 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.005+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f358c1a1780 con 0x7f358c10a580 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.006+0000 7f35837fe700 1 -- 172.21.15.115:0/1462139182 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f35880094f0 con 0x7f358c10a580 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f357803e530 msgr2=0x7f3578040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 --2- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f357803e530 0x7f3578040a40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f357c006fd0 tx=0x7f357c006e40 comp rx=0 tx=0).stop 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 msgr2=0x7f358c19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 --2- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c19dd50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f35880119b0 tx=0x7f3588011a90 comp rx=0 tx=0).stop 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 shutdown_connections 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 --2- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f357803e530 0x7f3578040a40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 --2- 172.21.15.115:0/1462139182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f358c10a580 0x7f358c19dd50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 >> 172.21.15.115:0/1462139182 conn(0x7f358c103fe0 msgr2=0x7f358c1048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 shutdown_connections 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:50.008+0000 7f359460c700 1 -- 172.21.15.115:0/1462139182 wait complete. 2024-05-09T05:25:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:mgr is available 2024-05-09T05:25:58.359 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:57 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4084462539' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T05:25:58.515 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:58.515 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T05:25:58.515 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout fsid = 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:25:58.515 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T05:25:58.515 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T05:25:58.515 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T05:25:58.515 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T05:25:58.516 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T05:25:58.516 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:58.516 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T05:25:58.516 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T05:25:58.516 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-05-09T05:25:58.516 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T05:25:58.516 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-09T05:25:58.516 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.708+0000 7f318eb8c700 1 Processor -- start 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.708+0000 7f318eb8c700 1 -- start start 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.708+0000 7f318eb8c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f3188108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.708+0000 7f318eb8c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f31881093d0 con 0x7f31881089d0 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.709+0000 7f318c928700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f3188108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.709+0000 7f318c928700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f3188108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44296/0 (socket says 172.21.15.115:44296) 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.709+0000 7f318c928700 1 -- 172.21.15.115:0/3243590343 learned_addr learned my addr 172.21.15.115:0/3243590343 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:58.517 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.709+0000 7f318c928700 1 -- 172.21.15.115:0/3243590343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3188109ba0 con 0x7f31881089d0 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.709+0000 7f318c928700 1 --2- 172.21.15.115:0/3243590343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f3188108de0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f317c0098a0 tx=0x7f317c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfc5f2fa5a983ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.709+0000 7f31877fe700 1 -- 172.21.15.115:0/3243590343 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f317c004160 con 0x7f31881089d0 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.709+0000 7f31877fe700 1 -- 172.21.15.115:0/3243590343 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f317c004300 con 0x7f31881089d0 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.709+0000 7f31877fe700 1 -- 172.21.15.115:0/3243590343 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f317c0119e0 con 0x7f31881089d0 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.710+0000 7f318eb8c700 1 -- 172.21.15.115:0/3243590343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 msgr2=0x7f3188108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.710+0000 7f318eb8c700 1 --2- 172.21.15.115:0/3243590343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f3188108de0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f317c0098a0 tx=0x7f317c009bb0 comp rx=0 tx=0).stop 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.710+0000 7f318eb8c700 1 -- 172.21.15.115:0/3243590343 shutdown_connections 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.710+0000 7f318eb8c700 1 --2- 172.21.15.115:0/3243590343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f3188108de0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.710+0000 7f318eb8c700 1 -- 172.21.15.115:0/3243590343 >> 172.21.15.115:0/3243590343 conn(0x7f31881040b0 msgr2=0x7f3188106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.710+0000 7f318eb8c700 1 -- 172.21.15.115:0/3243590343 shutdown_connections 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.710+0000 7f318eb8c700 1 -- 172.21.15.115:0/3243590343 wait complete. 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.711+0000 7f318eb8c700 1 Processor -- start 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.711+0000 7f318eb8c700 1 -- start start 2024-05-09T05:25:58.518 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.711+0000 7f318eb8c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f31881073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.711+0000 7f318eb8c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f318810cf50 con 0x7f31881089d0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.711+0000 7f318c928700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f31881073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.711+0000 7f318c928700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f31881073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44302/0 (socket says 172.21.15.115:44302) 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.711+0000 7f318c928700 1 -- 172.21.15.115:0/4084462539 learned_addr learned my addr 172.21.15.115:0/4084462539 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.711+0000 7f318c928700 1 -- 172.21.15.115:0/4084462539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31881078e0 con 0x7f31881089d0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.711+0000 7f318c928700 1 --2- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f31881073a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f317c009fb0 tx=0x7f317c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.712+0000 7f3185ffb700 1 -- 172.21.15.115:0/4084462539 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f317c0038b0 con 0x7f31881089d0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.712+0000 7f3185ffb700 1 -- 172.21.15.115:0/4084462539 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f317c003a50 con 0x7f31881089d0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.712+0000 7f3185ffb700 1 -- 172.21.15.115:0/4084462539 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f317c019550 con 0x7f31881089d0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.712+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3188105b30 con 0x7f31881089d0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.712+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3188106010 con 0x7f31881089d0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.713+0000 7f3185ffb700 1 -- 172.21.15.115:0/4084462539 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f317c019850 con 0x7f31881089d0 2024-05-09T05:25:58.519 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.713+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3188064e20 con 0x7f31881089d0 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.713+0000 7f3185ffb700 1 --2- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f317803e490 0x7f31780409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.713+0000 7f3185ffb700 1 -- 172.21.15.115:0/4084462539 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f317c053c60 con 0x7f31881089d0 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.713+0000 7f3187fff700 1 --2- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f317803e490 0x7f31780409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.713+0000 7f3187fff700 1 --2- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f317803e490 0x7f31780409a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3174006fd0 tx=0x7f3174006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.718+0000 7f3185ffb700 1 -- 172.21.15.115:0/4084462539 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f317c018dc0 con 0x7f31881089d0 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.918+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3188197020 con 0x7f31881089d0 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.920+0000 7f3185ffb700 1 -- 172.21.15.115:0/4084462539 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+434 (secure 0 0 0) 0x7f317c053680 con 0x7f31881089d0 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f317803e490 msgr2=0x7f31780409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 --2- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f317803e490 0x7f31780409a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3174006fd0 tx=0x7f3174006e40 comp rx=0 tx=0).stop 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 msgr2=0x7f31881073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 --2- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f31881073a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f317c009fb0 tx=0x7f317c011a60 comp rx=0 tx=0).stop 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 shutdown_connections 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 --2- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f317803e490 0x7f31780409a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:58.520 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 --2- 172.21.15.115:0/4084462539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31881089d0 0x7f31881073a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:58.521 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 >> 172.21.15.115:0/4084462539 conn(0x7f31881040b0 msgr2=0x7f31881943c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:58.521 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 shutdown_connections 2024-05-09T05:25:58.521 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:57.922+0000 7f318eb8c700 1 -- 172.21.15.115:0/4084462539 wait complete. 2024-05-09T05:25:58.521 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling cephadm module... 2024-05-09T05:25:59.553 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:25:59 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3338246635' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T05:25:59.554 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:59 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: ignoring --setuser ceph since I am not root 2024-05-09T05:25:59.554 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:59 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: ignoring --setgroup ceph since I am not root 2024-05-09T05:25:59.554 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:59 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:59.446+0000 7f7112b59200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T05:25:59.837 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:59 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:59.637+0000 7f7112b59200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:25:59.837 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:25:59 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:25:59.742+0000 7f7112b59200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.929+0000 7f85140d4700 1 Processor -- start 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.929+0000 7f85140d4700 1 -- start start 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.929+0000 7f85140d4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c10ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.929+0000 7f85140d4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f850c07d6a0 con 0x7f850c10a610 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.929+0000 7f8511e70700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c10ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.929+0000 7f8511e70700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c10ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44308/0 (socket says 172.21.15.115:44308) 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.929+0000 7f8511e70700 1 -- 172.21.15.115:0/202109940 learned_addr learned my addr 172.21.15.115:0/202109940 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.930+0000 7f8511e70700 1 -- 172.21.15.115:0/202109940 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c10cf70 con 0x7f850c10a610 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.930+0000 7f8511e70700 1 --2- 172.21.15.115:0/202109940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c10ca30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f85080098a0 tx=0x7f8508009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403a0fc6c6dc6b5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.930+0000 7f8510e6e700 1 -- 172.21.15.115:0/202109940 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8508004160 con 0x7f850c10a610 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.930+0000 7f8510e6e700 1 -- 172.21.15.115:0/202109940 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8508004300 con 0x7f850c10a610 2024-05-09T05:25:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.930+0000 7f8510e6e700 1 -- 172.21.15.115:0/202109940 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85080119e0 con 0x7f850c10a610 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.931+0000 7f85140d4700 1 -- 172.21.15.115:0/202109940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 msgr2=0x7f850c10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.931+0000 7f85140d4700 1 --2- 172.21.15.115:0/202109940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c10ca30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f85080098a0 tx=0x7f8508009bb0 comp rx=0 tx=0).stop 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.931+0000 7f85140d4700 1 -- 172.21.15.115:0/202109940 shutdown_connections 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.931+0000 7f85140d4700 1 --2- 172.21.15.115:0/202109940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c10ca30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.931+0000 7f85140d4700 1 -- 172.21.15.115:0/202109940 >> 172.21.15.115:0/202109940 conn(0x7f850c1040b0 msgr2=0x7f850c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.931+0000 7f85140d4700 1 -- 172.21.15.115:0/202109940 shutdown_connections 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.931+0000 7f85140d4700 1 -- 172.21.15.115:0/202109940 wait complete. 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.932+0000 7f85140d4700 1 Processor -- start 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.932+0000 7f85140d4700 1 -- start start 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.932+0000 7f85140d4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.932+0000 7f85140d4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f850c10de70 con 0x7f850c10a610 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.932+0000 7f8511e70700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.932+0000 7f8511e70700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c19ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44310/0 (socket says 172.21.15.115:44310) 2024-05-09T05:25:59.849 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.932+0000 7f8511e70700 1 -- 172.21.15.115:0/3338246635 learned_addr learned my addr 172.21.15.115:0/3338246635 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.932+0000 7f8511e70700 1 -- 172.21.15.115:0/3338246635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c19e310 con 0x7f850c10a610 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.932+0000 7f8511e70700 1 --2- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c19ddd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f85080119b0 tx=0x7f8508011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.933+0000 7f8502ffd700 1 -- 172.21.15.115:0/3338246635 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8508003af0 con 0x7f850c10a610 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.933+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c19e5a0 con 0x7f850c10a610 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.933+0000 7f8502ffd700 1 -- 172.21.15.115:0/3338246635 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f850801a8a0 con 0x7f850c10a610 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.933+0000 7f8502ffd700 1 -- 172.21.15.115:0/3338246635 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8508024830 con 0x7f850c10a610 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.933+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c1a1170 con 0x7f850c10a610 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.934+0000 7f8502ffd700 1 -- 172.21.15.115:0/3338246635 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f8508022020 con 0x7f850c10a610 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.934+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c064e20 con 0x7f850c10a610 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.934+0000 7f8502ffd700 1 --2- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f84f803e530 0x7f84f8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.934+0000 7f8502ffd700 1 -- 172.21.15.115:0/3338246635 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8508053520 con 0x7f850c10a610 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.934+0000 7f851166f700 1 --2- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f84f803e530 0x7f84f8040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.934+0000 7f851166f700 1 --2- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f84f803e530 0x7f84f8040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84fc006fd0 tx=0x7f84fc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:25:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:58.939+0000 7f8502ffd700 1 -- 172.21.15.115:0/3338246635 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f850800e260 con 0x7f850c10a610 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.198+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f850c1a1890 con 0x7f850c10a610 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.249+0000 7f8502ffd700 1 -- 172.21.15.115:0/3338246635 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f8508053280 con 0x7f850c10a610 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.251+0000 7f8502ffd700 1 -- 172.21.15.115:0/3338246635 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f850802dc90 con 0x7f850c10a610 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f84f803e530 msgr2=0x7f84f8040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 --2- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f84f803e530 0x7f84f8040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84fc006fd0 tx=0x7f84fc006e40 comp rx=0 tx=0).stop 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 msgr2=0x7f850c19ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 --2- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c19ddd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f85080119b0 tx=0x7f8508011a90 comp rx=0 tx=0).stop 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 shutdown_connections 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 --2- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f84f803e530 0x7f84f8040a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 --2- 172.21.15.115:0/3338246635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f850c10a610 0x7f850c19ddd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 >> 172.21.15.115:0/3338246635 conn(0x7f850c1040b0 msgr2=0x7f850c105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 shutdown_connections 2024-05-09T05:25:59.851 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:25:59.252+0000 7f85140d4700 1 -- 172.21.15.115:0/3338246635 wait complete. 2024-05-09T05:26:00.123 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:00.070+0000 7f7112b59200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T05:26:00.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:00 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3338246635' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T05:26:00.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:00 smithi115 ceph-mon[30349]: mgrmap e5: x(active, since 11s) 2024-05-09T05:26:00.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:00.158+0000 7f7112b59200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.240+0000 7f57becf6700 1 Processor -- start 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.240+0000 7f57becf6700 1 -- start start 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.240+0000 7f57becf6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b810c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.240+0000 7f57becf6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57b807ce70 con 0x7f57b810a540 2024-05-09T05:26:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.240+0000 7f57bca92700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b810c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.240+0000 7f57bca92700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b810c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44332/0 (socket says 172.21.15.115:44332) 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.240+0000 7f57bca92700 1 -- 172.21.15.115:0/2398251760 learned_addr learned my addr 172.21.15.115:0/2398251760 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.240+0000 7f57bca92700 1 -- 172.21.15.115:0/2398251760 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b810cea0 con 0x7f57b810a540 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.241+0000 7f57bca92700 1 --2- 172.21.15.115:0/2398251760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b810c960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0098a0 tx=0x7f57ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afb14dba842b40d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.241+0000 7f57b77fe700 1 -- 172.21.15.115:0/2398251760 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ac004160 con 0x7f57b810a540 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.241+0000 7f57b77fe700 1 -- 172.21.15.115:0/2398251760 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f57ac004300 con 0x7f57b810a540 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.241+0000 7f57b77fe700 1 -- 172.21.15.115:0/2398251760 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ac0119e0 con 0x7f57b810a540 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.242+0000 7f57becf6700 1 -- 172.21.15.115:0/2398251760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 msgr2=0x7f57b810c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.242+0000 7f57becf6700 1 --2- 172.21.15.115:0/2398251760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b810c960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0098a0 tx=0x7f57ac009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.242+0000 7f57becf6700 1 -- 172.21.15.115:0/2398251760 shutdown_connections 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.242+0000 7f57becf6700 1 --2- 172.21.15.115:0/2398251760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b810c960 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.242+0000 7f57becf6700 1 -- 172.21.15.115:0/2398251760 >> 172.21.15.115:0/2398251760 conn(0x7f57b8103fe0 msgr2=0x7f57b8106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.242+0000 7f57becf6700 1 -- 172.21.15.115:0/2398251760 shutdown_connections 2024-05-09T05:26:01.120 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.242+0000 7f57becf6700 1 -- 172.21.15.115:0/2398251760 wait complete. 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.242+0000 7f57becf6700 1 Processor -- start 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.242+0000 7f57becf6700 1 -- start start 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.243+0000 7f57becf6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.243+0000 7f57becf6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f57b810de00 con 0x7f57b810a540 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.243+0000 7f57bca92700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.243+0000 7f57bca92700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44344/0 (socket says 172.21.15.115:44344) 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.243+0000 7f57bca92700 1 -- 172.21.15.115:0/138946524 learned_addr learned my addr 172.21.15.115:0/138946524 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.243+0000 7f57bca92700 1 -- 172.21.15.115:0/138946524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b819e2e0 con 0x7f57b810a540 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.243+0000 7f57bca92700 1 --2- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b819dda0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0119b0 tx=0x7f57ac011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.244+0000 7f57b5ffb700 1 -- 172.21.15.115:0/138946524 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ac003af0 con 0x7f57b810a540 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.244+0000 7f57b5ffb700 1 -- 172.21.15.115:0/138946524 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f57ac01a8a0 con 0x7f57b810a540 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.244+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b819e570 con 0x7f57b810a540 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.244+0000 7f57b5ffb700 1 -- 172.21.15.115:0/138946524 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57ac024830 con 0x7f57b810a540 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.244+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81a1180 con 0x7f57b810a540 2024-05-09T05:26:01.121 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.245+0000 7f57b5ffb700 1 -- 172.21.15.115:0/138946524 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f57ac022020 con 0x7f57b810a540 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.245+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b8064e20 con 0x7f57b810a540 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.245+0000 7f57b5ffb700 1 --2- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f57a803e580 0x7f57a8040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.245+0000 7f57b5ffb700 1 -- 172.21.15.115:0/138946524 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f57ac0535c0 con 0x7f57b810a540 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.245+0000 7f57b7fff700 1 -- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f57a803e580 msgr2=0x7f57a8040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/1212404088 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.245+0000 7f57b7fff700 1 --2- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f57a803e580 0x7f57a8040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.250+0000 7f57b5ffb700 1 -- 172.21.15.115:0/138946524 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57ac00e260 con 0x7f57b810a540 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.446+0000 7f57b7fff700 1 -- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f57a803e580 msgr2=0x7f57a8040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/1212404088 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.446+0000 7f57b7fff700 1 --2- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f57a803e580 0x7f57a8040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.517+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f57b81a13a0 con 0x7f57b810a540 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.517+0000 7f57b5ffb700 1 -- 172.21.15.115:0/138946524 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f57b81a13a0 con 0x7f57b810a540 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f57a803e580 msgr2=0x7f57a8040a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 --2- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f57a803e580 0x7f57a8040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:01.122 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 msgr2=0x7f57b819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:01.123 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 --2- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b819dda0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0119b0 tx=0x7f57ac011a90 comp rx=0 tx=0).stop 2024-05-09T05:26:01.123 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 shutdown_connections 2024-05-09T05:26:01.123 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 --2- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7f57a803e580 0x7f57a8040a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:01.123 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 --2- 172.21.15.115:0/138946524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b810a540 0x7f57b819dda0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:01.123 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 >> 172.21.15.115:0/138946524 conn(0x7f57b8103fe0 msgr2=0x7f57b8104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:01.123 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 shutdown_connections 2024-05-09T05:26:01.123 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:00.520+0000 7f57becf6700 1 -- 172.21.15.115:0/138946524 wait complete. 2024-05-09T05:26:01.123 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for the mgr to restart... 2024-05-09T05:26:01.123 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr epoch 5... 2024-05-09T05:26:01.364 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:01 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/138946524' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T05:26:01.647 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:01 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:01.370+0000 7f7112b59200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T05:26:01.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:01 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:01.551+0000 7f7112b59200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T05:26:02.017 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:01 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:01.728+0000 7f7112b59200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T05:26:02.017 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:02 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:02.022+0000 7f7112b59200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T05:26:02.316 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:02 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:02.117+0000 7f7112b59200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T05:26:02.316 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:02 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:02.321+0000 7f7112b59200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T05:26:02.647 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:02 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:02.517+0000 7f7112b59200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:26:03.330 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:03 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:03.114+0000 7f7112b59200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T05:26:03.330 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:03 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:03.336+0000 7f7112b59200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T05:26:04.147 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:03 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:03.761+0000 7f7112b59200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T05:26:04.148 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:03 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:03.857+0000 7f7112b59200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T05:26:04.584 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:04.227+0000 7f7112b59200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T05:26:04.584 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:04.589+0000 7f7112b59200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T05:26:04.897 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:04.685+0000 7f7112b59200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T05:26:05.397 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:04.933+0000 7f7112b59200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T05:26:05.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:05 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:05.071+0000 7f7112b59200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T05:26:05.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:05 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:05.169+0000 7f7112b59200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T05:26:05.897 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:05 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:05.497+0000 7f7112b59200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T05:26:05.898 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:05 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:05.590+0000 7f7112b59200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T05:26:06.461 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:06 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:06.193+0000 7f7112b59200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T05:26:06.461 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:06 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:06.413+0000 7f7112b59200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: Active manager daemon x restarted 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: Activating manager daemon x 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: mgrmap e6: x(active, starting, since 0.00369179s) 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T05:26:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: Manager daemon x is now available 2024-05-09T05:26:06.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:06.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:06.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:06.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:06 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:07.691 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:07 smithi115 ceph-mon[30349]: Found migration_current of "None". Setting to last migration. 2024-05-09T05:26:07.691 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:07 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T05:26:07.691 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:07 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T05:26:07.691 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:07 smithi115 ceph-mon[30349]: mgrmap e7: x(active, since 1.00626s) 2024-05-09T05:26:07.918 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.600+0000 7ff4de781700 1 Processor -- start 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.600+0000 7ff4de781700 1 -- start start 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.600+0000 7ff4de781700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d810c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.600+0000 7ff4de781700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d807d860 con 0x7ff4d810a5c0 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.600+0000 7ff4d7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d810c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.600+0000 7ff4d7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d810c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44354/0 (socket says 172.21.15.115:44354) 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.600+0000 7ff4d7fff700 1 -- 172.21.15.115:0/93073245 learned_addr learned my addr 172.21.15.115:0/93073245 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.601+0000 7ff4d7fff700 1 -- 172.21.15.115:0/93073245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d810cf20 con 0x7ff4d810a5c0 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.601+0000 7ff4d7fff700 1 --2- 172.21.15.115:0/93073245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d810c9e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80098a0 tx=0x7ff4c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23620e6367eff11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.601+0000 7ff4d6ffd700 1 -- 172.21.15.115:0/93073245 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c8004160 con 0x7ff4d810a5c0 2024-05-09T05:26:07.919 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.601+0000 7ff4d6ffd700 1 -- 172.21.15.115:0/93073245 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff4c8004300 con 0x7ff4d810a5c0 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.601+0000 7ff4d6ffd700 1 -- 172.21.15.115:0/93073245 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c80119e0 con 0x7ff4d810a5c0 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.602+0000 7ff4de781700 1 -- 172.21.15.115:0/93073245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 msgr2=0x7ff4d810c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.602+0000 7ff4de781700 1 --2- 172.21.15.115:0/93073245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d810c9e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80098a0 tx=0x7ff4c8009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.602+0000 7ff4de781700 1 -- 172.21.15.115:0/93073245 shutdown_connections 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.602+0000 7ff4de781700 1 --2- 172.21.15.115:0/93073245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d810c9e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.602+0000 7ff4de781700 1 -- 172.21.15.115:0/93073245 >> 172.21.15.115:0/93073245 conn(0x7ff4d8103fc0 msgr2=0x7ff4d8106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.602+0000 7ff4de781700 1 -- 172.21.15.115:0/93073245 shutdown_connections 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.602+0000 7ff4de781700 1 -- 172.21.15.115:0/93073245 wait complete. 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.603+0000 7ff4de781700 1 Processor -- start 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.603+0000 7ff4de781700 1 -- start start 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.603+0000 7ff4de781700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d8109d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.603+0000 7ff4de781700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d810de20 con 0x7ff4d810a5c0 2024-05-09T05:26:07.920 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.603+0000 7ff4d7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d8109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.603+0000 7ff4d7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d8109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44362/0 (socket says 172.21.15.115:44362) 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.603+0000 7ff4d7fff700 1 -- 172.21.15.115:0/3264941677 learned_addr learned my addr 172.21.15.115:0/3264941677 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.604+0000 7ff4d7fff700 1 -- 172.21.15.115:0/3264941677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d810a280 con 0x7ff4d810a5c0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.604+0000 7ff4d7fff700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d8109d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80116a0 tx=0x7ff4c8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.604+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c80039f0 con 0x7ff4d810a5c0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.604+0000 7ff4de781700 1 -- 172.21.15.115:0/3264941677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d81084d0 con 0x7ff4d810a5c0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.604+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff4c8003b90 con 0x7ff4d810a5c0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.604+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c801a600 con 0x7ff4d810a5c0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.604+0000 7ff4de781700 1 -- 172.21.15.115:0/3264941677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d81089b0 con 0x7ff4d810a5c0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.605+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7ff4c801a900 con 0x7ff4d810a5c0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.605+0000 7ff4d57fa700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 0x7ff4c40409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.605+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 --> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4c4041100 con 0x7ff4c403e4e0 2024-05-09T05:26:07.921 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.605+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4c8053c40 con 0x7ff4d810a5c0 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.605+0000 7ff4d77fe700 1 -- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 msgr2=0x7ff4c40409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/1212404088 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.605+0000 7ff4d77fe700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 0x7ff4c40409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.806+0000 7ff4d77fe700 1 -- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 msgr2=0x7ff4c40409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/1212404088 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:01.806+0000 7ff4d77fe700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 0x7ff4c40409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:02.207+0000 7ff4d77fe700 1 -- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 msgr2=0x7ff4c40409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/1212404088 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:02.207+0000 7ff4d77fe700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 0x7ff4c40409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:03.008+0000 7ff4d77fe700 1 -- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 msgr2=0x7ff4c40409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/1212404088 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:03.008+0000 7ff4d77fe700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 0x7ff4c40409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:04.609+0000 7ff4d77fe700 1 -- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 msgr2=0x7ff4c40409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/1212404088 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:04.609+0000 7ff4d77fe700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 0x7ff4c40409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:06.416+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7ff4c801abf0 con 0x7ff4d810a5c0 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:06.417+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 msgr2=0x7ff4c40409f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:06.417+0000 7ff4d57fa700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 0x7ff4c40409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:07.922 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.418+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7ff4c8055000 con 0x7ff4d810a5c0 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.418+0000 7ff4d57fa700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff4c4041e20 0x7ff4c4044240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.418+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4c8025e70 con 0x7ff4c4041e20 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.418+0000 7ff4d77fe700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff4c4041e20 0x7ff4c4044240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.419+0000 7ff4d77fe700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff4c4041e20 0x7ff4c4044240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0003a10 tx=0x7ff4c0009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.420+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 <== mgr.14118 v2:172.21.15.115:6800/792641138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff4c8025e70 con 0x7ff4c4041e20 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 -- 172.21.15.115:0/3264941677 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff4d8064e20 con 0x7ff4c4041e20 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4d57fa700 1 -- 172.21.15.115:0/3264941677 <== mgr.14118 v2:172.21.15.115:6800/792641138 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff4d8064e20 con 0x7ff4c4041e20 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 -- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff4c4041e20 msgr2=0x7ff4c4044240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff4c4041e20 0x7ff4c4044240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0003a10 tx=0x7ff4c0009290 comp rx=0 tx=0).stop 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 -- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 msgr2=0x7ff4d8109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d8109d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80116a0 tx=0x7ff4c8011a60 comp rx=0 tx=0).stop 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 -- 172.21.15.115:0/3264941677 shutdown_connections 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4d810a5c0 0x7ff4d8109d40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:07.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff4c4041e20 0x7ff4c4044240 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:07.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 --2- 172.21.15.115:0/3264941677 >> [v2:172.21.15.115:6800/1212404088,v1:172.21.15.115:6801/1212404088] conn(0x7ff4c403e4e0 0x7ff4c40409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:07.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 -- 172.21.15.115:0/3264941677 >> 172.21.15.115:0/3264941677 conn(0x7ff4d8103fc0 msgr2=0x7ff4d8104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:07.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 -- 172.21.15.115:0/3264941677 shutdown_connections 2024-05-09T05:26:07.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:07.428+0000 7ff4de781700 1 -- 172.21.15.115:0/3264941677 wait complete. 2024-05-09T05:26:07.924 INFO:teuthology.orchestra.run.smithi115.stdout:mgr epoch 5 is available 2024-05-09T05:26:07.924 INFO:teuthology.orchestra.run.smithi115.stdout:Setting orchestrator backend to cephadm... 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: [09/May/2024:05:26:07] ENGINE Bus STARTING 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: [09/May/2024:05:26:08] ENGINE Serving on http://172.21.15.115:8765 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: [09/May/2024:05:26:08] ENGINE Serving on https://172.21.15.115:7150 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: [09/May/2024:05:26:08] ENGINE Bus STARTED 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:08.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:08.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:08 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:09.154 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.402+0000 7f660384f700 1 Processor -- start 2024-05-09T05:26:09.154 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.402+0000 7f660384f700 1 -- start start 2024-05-09T05:26:09.154 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f660384f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:09.154 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f660384f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc10a4d0 con 0x7f65fc109ad0 2024-05-09T05:26:09.154 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f66015eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:09.154 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f66015eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc109ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36192/0 (socket says 172.21.15.115:36192) 2024-05-09T05:26:09.154 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f66015eb700 1 -- 172.21.15.115:0/2727533305 learned_addr learned my addr 172.21.15.115:0/2727533305 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:09.155 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f66015eb700 1 -- 172.21.15.115:0/2727533305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc10aca0 con 0x7f65fc109ad0 2024-05-09T05:26:09.155 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f66015eb700 1 --2- 172.21.15.115:0/2727533305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc109ee0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009e80 tx=0x7f65f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce6114db55c5a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:09.155 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f65f3fff700 1 -- 172.21.15.115:0/2727533305 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65f8017070 con 0x7f65fc109ad0 2024-05-09T05:26:09.159 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f65f3fff700 1 -- 172.21.15.115:0/2727533305 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f65f800b8c0 con 0x7f65fc109ad0 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.403+0000 7f65f3fff700 1 -- 172.21.15.115:0/2727533305 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65f8004420 con 0x7f65fc109ad0 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.404+0000 7f660384f700 1 -- 172.21.15.115:0/2727533305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 msgr2=0x7f65fc109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.404+0000 7f660384f700 1 --2- 172.21.15.115:0/2727533305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc109ee0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009e80 tx=0x7f65f800b330 comp rx=0 tx=0).stop 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.404+0000 7f660384f700 1 -- 172.21.15.115:0/2727533305 shutdown_connections 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.404+0000 7f660384f700 1 --2- 172.21.15.115:0/2727533305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc109ee0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.404+0000 7f660384f700 1 -- 172.21.15.115:0/2727533305 >> 172.21.15.115:0/2727533305 conn(0x7f65fc105190 msgr2=0x7f65fc1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.404+0000 7f660384f700 1 -- 172.21.15.115:0/2727533305 shutdown_connections 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.404+0000 7f660384f700 1 -- 172.21.15.115:0/2727533305 wait complete. 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.405+0000 7f660384f700 1 Processor -- start 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.405+0000 7f660384f700 1 -- start start 2024-05-09T05:26:09.160 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.405+0000 7f660384f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.405+0000 7f660384f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc10e050 con 0x7f65fc109ad0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.405+0000 7f66015eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.405+0000 7f66015eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36200/0 (socket says 172.21.15.115:36200) 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.405+0000 7f66015eb700 1 -- 172.21.15.115:0/404965813 learned_addr learned my addr 172.21.15.115:0/404965813 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.406+0000 7f66015eb700 1 -- 172.21.15.115:0/404965813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc19e3b0 con 0x7f65fc109ad0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.406+0000 7f66015eb700 1 --2- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc19de70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f65f800bca0 tx=0x7f65f800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.406+0000 7f65f27fc700 1 -- 172.21.15.115:0/404965813 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65f8020020 con 0x7f65fc109ad0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.406+0000 7f65f27fc700 1 -- 172.21.15.115:0/404965813 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f65f8012430 con 0x7f65fc109ad0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.406+0000 7f65f27fc700 1 -- 172.21.15.115:0/404965813 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65f8017040 con 0x7f65fc109ad0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.406+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65fc19e640 con 0x7f65fc109ad0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.406+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65fc1a1250 con 0x7f65fc109ad0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.407+0000 7f65f27fc700 1 -- 172.21.15.115:0/404965813 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f65f8023020 con 0x7f65fc109ad0 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.407+0000 7f65f27fc700 1 --2- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f65e803e420 0x7f65e8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:09.161 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.407+0000 7f65f27fc700 1 -- 172.21.15.115:0/404965813 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f65f800f050 con 0x7f65fc109ad0 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.407+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65fc064e20 con 0x7f65fc109ad0 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.407+0000 7f6600dea700 1 --2- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f65e803e420 0x7f65e8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.408+0000 7f6600dea700 1 --2- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f65e803e420 0x7f65e8040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65ec006fd0 tx=0x7f65ec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.413+0000 7f65f27fc700 1 -- 172.21.15.115:0/404965813 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65f802a020 con 0x7f65fc109ad0 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.636+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f65fc107200 con 0x7f65e803e420 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.641+0000 7f65f27fc700 1 -- 172.21.15.115:0/404965813 <== mgr.14118 v2:172.21.15.115:6800/792641138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f65fc107200 con 0x7f65e803e420 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f65e803e420 msgr2=0x7f65e8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 --2- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f65e803e420 0x7f65e8040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65ec006fd0 tx=0x7f65ec006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 msgr2=0x7f65fc19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 --2- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc19de70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f65f800bca0 tx=0x7f65f800bd80 comp rx=0 tx=0).stop 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 shutdown_connections 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 --2- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f65e803e420 0x7f65e8040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 --2- 172.21.15.115:0/404965813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65fc109ad0 0x7f65fc19de70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 >> 172.21.15.115:0/404965813 conn(0x7f65fc105190 msgr2=0x7f65fc106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:09.163 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 shutdown_connections 2024-05-09T05:26:09.163 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:08.644+0000 7f660384f700 1 -- 172.21.15.115:0/404965813 wait complete. 2024-05-09T05:26:09.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:09 smithi115 ceph-mon[30349]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:09.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:09 smithi115 ceph-mon[30349]: mgrmap e8: x(active, since 2s) 2024-05-09T05:26:10.396 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T05:26:10.396 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.576+0000 7fa8404d7700 1 Processor -- start 2024-05-09T05:26:10.396 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.576+0000 7fa8404d7700 1 -- start start 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.576+0000 7fa8404d7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa838108560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.576+0000 7fa8404d7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa838108b50 con 0x7fa838108150 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.576+0000 7fa83e273700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa838108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.576+0000 7fa83e273700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa838108560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36210/0 (socket says 172.21.15.115:36210) 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.576+0000 7fa83e273700 1 -- 172.21.15.115:0/4232068857 learned_addr learned my addr 172.21.15.115:0/4232068857 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.577+0000 7fa83e273700 1 -- 172.21.15.115:0/4232068857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa838109320 con 0x7fa838108150 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.577+0000 7fa83e273700 1 --2- 172.21.15.115:0/4232068857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa838108560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa8340098a0 tx=0x7fa834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb273a8d0c8b0d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.577+0000 7fa83d271700 1 -- 172.21.15.115:0/4232068857 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa834004160 con 0x7fa838108150 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.577+0000 7fa83d271700 1 -- 172.21.15.115:0/4232068857 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa834004300 con 0x7fa838108150 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.577+0000 7fa83d271700 1 -- 172.21.15.115:0/4232068857 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8340119e0 con 0x7fa838108150 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.578+0000 7fa8404d7700 1 -- 172.21.15.115:0/4232068857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 msgr2=0x7fa838108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.578+0000 7fa8404d7700 1 --2- 172.21.15.115:0/4232068857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa838108560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa8340098a0 tx=0x7fa834009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:10.397 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.578+0000 7fa8404d7700 1 -- 172.21.15.115:0/4232068857 shutdown_connections 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.578+0000 7fa8404d7700 1 --2- 172.21.15.115:0/4232068857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa838108560 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.578+0000 7fa8404d7700 1 -- 172.21.15.115:0/4232068857 >> 172.21.15.115:0/4232068857 conn(0x7fa838103810 msgr2=0x7fa838105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.578+0000 7fa8404d7700 1 -- 172.21.15.115:0/4232068857 shutdown_connections 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.578+0000 7fa8404d7700 1 -- 172.21.15.115:0/4232068857 wait complete. 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.578+0000 7fa8404d7700 1 Processor -- start 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.578+0000 7fa8404d7700 1 -- start start 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa8404d7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa83819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa8404d7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810c6d0 con 0x7fa838108150 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa83e273700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa83819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa83e273700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa83819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36214/0 (socket says 172.21.15.115:36214) 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa83e273700 1 -- 172.21.15.115:0/2577427976 learned_addr learned my addr 172.21.15.115:0/2577427976 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa83e273700 1 -- 172.21.15.115:0/2577427976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83819d280 con 0x7fa838108150 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa83e273700 1 --2- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa83819cd40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa8340119b0 tx=0x7fa834011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:10.398 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa82f7fe700 1 -- 172.21.15.115:0/2577427976 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa834003970 con 0x7fa838108150 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa82f7fe700 1 -- 172.21.15.115:0/2577427976 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa834003b10 con 0x7fa838108150 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83819d510 con 0x7fa838108150 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa82f7fe700 1 -- 172.21.15.115:0/2577427976 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa834024610 con 0x7fa838108150 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.579+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8381a0110 con 0x7fa838108150 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.580+0000 7fa82f7fe700 1 -- 172.21.15.115:0/2577427976 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa83402d3f0 con 0x7fa838108150 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.581+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa838064e20 con 0x7fa838108150 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.581+0000 7fa82f7fe700 1 --2- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fa82403e530 0x7fa824040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.581+0000 7fa82f7fe700 1 -- 172.21.15.115:0/2577427976 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa834054810 con 0x7fa838108150 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.581+0000 7fa83da72700 1 --2- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fa82403e530 0x7fa824040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.581+0000 7fa83da72700 1 --2- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fa82403e530 0x7fa824040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa828006fd0 tx=0x7fa828006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.586+0000 7fa82f7fe700 1 -- 172.21.15.115:0/2577427976 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8340094f0 con 0x7fa838108150 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.791+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa83802e960 con 0x7fa82403e530 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.792+0000 7fa82f7fe700 1 -- 172.21.15.115:0/2577427976 <== mgr.14118 v2:172.21.15.115:6800/792641138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa83802e960 con 0x7fa82403e530 2024-05-09T05:26:10.399 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fa82403e530 msgr2=0x7fa824040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 --2- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fa82403e530 0x7fa824040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa828006fd0 tx=0x7fa828006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 msgr2=0x7fa83819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 --2- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa83819cd40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa8340119b0 tx=0x7fa834011a90 comp rx=0 tx=0).stop 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 shutdown_connections 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 --2- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fa82403e530 0x7fa824040a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 --2- 172.21.15.115:0/2577427976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa838108150 0x7fa83819cd40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 >> 172.21.15.115:0/2577427976 conn(0x7fa838103810 msgr2=0x7fa8381067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 shutdown_connections 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:09.794+0000 7fa8404d7700 1 -- 172.21.15.115:0/2577427976 wait complete. 2024-05-09T05:26:10.400 INFO:teuthology.orchestra.run.smithi115.stdout:Generating ssh key... 2024-05-09T05:26:11.048 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:10 smithi115 ceph-mon[30349]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:11.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: Generating public/private rsa key pair. 2024-05-09T05:26:11.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: Your identification has been saved in /tmp/tmpngd1ittr/key. 2024-05-09T05:26:11.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: Your public key has been saved in /tmp/tmpngd1ittr/key.pub. 2024-05-09T05:26:11.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: The key fingerprint is: 2024-05-09T05:26:11.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: SHA256:fECeWdKeaMt47eJkDQky8k/Tz2/zEg5j7ilv/aY3SJE ceph-69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:26:11.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: The key's randomart image is: 2024-05-09T05:26:11.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: +---[RSA 3072]----+ 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: | o.. | 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: | o =. | 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: | . o . =o . . | 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: | o o +ooo E | 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: | . o+So. . | 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: | o..+*= o | 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: | ..o++* o | 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: | oo +oB + | 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: | ..*+.oOo. | 2024-05-09T05:26:11.649 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: +----[SHA256]-----+ 2024-05-09T05:26:11.961 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:11 smithi115 ceph-mon[30349]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:11.962 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:11 smithi115 ceph-mon[30349]: Generating ssh key... 2024-05-09T05:26:11.962 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:11 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:11.962 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:11 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:11.969 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.841+0000 7ff62d25d700 1 Processor -- start 2024-05-09T05:26:11.969 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.841+0000 7ff62d25d700 1 -- start start 2024-05-09T05:26:11.969 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.841+0000 7ff62d25d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff628108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:11.969 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.841+0000 7ff62d25d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281093d0 con 0x7ff6281089d0 2024-05-09T05:26:11.969 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.841+0000 7ff626d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff628108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:11.969 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.841+0000 7ff626d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff628108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36220/0 (socket says 172.21.15.115:36220) 2024-05-09T05:26:11.969 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.841+0000 7ff626d9d700 1 -- 172.21.15.115:0/1889145207 learned_addr learned my addr 172.21.15.115:0/1889145207 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.841+0000 7ff626d9d700 1 -- 172.21.15.115:0/1889145207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff628109ba0 con 0x7ff6281089d0 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.842+0000 7ff626d9d700 1 --2- 172.21.15.115:0/1889145207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff628108de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff6180098a0 tx=0x7ff618009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca9a9943ad47f14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.842+0000 7ff625d9b700 1 -- 172.21.15.115:0/1889145207 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff618004160 con 0x7ff6281089d0 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.842+0000 7ff625d9b700 1 -- 172.21.15.115:0/1889145207 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff618004300 con 0x7ff6281089d0 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.842+0000 7ff625d9b700 1 -- 172.21.15.115:0/1889145207 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6180119e0 con 0x7ff6281089d0 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.842+0000 7ff62d25d700 1 -- 172.21.15.115:0/1889145207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 msgr2=0x7ff628108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.842+0000 7ff62d25d700 1 --2- 172.21.15.115:0/1889145207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff628108de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff6180098a0 tx=0x7ff618009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.842+0000 7ff62d25d700 1 -- 172.21.15.115:0/1889145207 shutdown_connections 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.843+0000 7ff62d25d700 1 --2- 172.21.15.115:0/1889145207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff628108de0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.843+0000 7ff62d25d700 1 -- 172.21.15.115:0/1889145207 >> 172.21.15.115:0/1889145207 conn(0x7ff6281040b0 msgr2=0x7ff628106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.843+0000 7ff62d25d700 1 -- 172.21.15.115:0/1889145207 shutdown_connections 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.843+0000 7ff62d25d700 1 -- 172.21.15.115:0/1889145207 wait complete. 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.843+0000 7ff62d25d700 1 Processor -- start 2024-05-09T05:26:11.970 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.844+0000 7ff62d25d700 1 -- start start 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.844+0000 7ff62d25d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff62819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.844+0000 7ff62d25d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810cf50 con 0x7ff6281089d0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.844+0000 7ff626d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff62819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.844+0000 7ff626d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff62819de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36226/0 (socket says 172.21.15.115:36226) 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.844+0000 7ff626d9d700 1 -- 172.21.15.115:0/638290330 learned_addr learned my addr 172.21.15.115:0/638290330 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.844+0000 7ff626d9d700 1 -- 172.21.15.115:0/638290330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62819e370 con 0x7ff6281089d0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.844+0000 7ff626d9d700 1 --2- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff62819de30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff618009fb0 tx=0x7ff618011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.844+0000 7ff60ffff700 1 -- 172.21.15.115:0/638290330 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6180038b0 con 0x7ff6281089d0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.845+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62819e600 con 0x7ff6281089d0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.845+0000 7ff60ffff700 1 -- 172.21.15.115:0/638290330 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff618003a50 con 0x7ff6281089d0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.845+0000 7ff60ffff700 1 -- 172.21.15.115:0/638290330 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff618019550 con 0x7ff6281089d0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.845+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6281a1210 con 0x7ff6281089d0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.846+0000 7ff60ffff700 1 -- 172.21.15.115:0/638290330 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff618019820 con 0x7ff6281089d0 2024-05-09T05:26:11.971 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.846+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff628064e20 con 0x7ff6281089d0 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.846+0000 7ff60ffff700 1 --2- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff61003e530 0x7ff610040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.846+0000 7ff60ffff700 1 -- 172.21.15.115:0/638290330 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff618053ac0 con 0x7ff6281089d0 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.846+0000 7ff62659c700 1 --2- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff61003e530 0x7ff610040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.846+0000 7ff62659c700 1 --2- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff61003e530 0x7ff610040a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff61c006fd0 tx=0x7ff61c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:10.851+0000 7ff60ffff700 1 -- 172.21.15.115:0/638290330 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff618017e90 con 0x7ff6281089d0 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.053+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ff62802e8d0 con 0x7ff61003e530 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.391+0000 7ff60ffff700 1 -- 172.21.15.115:0/638290330 <== mgr.14118 v2:172.21.15.115:6800/792641138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff62802e8d0 con 0x7ff61003e530 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff61003e530 msgr2=0x7ff610040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 --2- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff61003e530 0x7ff610040a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff61c006fd0 tx=0x7ff61c006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 msgr2=0x7ff62819de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 --2- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff62819de30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff618009fb0 tx=0x7ff618011a60 comp rx=0 tx=0).stop 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 shutdown_connections 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 --2- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7ff61003e530 0x7ff610040a40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 --2- 172.21.15.115:0/638290330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6281089d0 0x7ff62819de30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:11.972 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 >> 172.21.15.115:0/638290330 conn(0x7ff6281040b0 msgr2=0x7ff6281943e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:11.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 shutdown_connections 2024-05-09T05:26:11.973 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:11.393+0000 7ff62d25d700 1 -- 172.21.15.115:0/638290330 wait complete. 2024-05-09T05:26:13.169 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoFfGMwA/8lkOGyOqFfL0OlSV1YErc2apcgLPrNr5h6cphil3iusu3zA/+9ytig4q7j+qOK1ZZzDGmRPuAmjLHzkjrNSNeNUIJ0SFjltjLPG+tyGY9eqpvLql42vGBRMs1Gc6eAPEJCunIyjboHPHQKlgptJBCFNs2Jx65VN3Q2hmPfR3CFTLpgw7keJb/XsTUd25YVFTo1THovuprHGQc4xz7A+6sAg/hvgLybz5Q7mqUFP3Unwp4oNne2Z5fF3ElvKHTSvoudZOE9lisdvljprdly5Y0fN98266iWfbprPWmAvvYlf26m4l04jsgZGIhuJAviuU4QtBq9Mr91kHMLLRG2yhDEduPddb5x2ofnXzovF4fg4uO6HpUieIzCN0bfmu52Vupu5oPE7fjbZehrvJCXYRWq0Wjev31x0ivBTmo1sh4wmq4Fvh6/N0j3EErPPWNss+eNgQIz3UPpD1kPXDj7nufY8il24QUXfQgpJTvBvXG4LQXRxpTWIe70RU= ceph-69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:26:13.169 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.375+0000 7fd52a294700 1 Processor -- start 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.375+0000 7fd52a294700 1 -- start start 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.375+0000 7fd52a294700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52410c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.375+0000 7fd52a294700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd52407d700 con 0x7fd52410a5b0 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.375+0000 7fd523fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52410c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.375+0000 7fd523fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52410c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36242/0 (socket says 172.21.15.115:36242) 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.375+0000 7fd523fff700 1 -- 172.21.15.115:0/2285373800 learned_addr learned my addr 172.21.15.115:0/2285373800 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.375+0000 7fd523fff700 1 -- 172.21.15.115:0/2285373800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52410cf10 con 0x7fd52410a5b0 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.376+0000 7fd523fff700 1 --2- 172.21.15.115:0/2285373800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52410c9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd5140098a0 tx=0x7fd514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b55eca2feee7fde server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.376+0000 7fd522ffd700 1 -- 172.21.15.115:0/2285373800 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514004160 con 0x7fd52410a5b0 2024-05-09T05:26:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.376+0000 7fd522ffd700 1 -- 172.21.15.115:0/2285373800 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd514004300 con 0x7fd52410a5b0 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.376+0000 7fd522ffd700 1 -- 172.21.15.115:0/2285373800 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5140119e0 con 0x7fd52410a5b0 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.376+0000 7fd52a294700 1 -- 172.21.15.115:0/2285373800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 msgr2=0x7fd52410c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.376+0000 7fd52a294700 1 --2- 172.21.15.115:0/2285373800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52410c9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd5140098a0 tx=0x7fd514009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.377+0000 7fd52a294700 1 -- 172.21.15.115:0/2285373800 shutdown_connections 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.377+0000 7fd52a294700 1 --2- 172.21.15.115:0/2285373800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52410c9d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.377+0000 7fd52a294700 1 -- 172.21.15.115:0/2285373800 >> 172.21.15.115:0/2285373800 conn(0x7fd5241040b0 msgr2=0x7fd524106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.377+0000 7fd52a294700 1 -- 172.21.15.115:0/2285373800 shutdown_connections 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.377+0000 7fd52a294700 1 -- 172.21.15.115:0/2285373800 wait complete. 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.377+0000 7fd52a294700 1 Processor -- start 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.377+0000 7fd52a294700 1 -- start start 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd52a294700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd52a294700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd52410de70 con 0x7fd52410a5b0 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd523fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:13.171 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd523fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52419dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36248/0 (socket says 172.21.15.115:36248) 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd523fff700 1 -- 172.21.15.115:0/2568068693 learned_addr learned my addr 172.21.15.115:0/2568068693 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd523fff700 1 -- 172.21.15.115:0/2568068693 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52419e2c0 con 0x7fd52410a5b0 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd523fff700 1 --2- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52419dd80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd5140119b0 tx=0x7fd514011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd5217fa700 1 -- 172.21.15.115:0/2568068693 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514003af0 con 0x7fd52410a5b0 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd5217fa700 1 -- 172.21.15.115:0/2568068693 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd51401a8a0 con 0x7fd52410a5b0 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52419e550 con 0x7fd52410a5b0 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd5217fa700 1 -- 172.21.15.115:0/2568068693 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514024830 con 0x7fd52410a5b0 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.378+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5241a1160 con 0x7fd52410a5b0 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.379+0000 7fd5217fa700 1 -- 172.21.15.115:0/2568068693 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd514022020 con 0x7fd52410a5b0 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.379+0000 7fd5217fa700 1 --2- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fd50c03e530 0x7fd50c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:13.172 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.379+0000 7fd5217fa700 1 -- 172.21.15.115:0/2568068693 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd514054810 con 0x7fd52410a5b0 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.380+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd524064e20 con 0x7fd52410a5b0 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.380+0000 7fd5237fe700 1 --2- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fd50c03e530 0x7fd50c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.380+0000 7fd5237fe700 1 --2- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fd50c03e530 0x7fd50c040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd518006fd0 tx=0x7fd518006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.385+0000 7fd5217fa700 1 -- 172.21.15.115:0/2568068693 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5140094f0 con 0x7fd52410a5b0 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.590+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd524106140 con 0x7fd50c03e530 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.591+0000 7fd5217fa700 1 -- 172.21.15.115:0/2568068693 <== mgr.14118 v2:172.21.15.115:6800/792641138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd524106140 con 0x7fd50c03e530 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fd50c03e530 msgr2=0x7fd50c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 --2- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fd50c03e530 0x7fd50c040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd518006fd0 tx=0x7fd518006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 msgr2=0x7fd52419dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 --2- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52419dd80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd5140119b0 tx=0x7fd514011a90 comp rx=0 tx=0).stop 2024-05-09T05:26:13.173 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 shutdown_connections 2024-05-09T05:26:13.174 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 --2- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fd50c03e530 0x7fd50c040a40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:13.174 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 --2- 172.21.15.115:0/2568068693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd52410a5b0 0x7fd52419dd80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:13.174 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 >> 172.21.15.115:0/2568068693 conn(0x7fd5241040b0 msgr2=0x7fd524105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:13.174 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 shutdown_connections 2024-05-09T05:26:13.174 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:12.593+0000 7fd52a294700 1 -- 172.21.15.115:0/2568068693 wait complete. 2024-05-09T05:26:13.174 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T05:26:13.174 INFO:teuthology.orchestra.run.smithi115.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T05:26:13.174 INFO:teuthology.orchestra.run.smithi115.stdout:Adding host smithi115... 2024-05-09T05:26:13.454 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:13 smithi115 ceph-mon[30349]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:14.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:14 smithi115 ceph-mon[30349]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "addr": "172.21.15.115", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:15.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:15 smithi115 ceph-mon[30349]: Deploying cephadm binary to smithi115 2024-05-09T05:26:18.444 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Added host 'smithi115' with addr '172.21.15.115' 2024-05-09T05:26:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.581+0000 7fb11c765700 1 Processor -- start 2024-05-09T05:26:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.581+0000 7fb11c765700 1 -- start start 2024-05-09T05:26:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.581+0000 7fb11c765700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11410ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.581+0000 7fb11c765700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb11407d6c0 con 0x7fb11410a5f0 2024-05-09T05:26:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.581+0000 7fb11a501700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11410ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:18.445 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.581+0000 7fb11a501700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11410ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55932/0 (socket says 172.21.15.115:55932) 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.581+0000 7fb11a501700 1 -- 172.21.15.115:0/879747705 learned_addr learned my addr 172.21.15.115:0/879747705 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.582+0000 7fb11a501700 1 -- 172.21.15.115:0/879747705 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11410cf50 con 0x7fb11410a5f0 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.582+0000 7fb11a501700 1 --2- 172.21.15.115:0/879747705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11410ca10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1100098a0 tx=0x7fb110009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1501752bd1a00a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.582+0000 7fb1194ff700 1 -- 172.21.15.115:0/879747705 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb110004160 con 0x7fb11410a5f0 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.582+0000 7fb1194ff700 1 -- 172.21.15.115:0/879747705 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb110004300 con 0x7fb11410a5f0 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.582+0000 7fb1194ff700 1 -- 172.21.15.115:0/879747705 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1100119e0 con 0x7fb11410a5f0 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.583+0000 7fb11c765700 1 -- 172.21.15.115:0/879747705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 msgr2=0x7fb11410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.583+0000 7fb11c765700 1 --2- 172.21.15.115:0/879747705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11410ca10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1100098a0 tx=0x7fb110009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.583+0000 7fb11c765700 1 -- 172.21.15.115:0/879747705 shutdown_connections 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.583+0000 7fb11c765700 1 --2- 172.21.15.115:0/879747705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11410ca10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:18.446 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.583+0000 7fb11c765700 1 -- 172.21.15.115:0/879747705 >> 172.21.15.115:0/879747705 conn(0x7fb1141040b0 msgr2=0x7fb114106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.583+0000 7fb11c765700 1 -- 172.21.15.115:0/879747705 shutdown_connections 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.583+0000 7fb11c765700 1 -- 172.21.15.115:0/879747705 wait complete. 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11c765700 1 Processor -- start 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11c765700 1 -- start start 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11c765700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11c765700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb11410de50 con 0x7fb11410a5f0 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11a501700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11a501700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55936/0 (socket says 172.21.15.115:55936) 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11a501700 1 -- 172.21.15.115:0/1091762519 learned_addr learned my addr 172.21.15.115:0/1091762519 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11a501700 1 -- 172.21.15.115:0/1091762519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11419e2d0 con 0x7fb11410a5f0 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11a501700 1 --2- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11419dd90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb1100119b0 tx=0x7fb110011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb10b7fe700 1 -- 172.21.15.115:0/1091762519 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb110003af0 con 0x7fb11410a5f0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb10b7fe700 1 -- 172.21.15.115:0/1091762519 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb11001a8a0 con 0x7fb11410a5f0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11419e560 con 0x7fb11410a5f0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.584+0000 7fb10b7fe700 1 -- 172.21.15.115:0/1091762519 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb110024830 con 0x7fb11410a5f0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.585+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1141a1170 con 0x7fb11410a5f0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.585+0000 7fb10b7fe700 1 -- 172.21.15.115:0/1091762519 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb110022020 con 0x7fb11410a5f0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.586+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb114064e20 con 0x7fb11410a5f0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.586+0000 7fb10b7fe700 1 --2- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fb10003e580 0x7fb100040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.586+0000 7fb10b7fe700 1 -- 172.21.15.115:0/1091762519 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb110053890 con 0x7fb11410a5f0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.586+0000 7fb119d00700 1 --2- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fb10003e580 0x7fb100040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.586+0000 7fb119d00700 1 --2- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fb10003e580 0x7fb100040a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb104006fd0 tx=0x7fb104006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.591+0000 7fb10b7fe700 1 -- 172.21.15.115:0/1091762519 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb110024a20 con 0x7fb11410a5f0 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:13.796+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi115", "addr": "172.21.15.115", "target": ["mon-mgr", ""]}) -- 0x7fb114106120 con 0x7fb10003e580 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.895+0000 7fb10b7fe700 1 -- 172.21.15.115:0/1091762519 <== mgr.14118 v2:172.21.15.115:6800/792641138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb114106120 con 0x7fb10003e580 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fb10003e580 msgr2=0x7fb100040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 --2- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fb10003e580 0x7fb100040a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb104006fd0 tx=0x7fb104006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 msgr2=0x7fb11419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 --2- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11419dd90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb1100119b0 tx=0x7fb110011a90 comp rx=0 tx=0).stop 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 shutdown_connections 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 --2- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fb10003e580 0x7fb100040a90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 --2- 172.21.15.115:0/1091762519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb11410a5f0 0x7fb11419dd90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 >> 172.21.15.115:0/1091762519 conn(0x7fb1141040b0 msgr2=0x7fb114105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 shutdown_connections 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:17.898+0000 7fb11c765700 1 -- 172.21.15.115:0/1091762519 wait complete. 2024-05-09T05:26:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying unmanaged mon service... 2024-05-09T05:26:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:18 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:19.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:18 smithi115 ceph-mon[30349]: Added host smithi115 2024-05-09T05:26:19.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:18 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:20.080 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T05:26:20.080 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.014+0000 7fbecea72700 1 Processor -- start 2024-05-09T05:26:20.080 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.014+0000 7fbecea72700 1 -- start start 2024-05-09T05:26:20.080 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.014+0000 7fbecea72700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec8108d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:20.080 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.014+0000 7fbecea72700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbec8109360 con 0x7fbec8108960 2024-05-09T05:26:20.080 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.015+0000 7fbecc80e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec8108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:20.080 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.015+0000 7fbecc80e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec8108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55938/0 (socket says 172.21.15.115:55938) 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.015+0000 7fbecc80e700 1 -- 172.21.15.115:0/3614123626 learned_addr learned my addr 172.21.15.115:0/3614123626 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.015+0000 7fbecc80e700 1 -- 172.21.15.115:0/3614123626 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec8109b30 con 0x7fbec8108960 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.015+0000 7fbecc80e700 1 --2- 172.21.15.115:0/3614123626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec8108d70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0098a0 tx=0x7fbebc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c910fc2bcff685ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.015+0000 7fbec77fe700 1 -- 172.21.15.115:0/3614123626 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbebc004160 con 0x7fbec8108960 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.015+0000 7fbec77fe700 1 -- 172.21.15.115:0/3614123626 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fbebc004300 con 0x7fbec8108960 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.015+0000 7fbec77fe700 1 -- 172.21.15.115:0/3614123626 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbebc0119e0 con 0x7fbec8108960 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.016+0000 7fbecea72700 1 -- 172.21.15.115:0/3614123626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 msgr2=0x7fbec8108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.016+0000 7fbecea72700 1 --2- 172.21.15.115:0/3614123626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec8108d70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0098a0 tx=0x7fbebc009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.016+0000 7fbecea72700 1 -- 172.21.15.115:0/3614123626 shutdown_connections 2024-05-09T05:26:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.016+0000 7fbecea72700 1 --2- 172.21.15.115:0/3614123626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec8108d70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.016+0000 7fbecea72700 1 -- 172.21.15.115:0/3614123626 >> 172.21.15.115:0/3614123626 conn(0x7fbec8103fe0 msgr2=0x7fbec8106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.016+0000 7fbecea72700 1 -- 172.21.15.115:0/3614123626 shutdown_connections 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.016+0000 7fbecea72700 1 -- 172.21.15.115:0/3614123626 wait complete. 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecea72700 1 Processor -- start 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecea72700 1 -- start start 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecea72700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec807c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecea72700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810cee0 con 0x7fbec8108960 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecc80e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec807c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecc80e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec807c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55952/0 (socket says 172.21.15.115:55952) 2024-05-09T05:26:20.082 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecc80e700 1 -- 172.21.15.115:0/3933264895 learned_addr learned my addr 172.21.15.115:0/3933264895 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:20.083 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecc80e700 1 -- 172.21.15.115:0/3933264895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec807ce80 con 0x7fbec8108960 2024-05-09T05:26:20.083 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecc80e700 1 --2- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec807c940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0119b0 tx=0x7fbebc011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:20.083 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbec5ffb700 1 -- 172.21.15.115:0/3933264895 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbebc003970 con 0x7fbec8108960 2024-05-09T05:26:20.083 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbec5ffb700 1 -- 172.21.15.115:0/3933264895 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fbebc003b10 con 0x7fbec8108960 2024-05-09T05:26:20.083 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec81a0b90 con 0x7fbec8108960 2024-05-09T05:26:20.083 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.017+0000 7fbec5ffb700 1 -- 172.21.15.115:0/3933264895 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbebc024610 con 0x7fbec8108960 2024-05-09T05:26:20.083 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.018+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec81a1010 con 0x7fbec8108960 2024-05-09T05:26:20.083 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.019+0000 7fbec5ffb700 1 -- 172.21.15.115:0/3933264895 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fbebc02d3f0 con 0x7fbec8108960 2024-05-09T05:26:20.083 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.018+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec8064e20 con 0x7fbec8108960 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.019+0000 7fbec5ffb700 1 --2- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fbeb803e580 0x7fbeb8040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.019+0000 7fbec5ffb700 1 -- 172.21.15.115:0/3933264895 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbebc053920 con 0x7fbec8108960 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.019+0000 7fbec7fff700 1 --2- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fbeb803e580 0x7fbeb8040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.019+0000 7fbec7fff700 1 --2- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fbeb803e580 0x7fbeb8040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4006fd0 tx=0x7fbeb4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.024+0000 7fbec5ffb700 1 -- 172.21.15.115:0/3933264895 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbebc00e260 con 0x7fbec8108960 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.252+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbec8104fe0 con 0x7fbeb803e580 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.258+0000 7fbec5ffb700 1 -- 172.21.15.115:0/3933264895 <== mgr.14118 v2:172.21.15.115:6800/792641138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbec8104fe0 con 0x7fbeb803e580 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fbeb803e580 msgr2=0x7fbeb8040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 --2- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fbeb803e580 0x7fbeb8040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4006fd0 tx=0x7fbeb4006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:20.084 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 msgr2=0x7fbec807c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:20.085 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 --2- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec807c940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0119b0 tx=0x7fbebc011a90 comp rx=0 tx=0).stop 2024-05-09T05:26:20.085 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 shutdown_connections 2024-05-09T05:26:20.085 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 --2- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fbeb803e580 0x7fbeb8040a90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:20.085 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 --2- 172.21.15.115:0/3933264895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec8108960 0x7fbec807c940 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:20.085 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 >> 172.21.15.115:0/3933264895 conn(0x7fbec8103fe0 msgr2=0x7fbec81048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:20.085 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 shutdown_connections 2024-05-09T05:26:20.085 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:19.260+0000 7fbecea72700 1 -- 172.21.15.115:0/3933264895 wait complete. 2024-05-09T05:26:20.085 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying unmanaged mgr service... 2024-05-09T05:26:20.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:20 smithi115 ceph-mon[30349]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:20.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:20 smithi115 ceph-mon[30349]: Saving service mon spec with placement count:5 2024-05-09T05:26:20.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:20 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:21.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T05:26:21.381 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.624+0000 7f3f7b5bc700 1 Processor -- start 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.625+0000 7f3f7b5bc700 1 -- start start 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.625+0000 7f3f7b5bc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7410a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.625+0000 7f3f7b5bc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7410a930 con 0x7f3f74109f10 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.625+0000 7f3f79358700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7410a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.625+0000 7f3f79358700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7410a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55960/0 (socket says 172.21.15.115:55960) 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.625+0000 7f3f79358700 1 -- 172.21.15.115:0/1384135159 learned_addr learned my addr 172.21.15.115:0/1384135159 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.625+0000 7f3f79358700 1 -- 172.21.15.115:0/1384135159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f74108b40 con 0x7f3f74109f10 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.625+0000 7f3f79358700 1 --2- 172.21.15.115:0/1384135159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7410a320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800bb40 tx=0x7f3f6800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5890f37cc680ba94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f67fff700 1 -- 172.21.15.115:0/1384135159 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f6800dbe0 con 0x7f3f74109f10 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f67fff700 1 -- 172.21.15.115:0/1384135159 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3f680044a0 con 0x7f3f74109f10 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f67fff700 1 -- 172.21.15.115:0/1384135159 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f68003e30 con 0x7f3f74109f10 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/1384135159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 msgr2=0x7f3f7410a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:21.382 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f7b5bc700 1 --2- 172.21.15.115:0/1384135159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7410a320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800bb40 tx=0x7f3f6800be50 comp rx=0 tx=0).stop 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/1384135159 shutdown_connections 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f7b5bc700 1 --2- 172.21.15.115:0/1384135159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7410a320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/1384135159 >> 172.21.15.115:0/1384135159 conn(0x7f3f74103780 msgr2=0x7f3f74105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/1384135159 shutdown_connections 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.626+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/1384135159 wait complete. 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.627+0000 7f3f7b5bc700 1 Processor -- start 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.627+0000 7f3f7b5bc700 1 -- start start 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.627+0000 7f3f7b5bc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.627+0000 7f3f7b5bc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3f74107a30 con 0x7f3f74109f10 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.627+0000 7f3f79358700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.627+0000 7f3f79358700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55964/0 (socket says 172.21.15.115:55964) 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.627+0000 7f3f79358700 1 -- 172.21.15.115:0/2671548286 learned_addr learned my addr 172.21.15.115:0/2671548286 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.628+0000 7f3f79358700 1 -- 172.21.15.115:0/2671548286 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7419c090 con 0x7f3f74109f10 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.628+0000 7f3f79358700 1 --2- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7419bb50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800bfd0 tx=0x7f3f68003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:21.383 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.628+0000 7f3f667fc700 1 -- 172.21.15.115:0/2671548286 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f68027070 con 0x7f3f74109f10 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.628+0000 7f3f667fc700 1 -- 172.21.15.115:0/2671548286 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3f68010a40 con 0x7f3f74109f10 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.628+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7419c320 con 0x7f3f74109f10 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.628+0000 7f3f667fc700 1 -- 172.21.15.115:0/2671548286 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f680229f0 con 0x7f3f74109f10 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.628+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7419ef30 con 0x7f3f74109f10 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.629+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f74064e20 con 0x7f3f74109f10 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.629+0000 7f3f667fc700 1 -- 172.21.15.115:0/2671548286 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3f6802a070 con 0x7f3f74109f10 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.629+0000 7f3f667fc700 1 --2- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f3f6003e4e0 0x7f3f600409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.629+0000 7f3f667fc700 1 -- 172.21.15.115:0/2671548286 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f6801f070 con 0x7f3f74109f10 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.629+0000 7f3f78b57700 1 --2- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f3f6003e4e0 0x7f3f600409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.630+0000 7f3f78b57700 1 --2- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f3f6003e4e0 0x7f3f600409f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f70006fd0 tx=0x7f3f70006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.635+0000 7f3f667fc700 1 -- 172.21.15.115:0/2671548286 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f68015070 con 0x7f3f74109f10 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.855+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3f7419f1c0 con 0x7f3f6003e4e0 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.860+0000 7f3f667fc700 1 -- 172.21.15.115:0/2671548286 <== mgr.14118 v2:172.21.15.115:6800/792641138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3f7419f1c0 con 0x7f3f6003e4e0 2024-05-09T05:26:21.384 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f3f6003e4e0 msgr2=0x7f3f600409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:21.385 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 --2- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f3f6003e4e0 0x7f3f600409f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f70006fd0 tx=0x7f3f70006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:21.385 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 msgr2=0x7f3f7419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:21.385 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 --2- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7419bb50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800bfd0 tx=0x7f3f68003f80 comp rx=0 tx=0).stop 2024-05-09T05:26:21.385 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 shutdown_connections 2024-05-09T05:26:21.385 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 --2- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f3f6003e4e0 0x7f3f600409f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:21.385 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 --2- 172.21.15.115:0/2671548286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f74109f10 0x7f3f7419bb50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:21.385 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 >> 172.21.15.115:0/2671548286 conn(0x7f3f74103780 msgr2=0x7f3f74191e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:21.385 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 shutdown_connections 2024-05-09T05:26:21.385 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:20.862+0000 7f3f7b5bc700 1 -- 172.21.15.115:0/2671548286 wait complete. 2024-05-09T05:26:22.022 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:21 smithi115 ceph-mon[30349]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:22.022 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:21 smithi115 ceph-mon[30349]: Saving service mgr spec with placement count:2 2024-05-09T05:26:22.022 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:21 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:22.022 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:21 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:22.022 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:21 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:22.530 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.810+0000 7fc1a6d17700 1 Processor -- start 2024-05-09T05:26:22.530 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.810+0000 7fc1a6d17700 1 -- start start 2024-05-09T05:26:22.530 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc1a6d17700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a007dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:22.530 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc1a6d17700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a007e1a0 con 0x7fc1a007f7e0 2024-05-09T05:26:22.530 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc1a4ab3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a007dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:22.530 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc1a4ab3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a007dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55968/0 (socket says 172.21.15.115:55968) 2024-05-09T05:26:22.530 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc1a4ab3700 1 -- 172.21.15.115:0/1608854080 learned_addr learned my addr 172.21.15.115:0/1608854080 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:22.530 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc1a4ab3700 1 -- 172.21.15.115:0/1608854080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a007e320 con 0x7fc1a007f7e0 2024-05-09T05:26:22.530 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc1a4ab3700 1 --2- 172.21.15.115:0/1608854080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a007dc60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc1940098a0 tx=0x7fc194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fd5bd5373b2366 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc19f7fe700 1 -- 172.21.15.115:0/1608854080 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc194004160 con 0x7fc1a007f7e0 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc19f7fe700 1 -- 172.21.15.115:0/1608854080 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc194004300 con 0x7fc1a007f7e0 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.811+0000 7fc19f7fe700 1 -- 172.21.15.115:0/1608854080 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc194011960 con 0x7fc1a007f7e0 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.812+0000 7fc1a6d17700 1 -- 172.21.15.115:0/1608854080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 msgr2=0x7fc1a007dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.812+0000 7fc1a6d17700 1 --2- 172.21.15.115:0/1608854080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a007dc60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc1940098a0 tx=0x7fc194009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.812+0000 7fc1a6d17700 1 -- 172.21.15.115:0/1608854080 shutdown_connections 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.812+0000 7fc1a6d17700 1 --2- 172.21.15.115:0/1608854080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a007dc60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.812+0000 7fc1a6d17700 1 -- 172.21.15.115:0/1608854080 >> 172.21.15.115:0/1608854080 conn(0x7fc1a0105140 msgr2=0x7fc1a01075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.812+0000 7fc1a6d17700 1 -- 172.21.15.115:0/1608854080 shutdown_connections 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.812+0000 7fc1a6d17700 1 -- 172.21.15.115:0/1608854080 wait complete. 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.813+0000 7fc1a6d17700 1 Processor -- start 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.813+0000 7fc1a6d17700 1 -- start start 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.813+0000 7fc1a6d17700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:22.531 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.813+0000 7fc1a6d17700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0110050 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.813+0000 7fc1a4ab3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.813+0000 7fc1a4ab3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a019ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55976/0 (socket says 172.21.15.115:55976) 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.813+0000 7fc1a4ab3700 1 -- 172.21.15.115:0/4138502810 learned_addr learned my addr 172.21.15.115:0/4138502810 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.814+0000 7fc1a4ab3700 1 -- 172.21.15.115:0/4138502810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a01a04a0 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.814+0000 7fc1a4ab3700 1 --2- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a019ff60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc194011670 tx=0x7fc194011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.814+0000 7fc19dffb700 1 -- 172.21.15.115:0/4138502810 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc194003a60 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.814+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a01a0730 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.814+0000 7fc19dffb700 1 -- 172.21.15.115:0/4138502810 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc19401b8d0 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.814+0000 7fc19dffb700 1 -- 172.21.15.115:0/4138502810 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc194023750 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.814+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a01a3340 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.815+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a0064e20 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.815+0000 7fc19dffb700 1 -- 172.21.15.115:0/4138502810 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc194023a40 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.816+0000 7fc19dffb700 1 --2- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fc19003e530 0x7fc190040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.816+0000 7fc19dffb700 1 -- 172.21.15.115:0/4138502810 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc194053f90 con 0x7fc1a007f7e0 2024-05-09T05:26:22.532 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.816+0000 7fc19ffff700 1 --2- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fc19003e530 0x7fc190040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.816+0000 7fc19ffff700 1 --2- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fc19003e530 0x7fc190040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc18c006fd0 tx=0x7fc18c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:21.821+0000 7fc19dffb700 1 -- 172.21.15.115:0/4138502810 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc19401f030 con 0x7fc1a007f7e0 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.026+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc1a01a3a20 con 0x7fc1a007f7e0 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.033+0000 7fc19dffb700 1 -- 172.21.15.115:0/4138502810 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fc19400b3e0 con 0x7fc1a007f7e0 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fc19003e530 msgr2=0x7fc190040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 --2- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fc19003e530 0x7fc190040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc18c006fd0 tx=0x7fc18c006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 msgr2=0x7fc1a019ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 --2- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a019ff60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc194011670 tx=0x7fc194011a30 comp rx=0 tx=0).stop 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 shutdown_connections 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 --2- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7fc19003e530 0x7fc190040a40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 --2- 172.21.15.115:0/4138502810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a007f7e0 0x7fc1a019ff60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 >> 172.21.15.115:0/4138502810 conn(0x7fc1a0105140 msgr2=0x7fc1a0105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 shutdown_connections 2024-05-09T05:26:22.533 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:22.035+0000 7fc1a6d17700 1 -- 172.21.15.115:0/4138502810 wait complete. 2024-05-09T05:26:23.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:23 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4138502810' entity='client.admin' 2024-05-09T05:26:23.923 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.030+0000 7f2bccd95700 1 Processor -- start 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.030+0000 7f2bccd95700 1 -- start start 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.030+0000 7f2bccd95700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc8108d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.030+0000 7f2bccd95700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc8109360 con 0x7f2bc8108960 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.030+0000 7f2bc659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc8108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.030+0000 7f2bc659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc8108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55160/0 (socket says 172.21.15.115:55160) 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.030+0000 7f2bc659c700 1 -- 172.21.15.115:0/4083306627 learned_addr learned my addr 172.21.15.115:0/4083306627 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.031+0000 7f2bc659c700 1 -- 172.21.15.115:0/4083306627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc8109b30 con 0x7f2bc8108960 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.031+0000 7f2bc659c700 1 --2- 172.21.15.115:0/4083306627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc8108d70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80098a0 tx=0x7f2bb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e2c6794b1de1ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.033+0000 7f2bc559a700 1 -- 172.21.15.115:0/4083306627 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bb8004160 con 0x7f2bc8108960 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.033+0000 7f2bc559a700 1 -- 172.21.15.115:0/4083306627 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bb8004300 con 0x7f2bc8108960 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.033+0000 7f2bc559a700 1 -- 172.21.15.115:0/4083306627 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bb8011960 con 0x7f2bc8108960 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.034+0000 7f2bccd95700 1 -- 172.21.15.115:0/4083306627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 msgr2=0x7f2bc8108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.034+0000 7f2bccd95700 1 --2- 172.21.15.115:0/4083306627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc8108d70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80098a0 tx=0x7f2bb8009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.034+0000 7f2bccd95700 1 -- 172.21.15.115:0/4083306627 shutdown_connections 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.034+0000 7f2bccd95700 1 --2- 172.21.15.115:0/4083306627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc8108d70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.034+0000 7f2bccd95700 1 -- 172.21.15.115:0/4083306627 >> 172.21.15.115:0/4083306627 conn(0x7f2bc8103fe0 msgr2=0x7f2bc8106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.035+0000 7f2bccd95700 1 -- 172.21.15.115:0/4083306627 shutdown_connections 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.035+0000 7f2bccd95700 1 -- 172.21.15.115:0/4083306627 wait complete. 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.035+0000 7f2bccd95700 1 Processor -- start 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.036+0000 7f2bccd95700 1 -- start start 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.036+0000 7f2bccd95700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.036+0000 7f2bccd95700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc810cee0 con 0x7f2bc8108960 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.036+0000 7f2bc659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.036+0000 7f2bc659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55164/0 (socket says 172.21.15.115:55164) 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.036+0000 7f2bc659c700 1 -- 172.21.15.115:0/914321688 learned_addr learned my addr 172.21.15.115:0/914321688 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:23.925 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.036+0000 7f2bc659c700 1 -- 172.21.15.115:0/914321688 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc819d240 con 0x7f2bc8108960 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.036+0000 7f2bc659c700 1 --2- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc819cd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006af0 tx=0x7f2bb8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.037+0000 7f2bb77fe700 1 -- 172.21.15.115:0/914321688 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bb8003af0 con 0x7f2bc8108960 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.037+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc819d4d0 con 0x7f2bc8108960 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.037+0000 7f2bb77fe700 1 -- 172.21.15.115:0/914321688 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bb801b8d0 con 0x7f2bc8108960 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.037+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc807bfc0 con 0x7f2bc8108960 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.037+0000 7f2bb77fe700 1 -- 172.21.15.115:0/914321688 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bb80237e0 con 0x7f2bc8108960 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.038+0000 7f2bb77fe700 1 -- 172.21.15.115:0/914321688 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2bb8023980 con 0x7f2bc8108960 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.038+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc8064e20 con 0x7f2bc8108960 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.038+0000 7f2bb77fe700 1 --2- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f2bb003e530 0x7f2bb0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.038+0000 7f2bb77fe700 1 -- 172.21.15.115:0/914321688 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2bb8054e40 con 0x7f2bc8108960 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.038+0000 7f2bc5d9b700 1 --2- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f2bb003e530 0x7f2bb0040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.039+0000 7f2bc5d9b700 1 --2- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f2bb003e530 0x7f2bb0040a40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc006fd0 tx=0x7f2bbc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:23.926 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.044+0000 7f2bb77fe700 1 -- 172.21.15.115:0/914321688 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2bb801f030 con 0x7f2bc8108960 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.257+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f2bc807c690 con 0x7f2bc8108960 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.261+0000 7f2bb77fe700 1 -- 172.21.15.115:0/914321688 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f2bb802b3a0 con 0x7f2bc8108960 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f2bb003e530 msgr2=0x7f2bb0040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 --2- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f2bb003e530 0x7f2bb0040a40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc006fd0 tx=0x7f2bbc006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 msgr2=0x7f2bc819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 --2- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc819cd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006af0 tx=0x7f2bb8011a30 comp rx=0 tx=0).stop 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 shutdown_connections 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 --2- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f2bb003e530 0x7f2bb0040a40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 --2- 172.21.15.115:0/914321688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bc8108960 0x7f2bc819cd00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 >> 172.21.15.115:0/914321688 conn(0x7f2bc8103fe0 msgr2=0x7f2bc81048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 shutdown_connections 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:23.263+0000 7f2bccd95700 1 -- 172.21.15.115:0/914321688 wait complete. 2024-05-09T05:26:23.927 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling the dashboard module... 2024-05-09T05:26:24.509 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:24 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/914321688' entity='client.admin' 2024-05-09T05:26:25.377 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:25 smithi115 ceph-mon[30349]: from='mgr.14118 172.21.15.115:0/2951475448' entity='mgr.x' 2024-05-09T05:26:25.377 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:25 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4230003096' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T05:26:25.647 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:25 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: ignoring --setuser ceph since I am not root 2024-05-09T05:26:25.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:25 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: ignoring --setgroup ceph since I am not root 2024-05-09T05:26:25.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:25 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:25.511+0000 7fd1a65e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T05:26:25.816 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.762+0000 7f8588397700 1 Processor -- start 2024-05-09T05:26:25.816 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.762+0000 7f8588397700 1 -- start start 2024-05-09T05:26:25.816 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.763+0000 7f8588397700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f8580108d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:25.816 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.763+0000 7f8588397700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8580109360 con 0x7f8580108960 2024-05-09T05:26:25.816 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.763+0000 7f8586133700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f8580108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:25.816 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.763+0000 7f8586133700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f8580108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55170/0 (socket says 172.21.15.115:55170) 2024-05-09T05:26:25.816 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.763+0000 7f8586133700 1 -- 172.21.15.115:0/496311351 learned_addr learned my addr 172.21.15.115:0/496311351 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:25.816 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.763+0000 7f8586133700 1 -- 172.21.15.115:0/496311351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8580109b30 con 0x7f8580108960 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.763+0000 7f8586133700 1 --2- 172.21.15.115:0/496311351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f8580108d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f857c0098a0 tx=0x7f857c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92f2a238a35137e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.764+0000 7f8585131700 1 -- 172.21.15.115:0/496311351 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f857c004160 con 0x7f8580108960 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.764+0000 7f8585131700 1 -- 172.21.15.115:0/496311351 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f857c004300 con 0x7f8580108960 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.764+0000 7f8585131700 1 -- 172.21.15.115:0/496311351 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f857c011960 con 0x7f8580108960 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.765+0000 7f8588397700 1 -- 172.21.15.115:0/496311351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 msgr2=0x7f8580108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.765+0000 7f8588397700 1 --2- 172.21.15.115:0/496311351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f8580108d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f857c0098a0 tx=0x7f857c009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.765+0000 7f8588397700 1 -- 172.21.15.115:0/496311351 shutdown_connections 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.765+0000 7f8588397700 1 --2- 172.21.15.115:0/496311351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f8580108d70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.765+0000 7f8588397700 1 -- 172.21.15.115:0/496311351 >> 172.21.15.115:0/496311351 conn(0x7f8580103fe0 msgr2=0x7f8580106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.765+0000 7f8588397700 1 -- 172.21.15.115:0/496311351 shutdown_connections 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.766+0000 7f8588397700 1 -- 172.21.15.115:0/496311351 wait complete. 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.767+0000 7f8588397700 1 Processor -- start 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.767+0000 7f8588397700 1 -- start start 2024-05-09T05:26:25.817 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.767+0000 7f8588397700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f858007c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.767+0000 7f8588397700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f858010cee0 con 0x7f8580108960 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.767+0000 7f8586133700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f858007c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.767+0000 7f8586133700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f858007c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55178/0 (socket says 172.21.15.115:55178) 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.767+0000 7f8586133700 1 -- 172.21.15.115:0/4230003096 learned_addr learned my addr 172.21.15.115:0/4230003096 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.767+0000 7f8586133700 1 -- 172.21.15.115:0/4230003096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858007cee0 con 0x7f8580108960 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.768+0000 7f8586133700 1 --2- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f858007c9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f857c0119b0 tx=0x7f857c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.768+0000 7f85777fe700 1 -- 172.21.15.115:0/4230003096 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f857c003aa0 con 0x7f8580108960 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.768+0000 7f85777fe700 1 -- 172.21.15.115:0/4230003096 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f857c01b8d0 con 0x7f8580108960 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.768+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801a0b50 con 0x7f8580108960 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.768+0000 7f85777fe700 1 -- 172.21.15.115:0/4230003096 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f857c023860 con 0x7f8580108960 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.768+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801a1030 con 0x7f8580108960 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.769+0000 7f85777fe700 1 -- 172.21.15.115:0/4230003096 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f857c02c3f0 con 0x7f8580108960 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.769+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8580064e20 con 0x7f8580108960 2024-05-09T05:26:25.818 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.769+0000 7f85777fe700 1 --2- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f856c03e530 0x7f856c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.769+0000 7f85777fe700 1 -- 172.21.15.115:0/4230003096 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f857c054d00 con 0x7f8580108960 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.769+0000 7f8585932700 1 --2- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f856c03e530 0x7f856c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.770+0000 7f8585932700 1 --2- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f856c03e530 0x7f856c040a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8570006fd0 tx=0x7f8570006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:24.775+0000 7f85777fe700 1 -- 172.21.15.115:0/4230003096 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f857c01f030 con 0x7f8580108960 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.122+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f85801a13d0 con 0x7f8580108960 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.280+0000 7f85777fe700 1 -- 172.21.15.115:0/4230003096 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f857c00f2e0 con 0x7f8580108960 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.282+0000 7f85777fe700 1 -- 172.21.15.115:0/4230003096 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f857c01b490 con 0x7f8580108960 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.283+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f856c03e530 msgr2=0x7f856c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.283+0000 7f8588397700 1 --2- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f856c03e530 0x7f856c040a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8570006fd0 tx=0x7f8570006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.283+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 msgr2=0x7f858007c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.283+0000 7f8588397700 1 --2- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f858007c9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f857c0119b0 tx=0x7f857c011a90 comp rx=0 tx=0).stop 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.283+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 shutdown_connections 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.283+0000 7f8588397700 1 --2- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f856c03e530 0x7f856c040a40 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8570006fd0 tx=0x7f8570006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.283+0000 7f8588397700 1 --2- 172.21.15.115:0/4230003096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108960 0x7f858007c9a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:25.820 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.283+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 >> 172.21.15.115:0/4230003096 conn(0x7f8580103fe0 msgr2=0x7f85801048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:25.820 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.284+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 shutdown_connections 2024-05-09T05:26:25.820 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:25.284+0000 7f8588397700 1 -- 172.21.15.115:0/4230003096 wait complete. 2024-05-09T05:26:26.042 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:25 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:25.709+0000 7fd1a65e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:26:26.043 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:25 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:25.816+0000 7fd1a65e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T05:26:26.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:26 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4230003096' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T05:26:26.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:26 smithi115 ceph-mon[30349]: mgrmap e9: x(active, since 18s) 2024-05-09T05:26:26.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:26 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:26.138+0000 7fd1a65e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T05:26:26.399 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:26 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:26.230+0000 7fd1a65e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T05:26:27.107 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.261+0000 7fad0daf6700 1 Processor -- start 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.261+0000 7fad0daf6700 1 -- start start 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.261+0000 7fad0daf6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad0810a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.261+0000 7fad0daf6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810a940 con 0x7fad08109f40 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.262+0000 7fad077fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad0810a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.262+0000 7fad077fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad0810a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55200/0 (socket says 172.21.15.115:55200) 2024-05-09T05:26:27.108 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.262+0000 7fad077fe700 1 -- 172.21.15.115:0/4125278714 learned_addr learned my addr 172.21.15.115:0/4125278714 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.262+0000 7fad077fe700 1 -- 172.21.15.115:0/4125278714 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad08108bc0 con 0x7fad08109f40 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.262+0000 7fad077fe700 1 --2- 172.21.15.115:0/4125278714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad0810a350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7facfc0098a0 tx=0x7facfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3afa82853d61ab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.262+0000 7fad067fc700 1 -- 172.21.15.115:0/4125278714 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facfc004160 con 0x7fad08109f40 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.262+0000 7fad067fc700 1 -- 172.21.15.115:0/4125278714 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facfc004300 con 0x7fad08109f40 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.262+0000 7fad067fc700 1 -- 172.21.15.115:0/4125278714 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facfc011960 con 0x7fad08109f40 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.263+0000 7fad0daf6700 1 -- 172.21.15.115:0/4125278714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 msgr2=0x7fad0810a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.263+0000 7fad0daf6700 1 --2- 172.21.15.115:0/4125278714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad0810a350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7facfc0098a0 tx=0x7facfc009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.263+0000 7fad0daf6700 1 -- 172.21.15.115:0/4125278714 shutdown_connections 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.263+0000 7fad0daf6700 1 --2- 172.21.15.115:0/4125278714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad0810a350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.263+0000 7fad0daf6700 1 -- 172.21.15.115:0/4125278714 >> 172.21.15.115:0/4125278714 conn(0x7fad08103780 msgr2=0x7fad08105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.263+0000 7fad0daf6700 1 -- 172.21.15.115:0/4125278714 shutdown_connections 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.263+0000 7fad0daf6700 1 -- 172.21.15.115:0/4125278714 wait complete. 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.264+0000 7fad0daf6700 1 Processor -- start 2024-05-09T05:26:27.109 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.264+0000 7fad0daf6700 1 -- start start 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.264+0000 7fad0daf6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad08108450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.264+0000 7fad0daf6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810cf80 con 0x7fad08109f40 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad077fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad08108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad077fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad08108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55202/0 (socket says 172.21.15.115:55202) 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad077fe700 1 -- 172.21.15.115:0/1651454578 learned_addr learned my addr 172.21.15.115:0/1651454578 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad077fe700 1 -- 172.21.15.115:0/1651454578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad08108990 con 0x7fad08109f40 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad077fe700 1 --2- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad08108450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7facfc011980 tx=0x7facfc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad04ff9700 1 -- 172.21.15.115:0/1651454578 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facfc003a60 con 0x7fad08109f40 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad081a0b50 con 0x7fad08109f40 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad04ff9700 1 -- 172.21.15.115:0/1651454578 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facfc01b8d0 con 0x7fad08109f40 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad04ff9700 1 -- 172.21.15.115:0/1651454578 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facfc0237d0 con 0x7fad08109f40 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.265+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081a0fd0 con 0x7fad08109f40 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.266+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad08064e20 con 0x7fad08109f40 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.266+0000 7fad04ff9700 1 -- 172.21.15.115:0/1651454578 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7facfc023a80 con 0x7fad08109f40 2024-05-09T05:26:27.110 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.267+0000 7fad04ff9700 1 --2- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7facf003e530 0x7facf0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.267+0000 7fad04ff9700 1 -- 172.21.15.115:0/1651454578 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7facfc053f70 con 0x7fad08109f40 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.267+0000 7fad06ffd700 1 -- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7facf003e530 msgr2=0x7facf0040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/792641138 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.267+0000 7fad06ffd700 1 --2- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7facf003e530 0x7facf0040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.276+0000 7fad04ff9700 1 -- 172.21.15.115:0/1651454578 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7facfc01f030 con 0x7fad08109f40 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.468+0000 7fad06ffd700 1 -- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7facf003e530 msgr2=0x7facf0040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/792641138 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.468+0000 7fad06ffd700 1 --2- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7facf003e530 0x7facf0040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.547+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fad081a1820 con 0x7fad08109f40 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.548+0000 7fad04ff9700 1 -- 172.21.15.115:0/1651454578 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7facfc01bda0 con 0x7fad08109f40 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.550+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7facf003e530 msgr2=0x7facf0040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.550+0000 7fad0daf6700 1 --2- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7facf003e530 0x7facf0040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.550+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 msgr2=0x7fad08108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.550+0000 7fad0daf6700 1 --2- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad08108450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7facfc011980 tx=0x7facfc011a60 comp rx=0 tx=0).stop 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.550+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 shutdown_connections 2024-05-09T05:26:27.111 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.550+0000 7fad0daf6700 1 --2- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7facf003e530 0x7facf0040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:27.112 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.550+0000 7fad0daf6700 1 --2- 172.21.15.115:0/1651454578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad08109f40 0x7fad08108450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:27.112 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.550+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 >> 172.21.15.115:0/1651454578 conn(0x7fad08103780 msgr2=0x7fad08103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:27.112 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.551+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 shutdown_connections 2024-05-09T05:26:27.112 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:26.551+0000 7fad0daf6700 1 -- 172.21.15.115:0/1651454578 wait complete. 2024-05-09T05:26:27.112 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for the mgr to restart... 2024-05-09T05:26:27.112 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr epoch 9... 2024-05-09T05:26:27.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:27 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1651454578' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T05:26:27.747 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:27 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:27.461+0000 7fd1a65e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T05:26:27.748 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:27 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:27.643+0000 7fd1a65e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T05:26:28.107 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:27 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:27.816+0000 7fd1a65e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T05:26:28.108 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:28 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:28.112+0000 7fd1a65e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T05:26:28.397 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:28 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:28.207+0000 7fd1a65e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T05:26:28.897 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:28 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:28.412+0000 7fd1a65e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T05:26:28.898 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:28 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:28.609+0000 7fd1a65e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:26:29.647 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:29 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:29.210+0000 7fd1a65e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T05:26:29.648 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:29 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:29.438+0000 7fd1a65e1200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T05:26:30.147 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:29 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:29.871+0000 7fd1a65e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T05:26:30.148 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:29 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:29.971+0000 7fd1a65e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T05:26:30.647 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:30 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:30.344+0000 7fd1a65e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T05:26:31.043 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:30 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:30.706+0000 7fd1a65e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T05:26:31.043 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:30 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:30.801+0000 7fd1a65e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T05:26:31.397 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:31 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:31.047+0000 7fd1a65e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T05:26:31.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:31 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:31.183+0000 7fd1a65e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T05:26:31.398 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:31 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:31.282+0000 7fd1a65e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T05:26:31.897 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:31 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:31.612+0000 7fd1a65e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T05:26:31.897 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:31 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:31.707+0000 7fd1a65e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T05:26:32.594 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:32 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:32.321+0000 7fd1a65e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T05:26:32.595 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:26:32 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:26:32.545+0000 7fd1a65e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T05:26:32.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: Active manager daemon x restarted 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: Activating manager daemon x 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: mgrmap e10: x(active, starting, since 0.00459389s) 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: Manager daemon x is now available 2024-05-09T05:26:32.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:33.814 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T05:26:33.815 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T05:26:33.815 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:33 smithi115 ceph-mon[30349]: [09/May/2024:05:26:33] ENGINE Bus STARTING 2024-05-09T05:26:33.815 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:33 smithi115 ceph-mon[30349]: [09/May/2024:05:26:33] ENGINE Serving on http://172.21.15.115:8765 2024-05-09T05:26:33.815 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:33 smithi115 ceph-mon[30349]: [09/May/2024:05:26:33] ENGINE Serving on https://172.21.15.115:7150 2024-05-09T05:26:33.815 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:33 smithi115 ceph-mon[30349]: [09/May/2024:05:26:33] ENGINE Bus STARTED 2024-05-09T05:26:33.815 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:33 smithi115 ceph-mon[30349]: mgrmap e11: x(active, since 1.00669s) 2024-05-09T05:26:34.332 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-05-09T05:26:34.332 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.545+0000 7f4f9a5dd700 1 Processor -- start 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.545+0000 7f4f9a5dd700 1 -- start start 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.545+0000 7f4f9a5dd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f94109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.545+0000 7f4f9a5dd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9410a490 con 0x7f4f94109a90 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.546+0000 7f4f93fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f94109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.546+0000 7f4f93fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f94109ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55210/0 (socket says 172.21.15.115:55210) 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.546+0000 7f4f93fff700 1 -- 172.21.15.115:0/3334041221 learned_addr learned my addr 172.21.15.115:0/3334041221 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.546+0000 7f4f93fff700 1 -- 172.21.15.115:0/3334041221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9410ac60 con 0x7f4f94109a90 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.546+0000 7f4f93fff700 1 --2- 172.21.15.115:0/3334041221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f94109ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f840098a0 tx=0x7f4f84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec926f73516016b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.546+0000 7f4f937fe700 1 -- 172.21.15.115:0/3334041221 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f84004160 con 0x7f4f94109a90 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.546+0000 7f4f937fe700 1 -- 172.21.15.115:0/3334041221 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f84004300 con 0x7f4f94109a90 2024-05-09T05:26:34.333 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.546+0000 7f4f937fe700 1 -- 172.21.15.115:0/3334041221 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f84011960 con 0x7f4f94109a90 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.547+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/3334041221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 msgr2=0x7f4f94109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.547+0000 7f4f9a5dd700 1 --2- 172.21.15.115:0/3334041221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f94109ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f840098a0 tx=0x7f4f84009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.547+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/3334041221 shutdown_connections 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.547+0000 7f4f9a5dd700 1 --2- 172.21.15.115:0/3334041221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f94109ea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.547+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/3334041221 >> 172.21.15.115:0/3334041221 conn(0x7f4f941050b0 msgr2=0x7f4f94107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.547+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/3334041221 shutdown_connections 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.547+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/3334041221 wait complete. 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.548+0000 7f4f9a5dd700 1 Processor -- start 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.548+0000 7f4f9a5dd700 1 -- start start 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f9a5dd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f9419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f9a5dd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9410e010 con 0x7f4f94109a90 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f93fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f9419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f93fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f9419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55224/0 (socket says 172.21.15.115:55224) 2024-05-09T05:26:34.334 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f93fff700 1 -- 172.21.15.115:0/4015850126 learned_addr learned my addr 172.21.15.115:0/4015850126 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f93fff700 1 -- 172.21.15.115:0/4015850126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9419e370 con 0x7f4f94109a90 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f93fff700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f9419de30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f84006af0 tx=0x7f4f84011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f84003af0 con 0x7f4f94109a90 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f8401b8d0 con 0x7f4f94109a90 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/4015850126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9419e600 con 0x7f4f94109a90 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.549+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f84023820 con 0x7f4f94109a90 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.550+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/4015850126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f941a1210 con 0x7f4f94109a90 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.551+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f4f8402c3f0 con 0x7f4f94109a90 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.551+0000 7f4f91ffb700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 0x7f4f80040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.551+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 --> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f80041200 con 0x7f4f8003e5e0 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.551+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4f84053f40 con 0x7f4f94109a90 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.551+0000 7f4f8bfff700 1 -- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 msgr2=0x7f4f80040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/792641138 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.551+0000 7f4f8bfff700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 0x7f4f80040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:26:34.335 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.752+0000 7f4f8bfff700 1 -- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 msgr2=0x7f4f80040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/792641138 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:27.752+0000 7f4f8bfff700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 0x7f4f80040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:28.153+0000 7f4f8bfff700 1 -- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 msgr2=0x7f4f80040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/792641138 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:28.153+0000 7f4f8bfff700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 0x7f4f80040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:28.954+0000 7f4f8bfff700 1 -- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 msgr2=0x7f4f80040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/792641138 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:28.954+0000 7f4f8bfff700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 0x7f4f80040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:30.555+0000 7f4f8bfff700 1 -- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 msgr2=0x7f4f80040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/792641138 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:30.555+0000 7f4f8bfff700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 0x7f4f80040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:32.549+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f4f84016bd0 con 0x7f4f94109a90 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:32.549+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 msgr2=0x7f4f80040af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:32.549+0000 7f4f91ffb700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 0x7f4f80040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.550+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f4f84055b10 con 0x7f4f94109a90 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.551+0000 7f4f91ffb700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f4f84053ad0 0x7f4f80043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.551+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f84009530 con 0x7f4f84053ad0 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.551+0000 7f4f8bfff700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f4f84053ad0 0x7f4f80043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:34.336 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.551+0000 7f4f8bfff700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f4f84053ad0 0x7f4f80043fd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c003a10 tx=0x7f4f7c009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.552+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4f84009530 con 0x7f4f84053ad0 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.560+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/4015850126 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4f94064e20 con 0x7f4f84053ad0 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f91ffb700 1 -- 172.21.15.115:0/4015850126 <== mgr.14150 v2:172.21.15.115:6800/2041699318 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f4f94064e20 con 0x7f4f84053ad0 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f4f84053ad0 msgr2=0x7f4f80043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f4f84053ad0 0x7f4f80043fd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c003a10 tx=0x7f4f7c009270 comp rx=0 tx=0).stop 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 msgr2=0x7f4f9419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f9419de30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f84006af0 tx=0x7f4f84011a30 comp rx=0 tx=0).stop 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/4015850126 shutdown_connections 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f4f84053ad0 0x7f4f80043fd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f94109a90 0x7f4f9419de30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 --2- 172.21.15.115:0/4015850126 >> [v2:172.21.15.115:6800/792641138,v1:172.21.15.115:6801/792641138] conn(0x7f4f8003e5e0 0x7f4f80040af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/4015850126 >> 172.21.15.115:0/4015850126 conn(0x7f4f941050b0 msgr2=0x7f4f941059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/4015850126 shutdown_connections 2024-05-09T05:26:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:33.561+0000 7f4f9a5dd700 1 -- 172.21.15.115:0/4015850126 wait complete. 2024-05-09T05:26:34.338 INFO:teuthology.orchestra.run.smithi115.stdout:mgr epoch 9 is available 2024-05-09T05:26:34.338 INFO:teuthology.orchestra.run.smithi115.stdout:Generating a dashboard self-signed certificate... 2024-05-09T05:26:34.725 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:34 smithi115 ceph-mon[30349]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T05:26:34.726 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:34 smithi115 ceph-mon[30349]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T05:26:35.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:35 smithi115 ceph-mon[30349]: mgrmap e12: x(active, since 2s) 2024-05-09T05:26:35.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:35 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:35.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:35 smithi115 ceph-mon[30349]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:35.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:35 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:35.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:35 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:35.937 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T05:26:35.937 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.784+0000 7f67388d1700 1 Processor -- start 2024-05-09T05:26:35.937 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.784+0000 7f67388d1700 1 -- start start 2024-05-09T05:26:35.937 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.784+0000 7f67388d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f6734108d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:35.937 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.785+0000 7f67388d1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6734109360 con 0x7f6734108960 2024-05-09T05:26:35.937 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.785+0000 7f673259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f6734108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.785+0000 7f673259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f6734108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40220/0 (socket says 172.21.15.115:40220) 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.785+0000 7f673259c700 1 -- 172.21.15.115:0/2613803019 learned_addr learned my addr 172.21.15.115:0/2613803019 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.785+0000 7f673259c700 1 -- 172.21.15.115:0/2613803019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6734109b30 con 0x7f6734108960 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.785+0000 7f673259c700 1 --2- 172.21.15.115:0/2613803019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f6734108d70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67280098a0 tx=0x7f6728009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344dd49472482960 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.785+0000 7f673159a700 1 -- 172.21.15.115:0/2613803019 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6728004160 con 0x7f6734108960 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.786+0000 7f673159a700 1 -- 172.21.15.115:0/2613803019 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6728004300 con 0x7f6734108960 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.786+0000 7f673159a700 1 -- 172.21.15.115:0/2613803019 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6728011960 con 0x7f6734108960 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.786+0000 7f67388d1700 1 -- 172.21.15.115:0/2613803019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 msgr2=0x7f6734108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.786+0000 7f67388d1700 1 --2- 172.21.15.115:0/2613803019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f6734108d70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67280098a0 tx=0x7f6728009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.786+0000 7f67388d1700 1 -- 172.21.15.115:0/2613803019 shutdown_connections 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.786+0000 7f67388d1700 1 --2- 172.21.15.115:0/2613803019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f6734108d70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.787+0000 7f67388d1700 1 -- 172.21.15.115:0/2613803019 >> 172.21.15.115:0/2613803019 conn(0x7f6734103fe0 msgr2=0x7f6734106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.787+0000 7f67388d1700 1 -- 172.21.15.115:0/2613803019 shutdown_connections 2024-05-09T05:26:35.938 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.787+0000 7f67388d1700 1 -- 172.21.15.115:0/2613803019 wait complete. 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.787+0000 7f67388d1700 1 Processor -- start 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.787+0000 7f67388d1700 1 -- start start 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f67388d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f67341a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f67388d1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f673410cee0 con 0x7f6734108960 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f673259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f67341a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f673259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f67341a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40236/0 (socket says 172.21.15.115:40236) 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f673259c700 1 -- 172.21.15.115:0/1653116452 learned_addr learned my addr 172.21.15.115:0/1653116452 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f673259c700 1 -- 172.21.15.115:0/1653116452 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67341a0550 con 0x7f6734108960 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f673259c700 1 --2- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f67341a0010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67280119b0 tx=0x7f6728011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f67237fe700 1 -- 172.21.15.115:0/1653116452 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6728003aa0 con 0x7f6734108960 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67341a07e0 con 0x7f6734108960 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f67237fe700 1 -- 172.21.15.115:0/1653116452 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f672801b8d0 con 0x7f6734108960 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f67237fe700 1 -- 172.21.15.115:0/1653116452 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6728023860 con 0x7f6734108960 2024-05-09T05:26:35.939 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.788+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67341a33f0 con 0x7f6734108960 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.789+0000 7f67237fe700 1 -- 172.21.15.115:0/1653116452 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f672802c3f0 con 0x7f6734108960 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.790+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6734064e20 con 0x7f6734108960 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.790+0000 7f67237fe700 1 --2- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f671c03e530 0x7f671c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.790+0000 7f67237fe700 1 -- 172.21.15.115:0/1653116452 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6728053fb0 con 0x7f6734108960 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.790+0000 7f6731d9b700 1 --2- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f671c03e530 0x7f671c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.790+0000 7f6731d9b700 1 --2- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f671c03e530 0x7f671c040a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6724006fd0 tx=0x7f6724006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:34.800+0000 7f67237fe700 1 -- 172.21.15.115:0/1653116452 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f672801f030 con 0x7f6734108960 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.015+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6734106380 con 0x7f671c03e530 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.381+0000 7f67237fe700 1 -- 172.21.15.115:0/1653116452 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6734106380 con 0x7f671c03e530 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f671c03e530 msgr2=0x7f671c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 --2- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f671c03e530 0x7f671c040a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6724006fd0 tx=0x7f6724006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 msgr2=0x7f67341a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 --2- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f67341a0010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67280119b0 tx=0x7f6728011a90 comp rx=0 tx=0).stop 2024-05-09T05:26:35.940 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 shutdown_connections 2024-05-09T05:26:35.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 --2- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f671c03e530 0x7f671c040a40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:35.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 --2- 172.21.15.115:0/1653116452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6734108960 0x7f67341a0010 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:35.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 >> 172.21.15.115:0/1653116452 conn(0x7f6734103fe0 msgr2=0x7f6734104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:35.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 shutdown_connections 2024-05-09T05:26:35.941 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:35.384+0000 7f67388d1700 1 -- 172.21.15.115:0/1653116452 wait complete. 2024-05-09T05:26:35.941 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial admin user... 2024-05-09T05:26:37.268 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$4cveBVDkG1UtAL9s0ejIu.y.BEqZ/TFA2M5owYRGTRH53FD9GaTYq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715232396, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.293+0000 7f670f24e700 1 Processor -- start 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.293+0000 7f670f24e700 1 -- start start 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.293+0000 7f670f24e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f6708109e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.293+0000 7f670f24e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f670810a440 con 0x7f6708109a40 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.293+0000 7f670cfea700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f6708109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.293+0000 7f670cfea700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f6708109e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40240/0 (socket says 172.21.15.115:40240) 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.293+0000 7f670cfea700 1 -- 172.21.15.115:0/4291754000 learned_addr learned my addr 172.21.15.115:0/4291754000 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.294+0000 7f670cfea700 1 -- 172.21.15.115:0/4291754000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670810ac10 con 0x7f6708109a40 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.294+0000 7f670cfea700 1 --2- 172.21.15.115:0/4291754000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f6708109e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098a0 tx=0x7f66fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96665be05a9cd636 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.294+0000 7f67077fe700 1 -- 172.21.15.115:0/4291754000 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66fc004160 con 0x7f6708109a40 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.294+0000 7f67077fe700 1 -- 172.21.15.115:0/4291754000 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66fc004300 con 0x7f6708109a40 2024-05-09T05:26:37.269 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.294+0000 7f67077fe700 1 -- 172.21.15.115:0/4291754000 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66fc011960 con 0x7f6708109a40 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.295+0000 7f670f24e700 1 -- 172.21.15.115:0/4291754000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 msgr2=0x7f6708109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.295+0000 7f670f24e700 1 --2- 172.21.15.115:0/4291754000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f6708109e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098a0 tx=0x7f66fc009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.295+0000 7f670f24e700 1 -- 172.21.15.115:0/4291754000 shutdown_connections 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.295+0000 7f670f24e700 1 --2- 172.21.15.115:0/4291754000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f6708109e50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.295+0000 7f670f24e700 1 -- 172.21.15.115:0/4291754000 >> 172.21.15.115:0/4291754000 conn(0x7f67081050c0 msgr2=0x7f6708107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.295+0000 7f670f24e700 1 -- 172.21.15.115:0/4291754000 shutdown_connections 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.295+0000 7f670f24e700 1 -- 172.21.15.115:0/4291754000 wait complete. 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.296+0000 7f670f24e700 1 Processor -- start 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.296+0000 7f670f24e700 1 -- start start 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.296+0000 7f670f24e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f670819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.296+0000 7f670f24e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f670810dfc0 con 0x7f6708109a40 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.296+0000 7f670cfea700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f670819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.296+0000 7f670cfea700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f670819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40250/0 (socket says 172.21.15.115:40250) 2024-05-09T05:26:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.296+0000 7f670cfea700 1 -- 172.21.15.115:0/1326592564 learned_addr learned my addr 172.21.15.115:0/1326592564 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.296+0000 7f670cfea700 1 -- 172.21.15.115:0/1326592564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670819e320 con 0x7f6708109a40 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.296+0000 7f670cfea700 1 --2- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f670819dde0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0119b0 tx=0x7f66fc011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.297+0000 7f6705ffb700 1 -- 172.21.15.115:0/1326592564 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66fc003aa0 con 0x7f6708109a40 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.297+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670819e5b0 con 0x7f6708109a40 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.297+0000 7f6705ffb700 1 -- 172.21.15.115:0/1326592564 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66fc01b8d0 con 0x7f6708109a40 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.297+0000 7f6705ffb700 1 -- 172.21.15.115:0/1326592564 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66fc023860 con 0x7f6708109a40 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.297+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67081a11c0 con 0x7f6708109a40 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.298+0000 7f6705ffb700 1 -- 172.21.15.115:0/1326592564 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f66fc02c3f0 con 0x7f6708109a40 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.298+0000 7f6705ffb700 1 --2- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f66f803e530 0x7f66f8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.298+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6708064e20 con 0x7f6708109a40 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.298+0000 7f6705ffb700 1 -- 172.21.15.115:0/1326592564 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f66fc054e40 con 0x7f6708109a40 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.298+0000 7f6707fff700 1 --2- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f66f803e530 0x7f66f8040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.299+0000 7f6707fff700 1 --2- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f66f803e530 0x7f66f8040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006fd0 tx=0x7f66f4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:37.271 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.304+0000 7f6705ffb700 1 -- 172.21.15.115:0/1326592564 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66fc01f030 con 0x7f6708109a40 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.529+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f67081a1450 con 0x7f66f803e530 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.753+0000 7f6705ffb700 1 -- 172.21.15.115:0/1326592564 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f67081a1450 con 0x7f66f803e530 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f66f803e530 msgr2=0x7f66f8040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 --2- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f66f803e530 0x7f66f8040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006fd0 tx=0x7f66f4006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 msgr2=0x7f670819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 --2- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f670819dde0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0119b0 tx=0x7f66fc011a90 comp rx=0 tx=0).stop 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 shutdown_connections 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 --2- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f66f803e530 0x7f66f8040a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 --2- 172.21.15.115:0/1326592564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6708109a40 0x7f670819dde0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 >> 172.21.15.115:0/1326592564 conn(0x7f67081050c0 msgr2=0x7f6708105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 shutdown_connections 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:36.756+0000 7f670f24e700 1 -- 172.21.15.115:0/1326592564 wait complete. 2024-05-09T05:26:37.272 INFO:teuthology.orchestra.run.smithi115.stdout:Fetching dashboard port number... 2024-05-09T05:26:37.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:37 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:37.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:37 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:38.502 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T05:26:38.502 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.700+0000 7f876a5bb700 1 Processor -- start 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.700+0000 7f876a5bb700 1 -- start start 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.700+0000 7f876a5bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f8764108d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.700+0000 7f876a5bb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8764109360 con 0x7f8764108960 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.701+0000 7f8763fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f8764108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.701+0000 7f8763fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f8764108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40256/0 (socket says 172.21.15.115:40256) 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.701+0000 7f8763fff700 1 -- 172.21.15.115:0/936860325 learned_addr learned my addr 172.21.15.115:0/936860325 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.701+0000 7f8763fff700 1 -- 172.21.15.115:0/936860325 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8764109b30 con 0x7f8764108960 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.701+0000 7f8763fff700 1 --2- 172.21.15.115:0/936860325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f8764108d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87540098a0 tx=0x7f8754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5c5aaddf1bff50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.701+0000 7f8762ffd700 1 -- 172.21.15.115:0/936860325 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8754004160 con 0x7f8764108960 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.701+0000 7f8762ffd700 1 -- 172.21.15.115:0/936860325 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8754004300 con 0x7f8764108960 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.701+0000 7f8762ffd700 1 -- 172.21.15.115:0/936860325 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8754011960 con 0x7f8764108960 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.702+0000 7f876a5bb700 1 -- 172.21.15.115:0/936860325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 msgr2=0x7f8764108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.702+0000 7f876a5bb700 1 --2- 172.21.15.115:0/936860325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f8764108d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87540098a0 tx=0x7f8754009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.702+0000 7f876a5bb700 1 -- 172.21.15.115:0/936860325 shutdown_connections 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.702+0000 7f876a5bb700 1 --2- 172.21.15.115:0/936860325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f8764108d70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.702+0000 7f876a5bb700 1 -- 172.21.15.115:0/936860325 >> 172.21.15.115:0/936860325 conn(0x7f8764103fe0 msgr2=0x7f8764106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.702+0000 7f876a5bb700 1 -- 172.21.15.115:0/936860325 shutdown_connections 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.702+0000 7f876a5bb700 1 -- 172.21.15.115:0/936860325 wait complete. 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.703+0000 7f876a5bb700 1 Processor -- start 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.703+0000 7f876a5bb700 1 -- start start 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f876a5bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f876407c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f876a5bb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f876410cee0 con 0x7f8764108960 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f8763fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f876407c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f8763fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f876407c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40264/0 (socket says 172.21.15.115:40264) 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f8763fff700 1 -- 172.21.15.115:0/311157577 learned_addr learned my addr 172.21.15.115:0/311157577 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:38.504 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f8763fff700 1 -- 172.21.15.115:0/311157577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876407ce80 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f8763fff700 1 --2- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f876407c940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87540119b0 tx=0x7f8754011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f87617fa700 1 -- 172.21.15.115:0/311157577 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8754003aa0 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f87617fa700 1 -- 172.21.15.115:0/311157577 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f875401b8d0 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87641a0bb0 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.704+0000 7f87617fa700 1 -- 172.21.15.115:0/311157577 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8754023860 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.705+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87641a1030 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.705+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8764064e20 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.706+0000 7f87617fa700 1 -- 172.21.15.115:0/311157577 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f875402c3f0 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.706+0000 7f87617fa700 1 --2- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f874c03e590 0x7f874c040aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.706+0000 7f87617fa700 1 -- 172.21.15.115:0/311157577 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8754054250 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.706+0000 7f87637fe700 1 --2- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f874c03e590 0x7f874c040aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.707+0000 7f87637fe700 1 --2- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f874c03e590 0x7f874c040aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8758006fd0 tx=0x7f8758006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.711+0000 7f87617fa700 1 -- 172.21.15.115:0/311157577 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f875401f030 con 0x7f8764108960 2024-05-09T05:26:38.505 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.929+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f87641a18e0 con 0x7f8764108960 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.930+0000 7f87617fa700 1 -- 172.21.15.115:0/311157577 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f875400f2e0 con 0x7f8764108960 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f874c03e590 msgr2=0x7f874c040aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 --2- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f874c03e590 0x7f874c040aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8758006fd0 tx=0x7f8758006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 msgr2=0x7f876407c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 --2- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f876407c940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87540119b0 tx=0x7f8754011a90 comp rx=0 tx=0).stop 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 shutdown_connections 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 --2- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f874c03e590 0x7f874c040aa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 --2- 172.21.15.115:0/311157577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8764108960 0x7f876407c940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 >> 172.21.15.115:0/311157577 conn(0x7f8764103fe0 msgr2=0x7f87641048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 shutdown_connections 2024-05-09T05:26:38.506 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:37.932+0000 7f876a5bb700 1 -- 172.21.15.115:0/311157577 wait complete. 2024-05-09T05:26:38.509 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T05:26:38.509 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-05-09T05:26:38.517 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T05:26:38.517 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-05-09T05:26:38.517 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-05-09T05:26:38.517 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T05:26:38.519 INFO:teuthology.orchestra.run.smithi115.stdout:Ceph Dashboard is now available at: 2024-05-09T05:26:38.519 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:38.519 INFO:teuthology.orchestra.run.smithi115.stdout: URL: https://smithi115.front.sepia.ceph.com:8443/ 2024-05-09T05:26:38.519 INFO:teuthology.orchestra.run.smithi115.stdout: User: admin 2024-05-09T05:26:38.519 INFO:teuthology.orchestra.run.smithi115.stdout: Password: 29pmiwbfmg 2024-05-09T05:26:38.519 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:38.519 INFO:teuthology.orchestra.run.smithi115.stdout:Saving cluster configuration to /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config directory 2024-05-09T05:26:38.520 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling autotune for osd_memory_target 2024-05-09T05:26:38.598 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:38 smithi115 ceph-mon[30349]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:38.598 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:38 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/311157577' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T05:26:40.163 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.302+0000 7f2c49c03700 1 Processor -- start 2024-05-09T05:26:40.163 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.302+0000 7f2c49c03700 1 -- start start 2024-05-09T05:26:40.163 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.302+0000 7f2c49c03700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4410a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:40.163 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.302+0000 7f2c49c03700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410a9d0 con 0x7f2c44109ff0 2024-05-09T05:26:40.163 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.302+0000 7f2c437fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4410a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:40.163 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.302+0000 7f2c437fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4410a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40274/0 (socket says 172.21.15.115:40274) 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.303+0000 7f2c437fe700 1 -- 172.21.15.115:0/2006316078 learned_addr learned my addr 172.21.15.115:0/2006316078 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.303+0000 7f2c437fe700 1 -- 172.21.15.115:0/2006316078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4410ab50 con 0x7f2c44109ff0 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.303+0000 7f2c437fe700 1 --2- 172.21.15.115:0/2006316078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4410a400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098a0 tx=0x7f2c34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e31f4646a80c53c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.303+0000 7f2c427fc700 1 -- 172.21.15.115:0/2006316078 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c34004160 con 0x7f2c44109ff0 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.303+0000 7f2c427fc700 1 -- 172.21.15.115:0/2006316078 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c34004300 con 0x7f2c44109ff0 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.303+0000 7f2c427fc700 1 -- 172.21.15.115:0/2006316078 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c34011960 con 0x7f2c44109ff0 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.304+0000 7f2c49c03700 1 -- 172.21.15.115:0/2006316078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 msgr2=0x7f2c4410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.304+0000 7f2c49c03700 1 --2- 172.21.15.115:0/2006316078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4410a400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098a0 tx=0x7f2c34009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.304+0000 7f2c49c03700 1 -- 172.21.15.115:0/2006316078 shutdown_connections 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.304+0000 7f2c49c03700 1 --2- 172.21.15.115:0/2006316078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4410a400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.304+0000 7f2c49c03700 1 -- 172.21.15.115:0/2006316078 >> 172.21.15.115:0/2006316078 conn(0x7f2c44103780 msgr2=0x7f2c44105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.304+0000 7f2c49c03700 1 -- 172.21.15.115:0/2006316078 shutdown_connections 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.304+0000 7f2c49c03700 1 -- 172.21.15.115:0/2006316078 wait complete. 2024-05-09T05:26:40.164 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.305+0000 7f2c49c03700 1 Processor -- start 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.305+0000 7f2c49c03700 1 -- start start 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.305+0000 7f2c49c03700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4419ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.305+0000 7f2c49c03700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44110080 con 0x7f2c44109ff0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.305+0000 7f2c437fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4419ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.305+0000 7f2c437fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4419ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40286/0 (socket says 172.21.15.115:40286) 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.305+0000 7f2c437fe700 1 -- 172.21.15.115:0/4217208866 learned_addr learned my addr 172.21.15.115:0/4217208866 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.306+0000 7f2c437fe700 1 -- 172.21.15.115:0/4217208866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c441a04c0 con 0x7f2c44109ff0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.306+0000 7f2c437fe700 1 --2- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4419ff80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2c34011670 tx=0x7f2c34011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.306+0000 7f2c40ff9700 1 -- 172.21.15.115:0/4217208866 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c34003a60 con 0x7f2c44109ff0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.306+0000 7f2c40ff9700 1 -- 172.21.15.115:0/4217208866 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c3401b8d0 con 0x7f2c44109ff0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.306+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c441a0750 con 0x7f2c44109ff0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.306+0000 7f2c40ff9700 1 -- 172.21.15.115:0/4217208866 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c34023790 con 0x7f2c44109ff0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.306+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c441a3360 con 0x7f2c44109ff0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.307+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c44064e20 con 0x7f2c44109ff0 2024-05-09T05:26:40.165 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.308+0000 7f2c40ff9700 1 -- 172.21.15.115:0/4217208866 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f2c34023a40 con 0x7f2c44109ff0 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.308+0000 7f2c40ff9700 1 --2- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c2c03e5e0 0x7f2c2c040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.308+0000 7f2c40ff9700 1 -- 172.21.15.115:0/4217208866 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2c34054040 con 0x7f2c44109ff0 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.308+0000 7f2c42ffd700 1 --2- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c2c03e5e0 0x7f2c2c040af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.308+0000 7f2c42ffd700 1 --2- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c2c03e5e0 0x7f2c2c040af0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c38006fd0 tx=0x7f2c38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.313+0000 7f2c40ff9700 1 -- 172.21.15.115:0/4217208866 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c3401f030 con 0x7f2c44109ff0 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.349+0000 7f2c40ff9700 1 -- 172.21.15.115:0/4217208866 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f2c3402c910 con 0x7f2c44109ff0 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.518+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f2c441a3670 con 0x7f2c44109ff0 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.518+0000 7f2c40ff9700 1 -- 172.21.15.115:0/4217208866 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f2c34056540 con 0x7f2c44109ff0 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.520+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c2c03e5e0 msgr2=0x7f2c2c040af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.520+0000 7f2c49c03700 1 --2- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c2c03e5e0 0x7f2c2c040af0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c38006fd0 tx=0x7f2c38006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.520+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 msgr2=0x7f2c4419ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.520+0000 7f2c49c03700 1 --2- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4419ff80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2c34011670 tx=0x7f2c34011a30 comp rx=0 tx=0).stop 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.520+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 shutdown_connections 2024-05-09T05:26:40.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.520+0000 7f2c49c03700 1 --2- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c2c03e5e0 0x7f2c2c040af0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:40.167 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.520+0000 7f2c49c03700 1 --2- 172.21.15.115:0/4217208866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c44109ff0 0x7f2c4419ff80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:40.167 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.521+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 >> 172.21.15.115:0/4217208866 conn(0x7f2c44103780 msgr2=0x7f2c44104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:40.167 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.521+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 shutdown_connections 2024-05-09T05:26:40.167 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:39.521+0000 7f2c49c03700 1 -- 172.21.15.115:0/4217208866 wait complete. 2024-05-09T05:26:40.632 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:40 smithi115 ceph-mon[30349]: mgrmap e13: x(active, since 6s) 2024-05-09T05:26:41.502 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.637+0000 7efe951ad700 1 Processor -- start 2024-05-09T05:26:41.502 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.637+0000 7efe951ad700 1 -- start start 2024-05-09T05:26:41.502 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.637+0000 7efe951ad700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe90108d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:41.502 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.637+0000 7efe951ad700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe90109360 con 0x7efe90108960 2024-05-09T05:26:41.502 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.638+0000 7efe8ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe90108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:41.502 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.638+0000 7efe8ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe90108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40294/0 (socket says 172.21.15.115:40294) 2024-05-09T05:26:41.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.638+0000 7efe8ed9d700 1 -- 172.21.15.115:0/1414624888 learned_addr learned my addr 172.21.15.115:0/1414624888 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:41.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.638+0000 7efe8ed9d700 1 -- 172.21.15.115:0/1414624888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe90109b30 con 0x7efe90108960 2024-05-09T05:26:41.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.638+0000 7efe8ed9d700 1 --2- 172.21.15.115:0/1414624888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe90108d70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe800098a0 tx=0x7efe80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573b55e8141ded23 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:41.503 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.638+0000 7efe8dd9b700 1 -- 172.21.15.115:0/1414624888 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe80004160 con 0x7efe90108960 2024-05-09T05:26:41.507 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.638+0000 7efe8dd9b700 1 -- 172.21.15.115:0/1414624888 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe80004300 con 0x7efe90108960 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.638+0000 7efe8dd9b700 1 -- 172.21.15.115:0/1414624888 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe80011960 con 0x7efe90108960 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.639+0000 7efe951ad700 1 -- 172.21.15.115:0/1414624888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 msgr2=0x7efe90108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.639+0000 7efe951ad700 1 --2- 172.21.15.115:0/1414624888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe90108d70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe800098a0 tx=0x7efe80009bb0 comp rx=0 tx=0).stop 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.639+0000 7efe951ad700 1 -- 172.21.15.115:0/1414624888 shutdown_connections 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.639+0000 7efe951ad700 1 --2- 172.21.15.115:0/1414624888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe90108d70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.639+0000 7efe951ad700 1 -- 172.21.15.115:0/1414624888 >> 172.21.15.115:0/1414624888 conn(0x7efe90103fe0 msgr2=0x7efe90106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.639+0000 7efe951ad700 1 -- 172.21.15.115:0/1414624888 shutdown_connections 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.639+0000 7efe951ad700 1 -- 172.21.15.115:0/1414624888 wait complete. 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.640+0000 7efe951ad700 1 Processor -- start 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.640+0000 7efe951ad700 1 -- start start 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.640+0000 7efe951ad700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe9019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.640+0000 7efe951ad700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe9010cee0 con 0x7efe90108960 2024-05-09T05:26:41.508 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.640+0000 7efe8ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe9019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.640+0000 7efe8ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe9019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40304/0 (socket says 172.21.15.115:40304) 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.640+0000 7efe8ed9d700 1 -- 172.21.15.115:0/3637450079 learned_addr learned my addr 172.21.15.115:0/3637450079 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.641+0000 7efe8ed9d700 1 -- 172.21.15.115:0/3637450079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9019d240 con 0x7efe90108960 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.641+0000 7efe8ed9d700 1 --2- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe9019cd00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe80006af0 tx=0x7efe80011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.641+0000 7efe77fff700 1 -- 172.21.15.115:0/3637450079 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe80003af0 con 0x7efe90108960 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.641+0000 7efe77fff700 1 -- 172.21.15.115:0/3637450079 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe8001b8d0 con 0x7efe90108960 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.641+0000 7efe77fff700 1 -- 172.21.15.115:0/3637450079 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe80023820 con 0x7efe90108960 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.641+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9019d4d0 con 0x7efe90108960 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.641+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe901a00e0 con 0x7efe90108960 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.642+0000 7efe77fff700 1 -- 172.21.15.115:0/3637450079 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7efe8002c3f0 con 0x7efe90108960 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.642+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe90064e20 con 0x7efe90108960 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.642+0000 7efe77fff700 1 --2- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efe7803e620 0x7efe78040b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.642+0000 7efe77fff700 1 -- 172.21.15.115:0/3637450079 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efe80054160 con 0x7efe90108960 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.642+0000 7efe8e59c700 1 --2- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efe7803e620 0x7efe78040b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:41.509 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.643+0000 7efe8e59c700 1 --2- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efe7803e620 0x7efe78040b30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe84006fd0 tx=0x7efe84006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.647+0000 7efe77fff700 1 -- 172.21.15.115:0/3637450079 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe8001f030 con 0x7efe90108960 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.965+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7efe901a02b0 con 0x7efe90108960 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.968+0000 7efe77fff700 1 -- 172.21.15.115:0/3637450079 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7efe80010380 con 0x7efe90108960 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efe7803e620 msgr2=0x7efe78040b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 --2- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efe7803e620 0x7efe78040b30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe84006fd0 tx=0x7efe84006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 msgr2=0x7efe9019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 --2- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe9019cd00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe80006af0 tx=0x7efe80011a30 comp rx=0 tx=0).stop 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 shutdown_connections 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 --2- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efe7803e620 0x7efe78040b30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 --2- 172.21.15.115:0/3637450079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe90108960 0x7efe9019cd00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 >> 172.21.15.115:0/3637450079 conn(0x7efe90103fe0 msgr2=0x7efe901048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 shutdown_connections 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr 2024-05-09T05:26:40.970+0000 7efe951ad700 1 -- 172.21.15.115:0/3637450079 wait complete. 2024-05-09T05:26:41.510 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: ceph telemetry on 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout:For more information see: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T05:26:41.511 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:41.512 INFO:teuthology.orchestra.run.smithi115.stdout:Bootstrap complete. 2024-05-09T05:26:41.540 INFO:tasks.cephadm:Fetching config... 2024-05-09T05:26:41.540 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:26:41.541 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T05:26:41.569 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T05:26:41.569 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:26:41.569 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T05:26:41.634 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T05:26:41.635 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:26:41.635 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T05:26:41.708 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T05:26:41.708 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:26:41.708 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T05:26:41.768 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T05:26:41.769 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoFfGMwA/8lkOGyOqFfL0OlSV1YErc2apcgLPrNr5h6cphil3iusu3zA/+9ytig4q7j+qOK1ZZzDGmRPuAmjLHzkjrNSNeNUIJ0SFjltjLPG+tyGY9eqpvLql42vGBRMs1Gc6eAPEJCunIyjboHPHQKlgptJBCFNs2Jx65VN3Q2hmPfR3CFTLpgw7keJb/XsTUd25YVFTo1THovuprHGQc4xz7A+6sAg/hvgLybz5Q7mqUFP3Unwp4oNne2Z5fF3ElvKHTSvoudZOE9lisdvljprdly5Y0fN98266iWfbprPWmAvvYlf26m4l04jsgZGIhuJAviuU4QtBq9Mr91kHMLLRG2yhDEduPddb5x2ofnXzovF4fg4uO6HpUieIzCN0bfmu52Vupu5oPE7fjbZehrvJCXYRWq0Wjev31x0ivBTmo1sh4wmq4Fvh6/N0j3EErPPWNss+eNgQIz3UPpD1kPXDj7nufY8il24QUXfQgpJTvBvXG4LQXRxpTWIe70RU= ceph-69de8a38-0dc4-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T05:26:41.846 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoFfGMwA/8lkOGyOqFfL0OlSV1YErc2apcgLPrNr5h6cphil3iusu3zA/+9ytig4q7j+qOK1ZZzDGmRPuAmjLHzkjrNSNeNUIJ0SFjltjLPG+tyGY9eqpvLql42vGBRMs1Gc6eAPEJCunIyjboHPHQKlgptJBCFNs2Jx65VN3Q2hmPfR3CFTLpgw7keJb/XsTUd25YVFTo1THovuprHGQc4xz7A+6sAg/hvgLybz5Q7mqUFP3Unwp4oNne2Z5fF3ElvKHTSvoudZOE9lisdvljprdly5Y0fN98266iWfbprPWmAvvYlf26m4l04jsgZGIhuJAviuU4QtBq9Mr91kHMLLRG2yhDEduPddb5x2ofnXzovF4fg4uO6HpUieIzCN0bfmu52Vupu5oPE7fjbZehrvJCXYRWq0Wjev31x0ivBTmo1sh4wmq4Fvh6/N0j3EErPPWNss+eNgQIz3UPpD1kPXDj7nufY8il24QUXfQgpJTvBvXG4LQXRxpTWIe70RU= ceph-69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:26:41.858 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoFfGMwA/8lkOGyOqFfL0OlSV1YErc2apcgLPrNr5h6cphil3iusu3zA/+9ytig4q7j+qOK1ZZzDGmRPuAmjLHzkjrNSNeNUIJ0SFjltjLPG+tyGY9eqpvLql42vGBRMs1Gc6eAPEJCunIyjboHPHQKlgptJBCFNs2Jx65VN3Q2hmPfR3CFTLpgw7keJb/XsTUd25YVFTo1THovuprHGQc4xz7A+6sAg/hvgLybz5Q7mqUFP3Unwp4oNne2Z5fF3ElvKHTSvoudZOE9lisdvljprdly5Y0fN98266iWfbprPWmAvvYlf26m4l04jsgZGIhuJAviuU4QtBq9Mr91kHMLLRG2yhDEduPddb5x2ofnXzovF4fg4uO6HpUieIzCN0bfmu52Vupu5oPE7fjbZehrvJCXYRWq0Wjev31x0ivBTmo1sh4wmq4Fvh6/N0j3EErPPWNss+eNgQIz3UPpD1kPXDj7nufY8il24QUXfQgpJTvBvXG4LQXRxpTWIe70RU= ceph-69de8a38-0dc4-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T05:26:41.902 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoFfGMwA/8lkOGyOqFfL0OlSV1YErc2apcgLPrNr5h6cphil3iusu3zA/+9ytig4q7j+qOK1ZZzDGmRPuAmjLHzkjrNSNeNUIJ0SFjltjLPG+tyGY9eqpvLql42vGBRMs1Gc6eAPEJCunIyjboHPHQKlgptJBCFNs2Jx65VN3Q2hmPfR3CFTLpgw7keJb/XsTUd25YVFTo1THovuprHGQc4xz7A+6sAg/hvgLybz5Q7mqUFP3Unwp4oNne2Z5fF3ElvKHTSvoudZOE9lisdvljprdly5Y0fN98266iWfbprPWmAvvYlf26m4l04jsgZGIhuJAviuU4QtBq9Mr91kHMLLRG2yhDEduPddb5x2ofnXzovF4fg4uO6HpUieIzCN0bfmu52Vupu5oPE7fjbZehrvJCXYRWq0Wjev31x0ivBTmo1sh4wmq4Fvh6/N0j3EErPPWNss+eNgQIz3UPpD1kPXDj7nufY8il24QUXfQgpJTvBvXG4LQXRxpTWIe70RU= ceph-69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:26:41.916 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoFfGMwA/8lkOGyOqFfL0OlSV1YErc2apcgLPrNr5h6cphil3iusu3zA/+9ytig4q7j+qOK1ZZzDGmRPuAmjLHzkjrNSNeNUIJ0SFjltjLPG+tyGY9eqpvLql42vGBRMs1Gc6eAPEJCunIyjboHPHQKlgptJBCFNs2Jx65VN3Q2hmPfR3CFTLpgw7keJb/XsTUd25YVFTo1THovuprHGQc4xz7A+6sAg/hvgLybz5Q7mqUFP3Unwp4oNne2Z5fF3ElvKHTSvoudZOE9lisdvljprdly5Y0fN98266iWfbprPWmAvvYlf26m4l04jsgZGIhuJAviuU4QtBq9Mr91kHMLLRG2yhDEduPddb5x2ofnXzovF4fg4uO6HpUieIzCN0bfmu52Vupu5oPE7fjbZehrvJCXYRWq0Wjev31x0ivBTmo1sh4wmq4Fvh6/N0j3EErPPWNss+eNgQIz3UPpD1kPXDj7nufY8il24QUXfQgpJTvBvXG4LQXRxpTWIe70RU= ceph-69de8a38-0dc4-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T05:26:41.967 INFO:teuthology.orchestra.run.smithi195.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoFfGMwA/8lkOGyOqFfL0OlSV1YErc2apcgLPrNr5h6cphil3iusu3zA/+9ytig4q7j+qOK1ZZzDGmRPuAmjLHzkjrNSNeNUIJ0SFjltjLPG+tyGY9eqpvLql42vGBRMs1Gc6eAPEJCunIyjboHPHQKlgptJBCFNs2Jx65VN3Q2hmPfR3CFTLpgw7keJb/XsTUd25YVFTo1THovuprHGQc4xz7A+6sAg/hvgLybz5Q7mqUFP3Unwp4oNne2Z5fF3ElvKHTSvoudZOE9lisdvljprdly5Y0fN98266iWfbprPWmAvvYlf26m4l04jsgZGIhuJAviuU4QtBq9Mr91kHMLLRG2yhDEduPddb5x2ofnXzovF4fg4uO6HpUieIzCN0bfmu52Vupu5oPE7fjbZehrvJCXYRWq0Wjev31x0ivBTmo1sh4wmq4Fvh6/N0j3EErPPWNss+eNgQIz3UPpD1kPXDj7nufY8il24QUXfQgpJTvBvXG4LQXRxpTWIe70RU= ceph-69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:26:41.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T05:26:42.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:41 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3637450079' entity='client.admin' 2024-05-09T05:26:42.223 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:26:44.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.373+0000 7feabb3b1700 1 -- 172.21.15.115:0/4197402904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 msgr2=0x7feab4103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:44.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.373+0000 7feabb3b1700 1 --2- 172.21.15.115:0/4197402904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 0x7feab4103fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feab0009910 tx=0x7feab0009c20 comp rx=0 tx=0).stop 2024-05-09T05:26:44.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.373+0000 7feabb3b1700 1 -- 172.21.15.115:0/4197402904 shutdown_connections 2024-05-09T05:26:44.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.373+0000 7feabb3b1700 1 --2- 172.21.15.115:0/4197402904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 0x7feab4103fe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:44.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.373+0000 7feabb3b1700 1 -- 172.21.15.115:0/4197402904 >> 172.21.15.115:0/4197402904 conn(0x7feab40ff210 msgr2=0x7feab41016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:44.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.373+0000 7feabb3b1700 1 -- 172.21.15.115:0/4197402904 shutdown_connections 2024-05-09T05:26:44.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.373+0000 7feabb3b1700 1 -- 172.21.15.115:0/4197402904 wait complete. 2024-05-09T05:26:44.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feabb3b1700 1 Processor -- start 2024-05-09T05:26:44.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feabb3b1700 1 -- start start 2024-05-09T05:26:44.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feabb3b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 0x7feab4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:44.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feabb3b1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feab4109a40 con 0x7feab4103bd0 2024-05-09T05:26:44.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feab914d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 0x7feab4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:44.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feab914d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 0x7feab4199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35298/0 (socket says 172.21.15.115:35298) 2024-05-09T05:26:44.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feab914d700 1 -- 172.21.15.115:0/746528841 learned_addr learned my addr 172.21.15.115:0/746528841 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feab914d700 1 -- 172.21.15.115:0/746528841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab4199b90 con 0x7feab4103bd0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feab914d700 1 --2- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 0x7feab4199650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feab0005950 tx=0x7feab0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feaaa7fc700 1 -- 172.21.15.115:0/746528841 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feab0027070 con 0x7feab4103bd0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feaaa7fc700 1 -- 172.21.15.115:0/746528841 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feab000bdf0 con 0x7feab4103bd0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.374+0000 7feaaa7fc700 1 -- 172.21.15.115:0/746528841 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feab000fdb0 con 0x7feab4103bd0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.375+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab4199e20 con 0x7feab4103bd0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.375+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab419ca30 con 0x7feab4103bd0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.376+0000 7feaaa7fc700 1 -- 172.21.15.115:0/746528841 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7feab002a070 con 0x7feab4103bd0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.376+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab4192890 con 0x7feab4103bd0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.376+0000 7feaaa7fc700 1 --2- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaa003e570 0x7feaa0040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.376+0000 7feaaa7fc700 1 -- 172.21.15.115:0/746528841 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feab0053950 con 0x7feab4103bd0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.376+0000 7feab894c700 1 --2- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaa003e570 0x7feaa0040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:44.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.376+0000 7feab894c700 1 --2- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaa003e570 0x7feaa0040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feaa4006fd0 tx=0x7feaa4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.381+0000 7feaaa7fc700 1 -- 172.21.15.115:0/746528841 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feab0013070 con 0x7feab4103bd0 2024-05-09T05:26:44.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.582+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7feab410a1c0 con 0x7feab4103bd0 2024-05-09T05:26:44.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.585+0000 7feaaa7fc700 1 -- 172.21.15.115:0/746528841 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7feab00310c0 con 0x7feab4103bd0 2024-05-09T05:26:44.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaa003e570 msgr2=0x7feaa0040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:44.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 --2- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaa003e570 0x7feaa0040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feaa4006fd0 tx=0x7feaa4006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:44.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 msgr2=0x7feab4199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:44.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 --2- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 0x7feab4199650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feab0005950 tx=0x7feab0005810 comp rx=0 tx=0).stop 2024-05-09T05:26:44.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 shutdown_connections 2024-05-09T05:26:44.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 --2- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaa003e570 0x7feaa0040a80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:44.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 --2- 172.21.15.115:0/746528841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab4103bd0 0x7feab4199650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:44.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 >> 172.21.15.115:0/746528841 conn(0x7feab40ff210 msgr2=0x7feab40fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:44.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 shutdown_connections 2024-05-09T05:26:44.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:44.587+0000 7feabb3b1700 1 -- 172.21.15.115:0/746528841 wait complete. 2024-05-09T05:26:44.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:44.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:44.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:26:44.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:44.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:44.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:45.074 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T05:26:45.075 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T05:26:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:26:45.726 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:45 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/746528841' entity='client.admin' 2024-05-09T05:26:45.726 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:45.726 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:46.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.714+0000 7fa2ef71a700 1 -- 172.21.15.115:0/2024494970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 msgr2=0x7fa2e8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.714+0000 7fa2ef71a700 1 --2- 172.21.15.115:0/2024494970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 0x7fa2e8106b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009910 tx=0x7fa2e4009c20 comp rx=0 tx=0).stop 2024-05-09T05:26:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.714+0000 7fa2ef71a700 1 -- 172.21.15.115:0/2024494970 shutdown_connections 2024-05-09T05:26:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.714+0000 7fa2ef71a700 1 --2- 172.21.15.115:0/2024494970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 0x7fa2e8106b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.714+0000 7fa2ef71a700 1 -- 172.21.15.115:0/2024494970 >> 172.21.15.115:0/2024494970 conn(0x7fa2e80fe170 msgr2=0x7fa2e81005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.714+0000 7fa2ef71a700 1 -- 172.21.15.115:0/2024494970 shutdown_connections 2024-05-09T05:26:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.714+0000 7fa2ef71a700 1 -- 172.21.15.115:0/2024494970 wait complete. 2024-05-09T05:26:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.715+0000 7fa2ef71a700 1 Processor -- start 2024-05-09T05:26:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.715+0000 7fa2ef71a700 1 -- start start 2024-05-09T05:26:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.715+0000 7fa2ef71a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 0x7fa2e8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.715+0000 7fa2ef71a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8109a60 con 0x7fa2e81046f0 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.715+0000 7fa2ed4b6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 0x7fa2e8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.715+0000 7fa2ed4b6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 0x7fa2e8199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35310/0 (socket says 172.21.15.115:35310) 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.715+0000 7fa2ed4b6700 1 -- 172.21.15.115:0/1144653692 learned_addr learned my addr 172.21.15.115:0/1144653692 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.715+0000 7fa2ed4b6700 1 -- 172.21.15.115:0/1144653692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e8199b90 con 0x7fa2e81046f0 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.715+0000 7fa2ed4b6700 1 --2- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 0x7fa2e8199650 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005790 tx=0x7fa2e4004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.716+0000 7fa2de7fc700 1 -- 172.21.15.115:0/1144653692 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2e4027070 con 0x7fa2e81046f0 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.716+0000 7fa2de7fc700 1 -- 172.21.15.115:0/1144653692 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2e400bdf0 con 0x7fa2e81046f0 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.716+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e8199e20 con 0x7fa2e81046f0 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.716+0000 7fa2de7fc700 1 -- 172.21.15.115:0/1144653692 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2e400fd90 con 0x7fa2e81046f0 2024-05-09T05:26:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.716+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e819ca30 con 0x7fa2e81046f0 2024-05-09T05:26:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.716+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e810a1e0 con 0x7fa2e81046f0 2024-05-09T05:26:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.717+0000 7fa2de7fc700 1 -- 172.21.15.115:0/1144653692 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fa2e402a070 con 0x7fa2e81046f0 2024-05-09T05:26:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.717+0000 7fa2de7fc700 1 --2- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa2d403e510 0x7fa2d4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.717+0000 7fa2de7fc700 1 -- 172.21.15.115:0/1144653692 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa2e4053930 con 0x7fa2e81046f0 2024-05-09T05:26:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.717+0000 7fa2eccb5700 1 --2- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa2d403e510 0x7fa2d4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.717+0000 7fa2eccb5700 1 --2- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa2d403e510 0x7fa2d4040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006fd0 tx=0x7fa2d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.722+0000 7fa2de7fc700 1 -- 172.21.15.115:0/1144653692 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2e4013070 con 0x7fa2e81046f0 2024-05-09T05:26:46.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.932+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa2e802e700 con 0x7fa2d403e510 2024-05-09T05:26:46.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.936+0000 7fa2de7fc700 1 -- 172.21.15.115:0/1144653692 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa2e802e700 con 0x7fa2d403e510 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa2d403e510 msgr2=0x7fa2d4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 --2- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa2d403e510 0x7fa2d4040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006fd0 tx=0x7fa2d8006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 msgr2=0x7fa2e8199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 --2- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 0x7fa2e8199650 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4005790 tx=0x7fa2e4004f60 comp rx=0 tx=0).stop 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 shutdown_connections 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 --2- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa2d403e510 0x7fa2d4040a20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 --2- 172.21.15.115:0/1144653692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e81046f0 0x7fa2e8199650 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 >> 172.21.15.115:0/1144653692 conn(0x7fa2e80fe170 msgr2=0x7fa2e80feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 shutdown_connections 2024-05-09T05:26:46.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:46.938+0000 7fa2ef71a700 1 -- 172.21.15.115:0/1144653692 wait complete. 2024-05-09T05:26:47.583 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-05-09T05:26:47.583 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:26:47.583 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-05-09T05:26:47.602 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:26:47.602 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:26:47.661 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-05-09T05:26:47.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch host add smithi161 2024-05-09T05:26:47.927 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:26:48.207 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:48.207 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:48.207 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:48.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:26:48.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:26:48.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: Updating smithi115:/etc/ceph/ceph.conf 2024-05-09T05:26:48.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:26:48.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:26:48.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:48.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:48.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:49.044 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:48 smithi115 ceph-mon[30349]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:26:49.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.416+0000 7f093a9c3700 1 -- 172.21.15.115:0/2050974808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 msgr2=0x7f0934108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:49.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.416+0000 7f093a9c3700 1 --2- 172.21.15.115:0/2050974808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 0x7f0934108e90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0924009910 tx=0x7f0924009c20 comp rx=0 tx=0).stop 2024-05-09T05:26:49.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.416+0000 7f093a9c3700 1 -- 172.21.15.115:0/2050974808 shutdown_connections 2024-05-09T05:26:49.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.416+0000 7f093a9c3700 1 --2- 172.21.15.115:0/2050974808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 0x7f0934108e90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:49.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.416+0000 7f093a9c3700 1 -- 172.21.15.115:0/2050974808 >> 172.21.15.115:0/2050974808 conn(0x7f09340fe570 msgr2=0x7f09341009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:49.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.416+0000 7f093a9c3700 1 -- 172.21.15.115:0/2050974808 shutdown_connections 2024-05-09T05:26:49.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.416+0000 7f093a9c3700 1 -- 172.21.15.115:0/2050974808 wait complete. 2024-05-09T05:26:49.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.417+0000 7f093a9c3700 1 Processor -- start 2024-05-09T05:26:49.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.417+0000 7f093a9c3700 1 -- start start 2024-05-09T05:26:49.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f093a9c3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 0x7f093419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:49.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f093a9c3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f093410ce80 con 0x7f0934108a80 2024-05-09T05:26:49.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f0933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 0x7f093419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:49.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f0933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 0x7f093419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35328/0 (socket says 172.21.15.115:35328) 2024-05-09T05:26:49.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f0933fff700 1 -- 172.21.15.115:0/1569259778 learned_addr learned my addr 172.21.15.115:0/1569259778 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f0933fff700 1 -- 172.21.15.115:0/1569259778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093406c9a0 con 0x7f0934108a80 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f0933fff700 1 --2- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 0x7f093419a6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0924005660 tx=0x7f0924005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f09317fa700 1 -- 172.21.15.115:0/1569259778 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0924026070 con 0x7f0934108a80 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093419acc0 con 0x7f0934108a80 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f09317fa700 1 -- 172.21.15.115:0/1569259778 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f092400b750 con 0x7f0934108a80 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f09317fa700 1 -- 172.21.15.115:0/1569259778 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0924003810 con 0x7f0934108a80 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.418+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093419b180 con 0x7f0934108a80 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.419+0000 7f09317fa700 1 -- 172.21.15.115:0/1569259778 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f0924029070 con 0x7f0934108a80 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.419+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0934104190 con 0x7f0934108a80 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.420+0000 7f09317fa700 1 --2- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f092003e510 0x7f0920040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.420+0000 7f09317fa700 1 -- 172.21.15.115:0/1569259778 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f092401e070 con 0x7f0934108a80 2024-05-09T05:26:49.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.420+0000 7f09337fe700 1 --2- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f092003e510 0x7f0920040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:49.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.420+0000 7f09337fe700 1 --2- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f092003e510 0x7f0920040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f091c006fd0 tx=0x7f091c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:49.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.425+0000 7f09317fa700 1 -- 172.21.15.115:0/1569259778 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f093406c9a0 con 0x7f0934108a80 2024-05-09T05:26:49.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:49.633+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) -- 0x7f093406c0f0 con 0x7f092003e510 2024-05-09T05:26:50.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:50 smithi115 ceph-mon[30349]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:50.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:50 smithi115 ceph-mon[30349]: Deploying cephadm binary to smithi161 2024-05-09T05:26:53.728 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:53 smithi115 ceph-mon[30349]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:26:53.814 INFO:teuthology.orchestra.run.smithi115.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-05-09T05:26:53.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.818+0000 7f09317fa700 1 -- 172.21.15.115:0/1569259778 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f093406c0f0 con 0x7f092003e510 2024-05-09T05:26:53.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f092003e510 msgr2=0x7f0920040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:53.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 --2- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f092003e510 0x7f0920040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f091c006fd0 tx=0x7f091c006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 msgr2=0x7f093419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 --2- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 0x7f093419a6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0924005660 tx=0x7f0924005740 comp rx=0 tx=0).stop 2024-05-09T05:26:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 shutdown_connections 2024-05-09T05:26:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 --2- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f092003e510 0x7f0920040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 --2- 172.21.15.115:0/1569259778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0934108a80 0x7f093419a6c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 >> 172.21.15.115:0/1569259778 conn(0x7f09340fe570 msgr2=0x7f09340ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 shutdown_connections 2024-05-09T05:26:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:53.821+0000 7f093a9c3700 1 -- 172.21.15.115:0/1569259778 wait complete. 2024-05-09T05:26:54.379 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T05:26:54.618 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:26:54.952 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:54.953 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:54 smithi115 ceph-mon[30349]: Added host smithi161 2024-05-09T05:26:54.953 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:26:54.953 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:56.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:55 smithi115 ceph-mon[30349]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:26:56.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.219+0000 7fc07be6a700 1 -- 172.21.15.115:0/182890165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 msgr2=0x7fc074102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.219+0000 7fc07be6a700 1 --2- 172.21.15.115:0/182890165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 0x7fc074102890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc07000c210 tx=0x7fc07000c520 comp rx=0 tx=0).stop 2024-05-09T05:26:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.219+0000 7fc07be6a700 1 -- 172.21.15.115:0/182890165 shutdown_connections 2024-05-09T05:26:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.219+0000 7fc07be6a700 1 --2- 172.21.15.115:0/182890165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 0x7fc074102890 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.219+0000 7fc07be6a700 1 -- 172.21.15.115:0/182890165 >> 172.21.15.115:0/182890165 conn(0x7fc074077d50 msgr2=0x7fc0740781b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.219+0000 7fc07be6a700 1 -- 172.21.15.115:0/182890165 shutdown_connections 2024-05-09T05:26:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.219+0000 7fc07be6a700 1 -- 172.21.15.115:0/182890165 wait complete. 2024-05-09T05:26:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.220+0000 7fc07be6a700 1 Processor -- start 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.220+0000 7fc07be6a700 1 -- start start 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.220+0000 7fc07be6a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 0x7fc074197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.220+0000 7fc07be6a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc07410bd80 con 0x7fc074102480 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.220+0000 7fc079c06700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 0x7fc074197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.220+0000 7fc079c06700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 0x7fc074197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33662/0 (socket says 172.21.15.115:33662) 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.220+0000 7fc079c06700 1 -- 172.21.15.115:0/3210648190 learned_addr learned my addr 172.21.15.115:0/3210648190 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.220+0000 7fc079c06700 1 -- 172.21.15.115:0/3210648190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc074197b70 con 0x7fc074102480 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.220+0000 7fc079c06700 1 --2- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 0x7fc074197630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc07000c9a0 tx=0x7fc070008a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.221+0000 7fc06affd700 1 -- 172.21.15.115:0/3210648190 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc070008050 con 0x7fc074102480 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.221+0000 7fc06affd700 1 -- 172.21.15.115:0/3210648190 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0700081f0 con 0x7fc074102480 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.221+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc074197e00 con 0x7fc074102480 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.221+0000 7fc06affd700 1 -- 172.21.15.115:0/3210648190 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0700041d0 con 0x7fc074102480 2024-05-09T05:26:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.221+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07419be60 con 0x7fc074102480 2024-05-09T05:26:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.222+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc074103930 con 0x7fc074102480 2024-05-09T05:26:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.222+0000 7fc06affd700 1 -- 172.21.15.115:0/3210648190 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fc070022020 con 0x7fc074102480 2024-05-09T05:26:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.222+0000 7fc06affd700 1 --2- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc06003e5c0 0x7fc060040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.222+0000 7fc06affd700 1 -- 172.21.15.115:0/3210648190 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc070010050 con 0x7fc074102480 2024-05-09T05:26:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.222+0000 7fc079405700 1 --2- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc06003e5c0 0x7fc060040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.223+0000 7fc079405700 1 --2- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc06003e5c0 0x7fc060040ad0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc064006fd0 tx=0x7fc064006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.228+0000 7fc06affd700 1 -- 172.21.15.115:0/3210648190 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc070004370 con 0x7fc074102480 2024-05-09T05:26:56.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.436+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc074064e20 con 0x7fc06003e5c0 2024-05-09T05:26:56.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.437+0000 7fc06affd700 1 -- 172.21.15.115:0/3210648190 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fc074064e20 con 0x7fc06003e5c0 2024-05-09T05:26:56.432 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:26:56.432 INFO:teuthology.orchestra.run.smithi115.stdout:[{"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-05-09T05:26:56.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.439+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc06003e5c0 msgr2=0x7fc060040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:56.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.439+0000 7fc07be6a700 1 --2- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc06003e5c0 0x7fc060040ad0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc064006fd0 tx=0x7fc064006e40 comp rx=0 tx=0).stop 2024-05-09T05:26:56.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.440+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 msgr2=0x7fc074197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:56.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.440+0000 7fc07be6a700 1 --2- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 0x7fc074197630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc07000c9a0 tx=0x7fc070008a50 comp rx=0 tx=0).stop 2024-05-09T05:26:56.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.440+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 shutdown_connections 2024-05-09T05:26:56.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.440+0000 7fc07be6a700 1 --2- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc06003e5c0 0x7fc060040ad0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:56.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.440+0000 7fc07be6a700 1 --2- 172.21.15.115:0/3210648190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc074102480 0x7fc074197630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:56.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.440+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 >> 172.21.15.115:0/3210648190 conn(0x7fc074077d50 msgr2=0x7fc074103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:56.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.440+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 shutdown_connections 2024-05-09T05:26:56.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:56.440+0000 7fc07be6a700 1 -- 172.21.15.115:0/3210648190 wait complete. 2024-05-09T05:26:56.956 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi195 2024-05-09T05:26:56.957 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:26:56.957 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.conf 2024-05-09T05:26:56.979 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:26:56.979 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:26:57.037 INFO:tasks.cephadm:Adding host smithi195 to orchestrator... 2024-05-09T05:26:57.037 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch host add smithi195 2024-05-09T05:26:57.272 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:26:57.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:57 smithi115 ceph-mon[30349]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:26:57.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:57 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:26:57.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:57 smithi115 ceph-mon[30349]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:26:58.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.833+0000 7fdd0fec4700 1 -- 172.21.15.115:0/4224480880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 msgr2=0x7fdd0810a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:26:58.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.833+0000 7fdd0fec4700 1 --2- 172.21.15.115:0/4224480880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 0x7fdd0810a030 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009910 tx=0x7fdd04009c20 comp rx=0 tx=0).stop 2024-05-09T05:26:58.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.833+0000 7fdd0fec4700 1 -- 172.21.15.115:0/4224480880 shutdown_connections 2024-05-09T05:26:58.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.833+0000 7fdd0fec4700 1 --2- 172.21.15.115:0/4224480880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 0x7fdd0810a030 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:26:58.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.833+0000 7fdd0fec4700 1 -- 172.21.15.115:0/4224480880 >> 172.21.15.115:0/4224480880 conn(0x7fdd080ff660 msgr2=0x7fdd08101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.833+0000 7fdd0fec4700 1 -- 172.21.15.115:0/4224480880 shutdown_connections 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.833+0000 7fdd0fec4700 1 -- 172.21.15.115:0/4224480880 wait complete. 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdd0fec4700 1 Processor -- start 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdd0fec4700 1 -- start start 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdd0fec4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 0x7fdd0819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdd0fec4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810dfb0 con 0x7fdd08109c20 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdd0dc60700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 0x7fdd0819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdd0dc60700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 0x7fdd0819f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33682/0 (socket says 172.21.15.115:33682) 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdd0dc60700 1 -- 172.21.15.115:0/2994268508 learned_addr learned my addr 172.21.15.115:0/2994268508 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdd0dc60700 1 -- 172.21.15.115:0/2994268508 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0819fa30 con 0x7fdd08109c20 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdd0dc60700 1 --2- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 0x7fdd0819f4f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd04005660 tx=0x7fdd04005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.834+0000 7fdcfeffd700 1 -- 172.21.15.115:0/2994268508 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd04026070 con 0x7fdd08109c20 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.835+0000 7fdcfeffd700 1 -- 172.21.15.115:0/2994268508 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd0400b750 con 0x7fdd08109c20 2024-05-09T05:26:58.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.835+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0819fcc0 con 0x7fdd08109c20 2024-05-09T05:26:58.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.835+0000 7fdcfeffd700 1 -- 172.21.15.115:0/2994268508 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd04003810 con 0x7fdd08109c20 2024-05-09T05:26:58.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.835+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd08199680 con 0x7fdd08109c20 2024-05-09T05:26:58.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.835+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd081051f0 con 0x7fdd08109c20 2024-05-09T05:26:58.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.836+0000 7fdcfeffd700 1 -- 172.21.15.115:0/2994268508 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fdd04029070 con 0x7fdd08109c20 2024-05-09T05:26:58.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.836+0000 7fdcfeffd700 1 --2- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdcf403e510 0x7fdcf4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:26:58.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.836+0000 7fdcfeffd700 1 -- 172.21.15.115:0/2994268508 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdd0401e070 con 0x7fdd08109c20 2024-05-09T05:26:58.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.836+0000 7fdd0d45f700 1 --2- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdcf403e510 0x7fdcf4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:26:58.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.836+0000 7fdd0d45f700 1 --2- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdcf403e510 0x7fdcf4040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8006fd0 tx=0x7fdcf8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:26:58.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:58.841+0000 7fdcfeffd700 1 -- 172.21.15.115:0/2994268508 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd04013070 con 0x7fdd08109c20 2024-05-09T05:26:59.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:26:59.049+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}) -- 0x7fdd0819ab90 con 0x7fdcf403e510 2024-05-09T05:26:59.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:59 smithi115 ceph-mon[30349]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:26:59.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:59 smithi115 ceph-mon[30349]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:26:59.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:26:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:00.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:00.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:00.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:00.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:27:00.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:00.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:00.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-09T05:27:00.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: Deploying cephadm binary to smithi195 2024-05-09T05:27:00.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:00.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:00 smithi115 ceph-mon[30349]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:27:01.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:01 smithi115 ceph-mon[30349]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:03.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.187+0000 7fdcfeffd700 1 -- 172.21.15.115:0/2994268508 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fdd0819ab90 con 0x7fdcf403e510 2024-05-09T05:27:03.182 INFO:teuthology.orchestra.run.smithi115.stdout:Added host 'smithi195' with addr '172.21.15.195' 2024-05-09T05:27:03.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.189+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdcf403e510 msgr2=0x7fdcf4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:03.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.189+0000 7fdd0fec4700 1 --2- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdcf403e510 0x7fdcf4040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8006fd0 tx=0x7fdcf8006e40 comp rx=0 tx=0).stop 2024-05-09T05:27:03.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.189+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 msgr2=0x7fdd0819f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:03.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.189+0000 7fdd0fec4700 1 --2- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 0x7fdd0819f4f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd04005660 tx=0x7fdd04005740 comp rx=0 tx=0).stop 2024-05-09T05:27:03.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.189+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 shutdown_connections 2024-05-09T05:27:03.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.189+0000 7fdd0fec4700 1 --2- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdcf403e510 0x7fdcf4040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:03.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.189+0000 7fdd0fec4700 1 --2- 172.21.15.115:0/2994268508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd08109c20 0x7fdd0819f4f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:03.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.189+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 >> 172.21.15.115:0/2994268508 conn(0x7fdd080ff660 msgr2=0x7fdd08100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:03.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.190+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 shutdown_connections 2024-05-09T05:27:03.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:03.190+0000 7fdd0fec4700 1 -- 172.21.15.115:0/2994268508 wait complete. 2024-05-09T05:27:04.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:03 smithi115 ceph-mon[30349]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:04.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:03 smithi115 ceph-mon[30349]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:27:04.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:04.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:04.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:04.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:04.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:03 smithi115 ceph-mon[30349]: Added host smithi195 2024-05-09T05:27:04.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:04.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:04.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T05:27:04.313 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:27:05.485 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:05 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:05.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.718+0000 7f1e462ad700 1 -- 172.21.15.115:0/2065809300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 msgr2=0x7f1e401041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:05.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.718+0000 7f1e462ad700 1 --2- 172.21.15.115:0/2065809300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 0x7f1e401041e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009910 tx=0x7f1e30009c20 comp rx=0 tx=0).stop 2024-05-09T05:27:05.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.718+0000 7f1e462ad700 1 -- 172.21.15.115:0/2065809300 shutdown_connections 2024-05-09T05:27:05.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.718+0000 7f1e462ad700 1 --2- 172.21.15.115:0/2065809300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 0x7f1e401041e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:05.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.718+0000 7f1e462ad700 1 -- 172.21.15.115:0/2065809300 >> 172.21.15.115:0/2065809300 conn(0x7f1e400ff6a0 msgr2=0x7f1e40101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:05.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.718+0000 7f1e462ad700 1 -- 172.21.15.115:0/2065809300 shutdown_connections 2024-05-09T05:27:05.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.718+0000 7f1e462ad700 1 -- 172.21.15.115:0/2065809300 wait complete. 2024-05-09T05:27:05.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.718+0000 7f1e462ad700 1 Processor -- start 2024-05-09T05:27:05.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.718+0000 7f1e462ad700 1 -- start start 2024-05-09T05:27:05.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e462ad700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 0x7f1e40075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e462ad700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010df70 con 0x7f1e40103dd0 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 0x7f1e40075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 0x7f1e40075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34470/0 (socket says 172.21.15.115:34470) 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e3ffff700 1 -- 172.21.15.115:0/4125160392 learned_addr learned my addr 172.21.15.115:0/4125160392 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e3ffff700 1 -- 172.21.15.115:0/4125160392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e40077910 con 0x7f1e40103dd0 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e3ffff700 1 --2- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 0x7f1e40075ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e30005660 tx=0x7f1e30005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e3dffb700 1 -- 172.21.15.115:0/4125160392 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e30026070 con 0x7f1e40103dd0 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e3dffb700 1 -- 172.21.15.115:0/4125160392 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e3000b6f0 con 0x7f1e40103dd0 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e40076000 con 0x7f1e40103dd0 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e3dffb700 1 -- 172.21.15.115:0/4125160392 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e300037b0 con 0x7f1e40103dd0 2024-05-09T05:27:05.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.719+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e400764c0 con 0x7f1e40103dd0 2024-05-09T05:27:05.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.720+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e40105280 con 0x7f1e40103dd0 2024-05-09T05:27:05.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.721+0000 7f1e3dffb700 1 -- 172.21.15.115:0/4125160392 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f1e30029070 con 0x7f1e40103dd0 2024-05-09T05:27:05.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.721+0000 7f1e3dffb700 1 --2- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e2c03e510 0x7f1e2c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:05.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.721+0000 7f1e3dffb700 1 -- 172.21.15.115:0/4125160392 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1e3001e070 con 0x7f1e40103dd0 2024-05-09T05:27:05.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.721+0000 7f1e37fff700 1 --2- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e2c03e510 0x7f1e2c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:05.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.721+0000 7f1e37fff700 1 --2- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e2c03e510 0x7f1e2c040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1e28006fd0 tx=0x7f1e28006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:05.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.726+0000 7f1e3dffb700 1 -- 172.21.15.115:0/4125160392 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e30013070 con 0x7f1e40103dd0 2024-05-09T05:27:05.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.937+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e40076740 con 0x7f1e2c03e510 2024-05-09T05:27:05.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.939+0000 7f1e3dffb700 1 -- 172.21.15.115:0/4125160392 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f1e40076740 con 0x7f1e2c03e510 2024-05-09T05:27:05.933 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:27:05.933 INFO:teuthology.orchestra.run.smithi115.stdout:[{"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}, {"addr": "172.21.15.195", "hostname": "smithi195", "labels": [], "status": ""}] 2024-05-09T05:27:05.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e2c03e510 msgr2=0x7f1e2c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:05.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 --2- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e2c03e510 0x7f1e2c040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1e28006fd0 tx=0x7f1e28006e40 comp rx=0 tx=0).stop 2024-05-09T05:27:05.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 msgr2=0x7f1e40075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:05.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 --2- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 0x7f1e40075ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e30005660 tx=0x7f1e30005740 comp rx=0 tx=0).stop 2024-05-09T05:27:05.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 shutdown_connections 2024-05-09T05:27:05.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 --2- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e2c03e510 0x7f1e2c040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:05.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 --2- 172.21.15.115:0/4125160392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e40103dd0 0x7f1e40075ac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:05.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 >> 172.21.15.115:0/4125160392 conn(0x7f1e400ff6a0 msgr2=0x7f1e40109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:05.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 shutdown_connections 2024-05-09T05:27:05.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:05.941+0000 7f1e462ad700 1 -- 172.21.15.115:0/4125160392 wait complete. 2024-05-09T05:27:06.552 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:06 smithi115 ceph-mon[30349]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:06.553 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:06.580 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T05:27:06.580 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-09T05:27:06.815 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:27:07.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:07 smithi115 ceph-mon[30349]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:27:08.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.189+0000 7f75d1d66700 1 -- 172.21.15.115:0/1031052270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 msgr2=0x7f75cc103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.189+0000 7f75d1d66700 1 --2- 172.21.15.115:0/1031052270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 0x7f75cc103850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009910 tx=0x7f75bc009c20 comp rx=0 tx=0).stop 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.189+0000 7f75d1d66700 1 -- 172.21.15.115:0/1031052270 shutdown_connections 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.189+0000 7f75d1d66700 1 --2- 172.21.15.115:0/1031052270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 0x7f75cc103850 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.189+0000 7f75d1d66700 1 -- 172.21.15.115:0/1031052270 >> 172.21.15.115:0/1031052270 conn(0x7f75cc077d50 msgr2=0x7f75cc0781b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.189+0000 7f75d1d66700 1 -- 172.21.15.115:0/1031052270 shutdown_connections 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.189+0000 7f75d1d66700 1 -- 172.21.15.115:0/1031052270 wait complete. 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.190+0000 7f75d1d66700 1 Processor -- start 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.190+0000 7f75d1d66700 1 -- start start 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.190+0000 7f75d1d66700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 0x7f75cc19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:08.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.190+0000 7f75d1d66700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d0c0 con 0x7f75cc103440 2024-05-09T05:27:08.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.190+0000 7f75cb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 0x7f75cc19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:08.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.190+0000 7f75cb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 0x7f75cc19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34488/0 (socket says 172.21.15.115:34488) 2024-05-09T05:27:08.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.190+0000 7f75cb7fe700 1 -- 172.21.15.115:0/728406180 learned_addr learned my addr 172.21.15.115:0/728406180 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:27:08.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.191+0000 7f75cb7fe700 1 -- 172.21.15.115:0/728406180 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc102f80 con 0x7f75cc103440 2024-05-09T05:27:08.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.191+0000 7f75cb7fe700 1 --2- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 0x7f75cc19a930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75bc005660 tx=0x7f75bc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:08.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.191+0000 7f75c8ff9700 1 -- 172.21.15.115:0/728406180 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75bc026070 con 0x7f75cc103440 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.191+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc19aed0 con 0x7f75cc103440 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.191+0000 7f75c8ff9700 1 -- 172.21.15.115:0/728406180 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75bc00b750 con 0x7f75cc103440 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.191+0000 7f75c8ff9700 1 -- 172.21.15.115:0/728406180 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75bc0037b0 con 0x7f75cc103440 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.191+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc19b390 con 0x7f75cc103440 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.192+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75cc104460 con 0x7f75cc103440 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.192+0000 7f75c8ff9700 1 -- 172.21.15.115:0/728406180 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f75bc029070 con 0x7f75cc103440 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.192+0000 7f75c8ff9700 1 --2- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f75b403e510 0x7f75b4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.192+0000 7f75c8ff9700 1 -- 172.21.15.115:0/728406180 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f75bc01e070 con 0x7f75cc103440 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.192+0000 7f75caffd700 1 --2- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f75b403e510 0x7f75b4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:08.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.193+0000 7f75caffd700 1 --2- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f75b403e510 0x7f75b4040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f75c0006fd0 tx=0x7f75c0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:08.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.198+0000 7f75c8ff9700 1 -- 172.21.15.115:0/728406180 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75bc013070 con 0x7f75cc103440 2024-05-09T05:27:08.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:08 smithi115 ceph-mon[30349]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:08.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:08.409+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f75cc06b870 con 0x7f75cc103440 2024-05-09T05:27:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.266+0000 7f75c8ff9700 1 -- 172.21.15.115:0/728406180 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f75bc021410 con 0x7f75cc103440 2024-05-09T05:27:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.268+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f75b403e510 msgr2=0x7f75b4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.268+0000 7f75d1d66700 1 --2- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f75b403e510 0x7f75b4040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f75c0006fd0 tx=0x7f75c0006e40 comp rx=0 tx=0).stop 2024-05-09T05:27:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.268+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 msgr2=0x7f75cc19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.268+0000 7f75d1d66700 1 --2- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 0x7f75cc19a930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75bc005660 tx=0x7f75bc005740 comp rx=0 tx=0).stop 2024-05-09T05:27:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.268+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 shutdown_connections 2024-05-09T05:27:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.269+0000 7f75d1d66700 1 --2- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f75b403e510 0x7f75b4040a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.269+0000 7f75d1d66700 1 --2- 172.21.15.115:0/728406180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75cc103440 0x7f75cc19a930 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.269+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 >> 172.21.15.115:0/728406180 conn(0x7f75cc077d50 msgr2=0x7f75cc1048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.269+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 shutdown_connections 2024-05-09T05:27:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:09.269+0000 7f75d1d66700 1 -- 172.21.15.115:0/728406180 wait complete. 2024-05-09T05:27:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:adjusted tunables profile to default 2024-05-09T05:27:09.597 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:09 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/728406180' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T05:27:09.597 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:09 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:09.598 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:09 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:09.598 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:09 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:09.598 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:09 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:09.598 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:09 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:27:09.598 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:09 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:09.598 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:09 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:09.859 INFO:tasks.cephadm:Adding mon.a on smithi115 2024-05-09T05:27:09.859 INFO:tasks.cephadm:Adding mon.b on smithi161 2024-05-09T05:27:09.859 INFO:tasks.cephadm:Adding mon.c on smithi195 2024-05-09T05:27:09.859 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi115:172.21.15.115=a;smithi161:172.21.15.161=b;smithi195:172.21.15.195=c' 2024-05-09T05:27:10.089 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:10.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:10 smithi115 ceph-mon[30349]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:10.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:10 smithi115 ceph-mon[30349]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-09T05:27:10.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:10 smithi115 ceph-mon[30349]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:10.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:10 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/728406180' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T05:27:10.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:10 smithi115 ceph-mon[30349]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:10.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:10 smithi115 ceph-mon[30349]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:27:10.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:10.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:10.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:11.523 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.528+0000 7f3e8847e700 1 -- 172.21.15.195:0/2217345902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 msgr2=0x7f3e8010a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:11.523 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.528+0000 7f3e8847e700 1 --2- 172.21.15.195:0/2217345902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 0x7f3e8010a030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009910 tx=0x7f3e7c009c20 comp rx=0 tx=0).stop 2024-05-09T05:27:11.523 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.528+0000 7f3e8847e700 1 -- 172.21.15.195:0/2217345902 shutdown_connections 2024-05-09T05:27:11.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.528+0000 7f3e8847e700 1 --2- 172.21.15.195:0/2217345902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 0x7f3e8010a030 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:11.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.528+0000 7f3e8847e700 1 -- 172.21.15.195:0/2217345902 >> 172.21.15.195:0/2217345902 conn(0x7f3e800ff660 msgr2=0x7f3e80101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:11.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.528+0000 7f3e8847e700 1 -- 172.21.15.195:0/2217345902 shutdown_connections 2024-05-09T05:27:11.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.528+0000 7f3e8847e700 1 -- 172.21.15.195:0/2217345902 wait complete. 2024-05-09T05:27:11.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.529+0000 7f3e8847e700 1 Processor -- start 2024-05-09T05:27:11.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.529+0000 7f3e8847e700 1 -- start start 2024-05-09T05:27:11.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.529+0000 7f3e8847e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 0x7f3e80075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:11.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.529+0000 7f3e8847e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8010dfb0 con 0x7f3e80109c20 2024-05-09T05:27:11.525 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.530+0000 7f3e8621a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 0x7f3e80075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:11.525 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.530+0000 7f3e8621a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 0x7f3e80075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.195:44936/0 (socket says 172.21.15.195:44936) 2024-05-09T05:27:11.525 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.530+0000 7f3e8621a700 1 -- 172.21.15.195:0/981105371 learned_addr learned my addr 172.21.15.195:0/981105371 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:27:11.525 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.530+0000 7f3e8621a700 1 -- 172.21.15.195:0/981105371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e80077910 con 0x7f3e80109c20 2024-05-09T05:27:11.526 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.530+0000 7f3e8621a700 1 --2- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 0x7f3e80075ac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005660 tx=0x7f3e7c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:11.526 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.530+0000 7f3e777fe700 1 -- 172.21.15.195:0/981105371 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e7c026070 con 0x7f3e80109c20 2024-05-09T05:27:11.526 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.530+0000 7f3e777fe700 1 -- 172.21.15.195:0/981105371 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e7c00b750 con 0x7f3e80109c20 2024-05-09T05:27:11.526 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.530+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e80076000 con 0x7f3e80109c20 2024-05-09T05:27:11.526 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.530+0000 7f3e777fe700 1 -- 172.21.15.195:0/981105371 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e7c003810 con 0x7f3e80109c20 2024-05-09T05:27:11.526 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.531+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e80076400 con 0x7f3e80109c20 2024-05-09T05:27:11.526 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.531+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e801051f0 con 0x7f3e80109c20 2024-05-09T05:27:11.526 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.532+0000 7f3e777fe700 1 -- 172.21.15.195:0/981105371 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f3e7c029070 con 0x7f3e80109c20 2024-05-09T05:27:11.527 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.532+0000 7f3e777fe700 1 --2- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e6c03e510 0x7f3e6c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:11.527 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.532+0000 7f3e777fe700 1 -- 172.21.15.195:0/981105371 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3e7c01e070 con 0x7f3e80109c20 2024-05-09T05:27:11.527 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.532+0000 7f3e85a19700 1 --2- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e6c03e510 0x7f3e6c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:11.527 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.532+0000 7f3e85a19700 1 --2- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e6c03e510 0x7f3e6c040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3e70006fd0 tx=0x7f3e70006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:11.531 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.538+0000 7f3e777fe700 1 -- 172.21.15.195:0/981105371 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e7c013070 con 0x7f3e80109c20 2024-05-09T05:27:11.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:11 smithi115 ceph-mon[30349]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:27:11.749 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.754+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi115:172.21.15.115=a;smithi161:172.21.15.161=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}) -- 0x7f3e80076680 con 0x7f3e6c03e510 2024-05-09T05:27:11.757 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.762+0000 7f3e777fe700 1 -- 172.21.15.195:0/981105371 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3e80076680 con 0x7f3e6c03e510 2024-05-09T05:27:11.758 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mon update... 2024-05-09T05:27:11.759 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e6c03e510 msgr2=0x7f3e6c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:11.759 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 --2- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e6c03e510 0x7f3e6c040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3e70006fd0 tx=0x7f3e70006e40 comp rx=0 tx=0).stop 2024-05-09T05:27:11.759 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 msgr2=0x7f3e80075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:11.759 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 --2- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 0x7f3e80075ac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005660 tx=0x7f3e7c005740 comp rx=0 tx=0).stop 2024-05-09T05:27:11.759 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 shutdown_connections 2024-05-09T05:27:11.759 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 --2- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e6c03e510 0x7f3e6c040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:11.759 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 --2- 172.21.15.195:0/981105371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e80109c20 0x7f3e80075ac0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:11.760 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 >> 172.21.15.195:0/981105371 conn(0x7f3e800ff660 msgr2=0x7f3e80100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:11.760 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 shutdown_connections 2024-05-09T05:27:11.760 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:11.764+0000 7f3e8847e700 1 -- 172.21.15.195:0/981105371 wait complete. 2024-05-09T05:27:12.398 DEBUG:teuthology.orchestra.run.smithi161:mon.b> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.b.service 2024-05-09T05:27:12.400 DEBUG:teuthology.orchestra.run.smithi195:mon.c> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.c.service 2024-05-09T05:27:12.403 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T05:27:12.403 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T05:27:12.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:12 smithi115 ceph-mon[30349]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:12.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:12.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:12.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:12.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:12.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:12.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:12.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:13.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:13 smithi115 ceph-mon[30349]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi115:172.21.15.115=a;smithi161:172.21.15.161=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:27:13.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:13 smithi115 ceph-mon[30349]: Saving service mon spec with placement smithi115:172.21.15.115=a;smithi161:172.21.15.161=b;smithi195:172.21.15.195=c;count:3 2024-05-09T05:27:13.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:13 smithi115 ceph-mon[30349]: Deploying daemon mon.c on smithi195 2024-05-09T05:27:14.120 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:14.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:14 smithi115 ceph-mon[30349]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.525+0000 7f0facbf3700 1 -- 172.21.15.195:0/4265676337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 msgr2=0x7f0fa8104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.525+0000 7f0facbf3700 1 --2- 172.21.15.195:0/4265676337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 0x7f0fa8104200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009910 tx=0x7f0f98009c20 comp rx=0 tx=0).stop 2024-05-09T05:27:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.525+0000 7f0facbf3700 1 -- 172.21.15.195:0/4265676337 shutdown_connections 2024-05-09T05:27:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.525+0000 7f0facbf3700 1 --2- 172.21.15.195:0/4265676337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 0x7f0fa8104200 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.525+0000 7f0facbf3700 1 -- 172.21.15.195:0/4265676337 >> 172.21.15.195:0/4265676337 conn(0x7f0fa80ff6a0 msgr2=0x7f0fa8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.525+0000 7f0facbf3700 1 -- 172.21.15.195:0/4265676337 shutdown_connections 2024-05-09T05:27:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.525+0000 7f0facbf3700 1 -- 172.21.15.195:0/4265676337 wait complete. 2024-05-09T05:27:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.526+0000 7f0facbf3700 1 Processor -- start 2024-05-09T05:27:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.526+0000 7f0facbf3700 1 -- start start 2024-05-09T05:27:16.522 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.527+0000 7f0facbf3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 0x7f0fa819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:16.522 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.527+0000 7f0facbf3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa810df90 con 0x7f0fa8103df0 2024-05-09T05:27:16.522 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.527+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 0x7f0fa819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:16.522 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.527+0000 7f0fa659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 0x7f0fa819f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.195:44956/0 (socket says 172.21.15.195:44956) 2024-05-09T05:27:16.522 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.527+0000 7f0fa659c700 1 -- 172.21.15.195:0/3622709557 learned_addr learned my addr 172.21.15.195:0/3622709557 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:27:16.523 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.527+0000 7f0fa659c700 1 -- 172.21.15.195:0/3622709557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa819fa90 con 0x7f0fa8103df0 2024-05-09T05:27:16.523 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.528+0000 7f0fa659c700 1 --2- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 0x7f0fa819f550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005660 tx=0x7f0f98005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:16.523 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.528+0000 7f0f977fe700 1 -- 172.21.15.195:0/3622709557 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f98026070 con 0x7f0fa8103df0 2024-05-09T05:27:16.523 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.528+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa819fd80 con 0x7f0fa8103df0 2024-05-09T05:27:16.523 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.528+0000 7f0f977fe700 1 -- 172.21.15.195:0/3622709557 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f9800b6f0 con 0x7f0fa8103df0 2024-05-09T05:27:16.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.528+0000 7f0f977fe700 1 -- 172.21.15.195:0/3622709557 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f980037b0 con 0x7f0fa8103df0 2024-05-09T05:27:16.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.528+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81998b0 con 0x7f0fa8103df0 2024-05-09T05:27:16.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.529+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa81052a0 con 0x7f0fa8103df0 2024-05-09T05:27:16.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.529+0000 7f0f977fe700 1 -- 172.21.15.195:0/3622709557 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f0f98029070 con 0x7f0fa8103df0 2024-05-09T05:27:16.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.529+0000 7f0f977fe700 1 --2- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0f9003e510 0x7f0f90040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:16.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.529+0000 7f0f977fe700 1 -- 172.21.15.195:0/3622709557 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0f9801e070 con 0x7f0fa8103df0 2024-05-09T05:27:16.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.529+0000 7f0fa5d9b700 1 --2- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0f9003e510 0x7f0f90040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:16.524 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.530+0000 7f0fa5d9b700 1 --2- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0f9003e510 0x7f0f90040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006fd0 tx=0x7f0f9c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:16.529 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.535+0000 7f0f977fe700 1 -- 172.21.15.195:0/3622709557 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f98013070 con 0x7f0fa8103df0 2024-05-09T05:27:16.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:16 smithi115 ceph-mon[30349]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:16.657 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(???) e0 preinit fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:16.682 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.688+0000 7f0f977fe700 1 -- 172.21.15.195:0/3622709557 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0f9802a730 con 0x7f0fa8103df0 2024-05-09T05:27:16.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:16.835+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0fa819ffb0 con 0x7f0fa8103df0 2024-05-09T05:27:17.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).mds e1 new map 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout: e1 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout: legacy client fscid: -1 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout: 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout: No filesystems configured 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T05:27:17.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Deploying cephadm binary to smithi195 2024-05-09T05:27:17.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Added host smithi195 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/728406180' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-09T05:27:17.121 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/728406180' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:17.122 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:17.123 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:17.123 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi115:172.21.15.115=a;smithi161:172.21.15.161=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:27:17.123 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Saving service mon spec with placement smithi115:172.21.15.115=a;smithi161:172.21.15.161=b;smithi195:172.21.15.195=c;count:3 2024-05-09T05:27:17.123 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: Deploying daemon mon.c on smithi195 2024-05-09T05:27:17.123 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:17.123 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:17.123 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 ceph-mon[33844]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T05:27:17.123 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 bash[33799]: 2e0df25d136516649ed1a9fa3bb759a33248b7494339a65ed49baf0696203b0d 2024-05-09T05:27:17.123 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:16 smithi195 systemd[1]: Started Ceph mon.c for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:27:21.694 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.699+0000 7f0f977fe700 1 -- 172.21.15.195:0/3622709557 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f0f98030050 con 0x7f0fa8103df0 2024-05-09T05:27:21.695 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:27:21.695 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":2,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","modified":"2024-05-09T05:27:16.688162Z","created":"2024-05-09T05:25:30.870570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T05:27:21.698 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0f9003e510 msgr2=0x7f0f90040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:21.698 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 --2- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0f9003e510 0x7f0f90040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006fd0 tx=0x7f0f9c006e40 comp rx=0 tx=0).stop 2024-05-09T05:27:21.698 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 msgr2=0x7f0fa819f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:21.698 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 --2- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 0x7f0fa819f550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005660 tx=0x7f0f98005740 comp rx=0 tx=0).stop 2024-05-09T05:27:21.699 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 shutdown_connections 2024-05-09T05:27:21.699 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 --2- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0f9003e510 0x7f0f90040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:21.699 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 --2- 172.21.15.195:0/3622709557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fa8103df0 0x7f0fa819f550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:21.699 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 >> 172.21.15.195:0/3622709557 conn(0x7f0fa80ff6a0 msgr2=0x7f0fa8109920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:21.699 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 shutdown_connections 2024-05-09T05:27:21.699 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:21.703+0000 7f0facbf3700 1 -- 172.21.15.195:0/3622709557 wait complete. 2024-05-09T05:27:21.699 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 2 2024-05-09T05:27:21.960 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:21.960 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:27:21.960 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:21.960 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: mon.a calling monitor election 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/3622709557' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: mon.c calling monitor election 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: monmap epoch 2 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: last_changed 2024-05-09T05:27:16.688162+0000 2024-05-09T05:27:21.961 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: min_mon_release 19 (squid) 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: election_strategy: 1 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: fsmap 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: mgrmap e13: x(active, since 49s) 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: overall HEALTH_OK 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:21.962 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:27:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: mon.a calling monitor election 2024-05-09T05:27:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/3622709557' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T05:27:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: mon.c calling monitor election 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: monmap epoch 2 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: last_changed 2024-05-09T05:27:16.688162+0000 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: min_mon_release 19 (squid) 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: election_strategy: 1 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-09T05:27:22.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: fsmap 2024-05-09T05:27:22.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:22.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: mgrmap e13: x(active, since 49s) 2024-05-09T05:27:22.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: overall HEALTH_OK 2024-05-09T05:27:22.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:22.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:22.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:22.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:22.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:23.049 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:22 smithi115 ceph-mon[30349]: Deploying daemon mon.b on smithi161 2024-05-09T05:27:23.049 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:22 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:23.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:22 smithi195 ceph-mon[33844]: Deploying daemon mon.b on smithi161 2024-05-09T05:27:23.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:22 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:23.236 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T05:27:23.236 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T05:27:23.479 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:27:23.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:23 smithi195 ceph-mon[33844]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:24.060 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:23 smithi115 ceph-mon[30349]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:24.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.928+0000 7fb51ef36700 1 -- 172.21.15.195:0/655517458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 msgr2=0x7fb518109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:24.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.928+0000 7fb51ef36700 1 --2- 172.21.15.195:0/655517458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 0x7fb518109040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009910 tx=0x7fb50c009c20 comp rx=0 tx=0).stop 2024-05-09T05:27:24.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.928+0000 7fb51ef36700 1 -- 172.21.15.195:0/655517458 shutdown_connections 2024-05-09T05:27:24.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.928+0000 7fb51ef36700 1 --2- 172.21.15.195:0/655517458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 0x7fb518109040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:24.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.928+0000 7fb51ef36700 1 -- 172.21.15.195:0/655517458 >> 172.21.15.195:0/655517458 conn(0x7fb5180fe650 msgr2=0x7fb518100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:24.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.928+0000 7fb51ef36700 1 -- 172.21.15.195:0/655517458 shutdown_connections 2024-05-09T05:27:24.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.928+0000 7fb51ef36700 1 -- 172.21.15.195:0/655517458 wait complete. 2024-05-09T05:27:24.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb51ef36700 1 Processor -- start 2024-05-09T05:27:24.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb51ef36700 1 -- start start 2024-05-09T05:27:24.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb51ef36700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 0x7fb51819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:24.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb51ef36700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 0x7fb518198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:24.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb51ef36700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810cfc0 con 0x7fb518108c30 2024-05-09T05:27:24.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb51ef36700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810ce40 con 0x7fb51819eb50 2024-05-09T05:27:24.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb517fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 0x7fb518198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:24.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb51ccd2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 0x7fb51819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:24.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb51ccd2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 0x7fb51819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.195:54256/0 (socket says 172.21.15.195:54256) 2024-05-09T05:27:24.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb517fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 0x7fb518198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:56382/0 (socket says 172.21.15.195:56382) 2024-05-09T05:27:24.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb51ccd2700 1 -- 172.21.15.195:0/532862809 learned_addr learned my addr 172.21.15.195:0/532862809 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:27:24.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb517fff700 1 -- 172.21.15.195:0/532862809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 msgr2=0x7fb518198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T05:27:24.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb517fff700 1 -- 172.21.15.195:0/532862809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 msgr2=0x7fb518198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T05:27:24.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.929+0000 7fb517fff700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 0x7fb518198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T05:27:24.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb517fff700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 0x7fb518198850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:27:24.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb51ccd2700 1 -- 172.21.15.195:0/532862809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 msgr2=0x7fb518198850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb51ccd2700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 0x7fb518198850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb51ccd2700 1 -- 172.21.15.195:0/532862809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51819f140 con 0x7fb518108c30 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb51ccd2700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 0x7fb51819e610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb50c005700 tx=0x7fb50c0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb515ffb700 1 -- 172.21.15.195:0/532862809 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb50c004f40 con 0x7fb518108c30 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb515ffb700 1 -- 172.21.15.195:0/532862809 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb50c00bdf0 con 0x7fb518108c30 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb515ffb700 1 -- 172.21.15.195:0/532862809 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb50c022dc0 con 0x7fb518108c30 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb518198df0 con 0x7fb518108c30 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.930+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181993a0 con 0x7fb518108c30 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.931+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb518104200 con 0x7fb518108c30 2024-05-09T05:27:24.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.931+0000 7fb515ffb700 1 -- 172.21.15.195:0/532862809 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fb50c02a070 con 0x7fb518108c30 2024-05-09T05:27:24.932 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.931+0000 7fb515ffb700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb50803e300 0x7fb508040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:24.932 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.931+0000 7fb515ffb700 1 -- 172.21.15.195:0/532862809 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb50c01e030 con 0x7fb518108c30 2024-05-09T05:27:24.932 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.932+0000 7fb517fff700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb50803e300 0x7fb508040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:24.932 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.932+0000 7fb517fff700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb50803e300 0x7fb508040810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb518101d90 tx=0x7fb504008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:24.933 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:24.937+0000 7fb515ffb700 1 -- 172.21.15.195:0/532862809 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb50c013070 con 0x7fb518108c30 2024-05-09T05:27:25.217 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.222+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb518199650 con 0x7fb518108c30 2024-05-09T05:27:25.218 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.223+0000 7fb515ffb700 1 -- 172.21.15.195:0/532862809 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fb50c031090 con 0x7fb518108c30 2024-05-09T05:27:25.218 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:27:25.218 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":2,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","modified":"2024-05-09T05:27:16.688162Z","created":"2024-05-09T05:25:30.870570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T05:27:25.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb50803e300 msgr2=0x7fb508040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb50803e300 0x7fb508040810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb518101d90 tx=0x7fb504008040 comp rx=0 tx=0).stop 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 msgr2=0x7fb51819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 0x7fb51819e610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb50c005700 tx=0x7fb50c0057e0 comp rx=0 tx=0).stop 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 shutdown_connections 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb50803e300 0x7fb508040810 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108c30 0x7fb51819e610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 --2- 172.21.15.195:0/532862809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb51819eb50 0x7fb518198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 >> 172.21.15.195:0/532862809 conn(0x7fb5180fe650 msgr2=0x7fb5180ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 shutdown_connections 2024-05-09T05:27:25.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:25.225+0000 7fb51ef36700 1 -- 172.21.15.195:0/532862809 wait complete. 2024-05-09T05:27:25.222 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 2 2024-05-09T05:27:25.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 podman[33785]: 2024-05-09 05:27:25.738839524 +0000 UTC m=+0.349157967 container start 72441f6aa892b453a4dde7f34d7757a2d62a9ae3682484a71d9b3ea79c27b3e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:27:25.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T05:27:25.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T05:27:25.768 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pidfile_write: ignore empty --pid-file 2024-05-09T05:27:25.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:25 smithi195 ceph-mon[33844]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:25.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:25 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/532862809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T05:27:26.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:25 smithi115 ceph-mon[30349]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:25 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/532862809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T05:27:26.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: load: jerasure load: lrc 2024-05-09T05:27:26.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: RocksDB version: 7.9.2 2024-05-09T05:27:26.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Git sha 0 2024-05-09T05:27:26.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T05:27:26.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: DB SUMMARY 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: DB Session ID: ZIB9DSZT17DGOMVC3DRS 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: CURRENT file: CURRENT 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.error_if_exists: 0 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.create_if_missing: 0 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.paranoid_checks: 1 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.env: 0x55aa73335500 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T05:27:26.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.info_log: 0x55aa74630020 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.statistics: (nil) 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.use_fsync: 0 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_log_file_size: 0 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.allow_fallocate: 1 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.use_direct_reads: 0 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T05:27:26.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.db_log_dir: 2024-05-09T05:27:26.165 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.wal_dir: 2024-05-09T05:27:26.165 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T05:27:26.165 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T05:27:26.165 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T05:27:26.165 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T05:27:26.166 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T05:27:26.166 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T05:27:26.166 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T05:27:26.166 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T05:27:26.166 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.write_buffer_manager: 0x55aa744fa8c0 2024-05-09T05:27:26.166 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T05:27:26.166 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.unordered_write: 0 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.row_cache: None 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.wal_filter: None 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T05:27:26.167 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.two_write_queues: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.wal_compression: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.atomic_flush: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.log_readahead_size: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T05:27:26.168 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_background_jobs: 2 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_background_compactions: -1 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_subcompactions: 1 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_open_files: -1 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T05:27:26.169 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_background_flushes: -1 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Compression algorithms supported: 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: kZSTD supported: 0 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: kXpressCompression supported: 0 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: kZlibCompression supported: 1 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: kSnappyCompression supported: 1 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: kLZ4Compression supported: 1 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: kBZip2Compression supported: 0 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T05:27:26.170 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.merge_operator: 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_filter: None 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55aa746301e0) 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks: 1 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_top_level_index_and_filter: 1 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_type: 0 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_index_type: 0 2024-05-09T05:27:26.171 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_shortening: 1 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: checksum: 4 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: no_block_cache: 0 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache: 0x55aa7452d1f0 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_name: BinnedLRUCache 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_options: 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: capacity : 536870912 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_shard_bits : 4 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: strict_capacity_limit : 0 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: high_pri_pool_ratio: 0.000 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_compressed: (nil) 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: persistent_cache: (nil) 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size: 4096 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size_deviation: 10 2024-05-09T05:27:26.172 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_restart_interval: 16 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_block_restart_interval: 1 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: metadata_block_size: 4096 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: partition_filters: 0 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: use_delta_encoding: 1 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: filter_policy: bloomfilter 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: whole_key_filtering: 1 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: verify_compression: 0 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: read_amp_bytes_per_bit: 0 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: format_version: 5 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_index_compression: 1 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_align: 0 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: max_auto_readahead_size: 262144 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: prepopulate_block_cache: 0 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: initial_auto_readahead_size: 8192 2024-05-09T05:27:26.173 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression: NoCompression 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.num_levels: 7 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T05:27:26.174 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T05:27:26.175 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T05:27:26.176 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T05:27:26.177 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.inplace_update_support: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.bloom_locality: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.max_successive_merges: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.ttl: 2592000 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T05:27:26.178 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.enable_blob_files: false 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.min_blob_size: 0 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 063bc550-51f4-4a6c-951f-1d54ebcf91cf 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715232445779674, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-09T05:27:26.179 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715232445780103, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715232445, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "063bc550-51f4-4a6c-951f-1d54ebcf91cf", "db_session_id": "ZIB9DSZT17DGOMVC3DRS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715232445780184, "job": 1, "event": "recovery_finished"} 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55aa74511c00 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: DB pointer 0x55aa79304000 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** DB Stats ** 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T05:27:26.180 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-09T05:27:26.181 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache BinnedLRUCache@0x55aa7452d1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: using public_addr v2:172.21.15.161:0/0 -> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-05-09T05:27:26.182 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: starting mon.b rank -1 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:26.183 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(???) e0 preinit fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:26.183 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).mds e1 new map 2024-05-09T05:27:26.183 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-09T05:27:26.183 INFO:journalctl@ceph.mon.b.smithi161.stdout: e1 2024-05-09T05:27:26.183 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T05:27:26.184 INFO:journalctl@ceph.mon.b.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T05:27:26.184 INFO:journalctl@ceph.mon.b.smithi161.stdout: legacy client fscid: -1 2024-05-09T05:27:26.184 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-09T05:27:26.184 INFO:journalctl@ceph.mon.b.smithi161.stdout: No filesystems configured 2024-05-09T05:27:26.184 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T05:27:26.184 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T05:27:26.184 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T05:27:26.184 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T05:27:26.184 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:26.185 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Deploying cephadm binary to smithi195 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Added host smithi195 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:26.186 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/728406180' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:26.187 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/728406180' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.188 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi115:172.21.15.115=a;smithi161:172.21.15.161=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Saving service mon spec with placement smithi115:172.21.15.115=a;smithi161:172.21.15.161=b;smithi195:172.21.15.195=c;count:3 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Deploying daemon mon.c on smithi195 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.189 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.a calling monitor election 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/3622709557' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.c calling monitor election 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T05:27:26.190 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: monmap epoch 2 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: last_changed 2024-05-09T05:27:16.688162+0000 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: min_mon_release 19 (squid) 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: election_strategy: 1 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: fsmap 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mgrmap e13: x(active, since 49s) 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: overall HEALTH_OK 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.191 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: Deploying daemon mon.b on smithi161 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/532862809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 ceph-mon[33805]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 bash[33785]: 72441f6aa892b453a4dde7f34d7757a2d62a9ae3682484a71d9b3ea79c27b3e0 2024-05-09T05:27:26.192 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:25 smithi161 systemd[1]: Started Ceph mon.b for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:27:26.834 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T05:27:26.835 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T05:27:27.082 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:27:31.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: mon.c calling monitor election 2024-05-09T05:27:31.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:27:31.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: mon.a calling monitor election 2024-05-09T05:27:31.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: monmap epoch 3 2024-05-09T05:27:31.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: last_changed 2024-05-09T05:27:25.808922+0000 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: min_mon_release 19 (squid) 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: election_strategy: 1 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: fsmap 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:31.119 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: mgrmap e13: x(active, since 58s) 2024-05-09T05:27:31.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: overall HEALTH_OK 2024-05-09T05:27:31.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:31.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:31.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:31.120 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: mon.c calling monitor election 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: mon.a calling monitor election 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: monmap epoch 3 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: last_changed 2024-05-09T05:27:25.808922+0000 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: min_mon_release 19 (squid) 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: election_strategy: 1 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: fsmap 2024-05-09T05:27:31.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:31.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: mgrmap e13: x(active, since 58s) 2024-05-09T05:27:31.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: overall HEALTH_OK 2024-05-09T05:27:31.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:31.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:31.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:31.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:31.386 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.390+0000 7fbb7ca64700 1 -- 172.21.15.195:0/3824559201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb600035c0 msgr2=0x7fbb60005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:31.386 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.390+0000 7fbb7ca64700 1 --2- 172.21.15.195:0/3824559201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb600035c0 0x7fbb60005aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbb78102a60 tx=0x7fbb6c00feb0 comp rx=0 tx=0).stop 2024-05-09T05:27:31.386 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.390+0000 7fbb7ca64700 1 -- 172.21.15.195:0/3824559201 shutdown_connections 2024-05-09T05:27:31.386 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.390+0000 7fbb7ca64700 1 --2- 172.21.15.195:0/3824559201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb600035c0 0x7fbb60005aa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:31.386 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.390+0000 7fbb7ca64700 1 -- 172.21.15.195:0/3824559201 >> 172.21.15.195:0/3824559201 conn(0x7fbb780fe670 msgr2=0x7fbb78100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:31.386 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.390+0000 7fbb7ca64700 1 -- 172.21.15.195:0/3824559201 shutdown_connections 2024-05-09T05:27:31.386 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.390+0000 7fbb7ca64700 1 -- 172.21.15.195:0/3824559201 wait complete. 2024-05-09T05:27:31.390 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.391+0000 7fbb7ca64700 1 Processor -- start 2024-05-09T05:27:31.390 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.391+0000 7fbb7ca64700 1 -- start start 2024-05-09T05:27:31.390 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7ca64700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb600035c0 0x7fbb7819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:31.390 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7ca64700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb7819ebd0 0x7fbb781988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7ca64700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb78198df0 0x7fbb78199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7ca64700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7810a990 con 0x7fbb7819ebd0 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7ca64700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7810a810 con 0x7fbb78198df0 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7ca64700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7810ab10 con 0x7fbb600035c0 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb75d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb7819ebd0 0x7fbb781988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb600035c0 0x7fbb7819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb75d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb7819ebd0 0x7fbb781988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.195:58486/0 (socket says 172.21.15.195:58486) 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb75d9b700 1 -- 172.21.15.195:0/126203069 learned_addr learned my addr 172.21.15.195:0/126203069 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb76d9d700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb78198df0 0x7fbb78199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7659c700 1 -- 172.21.15.195:0/126203069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb600035c0 msgr2=0x7fbb7819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7659c700 1 -- 172.21.15.195:0/126203069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb600035c0 msgr2=0x7fbb7819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7659c700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb600035c0 0x7fbb7819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb7659c700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb600035c0 0x7fbb7819e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:27:31.391 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb76d9d700 1 -- 172.21.15.195:0/126203069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb78198df0 msgr2=0x7fbb78199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb76d9d700 1 -- 172.21.15.195:0/126203069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb78198df0 msgr2=0x7fbb78199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb76d9d700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb78198df0 0x7fbb78199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb76d9d700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb78198df0 0x7fbb78199200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb75d9b700 1 -- 172.21.15.195:0/126203069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb600035c0 msgr2=0x7fbb7819e690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb75d9b700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb600035c0 0x7fbb7819e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb75d9b700 1 -- 172.21.15.195:0/126203069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb78198df0 msgr2=0x7fbb78199200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb75d9b700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb78198df0 0x7fbb78199200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.392+0000 7fbb75d9b700 1 -- 172.21.15.195:0/126203069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb781a5520 con 0x7fbb7819ebd0 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.393+0000 7fbb75d9b700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb7819ebd0 0x7fbb781988b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b780 tx=0x7fbb6800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.393+0000 7fbb677fe700 1 -- 172.21.15.195:0/126203069 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68004230 con 0x7fbb7819ebd0 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.393+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7806b310 con 0x7fbb7819ebd0 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.393+0000 7fbb677fe700 1 -- 172.21.15.195:0/126203069 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb680043d0 con 0x7fbb7819ebd0 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.393+0000 7fbb677fe700 1 -- 172.21.15.195:0/126203069 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68011ac0 con 0x7fbb7819ebd0 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.393+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7806b850 con 0x7fbb7819ebd0 2024-05-09T05:27:31.392 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.394+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb78064e20 con 0x7fbb7819ebd0 2024-05-09T05:27:31.393 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.394+0000 7fbb677fe700 1 -- 172.21.15.195:0/126203069 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fbb68011c80 con 0x7fbb7819ebd0 2024-05-09T05:27:31.393 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.394+0000 7fbb677fe700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbb5803e9a0 0x7fbb58040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:31.393 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.394+0000 7fbb677fe700 1 -- 172.21.15.195:0/126203069 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbb68052470 con 0x7fbb7819ebd0 2024-05-09T05:27:31.393 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.395+0000 7fbb7659c700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbb5803e9a0 0x7fbb58040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:31.393 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.395+0000 7fbb7659c700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbb5803e9a0 0x7fbb58040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c005ed0 tx=0x7fbb6c005e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:31.395 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.400+0000 7fbb677fe700 1 -- 172.21.15.195:0/126203069 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb68015df0 con 0x7fbb7819ebd0 2024-05-09T05:27:31.706 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.710+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbb7806bba0 con 0x7fbb7819ebd0 2024-05-09T05:27:31.706 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.711+0000 7fbb677fe700 1 -- 172.21.15.195:0/126203069 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fbb68003bc0 con 0x7fbb7819ebd0 2024-05-09T05:27:31.706 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-05-09T05:27:31.706 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":3,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","modified":"2024-05-09T05:27:25.808922Z","created":"2024-05-09T05:25:30.870570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T05:27:31.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbb5803e9a0 msgr2=0x7fbb58040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:31.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbb5803e9a0 0x7fbb58040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c005ed0 tx=0x7fbb6c005e60 comp rx=0 tx=0).stop 2024-05-09T05:27:31.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb7819ebd0 msgr2=0x7fbb781988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:31.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb7819ebd0 0x7fbb781988b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b780 tx=0x7fbb6800bb40 comp rx=0 tx=0).stop 2024-05-09T05:27:31.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 shutdown_connections 2024-05-09T05:27:31.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbb5803e9a0 0x7fbb58040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:31.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb78198df0 0x7fbb78199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:31.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb600035c0 0x7fbb7819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:31.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 --2- 172.21.15.195:0/126203069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb7819ebd0 0x7fbb781988b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:31.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 >> 172.21.15.195:0/126203069 conn(0x7fbb780fe670 msgr2=0x7fbb78101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:31.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 shutdown_connections 2024-05-09T05:27:31.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:31.714+0000 7fbb7ca64700 1 -- 172.21.15.195:0/126203069 wait complete. 2024-05-09T05:27:31.710 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 3 2024-05-09T05:27:32.096 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:31 smithi195 ceph-mon[33844]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:32.096 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:31 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:32.096 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:31 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/126203069' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T05:27:32.096 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:31 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:32.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:31 smithi115 ceph-mon[30349]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:32.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:31 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:32.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:31 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/126203069' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T05:27:32.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:31 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:32.811 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T05:27:32.811 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T05:27:33.063 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:27:33.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: mon.c calling monitor election 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: mon.a calling monitor election 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:33.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: monmap epoch 3 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: last_changed 2024-05-09T05:27:25.808922+0000 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: min_mon_release 19 (squid) 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: election_strategy: 1 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: fsmap 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: mgrmap e13: x(active, since 58s) 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: overall HEALTH_OK 2024-05-09T05:27:33.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:33.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:33.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/126203069' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T05:27:33.164 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:33.785 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: mon.b calling monitor election 2024-05-09T05:27:33.785 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: Updating smithi115:/etc/ceph/ceph.conf 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: mon.b calling monitor election 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: mon.a calling monitor election 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: mon.c calling monitor election 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: monmap epoch 3 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:33.786 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: last_changed 2024-05-09T05:27:25.808922+0000 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: min_mon_release 19 (squid) 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: election_strategy: 1 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: fsmap 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: mgrmap e13: x(active, since 60s) 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: overall HEALTH_OK 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:33.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:33 smithi115 ceph-mon[30349]: Reconfiguring daemon mon.a on smithi115 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: mon.b calling monitor election 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: Updating smithi115:/etc/ceph/ceph.conf 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: mon.b calling monitor election 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: mon.a calling monitor election 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: mon.c calling monitor election 2024-05-09T05:27:33.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: monmap epoch 3 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: last_changed 2024-05-09T05:27:25.808922+0000 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: min_mon_release 19 (squid) 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: election_strategy: 1 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: fsmap 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: mgrmap e13: x(active, since 60s) 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: overall HEALTH_OK 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.869 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:33.870 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:33 smithi195 ceph-mon[33844]: Reconfiguring daemon mon.a on smithi115 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: mon.b calling monitor election 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: Updating smithi115:/etc/ceph/ceph.conf 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: mon.b calling monitor election 2024-05-09T05:27:33.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: mon.a calling monitor election 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: mon.c calling monitor election 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: monmap epoch 3 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: last_changed 2024-05-09T05:27:25.808922+0000 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: created 2024-05-09T05:25:30.870570+0000 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: min_mon_release 19 (squid) 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: election_strategy: 1 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: 0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: fsmap 2024-05-09T05:27:33.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: mgrmap e13: x(active, since 60s) 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: overall HEALTH_OK 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.914 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:33.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T05:27:33.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:33.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T05:27:33.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:33.915 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:33 smithi161 ceph-mon[33805]: Reconfiguring daemon mon.a on smithi115 2024-05-09T05:27:34.747 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:34.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:34.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:27:35.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.612+0000 7fd764e9a700 1 -- 172.21.15.115:0/3589999147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd760108a50 msgr2=0x7fd760108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:35.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.612+0000 7fd764e9a700 1 --2- 172.21.15.115:0/3589999147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd760108a50 0x7fd760108e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd750009910 tx=0x7fd750009c20 comp rx=0 tx=0).stop 2024-05-09T05:27:35.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.612+0000 7fd764e9a700 1 -- 172.21.15.115:0/3589999147 shutdown_connections 2024-05-09T05:27:35.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.612+0000 7fd764e9a700 1 --2- 172.21.15.115:0/3589999147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd760108a50 0x7fd760108e60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:35.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.612+0000 7fd764e9a700 1 -- 172.21.15.115:0/3589999147 >> 172.21.15.115:0/3589999147 conn(0x7fd7600fe580 msgr2=0x7fd7601009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:35.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.612+0000 7fd764e9a700 1 -- 172.21.15.115:0/3589999147 shutdown_connections 2024-05-09T05:27:35.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.612+0000 7fd764e9a700 1 -- 172.21.15.115:0/3589999147 wait complete. 2024-05-09T05:27:35.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd764e9a700 1 Processor -- start 2024-05-09T05:27:35.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd764e9a700 1 -- start start 2024-05-09T05:27:35.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd764e9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd760108a50 0x7fd76019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:35.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd764e9a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76019af00 0x7fd76019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:35.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd764e9a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7601a3a30 0x7fd7601a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd764e9a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd76010ce10 con 0x7fd7601a3a30 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd764e9a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd76010cc90 con 0x7fd760108a50 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd764e9a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd76010cf90 con 0x7fd76019af00 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd75ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7601a3a30 0x7fd7601a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd75ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7601a3a30 0x7fd7601a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37180/0 (socket says 172.21.15.115:37180) 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd75ed9d700 1 -- 172.21.15.115:0/4041593189 learned_addr learned my addr 172.21.15.115:0/4041593189 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.613+0000 7fd75e59c700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd760108a50 0x7fd76019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd75dd9b700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76019af00 0x7fd76019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd75ed9d700 1 -- 172.21.15.115:0/4041593189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76019af00 msgr2=0x7fd76019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd75ed9d700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76019af00 0x7fd76019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd75ed9d700 1 -- 172.21.15.115:0/4041593189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd760108a50 msgr2=0x7fd76019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd75ed9d700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd760108a50 0x7fd76019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd75ed9d700 1 -- 172.21.15.115:0/4041593189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7601a65e0 con 0x7fd7601a3a30 2024-05-09T05:27:35.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd75dd9b700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76019af00 0x7fd76019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:27:35.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd75ed9d700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7601a3a30 0x7fd7601a5e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd75800ba50 tx=0x7fd75800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:35.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd74f7fe700 1 -- 172.21.15.115:0/4041593189 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758019070 con 0x7fd7601a3a30 2024-05-09T05:27:35.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd74f7fe700 1 -- 172.21.15.115:0/4041593189 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd75800ce40 con 0x7fd7601a3a30 2024-05-09T05:27:35.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7601a68d0 con 0x7fd7601a3a30 2024-05-09T05:27:35.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd74f7fe700 1 -- 172.21.15.115:0/4041593189 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758007350 con 0x7fd7601a3a30 2024-05-09T05:27:35.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.614+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7601a6e10 con 0x7fd7601a3a30 2024-05-09T05:27:35.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.615+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd760064e20 con 0x7fd7601a3a30 2024-05-09T05:27:35.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.615+0000 7fd74f7fe700 1 -- 172.21.15.115:0/4041593189 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fd75801c070 con 0x7fd7601a3a30 2024-05-09T05:27:35.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.615+0000 7fd74f7fe700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd74803e9a0 0x7fd748040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:35.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.615+0000 7fd74f7fe700 1 -- 172.21.15.115:0/4041593189 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd758051de0 con 0x7fd7601a3a30 2024-05-09T05:27:35.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.617+0000 7fd75e59c700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd74803e9a0 0x7fd748040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:35.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.617+0000 7fd75e59c700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd74803e9a0 0x7fd748040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd750004690 tx=0x7fd750005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:35.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.621+0000 7fd74f7fe700 1 -- 172.21.15.115:0/4041593189 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd758014440 con 0x7fd7601a3a30 2024-05-09T05:27:35.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.823+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd7601a7230 con 0x7fd7601a3a30 2024-05-09T05:27:35.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.824+0000 7fd74f7fe700 1 -- 172.21.15.115:0/4041593189 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fd758013a90 con 0x7fd7601a3a30 2024-05-09T05:27:35.820 INFO:teuthology.orchestra.run.smithi115.stdout:# minimal ceph.conf for 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:35.820 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-05-09T05:27:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:27:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: mon_host = [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd74803e9a0 msgr2=0x7fd748040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd74803e9a0 0x7fd748040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd750004690 tx=0x7fd750005fb0 comp rx=0 tx=0).stop 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7601a3a30 msgr2=0x7fd7601a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7601a3a30 0x7fd7601a5e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd75800ba50 tx=0x7fd75800bd60 comp rx=0 tx=0).stop 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 shutdown_connections 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd74803e9a0 0x7fd748040eb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd760108a50 0x7fd76019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7601a3a30 0x7fd7601a5e50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 --2- 172.21.15.115:0/4041593189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76019af00 0x7fd76019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 >> 172.21.15.115:0/4041593189 conn(0x7fd7600fe580 msgr2=0x7fd7600ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 shutdown_connections 2024-05-09T05:27:35.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:35.827+0000 7fd764e9a700 1 -- 172.21.15.115:0/4041593189 wait complete. 2024-05-09T05:27:36.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:36 smithi115 ceph-mon[30349]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:36.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:36.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:36.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:36.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T05:27:36.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:36.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:36 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4041593189' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:36.420 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T05:27:36.420 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:27:36.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T05:27:36.450 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:27:36.450 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:27:36.517 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:27:36.517 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T05:27:36.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:36 smithi161 ceph-mon[33805]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:36.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:36.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:36.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:36.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T05:27:36.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:36.542 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:36 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4041593189' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:36.549 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:27:36.549 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:27:36.614 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:27:36.614 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T05:27:36.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:36 smithi195 ceph-mon[33844]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:36.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:36.618 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:36.618 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:36.618 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T05:27:36.618 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:36.618 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:36 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4041593189' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:36.647 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:27:36.647 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:27:36.717 INFO:tasks.cephadm:Adding mgr.x on smithi115 2024-05-09T05:27:36.717 INFO:tasks.cephadm:Adding mgr.y on smithi161 2024-05-09T05:27:36.717 INFO:tasks.cephadm:Adding mgr.z on smithi195 2024-05-09T05:27:36.717 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi115=x;smithi161=y;smithi195=z' 2024-05-09T05:27:36.996 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:27:37.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:37 smithi195 ceph-mon[33844]: Reconfiguring mon.b (monmap changed)... 2024-05-09T05:27:37.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:37 smithi195 ceph-mon[33844]: Reconfiguring daemon mon.b on smithi161 2024-05-09T05:27:37.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:37 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:37.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:37 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:37.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:37 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:37.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:37 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T05:27:37.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:37 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:37.623 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:37 smithi115 ceph-mon[30349]: Reconfiguring mon.b (monmap changed)... 2024-05-09T05:27:37.624 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:37 smithi115 ceph-mon[30349]: Reconfiguring daemon mon.b on smithi161 2024-05-09T05:27:37.624 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:37 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:37.624 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:37 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:37.624 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:37 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:37.624 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:37 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T05:27:37.624 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:37 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:37.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:37 smithi161 ceph-mon[33805]: Reconfiguring mon.b (monmap changed)... 2024-05-09T05:27:37.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:37 smithi161 ceph-mon[33805]: Reconfiguring daemon mon.b on smithi161 2024-05-09T05:27:37.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:37 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:37.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:37 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:37.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:37 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T05:27:37.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:37 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T05:27:37.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:37 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:38.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:38 smithi195 ceph-mon[33844]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:38.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:38 smithi195 ceph-mon[33844]: Reconfiguring mon.c (monmap changed)... 2024-05-09T05:27:38.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:38 smithi195 ceph-mon[33844]: Reconfiguring daemon mon.c on smithi195 2024-05-09T05:27:38.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:38 smithi115 ceph-mon[30349]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:38.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:38 smithi115 ceph-mon[30349]: Reconfiguring mon.c (monmap changed)... 2024-05-09T05:27:38.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:38 smithi115 ceph-mon[30349]: Reconfiguring daemon mon.c on smithi195 2024-05-09T05:27:38.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:38 smithi161 ceph-mon[33805]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:38.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:38 smithi161 ceph-mon[33805]: Reconfiguring mon.c (monmap changed)... 2024-05-09T05:27:38.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:38 smithi161 ceph-mon[33805]: Reconfiguring daemon mon.c on smithi195 2024-05-09T05:27:38.968 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.971+0000 7f6d6442b700 1 -- 172.21.15.195:0/2532502497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c103d10 msgr2=0x7f6d5c104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.971+0000 7f6d6442b700 1 --2- 172.21.15.195:0/2532502497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c103d10 0x7f6d5c104120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00b0a0 tx=0x7f6d4c00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.971+0000 7f6d6442b700 1 -- 172.21.15.195:0/2532502497 shutdown_connections 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.971+0000 7f6d6442b700 1 --2- 172.21.15.195:0/2532502497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c103d10 0x7f6d5c104120 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.971+0000 7f6d6442b700 1 -- 172.21.15.195:0/2532502497 >> 172.21.15.195:0/2532502497 conn(0x7f6d5c0ff350 msgr2=0x7f6d5c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.971+0000 7f6d6442b700 1 -- 172.21.15.195:0/2532502497 shutdown_connections 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.971+0000 7f6d6442b700 1 -- 172.21.15.195:0/2532502497 wait complete. 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d6442b700 1 Processor -- start 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d6442b700 1 -- start start 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d6442b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d5c103d10 0x7f6d5c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:38.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d6442b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c199da0 0x7f6d5c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d6442b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d5c19f270 0x7f6d5c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d6442b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c109b40 con 0x7f6d5c199da0 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d6442b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c1099c0 con 0x7f6d5c103d10 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d6442b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c109cc0 con 0x7f6d5c19f270 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d629c8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d5c19f270 0x7f6d5c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d629c8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d5c19f270 0x7f6d5c1a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:47866/0 (socket says 172.21.15.195:47866) 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d629c8700 1 -- 172.21.15.195:0/2589158239 learned_addr learned my addr 172.21.15.195:0/2589158239 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.972+0000 7f6d619c6700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c199da0 0x7f6d5c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d621c7700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d5c103d10 0x7f6d5c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d629c8700 1 -- 172.21.15.195:0/2589158239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d5c103d10 msgr2=0x7f6d5c199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d629c8700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d5c103d10 0x7f6d5c199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d629c8700 1 -- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c199da0 msgr2=0x7f6d5c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d629c8700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c199da0 0x7f6d5c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:38.970 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d629c8700 1 -- 172.21.15.195:0/2589158239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c1a1c00 con 0x7f6d5c19f270 2024-05-09T05:27:38.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d619c6700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c199da0 0x7f6d5c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:27:38.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d629c8700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d5c19f270 0x7f6d5c1a1690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400b550 tx=0x7f6d5400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:38.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d5b7fe700 1 -- 172.21.15.195:0/2589158239 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54015020 con 0x7f6d5c19f270 2024-05-09T05:27:38.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c1a1ef0 con 0x7f6d5c19f270 2024-05-09T05:27:38.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d5b7fe700 1 -- 172.21.15.195:0/2589158239 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d54011cc0 con 0x7f6d5c19f270 2024-05-09T05:27:38.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d5b7fe700 1 -- 172.21.15.195:0/2589158239 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54010c90 con 0x7f6d5c19f270 2024-05-09T05:27:38.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.973+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c1a23c0 con 0x7f6d5c19f270 2024-05-09T05:27:38.972 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.974+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c1a2650 con 0x7f6d5c19f270 2024-05-09T05:27:38.972 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.974+0000 7f6d5b7fe700 1 -- 172.21.15.195:0/2589158239 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f6d54018020 con 0x7f6d5c19f270 2024-05-09T05:27:38.972 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.974+0000 7f6d5b7fe700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d4403e9a0 0x7f6d44040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:38.972 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.975+0000 7f6d5b7fe700 1 -- 172.21.15.195:0/2589158239 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6d54052010 con 0x7f6d5c19f270 2024-05-09T05:27:38.972 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.975+0000 7f6d621c7700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d4403e9a0 0x7f6d44040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:38.972 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.976+0000 7f6d621c7700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d4403e9a0 0x7f6d44040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00d010 tx=0x7f6d4c005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:38.975 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:38.980+0000 7f6d5b7fe700 1 -- 172.21.15.195:0/2589158239 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d5401f020 con 0x7f6d5c19f270 2024-05-09T05:27:39.194 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.199+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi115=x;smithi161=y;smithi195=z", "target": ["mon-mgr", ""]}) -- 0x7f6d5c101160 con 0x7f6d4403e9a0 2024-05-09T05:27:39.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.206+0000 7f6d5b7fe700 1 -- 172.21.15.195:0/2589158239 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6d5c101160 con 0x7f6d4403e9a0 2024-05-09T05:27:39.202 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mgr update... 2024-05-09T05:27:39.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d4403e9a0 msgr2=0x7f6d44040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:39.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d4403e9a0 0x7f6d44040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00d010 tx=0x7f6d4c005040 comp rx=0 tx=0).stop 2024-05-09T05:27:39.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d5c19f270 msgr2=0x7f6d5c1a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:39.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d5c19f270 0x7f6d5c1a1690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400b550 tx=0x7f6d5400b860 comp rx=0 tx=0).stop 2024-05-09T05:27:39.204 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 shutdown_connections 2024-05-09T05:27:39.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d4403e9a0 0x7f6d44040eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:39.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d5c103d10 0x7f6d5c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:39.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d5c19f270 0x7f6d5c1a1690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:39.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 --2- 172.21.15.195:0/2589158239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d5c199da0 0x7f6d5c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:39.205 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 >> 172.21.15.195:0/2589158239 conn(0x7f6d5c0ff350 msgr2=0x7f6d5c100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:39.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 shutdown_connections 2024-05-09T05:27:39.206 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:27:39.209+0000 7f6d6442b700 1 -- 172.21.15.195:0/2589158239 wait complete. 2024-05-09T05:27:39.893 DEBUG:teuthology.orchestra.run.smithi161:mgr.y> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.y.service 2024-05-09T05:27:39.896 DEBUG:teuthology.orchestra.run.smithi195:mgr.z> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.z.service 2024-05-09T05:27:39.898 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T05:27:39.898 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:27:39.898 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T05:27:39.914 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T05:27:39.914 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-05-09T05:27:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T05:27:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:39.970 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T05:27:39.970 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:39.970 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:39.970 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-05-09 05:26:42.498483119 +0000 2024-05-09T05:27:39.970 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-05-09 05:22:10.016047403 +0000 2024-05-09T05:27:39.970 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-05-09 05:22:10.016047403 +0000 2024-05-09T05:27:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-05-09 05:22:10.016047403 +0000 2024-05-09T05:27:39.971 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T05:27:40.036 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-05-09T05:27:40.036 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-05-09T05:27:40.036 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000321485 s, 1.6 MB/s 2024-05-09T05:27:40.038 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T05:27:40.096 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-05-09T05:27:40.152 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T05:27:40.152 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:40.152 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-09T05:27:40.152 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:40.152 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:40.152 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-05-09 05:26:42.525482659 +0000 2024-05-09T05:27:40.152 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-05-09 05:22:10.364041473 +0000 2024-05-09T05:27:40.152 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-05-09 05:22:10.364041473 +0000 2024-05-09T05:27:40.152 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-05-09 05:22:10.364041473 +0000 2024-05-09T05:27:40.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T05:27:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-05-09T05:27:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-05-09T05:27:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000195681 s, 2.6 MB/s 2024-05-09T05:27:40.217 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T05:27:40.273 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-05-09T05:27:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T05:27:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:40.329 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T05:27:40.329 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:40.329 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:40.329 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-05-09 05:26:42.553482183 +0000 2024-05-09T05:27:40.329 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-05-09 05:22:10.706035646 +0000 2024-05-09T05:27:40.329 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-05-09 05:22:10.706035646 +0000 2024-05-09T05:27:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-05-09 05:22:10.706035646 +0000 2024-05-09T05:27:40.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T05:27:40.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:40.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi115=x;smithi161=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:27:40.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: Saving service mgr spec with placement smithi115=x;smithi161=y;smithi195=z;count:3 2024-05-09T05:27:40.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T05:27:40.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:40.393 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-05-09T05:27:40.393 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-05-09T05:27:40.393 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000213181 s, 2.4 MB/s 2024-05-09T05:27:40.394 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T05:27:40.450 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-05-09T05:27:40.506 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T05:27:40.506 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:40.507 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T05:27:40.507 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:40.507 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:40.507 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-05-09 05:26:42.577481774 +0000 2024-05-09T05:27:40.507 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-05-09 05:22:11.074029375 +0000 2024-05-09T05:27:40.507 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-05-09 05:22:11.074029375 +0000 2024-05-09T05:27:40.507 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-05-09 05:22:11.074029375 +0000 2024-05-09T05:27:40.507 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T05:27:40.571 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-05-09T05:27:40.571 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-05-09T05:27:40.571 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000212702 s, 2.4 MB/s 2024-05-09T05:27:40.572 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T05:27:40.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi115=x;smithi161=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: Saving service mgr spec with placement smithi115=x;smithi161=y;smithi195=z;count:3 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T05:27:40.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T05:27:40.618 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T05:27:40.618 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:40.628 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-09T05:27:40.628 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T05:27:40.648 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T05:27:40.648 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-05-09T05:27:40.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi115=x;smithi161=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: Saving service mgr spec with placement smithi115=x;smithi161=y;smithi195=z;count:3 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T05:27:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:40.680 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T05:27:40.680 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:40.680 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-09T05:27:40.680 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:40.680 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:40.680 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-09 05:27:25.207429408 +0000 2024-05-09T05:27:40.680 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-09 05:22:12.686002004 +0000 2024-05-09T05:27:40.680 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-09 05:22:12.686002004 +0000 2024-05-09T05:27:40.680 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-05-09 05:22:12.686002004 +0000 2024-05-09T05:27:40.681 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T05:27:40.747 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-09T05:27:40.747 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-09T05:27:40.747 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000280532 s, 1.8 MB/s 2024-05-09T05:27:40.748 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T05:27:40.807 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-05-09T05:27:40.864 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T05:27:40.865 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:40.865 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-09T05:27:40.865 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:40.865 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:40.865 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-09 05:27:25.208429388 +0000 2024-05-09T05:27:40.865 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-09 05:22:13.047994284 +0000 2024-05-09T05:27:40.865 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-09 05:22:13.047994284 +0000 2024-05-09T05:27:40.865 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-05-09 05:22:13.047994284 +0000 2024-05-09T05:27:40.865 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T05:27:40.930 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-09T05:27:40.930 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-09T05:27:40.930 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000220418 s, 2.3 MB/s 2024-05-09T05:27:40.931 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T05:27:40.990 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-05-09T05:27:41.047 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T05:27:41.047 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:41.047 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-09T05:27:41.047 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:41.047 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:41.047 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-09 05:27:25.208429388 +0000 2024-05-09T05:27:41.047 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-09 05:22:13.385987075 +0000 2024-05-09T05:27:41.047 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-09 05:22:13.385987075 +0000 2024-05-09T05:27:41.047 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-05-09 05:22:13.385987075 +0000 2024-05-09T05:27:41.048 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T05:27:41.111 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-09T05:27:41.111 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-09T05:27:41.112 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000174348 s, 2.9 MB/s 2024-05-09T05:27:41.113 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T05:27:41.170 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-05-09T05:27:41.226 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T05:27:41.227 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:41.227 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-09T05:27:41.227 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:41.227 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:41.227 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-09 05:27:25.209429367 +0000 2024-05-09T05:27:41.227 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-09 05:22:13.739979526 +0000 2024-05-09T05:27:41.227 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-09 05:22:13.739979526 +0000 2024-05-09T05:27:41.227 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-05-09 05:22:13.739979526 +0000 2024-05-09T05:27:41.227 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T05:27:41.289 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-09T05:27:41.290 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-09T05:27:41.290 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000200777 s, 2.6 MB/s 2024-05-09T05:27:41.291 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T05:27:41.348 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-05-09T05:27:41.349 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T05:27:41.366 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T05:27:41.366 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-05-09T05:27:41.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:41 smithi195 ceph-mon[33844]: Deploying daemon mgr.z on smithi195 2024-05-09T05:27:41.382 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T05:27:41.382 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:41.382 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-09T05:27:41.382 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:41.382 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:41.382 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-09 05:27:15.623088454 +0000 2024-05-09T05:27:41.382 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-09 05:22:10.640259658 +0000 2024-05-09T05:27:41.382 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-09 05:22:10.640259658 +0000 2024-05-09T05:27:41.382 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-05-09 05:22:10.640259658 +0000 2024-05-09T05:27:41.383 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T05:27:41.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:41 smithi161 ceph-mon[33805]: Deploying daemon mgr.z on smithi195 2024-05-09T05:27:41.447 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-09T05:27:41.447 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-09T05:27:41.447 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000114146 s, 4.5 MB/s 2024-05-09T05:27:41.448 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T05:27:41.506 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-05-09T05:27:41.562 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T05:27:41.562 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:41.562 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T05:27:41.562 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:41.562 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:41.562 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-09 05:27:15.623088454 +0000 2024-05-09T05:27:41.562 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-09 05:22:11.008252657 +0000 2024-05-09T05:27:41.562 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-09 05:22:11.008252657 +0000 2024-05-09T05:27:41.562 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-05-09 05:22:11.008252657 +0000 2024-05-09T05:27:41.563 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T05:27:41.626 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-09T05:27:41.626 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-09T05:27:41.626 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 8.8034e-05 s, 5.8 MB/s 2024-05-09T05:27:41.627 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T05:27:41.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:41 smithi115 ceph-mon[30349]: Deploying daemon mgr.z on smithi195 2024-05-09T05:27:41.685 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-05-09T05:27:41.740 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T05:27:41.741 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:41.741 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T05:27:41.741 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:41.741 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:41.741 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-09 05:27:15.623088454 +0000 2024-05-09T05:27:41.741 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-09 05:22:11.344246264 +0000 2024-05-09T05:27:41.741 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-09 05:22:11.344246264 +0000 2024-05-09T05:27:41.741 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-05-09 05:22:11.344246264 +0000 2024-05-09T05:27:41.741 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T05:27:41.911 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-09T05:27:41.911 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-09T05:27:41.911 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 9.3676e-05 s, 5.5 MB/s 2024-05-09T05:27:41.912 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T05:27:41.927 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-05-09T05:27:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T05:27:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T05:27:41.985 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-09T05:27:41.985 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:27:41.985 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T05:27:41.986 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-05-09 05:27:41.873618697 +0000 2024-05-09T05:27:41.986 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-05-09 05:22:11.686239758 +0000 2024-05-09T05:27:41.986 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-05-09 05:22:11.686239758 +0000 2024-05-09T05:27:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-05-09 05:22:11.686239758 +0000 2024-05-09T05:27:41.986 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T05:27:42.105 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-05-09T05:27:42.105 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-05-09T05:27:42.105 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000111863 s, 4.6 MB/s 2024-05-09T05:27:42.107 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T05:27:42.124 INFO:tasks.cephadm:Deploying osd.0 on smithi115 with /dev/vg_nvme/lv_4... 2024-05-09T05:27:42.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T05:27:42.325 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:42 smithi195 ceph-mon[33844]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:42.369 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:27:42.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:42 smithi115 ceph-mon[30349]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:42.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:42 smithi161 ceph-mon[33805]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:42.866 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:42 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:42.737+0000 7f0205319200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T05:27:43.354 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:42 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:42.932+0000 7f0205319200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:27:43.354 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:43 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:43.038+0000 7f0205319200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T05:27:43.354 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:43 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:43.358+0000 7f0205319200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T05:27:43.617 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:43 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:43.448+0000 7f0205319200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T05:27:43.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:43 smithi115 ceph-mon[30349]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:43.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:43 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:43.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:43 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:43.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:43 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:43.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:43 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T05:27:43.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:43 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T05:27:43.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:43 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T05:27:43.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:43 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:43.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:43 smithi115 ceph-mon[30349]: Deploying daemon mgr.y on smithi161 2024-05-09T05:27:44.058 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:43 smithi161 ceph-mon[33805]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:44.058 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:43 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:44.058 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:43 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:44.058 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:43 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:44.059 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:43 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T05:27:44.059 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:43 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T05:27:44.059 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:43 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T05:27:44.059 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:43 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:44.059 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:43 smithi161 ceph-mon[33805]: Deploying daemon mgr.y on smithi161 2024-05-09T05:27:44.091 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T05:27:44.091 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T05:27:44.091 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-05-09T05:27:44.091 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-05-09T05:27:44.091 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113439 s, 924 MB/s 2024-05-09T05:27:44.091 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-05-09T05:27:44.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:43 smithi195 ceph-mon[33844]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:44.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:43 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:44.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:43 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:44.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:43 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:44.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:43 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T05:27:44.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:43 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T05:27:44.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:43 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T05:27:44.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:43 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:44.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:43 smithi195 ceph-mon[33844]: Deploying daemon mgr.y on smithi161 2024-05-09T05:27:44.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2024-05-09T05:27:44.908 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:27:44.989 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:44 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:44.652+0000 7f0205319200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T05:27:44.989 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:44 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:44.822+0000 7f0205319200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T05:27:44.989 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:44 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:44.993+0000 7f0205319200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T05:27:45.058 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:44 smithi161 systemd[1]: Starting Ceph mgr.y for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:27:45.321 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 podman[35189]: 2024-05-09 05:27:45.060248999 +0000 UTC m=+0.105045480 container create 550fab3ac83c87596e06f66c9c0a6f6dfc5680c5c015c96e4d57dbe12004400d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:27:45.321 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 podman[35189]: 2024-05-09 05:27:44.972935489 +0000 UTC m=+0.017731979 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:27:45.321 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 podman[35189]: 2024-05-09 05:27:45.2876014 +0000 UTC m=+0.332397872 container init 550fab3ac83c87596e06f66c9c0a6f6dfc5680c5c015c96e4d57dbe12004400d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T05:27:45.321 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 podman[35189]: 2024-05-09 05:27:45.290727367 +0000 UTC m=+0.335523840 container start 550fab3ac83c87596e06f66c9c0a6f6dfc5680c5c015c96e4d57dbe12004400d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T05:27:45.577 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:45 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:45.282+0000 7f0205319200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T05:27:45.577 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:45 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:45.375+0000 7f0205319200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T05:27:45.577 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:45 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:45.582+0000 7f0205319200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T05:27:45.645 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 ceph-mgr[35208]: -- 172.21.15.161:0/1447992801 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56020f8345a0 con 0x5602145be000 2024-05-09T05:27:45.646 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 bash[35189]: 550fab3ac83c87596e06f66c9c0a6f6dfc5680c5c015c96e4d57dbe12004400d 2024-05-09T05:27:45.646 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 systemd[1]: Started Ceph mgr.y for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:27:45.646 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:45.461+0000 7f026d3f4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T05:27:45.646 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:45.650+0000 7f026d3f4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:27:45.866 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:45 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:45.783+0000 7f0205319200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:27:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:45 smithi195 ceph-mon[33844]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:45 smithi115 ceph-mon[30349]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:45.911 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:45 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:45.760+0000 7f026d3f4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T05:27:45.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:45 smithi161 ceph-mon[33805]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:45.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:45.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:46.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.348+0000 7effbbcae700 1 -- 172.21.15.115:0/1015165021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 msgr2=0x7effb41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:46.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.348+0000 7effbbcae700 1 --2- 172.21.15.115:0/1015165021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 0x7effb41047a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effb00098f0 tx=0x7effb0009c00 comp rx=0 tx=0).stop 2024-05-09T05:27:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.348+0000 7effbbcae700 1 -- 172.21.15.115:0/1015165021 shutdown_connections 2024-05-09T05:27:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.348+0000 7effbbcae700 1 --2- 172.21.15.115:0/1015165021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effb4103260 0x7effb4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.348+0000 7effbbcae700 1 --2- 172.21.15.115:0/1015165021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb4104d30 0x7effb410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.348+0000 7effbbcae700 1 --2- 172.21.15.115:0/1015165021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 0x7effb41047a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.348+0000 7effbbcae700 1 -- 172.21.15.115:0/1015165021 >> 172.21.15.115:0/1015165021 conn(0x7effb40fe8c0 msgr2=0x7effb4100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:27:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.348+0000 7effbbcae700 1 -- 172.21.15.115:0/1015165021 shutdown_connections 2024-05-09T05:27:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.348+0000 7effbbcae700 1 -- 172.21.15.115:0/1015165021 wait complete. 2024-05-09T05:27:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effbbcae700 1 Processor -- start 2024-05-09T05:27:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effbbcae700 1 -- start start 2024-05-09T05:27:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effbbcae700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb4103260 0x7effb40764b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effbbcae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 0x7effb419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effbbcae700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effb4104d30 0x7effb41a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effbbcae700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7effb410ebd0 con 0x7effb4104390 2024-05-09T05:27:46.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effbbcae700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7effb410ea50 con 0x7effb4104d30 2024-05-09T05:27:46.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effbbcae700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7effb410ed50 con 0x7effb4103260 2024-05-09T05:27:46.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9249700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 0x7effb419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:46.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9249700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 0x7effb419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55272/0 (socket says 172.21.15.115:55272) 2024-05-09T05:27:46.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9249700 1 -- 172.21.15.115:0/1088810098 learned_addr learned my addr 172.21.15.115:0/1088810098 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:27:46.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9a4a700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb4103260 0x7effb40764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:46.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effba24b700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effb4104d30 0x7effb41a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9249700 1 -- 172.21.15.115:0/1088810098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb4103260 msgr2=0x7effb40764b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9249700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb4103260 0x7effb40764b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9249700 1 -- 172.21.15.115:0/1088810098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effb4104d30 msgr2=0x7effb41a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:27:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9249700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effb4104d30 0x7effb41a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:27:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9249700 1 -- 172.21.15.115:0/1088810098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb41a2430 con 0x7effb4104390 2024-05-09T05:27:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effb9a4a700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb4103260 0x7effb40764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:27:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.349+0000 7effba24b700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effb4104d30 0x7effb41a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:27:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.350+0000 7effb9249700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 0x7effb419f630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7effb0005e50 tx=0x7effb0009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.350+0000 7effaaffd700 1 -- 172.21.15.115:0/1088810098 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0026070 con 0x7effb4104390 2024-05-09T05:27:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.350+0000 7effaaffd700 1 -- 172.21.15.115:0/1088810098 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7effb0005040 con 0x7effb4104390 2024-05-09T05:27:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.350+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb4199400 con 0x7effb4104390 2024-05-09T05:27:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.350+0000 7effaaffd700 1 -- 172.21.15.115:0/1088810098 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0004a70 con 0x7effb4104390 2024-05-09T05:27:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.350+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb4199930 con 0x7effb4104390 2024-05-09T05:27:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.351+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb40512c0 con 0x7effb4104390 2024-05-09T05:27:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.351+0000 7effaaffd700 1 -- 172.21.15.115:0/1088810098 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7effb0029070 con 0x7effb4104390 2024-05-09T05:27:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.351+0000 7effaaffd700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7effa003e750 0x7effa0040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:27:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.351+0000 7effaaffd700 1 -- 172.21.15.115:0/1088810098 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7effb0053620 con 0x7effb4104390 2024-05-09T05:27:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.351+0000 7effb9a4a700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7effa003e750 0x7effa0040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:27:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.352+0000 7effb9a4a700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7effa003e750 0x7effa0040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7effa4009750 tx=0x7effa4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:27:46.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.356+0000 7effaaffd700 1 -- 172.21.15.115:0/1088810098 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7effb0013030 con 0x7effb4104390 2024-05-09T05:27:46.411 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:46 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:46.092+0000 7f026d3f4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T05:27:46.411 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:46 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:46.197+0000 7f026d3f4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T05:27:46.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:46.570+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7effb4063c00 con 0x7effa003e750 2024-05-09T05:27:46.776 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:46 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:46.424+0000 7f0205319200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T05:27:46.776 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:46 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:46.656+0000 7f0205319200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T05:27:46.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:46 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:27:46.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:46 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:27:46.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:46 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:47.088 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:46 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:27:47.088 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:46 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:27:47.088 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:46 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:46 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:27:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:46 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:27:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:46 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:47.366 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:47 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:47.092+0000 7f0205319200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T05:27:47.367 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:47 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:47.190+0000 7f0205319200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T05:27:47.801 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:47 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:47.464+0000 7f026d3f4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T05:27:47.801 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:47 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:47.634+0000 7f026d3f4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T05:27:47.801 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:47 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:47.805+0000 7f026d3f4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T05:27:47.866 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:47 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:47.563+0000 7f0205319200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T05:27:48.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:48.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:27:48.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: Reconfiguring daemon mgr.x on smithi115 2024-05-09T05:27:48.089 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.246 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:47 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:47.918+0000 7f0205319200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:48 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:48.011+0000 7f0205319200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T05:27:48.248 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T05:27:48.248 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T05:27:48.248 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:48.248 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: Reconfiguring daemon mgr.x on smithi115 2024-05-09T05:27:48.248 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.383 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:48 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:48.092+0000 7f026d3f4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T05:27:48.383 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:48 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:48.184+0000 7f026d3f4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T05:27:48.383 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:48 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:48.387+0000 7f026d3f4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T05:27:48.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:48.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:27:48.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: Reconfiguring daemon mgr.x on smithi115 2024-05-09T05:27:48.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:48.616 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:48 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:48.251+0000 7f0205319200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T05:27:48.617 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:48 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:48.385+0000 7f0205319200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T05:27:48.617 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:48 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:48.480+0000 7f0205319200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T05:27:48.661 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:48 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:48.578+0000 7f026d3f4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:27:49.116 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:48 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:48.812+0000 7f0205319200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T05:27:49.117 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:48 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:48.904+0000 7f0205319200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T05:27:49.661 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:49 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:49.211+0000 7f026d3f4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T05:27:49.662 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:49 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:49.437+0000 7f026d3f4200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T05:27:49.867 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:49 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:49.507+0000 7f0205319200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T05:27:49.867 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:27:49 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:27:49.724+0000 7f0205319200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T05:27:50.142 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:49 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:49.868+0000 7f026d3f4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T05:27:50.143 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:49 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:49.965+0000 7f026d3f4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T05:27:50.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:50.187+0000 7effaaffd700 1 -- 172.21.15.115:0/1088810098 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 14) ==== 99938+0+0 (secure 0 0 0) 0x7effb00054f0 con 0x7effb4104390 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: Standby manager daemon z started 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:27:50.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T05:27:50.399 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:50 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:27:50.411 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:50 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:50.343+0000 7f026d3f4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T05:27:50.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: Standby manager daemon z started 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T05:27:50.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:50 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: Standby manager daemon z started 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T05:27:50.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:50 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.195:0/383968103' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:27:51.038 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:50 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:50.703+0000 7f026d3f4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T05:27:51.038 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:50 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:50.798+0000 7f026d3f4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T05:27:51.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:51 smithi161 ceph-mon[33805]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-09T05:27:51.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T05:27:51.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:51 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3603241096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72834922-0c68-46a5-a24b-7bad43c395c4"}]: dispatch 2024-05-09T05:27:51.415 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:51 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3603241096' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72834922-0c68-46a5-a24b-7bad43c395c4"}]': finished 2024-05-09T05:27:51.416 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:51 smithi161 ceph-mon[33805]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T05:27:51.416 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:27:51.416 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:51 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/817381569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T05:27:51.416 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:51 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:51.043+0000 7f026d3f4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T05:27:51.416 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:51 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:51.178+0000 7f026d3f4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T05:27:51.416 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:51 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:51.275+0000 7f026d3f4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T05:27:51.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:51 smithi195 ceph-mon[33844]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-09T05:27:51.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T05:27:51.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:51 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3603241096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72834922-0c68-46a5-a24b-7bad43c395c4"}]: dispatch 2024-05-09T05:27:51.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:51 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3603241096' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72834922-0c68-46a5-a24b-7bad43c395c4"}]': finished 2024-05-09T05:27:51.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:51 smithi195 ceph-mon[33844]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T05:27:51.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:27:51.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:51 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/817381569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T05:27:51.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:51 smithi115 ceph-mon[30349]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-09T05:27:51.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T05:27:51.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:51 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3603241096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72834922-0c68-46a5-a24b-7bad43c395c4"}]: dispatch 2024-05-09T05:27:51.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:51 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3603241096' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72834922-0c68-46a5-a24b-7bad43c395c4"}]': finished 2024-05-09T05:27:51.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:51 smithi115 ceph-mon[30349]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T05:27:51.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:27:51.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:51 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/817381569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T05:27:51.911 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:51 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:51.606+0000 7f026d3f4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T05:27:51.911 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:51 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:51.699+0000 7f026d3f4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T05:27:52.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:52 smithi195 ceph-mon[33844]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:52 smithi115 ceph-mon[30349]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:52 smithi161 ceph-mon[33805]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:52.662 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:52 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:52.309+0000 7f026d3f4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T05:27:52.662 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:27:52 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:27:52.528+0000 7f026d3f4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T05:27:53.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:27:53.331+0000 7effaaffd700 1 -- 172.21.15.115:0/1088810098 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7effb002f1b0 con 0x7effb4104390 2024-05-09T05:27:53.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:53 smithi195 ceph-mon[33844]: Standby manager daemon y started 2024-05-09T05:27:53.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:53 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T05:27:53.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:53 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:27:53.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:53 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T05:27:53.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:53 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:27:53.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:53 smithi115 ceph-mon[30349]: Standby manager daemon y started 2024-05-09T05:27:53.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:53 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T05:27:53.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:53 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:27:53.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:53 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T05:27:53.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:53 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:27:53.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:53 smithi161 ceph-mon[33805]: Standby manager daemon y started 2024-05-09T05:27:53.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:53 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T05:27:53.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:53 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:27:53.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:53 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T05:27:53.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:53 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.161:0/1340105158' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:27:54.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:54 smithi161 ceph-mon[33805]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:54.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:54 smithi161 ceph-mon[33805]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-09T05:27:54.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:54 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T05:27:54.561 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:54 smithi115 ceph-mon[30349]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:54.561 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:54 smithi115 ceph-mon[30349]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-09T05:27:54.561 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T05:27:54.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:54 smithi195 ceph-mon[33844]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:54.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:54 smithi195 ceph-mon[33844]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-09T05:27:54.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:54 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T05:27:56.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:56 smithi195 ceph-mon[33844]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:56.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:56 smithi115 ceph-mon[30349]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:56.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:56 smithi161 ceph-mon[33805]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:57.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:57 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T05:27:57.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:57 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:57.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:57 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T05:27:57.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:57 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:57.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:57 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T05:27:57.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:57 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:27:58.562 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:58 smithi115 ceph-mon[30349]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:58.563 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:27:58 smithi115 ceph-mon[30349]: Deploying daemon osd.0 on smithi115 2024-05-09T05:27:58.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:58 smithi195 ceph-mon[33844]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:58.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:27:58 smithi195 ceph-mon[33844]: Deploying daemon osd.0 on smithi115 2024-05-09T05:27:58.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:58 smithi161 ceph-mon[33805]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:27:58.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:27:58 smithi161 ceph-mon[33805]: Deploying daemon osd.0 on smithi115 2024-05-09T05:28:00.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:00 smithi115 ceph-mon[30349]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:00.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:00 smithi195 ceph-mon[33844]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:00.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:00 smithi161 ceph-mon[33805]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:02.550 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:02 smithi115 ceph-mon[30349]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:02.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:02 smithi195 ceph-mon[33844]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:02 smithi161 ceph-mon[33805]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:03.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:03 smithi115 ceph-mon[30349]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:03.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:03.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:03.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:04.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:03 smithi195 ceph-mon[33844]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:04.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:03 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:04.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:03 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:04.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:03 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:03 smithi161 ceph-mon[33805]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:03 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:03 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:03 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:06.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:05 smithi195 ceph-mon[33844]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:06.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:05 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:05 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:05 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:06.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:05 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:06.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:05 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:05 smithi115 ceph-mon[30349]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:06.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:05 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:05 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:05 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:06.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:05 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:06.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:05 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:05 smithi161 ceph-mon[33805]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:06.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:05 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:05 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:05 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:06.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:05 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:06.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:05 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.525+0000 7effaaffd700 1 -- 172.21.15.115:0/1088810098 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7effb4063c00 con 0x7effa003e750 2024-05-09T05:28:06.520 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 0 on host 'smithi115' 2024-05-09T05:28:06.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7effa003e750 msgr2=0x7effa0040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:06.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7effa003e750 0x7effa0040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7effa4009750 tx=0x7effa4009310 comp rx=0 tx=0).stop 2024-05-09T05:28:06.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 msgr2=0x7effb419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:06.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 0x7effb419f630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7effb0005e50 tx=0x7effb0009400 comp rx=0 tx=0).stop 2024-05-09T05:28:06.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 shutdown_connections 2024-05-09T05:28:06.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7effa003e750 0x7effa0040c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:06.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effb4104d30 0x7effb41a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:06.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb4103260 0x7effb40764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:06.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 --2- 172.21.15.115:0/1088810098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effb4104390 0x7effb419f630 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:06.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 >> 172.21.15.115:0/1088810098 conn(0x7effb40fe8c0 msgr2=0x7effb41075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:28:06.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.527+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 shutdown_connections 2024-05-09T05:28:06.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:28:06.528+0000 7effbbcae700 1 -- 172.21.15.115:0/1088810098 wait complete. 2024-05-09T05:28:06.827 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:06.828 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.828 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:06.828 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:06 smithi115 ceph-mon[30349]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T05:28:07.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:07.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:07.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:07.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:06 smithi195 ceph-mon[33844]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T05:28:07.132 DEBUG:teuthology.orchestra.run.smithi115:osd.0> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.0.service 2024-05-09T05:28:07.134 INFO:tasks.cephadm:Deploying osd.1 on smithi161 with /dev/vg_nvme/lv_4... 2024-05-09T05:28:07.135 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T05:28:07.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:07.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:07.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:07.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:06 smithi161 ceph-mon[33805]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T05:28:07.382 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:28:07.811 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:07 smithi161 ceph-mon[33805]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:07.811 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:07 smithi161 ceph-mon[33805]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T05:28:07.811 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:07 smithi161 ceph-mon[33805]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T05:28:07.811 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:07 smithi161 ceph-mon[33805]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:28:07.812 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:07.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:07 smithi115 ceph-mon[30349]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:07.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:07 smithi115 ceph-mon[30349]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T05:28:07.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:07 smithi115 ceph-mon[30349]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T05:28:07.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:07 smithi115 ceph-mon[30349]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:28:07.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:08.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:07 smithi195 ceph-mon[33844]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:08.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:07 smithi195 ceph-mon[33844]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T05:28:08.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:07 smithi195 ceph-mon[33844]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T05:28:08.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:07 smithi195 ceph-mon[33844]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:28:08.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:08.644 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0[42417]: 2024-05-09T05:28:08.501+0000 7f36fe755700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T05:28:08.645 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0[42417]: 2024-05-09T05:28:08.504+0000 7f36fa50a700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:28:09.275 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T05:28:09.275 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T05:28:09.275 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-09T05:28:09.275 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-09T05:28:09.275 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111935 s, 937 MB/s 2024-05-09T05:28:09.275 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-09T05:28:09.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:09 smithi115 ceph-mon[30349]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:28:09.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:09 smithi115 ceph-mon[30349]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T05:28:09.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:09 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:09.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:09 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:09.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:09 smithi115 ceph-mon[30349]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:09.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:09 smithi161 ceph-mon[33805]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:28:09.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:09 smithi161 ceph-mon[33805]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T05:28:09.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:09 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:09.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:09 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:09.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:09 smithi161 ceph-mon[33805]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:09.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:09 smithi195 ceph-mon[33844]: from='osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:28:09.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:09 smithi195 ceph-mon[33844]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T05:28:09.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:09 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:09.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:09 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:09.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:09 smithi195 ceph-mon[33844]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T05:28:09.892 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-05-09T05:28:10.176 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] boot 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi115 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:28:10.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi115 to 17088M 2024-05-09T05:28:10.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:10.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:10.633 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] boot 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi115 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:28:10.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi115 to 17088M 2024-05-09T05:28:10.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:10.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:10.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: osd.0 [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] boot 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi115 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi115 to 17088M 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:10.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:10.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:11.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.741+0000 7f10c6d9e700 1 -- 172.21.15.161:0/228095494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 msgr2=0x7f10c010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:11.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.741+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/228095494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 0x7f10c010e510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f10b800b0a0 tx=0x7f10b800b3b0 comp rx=0 tx=0).stop 2024-05-09T05:28:11.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.741+0000 7f10c6d9e700 1 -- 172.21.15.161:0/228095494 shutdown_connections 2024-05-09T05:28:11.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.741+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/228095494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 0x7f10c010e510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:11.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.741+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/228095494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10c00776e0 0x7f10c0075bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:11.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.741+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/228095494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10c00760f0 0x7f10c0076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:11.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.741+0000 7f10c6d9e700 1 -- 172.21.15.161:0/228095494 >> 172.21.15.161:0/228095494 conn(0x7f10c00ff7b0 msgr2=0x7f10c0101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:28:11.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.741+0000 7f10c6d9e700 1 -- 172.21.15.161:0/228095494 shutdown_connections 2024-05-09T05:28:11.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.741+0000 7f10c6d9e700 1 -- 172.21.15.161:0/228095494 wait complete. 2024-05-09T05:28:11.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c6d9e700 1 Processor -- start 2024-05-09T05:28:11.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c6d9e700 1 -- start start 2024-05-09T05:28:11.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c6d9e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10c00760f0 0x7f10c019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:28:11.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c6d9e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10c00776e0 0x7f10c019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:28:11.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c6d9e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 0x7f10c01a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:28:11.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c6d9e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0111c60 con 0x7f10c00760f0 2024-05-09T05:28:11.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c6d9e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0111ae0 con 0x7f10c010c000 2024-05-09T05:28:11.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c6d9e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0111de0 con 0x7f10c00776e0 2024-05-09T05:28:11.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c659d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 0x7f10c01a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c659d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 0x7f10c01a3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41288/0 (socket says 172.21.15.161:41288) 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c659d700 1 -- 172.21.15.161:0/879692805 learned_addr learned my addr 172.21.15.161:0/879692805 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c559b700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10c00776e0 0x7f10c019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.742+0000 7f10c5d9c700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10c00760f0 0x7f10c019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10c659d700 1 -- 172.21.15.161:0/879692805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10c00776e0 msgr2=0x7f10c019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10c659d700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10c00776e0 0x7f10c019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10c659d700 1 -- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10c00760f0 msgr2=0x7f10c019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10c659d700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10c00760f0 0x7f10c019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10c659d700 1 -- 172.21.15.161:0/879692805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c01a4310 con 0x7f10c010c000 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10c659d700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 0x7f10c01a3c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009030 tx=0x7f10b80097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10b6ffd700 1 -- 172.21.15.161:0/879692805 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b801f030 con 0x7f10c010c000 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10b6ffd700 1 -- 172.21.15.161:0/879692805 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10b8005040 con 0x7f10c010c000 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c01a45a0 con 0x7f10c010c000 2024-05-09T05:28:11.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10b6ffd700 1 -- 172.21.15.161:0/879692805 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b801b690 con 0x7f10c010c000 2024-05-09T05:28:11.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.743+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c01a4aa0 con 0x7f10c010c000 2024-05-09T05:28:11.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.744+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c00512c0 con 0x7f10c010c000 2024-05-09T05:28:11.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.746+0000 7f10b6ffd700 1 -- 172.21.15.161:0/879692805 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f10b8022030 con 0x7f10c010c000 2024-05-09T05:28:11.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.747+0000 7f10b6ffd700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f10ac0b2870 0x7f10ac0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:28:11.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.747+0000 7f10b6ffd700 1 -- 172.21.15.161:0/879692805 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f10b800f020 con 0x7f10c010c000 2024-05-09T05:28:11.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.747+0000 7f10c5d9c700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f10ac0b2870 0x7f10ac0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:28:11.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.747+0000 7f10c5d9c700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f10ac0b2870 0x7f10ac0b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f10bc005950 tx=0x7f10bc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:28:11.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.752+0000 7f10b6ffd700 1 -- 172.21.15.161:0/879692805 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10b8029020 con 0x7f10c010c000 2024-05-09T05:28:11.768 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:11 smithi195 ceph-mon[33844]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T05:28:11.768 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:11 smithi195 ceph-mon[33844]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:11.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:11 smithi115 ceph-mon[30349]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T05:28:11.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:11 smithi115 ceph-mon[30349]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:11.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:11 smithi161 ceph-mon[33805]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T05:28:11.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:11 smithi161 ceph-mon[33805]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:11.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:11.967+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f10c0063c00 con 0x7f10ac0b2870 2024-05-09T05:28:12.688 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:12 smithi161 ceph-mon[33805]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:28:12.689 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:12 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:28:12.689 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:12 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:28:12.689 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:12 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:12.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:12 smithi195 ceph-mon[33844]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:28:12.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:12 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:28:12.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:12 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:28:12.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:12 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:12.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:12 smithi115 ceph-mon[30349]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:28:12.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:28:12.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:28:12.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:13.796 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:13 smithi161 ceph-mon[33805]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:13.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:13 smithi195 ceph-mon[33844]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:13.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:13 smithi115 ceph-mon[30349]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:15.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:15 smithi115 ceph-mon[30349]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:15.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:15 smithi115 ceph-mon[30349]: from='client.? 172.21.15.161:0/3387046316' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b5e817-6c25-4b93-8eed-13aff270aac2"}]: dispatch 2024-05-09T05:28:15.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:15 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b5e817-6c25-4b93-8eed-13aff270aac2"}]: dispatch 2024-05-09T05:28:15.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:15 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6b5e817-6c25-4b93-8eed-13aff270aac2"}]': finished 2024-05-09T05:28:15.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:15 smithi115 ceph-mon[30349]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T05:28:15.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:15 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:15.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:15 smithi115 ceph-mon[30349]: from='client.? 172.21.15.161:0/2191090000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T05:28:15.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:15 smithi161 ceph-mon[33805]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:15.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:15 smithi161 ceph-mon[33805]: from='client.? 172.21.15.161:0/3387046316' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b5e817-6c25-4b93-8eed-13aff270aac2"}]: dispatch 2024-05-09T05:28:15.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:15 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b5e817-6c25-4b93-8eed-13aff270aac2"}]: dispatch 2024-05-09T05:28:15.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:15 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6b5e817-6c25-4b93-8eed-13aff270aac2"}]': finished 2024-05-09T05:28:15.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:15 smithi161 ceph-mon[33805]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T05:28:15.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:15 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:15.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:15 smithi161 ceph-mon[33805]: from='client.? 172.21.15.161:0/2191090000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T05:28:16.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:15 smithi195 ceph-mon[33844]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:15 smithi195 ceph-mon[33844]: from='client.? 172.21.15.161:0/3387046316' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b5e817-6c25-4b93-8eed-13aff270aac2"}]: dispatch 2024-05-09T05:28:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:15 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b5e817-6c25-4b93-8eed-13aff270aac2"}]: dispatch 2024-05-09T05:28:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:15 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6b5e817-6c25-4b93-8eed-13aff270aac2"}]': finished 2024-05-09T05:28:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:15 smithi195 ceph-mon[33844]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T05:28:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:15 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:15 smithi195 ceph-mon[33844]: from='client.? 172.21.15.161:0/2191090000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T05:28:17.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:17 smithi115 ceph-mon[30349]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:17.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:17 smithi161 ceph-mon[33805]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:18.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:17 smithi195 ceph-mon[33844]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:19.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:19 smithi115 ceph-mon[30349]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:19 smithi161 ceph-mon[33805]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:20.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:19 smithi195 ceph-mon[33844]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:21.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:21 smithi115 ceph-mon[30349]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:21 smithi161 ceph-mon[33805]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:22.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:21 smithi195 ceph-mon[33844]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:22.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:22 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T05:28:22.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:22 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:22.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:22 smithi195 ceph-mon[33844]: Deploying daemon osd.1 on smithi161 2024-05-09T05:28:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:22 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T05:28:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:22 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:22 smithi115 ceph-mon[30349]: Deploying daemon osd.1 on smithi161 2024-05-09T05:28:22.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:22 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T05:28:22.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:22 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:22.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:22 smithi161 ceph-mon[33805]: Deploying daemon osd.1 on smithi161 2024-05-09T05:28:23.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:23 smithi115 ceph-mon[30349]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:23.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:23 smithi161 ceph-mon[33805]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:24.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:23 smithi195 ceph-mon[33844]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:25.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:25 smithi115 ceph-mon[30349]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:25.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:25 smithi161 ceph-mon[33805]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:26.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:25 smithi195 ceph-mon[33844]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:28.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:28 smithi161 ceph-mon[33805]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:28.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:28.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:28.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:28.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:28 smithi195 ceph-mon[33844]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:28.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:28.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:28.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:28.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:28 smithi115 ceph-mon[30349]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:28.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:28.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:28.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:30 smithi195 ceph-mon[33844]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:30.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:30.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:30.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:30 smithi115 ceph-mon[30349]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:30.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:30.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:30.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:30 smithi161 ceph-mon[33805]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:30.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:30.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:30.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:30.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.782+0000 7f10b6ffd700 1 -- 172.21.15.161:0/879692805 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f10c0063c00 con 0x7f10ac0b2870 2024-05-09T05:28:30.776 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 1 on host 'smithi161' 2024-05-09T05:28:30.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f10ac0b2870 msgr2=0x7f10ac0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:30.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f10ac0b2870 0x7f10ac0b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f10bc005950 tx=0x7f10bc00b410 comp rx=0 tx=0).stop 2024-05-09T05:28:30.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 msgr2=0x7f10c01a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:30.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 0x7f10c01a3c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009030 tx=0x7f10b80097f0 comp rx=0 tx=0).stop 2024-05-09T05:28:30.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 shutdown_connections 2024-05-09T05:28:30.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f10ac0b2870 0x7f10ac0b4d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:30.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c010c000 0x7f10c01a3c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:30.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10c00760f0 0x7f10c019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:30.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 --2- 172.21.15.161:0/879692805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10c00776e0 0x7f10c019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:30.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 >> 172.21.15.161:0/879692805 conn(0x7f10c00ff7b0 msgr2=0x7f10c0107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:28:30.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 shutdown_connections 2024-05-09T05:28:30.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:28:30.784+0000 7f10c6d9e700 1 -- 172.21.15.161:0/879692805 wait complete. 2024-05-09T05:28:31.271 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:31 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:31.271 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:31 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:31.271 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:31 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:31.271 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:31 smithi161 ceph-mon[33805]: from='osd.1 [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T05:28:31.271 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:31 smithi161 ceph-mon[33805]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T05:28:31.296 DEBUG:teuthology.orchestra.run.smithi161:osd.1> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.1.service 2024-05-09T05:28:31.298 INFO:tasks.cephadm:Deploying osd.2 on smithi195 with /dev/vg_nvme/lv_4... 2024-05-09T05:28:31.298 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T05:28:31.326 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:31 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:31.326 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:31 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:31.326 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:31 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:31.326 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:31 smithi195 ceph-mon[33844]: from='osd.1 [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T05:28:31.326 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:31 smithi195 ceph-mon[33844]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T05:28:31.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:31 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:31.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:31 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:31.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:31 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:31.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:31 smithi115 ceph-mon[30349]: from='osd.1 [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T05:28:31.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:31 smithi115 ceph-mon[30349]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T05:28:31.546 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:28:32.370 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:32 smithi195 ceph-mon[33844]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:32.370 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:32 smithi195 ceph-mon[33844]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T05:28:32.370 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:32 smithi195 ceph-mon[33844]: from='osd.1 [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:28:32.370 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:32 smithi195 ceph-mon[33844]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T05:28:32.370 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:32.370 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:32 smithi195 ceph-mon[33844]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:28:32.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:32 smithi115 ceph-mon[30349]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:32.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:32 smithi115 ceph-mon[30349]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T05:28:32.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:32 smithi115 ceph-mon[30349]: from='osd.1 [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:28:32.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:32 smithi115 ceph-mon[30349]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T05:28:32.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:32.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:32 smithi115 ceph-mon[30349]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:28:32.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:32 smithi161 ceph-mon[33805]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:32.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:32 smithi161 ceph-mon[33805]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T05:28:32.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:32 smithi161 ceph-mon[33805]: from='osd.1 [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:28:32.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:32 smithi161 ceph-mon[33805]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T05:28:32.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:32.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:32 smithi161 ceph-mon[33805]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:28:33.064 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:32 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1[38402]: 2024-05-09T05:28:32.784+0000 7f4663711700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T05:28:33.064 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:32 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1[38402]: 2024-05-09T05:28:32.788+0000 7f465f4c6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:28:33.422 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T05:28:33.422 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T05:28:33.423 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-05-09T05:28:33.423 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-05-09T05:28:33.423 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00836214 s, 1.3 GB/s 2024-05-09T05:28:33.423 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-05-09T05:28:33.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:33 smithi161 ceph-mon[33805]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:33.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:33 smithi161 ceph-mon[33805]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:28:33.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:33 smithi161 ceph-mon[33805]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T05:28:33.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:33.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:34.057 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:33 smithi195 ceph-mon[33844]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:34.057 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:33 smithi195 ceph-mon[33844]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:28:34.057 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:33 smithi195 ceph-mon[33844]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T05:28:34.057 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:34.058 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:34.086 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_4 2024-05-09T05:28:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:33 smithi115 ceph-mon[30349]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T05:28:34.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:33 smithi115 ceph-mon[30349]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:28:34.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:33 smithi115 ceph-mon[30349]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T05:28:34.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:34.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:34.374 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:28:35.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:28:35.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:28:35.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: osd.1 [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] boot 2024-05-09T05:28:35.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T05:28:35.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:35.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi161 2024-05-09T05:28:35.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:28:35.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi161 to 17088M 2024-05-09T05:28:35.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:35.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:35.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:28:35.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: osd.1 [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] boot 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi161 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi161 to 17088M 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:35.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:35.149 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:28:35.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:28:35.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: osd.1 [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] boot 2024-05-09T05:28:35.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T05:28:35.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:28:35.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi161 2024-05-09T05:28:35.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:28:35.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi161 to 17088M 2024-05-09T05:28:35.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:35.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:35.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:35.956 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.961+0000 7f7be0f90700 1 -- 172.21.15.195:0/342729342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 msgr2=0x7f7bdc10c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:35.956 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.961+0000 7f7be0f90700 1 --2- 172.21.15.195:0/342729342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 0x7f7bdc10c6f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400b0a0 tx=0x7f7bd400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:28:35.956 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.961+0000 7f7be0f90700 1 -- 172.21.15.195:0/342729342 shutdown_connections 2024-05-09T05:28:35.957 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.961+0000 7f7be0f90700 1 --2- 172.21.15.195:0/342729342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bdc1059b0 0x7f7bdc105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:35.957 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.961+0000 7f7be0f90700 1 --2- 172.21.15.195:0/342729342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 0x7f7bdc10c6f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:35.957 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.961+0000 7f7be0f90700 1 --2- 172.21.15.195:0/342729342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bdc103f60 0x7f7bdc104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:35.957 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.961+0000 7f7be0f90700 1 -- 172.21.15.195:0/342729342 >> 172.21.15.195:0/342729342 conn(0x7f7bdc0ff9d0 msgr2=0x7f7bdc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:28:35.957 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.961+0000 7f7be0f90700 1 -- 172.21.15.195:0/342729342 shutdown_connections 2024-05-09T05:28:35.957 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.961+0000 7f7be0f90700 1 -- 172.21.15.195:0/342729342 wait complete. 2024-05-09T05:28:35.959 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7be0f90700 1 Processor -- start 2024-05-09T05:28:35.959 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7be0f90700 1 -- start start 2024-05-09T05:28:35.959 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7be0f90700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bdc103f60 0x7f7bdc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:28:35.959 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7be0f90700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 0x7f7bdc19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:28:35.959 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7be0f90700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bdc1059b0 0x7f7bdc1a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:28:35.959 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7be0f90700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc10fda0 con 0x7f7bdc1059b0 2024-05-09T05:28:35.959 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7be0f90700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc10fc20 con 0x7f7bdc103f60 2024-05-09T05:28:35.959 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7be0f90700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc10ff20 con 0x7f7bdc1049e0 2024-05-09T05:28:35.959 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7bd9d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 0x7f7bdc19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7bd9d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 0x7f7bdc19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:52792/0 (socket says 172.21.15.195:52792) 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7bd9d9b700 1 -- 172.21.15.195:0/166570996 learned_addr learned my addr 172.21.15.195:0/166570996 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7bdad9d700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bdc1059b0 0x7f7bdc1a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.962+0000 7f7bda59c700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bdc103f60 0x7f7bdc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7bd9d9b700 1 -- 172.21.15.195:0/166570996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bdc103f60 msgr2=0x7f7bdc19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7bd9d9b700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bdc103f60 0x7f7bdc19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7bd9d9b700 1 -- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bdc1059b0 msgr2=0x7f7bdc1a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7bd9d9b700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bdc1059b0 0x7f7bdc1a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7bd9d9b700 1 -- 172.21.15.195:0/166570996 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc1a2440 con 0x7f7bdc1049e0 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7bd9d9b700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 0x7f7bdc19ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d950 tx=0x7f7bd000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7bcb7fe700 1 -- 172.21.15.195:0/166570996 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0016070 con 0x7f7bdc1049e0 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc1a2730 con 0x7f7bdc1049e0 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7bcb7fe700 1 -- 172.21.15.195:0/166570996 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bd0009dd0 con 0x7f7bdc1049e0 2024-05-09T05:28:35.960 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7bcb7fe700 1 -- 172.21.15.195:0/166570996 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd000f3c0 con 0x7f7bdc1049e0 2024-05-09T05:28:35.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.963+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc1a2c70 con 0x7f7bdc1049e0 2024-05-09T05:28:35.961 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.964+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bdc0512c0 con 0x7f7bdc1049e0 2024-05-09T05:28:35.962 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.966+0000 7f7bcb7fe700 1 -- 172.21.15.195:0/166570996 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f7bd0019070 con 0x7f7bdc1049e0 2024-05-09T05:28:35.962 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.966+0000 7f7bcb7fe700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7bc40b2870 0x7f7bc40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:28:35.962 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.967+0000 7f7bcb7fe700 1 -- 172.21.15.195:0/166570996 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f7bd00e0650 con 0x7f7bdc1049e0 2024-05-09T05:28:35.962 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.967+0000 7f7bda59c700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7bc40b2870 0x7f7bc40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:28:35.962 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.967+0000 7f7bda59c700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7bc40b2870 0x7f7bc40b4d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc005950 tx=0x7f7bcc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:28:35.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:35.970+0000 7f7bcb7fe700 1 -- 172.21.15.195:0/166570996 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7bd00a9b20 con 0x7f7bdc1049e0 2024-05-09T05:28:36.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:35 smithi195 ceph-mon[33844]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:36.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:35 smithi195 ceph-mon[33844]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T05:28:36.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:35 smithi115 ceph-mon[30349]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:36.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:35 smithi115 ceph-mon[30349]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T05:28:36.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:35 smithi161 ceph-mon[33805]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:36.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:35 smithi161 ceph-mon[33805]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T05:28:36.174 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:36.179+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7bdc02e9a0 con 0x7f7bc40b2870 2024-05-09T05:28:37.040 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:36 smithi195 ceph-mon[33844]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:28:37.040 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:28:37.040 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:28:37.040 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:37.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:36 smithi115 ceph-mon[30349]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:28:37.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:28:37.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:28:37.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:37.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:36 smithi161 ceph-mon[33805]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:28:37.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:28:37.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:28:37.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:37.932 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:37 smithi195 ceph-mon[33844]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:38.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:37 smithi115 ceph-mon[30349]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:38.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:37 smithi161 ceph-mon[33805]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:40.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:39 smithi195 ceph-mon[33844]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:40.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:39 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/1883720327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c0b77f-2058-4131-a82c-b8f42780f350"}]: dispatch 2024-05-09T05:28:40.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:39 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c0b77f-2058-4131-a82c-b8f42780f350"}]: dispatch 2024-05-09T05:28:40.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:39 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77c0b77f-2058-4131-a82c-b8f42780f350"}]': finished 2024-05-09T05:28:40.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:39 smithi195 ceph-mon[33844]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T05:28:40.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:39 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:40.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:39 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/167716121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T05:28:40.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:39 smithi115 ceph-mon[30349]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:40.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:39 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/1883720327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c0b77f-2058-4131-a82c-b8f42780f350"}]: dispatch 2024-05-09T05:28:40.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:39 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c0b77f-2058-4131-a82c-b8f42780f350"}]: dispatch 2024-05-09T05:28:40.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:39 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77c0b77f-2058-4131-a82c-b8f42780f350"}]': finished 2024-05-09T05:28:40.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:39 smithi115 ceph-mon[30349]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T05:28:40.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:39 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:40.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:39 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/167716121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T05:28:40.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:39 smithi161 ceph-mon[33805]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:40.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:39 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/1883720327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c0b77f-2058-4131-a82c-b8f42780f350"}]: dispatch 2024-05-09T05:28:40.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:39 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c0b77f-2058-4131-a82c-b8f42780f350"}]: dispatch 2024-05-09T05:28:40.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:39 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77c0b77f-2058-4131-a82c-b8f42780f350"}]': finished 2024-05-09T05:28:40.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:39 smithi161 ceph-mon[33805]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T05:28:40.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:39 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:40.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:39 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/167716121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T05:28:42.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:41 smithi195 ceph-mon[33844]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:42.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:41 smithi115 ceph-mon[30349]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:42.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:41 smithi161 ceph-mon[33805]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:44.053 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:43 smithi195 ceph-mon[33844]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:44.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:43 smithi115 ceph-mon[30349]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:44.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:43 smithi161 ceph-mon[33805]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:45.993 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:45 smithi195 ceph-mon[33844]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:46.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:45 smithi115 ceph-mon[30349]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:46.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:45 smithi161 ceph-mon[33805]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:46.897 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:46 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T05:28:46.897 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:46 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:46.897 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:46 smithi195 ceph-mon[33844]: Deploying daemon osd.2 on smithi195 2024-05-09T05:28:47.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:46 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T05:28:47.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:46 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:47.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:46 smithi115 ceph-mon[30349]: Deploying daemon osd.2 on smithi195 2024-05-09T05:28:47.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:46 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T05:28:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:46 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:47.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:46 smithi161 ceph-mon[33805]: Deploying daemon osd.2 on smithi195 2024-05-09T05:28:48.073 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:47 smithi195 ceph-mon[33844]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:48.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:47 smithi115 ceph-mon[30349]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:48.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:47 smithi161 ceph-mon[33805]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:50.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:49 smithi195 ceph-mon[33844]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:50.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:49 smithi115 ceph-mon[30349]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:50.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:49 smithi161 ceph-mon[33805]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:52.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:52 smithi195 ceph-mon[33844]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:52.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:52.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:52.569 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:52 smithi115 ceph-mon[30349]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:52 smithi161 ceph-mon[33805]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:54.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:54 smithi195 ceph-mon[33844]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:54.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:54 smithi115 ceph-mon[30349]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:54.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:54 smithi161 ceph-mon[33805]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:54.738 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.743+0000 7f7bcb7fe700 1 -- 172.21.15.195:0/166570996 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7bdc02e9a0 con 0x7f7bc40b2870 2024-05-09T05:28:54.738 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 2 on host 'smithi195' 2024-05-09T05:28:54.740 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7bc40b2870 msgr2=0x7f7bc40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:54.740 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7bc40b2870 0x7f7bc40b4d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc005950 tx=0x7f7bcc0058e0 comp rx=0 tx=0).stop 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 msgr2=0x7f7bdc19ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 0x7f7bdc19ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d950 tx=0x7f7bd000dc60 comp rx=0 tx=0).stop 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 shutdown_connections 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7bc40b2870 0x7f7bc40b4d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bdc103f60 0x7f7bdc19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bdc1059b0 0x7f7bdc1a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 --2- 172.21.15.195:0/166570996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7bdc1049e0 0x7f7bdc19ae20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.745+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 >> 172.21.15.195:0/166570996 conn(0x7f7bdc0ff9d0 msgr2=0x7f7bdc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.746+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 shutdown_connections 2024-05-09T05:28:54.741 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:28:54.746+0000 7f7be0f90700 1 -- 172.21.15.195:0/166570996 wait complete. 2024-05-09T05:28:55.338 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.338 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.338 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:55.338 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:55.338 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.338 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.338 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:55.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='osd.2 [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T05:28:55.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:55 smithi195 ceph-mon[33844]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T05:28:55.364 DEBUG:teuthology.orchestra.run.smithi195:osd.2> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.2.service 2024-05-09T05:28:55.366 INFO:tasks.cephadm:Deploying osd.3 on smithi115 with /dev/vg_nvme/lv_3... 2024-05-09T05:28:55.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T05:28:55.618 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:28:55.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:55.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:55.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:55.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='osd.2 [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T05:28:55.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:55 smithi115 ceph-mon[30349]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T05:28:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:28:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='osd.2 [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T05:28:55.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:55 smithi161 ceph-mon[33805]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T05:28:56.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:56 smithi195 ceph-mon[33844]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:56.566 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:56 smithi195 ceph-mon[33844]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T05:28:56.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:56 smithi195 ceph-mon[33844]: from='osd.2 [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:28:56.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:56 smithi195 ceph-mon[33844]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T05:28:56.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:56 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:56.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:56 smithi195 ceph-mon[33844]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:28:56.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:56 smithi115 ceph-mon[30349]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:56.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:56 smithi115 ceph-mon[30349]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T05:28:56.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:56 smithi115 ceph-mon[30349]: from='osd.2 [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:28:56.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:56 smithi115 ceph-mon[30349]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T05:28:56.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:56 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:56.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:56 smithi115 ceph-mon[30349]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:28:56.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:56 smithi161 ceph-mon[33805]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:56.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:56 smithi161 ceph-mon[33805]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T05:28:56.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:56 smithi161 ceph-mon[33805]: from='osd.2 [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:28:56.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:56 smithi161 ceph-mon[33805]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T05:28:56.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:56 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:56.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:56 smithi161 ceph-mon[33805]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:28:56.834 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:56 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2[38992]: 2024-05-09T05:28:56.705+0000 7f7ee89fe700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T05:28:56.834 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:56 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2[38992]: 2024-05-09T05:28:56.709+0000 7f7ee47b3700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:28:57.608 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T05:28:57.608 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T05:28:57.608 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-05-09T05:28:57.608 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-05-09T05:28:57.608 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131288 s, 799 MB/s 2024-05-09T05:28:57.608 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-05-09T05:28:57.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:57 smithi195 ceph-mon[33844]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:57.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:57 smithi195 ceph-mon[33844]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:28:57.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:57 smithi195 ceph-mon[33844]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T05:28:57.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:57 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:57.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:57 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:57.951 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:57 smithi115 ceph-mon[30349]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:57.951 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:57 smithi115 ceph-mon[30349]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:28:57.951 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:57 smithi115 ceph-mon[30349]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T05:28:57.951 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:57 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:57.951 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:57 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:58.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:57 smithi161 ceph-mon[33805]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-09T05:28:58.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:57 smithi161 ceph-mon[33805]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:28:58.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:57 smithi161 ceph-mon[33805]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T05:28:58.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:57 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:58.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:57 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:58.212 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2024-05-09T05:28:58.463 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:28:58.756 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:28:58.756 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:28:58.756 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: osd.2 [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] boot 2024-05-09T05:28:58.756 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T05:28:58.756 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:58.756 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi195 2024-05-09T05:28:58.756 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:58.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:58.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:28:58.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi195 to 17088M 2024-05-09T05:28:58.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:58.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:58.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:58.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:58.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: osd.2 [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] boot 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi195 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:28:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi195 to 17088M 2024-05-09T05:28:59.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:59.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:59.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:59.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:59.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T05:28:59.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:28:59.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:28:59.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: osd.2 [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] boot 2024-05-09T05:28:59.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T05:28:59.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi195 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi195 to 17088M 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:28:59.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T05:28:59.929 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:59 smithi115 ceph-mon[30349]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:28:59.929 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T05:28:59.929 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:59 smithi115 ceph-mon[30349]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T05:28:59.929 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T05:28:59.929 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46602]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T05:28:59.930 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46602]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:28:59.930 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46602]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:28:59.930 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46602]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:28:59.930 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46608]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T05:28:59.930 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46608]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:28:59.930 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46608]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:28:59.930 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46608]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:59 smithi195 ceph-mon[33844]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:00.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T05:29:00.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:59 smithi195 ceph-mon[33844]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T05:29:00.012 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:28:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T05:29:00.013 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:59 smithi195 sudo[42590]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T05:29:00.013 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:59 smithi195 sudo[42590]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.013 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:59 smithi195 sudo[42590]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.013 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:59 smithi195 sudo[42590]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.013 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:59 smithi195 sudo[42593]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T05:29:00.013 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:59 smithi195 sudo[42593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.013 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:59 smithi195 sudo[42593]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.013 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 05:28:59 smithi195 sudo[42593]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.056+0000 7f744059f700 1 -- 172.21.15.115:0/1235039281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104d10 msgr2=0x7f743810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:00.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.056+0000 7f744059f700 1 --2- 172.21.15.115:0/1235039281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104d10 0x7f743810b5c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f743000b0a0 tx=0x7f743000b3b0 comp rx=0 tx=0).stop 2024-05-09T05:29:00.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.056+0000 7f744059f700 1 -- 172.21.15.115:0/1235039281 shutdown_connections 2024-05-09T05:29:00.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.056+0000 7f744059f700 1 --2- 172.21.15.115:0/1235039281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104d10 0x7f743810b5c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.056+0000 7f744059f700 1 --2- 172.21.15.115:0/1235039281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7438103240 0x7f7438103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.056+0000 7f744059f700 1 --2- 172.21.15.115:0/1235039281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438104370 0x7f7438104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.056+0000 7f744059f700 1 -- 172.21.15.115:0/1235039281 >> 172.21.15.115:0/1235039281 conn(0x7f74380fe880 msgr2=0x7f7438100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.056+0000 7f744059f700 1 -- 172.21.15.115:0/1235039281 shutdown_connections 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.056+0000 7f744059f700 1 -- 172.21.15.115:0/1235039281 wait complete. 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f744059f700 1 Processor -- start 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f744059f700 1 -- start start 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f744059f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7438103240 0x7f743806c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f744059f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104370 0x7f743819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f744059f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438104d10 0x7f74381a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f744059f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f743810eb50 con 0x7f7438104370 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f744059f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f743810e9d0 con 0x7f7438104d10 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f744059f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f743810ecd0 con 0x7f7438103240 2024-05-09T05:29:00.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f743db3a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104370 0x7f743819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f743db3a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104370 0x7f743819f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49658/0 (socket says 172.21.15.115:49658) 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f743e33b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7438103240 0x7f743806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f743db3a700 1 -- 172.21.15.115:0/665494982 learned_addr learned my addr 172.21.15.115:0/665494982 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.057+0000 7f743eb3c700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438104d10 0x7f74381a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f743e33b700 1 -- 172.21.15.115:0/665494982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438104d10 msgr2=0x7f74381a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f743e33b700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438104d10 0x7f74381a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f743e33b700 1 -- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104370 msgr2=0x7f743819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f743e33b700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104370 0x7f743819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f743e33b700 1 -- 172.21.15.115:0/665494982 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74381a2290 con 0x7f7438103240 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f743db3a700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104370 0x7f743819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f743e33b700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7438103240 0x7f743806c070 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7438102f40 tx=0x7f743400eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f742f7fe700 1 -- 172.21.15.115:0/665494982 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743400cbd0 con 0x7f7438103240 2024-05-09T05:29:00.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f742f7fe700 1 -- 172.21.15.115:0/665494982 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f743400ee80 con 0x7f7438103240 2024-05-09T05:29:00.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74381a2580 con 0x7f7438103240 2024-05-09T05:29:00.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f742f7fe700 1 -- 172.21.15.115:0/665494982 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434003bf0 con 0x7f7438103240 2024-05-09T05:29:00.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.058+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74381a2ac0 con 0x7f7438103240 2024-05-09T05:29:00.055 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:59 smithi161 ceph-mon[33805]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:00.055 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T05:29:00.055 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:59 smithi161 ceph-mon[33805]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T05:29:00.056 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:28:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T05:29:00.056 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:59 smithi161 sudo[42050]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T05:29:00.056 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:59 smithi161 sudo[42050]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.056 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:59 smithi161 sudo[42050]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.056 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:59 smithi161 sudo[42050]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.056 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:59 smithi161 sudo[42053]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T05:29:00.056 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:59 smithi161 sudo[42053]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.056 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:59 smithi161 sudo[42053]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.056 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 05:28:59 smithi161 sudo[42053]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.059+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74380512c0 con 0x7f7438103240 2024-05-09T05:29:00.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.061+0000 7f742f7fe700 1 -- 172.21.15.115:0/665494982 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f7434003d90 con 0x7f7438103240 2024-05-09T05:29:00.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.061+0000 7f742f7fe700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f74240b27f0 0x7f74240b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:00.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.062+0000 7f743db3a700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f74240b27f0 0x7f74240b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:00.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.062+0000 7f742f7fe700 1 -- 172.21.15.115:0/665494982 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f74340e10b0 con 0x7f7438103240 2024-05-09T05:29:00.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.062+0000 7f743db3a700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f74240b27f0 0x7f74240b4d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f743806ba20 tx=0x7f7428005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:29:00.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.065+0000 7f742f7fe700 1 -- 172.21.15.115:0/665494982 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74340aa310 con 0x7f7438103240 2024-05-09T05:29:00.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.281+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7438063c00 con 0x7f74240b27f0 2024-05-09T05:29:00.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46613]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T05:29:00.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46613]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46613]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:28:59 smithi115 sudo[46613]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 sudo[46616]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T05:29:00.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 sudo[46616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 sudo[46616]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.277 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 sudo[46616]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 sudo[42596]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T05:29:00.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 sudo[42596]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 sudo[42596]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 sudo[42596]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 sudo[42599]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T05:29:00.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 sudo[42599]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 sudo[42599]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 sudo[42599]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 sudo[42056]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T05:29:00.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 sudo[42056]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 sudo[42056]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 sudo[42056]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 sudo[42059]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T05:29:00.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 sudo[42059]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T05:29:00.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 sudo[42059]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T05:29:00.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 sudo[42059]: pam_unix(sudo:session): session closed for user root 2024-05-09T05:29:00.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:00.981+0000 7f742f7fe700 1 -- 172.21.15.115:0/665494982 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f74340a5ea0 con 0x7f7438103240 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T05:29:01.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:29:01.014 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:01.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:29:01.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:29:01.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:01.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T05:29:01.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T05:29:01.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:01.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:01.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:29:01.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T05:29:01.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:29:01.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:29:01.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:29:01.163 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:02.015 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:01 smithi115 ceph-mon[30349]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:02.015 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:01 smithi115 ceph-mon[30349]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T05:29:02.015 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:01 smithi115 ceph-mon[30349]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T05:29:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:01 smithi195 ceph-mon[33844]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:01 smithi195 ceph-mon[33844]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T05:29:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:01 smithi195 ceph-mon[33844]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T05:29:02.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:01 smithi161 ceph-mon[33805]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:02.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:01 smithi161 ceph-mon[33805]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T05:29:02.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:01 smithi161 ceph-mon[33805]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T05:29:04.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:03 smithi195 ceph-mon[33844]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:04.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:03 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2831158600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "588aa4a2-6263-494a-93e3-a9ad906119eb"}]: dispatch 2024-05-09T05:29:04.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:03 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2831158600' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "588aa4a2-6263-494a-93e3-a9ad906119eb"}]': finished 2024-05-09T05:29:04.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:03 smithi195 ceph-mon[33844]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T05:29:04.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:03 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:04.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:03 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/900675389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T05:29:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:03 smithi115 ceph-mon[30349]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:03 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2831158600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "588aa4a2-6263-494a-93e3-a9ad906119eb"}]: dispatch 2024-05-09T05:29:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:03 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2831158600' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "588aa4a2-6263-494a-93e3-a9ad906119eb"}]': finished 2024-05-09T05:29:04.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:03 smithi115 ceph-mon[30349]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T05:29:04.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:04.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:03 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/900675389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T05:29:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:03 smithi161 ceph-mon[33805]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:03 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2831158600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "588aa4a2-6263-494a-93e3-a9ad906119eb"}]: dispatch 2024-05-09T05:29:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:03 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2831158600' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "588aa4a2-6263-494a-93e3-a9ad906119eb"}]': finished 2024-05-09T05:29:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:03 smithi161 ceph-mon[33805]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T05:29:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:03 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:04.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:03 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/900675389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T05:29:06.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:05 smithi195 ceph-mon[33844]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:06.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:05 smithi115 ceph-mon[30349]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:06.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:05 smithi161 ceph-mon[33805]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:07.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:07 smithi115 ceph-mon[30349]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:08.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:07 smithi195 ceph-mon[33844]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:08.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:07 smithi161 ceph-mon[33805]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:10.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:09 smithi195 ceph-mon[33844]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:10.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:09 smithi115 ceph-mon[30349]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:10.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:09 smithi161 ceph-mon[33805]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:11.018 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T05:29:11.018 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:11.086 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T05:29:11.086 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T05:29:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:12.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:11 smithi195 ceph-mon[33844]: Deploying daemon osd.3 on smithi115 2024-05-09T05:29:12.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:11 smithi195 ceph-mon[33844]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:11 smithi115 ceph-mon[30349]: Deploying daemon osd.3 on smithi115 2024-05-09T05:29:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:11 smithi115 ceph-mon[30349]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:12.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:11 smithi161 ceph-mon[33805]: Deploying daemon osd.3 on smithi115 2024-05-09T05:29:12.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:11 smithi161 ceph-mon[33805]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:14.096 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:13 smithi115 ceph-mon[30349]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:14.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:13 smithi195 ceph-mon[33844]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:14.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:13 smithi161 ceph-mon[33805]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:16.004 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:15 smithi115 ceph-mon[30349]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:16.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:15 smithi195 ceph-mon[33844]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:15 smithi161 ceph-mon[33805]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:17.371 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:17 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:17.371 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:17 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:17.371 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:17 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:17 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:17.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:17 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:17.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:17 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:17 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:17 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:17 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:18.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:18 smithi115 ceph-mon[30349]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:18.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:18 smithi195 ceph-mon[33844]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:18 smithi161 ceph-mon[33805]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:19.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:19 smithi115 ceph-mon[30349]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:19.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:19 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:19.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:19 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:19.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:19 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:19.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:19 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:19.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:19 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:20.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.007+0000 7f742f7fe700 1 -- 172.21.15.115:0/665494982 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7438063c00 con 0x7f74240b27f0 2024-05-09T05:29:20.003 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 3 on host 'smithi115' 2024-05-09T05:29:20.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f74240b27f0 msgr2=0x7f74240b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:20.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f74240b27f0 0x7f74240b4d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f743806ba20 tx=0x7f7428005f90 comp rx=0 tx=0).stop 2024-05-09T05:29:20.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7438103240 msgr2=0x7f743806c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:20.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7438103240 0x7f743806c070 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7438102f40 tx=0x7f743400eca0 comp rx=0 tx=0).stop 2024-05-09T05:29:20.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 shutdown_connections 2024-05-09T05:29:20.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f74240b27f0 0x7f74240b4d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:20.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438104d10 0x7f74381a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:20.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7438103240 0x7f743806c070 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:20.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 --2- 172.21.15.115:0/665494982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7438104370 0x7f743819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:20.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 >> 172.21.15.115:0/665494982 conn(0x7f74380fe880 msgr2=0x7f74381075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:29:20.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 shutdown_connections 2024-05-09T05:29:20.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:29:20.010+0000 7f744059f700 1 -- 172.21.15.115:0/665494982 wait complete. 2024-05-09T05:29:20.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:19 smithi195 ceph-mon[33844]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:20.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:19 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:20.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:19 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:20.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:19 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:20.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:19 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:20.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:19 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:20.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:19 smithi161 ceph-mon[33805]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:20.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:19 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:20.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:19 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:20.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:19 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:20.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:19 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:20.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:19 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:20.620 DEBUG:teuthology.orchestra.run.smithi115:osd.3> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.3.service 2024-05-09T05:29:20.622 INFO:tasks.cephadm:Deploying osd.4 on smithi161 with /dev/vg_nvme/lv_3... 2024-05-09T05:29:20.622 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T05:29:20.875 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:29:21.316 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:20 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:21.316 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:20 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:21.316 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:20 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:21.316 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:20 smithi161 ceph-mon[33805]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T05:29:21.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:20 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:21.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:20 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:21.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:20 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:21.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:20 smithi195 ceph-mon[33844]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T05:29:21.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:20 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:21.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:20 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:21.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:20 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:21.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:20 smithi115 ceph-mon[30349]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T05:29:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:22 smithi115 ceph-mon[30349]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:22 smithi115 ceph-mon[30349]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T05:29:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:22 smithi115 ceph-mon[30349]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T05:29:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:22 smithi115 ceph-mon[30349]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:29:22.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:22 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:22.148 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 05:29:22 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3[48887]: 2024-05-09T05:29:22.011+0000 7faf206ef700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T05:29:22.148 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 05:29:22 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3[48887]: 2024-05-09T05:29:22.019+0000 7faf1c4a4700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:29:22.251 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:22 smithi161 ceph-mon[33805]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:22.251 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:22 smithi161 ceph-mon[33805]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T05:29:22.251 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:22 smithi161 ceph-mon[33805]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T05:29:22.251 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:22 smithi161 ceph-mon[33805]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:29:22.251 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:22 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:22.306 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:22 smithi195 ceph-mon[33844]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:22.306 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:22 smithi195 ceph-mon[33844]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T05:29:22.306 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:22 smithi195 ceph-mon[33844]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T05:29:22.306 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:22 smithi195 ceph-mon[33844]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:29:22.306 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:22 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:22.814 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T05:29:22.814 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T05:29:22.814 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-09T05:29:22.814 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-09T05:29:22.814 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0196128 s, 535 MB/s 2024-05-09T05:29:22.815 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-09T05:29:23.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:23 smithi195 ceph-mon[33844]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:29:23.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:23 smithi195 ceph-mon[33844]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T05:29:23.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:23.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:23.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:23 smithi195 ceph-mon[33844]: osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] boot 2024-05-09T05:29:23.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:23 smithi195 ceph-mon[33844]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T05:29:23.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:23.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:23 smithi115 ceph-mon[30349]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:29:23.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:23 smithi115 ceph-mon[30349]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T05:29:23.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:23.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:23.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:23 smithi115 ceph-mon[30349]: osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] boot 2024-05-09T05:29:23.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:23 smithi115 ceph-mon[30349]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T05:29:23.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:23.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:23 smithi161 ceph-mon[33805]: from='osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:29:23.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:23 smithi161 ceph-mon[33805]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T05:29:23.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:23.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:23.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:23 smithi161 ceph-mon[33805]: osd.3 [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] boot 2024-05-09T05:29:23.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:23 smithi161 ceph-mon[33805]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T05:29:23.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:29:23.522 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-05-09T05:29:23.766 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:29:24.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:24.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:29:24.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:29:24.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:24.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T05:29:24.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:24.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:24.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:24.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:24.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:29:24.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:29:24.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-09T05:29:24.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T05:29:24.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:24.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:24.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:24.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:24.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:24.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:25.153 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:25 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi115 2024-05-09T05:29:25.153 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:25 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi115 to 8544M 2024-05-09T05:29:25.153 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:25 smithi161 ceph-mon[33805]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T05:29:25.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:25 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi115 2024-05-09T05:29:25.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:25 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi115 to 8544M 2024-05-09T05:29:25.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:25 smithi195 ceph-mon[33844]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T05:29:25.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.379+0000 7f5e05ac9700 1 -- 172.21.15.161:0/883656682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e0010a170 msgr2=0x7f5e0010c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:25.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.379+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/883656682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e0010a170 0x7f5e0010c620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009990 tx=0x7f5df8009ca0 comp rx=0 tx=0).stop 2024-05-09T05:29:25.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.379+0000 7f5e05ac9700 1 -- 172.21.15.161:0/883656682 shutdown_connections 2024-05-09T05:29:25.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.379+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/883656682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e0010a170 0x7f5e0010c620 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:25.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.379+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/883656682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e00102770 0x7f5e00102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:25.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.379+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/883656682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e00107810 0x7f5e00109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:25.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.379+0000 7f5e05ac9700 1 -- 172.21.15.161:0/883656682 >> 172.21.15.161:0/883656682 conn(0x7f5e000fe020 msgr2=0x7f5e001004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:29:25.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.379+0000 7f5e05ac9700 1 -- 172.21.15.161:0/883656682 shutdown_connections 2024-05-09T05:29:25.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.379+0000 7f5e05ac9700 1 -- 172.21.15.161:0/883656682 wait complete. 2024-05-09T05:29:25.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5e05ac9700 1 Processor -- start 2024-05-09T05:29:25.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5e05ac9700 1 -- start start 2024-05-09T05:29:25.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5e05ac9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e00102770 0x7f5e0019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:25.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5e05ac9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e00107810 0x7f5e0019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:25.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5e05ac9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e0010a170 0x7f5e001a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:25.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5e05ac9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0010fd80 con 0x7f5e0010a170 2024-05-09T05:29:25.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5e05ac9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0010fc00 con 0x7f5e00107810 2024-05-09T05:29:25.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5e05ac9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0010ff00 con 0x7f5e00102770 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5dfeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e00107810 0x7f5e0019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5dfeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e00107810 0x7f5e0019ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42604/0 (socket says 172.21.15.161:42604) 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5dfeffd700 1 -- 172.21.15.161:0/2146646238 learned_addr learned my addr 172.21.15.161:0/2146646238 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.380+0000 7f5dfffff700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e0010a170 0x7f5e001a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dff7fe700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e00102770 0x7f5e0019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dfeffd700 1 -- 172.21.15.161:0/2146646238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e00102770 msgr2=0x7f5e0019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dfeffd700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e00102770 0x7f5e0019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dfeffd700 1 -- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e0010a170 msgr2=0x7f5e001a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dfeffd700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e0010a170 0x7f5e001a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dfeffd700 1 -- 172.21.15.161:0/2146646238 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e001a2420 con 0x7f5e00107810 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dfeffd700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e00107810 0x7f5e0019ae00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5df400b530 tx=0x7f5df400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dfcff9700 1 -- 172.21.15.161:0/2146646238 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4015020 con 0x7f5e00107810 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e001a2710 con 0x7f5e00107810 2024-05-09T05:29:25.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dfcff9700 1 -- 172.21.15.161:0/2146646238 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df4011cc0 con 0x7f5e00107810 2024-05-09T05:29:25.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5dfcff9700 1 -- 172.21.15.161:0/2146646238 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4010c80 con 0x7f5e00107810 2024-05-09T05:29:25.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.381+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e001a2d20 con 0x7f5e00107810 2024-05-09T05:29:25.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.382+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e000512c0 con 0x7f5e00107810 2024-05-09T05:29:25.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.384+0000 7f5dfcff9700 1 -- 172.21.15.161:0/2146646238 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f5df4018020 con 0x7f5e00107810 2024-05-09T05:29:25.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.384+0000 7f5dfcff9700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f5de80b2880 0x7f5de80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:25.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.385+0000 7f5dfcff9700 1 -- 172.21.15.161:0/2146646238 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f5df40e0c70 con 0x7f5e00107810 2024-05-09T05:29:25.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.385+0000 7f5dff7fe700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f5de80b2880 0x7f5de80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:25.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.385+0000 7f5dff7fe700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f5de80b2880 0x7f5de80b4d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009550 tx=0x7f5df00091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:29:25.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.388+0000 7f5dfcff9700 1 -- 172.21.15.161:0/2146646238 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5df401f020 con 0x7f5e00107810 2024-05-09T05:29:25.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:25 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi115 2024-05-09T05:29:25.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:25 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi115 to 8544M 2024-05-09T05:29:25.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:25 smithi115 ceph-mon[30349]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T05:29:25.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:25.602+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5e00063c00 con 0x7f5de80b2880 2024-05-09T05:29:26.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:26 smithi161 ceph-mon[33805]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:26.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:29:26.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:29:26.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:26.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:26 smithi195 ceph-mon[33844]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:26.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:29:26.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:29:26.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:26.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:26 smithi115 ceph-mon[30349]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:26.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:29:26.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:29:26.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:27.227 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:27 smithi161 ceph-mon[33805]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:29:27.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:27 smithi195 ceph-mon[33844]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:29:27.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:27 smithi115 ceph-mon[30349]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:29:28.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:28 smithi195 ceph-mon[33844]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:28.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:28 smithi115 ceph-mon[30349]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:28.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:28 smithi161 ceph-mon[33805]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:29.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:29 smithi161 ceph-mon[33805]: from='client.? 172.21.15.161:0/1168572089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70e4d345-0e86-4383-8051-be261d261fd0"}]: dispatch 2024-05-09T05:29:29.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:29 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70e4d345-0e86-4383-8051-be261d261fd0"}]: dispatch 2024-05-09T05:29:29.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:29 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70e4d345-0e86-4383-8051-be261d261fd0"}]': finished 2024-05-09T05:29:29.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:29 smithi161 ceph-mon[33805]: osdmap e28: 5 total, 4 up, 5 in 2024-05-09T05:29:29.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:29 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:29.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:29 smithi195 ceph-mon[33844]: from='client.? 172.21.15.161:0/1168572089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70e4d345-0e86-4383-8051-be261d261fd0"}]: dispatch 2024-05-09T05:29:29.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:29 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70e4d345-0e86-4383-8051-be261d261fd0"}]: dispatch 2024-05-09T05:29:29.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:29 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70e4d345-0e86-4383-8051-be261d261fd0"}]': finished 2024-05-09T05:29:29.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:29 smithi195 ceph-mon[33844]: osdmap e28: 5 total, 4 up, 5 in 2024-05-09T05:29:29.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:29 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:29.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:29 smithi115 ceph-mon[30349]: from='client.? 172.21.15.161:0/1168572089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70e4d345-0e86-4383-8051-be261d261fd0"}]: dispatch 2024-05-09T05:29:29.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:29 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70e4d345-0e86-4383-8051-be261d261fd0"}]: dispatch 2024-05-09T05:29:29.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:29 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70e4d345-0e86-4383-8051-be261d261fd0"}]': finished 2024-05-09T05:29:29.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:29 smithi115 ceph-mon[30349]: osdmap e28: 5 total, 4 up, 5 in 2024-05-09T05:29:29.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:29 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:30.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:30 smithi195 ceph-mon[33844]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:30.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:30 smithi195 ceph-mon[33844]: from='client.? 172.21.15.161:0/2463907921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T05:29:30.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:30 smithi115 ceph-mon[30349]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:30.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:30 smithi115 ceph-mon[30349]: from='client.? 172.21.15.161:0/2463907921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T05:29:30.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:30 smithi161 ceph-mon[33805]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:30.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:30 smithi161 ceph-mon[33805]: from='client.? 172.21.15.161:0/2463907921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T05:29:32.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:32 smithi195 ceph-mon[33844]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:29:32.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:32 smithi115 ceph-mon[30349]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:29:32.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:32 smithi161 ceph-mon[33805]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:29:34.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:34 smithi195 ceph-mon[33844]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T05:29:34.389 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:34 smithi161 ceph-mon[33805]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T05:29:34.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:34 smithi115 ceph-mon[30349]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T05:29:36.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:36 smithi195 ceph-mon[33844]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:29:36.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T05:29:36.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:36.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:36 smithi115 ceph-mon[30349]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:29:36.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T05:29:36.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:36.399 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:36 smithi161 ceph-mon[33805]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:29:36.399 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T05:29:36.399 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:37.354 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:37 smithi161 ceph-mon[33805]: Deploying daemon osd.4 on smithi161 2024-05-09T05:29:37.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:37 smithi195 ceph-mon[33844]: Deploying daemon osd.4 on smithi161 2024-05-09T05:29:37.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:37 smithi115 ceph-mon[30349]: Deploying daemon osd.4 on smithi161 2024-05-09T05:29:38.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:38 smithi195 ceph-mon[33844]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:29:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:38 smithi115 ceph-mon[30349]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:29:38.401 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:38 smithi161 ceph-mon[33805]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:29:40.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:40 smithi195 ceph-mon[33844]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-09T05:29:40.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:40 smithi115 ceph-mon[30349]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-09T05:29:40.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:40 smithi161 ceph-mon[33805]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-09T05:29:42.288 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:42 smithi161 ceph-mon[33805]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T05:29:42.288 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:42.288 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:42.289 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:42 smithi195 ceph-mon[33844]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T05:29:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:42.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:42.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:42.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:42 smithi115 ceph-mon[30349]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T05:29:42.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:42.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:42.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:44.125 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:44 smithi161 ceph-mon[33805]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:44 smithi195 ceph-mon[33844]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:44.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:44 smithi115 ceph-mon[30349]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:45.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.168+0000 7f5dfcff9700 1 -- 172.21.15.161:0/2146646238 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5e00063c00 con 0x7f5de80b2880 2024-05-09T05:29:45.163 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 4 on host 'smithi161' 2024-05-09T05:29:45.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f5de80b2880 msgr2=0x7f5de80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f5de80b2880 0x7f5de80b4d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009550 tx=0x7f5df00091b0 comp rx=0 tx=0).stop 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e00107810 msgr2=0x7f5e0019ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e00107810 0x7f5e0019ae00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5df400b530 tx=0x7f5df400b840 comp rx=0 tx=0).stop 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 shutdown_connections 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f5de80b2880 0x7f5de80b4d90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e0010a170 0x7f5e001a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e00102770 0x7f5e0019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 --2- 172.21.15.161:0/2146646238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e00107810 0x7f5e0019ae00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 >> 172.21.15.161:0/2146646238 conn(0x7f5e000fe020 msgr2=0x7f5e00108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 shutdown_connections 2024-05-09T05:29:45.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:29:45.171+0000 7f5e05ac9700 1 -- 172.21.15.161:0/2146646238 wait complete. 2024-05-09T05:29:45.770 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='osd.4 [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T05:29:45.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:45 smithi161 ceph-mon[33805]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T05:29:45.845 DEBUG:teuthology.orchestra.run.smithi161:osd.4> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.4.service 2024-05-09T05:29:45.848 INFO:tasks.cephadm:Deploying osd.5 on smithi195 with /dev/vg_nvme/lv_3... 2024-05-09T05:29:45.848 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T05:29:45.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='osd.4 [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T05:29:45.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:45 smithi195 ceph-mon[33844]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T05:29:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:29:45.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='osd.4 [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T05:29:45.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:45 smithi115 ceph-mon[30349]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T05:29:46.095 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:29:47.238 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:47 smithi195 ceph-mon[33844]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T05:29:47.239 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:47 smithi195 ceph-mon[33844]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T05:29:47.239 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:47.239 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:47 smithi195 ceph-mon[33844]: from='osd.4 [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:29:47.239 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:47 smithi195 ceph-mon[33844]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:29:47.362 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:47 smithi161 ceph-mon[33805]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T05:29:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:47 smithi161 ceph-mon[33805]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T05:29:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:47 smithi161 ceph-mon[33805]: from='osd.4 [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:29:47.363 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:47 smithi161 ceph-mon[33805]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:29:47.363 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 05:29:47 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4[44874]: 2024-05-09T05:29:47.170+0000 7f2b1d1cb700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T05:29:47.363 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 05:29:47 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4[44874]: 2024-05-09T05:29:47.175+0000 7f2b18f80700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:29:47.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:47 smithi115 ceph-mon[30349]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T05:29:47.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:47 smithi115 ceph-mon[30349]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T05:29:47.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:47.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:47 smithi115 ceph-mon[30349]: from='osd.4 [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:29:47.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:47 smithi115 ceph-mon[30349]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:29:48.079 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T05:29:48.079 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T05:29:48.079 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-05-09T05:29:48.079 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-05-09T05:29:48.079 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00847001 s, 1.2 GB/s 2024-05-09T05:29:48.079 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-05-09T05:29:48.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:48 smithi195 ceph-mon[33844]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:48.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:48 smithi195 ceph-mon[33844]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:29:48.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:48 smithi195 ceph-mon[33844]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T05:29:48.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:48.379 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:48.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:48 smithi161 ceph-mon[33805]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:48.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:48 smithi161 ceph-mon[33805]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:29:48.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:48 smithi161 ceph-mon[33805]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T05:29:48.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:48.515 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:48 smithi115 ceph-mon[30349]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-05-09T05:29:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:48 smithi115 ceph-mon[30349]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:29:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:48 smithi115 ceph-mon[30349]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T05:29:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:48.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:48.707 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_3 2024-05-09T05:29:48.993 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:29:49.425 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:29:49.425 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: osd.4 [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] boot 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: osdmap e31: 5 total, 5 up, 5 in 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:49.426 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:29:49.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:29:49.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: osd.4 [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] boot 2024-05-09T05:29:49.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: osdmap e31: 5 total, 5 up, 5 in 2024-05-09T05:29:49.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:49.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:49.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:49.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:49.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:49.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:29:49.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: osd.4 [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] boot 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: osdmap e31: 5 total, 5 up, 5 in 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:29:49.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:29:50.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:50 smithi195 ceph-mon[33844]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:50.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:50 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi161 2024-05-09T05:29:50.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:50 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi161 to 8544M 2024-05-09T05:29:50.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:50 smithi195 ceph-mon[33844]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T05:29:50.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:50 smithi115 ceph-mon[30349]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:50.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:50 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi161 2024-05-09T05:29:50.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:50 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi161 to 8544M 2024-05-09T05:29:50.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:50 smithi115 ceph-mon[30349]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T05:29:50.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:50 smithi161 ceph-mon[33805]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:50.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:50 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi161 2024-05-09T05:29:50.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:50 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi161 to 8544M 2024-05-09T05:29:50.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:50 smithi161 ceph-mon[33805]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T05:29:50.752 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.752+0000 7fdcbbd71700 1 -- 172.21.15.195:0/3091562273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 msgr2=0x7fdcb4104370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:50.752 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.752+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/3091562273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 0x7fdcb4104370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009990 tx=0x7fdca4009ca0 comp rx=0 tx=0).stop 2024-05-09T05:29:50.752 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.752+0000 7fdcbbd71700 1 -- 172.21.15.195:0/3091562273 shutdown_connections 2024-05-09T05:29:50.752 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.752+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/3091562273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41049e0 0x7fdcb410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:50.752 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.752+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/3091562273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41059b0 0x7fdcb4105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:50.752 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.752+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/3091562273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 0x7fdcb4104370 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:50.752 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.752+0000 7fdcbbd71700 1 -- 172.21.15.195:0/3091562273 >> 172.21.15.195:0/3091562273 conn(0x7fdcb40ff9d0 msgr2=0x7fdcb4101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:29:50.752 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.752+0000 7fdcbbd71700 1 -- 172.21.15.195:0/3091562273 shutdown_connections 2024-05-09T05:29:50.752 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.752+0000 7fdcbbd71700 1 -- 172.21.15.195:0/3091562273 wait complete. 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcbbd71700 1 Processor -- start 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcbbd71700 1 -- start start 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcbbd71700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 0x7fdcb419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcbbd71700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41049e0 0x7fdcb419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcbbd71700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41059b0 0x7fdcb41a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcbbd71700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb410fe10 con 0x7fdcb41059b0 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcbbd71700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb410fc90 con 0x7fdcb41049e0 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcbbd71700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb410ff90 con 0x7fdcb4103f60 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb9b0d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 0x7fdcb419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb9b0d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 0x7fdcb419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:47408/0 (socket says 172.21.15.195:47408) 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb9b0d700 1 -- 172.21.15.195:0/2907162408 learned_addr learned my addr 172.21.15.195:0/2907162408 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcba30e700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41059b0 0x7fdcb41a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb9b0d700 1 -- 172.21.15.195:0/2907162408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41049e0 msgr2=0x7fdcb419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb930c700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41049e0 0x7fdcb419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:50.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb9b0d700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41049e0 0x7fdcb419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb9b0d700 1 -- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41059b0 msgr2=0x7fdcb41a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb9b0d700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41059b0 0x7fdcb41a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb9b0d700 1 -- 172.21.15.195:0/2907162408 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb41a25a0 con 0x7fdcb4103f60 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcba30e700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41059b0 0x7fdcb41a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.753+0000 7fdcb9b0d700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 0x7fdcb419aa40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000d950 tx=0x7fdcb000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.754+0000 7fdcaaffd700 1 -- 172.21.15.195:0/2907162408 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0017070 con 0x7fdcb4103f60 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.754+0000 7fdcaaffd700 1 -- 172.21.15.195:0/2907162408 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcb000de30 con 0x7fdcb4103f60 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.754+0000 7fdcaaffd700 1 -- 172.21.15.195:0/2907162408 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0003aa0 con 0x7fdcb4103f60 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.754+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb41a2890 con 0x7fdcb4103f60 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.754+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb41a2d80 con 0x7fdcb4103f60 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.755+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb40512c0 con 0x7fdcb4103f60 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.756+0000 7fdcaaffd700 1 -- 172.21.15.195:0/2907162408 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fdcb0003c40 con 0x7fdcb4103f60 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.757+0000 7fdcaaffd700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdca00b2900 0x7fdca00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:29:50.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.757+0000 7fdcaaffd700 1 -- 172.21.15.195:0/2907162408 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fdcb00e14c0 con 0x7fdcb4103f60 2024-05-09T05:29:50.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.757+0000 7fdcb930c700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdca00b2900 0x7fdca00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:29:50.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.758+0000 7fdcb930c700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdca00b2900 0x7fdca00b4e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdca400b5c0 tx=0x7fdca4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:29:50.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.761+0000 7fdcaaffd700 1 -- 172.21.15.195:0/2907162408 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdcb0020030 con 0x7fdcb4103f60 2024-05-09T05:29:50.964 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:29:50.969+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdcb4063c00 con 0x7fdca00b2900 2024-05-09T05:29:51.580 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:29:51.581 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:29:51.581 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:51.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:29:51.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:29:51.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:51.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:29:51.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:29:51.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:29:52.278 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:52 smithi195 ceph-mon[33844]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:52.278 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:52 smithi195 ceph-mon[33844]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:29:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:52 smithi115 ceph-mon[30349]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:52 smithi115 ceph-mon[30349]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:29:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:52 smithi161 ceph-mon[33805]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:52 smithi161 ceph-mon[33805]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:29:54.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:54 smithi195 ceph-mon[33844]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:54.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:54 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/1518201610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8883101f-256e-4fa9-aa1f-0b5167462282"}]: dispatch 2024-05-09T05:29:54.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:54 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8883101f-256e-4fa9-aa1f-0b5167462282"}]: dispatch 2024-05-09T05:29:54.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:54 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8883101f-256e-4fa9-aa1f-0b5167462282"}]': finished 2024-05-09T05:29:54.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:54 smithi195 ceph-mon[33844]: osdmap e33: 6 total, 5 up, 6 in 2024-05-09T05:29:54.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:54 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:29:54.449 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:54 smithi115 ceph-mon[30349]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:54.449 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:54 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/1518201610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8883101f-256e-4fa9-aa1f-0b5167462282"}]: dispatch 2024-05-09T05:29:54.449 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:54 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8883101f-256e-4fa9-aa1f-0b5167462282"}]: dispatch 2024-05-09T05:29:54.449 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:54 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8883101f-256e-4fa9-aa1f-0b5167462282"}]': finished 2024-05-09T05:29:54.449 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:54 smithi115 ceph-mon[30349]: osdmap e33: 6 total, 5 up, 6 in 2024-05-09T05:29:54.449 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:29:54.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:54 smithi161 ceph-mon[33805]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:54.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:54 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/1518201610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8883101f-256e-4fa9-aa1f-0b5167462282"}]: dispatch 2024-05-09T05:29:54.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:54 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8883101f-256e-4fa9-aa1f-0b5167462282"}]: dispatch 2024-05-09T05:29:54.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:54 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8883101f-256e-4fa9-aa1f-0b5167462282"}]': finished 2024-05-09T05:29:54.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:54 smithi161 ceph-mon[33805]: osdmap e33: 6 total, 5 up, 6 in 2024-05-09T05:29:54.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:54 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:29:55.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:55 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/802443748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T05:29:55.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:55 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/802443748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T05:29:55.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:55 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/802443748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T05:29:56.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:56 smithi195 ceph-mon[33844]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:56.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:56 smithi115 ceph-mon[30349]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:56.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:56 smithi161 ceph-mon[33805]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:58.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:29:58 smithi195 ceph-mon[33844]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:58.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:29:58 smithi115 ceph-mon[30349]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:29:58.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:29:58 smithi161 ceph-mon[33805]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:00.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:00 smithi195 ceph-mon[33844]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:00.260 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:00 smithi195 ceph-mon[33844]: overall HEALTH_OK 2024-05-09T05:30:00.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:00 smithi115 ceph-mon[30349]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:00.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:00 smithi115 ceph-mon[30349]: overall HEALTH_OK 2024-05-09T05:30:00.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:00 smithi161 ceph-mon[33805]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:00.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:00 smithi161 ceph-mon[33805]: overall HEALTH_OK 2024-05-09T05:30:01.585 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T05:30:01.586 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:01.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T05:30:01.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T05:30:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:02.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:02 smithi195 ceph-mon[33844]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:02.568 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:02 smithi195 ceph-mon[33844]: Deploying daemon osd.5 on smithi195 2024-05-09T05:30:02.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:02 smithi115 ceph-mon[30349]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:02.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:02 smithi115 ceph-mon[30349]: Deploying daemon osd.5 on smithi195 2024-05-09T05:30:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:02 smithi161 ceph-mon[33805]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:02 smithi161 ceph-mon[33805]: Deploying daemon osd.5 on smithi195 2024-05-09T05:30:04.520 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:04 smithi195 ceph-mon[33844]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:04 smithi115 ceph-mon[30349]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:04.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:04 smithi161 ceph-mon[33805]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:06.355 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:06 smithi195 ceph-mon[33844]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:06.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:06 smithi115 ceph-mon[30349]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:06.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:06 smithi161 ceph-mon[33805]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:07.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:07 smithi115 ceph-mon[30349]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:07.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:07.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:07.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:07.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:07 smithi161 ceph-mon[33805]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:07.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:07.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:07.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:08.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:07 smithi195 ceph-mon[33844]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:08.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:08.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:08.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:09.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:09 smithi195 ceph-mon[33844]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:09.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:09 smithi115 ceph-mon[30349]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:09.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:09 smithi161 ceph-mon[33805]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:10.230 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 5 on host 'smithi195' 2024-05-09T05:30:10.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.235+0000 7fdcaaffd700 1 -- 172.21.15.195:0/2907162408 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdcb4063c00 con 0x7fdca00b2900 2024-05-09T05:30:10.232 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdca00b2900 msgr2=0x7fdca00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:10.232 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdca00b2900 0x7fdca00b4e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdca400b5c0 tx=0x7fdca4005fb0 comp rx=0 tx=0).stop 2024-05-09T05:30:10.232 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 msgr2=0x7fdcb419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:10.232 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 0x7fdcb419aa40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000d950 tx=0x7fdcb000dd10 comp rx=0 tx=0).stop 2024-05-09T05:30:10.233 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 shutdown_connections 2024-05-09T05:30:10.233 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fdca00b2900 0x7fdca00b4e10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:10.233 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb41059b0 0x7fdcb41a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:10.233 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcb4103f60 0x7fdcb419aa40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:10.233 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 --2- 172.21.15.195:0/2907162408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb41049e0 0x7fdcb419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:10.233 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 >> 172.21.15.195:0/2907162408 conn(0x7fdcb40ff9d0 msgr2=0x7fdcb41086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:30:10.233 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 shutdown_connections 2024-05-09T05:30:10.233 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:30:10.237+0000 7fdcbbd71700 1 -- 172.21.15.195:0/2907162408 wait complete. 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='osd.5 [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T05:30:10.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:10 smithi195 ceph-mon[33844]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T05:30:10.977 DEBUG:teuthology.orchestra.run.smithi195:osd.5> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.5.service 2024-05-09T05:30:10.980 INFO:tasks.cephadm:Deploying osd.6 on smithi115 with /dev/vg_nvme/lv_2... 2024-05-09T05:30:10.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='osd.5 [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T05:30:11.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:10 smithi115 ceph-mon[30349]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='osd.5 [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T05:30:11.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:10 smithi161 ceph-mon[33805]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T05:30:11.228 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:30:12.558 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:12 smithi115 ceph-mon[30349]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:12.558 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:12 smithi115 ceph-mon[30349]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T05:30:12.558 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:12 smithi115 ceph-mon[30349]: from='osd.5 [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:30:12.558 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:12 smithi115 ceph-mon[30349]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T05:30:12.558 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:12 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:12.558 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:12 smithi115 ceph-mon[30349]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:30:12.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:12 smithi195 ceph-mon[33844]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:12.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:12 smithi195 ceph-mon[33844]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T05:30:12.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:12 smithi195 ceph-mon[33844]: from='osd.5 [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:30:12.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:12 smithi195 ceph-mon[33844]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T05:30:12.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:12 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:12.567 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:12 smithi195 ceph-mon[33844]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:30:12.567 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 05:30:12 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5[45486]: 2024-05-09T05:30:12.249+0000 7fc8ffe56700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T05:30:12.568 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 05:30:12 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5[45486]: 2024-05-09T05:30:12.260+0000 7fc8fbc0b700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:30:12.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:12 smithi161 ceph-mon[33805]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:12.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:12 smithi161 ceph-mon[33805]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T05:30:12.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:12 smithi161 ceph-mon[33805]: from='osd.5 [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:30:12.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:12 smithi161 ceph-mon[33805]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T05:30:12.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:12 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:12.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:12 smithi161 ceph-mon[33805]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:30:13.141 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T05:30:13.141 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T05:30:13.141 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-05-09T05:30:13.141 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-05-09T05:30:13.141 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114897 s, 913 MB/s 2024-05-09T05:30:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-05-09T05:30:13.459 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:13 smithi115 ceph-mon[30349]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:30:13.459 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:13 smithi115 ceph-mon[30349]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T05:30:13.459 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:13 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:13.459 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:13 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:13.563 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:13 smithi161 ceph-mon[33805]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:30:13.563 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:13 smithi161 ceph-mon[33805]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T05:30:13.563 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:13 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:13.563 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:13 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:13.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:13 smithi195 ceph-mon[33844]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:30:13.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:13 smithi195 ceph-mon[33844]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T05:30:13.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:13 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:13.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:13 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:13.760 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2024-05-09T05:30:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:30:14.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: osd.5 [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] boot 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: osdmap e36: 6 total, 6 up, 6 in 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:14.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:14.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:14.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:14.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:14 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:30:14.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:30:14.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:14.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:14.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: osd.5 [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] boot 2024-05-09T05:30:14.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: osdmap e36: 6 total, 6 up, 6 in 2024-05-09T05:30:14.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:14.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T05:30:14.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:14.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:14.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.398 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:14.399 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:14.399 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:14 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: osd.5 [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] boot 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: osdmap e36: 6 total, 6 up, 6 in 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:14.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:14.450 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:14.450 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:14 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:15.359 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:15 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi195 2024-05-09T05:30:15.359 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:15 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi195 to 8544M 2024-05-09T05:30:15.359 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:15 smithi115 ceph-mon[30349]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.606+0000 7f73770b7700 1 -- 172.21.15.115:0/4071048887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 msgr2=0x7f737010b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.606+0000 7f73770b7700 1 --2- 172.21.15.115:0/4071048887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 0x7f737010b770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7368009990 tx=0x7f7368009ca0 comp rx=0 tx=0).stop 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.606+0000 7f73770b7700 1 -- 172.21.15.115:0/4071048887 shutdown_connections 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.606+0000 7f73770b7700 1 --2- 172.21.15.115:0/4071048887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73701037a0 0x7f7370103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.606+0000 7f73770b7700 1 --2- 172.21.15.115:0/4071048887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 0x7f737010b770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.606+0000 7f73770b7700 1 --2- 172.21.15.115:0/4071048887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7370104180 0x7f7370104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.606+0000 7f73770b7700 1 -- 172.21.15.115:0/4071048887 >> 172.21.15.115:0/4071048887 conn(0x7f73700fe010 msgr2=0x7f73701004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.606+0000 7f73770b7700 1 -- 172.21.15.115:0/4071048887 shutdown_connections 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.606+0000 7f73770b7700 1 -- 172.21.15.115:0/4071048887 wait complete. 2024-05-09T05:30:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f73770b7700 1 Processor -- start 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f73770b7700 1 -- start start 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f73770b7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701037a0 0x7f7370102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f73770b7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7370104180 0x7f737019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f73770b7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 0x7f73701a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f73770b7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f737010ebc0 con 0x7f7370104ce0 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f73770b7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f737010ea40 con 0x7f7370104180 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f73770b7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f737010ed40 con 0x7f73701037a0 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f7375654700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 0x7f73701a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f7375654700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 0x7f73701a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47360/0 (socket says 172.21.15.115:47360) 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f7375654700 1 -- 172.21.15.115:0/4103723419 learned_addr learned my addr 172.21.15.115:0/4103723419 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.607+0000 7f7374e53700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701037a0 0x7f7370102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7367fff700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7370104180 0x7f737019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7375654700 1 -- 172.21.15.115:0/4103723419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701037a0 msgr2=0x7f7370102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7375654700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701037a0 0x7f7370102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7375654700 1 -- 172.21.15.115:0/4103723419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7370104180 msgr2=0x7f737019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7375654700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7370104180 0x7f737019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7375654700 1 -- 172.21.15.115:0/4103723419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73701a2390 con 0x7f7370104ce0 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7367fff700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7370104180 0x7f737019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7375654700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 0x7f73701a1b60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7368003760 tx=0x7f73680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7365ffb700 1 -- 172.21.15.115:0/4103723419 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368004830 con 0x7f7370104ce0 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73701a2620 con 0x7f7370104ce0 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7365ffb700 1 -- 172.21.15.115:0/4103723419 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7368003b90 con 0x7f7370104ce0 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f7365ffb700 1 -- 172.21.15.115:0/4103723419 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368005280 con 0x7f7370104ce0 2024-05-09T05:30:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.608+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73701a2a90 con 0x7f7370104ce0 2024-05-09T05:30:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.609+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73700512c0 con 0x7f7370104ce0 2024-05-09T05:30:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.611+0000 7f7365ffb700 1 -- 172.21.15.115:0/4103723419 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7368005420 con 0x7f7370104ce0 2024-05-09T05:30:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.612+0000 7f7365ffb700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f73580b2880 0x7f73580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:30:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.612+0000 7f7374e53700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f73580b2880 0x7f73580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:30:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.612+0000 7f7365ffb700 1 -- 172.21.15.115:0/4103723419 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f73680e2750 con 0x7f7370104ce0 2024-05-09T05:30:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.612+0000 7f7374e53700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f73580b2880 0x7f73580b4d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f736c009550 tx=0x7f736c0091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:30:15.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.616+0000 7f7365ffb700 1 -- 172.21.15.115:0/4103723419 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7368013070 con 0x7f7370104ce0 2024-05-09T05:30:15.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:15 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi195 2024-05-09T05:30:15.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:15 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi195 to 8544M 2024-05-09T05:30:15.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:15 smithi195 ceph-mon[33844]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T05:30:15.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:15 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi195 2024-05-09T05:30:15.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:15 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi195 to 8544M 2024-05-09T05:30:15.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:15 smithi161 ceph-mon[33805]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T05:30:15.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:15.835+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7370063cc0 con 0x7f73580b2880 2024-05-09T05:30:16.567 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:16 smithi115 ceph-mon[30349]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:16.567 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:16 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:30:16.567 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:16 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:30:16.567 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:16 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:16.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:16 smithi195 ceph-mon[33844]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:16.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:30:16.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:30:16.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:16 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:16 smithi161 ceph-mon[33805]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:16 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:30:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:16 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:30:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:16 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:17 smithi195 ceph-mon[33844]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:30:17.617 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:17 smithi115 ceph-mon[30349]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:30:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:17 smithi161 ceph-mon[33805]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:30:18.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:18 smithi195 ceph-mon[33844]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:18.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:18 smithi115 ceph-mon[30349]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:18 smithi161 ceph-mon[33805]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:19.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:19 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3948016565' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c3cc250-18ea-4a70-972e-3c58917ccced"}]: dispatch 2024-05-09T05:30:19.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:19 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3948016565' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c3cc250-18ea-4a70-972e-3c58917ccced"}]': finished 2024-05-09T05:30:19.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:19 smithi195 ceph-mon[33844]: osdmap e39: 7 total, 6 up, 7 in 2024-05-09T05:30:19.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:19 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:19.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:19 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3948016565' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c3cc250-18ea-4a70-972e-3c58917ccced"}]: dispatch 2024-05-09T05:30:19.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:19 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3948016565' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c3cc250-18ea-4a70-972e-3c58917ccced"}]': finished 2024-05-09T05:30:19.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:19 smithi115 ceph-mon[30349]: osdmap e39: 7 total, 6 up, 7 in 2024-05-09T05:30:19.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:19 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:19.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:19 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3948016565' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c3cc250-18ea-4a70-972e-3c58917ccced"}]: dispatch 2024-05-09T05:30:19.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:19 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3948016565' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c3cc250-18ea-4a70-972e-3c58917ccced"}]': finished 2024-05-09T05:30:19.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:19 smithi161 ceph-mon[33805]: osdmap e39: 7 total, 6 up, 7 in 2024-05-09T05:30:19.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:19 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:20.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:20 smithi195 ceph-mon[33844]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:20.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:20 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1332685269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T05:30:20.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:20 smithi115 ceph-mon[30349]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:20.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:20 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1332685269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T05:30:20.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:20 smithi161 ceph-mon[33805]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:20.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:20 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1332685269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T05:30:22.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:22 smithi195 ceph-mon[33844]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:30:22.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:22 smithi115 ceph-mon[30349]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:30:22.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:22 smithi161 ceph-mon[33805]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:30:24.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:24 smithi195 ceph-mon[33844]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T05:30:24.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:24 smithi115 ceph-mon[30349]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T05:30:24.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:24 smithi161 ceph-mon[33805]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T05:30:26.275 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:26 smithi115 ceph-mon[30349]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:30:26.275 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T05:30:26.275 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:26.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:26 smithi195 ceph-mon[33844]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:30:26.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T05:30:26.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:26.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:26 smithi161 ceph-mon[33805]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:30:26.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T05:30:26.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:27 smithi195 ceph-mon[33844]: Deploying daemon osd.6 on smithi115 2024-05-09T05:30:27.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:27 smithi115 ceph-mon[30349]: Deploying daemon osd.6 on smithi115 2024-05-09T05:30:27.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:27 smithi161 ceph-mon[33805]: Deploying daemon osd.6 on smithi115 2024-05-09T05:30:28.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:28 smithi115 ceph-mon[30349]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:30:28.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:28 smithi195 ceph-mon[33844]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:30:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:28 smithi161 ceph-mon[33805]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:30:30.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:30 smithi195 ceph-mon[33844]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:30:30.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:30 smithi115 ceph-mon[30349]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:30:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:30 smithi161 ceph-mon[33805]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:30:32.588 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:32 smithi115 ceph-mon[30349]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T05:30:32.588 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:32.588 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:32.588 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:32 smithi195 ceph-mon[33844]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T05:30:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:32 smithi161 ceph-mon[33805]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T05:30:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:34.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:34 smithi195 ceph-mon[33844]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:34.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:34 smithi115 ceph-mon[30349]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:34.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:34 smithi161 ceph-mon[33805]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:35.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.509+0000 7f7365ffb700 1 -- 172.21.15.115:0/4103723419 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7370063cc0 con 0x7f73580b2880 2024-05-09T05:30:35.504 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 6 on host 'smithi115' 2024-05-09T05:30:35.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f73580b2880 msgr2=0x7f73580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:35.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f73580b2880 0x7f73580b4d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f736c009550 tx=0x7f736c0091b0 comp rx=0 tx=0).stop 2024-05-09T05:30:35.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 msgr2=0x7f73701a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:35.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 0x7f73701a1b60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7368003760 tx=0x7f73680047b0 comp rx=0 tx=0).stop 2024-05-09T05:30:35.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 shutdown_connections 2024-05-09T05:30:35.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f73580b2880 0x7f73580b4d90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:35.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370104ce0 0x7f73701a1b60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:35.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73701037a0 0x7f7370102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:35.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 --2- 172.21.15.115:0/4103723419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7370104180 0x7f737019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:35.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 >> 172.21.15.115:0/4103723419 conn(0x7f73700fe010 msgr2=0x7f7370107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:30:35.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 shutdown_connections 2024-05-09T05:30:35.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:30:35.512+0000 7f73770b7700 1 -- 172.21.15.115:0/4103723419 wait complete. 2024-05-09T05:30:36.084 DEBUG:teuthology.orchestra.run.smithi115:osd.6> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.6.service 2024-05-09T05:30:36.089 INFO:tasks.cephadm:Deploying osd.7 on smithi161 with /dev/vg_nvme/lv_2... 2024-05-09T05:30:36.090 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T05:30:36.324 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:36 smithi115 ceph-mon[30349]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:36.324 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.324 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.324 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:36 smithi115 ceph-mon[30349]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T05:30:36.324 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.324 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:36 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.345 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:30:36.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:36 smithi161 ceph-mon[33805]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:36.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:36 smithi161 ceph-mon[33805]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T05:30:36.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:36 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:36 smithi195 ceph-mon[33844]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:36.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:36 smithi195 ceph-mon[33844]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T05:30:36.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:36.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:36 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:37.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:37 smithi161 ceph-mon[33805]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T05:30:37.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:37 smithi161 ceph-mon[33805]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T05:30:37.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:37 smithi161 ceph-mon[33805]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:30:37.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:37 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:37.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:37 smithi161 ceph-mon[33805]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:37.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:37 smithi195 ceph-mon[33844]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T05:30:37.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:37 smithi195 ceph-mon[33844]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T05:30:37.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:37 smithi195 ceph-mon[33844]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:30:37.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:37 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:37.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:37 smithi195 ceph-mon[33844]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:37.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:37 smithi115 ceph-mon[30349]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T05:30:37.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:37 smithi115 ceph-mon[30349]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T05:30:37.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:37 smithi115 ceph-mon[30349]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:30:37.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:37 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:37.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:37 smithi115 ceph-mon[30349]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:37.898 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 05:30:37 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6[55419]: 2024-05-09T05:30:37.495+0000 7f660413b700 -1 osd.6 0 waiting for initial osdmap 2024-05-09T05:30:37.898 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 05:30:37 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6[55419]: 2024-05-09T05:30:37.506+0000 7f65ffef0700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:30:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T05:30:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T05:30:38.519 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-09T05:30:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-09T05:30:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142154 s, 738 MB/s 2024-05-09T05:30:38.519 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-09T05:30:38.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:38 smithi161 ceph-mon[33805]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:30:38.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:38 smithi161 ceph-mon[33805]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T05:30:38.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:38.834 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:38.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:38 smithi195 ceph-mon[33844]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:30:38.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:38 smithi195 ceph-mon[33844]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T05:30:38.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:38.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:38.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:38 smithi115 ceph-mon[30349]: from='osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:30:38.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:38 smithi115 ceph-mon[30349]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T05:30:38.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:38.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:39.271 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-05-09T05:30:39.517 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:30:39.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:39 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:30:39.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:39 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:30:39.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:39 smithi115 ceph-mon[30349]: osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] boot 2024-05-09T05:30:39.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:39 smithi115 ceph-mon[30349]: osdmap e42: 7 total, 7 up, 7 in 2024-05-09T05:30:39.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:39 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:39.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:39 smithi115 ceph-mon[30349]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:39.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:39 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:30:39.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:39 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:30:39.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:39 smithi161 ceph-mon[33805]: osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] boot 2024-05-09T05:30:39.780 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:39 smithi161 ceph-mon[33805]: osdmap e42: 7 total, 7 up, 7 in 2024-05-09T05:30:39.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:39 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:39.781 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:39 smithi161 ceph-mon[33805]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:39.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:39 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:30:39.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:39 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:30:39.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:39 smithi195 ceph-mon[33844]: osd.6 [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] boot 2024-05-09T05:30:39.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:39 smithi195 ceph-mon[33844]: osdmap e42: 7 total, 7 up, 7 in 2024-05-09T05:30:39.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:39 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:30:39.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:39 smithi195 ceph-mon[33844]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-05-09T05:30:40.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T05:30:40.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi115 2024-05-09T05:30:40.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:40.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi115 to 5696M 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:40.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T05:30:40.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi115 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi115 to 5696M 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:40.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T05:30:40.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi115 2024-05-09T05:30:40.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:40.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:40.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:30:40.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi115 to 5696M 2024-05-09T05:30:40.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:40.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:40.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:40.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:40.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:40.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:30:40.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:41.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.224+0000 7f5774521700 1 -- 172.21.15.161:0/1229334128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 msgr2=0x7f576c104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:41.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.224+0000 7f5774521700 1 --2- 172.21.15.161:0/1229334128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 0x7f576c104b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f575c0099c0 tx=0x7f575c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:30:41.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.224+0000 7f5774521700 1 -- 172.21.15.161:0/1229334128 shutdown_connections 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.224+0000 7f5774521700 1 --2- 172.21.15.161:0/1229334128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f576c105080 0x7f576c10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.224+0000 7f5774521700 1 --2- 172.21.15.161:0/1229334128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f576c103830 0x7f576c103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.224+0000 7f5774521700 1 --2- 172.21.15.161:0/1229334128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 0x7f576c104b40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.224+0000 7f5774521700 1 -- 172.21.15.161:0/1229334128 >> 172.21.15.161:0/1229334128 conn(0x7f576c0fe020 msgr2=0x7f576c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.224+0000 7f5774521700 1 -- 172.21.15.161:0/1229334128 shutdown_connections 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.224+0000 7f5774521700 1 -- 172.21.15.161:0/1229334128 wait complete. 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5774521700 1 Processor -- start 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5774521700 1 -- start start 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5774521700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f576c103830 0x7f576c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5774521700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 0x7f576c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5774521700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f576c105080 0x7f576c1a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5774521700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f576c10edb0 con 0x7f576c103830 2024-05-09T05:30:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5774521700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f576c10ec30 con 0x7f576c104730 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5774521700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f576c10ef30 con 0x7f576c105080 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5771abc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 0x7f576c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5771abc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 0x7f576c199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41440/0 (socket says 172.21.15.161:41440) 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5771abc700 1 -- 172.21.15.161:0/632038175 learned_addr learned my addr 172.21.15.161:0/632038175 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f5772abe700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f576c105080 0x7f576c1a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.225+0000 7f57722bd700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f576c103830 0x7f576c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f5771abc700 1 -- 172.21.15.161:0/632038175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f576c105080 msgr2=0x7f576c1a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f5771abc700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f576c105080 0x7f576c1a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f5771abc700 1 -- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f576c103830 msgr2=0x7f576c199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f5771abc700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f576c103830 0x7f576c199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f5771abc700 1 -- 172.21.15.161:0/632038175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c1a14e0 con 0x7f576c104730 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f57722bd700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f576c103830 0x7f576c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:30:41.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f5771abc700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 0x7f576c199ec0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f575c000c00 tx=0x7f575c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:30:41.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f57637fe700 1 -- 172.21.15.161:0/632038175 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c027070 con 0x7f576c104730 2024-05-09T05:30:41.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f57637fe700 1 -- 172.21.15.161:0/632038175 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f575c00bd10 con 0x7f576c104730 2024-05-09T05:30:41.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c1a1770 con 0x7f576c104730 2024-05-09T05:30:41.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f57637fe700 1 -- 172.21.15.161:0/632038175 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c022de0 con 0x7f576c104730 2024-05-09T05:30:41.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.226+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c1a1c20 con 0x7f576c104730 2024-05-09T05:30:41.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.227+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576c0512c0 con 0x7f576c104730 2024-05-09T05:30:41.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.228+0000 7f57637fe700 1 -- 172.21.15.161:0/632038175 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f575c02a070 con 0x7f576c104730 2024-05-09T05:30:41.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.229+0000 7f57637fe700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f57580b2900 0x7f57580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:30:41.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.229+0000 7f57637fe700 1 -- 172.21.15.161:0/632038175 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f575c0e24c0 con 0x7f576c104730 2024-05-09T05:30:41.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.229+0000 7f57722bd700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f57580b2900 0x7f57580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:30:41.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.230+0000 7f57722bd700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f57580b2900 0x7f57580b4e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5768005fd0 tx=0x7f5768005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:30:41.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.233+0000 7f57637fe700 1 -- 172.21.15.161:0/632038175 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f575c05fdf0 con 0x7f576c104730 2024-05-09T05:30:41.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:30:41.441+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f576c02e7f0 con 0x7f57580b2900 2024-05-09T05:30:41.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:41 smithi195 ceph-mon[33844]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:41.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:30:41.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:30:41.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:41.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:41 smithi115 ceph-mon[30349]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:41.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:30:41.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:30:41.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:41.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:41 smithi161 ceph-mon[33805]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:41.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:30:41.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:30:41.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:42.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:42 smithi161 ceph-mon[33805]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:30:42.766 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:42 smithi195 ceph-mon[33844]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:30:42.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:42 smithi115 ceph-mon[30349]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:30:43.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:43 smithi161 ceph-mon[33805]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:43.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:43 smithi195 ceph-mon[33844]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:43.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:43 smithi115 ceph-mon[30349]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:44 smithi161 ceph-mon[33805]: from='client.? 172.21.15.161:0/1011985131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c95dabd-b86f-4ca6-be1c-a783b8716848"}]: dispatch 2024-05-09T05:30:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:44 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c95dabd-b86f-4ca6-be1c-a783b8716848"}]: dispatch 2024-05-09T05:30:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:44 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c95dabd-b86f-4ca6-be1c-a783b8716848"}]': finished 2024-05-09T05:30:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:44 smithi161 ceph-mon[33805]: osdmap e44: 8 total, 7 up, 8 in 2024-05-09T05:30:44.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:44 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:30:44.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:44 smithi195 ceph-mon[33844]: from='client.? 172.21.15.161:0/1011985131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c95dabd-b86f-4ca6-be1c-a783b8716848"}]: dispatch 2024-05-09T05:30:44.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:44 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c95dabd-b86f-4ca6-be1c-a783b8716848"}]: dispatch 2024-05-09T05:30:44.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:44 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c95dabd-b86f-4ca6-be1c-a783b8716848"}]': finished 2024-05-09T05:30:44.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:44 smithi195 ceph-mon[33844]: osdmap e44: 8 total, 7 up, 8 in 2024-05-09T05:30:44.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:44 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:30:44.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:44 smithi115 ceph-mon[30349]: from='client.? 172.21.15.161:0/1011985131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c95dabd-b86f-4ca6-be1c-a783b8716848"}]: dispatch 2024-05-09T05:30:44.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:44 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c95dabd-b86f-4ca6-be1c-a783b8716848"}]: dispatch 2024-05-09T05:30:44.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:44 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c95dabd-b86f-4ca6-be1c-a783b8716848"}]': finished 2024-05-09T05:30:44.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:44 smithi115 ceph-mon[30349]: osdmap e44: 8 total, 7 up, 8 in 2024-05-09T05:30:44.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:30:45.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:45 smithi195 ceph-mon[33844]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:45.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:45 smithi195 ceph-mon[33844]: from='client.? 172.21.15.161:0/953703395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T05:30:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:45 smithi115 ceph-mon[30349]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:45.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:45 smithi115 ceph-mon[30349]: from='client.? 172.21.15.161:0/953703395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T05:30:45.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:45 smithi161 ceph-mon[33805]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:45.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:45 smithi161 ceph-mon[33805]: from='client.? 172.21.15.161:0/953703395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T05:30:47.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:47 smithi115 ceph-mon[30349]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:47.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:47 smithi161 ceph-mon[33805]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:48.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:47 smithi195 ceph-mon[33844]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:49.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:49 smithi161 ceph-mon[33805]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:50.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:49 smithi195 ceph-mon[33844]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:50.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:49 smithi115 ceph-mon[30349]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:51.775 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:51 smithi161 ceph-mon[33805]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:52.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:51 smithi195 ceph-mon[33844]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:52.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:51 smithi115 ceph-mon[30349]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:52.720 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T05:30:52.720 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:52.720 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:52 smithi161 ceph-mon[33805]: Deploying daemon osd.7 on smithi161 2024-05-09T05:30:53.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T05:30:53.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:53.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:52 smithi195 ceph-mon[33844]: Deploying daemon osd.7 on smithi161 2024-05-09T05:30:53.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T05:30:53.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:30:53.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:52 smithi115 ceph-mon[30349]: Deploying daemon osd.7 on smithi161 2024-05-09T05:30:53.766 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:53 smithi161 ceph-mon[33805]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:53.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:53 smithi195 ceph-mon[33844]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:54.048 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:53 smithi115 ceph-mon[30349]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:55.935 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:55 smithi161 ceph-mon[33805]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:55 smithi195 ceph-mon[33844]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:56.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:55 smithi115 ceph-mon[30349]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:57.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:57 smithi161 ceph-mon[33805]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:58.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:57 smithi195 ceph-mon[33844]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:58.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:57 smithi115 ceph-mon[30349]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:30:58.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:58.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:58.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:59.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:59.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:30:59.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:59.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:30:59.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:00.077 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:30:59 smithi161 ceph-mon[33805]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:31:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:30:59 smithi195 ceph-mon[33844]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:31:00.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:30:59 smithi115 ceph-mon[30349]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:31:02.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.015+0000 7f57637fe700 1 -- 172.21.15.161:0/632038175 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f576c02e7f0 con 0x7f57580b2900 2024-05-09T05:31:02.010 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 7 on host 'smithi161' 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f57580b2900 msgr2=0x7f57580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f57580b2900 0x7f57580b4e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5768005fd0 tx=0x7f5768005f60 comp rx=0 tx=0).stop 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 msgr2=0x7f576c199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 0x7f576c199ec0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f575c000c00 tx=0x7f575c00f690 comp rx=0 tx=0).stop 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 shutdown_connections 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f57580b2900 0x7f57580b4e10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f576c105080 0x7f576c1a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f576c103830 0x7f576c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 --2- 172.21.15.161:0/632038175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f576c104730 0x7f576c199ec0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 >> 172.21.15.161:0/632038175 conn(0x7f576c0fe020 msgr2=0x7f576c107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:31:02.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 shutdown_connections 2024-05-09T05:31:02.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:31:02.018+0000 7f5774521700 1 -- 172.21.15.161:0/632038175 wait complete. 2024-05-09T05:31:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:01 smithi195 ceph-mon[33844]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:31:02.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:02.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:02.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:02.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:02.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:02.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:01 smithi195 ceph-mon[33844]: from='osd.7 [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T05:31:02.118 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:01 smithi195 ceph-mon[33844]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T05:31:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:01 smithi115 ceph-mon[30349]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:31:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:01 smithi115 ceph-mon[30349]: from='osd.7 [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T05:31:02.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:01 smithi115 ceph-mon[30349]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T05:31:02.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:01 smithi161 ceph-mon[33805]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:31:02.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:02.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:02.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:02.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:02.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:02.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:01 smithi161 ceph-mon[33805]: from='osd.7 [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T05:31:02.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:01 smithi161 ceph-mon[33805]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T05:31:02.833 DEBUG:teuthology.orchestra.run.smithi161:osd.7> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.7.service 2024-05-09T05:31:02.835 INFO:tasks.cephadm:Deploying osd.8 on smithi195 with /dev/vg_nvme/lv_2... 2024-05-09T05:31:02.836 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T05:31:03.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:02 smithi161 ceph-mon[33805]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T05:31:03.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:02 smithi161 ceph-mon[33805]: from='osd.7 [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:31:03.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:02 smithi161 ceph-mon[33805]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T05:31:03.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:02 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:03.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:02 smithi161 ceph-mon[33805]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:31:03.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:02 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:03.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:02 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:03.074 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:02 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:03.074 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 05:31:02 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7[51684]: 2024-05-09T05:31:02.899+0000 7f496a07d700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T05:31:03.075 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 05:31:02 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7[51684]: 2024-05-09T05:31:02.910+0000 7f4965e32700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:31:03.091 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:31:03.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:02 smithi195 ceph-mon[33844]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T05:31:03.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:02 smithi195 ceph-mon[33844]: from='osd.7 [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:31:03.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:02 smithi195 ceph-mon[33844]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T05:31:03.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:02 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:03.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:02 smithi195 ceph-mon[33844]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:31:03.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:02 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:03.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:02 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:03.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:02 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:03.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:02 smithi115 ceph-mon[30349]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T05:31:03.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:02 smithi115 ceph-mon[30349]: from='osd.7 [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:31:03.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:02 smithi115 ceph-mon[30349]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T05:31:03.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:02 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:03.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:02 smithi115 ceph-mon[30349]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:31:03.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:02 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:03.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:02 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:03.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:02 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:04.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:03 smithi161 ceph-mon[33805]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:31:04.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:03 smithi161 ceph-mon[33805]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:31:04.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:03 smithi161 ceph-mon[33805]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T05:31:04.080 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:03 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:04.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:03 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:04.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:03 smithi161 ceph-mon[33805]: osd.7 [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] boot 2024-05-09T05:31:04.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:03 smithi161 ceph-mon[33805]: osdmap e47: 8 total, 8 up, 8 in 2024-05-09T05:31:04.081 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:03 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:03 smithi115 ceph-mon[30349]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:31:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:03 smithi115 ceph-mon[30349]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:31:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:03 smithi115 ceph-mon[30349]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T05:31:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:03 smithi115 ceph-mon[30349]: osd.7 [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] boot 2024-05-09T05:31:04.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:03 smithi115 ceph-mon[30349]: osdmap e47: 8 total, 8 up, 8 in 2024-05-09T05:31:04.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:03 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:04.204 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:03 smithi195 ceph-mon[33844]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-05-09T05:31:04.204 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:03 smithi195 ceph-mon[33844]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:31:04.204 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:03 smithi195 ceph-mon[33844]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T05:31:04.204 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:03 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:04.204 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:03 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:04.204 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:03 smithi195 ceph-mon[33844]: osd.7 [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] boot 2024-05-09T05:31:04.204 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:03 smithi195 ceph-mon[33844]: osdmap e47: 8 total, 8 up, 8 in 2024-05-09T05:31:04.204 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:03 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:31:05.063 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T05:31:05.064 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T05:31:05.064 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-05-09T05:31:05.064 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-05-09T05:31:05.064 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00880032 s, 1.2 GB/s 2024-05-09T05:31:05.064 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-05-09T05:31:05.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:05 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:31:05.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:05 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:31:05.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:05 smithi195 ceph-mon[33844]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T05:31:05.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:05 smithi195 ceph-mon[33844]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:05 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:31:05.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:05 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:31:05.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:05 smithi115 ceph-mon[30349]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T05:31:05.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:05 smithi115 ceph-mon[30349]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:05.782 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_2 2024-05-09T05:31:05.871 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:05 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:31:05.871 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:05 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:31:05.871 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:05 smithi161 ceph-mon[33805]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T05:31:05.871 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:05 smithi161 ceph-mon[33805]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:06.034 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:31:06.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T05:31:06.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi161 2024-05-09T05:31:06.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:06.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:06.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:06.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi161 to 5696M 2024-05-09T05:31:06.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:06.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:06.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:06 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T05:31:06.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi161 2024-05-09T05:31:06.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:06.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:06.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:06.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi161 to 5696M 2024-05-09T05:31:06.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:06.898 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:06 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi161 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi161 to 5696M 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:06.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:06 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.795+0000 7f3e9b084700 1 -- 172.21.15.195:0/2792150846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941049e0 msgr2=0x7f3e9410c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.795+0000 7f3e9b084700 1 --2- 172.21.15.195:0/2792150846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941049e0 0x7f3e9410c6f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00b0a0 tx=0x7f3e8c00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.795+0000 7f3e9b084700 1 -- 172.21.15.195:0/2792150846 shutdown_connections 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.795+0000 7f3e9b084700 1 --2- 172.21.15.195:0/2792150846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e941059b0 0x7f3e94105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.795+0000 7f3e9b084700 1 --2- 172.21.15.195:0/2792150846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941049e0 0x7f3e9410c6f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.795+0000 7f3e9b084700 1 --2- 172.21.15.195:0/2792150846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e94103f60 0x7f3e94104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.795+0000 7f3e9b084700 1 -- 172.21.15.195:0/2792150846 >> 172.21.15.195:0/2792150846 conn(0x7f3e940ff9d0 msgr2=0x7f3e94101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.795+0000 7f3e9b084700 1 -- 172.21.15.195:0/2792150846 shutdown_connections 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.795+0000 7f3e9b084700 1 -- 172.21.15.195:0/2792150846 wait complete. 2024-05-09T05:31:07.792 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e9b084700 1 Processor -- start 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e9b084700 1 -- start start 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e9b084700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e94103f60 0x7f3e94078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e9b084700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e941049e0 0x7f3e94076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e9b084700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941059b0 0x7f3e94077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e9b084700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410fe70 con 0x7f3e941049e0 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e9b084700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410fcf0 con 0x7f3e94103f60 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e9b084700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410fff0 con 0x7f3e941059b0 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e99621700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941059b0 0x7f3e94077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e99621700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941059b0 0x7f3e94077160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:44140/0 (socket says 172.21.15.195:44140) 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e99621700 1 -- 172.21.15.195:0/3262627779 learned_addr learned my addr 172.21.15.195:0/3262627779 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e98e20700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e94103f60 0x7f3e94078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.796+0000 7f3e93fff700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e941049e0 0x7f3e94076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:31:07.793 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e99621700 1 -- 172.21.15.195:0/3262627779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e94103f60 msgr2=0x7f3e94078570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e99621700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e94103f60 0x7f3e94078570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e99621700 1 -- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e941049e0 msgr2=0x7f3e94076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e99621700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e941049e0 0x7f3e94076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e99621700 1 -- 172.21.15.195:0/3262627779 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e940778c0 con 0x7f3e941059b0 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e93fff700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e941049e0 0x7f3e94076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e99621700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941059b0 0x7f3e94077160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00b070 tx=0x7f3e8c0097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e91ffb700 1 -- 172.21.15.195:0/3262627779 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c01f030 con 0x7f3e941059b0 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e941a4a50 con 0x7f3e941059b0 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e91ffb700 1 -- 172.21.15.195:0/3262627779 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e8c007ac0 con 0x7f3e941059b0 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e91ffb700 1 -- 172.21.15.195:0/3262627779 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c004760 con 0x7f3e941059b0 2024-05-09T05:31:07.794 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.797+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e941a4ec0 con 0x7f3e941059b0 2024-05-09T05:31:07.795 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.798+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e94110630 con 0x7f3e941059b0 2024-05-09T05:31:07.795 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.800+0000 7f3e91ffb700 1 -- 172.21.15.195:0/3262627779 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3e8c003a00 con 0x7f3e941059b0 2024-05-09T05:31:07.795 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.800+0000 7f3e91ffb700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e840b2880 0x7f3e840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:31:07.795 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.800+0000 7f3e91ffb700 1 -- 172.21.15.195:0/3262627779 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f3e8c00f020 con 0x7f3e941059b0 2024-05-09T05:31:07.795 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.801+0000 7f3e98e20700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e840b2880 0x7f3e840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:31:07.796 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.801+0000 7f3e98e20700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e840b2880 0x7f3e840b4d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3e941053a0 tx=0x7f3e88009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:31:07.798 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:07.804+0000 7f3e91ffb700 1 -- 172.21.15.195:0/3262627779 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e8c029020 con 0x7f3e941059b0 2024-05-09T05:31:07.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:07 smithi195 ceph-mon[33844]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:07.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:07 smithi115 ceph-mon[30349]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:07.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:07 smithi161 ceph-mon[33805]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:08.011 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:08.017+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3e9402e9d0 con 0x7f3e840b2880 2024-05-09T05:31:08.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:08 smithi195 ceph-mon[33844]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:31:08.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:08 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:31:08.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:08 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:31:08.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:08 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:08.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:08 smithi115 ceph-mon[30349]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:31:08.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:08 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:31:08.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:08 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:31:08.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:08 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:08.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:08 smithi161 ceph-mon[33805]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:31:08.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:08 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:31:08.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:08 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:31:08.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:08 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:09.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:09 smithi195 ceph-mon[33844]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:09.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:09 smithi115 ceph-mon[30349]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:09.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:09 smithi161 ceph-mon[33805]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:11.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:11 smithi161 ceph-mon[33805]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:31:11.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:11 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/3545954467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92757b9e-da23-40c2-920c-2ed2a247b772"}]: dispatch 2024-05-09T05:31:11.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:11 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92757b9e-da23-40c2-920c-2ed2a247b772"}]: dispatch 2024-05-09T05:31:11.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:11 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92757b9e-da23-40c2-920c-2ed2a247b772"}]': finished 2024-05-09T05:31:11.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:11 smithi161 ceph-mon[33805]: osdmap e50: 9 total, 8 up, 9 in 2024-05-09T05:31:11.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:11.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:11 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/1259755038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T05:31:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:11 smithi195 ceph-mon[33844]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:31:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:11 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/3545954467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92757b9e-da23-40c2-920c-2ed2a247b772"}]: dispatch 2024-05-09T05:31:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:11 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92757b9e-da23-40c2-920c-2ed2a247b772"}]: dispatch 2024-05-09T05:31:12.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:11 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92757b9e-da23-40c2-920c-2ed2a247b772"}]': finished 2024-05-09T05:31:12.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:11 smithi195 ceph-mon[33844]: osdmap e50: 9 total, 8 up, 9 in 2024-05-09T05:31:12.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:12.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:11 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/1259755038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T05:31:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:11 smithi115 ceph-mon[30349]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:31:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:11 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/3545954467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92757b9e-da23-40c2-920c-2ed2a247b772"}]: dispatch 2024-05-09T05:31:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:11 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92757b9e-da23-40c2-920c-2ed2a247b772"}]: dispatch 2024-05-09T05:31:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:11 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92757b9e-da23-40c2-920c-2ed2a247b772"}]': finished 2024-05-09T05:31:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:11 smithi115 ceph-mon[30349]: osdmap e50: 9 total, 8 up, 9 in 2024-05-09T05:31:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:11 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/1259755038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T05:31:13.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:13 smithi161 ceph-mon[33805]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T05:31:14.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:13 smithi195 ceph-mon[33844]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T05:31:14.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:13 smithi115 ceph-mon[30349]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T05:31:15.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:15 smithi161 ceph-mon[33805]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-05-09T05:31:15.924 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:15 smithi195 ceph-mon[33844]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-05-09T05:31:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:15 smithi115 ceph-mon[30349]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-05-09T05:31:17.937 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:17 smithi195 ceph-mon[33844]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T05:31:18.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:17 smithi115 ceph-mon[30349]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T05:31:18.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:17 smithi161 ceph-mon[33805]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T05:31:18.950 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T05:31:18.950 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:18.950 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:18 smithi195 ceph-mon[33844]: Deploying daemon osd.8 on smithi195 2024-05-09T05:31:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T05:31:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:18 smithi115 ceph-mon[30349]: Deploying daemon osd.8 on smithi195 2024-05-09T05:31:19.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T05:31:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:18 smithi161 ceph-mon[33805]: Deploying daemon osd.8 on smithi195 2024-05-09T05:31:19.973 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:19 smithi195 ceph-mon[33844]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T05:31:20.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:19 smithi115 ceph-mon[30349]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T05:31:20.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:19 smithi161 ceph-mon[33805]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T05:31:22.024 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:21 smithi195 ceph-mon[33844]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:21 smithi115 ceph-mon[30349]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:22.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:21 smithi161 ceph-mon[33805]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:23.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:23 smithi195 ceph-mon[33844]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:24.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:23 smithi115 ceph-mon[30349]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:24.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:23 smithi161 ceph-mon[33805]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:25 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:25 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:25 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:25.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:25 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:25.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:25 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:25.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:25 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:25 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:26.525 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:26 smithi195 ceph-mon[33844]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:26.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:26 smithi115 ceph-mon[30349]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:26.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:26 smithi161 ceph-mon[33805]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:28.278 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.284+0000 7f3e91ffb700 1 -- 172.21.15.195:0/3262627779 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3e9402e9d0 con 0x7f3e840b2880 2024-05-09T05:31:28.278 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 8 on host 'smithi195' 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e840b2880 msgr2=0x7f3e840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e840b2880 0x7f3e840b4d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3e941053a0 tx=0x7f3e88009190 comp rx=0 tx=0).stop 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941059b0 msgr2=0x7f3e94077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941059b0 0x7f3e94077160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00b070 tx=0x7f3e8c0097f0 comp rx=0 tx=0).stop 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 shutdown_connections 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e840b2880 0x7f3e840b4d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e94103f60 0x7f3e94078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e941059b0 0x7f3e94077160 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 --2- 172.21.15.195:0/3262627779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e941049e0 0x7f3e94076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 >> 172.21.15.195:0/3262627779 conn(0x7f3e940ff9d0 msgr2=0x7f3e941086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.286+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 shutdown_connections 2024-05-09T05:31:28.281 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:31:28.287+0000 7f3e9b084700 1 -- 172.21.15.195:0/3262627779 wait complete. 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='osd.8 [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:28.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:28 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:28.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:28.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:28.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='osd.8 [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T05:31:28.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T05:31:28.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:28.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:28 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='osd.8 [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:28.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:28 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:28.956 DEBUG:teuthology.orchestra.run.smithi195:osd.8> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.8.service 2024-05-09T05:31:28.959 INFO:tasks.cephadm:Deploying osd.9 on smithi115 with /dev/vg_nvme/lv_1... 2024-05-09T05:31:28.959 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T05:31:29.209 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:31:29.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:29 smithi195 ceph-mon[33844]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T05:31:29.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:29 smithi195 ceph-mon[33844]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T05:31:29.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:29 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:29.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:29 smithi195 ceph-mon[33844]: from='osd.8 [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:31:29.458 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:29 smithi195 ceph-mon[33844]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:31:29.459 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 05:31:29 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8[52188]: 2024-05-09T05:31:29.303+0000 7f841e96b700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T05:31:29.459 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 05:31:29 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8[52188]: 2024-05-09T05:31:29.315+0000 7f841a720700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:31:29.552 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:29 smithi115 ceph-mon[30349]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T05:31:29.552 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:29 smithi115 ceph-mon[30349]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T05:31:29.552 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:29 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:29.552 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:29 smithi115 ceph-mon[30349]: from='osd.8 [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:31:29.552 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:29 smithi115 ceph-mon[30349]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:31:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:29 smithi161 ceph-mon[33805]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T05:31:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:29 smithi161 ceph-mon[33805]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T05:31:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:29 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:29 smithi161 ceph-mon[33805]: from='osd.8 [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:31:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:29 smithi161 ceph-mon[33805]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:31:30.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:30 smithi195 ceph-mon[33844]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:30.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:30 smithi195 ceph-mon[33844]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:31:30.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:30 smithi195 ceph-mon[33844]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T05:31:30.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:30.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:30.594 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:30 smithi115 ceph-mon[30349]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:30.594 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:30 smithi115 ceph-mon[30349]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:31:30.595 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:30 smithi115 ceph-mon[30349]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T05:31:30.595 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:30.595 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:30 smithi161 ceph-mon[33805]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:30 smithi161 ceph-mon[33805]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:31:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:30 smithi161 ceph-mon[33805]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T05:31:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:31.284 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T05:31:31.284 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T05:31:31.284 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-05-09T05:31:31.284 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-05-09T05:31:31.284 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013352 s, 785 MB/s 2024-05-09T05:31:31.284 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-05-09T05:31:31.590 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:31 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:31:31.590 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:31 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:31:31.590 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:31 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:31.590 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:31 smithi115 ceph-mon[30349]: osd.8 [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] boot 2024-05-09T05:31:31.590 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:31 smithi115 ceph-mon[30349]: osdmap e53: 9 total, 9 up, 9 in 2024-05-09T05:31:31.590 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:31 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:31.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:31 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:31:31.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:31 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:31:31.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:31 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:31.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:31 smithi195 ceph-mon[33844]: osd.8 [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] boot 2024-05-09T05:31:31.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:31 smithi195 ceph-mon[33844]: osdmap e53: 9 total, 9 up, 9 in 2024-05-09T05:31:31.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:31 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:31.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:31 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:31:31.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:31 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:31:31.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:31 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:31.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:31 smithi161 ceph-mon[33805]: osd.8 [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] boot 2024-05-09T05:31:31.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:31 smithi161 ceph-mon[33805]: osdmap e53: 9 total, 9 up, 9 in 2024-05-09T05:31:31.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:31 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:31:31.845 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2024-05-09T05:31:32.107 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:31:32.570 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:32.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:32.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T05:31:32.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:32.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:32.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:33.578 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:33 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi195 2024-05-09T05:31:33.578 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:33 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi195 to 5696M 2024-05-09T05:31:33.578 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:33 smithi115 ceph-mon[30349]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T05:31:33.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:33 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi195 2024-05-09T05:31:33.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:33 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi195 to 5696M 2024-05-09T05:31:33.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:33 smithi195 ceph-mon[33844]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T05:31:33.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:33 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi195 2024-05-09T05:31:33.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:33 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi195 to 5696M 2024-05-09T05:31:33.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:33 smithi161 ceph-mon[33805]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T05:31:33.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.809+0000 7f3a4cfff700 1 -- 172.21.15.115:0/2107056907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 msgr2=0x7f3a48103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:33.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.809+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/2107056907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 0x7f3a48103660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009990 tx=0x7f3a38009ca0 comp rx=0 tx=0).stop 2024-05-09T05:31:33.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.809+0000 7f3a4cfff700 1 -- 172.21.15.115:0/2107056907 shutdown_connections 2024-05-09T05:31:33.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.809+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/2107056907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a48104d20 0x7f3a4810b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:33.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.809+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/2107056907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 0x7f3a48103660 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:33.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.809+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/2107056907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a48104380 0x7f3a48104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:33.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.809+0000 7f3a4cfff700 1 -- 172.21.15.115:0/2107056907 >> 172.21.15.115:0/2107056907 conn(0x7f3a480fe890 msgr2=0x7f3a48100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:31:33.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.809+0000 7f3a4cfff700 1 -- 172.21.15.115:0/2107056907 shutdown_connections 2024-05-09T05:31:33.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.809+0000 7f3a4cfff700 1 -- 172.21.15.115:0/2107056907 wait complete. 2024-05-09T05:31:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.810+0000 7f3a4cfff700 1 Processor -- start 2024-05-09T05:31:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.810+0000 7f3a4cfff700 1 -- start start 2024-05-09T05:31:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4cfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 0x7f3a481997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:31:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4cfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a48104380 0x7f3a48199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:31:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4cfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a48104d20 0x7f3a481a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:31:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4cfff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810eba0 con 0x7f3a48103250 2024-05-09T05:31:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4cfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810ea20 con 0x7f3a48104d20 2024-05-09T05:31:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4cfff700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810ed20 con 0x7f3a48104380 2024-05-09T05:31:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 0x7f3a481997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 0x7f3a481997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57416/0 (socket says 172.21.15.115:57416) 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4659c700 1 -- 172.21.15.115:0/3497956043 learned_addr learned my addr 172.21.15.115:0/3497956043 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a46d9d700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a48104d20 0x7f3a481a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4659c700 1 -- 172.21.15.115:0/3497956043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a48104380 msgr2=0x7f3a48199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a45d9b700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a48104380 0x7f3a48199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4659c700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a48104380 0x7f3a48199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4659c700 1 -- 172.21.15.115:0/3497956043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a48104d20 msgr2=0x7f3a481a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4659c700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a48104d20 0x7f3a481a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4659c700 1 -- 172.21.15.115:0/3497956043 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a481a1310 con 0x7f3a48103250 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a46d9d700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a48104d20 0x7f3a481a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a4659c700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 0x7f3a481997b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a38003820 tx=0x7f3a380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.811+0000 7f3a3f7fe700 1 -- 172.21.15.115:0/3497956043 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38003c10 con 0x7f3a48103250 2024-05-09T05:31:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.812+0000 7f3a3f7fe700 1 -- 172.21.15.115:0/3497956043 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a38004830 con 0x7f3a48103250 2024-05-09T05:31:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.812+0000 7f3a3f7fe700 1 -- 172.21.15.115:0/3497956043 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38005260 con 0x7f3a48103250 2024-05-09T05:31:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.812+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a481a15a0 con 0x7f3a48103250 2024-05-09T05:31:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.812+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a481a1990 con 0x7f3a48103250 2024-05-09T05:31:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.813+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a480512c0 con 0x7f3a48103250 2024-05-09T05:31:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.814+0000 7f3a3f7fe700 1 -- 172.21.15.115:0/3497956043 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3a380054c0 con 0x7f3a48103250 2024-05-09T05:31:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.815+0000 7f3a3f7fe700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3a340b2880 0x7f3a340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:31:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.815+0000 7f3a45d9b700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3a340b2880 0x7f3a340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:31:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.815+0000 7f3a3f7fe700 1 -- 172.21.15.115:0/3497956043 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f3a380e24e0 con 0x7f3a48103250 2024-05-09T05:31:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.815+0000 7f3a45d9b700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3a340b2880 0x7f3a340b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a30005950 tx=0x7f3a300058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:31:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:33.820+0000 7f3a3f7fe700 1 -- 172.21.15.115:0/3497956043 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a38013070 con 0x7f3a48103250 2024-05-09T05:31:34.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:34.033+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3a48063cc0 con 0x7f3a340b2880 2024-05-09T05:31:34.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:34 smithi195 ceph-mon[33844]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:34.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:31:34.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:31:34.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:34 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:34 smithi115 ceph-mon[30349]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:31:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:31:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:34 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:34.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:34 smithi161 ceph-mon[33805]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:34.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:31:34.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:31:34.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:34 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:35.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:35 smithi195 ceph-mon[33844]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:31:35.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:35 smithi115 ceph-mon[30349]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:31:35.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:35 smithi161 ceph-mon[33805]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:31:36.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:36 smithi195 ceph-mon[33844]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-09T05:31:36.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:36 smithi115 ceph-mon[30349]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-09T05:31:36.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:36 smithi161 ceph-mon[33805]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-09T05:31:37.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:37 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1264636596' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d893599-0268-4ed6-9ea7-b74b6a623b3d"}]: dispatch 2024-05-09T05:31:37.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:37 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1264636596' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d893599-0268-4ed6-9ea7-b74b6a623b3d"}]': finished 2024-05-09T05:31:37.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:37 smithi195 ceph-mon[33844]: osdmap e56: 10 total, 9 up, 10 in 2024-05-09T05:31:37.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:37 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:37.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:37 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1264636596' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d893599-0268-4ed6-9ea7-b74b6a623b3d"}]: dispatch 2024-05-09T05:31:37.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:37 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1264636596' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d893599-0268-4ed6-9ea7-b74b6a623b3d"}]': finished 2024-05-09T05:31:37.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:37 smithi115 ceph-mon[30349]: osdmap e56: 10 total, 9 up, 10 in 2024-05-09T05:31:37.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:37 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:37.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:37 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1264636596' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d893599-0268-4ed6-9ea7-b74b6a623b3d"}]: dispatch 2024-05-09T05:31:37.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:37 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1264636596' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d893599-0268-4ed6-9ea7-b74b6a623b3d"}]': finished 2024-05-09T05:31:37.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:37 smithi161 ceph-mon[33805]: osdmap e56: 10 total, 9 up, 10 in 2024-05-09T05:31:37.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:37 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:38.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:38 smithi195 ceph-mon[33844]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-09T05:31:38.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:38 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1342035777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T05:31:38.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:38 smithi115 ceph-mon[30349]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-09T05:31:38.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:38 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1342035777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T05:31:38.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:38 smithi161 ceph-mon[33805]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-09T05:31:38.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:38 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1342035777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T05:31:40.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:40 smithi195 ceph-mon[33844]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-09T05:31:40.621 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:40 smithi115 ceph-mon[30349]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-09T05:31:40.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:40 smithi161 ceph-mon[33805]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-09T05:31:42.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:42 smithi195 ceph-mon[33844]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T05:31:42.632 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:42 smithi115 ceph-mon[30349]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T05:31:42.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:42 smithi161 ceph-mon[33805]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T05:31:44.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:44 smithi195 ceph-mon[33844]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:31:44.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:44 smithi115 ceph-mon[30349]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:31:44.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:44 smithi161 ceph-mon[33805]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:31:45.493 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T05:31:45.494 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T05:31:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:45.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T05:31:45.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:46.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:46 smithi115 ceph-mon[30349]: Deploying daemon osd.9 on smithi115 2024-05-09T05:31:46.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:46 smithi115 ceph-mon[30349]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:46.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:46 smithi195 ceph-mon[33844]: Deploying daemon osd.9 on smithi115 2024-05-09T05:31:46.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:46 smithi195 ceph-mon[33844]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:46.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:46 smithi161 ceph-mon[33805]: Deploying daemon osd.9 on smithi115 2024-05-09T05:31:46.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:46 smithi161 ceph-mon[33805]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:48.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:48 smithi195 ceph-mon[33844]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:48.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:48 smithi115 ceph-mon[30349]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:48.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:48 smithi161 ceph-mon[33805]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:50.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:50 smithi195 ceph-mon[33844]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:50.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:50 smithi115 ceph-mon[30349]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:50.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:50 smithi161 ceph-mon[33805]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:51.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:51 smithi115 ceph-mon[30349]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:51.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:51.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:51.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:51.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:51 smithi161 ceph-mon[33805]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:51.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:51.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:51.913 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:52.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:51 smithi195 ceph-mon[33844]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:52.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:52.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:52.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:53.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:53 smithi115 ceph-mon[30349]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:53.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:53 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:53.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:53 smithi161 ceph-mon[33805]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:53.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:53 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:53 smithi195 ceph-mon[33844]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:54.129 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:53 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.592+0000 7f3a3f7fe700 1 -- 172.21.15.115:0/3497956043 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a48063cc0 con 0x7f3a340b2880 2024-05-09T05:31:54.586 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 9 on host 'smithi115' 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3a340b2880 msgr2=0x7f3a340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3a340b2880 0x7f3a340b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a30005950 tx=0x7f3a300058e0 comp rx=0 tx=0).stop 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 msgr2=0x7f3a481997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 0x7f3a481997b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a38003820 tx=0x7f3a380047b0 comp rx=0 tx=0).stop 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 shutdown_connections 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3a340b2880 0x7f3a340b4d90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a48104d20 0x7f3a481a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a48103250 0x7f3a481997b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 --2- 172.21.15.115:0/3497956043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a48104380 0x7f3a48199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:31:54.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 >> 172.21.15.115:0/3497956043 conn(0x7f3a480fe890 msgr2=0x7f3a481075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:31:54.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 shutdown_connections 2024-05-09T05:31:54.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:31:54.594+0000 7f3a4cfff700 1 -- 172.21.15.115:0/3497956043 wait complete. 2024-05-09T05:31:54.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:54.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:54.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:54 smithi115 ceph-mon[30349]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T05:31:54.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:54 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:54.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:54 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:54 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:54.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:54 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:54.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:54 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:54 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:54 smithi161 ceph-mon[33805]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T05:31:54.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:54 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:54.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:54 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:54 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:54 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:31:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:54 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:31:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:54 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:54 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:54 smithi195 ceph-mon[33844]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T05:31:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:54 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:31:55.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:54 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:31:55.356 DEBUG:teuthology.orchestra.run.smithi115:osd.9> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.9.service 2024-05-09T05:31:55.358 INFO:tasks.cephadm:Deploying osd.10 on smithi161 with /dev/vg_nvme/lv_1... 2024-05-09T05:31:55.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T05:31:55.610 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:31:55.655 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:55 smithi115 ceph-mon[30349]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:55.656 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:55 smithi115 ceph-mon[30349]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T05:31:55.656 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:55 smithi115 ceph-mon[30349]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T05:31:55.656 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:55 smithi115 ceph-mon[30349]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:31:55.656 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:55.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:55 smithi161 ceph-mon[33805]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:55.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:55 smithi161 ceph-mon[33805]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T05:31:55.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:55 smithi161 ceph-mon[33805]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T05:31:55.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:55 smithi161 ceph-mon[33805]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:31:55.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:55 smithi195 ceph-mon[33844]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:55 smithi195 ceph-mon[33844]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T05:31:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:55 smithi195 ceph-mon[33844]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T05:31:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:55 smithi195 ceph-mon[33844]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-05-09T05:31:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:57.261 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 05:31:57 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T05:31:57.034+0000 7f3c86ea4700 -1 osd.9 0 waiting for initial osdmap 2024-05-09T05:31:57.262 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 05:31:57 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T05:31:57.046+0000 7f3c7ec51700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:31:57.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:57 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:31:57.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:57 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:31:57.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:57 smithi161 ceph-mon[33805]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:31:57.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:57 smithi161 ceph-mon[33805]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T05:31:57.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:57 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:57.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:57 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:57.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:57 smithi161 ceph-mon[33805]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:57.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:57 smithi161 ceph-mon[33805]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' 2024-05-09T05:31:57.721 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T05:31:57.721 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T05:31:57.721 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-09T05:31:57.721 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-09T05:31:57.721 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114943 s, 912 MB/s 2024-05-09T05:31:57.721 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-09T05:31:57.818 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:57 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:31:57.818 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:57 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:31:57.818 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:57 smithi115 ceph-mon[30349]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:31:57.818 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:57 smithi115 ceph-mon[30349]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T05:31:57.818 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:57 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:57.818 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:57 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:57.818 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:57 smithi115 ceph-mon[30349]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:57.818 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:57 smithi115 ceph-mon[30349]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' 2024-05-09T05:31:57.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:57 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:31:57.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:57 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:31:57.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:57 smithi195 ceph-mon[33844]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-05-09T05:31:57.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:57 smithi195 ceph-mon[33844]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T05:31:57.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:57 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:57.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:57 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:57.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:57 smithi195 ceph-mon[33844]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-05-09T05:31:57.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:57 smithi195 ceph-mon[33844]: from='osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430]' entity='osd.9' 2024-05-09T05:31:58.588 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-05-09T05:31:58.838 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:31:58.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:58.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:58 smithi195 ceph-mon[33844]: osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] boot 2024-05-09T05:31:58.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:58 smithi195 ceph-mon[33844]: osdmap e59: 10 total, 10 up, 10 in 2024-05-09T05:31:58.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:58 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:58.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:58.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:58 smithi161 ceph-mon[33805]: osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] boot 2024-05-09T05:31:58.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:58 smithi161 ceph-mon[33805]: osdmap e59: 10 total, 10 up, 10 in 2024-05-09T05:31:58.876 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:58 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:58.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:31:58.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:58 smithi115 ceph-mon[30349]: osd.9 [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] boot 2024-05-09T05:31:58.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:58 smithi115 ceph-mon[30349]: osdmap e59: 10 total, 10 up, 10 in 2024-05-09T05:31:58.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:58 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:32:00.005 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:00.005 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi115 2024-05-09T05:32:00.005 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.005 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.005 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.005 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.006 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.006 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.006 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi115 to 4272M 2024-05-09T05:32:00.006 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.006 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:00.006 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:00.006 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.006 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:31:59 smithi161 ceph-mon[33805]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T05:32:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi115 2024-05-09T05:32:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi115 to 4272M 2024-05-09T05:32:00.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:00.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:00.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:31:59 smithi195 ceph-mon[33844]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi115 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi115 to 4272M 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:00.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:00.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:31:59 smithi115 ceph-mon[30349]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T05:32:00.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.633+0000 7fab15a80700 1 -- 172.21.15.161:0/3452103706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104d30 msgr2=0x7fab1010b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:00.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.633+0000 7fab15a80700 1 --2- 172.21.15.161:0/3452103706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104d30 0x7fab1010b5e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fab0800b0a0 tx=0x7fab0800b3b0 comp rx=0 tx=0).stop 2024-05-09T05:32:00.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.633+0000 7fab15a80700 1 -- 172.21.15.161:0/3452103706 shutdown_connections 2024-05-09T05:32:00.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.633+0000 7fab15a80700 1 --2- 172.21.15.161:0/3452103706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104d30 0x7fab1010b5e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:00.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.633+0000 7fab15a80700 1 --2- 172.21.15.161:0/3452103706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab10103260 0x7fab10103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.633+0000 7fab15a80700 1 --2- 172.21.15.161:0/3452103706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab10104390 0x7fab101047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.633+0000 7fab15a80700 1 -- 172.21.15.161:0/3452103706 >> 172.21.15.161:0/3452103706 conn(0x7fab100fe8c0 msgr2=0x7fab10100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.633+0000 7fab15a80700 1 -- 172.21.15.161:0/3452103706 shutdown_connections 2024-05-09T05:32:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.633+0000 7fab15a80700 1 -- 172.21.15.161:0/3452103706 wait complete. 2024-05-09T05:32:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.634+0000 7fab15a80700 1 Processor -- start 2024-05-09T05:32:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.634+0000 7fab15a80700 1 -- start start 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab15a80700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab10103260 0x7fab100764b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab15a80700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104390 0x7fab1019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab15a80700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab10104d30 0x7fab101a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab15a80700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fab1010eb80 con 0x7fab10104d30 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab15a80700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab1010ea00 con 0x7fab10104390 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab15a80700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab1010ed00 con 0x7fab10103260 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104390 0x7fab1019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104390 0x7fab1019f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53864/0 (socket says 172.21.15.161:53864) 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0effd700 1 -- 172.21.15.161:0/4244631470 learned_addr learned my addr 172.21.15.161:0/4244631470 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-09T05:32:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0f7fe700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab10103260 0x7fab100764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0ffff700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab10104d30 0x7fab101a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0effd700 1 -- 172.21.15.161:0/4244631470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab10103260 msgr2=0x7fab100764b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0effd700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab10103260 0x7fab100764b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0effd700 1 -- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab10104d30 msgr2=0x7fab101a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0effd700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab10104d30 0x7fab101a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0effd700 1 -- 172.21.15.161:0/4244631470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab101a2290 con 0x7fab10104390 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0f7fe700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab10103260 0x7fab100764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0ffff700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab10104d30 0x7fab101a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0effd700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104390 0x7fab1019f520 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab0000c920 tx=0x7fab0000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.635+0000 7fab0cff9700 1 -- 172.21.15.161:0/4244631470 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00007b00 con 0x7fab10104390 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.636+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab101a2580 con 0x7fab10104390 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.636+0000 7fab0cff9700 1 -- 172.21.15.161:0/4244631470 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab00004cc0 con 0x7fab10104390 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.636+0000 7fab0cff9700 1 -- 172.21.15.161:0/4244631470 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab000054f0 con 0x7fab10104390 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.636+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab101a2ac0 con 0x7fab10104390 2024-05-09T05:32:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.637+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab100512c0 con 0x7fab10104390 2024-05-09T05:32:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.638+0000 7fab0cff9700 1 -- 172.21.15.161:0/4244631470 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fab00007ca0 con 0x7fab10104390 2024-05-09T05:32:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.639+0000 7fab0cff9700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faafc0b2880 0x7faafc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.639+0000 7fab0cff9700 1 -- 172.21.15.161:0/4244631470 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7fab000e1b00 con 0x7fab10104390 2024-05-09T05:32:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.639+0000 7fab0f7fe700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faafc0b2880 0x7faafc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:00.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.640+0000 7fab0f7fe700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faafc0b2880 0x7faafc0b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab10104290 tx=0x7faaf800a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.642+0000 7fab0cff9700 1 -- 172.21.15.161:0/4244631470 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab00014030 con 0x7fab10104390 2024-05-09T05:32:00.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:00.856+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fab10063c00 con 0x7faafc0b2880 2024-05-09T05:32:01.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:01 smithi161 ceph-mon[33805]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:01.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:01 smithi161 ceph-mon[33805]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:32:01.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:32:01.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:32:01.760 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:01 smithi195 ceph-mon[33844]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:01 smithi195 ceph-mon[33844]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:32:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:32:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:32:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:02.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:01 smithi115 ceph-mon[30349]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:01 smithi115 ceph-mon[30349]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:32:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:32:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:32:02.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:04.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:03 smithi195 ceph-mon[33844]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:03 smithi115 ceph-mon[30349]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:04.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:03 smithi161 ceph-mon[33805]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:05.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:04 smithi195 ceph-mon[33844]: from='client.? 172.21.15.161:0/2832010440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777e8971-300a-4291-b81c-13794e9d8f02"}]: dispatch 2024-05-09T05:32:05.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:04 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777e8971-300a-4291-b81c-13794e9d8f02"}]: dispatch 2024-05-09T05:32:05.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:04 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777e8971-300a-4291-b81c-13794e9d8f02"}]': finished 2024-05-09T05:32:05.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:04 smithi195 ceph-mon[33844]: osdmap e61: 11 total, 10 up, 11 in 2024-05-09T05:32:05.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:04 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:05.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:04 smithi195 ceph-mon[33844]: from='client.? 172.21.15.161:0/3985747060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T05:32:05.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:04 smithi115 ceph-mon[30349]: from='client.? 172.21.15.161:0/2832010440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777e8971-300a-4291-b81c-13794e9d8f02"}]: dispatch 2024-05-09T05:32:05.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:04 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777e8971-300a-4291-b81c-13794e9d8f02"}]: dispatch 2024-05-09T05:32:05.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:04 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777e8971-300a-4291-b81c-13794e9d8f02"}]': finished 2024-05-09T05:32:05.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:04 smithi115 ceph-mon[30349]: osdmap e61: 11 total, 10 up, 11 in 2024-05-09T05:32:05.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:04 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:05.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:04 smithi115 ceph-mon[30349]: from='client.? 172.21.15.161:0/3985747060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T05:32:05.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:04 smithi161 ceph-mon[33805]: from='client.? 172.21.15.161:0/2832010440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777e8971-300a-4291-b81c-13794e9d8f02"}]: dispatch 2024-05-09T05:32:05.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:04 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "777e8971-300a-4291-b81c-13794e9d8f02"}]: dispatch 2024-05-09T05:32:05.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:04 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777e8971-300a-4291-b81c-13794e9d8f02"}]': finished 2024-05-09T05:32:05.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:04 smithi161 ceph-mon[33805]: osdmap e61: 11 total, 10 up, 11 in 2024-05-09T05:32:05.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:04 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:05.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:04 smithi161 ceph-mon[33805]: from='client.? 172.21.15.161:0/3985747060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T05:32:06.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:05 smithi195 ceph-mon[33844]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:06.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:05 smithi115 ceph-mon[30349]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:06.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:05 smithi161 ceph-mon[33805]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:08.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:07 smithi195 ceph-mon[33844]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:08.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:07 smithi115 ceph-mon[30349]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:08.145 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:07 smithi161 ceph-mon[33805]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:10.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:09 smithi195 ceph-mon[33844]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:10.124 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:09 smithi161 ceph-mon[33805]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:10.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:09 smithi115 ceph-mon[30349]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:11 smithi195 ceph-mon[33844]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T05:32:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:12.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:11 smithi195 ceph-mon[33844]: Deploying daemon osd.10 on smithi161 2024-05-09T05:32:12.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:11 smithi115 ceph-mon[30349]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T05:32:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:11 smithi115 ceph-mon[30349]: Deploying daemon osd.10 on smithi161 2024-05-09T05:32:12.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:11 smithi161 ceph-mon[33805]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:12.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T05:32:12.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:12.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:11 smithi161 ceph-mon[33805]: Deploying daemon osd.10 on smithi161 2024-05-09T05:32:14.094 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:13 smithi161 ceph-mon[33805]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:14.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:13 smithi195 ceph-mon[33844]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:14.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:13 smithi115 ceph-mon[30349]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:16.027 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:15 smithi161 ceph-mon[33805]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:16.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:15 smithi195 ceph-mon[33844]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:16.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:15 smithi115 ceph-mon[30349]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:18.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:18 smithi195 ceph-mon[33844]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:18.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:18.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:18.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:18 smithi115 ceph-mon[30349]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:18 smithi161 ceph-mon[33805]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:20.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:20 smithi161 ceph-mon[33805]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:20.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:20 smithi195 ceph-mon[33844]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:20.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:20 smithi115 ceph-mon[30349]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:21.463 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 10 on host 'smithi161' 2024-05-09T05:32:21.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.469+0000 7fab0cff9700 1 -- 172.21.15.161:0/4244631470 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fab10063c00 con 0x7faafc0b2880 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faafc0b2880 msgr2=0x7faafc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faafc0b2880 0x7faafc0b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab10104290 tx=0x7faaf800a680 comp rx=0 tx=0).stop 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104390 msgr2=0x7fab1019f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104390 0x7fab1019f520 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab0000c920 tx=0x7fab0000cce0 comp rx=0 tx=0).stop 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 shutdown_connections 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faafc0b2880 0x7faafc0b4d90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab10104d30 0x7fab101a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab10103260 0x7fab100764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 --2- 172.21.15.161:0/4244631470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab10104390 0x7fab1019f520 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 >> 172.21.15.161:0/4244631470 conn(0x7fab100fe8c0 msgr2=0x7fab101075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:21.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 shutdown_connections 2024-05-09T05:32:21.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-09T05:32:21.472+0000 7fab15a80700 1 -- 172.21.15.161:0/4244631470 wait complete. 2024-05-09T05:32:21.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:21.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:21.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:21.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:21.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:21.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:21 smithi115 ceph-mon[30349]: from='osd.10 [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T05:32:21.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:21 smithi115 ceph-mon[30349]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T05:32:21.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:21.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:21.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:21.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:21.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:21.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:21 smithi161 ceph-mon[33805]: from='osd.10 [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T05:32:21.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:21 smithi161 ceph-mon[33805]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T05:32:21.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:21.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:21.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:21.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:21.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:21.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:21 smithi195 ceph-mon[33844]: from='osd.10 [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T05:32:21.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:21 smithi195 ceph-mon[33844]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T05:32:22.171 DEBUG:teuthology.orchestra.run.smithi161:osd.10> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.10.service 2024-05-09T05:32:22.174 INFO:tasks.cephadm:Deploying osd.11 on smithi195 with /dev/vg_nvme/lv_1... 2024-05-09T05:32:22.174 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T05:32:22.425 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:32:22.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:22 smithi195 ceph-mon[33844]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:22.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:22 smithi195 ceph-mon[33844]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T05:32:22.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:22 smithi195 ceph-mon[33844]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T05:32:22.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:22 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:22.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:22 smithi195 ceph-mon[33844]: from='osd.10 [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:32:22.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:22 smithi195 ceph-mon[33844]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:32:22.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:22 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:22.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:22 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:22.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:22 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:22.714 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:22 smithi161 ceph-mon[33805]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:22.714 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:22 smithi161 ceph-mon[33805]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T05:32:22.714 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:22 smithi161 ceph-mon[33805]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T05:32:22.714 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:22 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:22.714 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:22 smithi161 ceph-mon[33805]: from='osd.10 [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:32:22.714 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:22 smithi161 ceph-mon[33805]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:32:22.714 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:22 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:22.715 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:22 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:22.715 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:22 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:22.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:22 smithi115 ceph-mon[30349]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:22 smithi115 ceph-mon[30349]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T05:32:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:22 smithi115 ceph-mon[30349]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T05:32:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:22 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:22 smithi115 ceph-mon[30349]: from='osd.10 [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:32:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:22 smithi115 ceph-mon[30349]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-09T05:32:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:22 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:22 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:22.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:22 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:23.161 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 05:32:22 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T05:32:22.859+0000 7fbaef257700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T05:32:23.161 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 05:32:22 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T05:32:22.873+0000 7fbae7004700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:32:23.678 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:23 smithi195 ceph-mon[33844]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:32:23.678 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:23 smithi195 ceph-mon[33844]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T05:32:23.678 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:23.678 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:23.678 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:23 smithi195 ceph-mon[33844]: from='osd.10 ' entity='osd.10' 2024-05-09T05:32:23.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:23 smithi161 ceph-mon[33805]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:32:23.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:23 smithi161 ceph-mon[33805]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T05:32:23.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:23.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:23.767 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:23 smithi161 ceph-mon[33805]: from='osd.10 ' entity='osd.10' 2024-05-09T05:32:23.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:23 smithi115 ceph-mon[30349]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-09T05:32:23.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:23 smithi115 ceph-mon[30349]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T05:32:23.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:23.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:23.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:23 smithi115 ceph-mon[30349]: from='osd.10 ' entity='osd.10' 2024-05-09T05:32:24.551 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T05:32:24.551 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T05:32:24.551 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-05-09T05:32:24.551 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-05-09T05:32:24.551 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00873502 s, 1.2 GB/s 2024-05-09T05:32:24.551 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-05-09T05:32:24.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:24 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:32:24.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:24 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:32:24.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:24 smithi161 ceph-mon[33805]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:24.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:24.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:24 smithi161 ceph-mon[33805]: osd.10 [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] boot 2024-05-09T05:32:24.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:24 smithi161 ceph-mon[33805]: osdmap e64: 11 total, 11 up, 11 in 2024-05-09T05:32:24.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:24.705 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:24 smithi161 ceph-mon[33805]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T05:32:24.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:24 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:32:24.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:24 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:32:24.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:24 smithi195 ceph-mon[33844]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:24.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:24.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:24 smithi195 ceph-mon[33844]: osd.10 [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] boot 2024-05-09T05:32:24.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:24 smithi195 ceph-mon[33844]: osdmap e64: 11 total, 11 up, 11 in 2024-05-09T05:32:24.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:24.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:24 smithi195 ceph-mon[33844]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T05:32:24.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:24 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:32:24.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:24 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:32:24.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:24 smithi115 ceph-mon[30349]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 894 GiB / 894 GiB avail 2024-05-09T05:32:24.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:24.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:24 smithi115 ceph-mon[30349]: osd.10 [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] boot 2024-05-09T05:32:24.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:24 smithi115 ceph-mon[30349]: osdmap e64: 11 total, 11 up, 11 in 2024-05-09T05:32:24.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:32:24.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:24 smithi115 ceph-mon[30349]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T05:32:25.314 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_1 2024-05-09T05:32:25.614 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:32:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:25 smithi161 ceph-mon[33805]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T05:32:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:25 smithi161 ceph-mon[33805]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:25.814 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:25 smithi115 ceph-mon[30349]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T05:32:25.815 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:25 smithi115 ceph-mon[30349]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:25.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:25 smithi195 ceph-mon[33844]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T05:32:25.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:25 smithi195 ceph-mon[33844]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi161 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi161 to 4272M 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:26.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:26.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:26 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:27.040 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi161 2024-05-09T05:32:27.040 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi161 to 4272M 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:27.041 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:26 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi161 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi161 to 4272M 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:27.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:27.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:26 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:27.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.276+0000 7fbe7e29e700 1 -- 172.21.15.195:0/3473954610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 msgr2=0x7fbe7810b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:27.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.276+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/3473954610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 0x7fbe7810b5d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000b0a0 tx=0x7fbe7000b3b0 comp rx=0 tx=0).stop 2024-05-09T05:32:27.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.276+0000 7fbe7e29e700 1 -- 172.21.15.195:0/3473954610 shutdown_connections 2024-05-09T05:32:27.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.276+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/3473954610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 0x7fbe7810b5d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:27.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.276+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/3473954610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe78103250 0x7fbe78103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:27.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.276+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/3473954610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe78104380 0x7fbe78104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:27.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.276+0000 7fbe7e29e700 1 -- 172.21.15.195:0/3473954610 >> 172.21.15.195:0/3473954610 conn(0x7fbe780fe890 msgr2=0x7fbe78100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:27.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.276+0000 7fbe7e29e700 1 -- 172.21.15.195:0/3473954610 shutdown_connections 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.276+0000 7fbe7e29e700 1 -- 172.21.15.195:0/3473954610 wait complete. 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7e29e700 1 Processor -- start 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7e29e700 1 -- start start 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7e29e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe78103250 0x7fbe781996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7e29e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe78104380 0x7fbe78199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7e29e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 0x7fbe781a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7e29e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7810eb60 con 0x7fbe78103250 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7e29e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7810e9e0 con 0x7fbe78104380 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7e29e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7810ece0 con 0x7fbe78104d20 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7c83b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 0x7fbe781a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:27.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7c83b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 0x7fbe781a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:38092/0 (socket says 172.21.15.195:38092) 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.277+0000 7fbe7c83b700 1 -- 172.21.15.195:0/1016750136 learned_addr learned my addr 172.21.15.195:0/1016750136 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe77fff700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe78103250 0x7fbe781996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe6ffff700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe78104380 0x7fbe78199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe7c83b700 1 -- 172.21.15.195:0/1016750136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe78104380 msgr2=0x7fbe78199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe7c83b700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe78104380 0x7fbe78199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe7c83b700 1 -- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe78103250 msgr2=0x7fbe781996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe7c83b700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe78103250 0x7fbe781996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe7c83b700 1 -- 172.21.15.195:0/1016750136 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe781a1210 con 0x7fbe78104d20 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe7c83b700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 0x7fbe781a0b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009030 tx=0x7fbe700097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:27.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe75ffb700 1 -- 172.21.15.195:0/1016750136 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7001f030 con 0x7fbe78104d20 2024-05-09T05:32:27.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe75ffb700 1 -- 172.21.15.195:0/1016750136 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe70005040 con 0x7fbe78104d20 2024-05-09T05:32:27.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe781a14a0 con 0x7fbe78104d20 2024-05-09T05:32:27.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe75ffb700 1 -- 172.21.15.195:0/1016750136 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7001b690 con 0x7fbe78104d20 2024-05-09T05:32:27.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.278+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe781a1950 con 0x7fbe78104d20 2024-05-09T05:32:27.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.279+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe780512c0 con 0x7fbe78104d20 2024-05-09T05:32:27.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.281+0000 7fbe75ffb700 1 -- 172.21.15.195:0/1016750136 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fbe70022030 con 0x7fbe78104d20 2024-05-09T05:32:27.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.281+0000 7fbe75ffb700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe640b2880 0x7fbe640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:27.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.281+0000 7fbe75ffb700 1 -- 172.21.15.195:0/1016750136 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7fbe7000f020 con 0x7fbe78104d20 2024-05-09T05:32:27.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.282+0000 7fbe77fff700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe640b2880 0x7fbe640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:27.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.282+0000 7fbe77fff700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe640b2880 0x7fbe640b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe78104280 tx=0x7fbe6800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:27.280 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.286+0000 7fbe75ffb700 1 -- 172.21.15.195:0/1016750136 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe70029020 con 0x7fbe78104d20 2024-05-09T05:32:27.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:27.499+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbe78063c00 con 0x7fbe640b2880 2024-05-09T05:32:28.107 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:27 smithi195 ceph-mon[33844]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:28.108 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:32:28.108 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:32:28.108 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:28.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:27 smithi115 ceph-mon[30349]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:28.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:32:28.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:32:28.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:28.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:27 smithi161 ceph-mon[33805]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:28.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T05:32:28.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T05:32:28.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:29.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:28 smithi195 ceph-mon[33844]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:32:29.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:28 smithi115 ceph-mon[30349]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:32:29.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:28 smithi161 ceph-mon[33805]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:32:30.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:29 smithi195 ceph-mon[33844]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:30.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:29 smithi115 ceph-mon[30349]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:30.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:29 smithi161 ceph-mon[33805]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:31.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:30 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/333715488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "debd622f-268c-4946-af2a-e687f95c4d89"}]: dispatch 2024-05-09T05:32:31.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:30 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "debd622f-268c-4946-af2a-e687f95c4d89"}]: dispatch 2024-05-09T05:32:31.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:30 smithi195 ceph-mon[33844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "debd622f-268c-4946-af2a-e687f95c4d89"}]': finished 2024-05-09T05:32:31.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:30 smithi195 ceph-mon[33844]: osdmap e67: 12 total, 11 up, 12 in 2024-05-09T05:32:31.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:31.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:30 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/333715488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "debd622f-268c-4946-af2a-e687f95c4d89"}]: dispatch 2024-05-09T05:32:31.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:30 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "debd622f-268c-4946-af2a-e687f95c4d89"}]: dispatch 2024-05-09T05:32:31.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:30 smithi115 ceph-mon[30349]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "debd622f-268c-4946-af2a-e687f95c4d89"}]': finished 2024-05-09T05:32:31.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:30 smithi115 ceph-mon[30349]: osdmap e67: 12 total, 11 up, 12 in 2024-05-09T05:32:31.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:31.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:30 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/333715488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "debd622f-268c-4946-af2a-e687f95c4d89"}]: dispatch 2024-05-09T05:32:31.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:30 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "debd622f-268c-4946-af2a-e687f95c4d89"}]: dispatch 2024-05-09T05:32:31.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:30 smithi161 ceph-mon[33805]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "debd622f-268c-4946-af2a-e687f95c4d89"}]': finished 2024-05-09T05:32:31.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:30 smithi161 ceph-mon[33805]: osdmap e67: 12 total, 11 up, 12 in 2024-05-09T05:32:31.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:32.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:31 smithi195 ceph-mon[33844]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:32:32.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:31 smithi195 ceph-mon[33844]: from='client.? 172.21.15.195:0/233087186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T05:32:32.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:31 smithi115 ceph-mon[30349]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:32:32.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:31 smithi115 ceph-mon[30349]: from='client.? 172.21.15.195:0/233087186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T05:32:32.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:31 smithi161 ceph-mon[33805]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-09T05:32:32.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:31 smithi161 ceph-mon[33805]: from='client.? 172.21.15.195:0/233087186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T05:32:34.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:33 smithi195 ceph-mon[33844]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T05:32:34.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:33 smithi115 ceph-mon[30349]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T05:32:34.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:33 smithi161 ceph-mon[33805]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T05:32:35.929 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:35 smithi195 ceph-mon[33844]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:32:36.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:35 smithi115 ceph-mon[30349]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:32:36.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:35 smithi161 ceph-mon[33805]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:32:38.016 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:37 smithi195 ceph-mon[33844]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:32:38.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:37 smithi115 ceph-mon[30349]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:32:38.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:37 smithi161 ceph-mon[33805]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:32:38.969 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T05:32:38.970 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:38.970 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:38 smithi195 ceph-mon[33844]: Deploying daemon osd.11 on smithi195 2024-05-09T05:32:39.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T05:32:39.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:39.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:38 smithi115 ceph-mon[30349]: Deploying daemon osd.11 on smithi195 2024-05-09T05:32:39.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T05:32:39.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:39.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:38 smithi161 ceph-mon[33805]: Deploying daemon osd.11 on smithi195 2024-05-09T05:32:39.952 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:39 smithi195 ceph-mon[33844]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:32:40.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:39 smithi115 ceph-mon[30349]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:32:40.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:39 smithi161 ceph-mon[33805]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T05:32:42.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:41 smithi195 ceph-mon[33844]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:42.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:41 smithi115 ceph-mon[30349]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:42.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:41 smithi161 ceph-mon[33805]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:44.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:43 smithi195 ceph-mon[33844]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:43 smithi115 ceph-mon[30349]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:44.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:43 smithi161 ceph-mon[33805]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:45.048 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:44 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:45.048 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:44 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:45.048 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:44 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:45.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:45.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:45.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:44 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:45.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:44 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:45.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:44 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:45.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:44 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:46.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:46 smithi195 ceph-mon[33844]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:46 smithi115 ceph-mon[30349]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:46 smithi161 ceph-mon[33805]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:48.111 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 11 on host 'smithi195' 2024-05-09T05:32:48.111 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.117+0000 7fbe75ffb700 1 -- 172.21.15.195:0/1016750136 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbe78063c00 con 0x7fbe640b2880 2024-05-09T05:32:48.113 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe640b2880 msgr2=0x7fbe640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:48.113 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe640b2880 0x7fbe640b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe78104280 tx=0x7fbe6800b410 comp rx=0 tx=0).stop 2024-05-09T05:32:48.113 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 msgr2=0x7fbe781a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:48.113 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 0x7fbe781a0b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009030 tx=0x7fbe700097f0 comp rx=0 tx=0).stop 2024-05-09T05:32:48.113 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 shutdown_connections 2024-05-09T05:32:48.114 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe640b2880 0x7fbe640b4d90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:48.114 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe78104d20 0x7fbe781a0b90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:48.114 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe78103250 0x7fbe781996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:48.114 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 --2- 172.21.15.195:0/1016750136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe78104380 0x7fbe78199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:48.114 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 >> 172.21.15.195:0/1016750136 conn(0x7fbe780fe890 msgr2=0x7fbe781075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:48.114 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 shutdown_connections 2024-05-09T05:32:48.114 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:48.120+0000 7fbe7e29e700 1 -- 172.21.15.195:0/1016750136 wait complete. 2024-05-09T05:32:48.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:48.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:48.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:48.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='osd.11 [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T05:32:48.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T05:32:48.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.368 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:48.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='osd.11 [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:48.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:48.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:48.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:48.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='osd.11 [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T05:32:48.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T05:32:48.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:48.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:32:48.822 DEBUG:teuthology.orchestra.run.smithi195:osd.11> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.11.service 2024-05-09T05:32:48.825 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T05:32:48.825 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T05:32:49.076 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:32:49.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:49 smithi195 ceph-mon[33844]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T05:32:49.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:49 smithi195 ceph-mon[33844]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T05:32:49.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:49 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:49.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:49 smithi195 ceph-mon[33844]: from='osd.11 [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:32:49.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:49 smithi195 ceph-mon[33844]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:32:49.367 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 05:32:49 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T05:32:49.275+0000 7fe61cd0f700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T05:32:49.367 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 05:32:49 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T05:32:49.292+0000 7fe618ac4700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T05:32:49.473 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:49 smithi115 ceph-mon[30349]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T05:32:49.473 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:49 smithi115 ceph-mon[30349]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T05:32:49.473 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:49 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:49.474 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:49 smithi115 ceph-mon[30349]: from='osd.11 [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:32:49.474 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:49 smithi115 ceph-mon[30349]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:32:49.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:49 smithi161 ceph-mon[33805]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T05:32:49.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:49 smithi161 ceph-mon[33805]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T05:32:49.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:49 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:49.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:49 smithi161 ceph-mon[33805]: from='osd.11 [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:32:49.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:49 smithi161 ceph-mon[33805]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-05-09T05:32:50.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:50 smithi195 ceph-mon[33844]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:50.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:50 smithi195 ceph-mon[33844]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:32:50.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:50 smithi195 ceph-mon[33844]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T05:32:50.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:50 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:50.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:50 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:50.546 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:50 smithi115 ceph-mon[30349]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:50.546 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:50 smithi115 ceph-mon[30349]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:32:50.546 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:50 smithi115 ceph-mon[30349]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T05:32:50.546 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:50 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:50.546 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:50 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:50.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:50 smithi161 ceph-mon[33805]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-05-09T05:32:50.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:50 smithi161 ceph-mon[33805]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-05-09T05:32:50.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:50 smithi161 ceph-mon[33805]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T05:32:50.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:50 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:50.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:50 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.799+0000 7f3bcb30f700 1 -- 172.21.15.115:0/3615011383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4104830 msgr2=0x7f3bc4104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.799+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/3615011383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4104830 0x7f3bc4104c40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009990 tx=0x7f3bc0009ca0 comp rx=0 tx=0).stop 2024-05-09T05:32:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.799+0000 7f3bcb30f700 1 -- 172.21.15.115:0/3615011383 shutdown_connections 2024-05-09T05:32:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.799+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/3615011383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bc410c3e0 0x7f3bc410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.799+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/3615011383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4104830 0x7f3bc4104c40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.799+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/3615011383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc4109a80 0x7f3bc410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.799+0000 7f3bcb30f700 1 -- 172.21.15.115:0/3615011383 >> 172.21.15.115:0/3615011383 conn(0x7f3bc40fe020 msgr2=0x7f3bc41004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.799+0000 7f3bcb30f700 1 -- 172.21.15.115:0/3615011383 shutdown_connections 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.799+0000 7f3bcb30f700 1 -- 172.21.15.115:0/3615011383 wait complete. 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bcb30f700 1 Processor -- start 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bcb30f700 1 -- start start 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bcb30f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bc4104830 0x7f3bc419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bcb30f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4109a80 0x7f3bc419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bcb30f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc410c3e0 0x7f3bc41a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bcb30f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4111e60 con 0x7f3bc4109a80 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bcb30f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4111ce0 con 0x7f3bc410c3e0 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bcb30f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4111fe0 con 0x7f3bc4104830 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bc88aa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4109a80 0x7f3bc419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bc88aa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4109a80 0x7f3bc419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42088/0 (socket says 172.21.15.115:42088) 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bc88aa700 1 -- 172.21.15.115:0/1076298218 learned_addr learned my addr 172.21.15.115:0/1076298218 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bc90ab700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bc4104830 0x7f3bc419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.800+0000 7f3bc98ac700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc410c3e0 0x7f3bc41a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bc88aa700 1 -- 172.21.15.115:0/1076298218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bc4104830 msgr2=0x7f3bc419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bc88aa700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bc4104830 0x7f3bc419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bc88aa700 1 -- 172.21.15.115:0/1076298218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc410c3e0 msgr2=0x7f3bc41a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bc88aa700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc410c3e0 0x7f3bc41a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bc88aa700 1 -- 172.21.15.115:0/1076298218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc41a4560 con 0x7f3bc4109a80 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bc98ac700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc410c3e0 0x7f3bc41a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bc88aa700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4109a80 0x7f3bc419cf40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400c920 tx=0x7f3bb400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bba7fc700 1 -- 172.21.15.115:0/1076298218 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4007b00 con 0x7f3bc4109a80 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bba7fc700 1 -- 172.21.15.115:0/1076298218 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bb4004cc0 con 0x7f3bc4109a80 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc41a4850 con 0x7f3bc4109a80 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bba7fc700 1 -- 172.21.15.115:0/1076298218 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4005440 con 0x7f3bc4109a80 2024-05-09T05:32:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.801+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41a4d90 con 0x7f3bc4109a80 2024-05-09T05:32:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.802+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc40512c0 con 0x7f3bc4109a80 2024-05-09T05:32:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.804+0000 7f3bba7fc700 1 -- 172.21.15.115:0/1076298218 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3bb4004750 con 0x7f3bc4109a80 2024-05-09T05:32:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.805+0000 7f3bba7fc700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3bb00b2880 0x7f3bb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.805+0000 7f3bc90ab700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3bb00b2880 0x7f3bb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.805+0000 7f3bba7fc700 1 -- 172.21.15.115:0/1076298218 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f3bb40e1900 con 0x7f3bc4109a80 2024-05-09T05:32:50.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.805+0000 7f3bc90ab700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3bb00b2880 0x7f3bb00b4d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0006010 tx=0x7f3bc001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:50.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:50.809+0000 7f3bba7fc700 1 -- 172.21.15.115:0/1076298218 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3bb4014030 con 0x7f3bc4109a80 2024-05-09T05:32:51.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.013+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3bc402e5f0 con 0x7f3bc4109a80 2024-05-09T05:32:51.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.013+0000 7f3bba7fc700 1 -- 172.21.15.115:0/1076298218 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f3bb40a9ac0 con 0x7f3bc4109a80 2024-05-09T05:32:51.007 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:32:51.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3bb00b2880 msgr2=0x7f3bb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:51.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3bb00b2880 0x7f3bb00b4d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0006010 tx=0x7f3bc001a040 comp rx=0 tx=0).stop 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4109a80 msgr2=0x7f3bc419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4109a80 0x7f3bc419cf40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400c920 tx=0x7f3bb400cc30 comp rx=0 tx=0).stop 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 shutdown_connections 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3bb00b2880 0x7f3bb00b4d90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc410c3e0 0x7f3bc41a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bc4104830 0x7f3bc419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 --2- 172.21.15.115:0/1076298218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4109a80 0x7f3bc419cf40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 >> 172.21.15.115:0/1076298218 conn(0x7f3bc40fe020 msgr2=0x7f3bc410a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 shutdown_connections 2024-05-09T05:32:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:51.016+0000 7f3bcb30f700 1 -- 172.21.15.115:0/1076298218 wait complete. 2024-05-09T05:32:51.546 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:51 smithi195 ceph-mon[33844]: purged_snaps scrub starts 2024-05-09T05:32:51.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:51 smithi195 ceph-mon[33844]: purged_snaps scrub ok 2024-05-09T05:32:51.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:51 smithi195 ceph-mon[33844]: osd.11 [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] boot 2024-05-09T05:32:51.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:51 smithi195 ceph-mon[33844]: osdmap e70: 12 total, 12 up, 12 in 2024-05-09T05:32:51.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:51.547 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:51 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1076298218' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T05:32:51.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:51 smithi115 ceph-mon[30349]: purged_snaps scrub starts 2024-05-09T05:32:51.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:51 smithi115 ceph-mon[30349]: purged_snaps scrub ok 2024-05-09T05:32:51.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:51 smithi115 ceph-mon[30349]: osd.11 [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] boot 2024-05-09T05:32:51.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:51 smithi115 ceph-mon[30349]: osdmap e70: 12 total, 12 up, 12 in 2024-05-09T05:32:51.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:51.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:51 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1076298218' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T05:32:51.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:51 smithi161 ceph-mon[33805]: purged_snaps scrub starts 2024-05-09T05:32:51.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:51 smithi161 ceph-mon[33805]: purged_snaps scrub ok 2024-05-09T05:32:51.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:51 smithi161 ceph-mon[33805]: osd.11 [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] boot 2024-05-09T05:32:51.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:51 smithi161 ceph-mon[33805]: osdmap e70: 12 total, 12 up, 12 in 2024-05-09T05:32:51.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:32:51.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:51 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1076298218' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T05:32:51.714 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1715232770,"num_in_osds":12,"osd_in_since":1715232750,"num_remapped_pgs":0} 2024-05-09T05:32:51.715 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:32:51.964 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:32:52.357 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:52.357 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T05:32:52.357 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.357 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.357 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.357 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:52.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:52.358 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:52.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T05:32:52.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:52.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:52.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:52.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:32:52.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:32:53.444 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:53 smithi115 ceph-mon[30349]: Detected new or changed devices on smithi195 2024-05-09T05:32:53.444 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:53 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi195 to 4272M 2024-05-09T05:32:53.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:53 smithi195 ceph-mon[33844]: Detected new or changed devices on smithi195 2024-05-09T05:32:53.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:53 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi195 to 4272M 2024-05-09T05:32:53.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:53 smithi161 ceph-mon[33805]: Detected new or changed devices on smithi195 2024-05-09T05:32:53.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:53 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi195 to 4272M 2024-05-09T05:32:53.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.692+0000 7ff82d223700 1 -- 172.21.15.115:0/22654189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff828102790 msgr2=0x7ff828102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:53.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.692+0000 7ff82d223700 1 --2- 172.21.15.115:0/22654189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff828102790 0x7ff828102ba0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff810009990 tx=0x7ff810009ca0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.692+0000 7ff82d223700 1 -- 172.21.15.115:0/22654189 shutdown_connections 2024-05-09T05:32:53.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.692+0000 7ff82d223700 1 --2- 172.21.15.115:0/22654189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff828109300 0x7ff828109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.692+0000 7ff82d223700 1 --2- 172.21.15.115:0/22654189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8281030e0 0x7ff828111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.692+0000 7ff82d223700 1 --2- 172.21.15.115:0/22654189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff828102790 0x7ff828102ba0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.692+0000 7ff82d223700 1 -- 172.21.15.115:0/22654189 >> 172.21.15.115:0/22654189 conn(0x7ff8280fe520 msgr2=0x7ff828100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:53.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.692+0000 7ff82d223700 1 -- 172.21.15.115:0/22654189 shutdown_connections 2024-05-09T05:32:53.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.692+0000 7ff82d223700 1 -- 172.21.15.115:0/22654189 wait complete. 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.693+0000 7ff82d223700 1 Processor -- start 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.693+0000 7ff82d223700 1 -- start start 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.693+0000 7ff82d223700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff828102790 0x7ff828103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.693+0000 7ff82d223700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8281030e0 0x7ff828103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.693+0000 7ff82d223700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff828109300 0x7ff8281043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff82d223700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff828114450 con 0x7ff8281030e0 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff82d223700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff8281142d0 con 0x7ff828109300 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff82d223700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff8281145d0 con 0x7ff828102790 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff81e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8281030e0 0x7ff828103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:53.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff81e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8281030e0 0x7ff828103eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58930/0 (socket says 172.21.15.115:58930) 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff81e59c700 1 -- 172.21.15.115:0/3786806236 learned_addr learned my addr 172.21.15.115:0/3786806236 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff826d9d700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff828102790 0x7ff828103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff82759e700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff828109300 0x7ff8281043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff81e59c700 1 -- 172.21.15.115:0/3786806236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff828102790 msgr2=0x7ff828103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff81e59c700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff828102790 0x7ff828103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff81e59c700 1 -- 172.21.15.115:0/3786806236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff828109300 msgr2=0x7ff8281043f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff81e59c700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff828109300 0x7ff8281043f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff81e59c700 1 -- 172.21.15.115:0/3786806236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8281a90d0 con 0x7ff8281030e0 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff81e59c700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8281030e0 0x7ff828103eb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff8100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff824d99700 1 -- 172.21.15.115:0/3786806236 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810027070 con 0x7ff8281030e0 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff824d99700 1 -- 172.21.15.115:0/3786806236 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff81000fcc0 con 0x7ff8281030e0 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8281a9360 con 0x7ff8281030e0 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff824d99700 1 -- 172.21.15.115:0/3786806236 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810022d40 con 0x7ff8281030e0 2024-05-09T05:32:53.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.694+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8281a97b0 con 0x7ff8281030e0 2024-05-09T05:32:53.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.695+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8280512c0 con 0x7ff8281030e0 2024-05-09T05:32:53.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.697+0000 7ff824d99700 1 -- 172.21.15.115:0/3786806236 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff81002a070 con 0x7ff8281030e0 2024-05-09T05:32:53.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.698+0000 7ff824d99700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff8140b2880 0x7ff8140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:53.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.698+0000 7ff826d9d700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff8140b2880 0x7ff8140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:53.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.698+0000 7ff824d99700 1 -- 172.21.15.115:0/3786806236 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff8100e30e0 con 0x7ff8281030e0 2024-05-09T05:32:53.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.698+0000 7ff826d9d700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff8140b2880 0x7ff8140b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff82806ca70 tx=0x7ff81800a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:53.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.701+0000 7ff824d99700 1 -- 172.21.15.115:0/3786806236 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff810013070 con 0x7ff8281030e0 2024-05-09T05:32:53.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.907+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff82802e5f0 con 0x7ff8281030e0 2024-05-09T05:32:53.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.908+0000 7ff824d99700 1 -- 172.21.15.115:0/3786806236 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18526 (secure 0 0 0) 0x7ff8100ab330 con 0x7ff8281030e0 2024-05-09T05:32:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:32:53.902 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":71,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:32:51.276183+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff8140b2880 msgr2=0x7ff8140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff8140b2880 0x7ff8140b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff82806ca70 tx=0x7ff81800a3b0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8281030e0 msgr2=0x7ff828103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8281030e0 0x7ff828103eb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff8100047b0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 shutdown_connections 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff8140b2880 0x7ff8140b4d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff828109300 0x7ff8281043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff828102790 0x7ff828103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 --2- 172.21.15.115:0/3786806236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8281030e0 0x7ff828103eb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 >> 172.21.15.115:0/3786806236 conn(0x7ff8280fe520 msgr2=0x7ff828107190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 shutdown_connections 2024-05-09T05:32:53.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:53.910+0000 7ff82d223700 1 -- 172.21.15.115:0/3786806236 wait complete. 2024-05-09T05:32:54.613 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:54 smithi115 ceph-mon[30349]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:54.613 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:54 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3786806236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:32:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:54 smithi195 ceph-mon[33844]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:54.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:54 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3786806236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:32:54.646 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T05:28:58.644670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T05:32:54.646 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T05:32:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:54 smithi161 ceph-mon[33805]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:54 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3786806236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:32:54.890 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:32:56.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:56 smithi115 ceph-mon[30349]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:56.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:56 smithi195 ceph-mon[33844]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 -- 172.21.15.115:0/786358585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500108740 msgr2=0x7f15001100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 --2- 172.21.15.115:0/786358585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500108740 0x7f15001100a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f14f800b0d0 tx=0x7f14f800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 -- 172.21.15.115:0/786358585 shutdown_connections 2024-05-09T05:32:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 --2- 172.21.15.115:0/786358585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500108740 0x7f15001100a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 --2- 172.21.15.115:0/786358585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15001034c0 0x7f15001038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 --2- 172.21.15.115:0/786358585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1500107df0 0x7f1500108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 -- 172.21.15.115:0/786358585 >> 172.21.15.115:0/786358585 conn(0x7f15000fedf0 msgr2=0x7f1500101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 -- 172.21.15.115:0/786358585 shutdown_connections 2024-05-09T05:32:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 -- 172.21.15.115:0/786358585 wait complete. 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 Processor -- start 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.665+0000 7f150721f700 1 -- start start 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f150721f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15001034c0 0x7f15000ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f150721f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500107df0 0x7f15000ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f150721f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1500108740 0x7f15000ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f150721f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1500113310 con 0x7f1500107df0 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f150721f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1500113190 con 0x7f15001034c0 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f150721f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1500113490 con 0x7f1500108740 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500107df0 0x7f15000ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500107df0 0x7f15000ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58940/0 (socket says 172.21.15.115:58940) 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fffff700 1 -- 172.21.15.115:0/1386577784 learned_addr learned my addr 172.21.15.115:0/1386577784 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f15057bc700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1500108740 0x7f15000ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f1504fbb700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15001034c0 0x7f15000ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fffff700 1 -- 172.21.15.115:0/1386577784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1500108740 msgr2=0x7f15000ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fffff700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1500108740 0x7f15000ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fffff700 1 -- 172.21.15.115:0/1386577784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15001034c0 msgr2=0x7f15000ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fffff700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15001034c0 0x7f15000ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fffff700 1 -- 172.21.15.115:0/1386577784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15001a9210 con 0x7f1500107df0 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f1504fbb700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15001034c0 0x7f15000ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fffff700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500107df0 0x7f15000ff810 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00b500 tx=0x7f14ec00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.666+0000 7f14fdffb700 1 -- 172.21.15.115:0/1386577784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec015020 con 0x7f1500107df0 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.667+0000 7f14fdffb700 1 -- 172.21.15.115:0/1386577784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14ec011cc0 con 0x7f1500107df0 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.667+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15001a9500 con 0x7f1500107df0 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.667+0000 7f14fdffb700 1 -- 172.21.15.115:0/1386577784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec010c80 con 0x7f1500107df0 2024-05-09T05:32:56.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.667+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15001a9a40 con 0x7f1500107df0 2024-05-09T05:32:56.664 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:56 smithi161 ceph-mon[33805]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:56.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.668+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15000512c0 con 0x7f1500107df0 2024-05-09T05:32:56.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.669+0000 7f14fdffb700 1 -- 172.21.15.115:0/1386577784 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f14ec018020 con 0x7f1500107df0 2024-05-09T05:32:56.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.670+0000 7f14fdffb700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f14f00b2880 0x7f14f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:56.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.670+0000 7f1504fbb700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f14f00b2880 0x7f14f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:56.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.670+0000 7f14fdffb700 1 -- 172.21.15.115:0/1386577784 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f14ec0e1490 con 0x7f1500107df0 2024-05-09T05:32:56.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.670+0000 7f1504fbb700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f14f00b2880 0x7f14f00b4d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14f4007900 tx=0x7f14f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:56.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.674+0000 7f14fdffb700 1 -- 172.21.15.115:0/1386577784 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14ec01f020 con 0x7f1500107df0 2024-05-09T05:32:56.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.878+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f15001a9d00 con 0x7f1500107df0 2024-05-09T05:32:56.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.879+0000 7f14fdffb700 1 -- 172.21.15.115:0/1386577784 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f14ec0a9650 con 0x7f1500107df0 2024-05-09T05:32:56.872 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-05-09T05:32:56.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f14f00b2880 msgr2=0x7f14f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:56.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f14f00b2880 0x7f14f00b4d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14f4007900 tx=0x7f14f4008040 comp rx=0 tx=0).stop 2024-05-09T05:32:56.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500107df0 msgr2=0x7f15000ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:56.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500107df0 0x7f15000ff810 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00b500 tx=0x7f14ec00b810 comp rx=0 tx=0).stop 2024-05-09T05:32:56.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 shutdown_connections 2024-05-09T05:32:56.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f14f00b2880 0x7f14f00b4d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15001034c0 0x7f15000ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1500108740 0x7f15000ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 --2- 172.21.15.115:0/1386577784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1500107df0 0x7f15000ff810 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:56.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 >> 172.21.15.115:0/1386577784 conn(0x7f15000fedf0 msgr2=0x7f15001096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:56.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 shutdown_connections 2024-05-09T05:32:56.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:32:56.881+0000 7f150721f700 1 -- 172.21.15.115:0/1386577784 wait complete. 2024-05-09T05:32:57.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:57 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1386577784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T05:32:57.623 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:57 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1386577784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T05:32:57.650 INFO:tasks.cephadm:Adding mds.a on smithi115 2024-05-09T05:32:57.650 INFO:tasks.cephadm:Adding mds.d on smithi115 2024-05-09T05:32:57.650 INFO:tasks.cephadm:Adding mds.g on smithi115 2024-05-09T05:32:57.650 INFO:tasks.cephadm:Adding mds.j on smithi115 2024-05-09T05:32:57.650 INFO:tasks.cephadm:Adding mds.b on smithi161 2024-05-09T05:32:57.650 INFO:tasks.cephadm:Adding mds.e on smithi161 2024-05-09T05:32:57.650 INFO:tasks.cephadm:Adding mds.h on smithi161 2024-05-09T05:32:57.650 INFO:tasks.cephadm:Adding mds.k on smithi161 2024-05-09T05:32:57.651 INFO:tasks.cephadm:Adding mds.c on smithi195 2024-05-09T05:32:57.651 INFO:tasks.cephadm:Adding mds.f on smithi195 2024-05-09T05:32:57.651 INFO:tasks.cephadm:Adding mds.i on smithi195 2024-05-09T05:32:57.651 INFO:tasks.cephadm:Adding mds.l on smithi195 2024-05-09T05:32:57.651 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi115=a;smithi115=d;smithi115=g;smithi115=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l' 2024-05-09T05:32:57.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:57 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1386577784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T05:32:57.906 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:32:58.294 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:32:58 smithi195 ceph-mon[33844]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:58.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:32:58 smithi115 ceph-mon[30349]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:58.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:32:58 smithi161 ceph-mon[33805]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.645+0000 7eff68513700 1 -- 172.21.15.195:0/3457398413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 msgr2=0x7eff6010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.645+0000 7eff68513700 1 --2- 172.21.15.195:0/3457398413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 0x7eff6010b5d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7eff58009990 tx=0x7eff58009ca0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.645+0000 7eff68513700 1 -- 172.21.15.195:0/3457398413 shutdown_connections 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.645+0000 7eff68513700 1 --2- 172.21.15.195:0/3457398413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 0x7eff6010b5d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.645+0000 7eff68513700 1 --2- 172.21.15.195:0/3457398413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff60103250 0x7eff60103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.645+0000 7eff68513700 1 --2- 172.21.15.195:0/3457398413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff60104380 0x7eff60104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.645+0000 7eff68513700 1 -- 172.21.15.195:0/3457398413 >> 172.21.15.195:0/3457398413 conn(0x7eff600fe890 msgr2=0x7eff60100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 -- 172.21.15.195:0/3457398413 shutdown_connections 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 -- 172.21.15.195:0/3457398413 wait complete. 2024-05-09T05:32:59.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 Processor -- start 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 -- start start 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff60103250 0x7eff60107010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff60104380 0x7eff601056c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 0x7eff60105c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010eb60 con 0x7eff60103250 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010e9e0 con 0x7eff60104380 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff68513700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010ece0 con 0x7eff60104d20 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.646+0000 7eff66ab0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 0x7eff60105c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff66ab0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 0x7eff60105c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:41142/0 (socket says 172.21.15.195:41142) 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff66ab0700 1 -- 172.21.15.195:0/3268951800 learned_addr learned my addr 172.21.15.195:0/3268951800 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff65aae700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff60104380 0x7eff601056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff662af700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff60103250 0x7eff60107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff66ab0700 1 -- 172.21.15.195:0/3268951800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff60104380 msgr2=0x7eff601056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:59.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff66ab0700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff60104380 0x7eff601056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff66ab0700 1 -- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff60103250 msgr2=0x7eff60107010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff66ab0700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff60103250 0x7eff60107010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff66ab0700 1 -- 172.21.15.195:0/3268951800 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff60106360 con 0x7eff60104d20 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff662af700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff60103250 0x7eff60107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff66ab0700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 0x7eff60105c00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff58003760 tx=0x7eff580047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff5f7fe700 1 -- 172.21.15.195:0/3268951800 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58004830 con 0x7eff60104d20 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601a48e0 con 0x7eff60104d20 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff5f7fe700 1 -- 172.21.15.195:0/3268951800 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff58003b90 con 0x7eff60104d20 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff5f7fe700 1 -- 172.21.15.195:0/3268951800 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58005280 con 0x7eff60104d20 2024-05-09T05:32:59.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.647+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601a4c80 con 0x7eff60104d20 2024-05-09T05:32:59.645 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.648+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff600512c0 con 0x7eff60104d20 2024-05-09T05:32:59.645 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.650+0000 7eff5f7fe700 1 -- 172.21.15.195:0/3268951800 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7eff58005420 con 0x7eff60104d20 2024-05-09T05:32:59.645 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.651+0000 7eff5f7fe700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7eff480b2880 0x7eff480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:32:59.645 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.651+0000 7eff5f7fe700 1 -- 172.21.15.195:0/3268951800 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7eff580e3850 con 0x7eff60104d20 2024-05-09T05:32:59.645 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.651+0000 7eff662af700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7eff480b2880 0x7eff480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:32:59.646 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.651+0000 7eff662af700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7eff480b2880 0x7eff480b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff60104280 tx=0x7eff50009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:32:59.648 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.654+0000 7eff5f7fe700 1 -- 172.21.15.195:0/3268951800 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff58030070 con 0x7eff60104d20 2024-05-09T05:32:59.862 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.867+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi115=a;smithi115=d;smithi115=g;smithi115=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}) -- 0x7eff601009f0 con 0x7eff480b2880 2024-05-09T05:32:59.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.876+0000 7eff5f7fe700 1 -- 172.21.15.195:0/3268951800 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7eff601009f0 con 0x7eff480b2880 2024-05-09T05:32:59.870 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mds.all update... 2024-05-09T05:32:59.872 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7eff480b2880 msgr2=0x7eff480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:59.872 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7eff480b2880 0x7eff480b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff60104280 tx=0x7eff50009040 comp rx=0 tx=0).stop 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 msgr2=0x7eff60105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 0x7eff60105c00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff58003760 tx=0x7eff580047b0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 shutdown_connections 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7eff480b2880 0x7eff480b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff60104d20 0x7eff60105c00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff60103250 0x7eff60107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 --2- 172.21.15.195:0/3268951800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff60104380 0x7eff601056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 >> 172.21.15.195:0/3268951800 conn(0x7eff600fe890 msgr2=0x7eff601001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 shutdown_connections 2024-05-09T05:32:59.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-05-09T05:32:59.878+0000 7eff68513700 1 -- 172.21.15.195:0/3268951800 wait complete. 2024-05-09T05:33:00.553 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:00 smithi195 ceph-mon[33844]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:00.553 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:00.553 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:00 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:33:00.579 DEBUG:teuthology.orchestra.run.smithi115:mds.a> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.a.service 2024-05-09T05:33:00.582 DEBUG:teuthology.orchestra.run.smithi115:mds.d> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.d.service 2024-05-09T05:33:00.607 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:00 smithi115 ceph-mon[30349]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:00.607 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:00.607 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:00 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:33:00.608 DEBUG:teuthology.orchestra.run.smithi115:mds.g> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.g.service 2024-05-09T05:33:00.651 DEBUG:teuthology.orchestra.run.smithi115:mds.j> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.j.service 2024-05-09T05:33:00.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:00 smithi161 ceph-mon[33805]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:00.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:00.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:00 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:33:00.694 DEBUG:teuthology.orchestra.run.smithi161:mds.b> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.b.service 2024-05-09T05:33:00.697 DEBUG:teuthology.orchestra.run.smithi161:mds.e> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.e.service 2024-05-09T05:33:00.741 DEBUG:teuthology.orchestra.run.smithi161:mds.h> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.h.service 2024-05-09T05:33:00.783 DEBUG:teuthology.orchestra.run.smithi161:mds.k> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.k.service 2024-05-09T05:33:00.826 DEBUG:teuthology.orchestra.run.smithi195:mds.c> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.c.service 2024-05-09T05:33:00.829 DEBUG:teuthology.orchestra.run.smithi195:mds.f> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.f.service 2024-05-09T05:33:00.872 DEBUG:teuthology.orchestra.run.smithi195:mds.i> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.i.service 2024-05-09T05:33:00.915 DEBUG:teuthology.orchestra.run.smithi195:mds.l> sudo journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service 2024-05-09T05:33:00.958 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T05:33:00.958 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:33:01.198 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:01.561 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:01 smithi115 ceph-mon[30349]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi115=a;smithi115=d;smithi115=g;smithi115=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:33:01.561 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:01 smithi115 ceph-mon[30349]: Saving service mds.all spec with placement smithi115=a;smithi115=d;smithi115=g;smithi115=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-05-09T05:33:01.561 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:01.561 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:33:01.561 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:01.562 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:01.562 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:01.562 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:01 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:01 smithi195 ceph-mon[33844]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi115=a;smithi115=d;smithi115=g;smithi115=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:33:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:01 smithi195 ceph-mon[33844]: Saving service mds.all spec with placement smithi115=a;smithi115=d;smithi115=g;smithi115=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-05-09T05:33:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:33:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:01 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:01 smithi161 ceph-mon[33805]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi115=a;smithi115=d;smithi115=g;smithi115=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:33:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:01 smithi161 ceph-mon[33805]: Saving service mds.all spec with placement smithi115=a;smithi115=d;smithi115=g;smithi115=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-05-09T05:33:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:33:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:01 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:02.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:02 smithi195 ceph-mon[33844]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:02.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:02 smithi195 ceph-mon[33844]: Deploying daemon mds.l on smithi195 2024-05-09T05:33:02.580 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:02 smithi115 ceph-mon[30349]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:02.580 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:02 smithi115 ceph-mon[30349]: Deploying daemon mds.l on smithi195 2024-05-09T05:33:02.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:02 smithi161 ceph-mon[33805]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:02 smithi161 ceph-mon[33805]: Deploying daemon mds.l on smithi195 2024-05-09T05:33:03.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 -- 172.21.15.115:0/3098741031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8898107df0 msgr2=0x7f8898108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:03.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 --2- 172.21.15.115:0/3098741031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8898107df0 0x7f8898108200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f88800099c0 tx=0x7f8880009cd0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 -- 172.21.15.115:0/3098741031 shutdown_connections 2024-05-09T05:33:03.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 --2- 172.21.15.115:0/3098741031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88981034c0 0x7f88981038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 --2- 172.21.15.115:0/3098741031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898108740 0x7f88981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 --2- 172.21.15.115:0/3098741031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8898107df0 0x7f8898108200 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 -- 172.21.15.115:0/3098741031 >> 172.21.15.115:0/3098741031 conn(0x7f88980fedc0 msgr2=0x7f88981011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:03.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 -- 172.21.15.115:0/3098741031 shutdown_connections 2024-05-09T05:33:03.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 -- 172.21.15.115:0/3098741031 wait complete. 2024-05-09T05:33:03.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.013+0000 7f889cf02700 1 Processor -- start 2024-05-09T05:33:03.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889cf02700 1 -- start start 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889cf02700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88981034c0 0x7f889819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889cf02700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8898107df0 0x7f889819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889cf02700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898108740 0x7f88981a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889cf02700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8898113200 con 0x7f88981034c0 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889cf02700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8898113080 con 0x7f8898107df0 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889cf02700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8898113380 con 0x7f8898108740 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88981034c0 0x7f889819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88981034c0 0x7f889819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52846/0 (socket says 172.21.15.115:52846) 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889659c700 1 -- 172.21.15.115:0/3326988911 learned_addr learned my addr 172.21.15.115:0/3326988911 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f8896d9d700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898108740 0x7f88981a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f8895d9b700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8898107df0 0x7f889819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889659c700 1 -- 172.21.15.115:0/3326988911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898108740 msgr2=0x7f88981a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889659c700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898108740 0x7f88981a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889659c700 1 -- 172.21.15.115:0/3326988911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8898107df0 msgr2=0x7f889819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889659c700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8898107df0 0x7f889819d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889659c700 1 -- 172.21.15.115:0/3326988911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88981a1d30 con 0x7f88981034c0 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f8895d9b700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8898107df0 0x7f889819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.014+0000 7f889659c700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88981034c0 0x7f889819cf10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8888007e20 tx=0x7f888800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.015+0000 7f888f7fe700 1 -- 172.21.15.115:0/3326988911 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888016020 con 0x7f88981034c0 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.015+0000 7f888f7fe700 1 -- 172.21.15.115:0/3326988911 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f888800ad40 con 0x7f88981034c0 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.015+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88981a2020 con 0x7f88981034c0 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.015+0000 7f888f7fe700 1 -- 172.21.15.115:0/3326988911 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88880115e0 con 0x7f88981034c0 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.015+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88981a2530 con 0x7f88981034c0 2024-05-09T05:33:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.016+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88980512c0 con 0x7f88981034c0 2024-05-09T05:33:03.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.017+0000 7f888f7fe700 1 -- 172.21.15.115:0/3326988911 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8888019020 con 0x7f88981034c0 2024-05-09T05:33:03.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.019+0000 7f888f7fe700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f88840b2880 0x7f88840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:03.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.019+0000 7f8895d9b700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f88840b2880 0x7f88840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:03.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.019+0000 7f888f7fe700 1 -- 172.21.15.115:0/3326988911 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f88880e1f90 con 0x7f88981034c0 2024-05-09T05:33:03.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.019+0000 7f8895d9b700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f88840b2880 0x7f88840b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8880000c00 tx=0x7f888001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:03.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.022+0000 7f888f7fe700 1 -- 172.21.15.115:0/3326988911 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8888020070 con 0x7f88981034c0 2024-05-09T05:33:03.105 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:33:02 smithi195 systemd[1]: Starting Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:33:03.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.273+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f889819e270 con 0x7f88981034c0 2024-05-09T05:33:03.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.273+0000 7f888f7fe700 1 -- 172.21.15.115:0/3326988911 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f88880aa0b0 con 0x7f88981034c0 2024-05-09T05:33:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:33:03.268 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-09T05:33:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f88840b2880 msgr2=0x7f88840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f88840b2880 0x7f88840b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8880000c00 tx=0x7f888001a040 comp rx=0 tx=0).stop 2024-05-09T05:33:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88981034c0 msgr2=0x7f889819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88981034c0 0x7f889819cf10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8888007e20 tx=0x7f888800a340 comp rx=0 tx=0).stop 2024-05-09T05:33:03.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 shutdown_connections 2024-05-09T05:33:03.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f88840b2880 0x7f88840b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8898108740 0x7f88981a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88981034c0 0x7f889819cf10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 --2- 172.21.15.115:0/3326988911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8898107df0 0x7f889819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:03.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 >> 172.21.15.115:0/3326988911 conn(0x7f88980fedc0 msgr2=0x7f8898109740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:03.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 shutdown_connections 2024-05-09T05:33:03.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:03.276+0000 7f889cf02700 1 -- 172.21.15.115:0/3326988911 wait complete. 2024-05-09T05:33:03.271 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 1 2024-05-09T05:33:03.364 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:03 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3326988911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:03.364 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:33:03 smithi195 podman[63243]: 2024-05-09 05:33:03.015432073 +0000 UTC m=+0.018347031 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:03.364 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:33:03 smithi195 podman[63243]: 2024-05-09 05:33:03.122364796 +0000 UTC m=+0.125279734 container create 4ab77f7ac162183406c6b4c1d71443f1dbae1d3db5818693acc6cd646cce81c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T05:33:03.364 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:33:03 smithi195 podman[63243]: 2024-05-09 05:33:03.334345009 +0000 UTC m=+0.337259941 container init 4ab77f7ac162183406c6b4c1d71443f1dbae1d3db5818693acc6cd646cce81c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:33:03.364 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:33:03 smithi195 podman[63243]: 2024-05-09 05:33:03.337438868 +0000 UTC m=+0.340353799 container start 4ab77f7ac162183406c6b4c1d71443f1dbae1d3db5818693acc6cd646cce81c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:33:03.365 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:33:03 smithi195 ceph-mds[63262]: -- 172.21.15.195:0/2868810396 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564797ddfa40 con 0x564797e27a80 2024-05-09T05:33:03.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:33:03 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: starting mds.l at 2024-05-09T05:33:03.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:33:03 smithi195 bash[63243]: 4ab77f7ac162183406c6b4c1d71443f1dbae1d3db5818693acc6cd646cce81c1 2024-05-09T05:33:03.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:33:03 smithi195 systemd[1]: Started Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:03.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:03 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3326988911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:03 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3326988911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:04.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:04.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6832/2784864553,v1:172.21.15.195:6833/2784864553] up:boot 2024-05-09T05:33:04.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: fsmap 1 up:standby 2024-05-09T05:33:04.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:33:04.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:04.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:04.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:04.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:04.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:04.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:04 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:04.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6832/2784864553,v1:172.21.15.195:6833/2784864553] up:boot 2024-05-09T05:33:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: fsmap 1 up:standby 2024-05-09T05:33:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:33:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:04.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:04 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:04.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:04.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6832/2784864553,v1:172.21.15.195:6833/2784864553] up:boot 2024-05-09T05:33:04.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: fsmap 1 up:standby 2024-05-09T05:33:04.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:33:04.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:04.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:04.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:04.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:04.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:04.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:04 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:05.528 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:05 smithi161 ceph-mon[33805]: Deploying daemon mds.k on smithi161 2024-05-09T05:33:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:05 smithi115 ceph-mon[30349]: Deploying daemon mds.k on smithi161 2024-05-09T05:33:05.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:05 smithi195 ceph-mon[33844]: Deploying daemon mds.k on smithi161 2024-05-09T05:33:06.037 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:33:06.161 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 05:33:06 smithi161 podman[62649]: 2024-05-09 05:33:06.031476119 +0000 UTC m=+0.017876940 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:06.291 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:06.444 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:06 smithi161 ceph-mon[33805]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:06.445 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 05:33:06 smithi161 podman[62649]: 2024-05-09 05:33:06.170457988 +0000 UTC m=+0.156858789 container create f22c6e34e52f68223b288f4d46afdb3a07c8603894cbc77f75e20719f4d31e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:33:06.445 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 05:33:06 smithi161 podman[62649]: 2024-05-09 05:33:06.414984637 +0000 UTC m=+0.401385449 container init f22c6e34e52f68223b288f4d46afdb3a07c8603894cbc77f75e20719f4d31e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-05-09T05:33:06.445 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 05:33:06 smithi161 podman[62649]: 2024-05-09 05:33:06.417964133 +0000 UTC m=+0.404364929 container start f22c6e34e52f68223b288f4d46afdb3a07c8603894cbc77f75e20719f4d31e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:33:06.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:06 smithi115 ceph-mon[30349]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:06.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:06 smithi195 ceph-mon[33844]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:06.910 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 05:33:06 smithi161 ceph-mds[62668]: -- 172.21.15.161:0/3371471513 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648df8e9a40 con 0x5648df931600 2024-05-09T05:33:06.910 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 05:33:06 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k[62664]: starting mds.k at 2024-05-09T05:33:06.911 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 05:33:06 smithi161 bash[62649]: f22c6e34e52f68223b288f4d46afdb3a07c8603894cbc77f75e20719f4d31e56 2024-05-09T05:33:06.911 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 05:33:06 smithi161 systemd[1]: Started Ceph mds.k for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:07.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:07.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:07.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:07.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:07.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:07.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:07.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: Deploying daemon mds.j on smithi115 2024-05-09T05:33:07.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:07.737 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.161:6832/634985009,v1:172.21.15.161:6833/634985009] up:boot 2024-05-09T05:33:07.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: fsmap 2 up:standby 2024-05-09T05:33:07.738 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:07 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: Deploying daemon mds.j on smithi115 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.161:6832/634985009,v1:172.21.15.161:6833/634985009] up:boot 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: fsmap 2 up:standby 2024-05-09T05:33:07.830 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:07 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T05:33:07.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:07.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:07.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:07.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:07.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:07.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:07.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: Deploying daemon mds.j on smithi115 2024-05-09T05:33:07.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:07.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.161:6832/634985009,v1:172.21.15.161:6833/634985009] up:boot 2024-05-09T05:33:07.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: fsmap 2 up:standby 2024-05-09T05:33:07.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:07 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T05:33:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:07.999+0000 7fbce67d0700 1 -- 172.21.15.115:0/1373757367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce01029c0 msgr2=0x7fbce0102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:07.999+0000 7fbce67d0700 1 --2- 172.21.15.115:0/1373757367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce01029c0 0x7fbce0102dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400aaa0 tx=0x7fbcd400adb0 comp rx=0 tx=0).stop 2024-05-09T05:33:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:07.999+0000 7fbce67d0700 1 -- 172.21.15.115:0/1373757367 shutdown_connections 2024-05-09T05:33:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:07.999+0000 7fbce67d0700 1 --2- 172.21.15.115:0/1373757367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbce0103310 0x7fbce010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:07.999+0000 7fbce67d0700 1 --2- 172.21.15.115:0/1373757367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbce01089f0 0x7fbce0108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:07.999+0000 7fbce67d0700 1 --2- 172.21.15.115:0/1373757367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce01029c0 0x7fbce0102dd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:07.999+0000 7fbce67d0700 1 -- 172.21.15.115:0/1373757367 >> 172.21.15.115:0/1373757367 conn(0x7fbce00fe520 msgr2=0x7fbce0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:07.999+0000 7fbce67d0700 1 -- 172.21.15.115:0/1373757367 shutdown_connections 2024-05-09T05:33:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:07.999+0000 7fbce67d0700 1 -- 172.21.15.115:0/1373757367 wait complete. 2024-05-09T05:33:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.000+0000 7fbce67d0700 1 Processor -- start 2024-05-09T05:33:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.000+0000 7fbce67d0700 1 -- start start 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbce67d0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbce01029c0 0x7fbce019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbce67d0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce0103310 0x7fbce019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbce67d0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbce01089f0 0x7fbce0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbce67d0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01121a0 con 0x7fbce0103310 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbce67d0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0112020 con 0x7fbce01089f0 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbce67d0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0112320 con 0x7fbce01029c0 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbcdf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce0103310 0x7fbce019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbcdf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce0103310 0x7fbce019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52874/0 (socket says 172.21.15.115:52874) 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbcdf7fe700 1 -- 172.21.15.115:0/2955374675 learned_addr learned my addr 172.21.15.115:0/2955374675 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbce4d6d700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbce01089f0 0x7fbce0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbcdffff700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbce01029c0 0x7fbce019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.001+0000 7fbcdf7fe700 1 -- 172.21.15.115:0/2955374675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbce01029c0 msgr2=0x7fbce019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbcdf7fe700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbce01029c0 0x7fbce019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbcdf7fe700 1 -- 172.21.15.115:0/2955374675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbce01089f0 msgr2=0x7fbce0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbcdf7fe700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbce01089f0 0x7fbce0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbcdf7fe700 1 -- 172.21.15.115:0/2955374675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce01990d0 con 0x7fbce0103310 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbcdf7fe700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce0103310 0x7fbce019ecf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd4003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbcdd7fa700 1 -- 172.21.15.115:0/2955374675 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd401b040 con 0x7fbce0103310 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbcdd7fa700 1 -- 172.21.15.115:0/2955374675 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcd400ccf0 con 0x7fbce0103310 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce0199360 con 0x7fbce0103310 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbcdd7fa700 1 -- 172.21.15.115:0/2955374675 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40054f0 con 0x7fbce0103310 2024-05-09T05:33:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.002+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce0199840 con 0x7fbce0103310 2024-05-09T05:33:07.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.003+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce00512c0 con 0x7fbce0103310 2024-05-09T05:33:07.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.005+0000 7fbcdd7fa700 1 -- 172.21.15.115:0/2955374675 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fbcd4004800 con 0x7fbce0103310 2024-05-09T05:33:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.005+0000 7fbcdd7fa700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcc80b2880 0x7fbcc80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.006+0000 7fbcdffff700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcc80b2880 0x7fbcc80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.006+0000 7fbcdd7fa700 1 -- 172.21.15.115:0/2955374675 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fbcd40e5290 con 0x7fbce0103310 2024-05-09T05:33:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.006+0000 7fbcdffff700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcc80b2880 0x7fbcc80b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbce0106590 tx=0x7fbcd000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:08.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.009+0000 7fbcdd7fa700 1 -- 172.21.15.115:0/2955374675 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcd40ad4e0 con 0x7fbce0103310 2024-05-09T05:33:08.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.263+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbce002e5f0 con 0x7fbce0103310 2024-05-09T05:33:08.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.264+0000 7fbcdd7fa700 1 -- 172.21.15.115:0/2955374675 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fbcd40acde0 con 0x7fbce0103310 2024-05-09T05:33:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:33:08.258 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcc80b2880 msgr2=0x7fbcc80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcc80b2880 0x7fbcc80b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbce0106590 tx=0x7fbcd000b410 comp rx=0 tx=0).stop 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce0103310 msgr2=0x7fbce019ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce0103310 0x7fbce019ecf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd4003930 comp rx=0 tx=0).stop 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 shutdown_connections 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcc80b2880 0x7fbcc80b4d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbce01089f0 0x7fbce0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbce01029c0 0x7fbce019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 --2- 172.21.15.115:0/2955374675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbce0103310 0x7fbce019ecf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 >> 172.21.15.115:0/2955374675 conn(0x7fbce00fe520 msgr2=0x7fbce0100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 shutdown_connections 2024-05-09T05:33:08.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:08.266+0000 7fbce67d0700 1 -- 172.21.15.115:0/2955374675 wait complete. 2024-05-09T05:33:08.262 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 3 2024-05-09T05:33:08.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:08 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2955374675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:08.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:08 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2955374675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:08.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:08 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2955374675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:09.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:09 smithi115 ceph-mon[30349]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:09.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:09 smithi195 ceph-mon[33844]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:09.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:09 smithi161 ceph-mon[33805]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:11.146 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 05:33:10 smithi115 systemd[1]: Started Ceph mds.j for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: Deploying daemon mds.d on smithi115 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.115:6834/1419658869,v1:172.21.15.115:6835/1419658869] up:boot 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: fsmap 3 up:standby 2024-05-09T05:33:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:11 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: Deploying daemon mds.d on smithi115 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.115:6834/1419658869,v1:172.21.15.115:6835/1419658869] up:boot 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: fsmap 3 up:standby 2024-05-09T05:33:12.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:11 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T05:33:12.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:12.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:12.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:12.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:12.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:12.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:12.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:12.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: Deploying daemon mds.d on smithi115 2024-05-09T05:33:12.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.115:6834/1419658869,v1:172.21.15.115:6835/1419658869] up:boot 2024-05-09T05:33:12.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: fsmap 3 up:standby 2024-05-09T05:33:12.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:11 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T05:33:12.281 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:33:12.563 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:14.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:13 smithi195 ceph-mon[33844]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:14.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:13 smithi115 ceph-mon[30349]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:14.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:13 smithi161 ceph-mon[33805]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.393+0000 7ff2c35fa700 1 -- 172.21.15.115:0/4117366017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 msgr2=0x7ff2bc103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.393+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/4117366017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 0x7ff2bc103660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009990 tx=0x7ff2b8009ca0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.393+0000 7ff2c35fa700 1 -- 172.21.15.115:0/4117366017 shutdown_connections 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.393+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/4117366017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2bc1086a0 0x7ff2bc108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.393+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/4117366017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 0x7ff2bc103660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.393+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/4117366017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2bc103ba0 0x7ff2bc1080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.393+0000 7ff2c35fa700 1 -- 172.21.15.115:0/4117366017 >> 172.21.15.115:0/4117366017 conn(0x7ff2bc0fedf0 msgr2=0x7ff2bc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.393+0000 7ff2c35fa700 1 -- 172.21.15.115:0/4117366017 shutdown_connections 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.393+0000 7ff2c35fa700 1 -- 172.21.15.115:0/4117366017 wait complete. 2024-05-09T05:33:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c35fa700 1 Processor -- start 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c35fa700 1 -- start start 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c35fa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 0x7ff2bc1a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c35fa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2bc103ba0 0x7ff2bc1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c35fa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2bc1086a0 0x7ff2bc1a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c35fa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc11a940 con 0x7ff2bc103250 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c35fa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc11a7c0 con 0x7ff2bc1086a0 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c35fa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc11aac0 con 0x7ff2bc103ba0 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c1396700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 0x7ff2bc1a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c1396700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 0x7ff2bc1a1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50544/0 (socket says 172.21.15.115:50544) 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c1396700 1 -- 172.21.15.115:0/3346868539 learned_addr learned my addr 172.21.15.115:0/3346868539 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c0b95700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2bc103ba0 0x7ff2bc1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.394+0000 7ff2c1b97700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2bc1086a0 0x7ff2bc1a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2c1396700 1 -- 172.21.15.115:0/3346868539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2bc103ba0 msgr2=0x7ff2bc1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:14.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2c1396700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2bc103ba0 0x7ff2bc1a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2c1396700 1 -- 172.21.15.115:0/3346868539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2bc1086a0 msgr2=0x7ff2bc1a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2c1396700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2bc1086a0 0x7ff2bc1a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2c1396700 1 -- 172.21.15.115:0/3346868539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc1a61b0 con 0x7ff2bc103250 2024-05-09T05:33:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2c1396700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 0x7ff2bc1a1390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8000c00 tx=0x7ff2b80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2b27fc700 1 -- 172.21.15.115:0/3346868539 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8027070 con 0x7ff2bc103250 2024-05-09T05:33:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2b27fc700 1 -- 172.21.15.115:0/3346868539 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2b800fb80 con 0x7ff2bc103250 2024-05-09T05:33:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc1a6440 con 0x7ff2bc103250 2024-05-09T05:33:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2b27fc700 1 -- 172.21.15.115:0/3346868539 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8022d00 con 0x7ff2bc103250 2024-05-09T05:33:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.395+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc1a6920 con 0x7ff2bc103250 2024-05-09T05:33:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.396+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2bc0512c0 con 0x7ff2bc103250 2024-05-09T05:33:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.398+0000 7ff2b27fc700 1 -- 172.21.15.115:0/3346868539 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff2b802a070 con 0x7ff2bc103250 2024-05-09T05:33:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.398+0000 7ff2b27fc700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff2a80b2880 0x7ff2a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.398+0000 7ff2c0b95700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff2a80b2880 0x7ff2a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.398+0000 7ff2b27fc700 1 -- 172.21.15.115:0/3346868539 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff2b80e3060 con 0x7ff2bc103250 2024-05-09T05:33:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.399+0000 7ff2c0b95700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff2a80b2880 0x7ff2a80b4d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc101880 tx=0x7ff2ac00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:14.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.402+0000 7ff2b27fc700 1 -- 172.21.15.115:0/3346868539 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2b8031080 con 0x7ff2bc103250 2024-05-09T05:33:14.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.658+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2bc1ade70 con 0x7ff2bc103250 2024-05-09T05:33:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.658+0000 7ff2b27fc700 1 -- 172.21.15.115:0/3346868539 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7ff2b8013070 con 0x7ff2bc103250 2024-05-09T05:33:14.653 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:33:14.653 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T05:33:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff2a80b2880 msgr2=0x7ff2a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff2a80b2880 0x7ff2a80b4d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc101880 tx=0x7ff2ac00a380 comp rx=0 tx=0).stop 2024-05-09T05:33:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 msgr2=0x7ff2bc1a1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 0x7ff2bc1a1390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8000c00 tx=0x7ff2b80047b0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 shutdown_connections 2024-05-09T05:33:14.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff2a80b2880 0x7ff2a80b4d90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2bc1086a0 0x7ff2bc1a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2bc103250 0x7ff2bc1a1390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 --2- 172.21.15.115:0/3346868539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2bc103ba0 0x7ff2bc1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:14.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 >> 172.21.15.115:0/3346868539 conn(0x7ff2bc0fedf0 msgr2=0x7ff2bc106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:14.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 shutdown_connections 2024-05-09T05:33:14.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:14.661+0000 7ff2c35fa700 1 -- 172.21.15.115:0/3346868539 wait complete. 2024-05-09T05:33:14.669 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 4 2024-05-09T05:33:15.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:14 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3346868539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:15.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:14 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3346868539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:15.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:14 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3346868539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:16.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:16.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.115:6836/2746290424,v1:172.21.15.115:6837/2746290424] up:boot 2024-05-09T05:33:16.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: fsmap 4 up:standby 2024-05-09T05:33:16.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T05:33:16.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:16.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:15 smithi195 ceph-mon[33844]: Deploying daemon mds.e on smithi161 2024-05-09T05:33:16.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.115:6836/2746290424,v1:172.21.15.115:6837/2746290424] up:boot 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: fsmap 4 up:standby 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:16.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:15 smithi115 ceph-mon[30349]: Deploying daemon mds.e on smithi161 2024-05-09T05:33:16.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.115:6836/2746290424,v1:172.21.15.115:6837/2746290424] up:boot 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: fsmap 4 up:standby 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:16.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:15 smithi161 ceph-mon[33805]: Deploying daemon mds.e on smithi161 2024-05-09T05:33:17.411 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 05:33:17 smithi161 systemd[1]: Starting Ceph mds.e for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:33:17.520 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:33:17.722 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 05:33:17 smithi161 podman[62969]: 2024-05-09 05:33:17.355522781 +0000 UTC m=+0.018244991 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:17.722 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 05:33:17 smithi161 podman[62969]: 2024-05-09 05:33:17.476349555 +0000 UTC m=+0.139071745 container create 5bd2dbf594bf8c0e7d7c603573ac88e8f2620fcd4de454af26d114d9aeae8973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-e, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T05:33:17.722 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 05:33:17 smithi161 podman[62969]: 2024-05-09 05:33:17.729399508 +0000 UTC m=+0.392121699 container init 5bd2dbf594bf8c0e7d7c603573ac88e8f2620fcd4de454af26d114d9aeae8973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T05:33:17.775 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:18.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:17 smithi195 ceph-mon[33844]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:18.130 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:17 smithi115 ceph-mon[30349]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:18.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:17 smithi161 ceph-mon[33805]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:18.161 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 05:33:17 smithi161 podman[62969]: 2024-05-09 05:33:17.732144462 +0000 UTC m=+0.394866654 container start 5bd2dbf594bf8c0e7d7c603573ac88e8f2620fcd4de454af26d114d9aeae8973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T05:33:18.161 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 05:33:17 smithi161 ceph-mds[62988]: -- 172.21.15.161:0/498896590 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5584a0981a40 con 0x5584a09c9a80 2024-05-09T05:33:18.161 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 05:33:17 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-e[62984]: starting mds.e at 2024-05-09T05:33:18.161 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 05:33:17 smithi161 bash[62969]: 5bd2dbf594bf8c0e7d7c603573ac88e8f2620fcd4de454af26d114d9aeae8973 2024-05-09T05:33:18.161 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 05:33:17 smithi161 systemd[1]: Started Ceph mds.e for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:19.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.161:6834/1779202002,v1:172.21.15.161:6835/1779202002] up:boot 2024-05-09T05:33:19.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: fsmap 5 up:standby 2024-05-09T05:33:19.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T05:33:19.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:19.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:19.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:19.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:19.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:19.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:19.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:18 smithi195 ceph-mon[33844]: Deploying daemon mds.f on smithi195 2024-05-09T05:33:19.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.161:6834/1779202002,v1:172.21.15.161:6835/1779202002] up:boot 2024-05-09T05:33:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: fsmap 5 up:standby 2024-05-09T05:33:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T05:33:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:19.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:18 smithi115 ceph-mon[30349]: Deploying daemon mds.f on smithi195 2024-05-09T05:33:19.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.161:6834/1779202002,v1:172.21.15.161:6835/1779202002] up:boot 2024-05-09T05:33:19.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: fsmap 5 up:standby 2024-05-09T05:33:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T05:33:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:19.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:18 smithi161 ceph-mon[33805]: Deploying daemon mds.f on smithi195 2024-05-09T05:33:19.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.522+0000 7f0ac1d33700 1 -- 172.21.15.115:0/3021628955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 msgr2=0x7f0abc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:19.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.522+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/3021628955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 0x7f0abc108200 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00099c0 tx=0x7f0ab0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.522+0000 7f0ac1d33700 1 -- 172.21.15.115:0/3021628955 shutdown_connections 2024-05-09T05:33:19.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.522+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/3021628955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0abc108740 0x7f0abc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.522+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/3021628955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0abc1034c0 0x7f0abc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.522+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/3021628955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 0x7f0abc108200 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.522+0000 7f0ac1d33700 1 -- 172.21.15.115:0/3021628955 >> 172.21.15.115:0/3021628955 conn(0x7f0abc0fedc0 msgr2=0x7f0abc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:19.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.522+0000 7f0ac1d33700 1 -- 172.21.15.115:0/3021628955 shutdown_connections 2024-05-09T05:33:19.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.523+0000 7f0ac1d33700 1 -- 172.21.15.115:0/3021628955 wait complete. 2024-05-09T05:33:19.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.523+0000 7f0ac1d33700 1 Processor -- start 2024-05-09T05:33:19.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.523+0000 7f0ac1d33700 1 -- start start 2024-05-09T05:33:19.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0ac1d33700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0abc1034c0 0x7f0abc19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:19.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0ac1d33700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 0x7f0abc19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0ac1d33700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0abc108740 0x7f0abc19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0ac1d33700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc113200 con 0x7f0abc107df0 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0ac1d33700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc113080 con 0x7f0abc1034c0 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0ac1d33700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc113380 con 0x7f0abc108740 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 0x7f0abc19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 0x7f0abc19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50574/0 (socket says 172.21.15.115:50574) 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abaffd700 1 -- 172.21.15.115:0/533190284 learned_addr learned my addr 172.21.15.115:0/533190284 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abb7fe700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0abc1034c0 0x7f0abc19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abbfff700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0abc108740 0x7f0abc19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abaffd700 1 -- 172.21.15.115:0/533190284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0abc108740 msgr2=0x7f0abc19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:19.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abaffd700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0abc108740 0x7f0abc19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abaffd700 1 -- 172.21.15.115:0/533190284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0abc1034c0 msgr2=0x7f0abc19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abaffd700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0abc1034c0 0x7f0abc19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abaffd700 1 -- 172.21.15.115:0/533190284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc19fb60 con 0x7f0abc107df0 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abb7fe700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0abc1034c0 0x7f0abc19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0abaffd700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 0x7f0abc19b2e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000b580 tx=0x7f0ab000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.524+0000 7f0ab8ff9700 1 -- 172.21.15.115:0/533190284 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0026070 con 0x7f0abc107df0 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.525+0000 7f0ab8ff9700 1 -- 172.21.15.115:0/533190284 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ab0005030 con 0x7f0abc107df0 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.525+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc19fdf0 con 0x7f0abc107df0 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.525+0000 7f0ab8ff9700 1 -- 172.21.15.115:0/533190284 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00049f0 con 0x7f0abc107df0 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.525+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc1a02d0 con 0x7f0abc107df0 2024-05-09T05:33:19.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.525+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0abc0512c0 con 0x7f0abc107df0 2024-05-09T05:33:19.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.527+0000 7f0ab8ff9700 1 -- 172.21.15.115:0/533190284 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0ab0029070 con 0x7f0abc107df0 2024-05-09T05:33:19.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.528+0000 7f0ab8ff9700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0aa40b2880 0x7f0aa40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:19.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.528+0000 7f0abb7fe700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0aa40b2880 0x7f0aa40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:19.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.528+0000 7f0ab8ff9700 1 -- 172.21.15.115:0/533190284 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f0ab00e2ed0 con 0x7f0abc107df0 2024-05-09T05:33:19.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.528+0000 7f0abb7fe700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0aa40b2880 0x7f0aa40b4d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009dd0 tx=0x7f0aac009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:19.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.531+0000 7f0ab8ff9700 1 -- 172.21.15.115:0/533190284 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ab0013070 con 0x7f0abc107df0 2024-05-09T05:33:19.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.780+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0abc02e5f0 con 0x7f0abc107df0 2024-05-09T05:33:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.780+0000 7f0ab8ff9700 1 -- 172.21.15.115:0/533190284 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f0ab00ab120 con 0x7f0abc107df0 2024-05-09T05:33:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:33:19.775 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T05:33:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0aa40b2880 msgr2=0x7f0aa40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0aa40b2880 0x7f0aa40b4d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009dd0 tx=0x7f0aac009040 comp rx=0 tx=0).stop 2024-05-09T05:33:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 msgr2=0x7f0abc19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 0x7f0abc19b2e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000b580 tx=0x7f0ab000f510 comp rx=0 tx=0).stop 2024-05-09T05:33:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 shutdown_connections 2024-05-09T05:33:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0aa40b2880 0x7f0aa40b4d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0abc1034c0 0x7f0abc19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0abc108740 0x7f0abc19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 --2- 172.21.15.115:0/533190284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0abc107df0 0x7f0abc19b2e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 >> 172.21.15.115:0/533190284 conn(0x7f0abc0fedc0 msgr2=0x7f0abc06ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 shutdown_connections 2024-05-09T05:33:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:19.783+0000 7f0ac1d33700 1 -- 172.21.15.115:0/533190284 wait complete. 2024-05-09T05:33:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 6 2024-05-09T05:33:20.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:19 smithi195 ceph-mon[33844]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:20.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:19 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/533190284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:20.116 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:33:20 smithi195 systemd[1]: Starting Ceph mds.f for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:33:20.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:19 smithi115 ceph-mon[30349]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:20.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:19 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/533190284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:20.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:19 smithi161 ceph-mon[33805]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:20.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:19 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/533190284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:20.503 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:33:20 smithi195 podman[63569]: 2024-05-09 05:33:20.147605943 +0000 UTC m=+0.018685806 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:20.503 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:33:20 smithi195 podman[63569]: 2024-05-09 05:33:20.257170053 +0000 UTC m=+0.128249919 container create 993232dbb6ea64f3487ca8248c873f75db3aa8927d4dcc89f1c7872d3f86507e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:33:20.503 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:33:20 smithi195 podman[63569]: 2024-05-09 05:33:20.510026062 +0000 UTC m=+0.381105925 container init 993232dbb6ea64f3487ca8248c873f75db3aa8927d4dcc89f1c7872d3f86507e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507) 2024-05-09T05:33:20.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:33:20 smithi195 podman[63569]: 2024-05-09 05:33:20.512777784 +0000 UTC m=+0.383857654 container start 993232dbb6ea64f3487ca8248c873f75db3aa8927d4dcc89f1c7872d3f86507e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-05-09T05:33:20.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:33:20 smithi195 ceph-mds[63588]: -- 172.21.15.195:0/2050098171 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9729bba40 con 0x55d972a03180 2024-05-09T05:33:20.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:33:20 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: starting mds.f at 2024-05-09T05:33:20.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:33:20 smithi195 bash[63569]: 993232dbb6ea64f3487ca8248c873f75db3aa8927d4dcc89f1c7872d3f86507e 2024-05-09T05:33:20.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:33:20 smithi195 systemd[1]: Started Ceph mds.f for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:21.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: Deploying daemon mds.c on smithi195 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] up:boot 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: fsmap 6 up:standby 2024-05-09T05:33:21.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:21 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T05:33:22.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:22.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:22.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:22.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:22.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:22.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:22.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:22.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: Deploying daemon mds.c on smithi195 2024-05-09T05:33:22.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] up:boot 2024-05-09T05:33:22.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: fsmap 6 up:standby 2024-05-09T05:33:22.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:21 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: Deploying daemon mds.c on smithi195 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] up:boot 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: fsmap 6 up:standby 2024-05-09T05:33:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:21 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T05:33:22.498 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:33:22.753 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:22.866 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 05:33:22 smithi195 systemd[1]: Starting Ceph mds.c for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:33:23.300 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 05:33:23 smithi195 podman[63879]: 2024-05-09 05:33:22.917397279 +0000 UTC m=+0.018608401 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:23.300 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 05:33:23 smithi195 podman[63879]: 2024-05-09 05:33:23.046059288 +0000 UTC m=+0.147270395 container create c411b27886fdb37ef89b542d831a0e7920ca08c93af85616e7e965ec2e568081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-c, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:33:23.616 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 05:33:23 smithi195 podman[63879]: 2024-05-09 05:33:23.307272888 +0000 UTC m=+0.408483997 container init c411b27886fdb37ef89b542d831a0e7920ca08c93af85616e7e965ec2e568081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-09T05:33:23.616 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 05:33:23 smithi195 podman[63879]: 2024-05-09 05:33:23.310414422 +0000 UTC m=+0.411625534 container start c411b27886fdb37ef89b542d831a0e7920ca08c93af85616e7e965ec2e568081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T05:33:23.616 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 05:33:23 smithi195 ceph-mds[63898]: -- 172.21.15.195:0/388685365 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d236f1a40 con 0x562d23739600 2024-05-09T05:33:23.616 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 05:33:23 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-c[63894]: starting mds.c at 2024-05-09T05:33:23.616 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 05:33:23 smithi195 bash[63879]: c411b27886fdb37ef89b542d831a0e7920ca08c93af85616e7e965ec2e568081 2024-05-09T05:33:23.616 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 05:33:23 smithi195 systemd[1]: Started Ceph mds.c for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:24.040 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:23 smithi115 ceph-mon[30349]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:24.041 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:24.041 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:24.041 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:24.041 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:24.041 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:24.041 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:23 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:24.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:23 smithi195 ceph-mon[33844]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:24.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:24.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:24.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:24.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:24.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:24.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:23 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:24.151 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:23 smithi161 ceph-mon[33805]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:24.151 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:24.152 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:24.152 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:24.152 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:24.152 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:24.152 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:23 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:24.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 -- 172.21.15.115:0/627423035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 msgr2=0x7f9c001111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:24.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 --2- 172.21.15.115:0/627423035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 0x7f9c001111b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800b0d0 tx=0x7f9bf800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 -- 172.21.15.115:0/627423035 shutdown_connections 2024-05-09T05:33:24.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 --2- 172.21.15.115:0/627423035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c001045d0 0x7f9c001049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 --2- 172.21.15.115:0/627423035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 0x7f9c001111b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 --2- 172.21.15.115:0/627423035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c00108f00 0x7f9c00109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 -- 172.21.15.115:0/627423035 >> 172.21.15.115:0/627423035 conn(0x7f9c000ffea0 msgr2=0x7f9c001022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:24.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 -- 172.21.15.115:0/627423035 shutdown_connections 2024-05-09T05:33:24.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 -- 172.21.15.115:0/627423035 wait complete. 2024-05-09T05:33:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 Processor -- start 2024-05-09T05:33:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.434+0000 7f9c067cf700 1 -- start start 2024-05-09T05:33:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c067cf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c001045d0 0x7f9c0019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c067cf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c00108f00 0x7f9c0019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c067cf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 0x7f9c0019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c067cf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00114380 con 0x7f9c00109850 2024-05-09T05:33:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c067cf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00114200 con 0x7f9c00108f00 2024-05-09T05:33:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c067cf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00114500 con 0x7f9c001045d0 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c04d6c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 0x7f9c0019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c04d6c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 0x7f9c0019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34840/0 (socket says 172.21.15.115:34840) 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c04d6c700 1 -- 172.21.15.115:0/2008511609 learned_addr learned my addr 172.21.15.115:0/2008511609 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9bff7fe700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c00108f00 0x7f9c0019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9bfffff700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c001045d0 0x7f9c0019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c04d6c700 1 -- 172.21.15.115:0/2008511609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c001045d0 msgr2=0x7f9c0019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c04d6c700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c001045d0 0x7f9c0019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c04d6c700 1 -- 172.21.15.115:0/2008511609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c00108f00 msgr2=0x7f9c0019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c04d6c700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c00108f00 0x7f9c0019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c04d6c700 1 -- 172.21.15.115:0/2008511609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0019fc40 con 0x7f9c00109850 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9bfffff700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c001045d0 0x7f9c0019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9bff7fe700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c00108f00 0x7f9c0019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:33:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.435+0000 7f9c04d6c700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 0x7f9c0019f5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8007d10 tx=0x7f9bf8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:24.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.436+0000 7f9bfd7fa700 1 -- 172.21.15.115:0/2008511609 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf801f070 con 0x7f9c00109850 2024-05-09T05:33:24.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.436+0000 7f9bfd7fa700 1 -- 172.21.15.115:0/2008511609 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bf8003e10 con 0x7f9c00109850 2024-05-09T05:33:24.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.436+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0019fed0 con 0x7f9c00109850 2024-05-09T05:33:24.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.436+0000 7f9bfd7fa700 1 -- 172.21.15.115:0/2008511609 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf801b820 con 0x7f9c00109850 2024-05-09T05:33:24.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.436+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c001a7850 con 0x7f9c00109850 2024-05-09T05:33:24.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.437+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c00105a80 con 0x7f9c00109850 2024-05-09T05:33:24.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.438+0000 7f9bfd7fa700 1 -- 172.21.15.115:0/2008511609 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9bf8022070 con 0x7f9c00109850 2024-05-09T05:33:24.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.439+0000 7f9bfd7fa700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9be80b2900 0x7f9be80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:24.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.439+0000 7f9bfffff700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9be80b2900 0x7f9be80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:24.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.439+0000 7f9bfd7fa700 1 -- 172.21.15.115:0/2008511609 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f9bf800f020 con 0x7f9c00109850 2024-05-09T05:33:24.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.439+0000 7f9bfffff700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9be80b2900 0x7f9be80b4e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005fd0 tx=0x7f9bf0005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:24.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.443+0000 7f9bfd7fa700 1 -- 172.21.15.115:0/2008511609 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9bf8029020 con 0x7f9c00109850 2024-05-09T05:33:24.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.698+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c0019c0d0 con 0x7f9c00109850 2024-05-09T05:33:24.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.699+0000 7f9bfd7fa700 1 -- 172.21.15.115:0/2008511609 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f9bf80ab8f0 con 0x7f9c00109850 2024-05-09T05:33:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:33:24.693 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T05:33:24.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9be80b2900 msgr2=0x7f9be80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:24.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9be80b2900 0x7f9be80b4e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005fd0 tx=0x7f9bf0005ee0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 msgr2=0x7f9c0019f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 0x7f9c0019f5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8007d10 tx=0x7f9bf8007df0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 shutdown_connections 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9be80b2900 0x7f9be80b4e10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c00109850 0x7f9c0019f5a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c001045d0 0x7f9c0019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 --2- 172.21.15.115:0/2008511609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c00108f00 0x7f9c0019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 >> 172.21.15.115:0/2008511609 conn(0x7f9c000ffea0 msgr2=0x7f9c001086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 shutdown_connections 2024-05-09T05:33:24.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:24.701+0000 7f9c067cf700 1 -- 172.21.15.115:0/2008511609 wait complete. 2024-05-09T05:33:24.697 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 8 2024-05-09T05:33:24.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:24 smithi161 ceph-mon[33805]: Deploying daemon mds.h on smithi161 2024-05-09T05:33:24.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:24 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] up:boot 2024-05-09T05:33:24.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:24 smithi161 ceph-mon[33805]: fsmap 7 up:standby 2024-05-09T05:33:24.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:24 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T05:33:25.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:24 smithi195 ceph-mon[33844]: Deploying daemon mds.h on smithi161 2024-05-09T05:33:25.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:24 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] up:boot 2024-05-09T05:33:25.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:24 smithi195 ceph-mon[33844]: fsmap 7 up:standby 2024-05-09T05:33:25.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:24 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T05:33:25.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:24 smithi115 ceph-mon[30349]: Deploying daemon mds.h on smithi161 2024-05-09T05:33:25.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:24 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] up:boot 2024-05-09T05:33:25.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:24 smithi115 ceph-mon[30349]: fsmap 7 up:standby 2024-05-09T05:33:25.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:24 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T05:33:25.779 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:25 smithi115 ceph-mon[30349]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:25.779 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:25 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2008511609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:25.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:25 smithi161 ceph-mon[33805]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:25.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2008511609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:25.911 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 05:33:25 smithi161 systemd[1]: Starting Ceph mds.h for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:33:26.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:25 smithi195 ceph-mon[33844]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:26.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:25 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2008511609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:26.270 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 05:33:26 smithi161 podman[63277]: 2024-05-09 05:33:25.903671034 +0000 UTC m=+0.018097770 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:26.270 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 05:33:26 smithi161 podman[63277]: 2024-05-09 05:33:26.015718403 +0000 UTC m=+0.130145182 container create 460ce5a078fce9e23c7db156c89f5504c72bc608abbbc9bc349347f08dd285ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:33:26.270 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 05:33:26 smithi161 podman[63277]: 2024-05-09 05:33:26.277017633 +0000 UTC m=+0.391444367 container init 460ce5a078fce9e23c7db156c89f5504c72bc608abbbc9bc349347f08dd285ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-h, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T05:33:26.660 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 05:33:26 smithi161 podman[63277]: 2024-05-09 05:33:26.279854402 +0000 UTC m=+0.394281136 container start 460ce5a078fce9e23c7db156c89f5504c72bc608abbbc9bc349347f08dd285ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T05:33:26.660 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 05:33:26 smithi161 ceph-mds[63296]: -- 172.21.15.161:0/1914832610 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df0863da40 con 0x55df08685180 2024-05-09T05:33:26.661 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 05:33:26 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-h[63292]: starting mds.h at 2024-05-09T05:33:26.661 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 05:33:26 smithi161 bash[63277]: 460ce5a078fce9e23c7db156c89f5504c72bc608abbbc9bc349347f08dd285ea 2024-05-09T05:33:26.661 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 05:33:26 smithi161 systemd[1]: Started Ceph mds.h for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:27.510 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: Deploying daemon mds.i on smithi195 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] up:boot 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: fsmap 8 up:standby 2024-05-09T05:33:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:27 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T05:33:27.757 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:27.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:27.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:27.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:27.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:27.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:27.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:27.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: Deploying daemon mds.i on smithi195 2024-05-09T05:33:27.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] up:boot 2024-05-09T05:33:27.880 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: fsmap 8 up:standby 2024-05-09T05:33:27.881 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:27 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T05:33:27.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:27.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:27.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:27.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:27.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:27.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:27.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: Deploying daemon mds.i on smithi195 2024-05-09T05:33:27.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] up:boot 2024-05-09T05:33:27.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: fsmap 8 up:standby 2024-05-09T05:33:27.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:27 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T05:33:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:28 smithi115 ceph-mon[30349]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:28.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:28 smithi195 ceph-mon[33844]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:28.839 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:28 smithi195 systemd[1]: Starting Ceph mds.i for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:33:28.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:28 smithi161 ceph-mon[33805]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:29.116 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:28 smithi195 podman[64191]: 2024-05-09 05:33:28.747976093 +0000 UTC m=+0.018067627 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:29.116 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:28 smithi195 podman[64191]: 2024-05-09 05:33:28.860377348 +0000 UTC m=+0.130468858 container create c7a2a7b858744d3b3ef27a253e48f1b24e5117ae4e8dd22a60df79ce1e0e6dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-05-09T05:33:29.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.572+0000 7f8e0e256700 1 -- 172.21.15.115:0/4045130380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08103310 msgr2=0x7f8e0810f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:29.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.572+0000 7f8e0e256700 1 --2- 172.21.15.115:0/4045130380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08103310 0x7f8e0810f890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000b0a0 tx=0x7f8e0000b3b0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.572+0000 7f8e0e256700 1 -- 172.21.15.115:0/4045130380 shutdown_connections 2024-05-09T05:33:29.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.572+0000 7f8e0e256700 1 --2- 172.21.15.115:0/4045130380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e081089f0 0x7f8e08108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.572+0000 7f8e0e256700 1 --2- 172.21.15.115:0/4045130380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e08103310 0x7f8e0810f890 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.572+0000 7f8e0e256700 1 --2- 172.21.15.115:0/4045130380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e081029c0 0x7f8e08102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.572+0000 7f8e0e256700 1 -- 172.21.15.115:0/4045130380 >> 172.21.15.115:0/4045130380 conn(0x7f8e080fe6f0 msgr2=0x7f8e08100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:29.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.572+0000 7f8e0e256700 1 -- 172.21.15.115:0/4045130380 shutdown_connections 2024-05-09T05:33:29.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.572+0000 7f8e0e256700 1 -- 172.21.15.115:0/4045130380 wait complete. 2024-05-09T05:33:29.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.573+0000 7f8e0e256700 1 Processor -- start 2024-05-09T05:33:29.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.573+0000 7f8e0e256700 1 -- start start 2024-05-09T05:33:29.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e0e256700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e081029c0 0x7f8e0819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:29.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e0e256700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e08103310 0x7f8e0819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:29.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e0e256700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e081089f0 0x7f8e0819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:29.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e0e256700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08113250 con 0x7f8e081029c0 2024-05-09T05:33:29.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e0e256700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081130d0 con 0x7f8e08103310 2024-05-09T05:33:29.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e0e256700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081133d0 con 0x7f8e081089f0 2024-05-09T05:33:29.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e077fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e081029c0 0x7f8e0819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:29.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e077fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e081029c0 0x7f8e0819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34864/0 (socket says 172.21.15.115:34864) 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e077fe700 1 -- 172.21.15.115:0/23905873 learned_addr learned my addr 172.21.15.115:0/23905873 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e07fff700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e081089f0 0x7f8e0819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e06ffd700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e08103310 0x7f8e0819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e077fe700 1 -- 172.21.15.115:0/23905873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e081089f0 msgr2=0x7f8e0819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e077fe700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e081089f0 0x7f8e0819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e077fe700 1 -- 172.21.15.115:0/23905873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e08103310 msgr2=0x7f8e0819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e077fe700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e08103310 0x7f8e0819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e077fe700 1 -- 172.21.15.115:0/23905873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0819fb30 con 0x7f8e081029c0 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.574+0000 7f8e077fe700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e081029c0 0x7f8e0819ad10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c9e0 tx=0x7f8df800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.575+0000 7f8e04ff9700 1 -- 172.21.15.115:0/23905873 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8007aa0 con 0x7f8e081029c0 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.575+0000 7f8e04ff9700 1 -- 172.21.15.115:0/23905873 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8df8007c40 con 0x7f8e081029c0 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.575+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0819fdc0 con 0x7f8e081029c0 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.575+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e081a0350 con 0x7f8e081029c0 2024-05-09T05:33:29.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.575+0000 7f8e04ff9700 1 -- 172.21.15.115:0/23905873 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8018400 con 0x7f8e081029c0 2024-05-09T05:33:29.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.576+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e080512c0 con 0x7f8e081029c0 2024-05-09T05:33:29.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.577+0000 7f8e04ff9700 1 -- 172.21.15.115:0/23905873 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8df800f450 con 0x7f8e081029c0 2024-05-09T05:33:29.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.578+0000 7f8e04ff9700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8df00b2900 0x7f8df00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:29.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.578+0000 7f8e06ffd700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8df00b2900 0x7f8df00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:29.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.578+0000 7f8e04ff9700 1 -- 172.21.15.115:0/23905873 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f8df80e1b50 con 0x7f8e081029c0 2024-05-09T05:33:29.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.579+0000 7f8e06ffd700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8df00b2900 0x7f8df00b4e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e081021b0 tx=0x7f8dfc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:29.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.583+0000 7f8e04ff9700 1 -- 172.21.15.115:0/23905873 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8df8014030 con 0x7f8e081029c0 2024-05-09T05:33:29.616 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:29 smithi195 podman[64191]: 2024-05-09 05:33:29.221540207 +0000 UTC m=+0.491631717 container init c7a2a7b858744d3b3ef27a253e48f1b24e5117ae4e8dd22a60df79ce1e0e6dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:33:29.616 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:29 smithi195 podman[64191]: 2024-05-09 05:33:29.224818492 +0000 UTC m=+0.494910014 container start c7a2a7b858744d3b3ef27a253e48f1b24e5117ae4e8dd22a60df79ce1e0e6dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:33:29.616 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:29 smithi195 ceph-mds[64211]: -- 172.21.15.195:0/1093177400 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6c7a51a40 con 0x55d6c7a99180 2024-05-09T05:33:29.616 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:29 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: starting mds.i at 2024-05-09T05:33:29.616 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:29 smithi195 bash[64191]: c7a2a7b858744d3b3ef27a253e48f1b24e5117ae4e8dd22a60df79ce1e0e6dd5 2024-05-09T05:33:29.616 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:29 smithi195 systemd[1]: Started Ceph mds.i for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:29.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.858+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e081a7900 con 0x7f8e081029c0 2024-05-09T05:33:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.858+0000 7f8e04ff9700 1 -- 172.21.15.115:0/23905873 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f8df80a9d10 con 0x7f8e081029c0 2024-05-09T05:33:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:33:29.852 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T05:33:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8df00b2900 msgr2=0x7f8df00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8df00b2900 0x7f8df00b4e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e081021b0 tx=0x7f8dfc009260 comp rx=0 tx=0).stop 2024-05-09T05:33:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e081029c0 msgr2=0x7f8e0819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e081029c0 0x7f8e0819ad10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c9e0 tx=0x7f8df800ccf0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 shutdown_connections 2024-05-09T05:33:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8df00b2900 0x7f8df00b4e10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e081089f0 0x7f8e0819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e081029c0 0x7f8e0819ad10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 --2- 172.21.15.115:0/23905873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e08103310 0x7f8e0819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 >> 172.21.15.115:0/23905873 conn(0x7f8e080fe6f0 msgr2=0x7f8e08107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 shutdown_connections 2024-05-09T05:33:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:29.861+0000 7f8e0e256700 1 -- 172.21.15.115:0/23905873 wait complete. 2024-05-09T05:33:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 10 2024-05-09T05:33:30.463 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:30.463 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:30.463 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:30.463 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:30.463 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:30.463 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:30.463 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:30.463 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: Deploying daemon mds.g on smithi115 2024-05-09T05:33:30.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] up:boot 2024-05-09T05:33:30.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: fsmap 9 up:standby 2024-05-09T05:33:30.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T05:33:30.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:30 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/23905873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:30.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:30.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:30.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:30.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:30.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:30.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:30.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:30.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: Deploying daemon mds.g on smithi115 2024-05-09T05:33:30.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] up:boot 2024-05-09T05:33:30.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: fsmap 9 up:standby 2024-05-09T05:33:30.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T05:33:30.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:30 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/23905873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: Deploying daemon mds.g on smithi115 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] up:boot 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: fsmap 9 up:standby 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T05:33:30.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:30 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/23905873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:32.368 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 05:33:32 smithi115 systemd[1]: Starting Ceph mds.g for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:33:32.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:33:32.647 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 05:33:32 smithi115 podman[69124]: 2024-05-09 05:33:32.3179041 +0000 UTC m=+0.018528510 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:32.647 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 05:33:32 smithi115 podman[69124]: 2024-05-09 05:33:32.439254806 +0000 UTC m=+0.139879200 container create 94da5797eb467d8923b44e06dbca7f3ee27fdad8216f1b03b38517a89507a064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:33:32.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:32 smithi115 ceph-mon[30349]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:32.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:32 smithi161 ceph-mon[33805]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:32.844 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:32.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:32 smithi195 ceph-mon[33844]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:32.967 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 05:33:32 smithi115 podman[69124]: 2024-05-09 05:33:32.700717359 +0000 UTC m=+0.401341764 container init 94da5797eb467d8923b44e06dbca7f3ee27fdad8216f1b03b38517a89507a064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T05:33:32.967 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 05:33:32 smithi115 podman[69124]: 2024-05-09 05:33:32.704549773 +0000 UTC m=+0.405174168 container start 94da5797eb467d8923b44e06dbca7f3ee27fdad8216f1b03b38517a89507a064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:33:32.967 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 05:33:32 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g[69157]: starting mds.g at 2024-05-09T05:33:32.967 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 05:33:32 smithi115 bash[69124]: 94da5797eb467d8923b44e06dbca7f3ee27fdad8216f1b03b38517a89507a064 2024-05-09T05:33:32.967 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 05:33:32 smithi115 systemd[1]: Started Ceph mds.g for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:33.997 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:33.997 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:33.997 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:33.997 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:33.998 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:33.998 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:33.998 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:33.998 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: Deploying daemon mds.a on smithi115 2024-05-09T05:33:33.998 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.115:6838/3801478957,v1:172.21.15.115:6839/3801478957] up:boot 2024-05-09T05:33:33.998 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: fsmap 10 up:standby 2024-05-09T05:33:33.998 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T05:33:34.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:34.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:34.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:34.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:34.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:34.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:34.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:34.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: Deploying daemon mds.a on smithi115 2024-05-09T05:33:34.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.115:6838/3801478957,v1:172.21.15.115:6839/3801478957] up:boot 2024-05-09T05:33:34.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: fsmap 10 up:standby 2024-05-09T05:33:34.117 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T05:33:34.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:34.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: Deploying daemon mds.a on smithi115 2024-05-09T05:33:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.115:6838/3801478957,v1:172.21.15.115:6839/3801478957] up:boot 2024-05-09T05:33:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: fsmap 10 up:standby 2024-05-09T05:33:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T05:33:34.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.552+0000 7f236b0d3700 1 -- 172.21.15.115:0/3964970933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2364103310 msgr2=0x7f236410f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.552+0000 7f236b0d3700 1 --2- 172.21.15.115:0/3964970933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2364103310 0x7f236410f890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f235c00aaa0 tx=0x7f235c00adb0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.552+0000 7f236b0d3700 1 -- 172.21.15.115:0/3964970933 shutdown_connections 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.552+0000 7f236b0d3700 1 --2- 172.21.15.115:0/3964970933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23641089f0 0x7f2364108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.552+0000 7f236b0d3700 1 --2- 172.21.15.115:0/3964970933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2364103310 0x7f236410f890 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.552+0000 7f236b0d3700 1 --2- 172.21.15.115:0/3964970933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23641029c0 0x7f2364102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.552+0000 7f236b0d3700 1 -- 172.21.15.115:0/3964970933 >> 172.21.15.115:0/3964970933 conn(0x7f23640fe520 msgr2=0x7f2364100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.552+0000 7f236b0d3700 1 -- 172.21.15.115:0/3964970933 shutdown_connections 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.552+0000 7f236b0d3700 1 -- 172.21.15.115:0/3964970933 wait complete. 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f236b0d3700 1 Processor -- start 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f236b0d3700 1 -- start start 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f236b0d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23641029c0 0x7f236419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f236b0d3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2364103310 0x7f236419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f236b0d3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23641089f0 0x7f23641988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f236b0d3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2364112190 con 0x7f23641029c0 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f236b0d3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2364112010 con 0x7f23641089f0 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f236b0d3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2364112310 con 0x7f2364103310 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f2368e6f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23641029c0 0x7f236419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f235bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2364103310 0x7f236419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f2368e6f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23641029c0 0x7f236419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56562/0 (socket says 172.21.15.115:56562) 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.553+0000 7f2368e6f700 1 -- 172.21.15.115:0/4272046499 learned_addr learned my addr 172.21.15.115:0/4272046499 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2369670700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23641089f0 0x7f23641988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2368e6f700 1 -- 172.21.15.115:0/4272046499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2364103310 msgr2=0x7f236419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2368e6f700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2364103310 0x7f236419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2368e6f700 1 -- 172.21.15.115:0/4272046499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23641089f0 msgr2=0x7f23641988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2368e6f700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23641089f0 0x7f23641988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2368e6f700 1 -- 172.21.15.115:0/4272046499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2364198f30 con 0x7f23641029c0 2024-05-09T05:33:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2368e6f700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23641029c0 0x7f236419e6d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2360007f10 tx=0x7f236000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:34.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2359ffb700 1 -- 172.21.15.115:0/4272046499 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236000ddd0 con 0x7f23641029c0 2024-05-09T05:33:34.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2359ffb700 1 -- 172.21.15.115:0/4272046499 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2360018070 con 0x7f23641029c0 2024-05-09T05:33:34.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2364199220 con 0x7f23641029c0 2024-05-09T05:33:34.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f2359ffb700 1 -- 172.21.15.115:0/4272046499 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23600125b0 con 0x7f23641029c0 2024-05-09T05:33:34.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.554+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23641a5560 con 0x7f23641029c0 2024-05-09T05:33:34.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.555+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23640512c0 con 0x7f23641029c0 2024-05-09T05:33:34.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.558+0000 7f2359ffb700 1 -- 172.21.15.115:0/4272046499 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f23600139a0 con 0x7f23641029c0 2024-05-09T05:33:34.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.558+0000 7f2359ffb700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f234c0b4b10 0x7f234c0b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:34.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.558+0000 7f235bfff700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f234c0b4b10 0x7f234c0b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:34.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.558+0000 7f2359ffb700 1 -- 172.21.15.115:0/4272046499 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f23600e1d60 con 0x7f23641029c0 2024-05-09T05:33:34.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.559+0000 7f235bfff700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f234c0b4b10 0x7f234c0b7020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2364105f40 tx=0x7f235400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:34.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.562+0000 7f2359ffb700 1 -- 172.21.15.115:0/4272046499 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2360010070 con 0x7f23641029c0 2024-05-09T05:33:34.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.826+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2364199e20 con 0x7f23641029c0 2024-05-09T05:33:34.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.827+0000 7f2359ffb700 1 -- 172.21.15.115:0/4272046499 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8845 (secure 0 0 0) 0x7f23600a9f20 con 0x7f23641029c0 2024-05-09T05:33:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:33:34.821 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f234c0b4b10 msgr2=0x7f234c0b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f234c0b4b10 0x7f234c0b7020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2364105f40 tx=0x7f235400a380 comp rx=0 tx=0).stop 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23641029c0 msgr2=0x7f236419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23641029c0 0x7f236419e6d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2360007f10 tx=0x7f236000d3b0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 shutdown_connections 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f234c0b4b10 0x7f234c0b7020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23641089f0 0x7f23641988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23641029c0 0x7f236419e6d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 --2- 172.21.15.115:0/4272046499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2364103310 0x7f236419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 >> 172.21.15.115:0/4272046499 conn(0x7f23640fe520 msgr2=0x7f2364100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 shutdown_connections 2024-05-09T05:33:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:34.829+0000 7f236b0d3700 1 -- 172.21.15.115:0/4272046499 wait complete. 2024-05-09T05:33:34.824 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 11 2024-05-09T05:33:35.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:34 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4272046499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:35.121 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:34 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4272046499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:35.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:34 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4272046499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:36.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:35 smithi195 ceph-mon[33844]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:36.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:35 smithi115 ceph-mon[30349]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:36.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:35 smithi161 ceph-mon[33805]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:36.646 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 05:33:36 smithi115 systemd[1]: Starting Ceph mds.a for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:33:36.927 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 05:33:36 smithi115 podman[69588]: 2024-05-09 05:33:36.558395511 +0000 UTC m=+0.018195389 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:36.927 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 05:33:36 smithi115 podman[69588]: 2024-05-09 05:33:36.680929491 +0000 UTC m=+0.140729415 container create 215309d10b0c8d22b58bdb9e64d4585b672aa8cdbca0e9c08ab3396efb22cb76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T05:33:36.927 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 05:33:36 smithi115 podman[69588]: 2024-05-09 05:33:36.933915735 +0000 UTC m=+0.393715614 container init 215309d10b0c8d22b58bdb9e64d4585b672aa8cdbca0e9c08ab3396efb22cb76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-09T05:33:37.396 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 05:33:36 smithi115 podman[69588]: 2024-05-09 05:33:36.936785258 +0000 UTC m=+0.396585136 container start 215309d10b0c8d22b58bdb9e64d4585b672aa8cdbca0e9c08ab3396efb22cb76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-09T05:33:37.397 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 05:33:36 smithi115 ceph-mds[69607]: -- 172.21.15.115:0/1997123965 <== mon.0 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f513b6da40 con 0x55f513bb5a80 2024-05-09T05:33:37.397 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 05:33:36 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-a[69603]: starting mds.a at 2024-05-09T05:33:37.397 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 05:33:37 smithi115 bash[69588]: 215309d10b0c8d22b58bdb9e64d4585b672aa8cdbca0e9c08ab3396efb22cb76 2024-05-09T05:33:37.397 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 05:33:37 smithi115 systemd[1]: Started Ceph mds.a for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:38.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:38.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:38.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:38.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:38.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:38.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:38.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:38.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: Deploying daemon mds.b on smithi161 2024-05-09T05:33:38.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.115:6840/1081505697,v1:172.21.15.115:6841/1081505697] up:boot 2024-05-09T05:33:38.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: fsmap 11 up:standby 2024-05-09T05:33:38.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:38 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T05:33:38.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: Deploying daemon mds.b on smithi161 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.115:6840/1081505697,v1:172.21.15.115:6841/1081505697] up:boot 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: fsmap 11 up:standby 2024-05-09T05:33:38.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:38 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T05:33:38.399 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:33:38.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: Deploying daemon mds.b on smithi161 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.115:6840/1081505697,v1:172.21.15.115:6841/1081505697] up:boot 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: fsmap 11 up:standby 2024-05-09T05:33:38.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:38 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T05:33:38.659 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:39.611 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 05:33:39 smithi161 systemd[1]: Starting Ceph mds.b for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:33:39.866 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 05:33:39 smithi161 podman[63587]: 2024-05-09 05:33:39.520657478 +0000 UTC m=+0.017989309 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:33:39.867 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 05:33:39 smithi161 podman[63587]: 2024-05-09 05:33:39.621685021 +0000 UTC m=+0.119016898 container create 937d4c8c6dd8a6dcb914f0916a248c16ccca391ae36e891ef7cfabadc527cf81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:33:40.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:40 smithi161 ceph-mon[33805]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:40.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:40.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:40.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:40.161 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 05:33:39 smithi161 podman[63587]: 2024-05-09 05:33:39.873549886 +0000 UTC m=+0.370881717 container init 937d4c8c6dd8a6dcb914f0916a248c16ccca391ae36e891ef7cfabadc527cf81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T05:33:40.161 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 05:33:39 smithi161 podman[63587]: 2024-05-09 05:33:39.877357206 +0000 UTC m=+0.374689038 container start 937d4c8c6dd8a6dcb914f0916a248c16ccca391ae36e891ef7cfabadc527cf81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T05:33:40.161 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 05:33:39 smithi161 ceph-mds[63606]: -- 172.21.15.161:0/3290913135 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f08135a40 con 0x563f0817d180 2024-05-09T05:33:40.161 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 05:33:39 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-b[63602]: starting mds.b at 2024-05-09T05:33:40.161 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 05:33:39 smithi161 bash[63587]: 937d4c8c6dd8a6dcb914f0916a248c16ccca391ae36e891ef7cfabadc527cf81 2024-05-09T05:33:40.161 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 05:33:39 smithi161 systemd[1]: Started Ceph mds.b for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:33:40.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:40 smithi195 ceph-mon[33844]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:40.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:40.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:40.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:40.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.373+0000 7f3df28f6700 1 -- 172.21.15.115:0/2815220408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec108740 msgr2=0x7f3dec1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:40.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.373+0000 7f3df28f6700 1 --2- 172.21.15.115:0/2815220408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec108740 0x7f3dec1100a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3de400b0d0 tx=0x7f3de400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.373+0000 7f3df28f6700 1 -- 172.21.15.115:0/2815220408 shutdown_connections 2024-05-09T05:33:40.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.373+0000 7f3df28f6700 1 --2- 172.21.15.115:0/2815220408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dec1034c0 0x7f3dec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.373+0000 7f3df28f6700 1 --2- 172.21.15.115:0/2815220408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec108740 0x7f3dec1100a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.373+0000 7f3df28f6700 1 --2- 172.21.15.115:0/2815220408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dec107df0 0x7f3dec108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.373+0000 7f3df28f6700 1 -- 172.21.15.115:0/2815220408 >> 172.21.15.115:0/2815220408 conn(0x7f3dec0fedc0 msgr2=0x7f3dec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:40.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.373+0000 7f3df28f6700 1 -- 172.21.15.115:0/2815220408 shutdown_connections 2024-05-09T05:33:40.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.373+0000 7f3df28f6700 1 -- 172.21.15.115:0/2815220408 wait complete. 2024-05-09T05:33:40.368 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:40 smithi115 ceph-mon[30349]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:40.368 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:40.368 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:40.368 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:40.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.374+0000 7f3df28f6700 1 Processor -- start 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.374+0000 7f3df28f6700 1 -- start start 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.374+0000 7f3df28f6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec1034c0 0x7f3dec19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df28f6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dec107df0 0x7f3dec19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df28f6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dec108740 0x7f3dec19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df28f6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec113220 con 0x7f3dec1034c0 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df28f6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec1130a0 con 0x7f3dec107df0 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df28f6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec1133a0 con 0x7f3dec108740 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3debfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec1034c0 0x7f3dec19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3debfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec1034c0 0x7f3dec19bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56588/0 (socket says 172.21.15.115:56588) 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3debfff700 1 -- 172.21.15.115:0/1939695211 learned_addr learned my addr 172.21.15.115:0/1939695211 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df0e93700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dec108740 0x7f3dec19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3deb7fe700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dec107df0 0x7f3dec19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:40.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df0e93700 1 -- 172.21.15.115:0/1939695211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dec107df0 msgr2=0x7f3dec19c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df0e93700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dec107df0 0x7f3dec19c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df0e93700 1 -- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec1034c0 msgr2=0x7f3dec19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df0e93700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec1034c0 0x7f3dec19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df0e93700 1 -- 172.21.15.115:0/1939695211 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec1a6e70 con 0x7f3dec108740 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3debfff700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec1034c0 0x7f3dec19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3deb7fe700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dec107df0 0x7f3dec19c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.375+0000 7f3df0e93700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dec108740 0x7f3dec19c5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3de400b0d0 tx=0x7f3de4007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.376+0000 7f3de97fa700 1 -- 172.21.15.115:0/1939695211 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de401f070 con 0x7f3dec108740 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.376+0000 7f3de97fa700 1 -- 172.21.15.115:0/1939695211 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3de4004c50 con 0x7f3dec108740 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.376+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dec1998f0 con 0x7f3dec108740 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.376+0000 7f3de97fa700 1 -- 172.21.15.115:0/1939695211 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de401b430 con 0x7f3dec108740 2024-05-09T05:33:40.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.376+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dec199dd0 con 0x7f3dec108740 2024-05-09T05:33:40.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.377+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dec0512c0 con 0x7f3dec108740 2024-05-09T05:33:40.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.379+0000 7f3de97fa700 1 -- 172.21.15.115:0/1939695211 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3de4022070 con 0x7f3dec108740 2024-05-09T05:33:40.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.379+0000 7f3de97fa700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3dd40b2880 0x7f3dd40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.379+0000 7f3debfff700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3dd40b2880 0x7f3dd40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.379+0000 7f3de97fa700 1 -- 172.21.15.115:0/1939695211 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3de400f020 con 0x7f3dec108740 2024-05-09T05:33:40.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.380+0000 7f3debfff700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3dd40b2880 0x7f3dd40b4d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc011550 tx=0x7f3ddc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:40.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.388+0000 7f3de97fa700 1 -- 172.21.15.115:0/1939695211 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3de4029020 con 0x7f3dec108740 2024-05-09T05:33:40.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.642+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dec02e5f0 con 0x7f3dec108740 2024-05-09T05:33:40.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.644+0000 7f3de97fa700 1 -- 172.21.15.115:0/1939695211 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10500 (secure 0 0 0) 0x7f3de40ab320 con 0x7f3dec108740 2024-05-09T05:33:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:33:40.637 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T05:33:40.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3dd40b2880 msgr2=0x7f3dd40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:40.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3dd40b2880 0x7f3dd40b4d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc011550 tx=0x7f3ddc009260 comp rx=0 tx=0).stop 2024-05-09T05:33:40.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dec108740 msgr2=0x7f3dec19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dec108740 0x7f3dec19c5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3de400b0d0 tx=0x7f3de4007a00 comp rx=0 tx=0).stop 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 shutdown_connections 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3dd40b2880 0x7f3dd40b4d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3dec108740 0x7f3dec19c5c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3dec1034c0 0x7f3dec19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 --2- 172.21.15.115:0/1939695211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dec107df0 0x7f3dec19c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 >> 172.21.15.115:0/1939695211 conn(0x7f3dec0fedc0 msgr2=0x7f3dec107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 shutdown_connections 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:40.646+0000 7f3df28f6700 1 -- 172.21.15.115:0/1939695211 wait complete. 2024-05-09T05:33:40.640 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 13 2024-05-09T05:33:41.270 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:41.270 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:41 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] up:boot 2024-05-09T05:33:41.270 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:41 smithi115 ceph-mon[30349]: fsmap 12 up:standby 2024-05-09T05:33:41.270 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T05:33:41.271 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:41.271 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:33:41.271 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:41 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1939695211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:41.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-09T05:33:41.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:41.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:41 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] up:boot 2024-05-09T05:33:41.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:41 smithi161 ceph-mon[33805]: fsmap 12 up:standby 2024-05-09T05:33:41.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T05:33:41.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:41.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:33:41.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:41 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1939695211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:41.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:41.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:41 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] up:boot 2024-05-09T05:33:41.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:41 smithi195 ceph-mon[33844]: fsmap 12 up:standby 2024-05-09T05:33:41.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T05:33:41.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:41.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:33:41.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:41 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1939695211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:33:42.189 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:42.342 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:42 smithi115 ceph-mon[30349]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:42.342 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.342 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.342 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.342 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:42 smithi195 ceph-mon[33844]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:42 smithi161 ceph-mon[33805]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:42.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:43.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:43 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:43.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:43 smithi115 ceph-mon[30349]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:43.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:43 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:43.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:43 smithi161 ceph-mon[33805]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:43.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.943+0000 7f401fd9f700 1 -- 172.21.15.115:0/3341793619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 msgr2=0x7f40181038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:43.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.943+0000 7f401fd9f700 1 --2- 172.21.15.115:0/3341793619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 0x7f40181038d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f40140099c0 tx=0x7f4014009cd0 comp rx=0 tx=0).stop 2024-05-09T05:33:43.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.943+0000 7f401fd9f700 1 -- 172.21.15.115:0/3341793619 shutdown_connections 2024-05-09T05:33:43.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.943+0000 7f401fd9f700 1 --2- 172.21.15.115:0/3341793619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4018108740 0x7f40181100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:43.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.943+0000 7f401fd9f700 1 --2- 172.21.15.115:0/3341793619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 0x7f40181038d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:43.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.943+0000 7f401fd9f700 1 --2- 172.21.15.115:0/3341793619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4018107df0 0x7f4018108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:43.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.943+0000 7f401fd9f700 1 -- 172.21.15.115:0/3341793619 >> 172.21.15.115:0/3341793619 conn(0x7f40180fedf0 msgr2=0x7f4018101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.943+0000 7f401fd9f700 1 -- 172.21.15.115:0/3341793619 shutdown_connections 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.943+0000 7f401fd9f700 1 -- 172.21.15.115:0/3341793619 wait complete. 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401fd9f700 1 Processor -- start 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401fd9f700 1 -- start start 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401fd9f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 0x7f401819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401fd9f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4018107df0 0x7f401819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401fd9f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4018108740 0x7f401819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401fd9f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40181132c0 con 0x7f40181034c0 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401fd9f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4018113140 con 0x7f4018108740 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401fd9f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4018113440 con 0x7f4018107df0 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401db3b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 0x7f401819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401db3b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 0x7f401819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34714/0 (socket says 172.21.15.115:34714) 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401db3b700 1 -- 172.21.15.115:0/1894386745 learned_addr learned my addr 172.21.15.115:0/1894386745 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:43.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401e33c700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4018108740 0x7f401819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401d33a700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4018107df0 0x7f401819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401db3b700 1 -- 172.21.15.115:0/1894386745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4018107df0 msgr2=0x7f401819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401db3b700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4018107df0 0x7f401819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401db3b700 1 -- 172.21.15.115:0/1894386745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4018108740 msgr2=0x7f401819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401db3b700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4018108740 0x7f401819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401db3b700 1 -- 172.21.15.115:0/1894386745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401819fc70 con 0x7f40181034c0 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401d33a700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4018107df0 0x7f401819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.944+0000 7f401db3b700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 0x7f401819ada0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4014000c00 tx=0x7f401400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.945+0000 7f400effd700 1 -- 172.21.15.115:0/1894386745 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014026070 con 0x7f40181034c0 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.945+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401819ff00 con 0x7f40181034c0 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.945+0000 7f400effd700 1 -- 172.21.15.115:0/1894386745 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40140054c0 con 0x7f40181034c0 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.945+0000 7f400effd700 1 -- 172.21.15.115:0/1894386745 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014004c20 con 0x7f40181034c0 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.945+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40181a7880 con 0x7f40181034c0 2024-05-09T05:33:43.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.946+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40180512c0 con 0x7f40181034c0 2024-05-09T05:33:43.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.947+0000 7f400effd700 1 -- 172.21.15.115:0/1894386745 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4014029070 con 0x7f40181034c0 2024-05-09T05:33:43.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.948+0000 7f400effd700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f40040b2880 0x7f40040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:43.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.948+0000 7f401d33a700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f40040b2880 0x7f40040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:43.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.948+0000 7f400effd700 1 -- 172.21.15.115:0/1894386745 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f40140e2d40 con 0x7f40181034c0 2024-05-09T05:33:43.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.948+0000 7f401d33a700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f40040b2880 0x7f40040b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4008005fd0 tx=0x7f4008005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:43.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:43.951+0000 7f400effd700 1 -- 172.21.15.115:0/1894386745 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4014030080 con 0x7f40181034c0 2024-05-09T05:33:44.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:43 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:44.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:43 smithi195 ceph-mon[33844]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.181+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f401802e5f0 con 0x7f40181034c0 2024-05-09T05:33:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.182+0000 7f400effd700 1 -- 172.21.15.115:0/1894386745 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f4014013070 con 0x7f40181034c0 2024-05-09T05:33:44.176 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:33:44.176 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2024-05-09T05:33:44.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:33:44.176 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2024-05-09T05:33:44.176 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2024-05-09T05:33:44.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:33:44.176 INFO:teuthology.orchestra.run.smithi115.stdout:] 2024-05-09T05:33:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f40040b2880 msgr2=0x7f40040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f40040b2880 0x7f40040b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4008005fd0 tx=0x7f4008005dc0 comp rx=0 tx=0).stop 2024-05-09T05:33:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 msgr2=0x7f401819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 0x7f401819ada0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4014000c00 tx=0x7f401400f740 comp rx=0 tx=0).stop 2024-05-09T05:33:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 shutdown_connections 2024-05-09T05:33:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f40040b2880 0x7f40040b4d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4018108740 0x7f401819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40181034c0 0x7f401819ada0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 --2- 172.21.15.115:0/1894386745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4018107df0 0x7f401819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 >> 172.21.15.115:0/1894386745 conn(0x7f40180fedf0 msgr2=0x7f4018075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.184+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 shutdown_connections 2024-05-09T05:33:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:44.185+0000 7f401fd9f700 1 -- 172.21.15.115:0/1894386745 wait complete. 2024-05-09T05:33:44.888 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T05:33:44.888 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T05:33:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:45.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:33:45.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:45.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:45 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1894386745' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T05:33:45.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:45 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:33:45.496 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:45.497 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:45.497 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:45.497 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:33:45.497 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:45.497 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:45 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1894386745' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T05:33:45.497 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:45 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:33:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:33:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:45 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1894386745' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T05:33:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:45 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:33:45.790 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:46.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:46 smithi195 ceph-mon[33844]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:46.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:46 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:46.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:46 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:46.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:46 smithi115 ceph-mon[30349]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:46.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:46 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:46.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:46 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:46.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:46 smithi161 ceph-mon[33805]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:46.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:46 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:46.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:46 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:47.535 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:47.535 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:47 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:47.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 -- 172.21.15.115:0/2007266460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 msgr2=0x7f4808108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:47.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 --2- 172.21.15.115:0/2007266460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 0x7f4808108200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47f00099c0 tx=0x7f47f0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:33:47.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 -- 172.21.15.115:0/2007266460 shutdown_connections 2024-05-09T05:33:47.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 --2- 172.21.15.115:0/2007266460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48081034c0 0x7f48081038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:47.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 --2- 172.21.15.115:0/2007266460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4808108740 0x7f48081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:47.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 --2- 172.21.15.115:0/2007266460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 0x7f4808108200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:47.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 -- 172.21.15.115:0/2007266460 >> 172.21.15.115:0/2007266460 conn(0x7f48080fedc0 msgr2=0x7f48081011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:47.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 -- 172.21.15.115:0/2007266460 shutdown_connections 2024-05-09T05:33:47.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 -- 172.21.15.115:0/2007266460 wait complete. 2024-05-09T05:33:47.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 Processor -- start 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.561+0000 7f480f255700 1 -- start start 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f480f255700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48081034c0 0x7f48081a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f480f255700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 0x7f48081a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f480f255700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4808108740 0x7f480819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f480f255700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4808113200 con 0x7f4808107df0 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f480f255700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4808113080 con 0x7f48081034c0 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f480f255700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4808113380 con 0x7f4808108740 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f4807fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 0x7f48081a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f4807fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 0x7f48081a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34738/0 (socket says 172.21.15.115:34738) 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f4807fff700 1 -- 172.21.15.115:0/3090520770 learned_addr learned my addr 172.21.15.115:0/3090520770 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f480d7f2700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4808108740 0x7f480819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f480cff1700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48081034c0 0x7f48081a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f4807fff700 1 -- 172.21.15.115:0/3090520770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4808108740 msgr2=0x7f480819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:47.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f4807fff700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4808108740 0x7f480819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f4807fff700 1 -- 172.21.15.115:0/3090520770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48081034c0 msgr2=0x7f48081a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f4807fff700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48081034c0 0x7f48081a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f4807fff700 1 -- 172.21.15.115:0/3090520770 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480819b280 con 0x7f4808107df0 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f480cff1700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48081034c0 0x7f48081a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.562+0000 7f4807fff700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 0x7f48081a0f00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f47f000b580 tx=0x7f47f0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.563+0000 7f4805ffb700 1 -- 172.21.15.115:0/3090520770 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f000f8a0 con 0x7f4808107df0 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.563+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480819b510 con 0x7f4808107df0 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.563+0000 7f4805ffb700 1 -- 172.21.15.115:0/3090520770 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47f0004830 con 0x7f4808107df0 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.563+0000 7f4805ffb700 1 -- 172.21.15.115:0/3090520770 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0005350 con 0x7f4808107df0 2024-05-09T05:33:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.563+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48081a7770 con 0x7f4808107df0 2024-05-09T05:33:47.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.564+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4808104970 con 0x7f4808107df0 2024-05-09T05:33:47.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.566+0000 7f4805ffb700 1 -- 172.21.15.115:0/3090520770 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f47f0029060 con 0x7f4808107df0 2024-05-09T05:33:47.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.567+0000 7f4805ffb700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47e80b2880 0x7f47e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:47.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.567+0000 7f480cff1700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47e80b2880 0x7f47e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:47.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.567+0000 7f4805ffb700 1 -- 172.21.15.115:0/3090520770 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f47f00e2bd0 con 0x7f4808107df0 2024-05-09T05:33:47.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.568+0000 7f480cff1700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47e80b2880 0x7f47e80b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005950 tx=0x7f47f80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:47.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.571+0000 7f4805ffb700 1 -- 172.21.15.115:0/3090520770 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47f0013070 con 0x7f4808107df0 2024-05-09T05:33:47.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:47.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:47 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:47.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:47.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:47 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:47.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:47.789+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f480819bff0 con 0x7f4808107df0 2024-05-09T05:33:48.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:48 smithi195 ceph-mon[33844]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:48.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:48.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:48.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:48 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3090520770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T05:33:48.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:48.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:33:48.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:48 smithi115 ceph-mon[30349]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:48 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3090520770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T05:33:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:33:48.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:48 smithi161 ceph-mon[33805]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:48 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3090520770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T05:33:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:33:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:33:48.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:33:48.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.726+0000 7f4805ffb700 1 -- 172.21.15.115:0/3090520770 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f47f00aae20 con 0x7f4808107df0 2024-05-09T05:33:48.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47e80b2880 msgr2=0x7f47e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:48.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47e80b2880 0x7f47e80b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47f8005950 tx=0x7f47f80058e0 comp rx=0 tx=0).stop 2024-05-09T05:33:48.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 msgr2=0x7f48081a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:48.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 0x7f48081a0f00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f47f000b580 tx=0x7f47f0004790 comp rx=0 tx=0).stop 2024-05-09T05:33:48.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 shutdown_connections 2024-05-09T05:33:48.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47e80b2880 0x7f47e80b4d90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:48.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48081034c0 0x7f48081a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:48.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4808108740 0x7f480819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:48.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 --2- 172.21.15.115:0/3090520770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4808107df0 0x7f48081a0f00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:48.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.729+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 >> 172.21.15.115:0/3090520770 conn(0x7f48080fedc0 msgr2=0x7f480806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:48.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.730+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 shutdown_connections 2024-05-09T05:33:48.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:48.730+0000 7f480f255700 1 -- 172.21.15.115:0/3090520770 wait complete. 2024-05-09T05:33:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2024-05-09T05:33:49.391 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-09T05:33:49.633 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:50.035 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:49 smithi115 ceph-mon[30349]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:50.035 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:49 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3090520770' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T05:33:50.036 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:49 smithi115 ceph-mon[30349]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T05:33:50.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:49 smithi195 ceph-mon[33844]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:50.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:49 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3090520770' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T05:33:50.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:49 smithi195 ceph-mon[33844]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T05:33:50.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:49 smithi161 ceph-mon[33805]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:50.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:49 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3090520770' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T05:33:50.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:49 smithi161 ceph-mon[33805]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T05:33:50.824 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:50 smithi115 ceph-mon[30349]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T05:33:51.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:50 smithi195 ceph-mon[33844]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T05:33:51.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:50 smithi161 ceph-mon[33805]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T05:33:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.394+0000 7fedc79ba700 1 -- 172.21.15.115:0/4209766000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc0108760 msgr2=0x7fedc01100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.394+0000 7fedc79ba700 1 --2- 172.21.15.115:0/4209766000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc0108760 0x7fedc01100c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedb800b0d0 tx=0x7fedb800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.394+0000 7fedc79ba700 1 -- 172.21.15.115:0/4209766000 shutdown_connections 2024-05-09T05:33:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.394+0000 7fedc79ba700 1 --2- 172.21.15.115:0/4209766000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc0108760 0x7fedc01100c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.394+0000 7fedc79ba700 1 --2- 172.21.15.115:0/4209766000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc01034e0 0x7fedc01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.394+0000 7fedc79ba700 1 --2- 172.21.15.115:0/4209766000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedc0107e10 0x7fedc0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.394+0000 7fedc79ba700 1 -- 172.21.15.115:0/4209766000 >> 172.21.15.115:0/4209766000 conn(0x7fedc00fef50 msgr2=0x7fedc0101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.394+0000 7fedc79ba700 1 -- 172.21.15.115:0/4209766000 shutdown_connections 2024-05-09T05:33:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.394+0000 7fedc79ba700 1 -- 172.21.15.115:0/4209766000 wait complete. 2024-05-09T05:33:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc79ba700 1 Processor -- start 2024-05-09T05:33:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc79ba700 1 -- start start 2024-05-09T05:33:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc79ba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc01034e0 0x7fedc019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc79ba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedc0107e10 0x7fedc019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc79ba700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc0108760 0x7fedc019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc79ba700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fedc0114430 con 0x7fedc01034e0 2024-05-09T05:33:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc79ba700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fedc01142b0 con 0x7fedc0107e10 2024-05-09T05:33:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc79ba700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fedc01145b0 con 0x7fedc0108760 2024-05-09T05:33:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc5756700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc01034e0 0x7fedc019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc5756700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc01034e0 0x7fedc019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34756/0 (socket says 172.21.15.115:34756) 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.395+0000 7fedc5756700 1 -- 172.21.15.115:0/1039463058 learned_addr learned my addr 172.21.15.115:0/1039463058 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc5f57700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc0108760 0x7fedc019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc4f55700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedc0107e10 0x7fedc019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc5756700 1 -- 172.21.15.115:0/1039463058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc0108760 msgr2=0x7fedc019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc5756700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc0108760 0x7fedc019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc5756700 1 -- 172.21.15.115:0/1039463058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedc0107e10 msgr2=0x7fedc019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc5756700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedc0107e10 0x7fedc019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc5756700 1 -- 172.21.15.115:0/1039463058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc019fd30 con 0x7fedc01034e0 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc4f55700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedc0107e10 0x7fedc019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc5756700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc01034e0 0x7fedc019ae60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00b530 tx=0x7fedbc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedb67fc700 1 -- 172.21.15.115:0/1039463058 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc015020 con 0x7fedc01034e0 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedb67fc700 1 -- 172.21.15.115:0/1039463058 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedbc011cc0 con 0x7fedc01034e0 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc01a0020 con 0x7fedc01034e0 2024-05-09T05:33:51.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedb67fc700 1 -- 172.21.15.115:0/1039463058 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc010dc0 con 0x7fedc01034e0 2024-05-09T05:33:51.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.396+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc01109d0 con 0x7fedc01034e0 2024-05-09T05:33:51.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.397+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc0104990 con 0x7fedc01034e0 2024-05-09T05:33:51.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.399+0000 7fedb67fc700 1 -- 172.21.15.115:0/1039463058 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fedbc018020 con 0x7fedc01034e0 2024-05-09T05:33:51.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.399+0000 7fedb67fc700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fedac0b2880 0x7fedac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:51.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.400+0000 7fedc4f55700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fedac0b2880 0x7fedac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:51.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.400+0000 7fedb67fc700 1 -- 172.21.15.115:0/1039463058 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7fedbc0e1d20 con 0x7fedc01034e0 2024-05-09T05:33:51.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.400+0000 7fedc4f55700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fedac0b2880 0x7fedac0b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fedc01066e0 tx=0x7fedb0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:51.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.403+0000 7fedb67fc700 1 -- 172.21.15.115:0/1039463058 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fedbc01f020 con 0x7fedc01034e0 2024-05-09T05:33:51.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.606+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fedc019c1c0 con 0x7fedc01034e0 2024-05-09T05:33:51.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.748+0000 7fedb67fc700 1 -- 172.21.15.115:0/1039463058 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7fedbc0a9d70 con 0x7fedc01034e0 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fedac0b2880 msgr2=0x7fedac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fedac0b2880 0x7fedac0b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fedc01066e0 tx=0x7fedb0008040 comp rx=0 tx=0).stop 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc01034e0 msgr2=0x7fedc019ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc01034e0 0x7fedc019ae60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00b530 tx=0x7fedbc00b840 comp rx=0 tx=0).stop 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 shutdown_connections 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fedac0b2880 0x7fedac0b4d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc0108760 0x7fedc019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc01034e0 0x7fedc019ae60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 --2- 172.21.15.115:0/1039463058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedc0107e10 0x7fedc019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 >> 172.21.15.115:0/1039463058 conn(0x7fedc00fef50 msgr2=0x7fedc01075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.750+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 shutdown_connections 2024-05-09T05:33:51.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:51.751+0000 7fedc79ba700 1 -- 172.21.15.115:0/1039463058 wait complete. 2024-05-09T05:33:51.747 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2024-05-09T05:33:51.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:51 smithi115 ceph-mon[30349]: pgmap v282: 33 pgs: 15 creating+peering, 17 unknown, 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:51.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:51 smithi115 ceph-mon[30349]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T05:33:51.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:51 smithi115 ceph-mon[30349]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T05:33:51.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:51 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1039463058' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T05:33:52.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:51 smithi195 ceph-mon[33844]: pgmap v282: 33 pgs: 15 creating+peering, 17 unknown, 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:52.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:51 smithi195 ceph-mon[33844]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T05:33:52.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:51 smithi195 ceph-mon[33844]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T05:33:52.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:51 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1039463058' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T05:33:52.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:51 smithi161 ceph-mon[33805]: pgmap v282: 33 pgs: 15 creating+peering, 17 unknown, 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:52.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:51 smithi161 ceph-mon[33805]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T05:33:52.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:51 smithi161 ceph-mon[33805]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T05:33:52.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:51 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1039463058' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-09T05:33:52.473 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T05:33:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:53.078 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:52 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1039463058' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T05:33:53.078 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:52 smithi115 ceph-mon[30349]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T05:33:53.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:52 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1039463058' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T05:33:53.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:52 smithi195 ceph-mon[33844]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T05:33:53.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:52 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1039463058' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-09T05:33:53.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:52 smithi161 ceph-mon[33805]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T05:33:53.841 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:53 smithi115 ceph-mon[30349]: pgmap v285: 97 pgs: 15 creating+peering, 75 unknown, 7 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:53.841 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:53 smithi115 ceph-mon[30349]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T05:33:54.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:53 smithi195 ceph-mon[33844]: pgmap v285: 97 pgs: 15 creating+peering, 75 unknown, 7 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:54.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:53 smithi195 ceph-mon[33844]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T05:33:54.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:53 smithi161 ceph-mon[33805]: pgmap v285: 97 pgs: 15 creating+peering, 75 unknown, 7 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:54.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:53 smithi161 ceph-mon[33805]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T05:33:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.467+0000 7fd74f1cd700 1 -- 172.21.15.115:0/322031682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd748107df0 msgr2=0x7fd748108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.467+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/322031682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd748107df0 0x7fd748108200 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7340099c0 tx=0x7fd734009cd0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.467+0000 7fd74f1cd700 1 -- 172.21.15.115:0/322031682 shutdown_connections 2024-05-09T05:33:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.467+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/322031682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7481034c0 0x7fd7481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.467+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/322031682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748108740 0x7fd7481100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.467+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/322031682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd748107df0 0x7fd748108200 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.467+0000 7fd74f1cd700 1 -- 172.21.15.115:0/322031682 >> 172.21.15.115:0/322031682 conn(0x7fd7480fedc0 msgr2=0x7fd7481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.467+0000 7fd74f1cd700 1 -- 172.21.15.115:0/322031682 shutdown_connections 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.467+0000 7fd74f1cd700 1 -- 172.21.15.115:0/322031682 wait complete. 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74f1cd700 1 Processor -- start 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74f1cd700 1 -- start start 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74f1cd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7481034c0 0x7fd74819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74f1cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd748107df0 0x7fd74819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74f1cd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748108740 0x7fd74819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74f1cd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd748113270 con 0x7fd7481034c0 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74f1cd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481130f0 con 0x7fd748107df0 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74f1cd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481133f0 con 0x7fd748108740 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74cf69700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7481034c0 0x7fd74819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74cf69700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7481034c0 0x7fd74819bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35602/0 (socket says 172.21.15.115:35602) 2024-05-09T05:33:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd74cf69700 1 -- 172.21.15.115:0/2745698000 learned_addr learned my addr 172.21.15.115:0/2745698000 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.468+0000 7fd747fff700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd748107df0 0x7fd74819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd74d76a700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748108740 0x7fd74819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd74cf69700 1 -- 172.21.15.115:0/2745698000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748108740 msgr2=0x7fd74819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd74cf69700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748108740 0x7fd74819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd74cf69700 1 -- 172.21.15.115:0/2745698000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd748107df0 msgr2=0x7fd74819c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd74cf69700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd748107df0 0x7fd74819c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd74cf69700 1 -- 172.21.15.115:0/2745698000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7481a6fc0 con 0x7fd7481034c0 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd74cf69700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7481034c0 0x7fd74819bc50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00d950 tx=0x7fd73c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd745ffb700 1 -- 172.21.15.115:0/2745698000 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c016070 con 0x7fd7481034c0 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd745ffb700 1 -- 172.21.15.115:0/2745698000 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd73c009dd0 con 0x7fd7481034c0 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7481a72b0 con 0x7fd7481034c0 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd745ffb700 1 -- 172.21.15.115:0/2745698000 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00f3c0 con 0x7fd7481034c0 2024-05-09T05:33:54.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.469+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7481a7810 con 0x7fd7481034c0 2024-05-09T05:33:54.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.470+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7480512c0 con 0x7fd7481034c0 2024-05-09T05:33:54.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.472+0000 7fd745ffb700 1 -- 172.21.15.115:0/2745698000 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd73c019070 con 0x7fd7481034c0 2024-05-09T05:33:54.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.473+0000 7fd745ffb700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd7380b2900 0x7fd7380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:54.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.473+0000 7fd747fff700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd7380b2900 0x7fd7380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:54.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.473+0000 7fd745ffb700 1 -- 172.21.15.115:0/2745698000 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7fd73c0e2070 con 0x7fd7481034c0 2024-05-09T05:33:54.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.474+0000 7fd747fff700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd7380b2900 0x7fd7380b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd734009990 tx=0x7fd734005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:54.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.477+0000 7fd745ffb700 1 -- 172.21.15.115:0/2745698000 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd73c0a9ea0 con 0x7fd7481034c0 2024-05-09T05:33:54.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.731+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd74802e5f0 con 0x7fd7481034c0 2024-05-09T05:33:54.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.773+0000 7fd745ffb700 1 -- 172.21.15.115:0/2745698000 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd73c0a97a0 con 0x7fd7481034c0 2024-05-09T05:33:54.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd7380b2900 msgr2=0x7fd7380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:54.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd7380b2900 0x7fd7380b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd734009990 tx=0x7fd734005fb0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7481034c0 msgr2=0x7fd74819bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:54.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7481034c0 0x7fd74819bc50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00d950 tx=0x7fd73c00dc60 comp rx=0 tx=0).stop 2024-05-09T05:33:54.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 shutdown_connections 2024-05-09T05:33:54.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd7380b2900 0x7fd7380b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748108740 0x7fd74819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7481034c0 0x7fd74819bc50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 --2- 172.21.15.115:0/2745698000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd748107df0 0x7fd74819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:54.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.775+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 >> 172.21.15.115:0/2745698000 conn(0x7fd7480fedc0 msgr2=0x7fd7481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:54.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.776+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 shutdown_connections 2024-05-09T05:33:54.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:54.776+0000 7fd74f1cd700 1 -- 172.21.15.115:0/2745698000 wait complete. 2024-05-09T05:33:54.770 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T05:33:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:54 smithi195 ceph-mon[33844]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T05:33:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:54 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2745698000' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T05:33:55.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:54 smithi195 ceph-mon[33844]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T05:33:55.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:54 smithi115 ceph-mon[30349]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T05:33:55.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:54 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2745698000' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T05:33:55.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:54 smithi115 ceph-mon[30349]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T05:33:55.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:54 smithi161 ceph-mon[33805]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T05:33:55.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:54 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2745698000' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T05:33:55.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:54 smithi161 ceph-mon[33805]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T05:33:55.535 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-09T05:33:55.536 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-09T05:33:55.792 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:56.069 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:55 smithi115 ceph-mon[30349]: pgmap v288: 97 pgs: 4 creating+peering, 19 unknown, 74 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:56.069 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:55 smithi115 ceph-mon[30349]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T05:33:56.069 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:55 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2745698000' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T05:33:56.070 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:55 smithi115 ceph-mon[30349]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T05:33:56.070 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:55 smithi115 ceph-mon[30349]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T05:33:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:55 smithi195 ceph-mon[33844]: pgmap v288: 97 pgs: 4 creating+peering, 19 unknown, 74 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:55 smithi195 ceph-mon[33844]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T05:33:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:55 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2745698000' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T05:33:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:55 smithi195 ceph-mon[33844]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T05:33:56.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:55 smithi195 ceph-mon[33844]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T05:33:56.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:55 smithi161 ceph-mon[33805]: pgmap v288: 97 pgs: 4 creating+peering, 19 unknown, 74 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:33:56.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:55 smithi161 ceph-mon[33805]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T05:33:56.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:55 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2745698000' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T05:33:56.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:55 smithi161 ceph-mon[33805]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T05:33:56.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:55 smithi161 ceph-mon[33805]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T05:33:57.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:56 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] up:active 2024-05-09T05:33:57.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:56 smithi195 ceph-mon[33844]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:33:57.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:56 smithi195 ceph-mon[33844]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:33:57.118 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:56 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] up:active 2024-05-09T05:33:57.118 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:56 smithi115 ceph-mon[30349]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:33:57.118 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:56 smithi115 ceph-mon[30349]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:33:57.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:56 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] up:active 2024-05-09T05:33:57.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:56 smithi161 ceph-mon[33805]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:33:57.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:56 smithi161 ceph-mon[33805]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:33:57.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.697+0000 7f014f43a700 1 -- 172.21.15.115:0/1425717407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 msgr2=0x7f01481111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:57.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.697+0000 7f014f43a700 1 --2- 172.21.15.115:0/1425717407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 0x7f01481111b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f01400099c0 tx=0x7f0140009cd0 comp rx=0 tx=0).stop 2024-05-09T05:33:57.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.697+0000 7f014f43a700 1 -- 172.21.15.115:0/1425717407 shutdown_connections 2024-05-09T05:33:57.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.697+0000 7f014f43a700 1 --2- 172.21.15.115:0/1425717407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 0x7f01481111b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:57.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.697+0000 7f014f43a700 1 --2- 172.21.15.115:0/1425717407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0148102b50 0x7f0148102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:57.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.697+0000 7f014f43a700 1 --2- 172.21.15.115:0/1425717407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0148108f00 0x7f0148109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:57.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.697+0000 7f014f43a700 1 -- 172.21.15.115:0/1425717407 >> 172.21.15.115:0/1425717407 conn(0x7f01480fe6f0 msgr2=0x7f0148100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:57.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.698+0000 7f014f43a700 1 -- 172.21.15.115:0/1425717407 shutdown_connections 2024-05-09T05:33:57.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.698+0000 7f014f43a700 1 -- 172.21.15.115:0/1425717407 wait complete. 2024-05-09T05:33:57.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014f43a700 1 Processor -- start 2024-05-09T05:33:57.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014f43a700 1 -- start start 2024-05-09T05:33:57.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014f43a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0148102b50 0x7f0148103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:57.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014f43a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0148108f00 0x7f0148103fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:57.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014f43a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 0x7f0148104520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:57.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014f43a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01481143c0 con 0x7f0148109850 2024-05-09T05:33:57.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014f43a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0148114240 con 0x7f0148102b50 2024-05-09T05:33:57.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014f43a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0148114540 con 0x7f0148108f00 2024-05-09T05:33:57.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014d9d7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 0x7f0148104520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014d9d7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 0x7f0148104520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35624/0 (socket says 172.21.15.115:35624) 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014d9d7700 1 -- 172.21.15.115:0/2451718496 learned_addr learned my addr 172.21.15.115:0/2451718496 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014d1d6700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0148102b50 0x7f0148103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014c9d5700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0148108f00 0x7f0148103fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014d9d7700 1 -- 172.21.15.115:0/2451718496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0148108f00 msgr2=0x7f0148103fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014d9d7700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0148108f00 0x7f0148103fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014d9d7700 1 -- 172.21.15.115:0/2451718496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0148102b50 msgr2=0x7f0148103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014d9d7700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0148102b50 0x7f0148103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.699+0000 7f014d9d7700 1 -- 172.21.15.115:0/2451718496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01481a90f0 con 0x7f0148109850 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.700+0000 7f014d9d7700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 0x7f0148104520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f014000bf40 tx=0x7f01400036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.700+0000 7f013e7fc700 1 -- 172.21.15.115:0/2451718496 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140026070 con 0x7f0148109850 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.700+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01481a9380 con 0x7f0148109850 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.700+0000 7f013e7fc700 1 -- 172.21.15.115:0/2451718496 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0140004c70 con 0x7f0148109850 2024-05-09T05:33:57.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.700+0000 7f013e7fc700 1 -- 172.21.15.115:0/2451718496 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140003dc0 con 0x7f0148109850 2024-05-09T05:33:57.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.700+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01481a97d0 con 0x7f0148109850 2024-05-09T05:33:57.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.701+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01480512c0 con 0x7f0148109850 2024-05-09T05:33:57.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.702+0000 7f013e7fc700 1 -- 172.21.15.115:0/2451718496 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0140029070 con 0x7f0148109850 2024-05-09T05:33:57.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.703+0000 7f013e7fc700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f01340b2880 0x7f01340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:33:57.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.703+0000 7f014d1d6700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f01340b2880 0x7f01340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:33:57.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.703+0000 7f013e7fc700 1 -- 172.21.15.115:0/2451718496 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f01400e2a80 con 0x7f0148109850 2024-05-09T05:33:57.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.704+0000 7f014d1d6700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f01340b2880 0x7f01340b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f01440060b0 tx=0x7f014400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:33:57.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.708+0000 7f013e7fc700 1 -- 172.21.15.115:0/2451718496 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0140013070 con 0x7f0148109850 2024-05-09T05:33:57.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:57.925+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f014802e7b0 con 0x7f0148109850 2024-05-09T05:33:58.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:57 smithi195 ceph-mon[33844]: pgmap v290: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 208 B/s wr, 4 op/s 2024-05-09T05:33:58.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:57 smithi195 ceph-mon[33844]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T05:33:58.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:57 smithi195 ceph-mon[33844]: Cluster is now healthy 2024-05-09T05:33:58.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:57 smithi115 ceph-mon[30349]: pgmap v290: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 208 B/s wr, 4 op/s 2024-05-09T05:33:58.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:57 smithi115 ceph-mon[30349]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T05:33:58.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:57 smithi115 ceph-mon[30349]: Cluster is now healthy 2024-05-09T05:33:58.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:57 smithi161 ceph-mon[33805]: pgmap v290: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 208 B/s wr, 4 op/s 2024-05-09T05:33:58.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:57 smithi161 ceph-mon[33805]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T05:33:58.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:57 smithi161 ceph-mon[33805]: Cluster is now healthy 2024-05-09T05:33:58.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.791+0000 7f013e7fc700 1 -- 172.21.15.115:0/2451718496 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7f01400aa980 con 0x7f0148109850 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f01340b2880 msgr2=0x7f01340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f01340b2880 0x7f01340b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f01440060b0 tx=0x7f014400c040 comp rx=0 tx=0).stop 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 msgr2=0x7f0148104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 0x7f0148104520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f014000bf40 tx=0x7f01400036f0 comp rx=0 tx=0).stop 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 shutdown_connections 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f01340b2880 0x7f01340b4d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0148102b50 0x7f0148103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0148109850 0x7f0148104520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 --2- 172.21.15.115:0/2451718496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0148108f00 0x7f0148103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 >> 172.21.15.115:0/2451718496 conn(0x7f01480fe6f0 msgr2=0x7f0148108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 shutdown_connections 2024-05-09T05:33:58.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:33:58.795+0000 7f014f43a700 1 -- 172.21.15.115:0/2451718496 wait complete. 2024-05-09T05:33:59.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:58 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2451718496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T05:33:59.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:58 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2451718496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T05:33:59.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:58 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2451718496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-09T05:33:59.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T05:33:59.786 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:33:59 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T05:33:59.466+0000 7f9d66d1b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T05:33:59.852 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:33:59.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:59 smithi115 ceph-mon[30349]: pgmap v291: 97 pgs: 97 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-09T05:33:59.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:59 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2451718496' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T05:33:59.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:33:59 smithi115 ceph-mon[30349]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T05:34:00.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:59 smithi195 ceph-mon[33844]: pgmap v291: 97 pgs: 97 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-09T05:34:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:59 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2451718496' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T05:34:00.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:33:59 smithi195 ceph-mon[33844]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T05:34:00.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:59 smithi161 ceph-mon[33805]: pgmap v291: 97 pgs: 97 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-09T05:34:00.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:59 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2451718496' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-09T05:34:00.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:33:59 smithi161 ceph-mon[33805]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T05:34:01.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.625+0000 7f06b7389700 1 -- 172.21.15.115:0/1106538501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b0108740 msgr2=0x7f06b01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:01.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.625+0000 7f06b7389700 1 --2- 172.21.15.115:0/1106538501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b0108740 0x7f06b01100a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f06a80099c0 tx=0x7f06a8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:01.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.625+0000 7f06b7389700 1 -- 172.21.15.115:0/1106538501 shutdown_connections 2024-05-09T05:34:01.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.625+0000 7f06b7389700 1 --2- 172.21.15.115:0/1106538501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06b01034c0 0x7f06b01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:01.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.625+0000 7f06b7389700 1 --2- 172.21.15.115:0/1106538501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b0108740 0x7f06b01100a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:01.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.625+0000 7f06b7389700 1 --2- 172.21.15.115:0/1106538501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06b0107df0 0x7f06b0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:01.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.625+0000 7f06b7389700 1 -- 172.21.15.115:0/1106538501 >> 172.21.15.115:0/1106538501 conn(0x7f06b00fedc0 msgr2=0x7f06b01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:01.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.626+0000 7f06b7389700 1 -- 172.21.15.115:0/1106538501 shutdown_connections 2024-05-09T05:34:01.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.626+0000 7f06b7389700 1 -- 172.21.15.115:0/1106538501 wait complete. 2024-05-09T05:34:01.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.626+0000 7f06b7389700 1 Processor -- start 2024-05-09T05:34:01.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.626+0000 7f06b7389700 1 -- start start 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.626+0000 7f06b7389700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b01034c0 0x7f06b019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b7389700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06b0107df0 0x7f06b019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b7389700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06b0108740 0x7f06b019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b7389700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f06b0113220 con 0x7f06b01034c0 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b7389700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f06b01130a0 con 0x7f06b0107df0 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b7389700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f06b01133a0 con 0x7f06b0108740 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5125700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b01034c0 0x7f06b019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5125700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b01034c0 0x7f06b019ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35648/0 (socket says 172.21.15.115:35648) 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5125700 1 -- 172.21.15.115:0/593702538 learned_addr learned my addr 172.21.15.115:0/593702538 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:01.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5926700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06b0108740 0x7f06b019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5125700 1 -- 172.21.15.115:0/593702538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06b0108740 msgr2=0x7f06b019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5125700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06b0108740 0x7f06b019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b4924700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06b0107df0 0x7f06b019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5125700 1 -- 172.21.15.115:0/593702538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06b0107df0 msgr2=0x7f06b019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5125700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06b0107df0 0x7f06b019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5125700 1 -- 172.21.15.115:0/593702538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b019fbf0 con 0x7f06b01034c0 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5926700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06b0108740 0x7f06b019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b4924700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06b0107df0 0x7f06b019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b5125700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b01034c0 0x7f06b019ad20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0060b0 tx=0x7f06ac007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06a67fc700 1 -- 172.21.15.115:0/593702538 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac01a070 con 0x7f06b01034c0 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06a67fc700 1 -- 172.21.15.115:0/593702538 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06ac00de20 con 0x7f06b01034c0 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06a67fc700 1 -- 172.21.15.115:0/593702538 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac014690 con 0x7f06b01034c0 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b019fee0 con 0x7f06b01034c0 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.627+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b01a7800 con 0x7f06b01034c0 2024-05-09T05:34:01.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.629+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b00512c0 con 0x7f06b01034c0 2024-05-09T05:34:01.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.630+0000 7f06a67fc700 1 -- 172.21.15.115:0/593702538 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f06ac01d070 con 0x7f06b01034c0 2024-05-09T05:34:01.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.631+0000 7f06a67fc700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f069c0b2880 0x7f069c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:01.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.631+0000 7f06b4924700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f069c0b2880 0x7f069c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:01.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.631+0000 7f06a67fc700 1 -- 172.21.15.115:0/593702538 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8277+0+0 (secure 0 0 0) 0x7f06ac0e1af0 con 0x7f06b01034c0 2024-05-09T05:34:01.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.631+0000 7f06b4924700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f069c0b2880 0x7f069c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f06b0109900 tx=0x7f06a0009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:01.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.637+0000 7f06a67fc700 1 -- 172.21.15.115:0/593702538 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06ac010070 con 0x7f06b01034c0 2024-05-09T05:34:01.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:01.843+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f06b002e5f0 con 0x7f06b01034c0 2024-05-09T05:34:01.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:01 smithi115 ceph-mon[30349]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-09T05:34:02.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:01 smithi195 ceph-mon[33844]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-09T05:34:02.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:01 smithi161 ceph-mon[33805]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-09T05:34:03.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:02 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/593702538' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T05:34:03.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:02 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/593702538' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T05:34:03.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:02 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/593702538' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T05:34:03.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.802+0000 7f06a67fc700 1 -- 172.21.15.115:0/593702538 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7f06ac0a97e0 con 0x7f06b01034c0 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f069c0b2880 msgr2=0x7f069c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f069c0b2880 0x7f069c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f06b0109900 tx=0x7f06a0009550 comp rx=0 tx=0).stop 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b01034c0 msgr2=0x7f06b019ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b01034c0 0x7f06b019ad20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0060b0 tx=0x7f06ac007f90 comp rx=0 tx=0).stop 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 shutdown_connections 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f069c0b2880 0x7f069c0b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06b0108740 0x7f06b019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06b01034c0 0x7f06b019ad20 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0060b0 tx=0x7f06ac007f90 comp rx=0 tx=0).stop 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 --2- 172.21.15.115:0/593702538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06b0107df0 0x7f06b019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.806+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 >> 172.21.15.115:0/593702538 conn(0x7f06b00fedc0 msgr2=0x7f06b006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:03.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.807+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 shutdown_connections 2024-05-09T05:34:03.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:03.807+0000 7f06b7389700 1 -- 172.21.15.115:0/593702538 wait complete. 2024-05-09T05:34:03.805 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data_ec' created 2024-05-09T05:34:04.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:03 smithi195 ceph-mon[33844]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T05:34:04.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:03 smithi195 ceph-mon[33844]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T05:34:04.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:03 smithi115 ceph-mon[30349]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T05:34:04.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:03 smithi115 ceph-mon[30349]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T05:34:04.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:03 smithi161 ceph-mon[33805]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T05:34:04.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:03 smithi161 ceph-mon[33805]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T05:34:04.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-09T05:34:04.871 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:04.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:04 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/593702538' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T05:34:04.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:04 smithi115 ceph-mon[30349]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T05:34:04.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:04 smithi115 ceph-mon[30349]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T05:34:05.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:04 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/593702538' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T05:34:05.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:04 smithi195 ceph-mon[33844]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T05:34:05.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:04 smithi195 ceph-mon[33844]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T05:34:05.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:04 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/593702538' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T05:34:05.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:04 smithi161 ceph-mon[33805]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T05:34:05.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:04 smithi161 ceph-mon[33805]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T05:34:06.037 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:05 smithi115 ceph-mon[30349]: pgmap v297: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-05-09T05:34:06.037 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:05 smithi115 ceph-mon[30349]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T05:34:06.037 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:05 smithi115 ceph-mon[30349]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T05:34:06.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:05 smithi195 ceph-mon[33844]: pgmap v297: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-05-09T05:34:06.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:05 smithi195 ceph-mon[33844]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T05:34:06.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:05 smithi195 ceph-mon[33844]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T05:34:06.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:05 smithi161 ceph-mon[33805]: pgmap v297: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-05-09T05:34:06.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:05 smithi161 ceph-mon[33805]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T05:34:06.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:05 smithi161 ceph-mon[33805]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T05:34:06.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.588+0000 7fda03bd2700 1 -- 172.21.15.115:0/1642786917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 msgr2=0x7fd9fc1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:06.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.588+0000 7fda03bd2700 1 --2- 172.21.15.115:0/1642786917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 0x7fd9fc1049f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80099c0 tx=0x7fd9f8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.588+0000 7fda03bd2700 1 -- 172.21.15.115:0/1642786917 shutdown_connections 2024-05-09T05:34:06.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.588+0000 7fda03bd2700 1 --2- 172.21.15.115:0/1642786917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc109860 0x7fd9fc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.588+0000 7fda03bd2700 1 --2- 172.21.15.115:0/1642786917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 0x7fd9fc1049f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.588+0000 7fda03bd2700 1 --2- 172.21.15.115:0/1642786917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9fc108f10 0x7fd9fc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.588+0000 7fda03bd2700 1 -- 172.21.15.115:0/1642786917 >> 172.21.15.115:0/1642786917 conn(0x7fd9fc0ffed0 msgr2=0x7fd9fc102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:06.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.588+0000 7fda03bd2700 1 -- 172.21.15.115:0/1642786917 shutdown_connections 2024-05-09T05:34:06.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.588+0000 7fda03bd2700 1 -- 172.21.15.115:0/1642786917 wait complete. 2024-05-09T05:34:06.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda03bd2700 1 Processor -- start 2024-05-09T05:34:06.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda03bd2700 1 -- start start 2024-05-09T05:34:06.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda03bd2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 0x7fd9fc19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:06.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda03bd2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc108f10 0x7fd9fc19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:06.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda03bd2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9fc109860 0x7fd9fc19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:06.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda03bd2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc114340 con 0x7fd9fc1045e0 2024-05-09T05:34:06.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda03bd2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1141c0 con 0x7fd9fc108f10 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda03bd2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1144c0 con 0x7fd9fc109860 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda0196e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 0x7fd9fc19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda0196e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 0x7fd9fc19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53442/0 (socket says 172.21.15.115:53442) 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda0196e700 1 -- 172.21.15.115:0/823539534 learned_addr learned my addr 172.21.15.115:0/823539534 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.589+0000 7fda0216f700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9fc109860 0x7fd9fc19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fda0116d700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc108f10 0x7fd9fc19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fda0196e700 1 -- 172.21.15.115:0/823539534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9fc109860 msgr2=0x7fd9fc19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fda0196e700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9fc109860 0x7fd9fc19f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fda0196e700 1 -- 172.21.15.115:0/823539534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc108f10 msgr2=0x7fd9fc19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fda0196e700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc108f10 0x7fd9fc19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fda0196e700 1 -- 172.21.15.115:0/823539534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc19fc00 con 0x7fd9fc1045e0 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fda0196e700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 0x7fd9fc19ad30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8000c00 tx=0x7fd9f80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fd9f2ffd700 1 -- 172.21.15.115:0/823539534 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8004830 con 0x7fd9fc1045e0 2024-05-09T05:34:06.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fd9f2ffd700 1 -- 172.21.15.115:0/823539534 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9f800f7f0 con 0x7fd9fc1045e0 2024-05-09T05:34:06.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc19fe90 con 0x7fd9fc1045e0 2024-05-09T05:34:06.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fd9f2ffd700 1 -- 172.21.15.115:0/823539534 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80053a0 con 0x7fd9fc1045e0 2024-05-09T05:34:06.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.590+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc1a0310 con 0x7fd9fc1045e0 2024-05-09T05:34:06.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.591+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9fc0512c0 con 0x7fd9fc1045e0 2024-05-09T05:34:06.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.593+0000 7fd9f2ffd700 1 -- 172.21.15.115:0/823539534 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd9f8029060 con 0x7fd9fc1045e0 2024-05-09T05:34:06.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.594+0000 7fd9f2ffd700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd9e80b2880 0x7fd9e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:06.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.594+0000 7fda0116d700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd9e80b2880 0x7fd9e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:06.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.594+0000 7fd9f2ffd700 1 -- 172.21.15.115:0/823539534 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7fd9f80e30f0 con 0x7fd9fc1045e0 2024-05-09T05:34:06.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.594+0000 7fda0116d700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd9e80b2880 0x7fd9e80b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc107570 tx=0x7fd9ec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:06.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.598+0000 7fd9f2ffd700 1 -- 172.21.15.115:0/823539534 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9f8013070 con 0x7fd9fc1045e0 2024-05-09T05:34:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.802+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fd9fc02e5f0 con 0x7fd9fc1045e0 2024-05-09T05:34:06.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.832+0000 7fd9f2ffd700 1 -- 172.21.15.115:0/823539534 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7fd9f80aac80 con 0x7fd9fc1045e0 2024-05-09T05:34:06.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd9e80b2880 msgr2=0x7fd9e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:06.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd9e80b2880 0x7fd9e80b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc107570 tx=0x7fd9ec00b410 comp rx=0 tx=0).stop 2024-05-09T05:34:06.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 msgr2=0x7fd9fc19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:06.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 0x7fd9fc19ad30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8000c00 tx=0x7fd9f80047b0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 shutdown_connections 2024-05-09T05:34:06.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd9e80b2880 0x7fd9e80b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9fc109860 0x7fd9fc19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9fc1045e0 0x7fd9fc19ad30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 --2- 172.21.15.115:0/823539534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9fc108f10 0x7fd9fc19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:06.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 >> 172.21.15.115:0/823539534 conn(0x7fd9fc0ffed0 msgr2=0x7fd9fc102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:06.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 shutdown_connections 2024-05-09T05:34:06.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:06.835+0000 7fda03bd2700 1 -- 172.21.15.115:0/823539534 wait complete. 2024-05-09T05:34:06.831 INFO:teuthology.orchestra.run.smithi115.stderr:set pool 4 allow_ec_overwrites to true 2024-05-09T05:34:06.843 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:06 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/823539534' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T05:34:07.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:06 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/823539534' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T05:34:07.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:06 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/823539534' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-09T05:34:07.572 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-09T05:34:07.814 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:08.079 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:07 smithi115 ceph-mon[30349]: pgmap v300: 129 pgs: 9 creating+peering, 17 unknown, 103 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:08.079 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:07 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/823539534' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T05:34:08.079 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:07 smithi115 ceph-mon[30349]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T05:34:08.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:07 smithi195 ceph-mon[33844]: pgmap v300: 129 pgs: 9 creating+peering, 17 unknown, 103 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:08.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:07 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/823539534' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T05:34:08.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:07 smithi195 ceph-mon[33844]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T05:34:08.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:07 smithi161 ceph-mon[33805]: pgmap v300: 129 pgs: 9 creating+peering, 17 unknown, 103 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:08.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:07 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/823539534' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-09T05:34:08.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:07 smithi161 ceph-mon[33805]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T05:34:09.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.546+0000 7f229d65a700 1 -- 172.21.15.115:0/1317674377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 msgr2=0x7f2298108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:09.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.546+0000 7f229d65a700 1 --2- 172.21.15.115:0/1317674377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 0x7f2298108e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f228800c210 tx=0x7f228800c520 comp rx=0 tx=0).stop 2024-05-09T05:34:09.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.546+0000 7f229d65a700 1 -- 172.21.15.115:0/1317674377 shutdown_connections 2024-05-09T05:34:09.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.546+0000 7f229d65a700 1 --2- 172.21.15.115:0/1317674377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2298103310 0x7f229810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:09.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.546+0000 7f229d65a700 1 --2- 172.21.15.115:0/1317674377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 0x7f2298108e00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:09.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.546+0000 7f229d65a700 1 --2- 172.21.15.115:0/1317674377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22981029c0 0x7f2298102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:09.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.546+0000 7f229d65a700 1 -- 172.21.15.115:0/1317674377 >> 172.21.15.115:0/1317674377 conn(0x7f22980fe6f0 msgr2=0x7f2298100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:09.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.546+0000 7f229d65a700 1 -- 172.21.15.115:0/1317674377 shutdown_connections 2024-05-09T05:34:09.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.546+0000 7f229d65a700 1 -- 172.21.15.115:0/1317674377 wait complete. 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.547+0000 7f229d65a700 1 Processor -- start 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.547+0000 7f229d65a700 1 -- start start 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f229d65a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22981029c0 0x7f229819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f229d65a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2298103310 0x7f229819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f229d65a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 0x7f229819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f229d65a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2298112190 con 0x7f22981089f0 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f229d65a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2298112010 con 0x7f22981029c0 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f229d65a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2298112310 con 0x7f2298103310 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22977fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 0x7f229819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:09.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22977fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 0x7f229819f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53464/0 (socket says 172.21.15.115:53464) 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22977fe700 1 -- 172.21.15.115:0/1963677773 learned_addr learned my addr 172.21.15.115:0/1963677773 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f2296ffd700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22981029c0 0x7f229819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22967fc700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2298103310 0x7f229819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22977fe700 1 -- 172.21.15.115:0/1963677773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2298103310 msgr2=0x7f229819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22977fe700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2298103310 0x7f229819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22977fe700 1 -- 172.21.15.115:0/1963677773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22981029c0 msgr2=0x7f229819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22977fe700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22981029c0 0x7f229819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22977fe700 1 -- 172.21.15.115:0/1963677773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229819fc30 con 0x7f22981089f0 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f22977fe700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 0x7f229819f590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f229000d980 tx=0x7f229000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:09.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f228ffff700 1 -- 172.21.15.115:0/1963677773 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290017070 con 0x7f22981089f0 2024-05-09T05:34:09.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229819ff20 con 0x7f22981089f0 2024-05-09T05:34:09.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f228ffff700 1 -- 172.21.15.115:0/1963677773 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2290005020 con 0x7f22981089f0 2024-05-09T05:34:09.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f228ffff700 1 -- 172.21.15.115:0/1963677773 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229000f980 con 0x7f22981089f0 2024-05-09T05:34:09.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.548+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22981a7840 con 0x7f22981089f0 2024-05-09T05:34:09.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.549+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22980512c0 con 0x7f22981089f0 2024-05-09T05:34:09.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.551+0000 7f228ffff700 1 -- 172.21.15.115:0/1963677773 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f2290004a30 con 0x7f22981089f0 2024-05-09T05:34:09.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.552+0000 7f228ffff700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f22840b2880 0x7f22840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:09.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.552+0000 7f2296ffd700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f22840b2880 0x7f22840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:09.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.552+0000 7f228ffff700 1 -- 172.21.15.115:0/1963677773 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f22900e2610 con 0x7f22981089f0 2024-05-09T05:34:09.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.553+0000 7f2296ffd700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f22840b2880 0x7f22840b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f228800af70 tx=0x7f228800c920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:09.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.556+0000 7f228ffff700 1 -- 172.21.15.115:0/1963677773 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2290020020 con 0x7f22981089f0 2024-05-09T05:34:09.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.808+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f229802e5f0 con 0x7f22981089f0 2024-05-09T05:34:09.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.836+0000 7f228ffff700 1 -- 172.21.15.115:0/1963677773 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f22900aa0b0 con 0x7f22981089f0 2024-05-09T05:34:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.839+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f22840b2880 msgr2=0x7f22840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.839+0000 7f229d65a700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f22840b2880 0x7f22840b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f228800af70 tx=0x7f228800c920 comp rx=0 tx=0).stop 2024-05-09T05:34:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.839+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 msgr2=0x7f229819f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.839+0000 7f229d65a700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 0x7f229819f590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f229000d980 tx=0x7f229000dc90 comp rx=0 tx=0).stop 2024-05-09T05:34:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.840+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 shutdown_connections 2024-05-09T05:34:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.840+0000 7f229d65a700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f22840b2880 0x7f22840b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.840+0000 7f229d65a700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22981029c0 0x7f229819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.840+0000 7f229d65a700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22981089f0 0x7f229819f590 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.840+0000 7f229d65a700 1 --2- 172.21.15.115:0/1963677773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2298103310 0x7f229819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.840+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 >> 172.21.15.115:0/1963677773 conn(0x7f22980fe6f0 msgr2=0x7f2298101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.840+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 shutdown_connections 2024-05-09T05:34:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:09.840+0000 7f229d65a700 1 -- 172.21.15.115:0/1963677773 wait complete. 2024-05-09T05:34:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:added data pool 4 to fsmap 2024-05-09T05:34:10.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:09 smithi195 ceph-mon[33844]: pgmap v302: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:10.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:09 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1963677773' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T05:34:10.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:09 smithi115 ceph-mon[30349]: pgmap v302: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:10.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:09 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1963677773' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T05:34:10.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:09 smithi161 ceph-mon[33805]: pgmap v302: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:10.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:09 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1963677773' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-09T05:34:10.628 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:34:10.870 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:10.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:10 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1963677773' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T05:34:10.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:10 smithi115 ceph-mon[30349]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T05:34:10.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:10 smithi115 ceph-mon[30349]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:34:11.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:10 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1963677773' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T05:34:11.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:10 smithi195 ceph-mon[33844]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T05:34:11.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:10 smithi195 ceph-mon[33844]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:34:11.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:10 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1963677773' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-09T05:34:11.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:10 smithi161 ceph-mon[33805]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T05:34:11.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:10 smithi161 ceph-mon[33805]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:34:12.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:11 smithi195 ceph-mon[33844]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:11 smithi195 ceph-mon[33844]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T05:34:12.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:11 smithi195 ceph-mon[33844]: Cluster is now healthy 2024-05-09T05:34:12.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:11 smithi115 ceph-mon[30349]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:12.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:11 smithi115 ceph-mon[30349]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T05:34:12.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:11 smithi115 ceph-mon[30349]: Cluster is now healthy 2024-05-09T05:34:12.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:11 smithi161 ceph-mon[33805]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:12.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:11 smithi161 ceph-mon[33805]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T05:34:12.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:11 smithi161 ceph-mon[33805]: Cluster is now healthy 2024-05-09T05:34:12.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.795+0000 7f6dd6548700 1 -- 172.21.15.115:0/3077108559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd0107e10 msgr2=0x7f6dd0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:12.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.795+0000 7f6dd6548700 1 --2- 172.21.15.115:0/3077108559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd0107e10 0x7f6dd0108220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40099c0 tx=0x7f6dc4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:12.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.795+0000 7f6dd6548700 1 -- 172.21.15.115:0/3077108559 shutdown_connections 2024-05-09T05:34:12.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.795+0000 7f6dd6548700 1 --2- 172.21.15.115:0/3077108559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd01034e0 0x7f6dd01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:12.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.795+0000 7f6dd6548700 1 --2- 172.21.15.115:0/3077108559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dd0108760 0x7f6dd01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:12.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.795+0000 7f6dd6548700 1 --2- 172.21.15.115:0/3077108559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd0107e10 0x7f6dd0108220 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:12.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.795+0000 7f6dd6548700 1 -- 172.21.15.115:0/3077108559 >> 172.21.15.115:0/3077108559 conn(0x7f6dd00fef30 msgr2=0x7f6dd0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:12.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.795+0000 7f6dd6548700 1 -- 172.21.15.115:0/3077108559 shutdown_connections 2024-05-09T05:34:12.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.795+0000 7f6dd6548700 1 -- 172.21.15.115:0/3077108559 wait complete. 2024-05-09T05:34:12.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.796+0000 7f6dd6548700 1 Processor -- start 2024-05-09T05:34:12.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.796+0000 7f6dd6548700 1 -- start start 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.796+0000 7f6dd6548700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd01034e0 0x7f6dd019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.796+0000 7f6dd6548700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd0107e10 0x7f6dd019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dd6548700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dd0108760 0x7f6dd019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dd6548700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0113290 con 0x7f6dd01034e0 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dd6548700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0113110 con 0x7f6dd0107e10 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dd6548700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0113410 con 0x7f6dd0108760 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd01034e0 0x7f6dd019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd01034e0 0x7f6dd019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34140/0 (socket says 172.21.15.115:34140) 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcffff700 1 -- 172.21.15.115:0/1232258115 learned_addr learned my addr 172.21.15.115:0/1232258115 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dd4ae5700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dd0108760 0x7f6dd019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcf7fe700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd0107e10 0x7f6dd019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcffff700 1 -- 172.21.15.115:0/1232258115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dd0108760 msgr2=0x7f6dd019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcffff700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dd0108760 0x7f6dd019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcffff700 1 -- 172.21.15.115:0/1232258115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd0107e10 msgr2=0x7f6dd019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcffff700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd0107e10 0x7f6dd019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcffff700 1 -- 172.21.15.115:0/1232258115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd019fce0 con 0x7f6dd01034e0 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcf7fe700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd0107e10 0x7f6dd019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcffff700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd01034e0 0x7f6dd019ae10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000b530 tx=0x7f6dc000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcd7fa700 1 -- 172.21.15.115:0/1232258115 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0015020 con 0x7f6dd01034e0 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcd7fa700 1 -- 172.21.15.115:0/1232258115 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dc0011cc0 con 0x7f6dd01034e0 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd019ffd0 con 0x7f6dd01034e0 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.797+0000 7f6dcd7fa700 1 -- 172.21.15.115:0/1232258115 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0010dc0 con 0x7f6dd01034e0 2024-05-09T05:34:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.798+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd01a78f0 con 0x7f6dd01034e0 2024-05-09T05:34:12.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.799+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd00512c0 con 0x7f6dd01034e0 2024-05-09T05:34:12.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.800+0000 7f6dcd7fa700 1 -- 172.21.15.115:0/1232258115 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6dc0018020 con 0x7f6dd01034e0 2024-05-09T05:34:12.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.801+0000 7f6dcd7fa700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6db80b2880 0x7f6db80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.801+0000 7f6dcf7fe700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6db80b2880 0x7f6db80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.801+0000 7f6dcd7fa700 1 -- 172.21.15.115:0/1232258115 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f6dc00e2270 con 0x7f6dd01034e0 2024-05-09T05:34:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.801+0000 7f6dcf7fe700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6db80b2880 0x7f6db80b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4000c00 tx=0x7f6dc4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:12.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:12.805+0000 7f6dcd7fa700 1 -- 172.21.15.115:0/1232258115 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6dc001f020 con 0x7f6dd01034e0 2024-05-09T05:34:13.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.053+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dd019c170 con 0x7f6dd01034e0 2024-05-09T05:34:13.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.054+0000 7f6dcd7fa700 1 -- 172.21.15.115:0/1232258115 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11818 (secure 0 0 0) 0x7f6dc00a9d50 con 0x7f6dd01034e0 2024-05-09T05:34:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:34:13.049 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:34:09.810273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24469,"qdb_cluster":[24469]},"id":1}]} 2024-05-09T05:34:13.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.056+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6db80b2880 msgr2=0x7f6db80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:13.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.056+0000 7f6dd6548700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6db80b2880 0x7f6db80b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4000c00 tx=0x7f6dc4005fb0 comp rx=0 tx=0).stop 2024-05-09T05:34:13.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.056+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd01034e0 msgr2=0x7f6dd019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:13.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.056+0000 7f6dd6548700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd01034e0 0x7f6dd019ae10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000b530 tx=0x7f6dc000b840 comp rx=0 tx=0).stop 2024-05-09T05:34:13.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.057+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 shutdown_connections 2024-05-09T05:34:13.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.057+0000 7f6dd6548700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6db80b2880 0x7f6db80b4d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:13.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.057+0000 7f6dd6548700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dd0108760 0x7f6dd019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:13.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.057+0000 7f6dd6548700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6dd01034e0 0x7f6dd019ae10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:13.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.057+0000 7f6dd6548700 1 --2- 172.21.15.115:0/1232258115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd0107e10 0x7f6dd019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:13.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.057+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 >> 172.21.15.115:0/1232258115 conn(0x7f6dd00fef30 msgr2=0x7f6dd006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:13.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.057+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 shutdown_connections 2024-05-09T05:34:13.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:13.057+0000 7f6dd6548700 1 -- 172.21.15.115:0/1232258115 wait complete. 2024-05-09T05:34:13.052 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 17 2024-05-09T05:34:13.879 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:34:14.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:13 smithi195 ceph-mon[33844]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:14.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:13 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1232258115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:34:14.119 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:14.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:13 smithi115 ceph-mon[30349]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:14.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:13 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1232258115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:34:14.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:13 smithi161 ceph-mon[33805]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:14.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:13 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1232258115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:34:15.921 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:15 smithi115 ceph-mon[30349]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.040+0000 7f3603a4c700 1 -- 172.21.15.115:0/2867099317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 msgr2=0x7f35fc108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.040+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2867099317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 0x7f35fc108230 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0099c0 tx=0x7f35ec009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.040+0000 7f3603a4c700 1 -- 172.21.15.115:0/2867099317 shutdown_connections 2024-05-09T05:34:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.040+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2867099317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35fc108770 0x7f35fc1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.040+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2867099317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35fc103540 0x7f35fc103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.040+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2867099317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 0x7f35fc108230 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.040+0000 7f3603a4c700 1 -- 172.21.15.115:0/2867099317 >> 172.21.15.115:0/2867099317 conn(0x7f35fc078e50 msgr2=0x7f35fc079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.040+0000 7f3603a4c700 1 -- 172.21.15.115:0/2867099317 shutdown_connections 2024-05-09T05:34:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.040+0000 7f3603a4c700 1 -- 172.21.15.115:0/2867099317 wait complete. 2024-05-09T05:34:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3603a4c700 1 Processor -- start 2024-05-09T05:34:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3603a4c700 1 -- start start 2024-05-09T05:34:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3603a4c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35fc103540 0x7f35fc19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3603a4c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 0x7f35fc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3603a4c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35fc108770 0x7f35fc19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3603a4c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc113230 con 0x7f35fc107e20 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3603a4c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1130b0 con 0x7f35fc103540 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3603a4c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1133b0 con 0x7f35fc108770 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3600fe7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 0x7f35fc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3600fe7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 0x7f35fc19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34166/0 (socket says 172.21.15.115:34166) 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3600fe7700 1 -- 172.21.15.115:0/2356346946 learned_addr learned my addr 172.21.15.115:0/2356346946 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f36017e8700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35fc103540 0x7f35fc19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3601fe9700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35fc108770 0x7f35fc19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3600fe7700 1 -- 172.21.15.115:0/2356346946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35fc108770 msgr2=0x7f35fc19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3600fe7700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35fc108770 0x7f35fc19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3600fe7700 1 -- 172.21.15.115:0/2356346946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35fc103540 msgr2=0x7f35fc19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3600fe7700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35fc103540 0x7f35fc19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.041+0000 7f3600fe7700 1 -- 172.21.15.115:0/2356346946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc19fc60 con 0x7f35fc107e20 2024-05-09T05:34:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.042+0000 7f3601fe9700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35fc108770 0x7f35fc19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:34:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.042+0000 7f3600fe7700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 0x7f35fc19b330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0045e0 tx=0x7f35ec004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.042+0000 7f35f27fc700 1 -- 172.21.15.115:0/2356346946 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec026070 con 0x7f35fc107e20 2024-05-09T05:34:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.042+0000 7f35f27fc700 1 -- 172.21.15.115:0/2356346946 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35ec00f460 con 0x7f35fc107e20 2024-05-09T05:34:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.042+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35fc19fef0 con 0x7f35fc107e20 2024-05-09T05:34:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.042+0000 7f35f27fc700 1 -- 172.21.15.115:0/2356346946 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec009e60 con 0x7f35fc107e20 2024-05-09T05:34:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.042+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35fc1a03d0 con 0x7f35fc107e20 2024-05-09T05:34:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.043+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35fc0512c0 con 0x7f35fc107e20 2024-05-09T05:34:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.046+0000 7f35f27fc700 1 -- 172.21.15.115:0/2356346946 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f35ec029070 con 0x7f35fc107e20 2024-05-09T05:34:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.046+0000 7f35f27fc700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f35e80b2880 0x7f35e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.046+0000 7f36017e8700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f35e80b2880 0x7f35e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.046+0000 7f35f27fc700 1 -- 172.21.15.115:0/2356346946 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f35ec0e2fa0 con 0x7f35fc107e20 2024-05-09T05:34:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.047+0000 7f36017e8700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f35e80b2880 0x7f35e80b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005950 tx=0x7f35f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:16.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.050+0000 7f35f27fc700 1 -- 172.21.15.115:0/2356346946 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f35ec013070 con 0x7f35fc107e20 2024-05-09T05:34:16.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:15 smithi195 ceph-mon[33844]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:16.162 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:15 smithi161 ceph-mon[33805]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:16.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.257+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35fc02e5f0 con 0x7f35fc107e20 2024-05-09T05:34:16.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.259+0000 7f35f27fc700 1 -- 172.21.15.115:0/2356346946 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24198 (secure 0 0 0) 0x7f35ec0aab10 con 0x7f35fc107e20 2024-05-09T05:34:16.254 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:34:16.254 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":85,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:09.810128+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.261+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f35e80b2880 msgr2=0x7f35e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.261+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f35e80b2880 0x7f35e80b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005950 tx=0x7f35f800b410 comp rx=0 tx=0).stop 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.261+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 msgr2=0x7f35fc19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.261+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 0x7f35fc19b330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0045e0 tx=0x7f35ec004610 comp rx=0 tx=0).stop 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.261+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 shutdown_connections 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.262+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f35e80b2880 0x7f35e80b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.262+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35fc103540 0x7f35fc19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.262+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35fc108770 0x7f35fc19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.262+0000 7f3603a4c700 1 --2- 172.21.15.115:0/2356346946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35fc107e20 0x7f35fc19b330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:16.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.262+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 >> 172.21.15.115:0/2356346946 conn(0x7f35fc078e50 msgr2=0x7f35fc075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:16.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.262+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 shutdown_connections 2024-05-09T05:34:16.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:16.262+0000 7f3603a4c700 1 -- 172.21.15.115:0/2356346946 wait complete. 2024-05-09T05:34:16.897 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:16 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2356346946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:16.961 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:34:17.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:16 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2356346946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:17.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:16 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2356346946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:17.213 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:17.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:17 smithi115 ceph-mon[30349]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:18.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:17 smithi195 ceph-mon[33844]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:18.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:17 smithi161 ceph-mon[33805]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:19.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.124+0000 7efd18c27700 1 -- 172.21.15.115:0/4154143888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 msgr2=0x7efd141049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:19.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.124+0000 7efd18c27700 1 --2- 172.21.15.115:0/4154143888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 0x7efd141049f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd040099c0 tx=0x7efd04009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.124+0000 7efd18c27700 1 -- 172.21.15.115:0/4154143888 shutdown_connections 2024-05-09T05:34:19.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.124+0000 7efd18c27700 1 --2- 172.21.15.115:0/4154143888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd14109860 0x7efd141111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.124+0000 7efd18c27700 1 --2- 172.21.15.115:0/4154143888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 0x7efd141049f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.124+0000 7efd18c27700 1 --2- 172.21.15.115:0/4154143888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd14108f10 0x7efd14109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.124+0000 7efd18c27700 1 -- 172.21.15.115:0/4154143888 >> 172.21.15.115:0/4154143888 conn(0x7efd140ffe70 msgr2=0x7efd141022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:19.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.124+0000 7efd18c27700 1 -- 172.21.15.115:0/4154143888 shutdown_connections 2024-05-09T05:34:19.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.124+0000 7efd18c27700 1 -- 172.21.15.115:0/4154143888 wait complete. 2024-05-09T05:34:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.125+0000 7efd18c27700 1 Processor -- start 2024-05-09T05:34:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.125+0000 7efd18c27700 1 -- start start 2024-05-09T05:34:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.125+0000 7efd18c27700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 0x7efd1419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.125+0000 7efd18c27700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd14108f10 0x7efd1419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.125+0000 7efd18c27700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd14109860 0x7efd1419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.125+0000 7efd18c27700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd14114430 con 0x7efd141045e0 2024-05-09T05:34:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.125+0000 7efd18c27700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efd141142b0 con 0x7efd14109860 2024-05-09T05:34:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.125+0000 7efd18c27700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efd141145b0 con 0x7efd14108f10 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd1259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 0x7efd1419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd1259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 0x7efd1419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34190/0 (socket says 172.21.15.115:34190) 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd1259c700 1 -- 172.21.15.115:0/1390561743 learned_addr learned my addr 172.21.15.115:0/1390561743 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd11d9b700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd14108f10 0x7efd1419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd12d9d700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd14109860 0x7efd1419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd1259c700 1 -- 172.21.15.115:0/1390561743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd14108f10 msgr2=0x7efd1419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd1259c700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd14108f10 0x7efd1419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd1259c700 1 -- 172.21.15.115:0/1390561743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd14109860 msgr2=0x7efd1419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd1259c700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd14109860 0x7efd1419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd1259c700 1 -- 172.21.15.115:0/1390561743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1419fcf0 con 0x7efd141045e0 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd12d9d700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd14109860 0x7efd1419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd1259c700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 0x7efd1419ae20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd04000c00 tx=0x7efd0400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd037fe700 1 -- 172.21.15.115:0/1390561743 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04026070 con 0x7efd141045e0 2024-05-09T05:34:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd037fe700 1 -- 172.21.15.115:0/1390561743 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd04005030 con 0x7efd141045e0 2024-05-09T05:34:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1419ff80 con 0x7efd141045e0 2024-05-09T05:34:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd141a0460 con 0x7efd141045e0 2024-05-09T05:34:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.126+0000 7efd037fe700 1 -- 172.21.15.115:0/1390561743 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04004b70 con 0x7efd141045e0 2024-05-09T05:34:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.128+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd140512c0 con 0x7efd141045e0 2024-05-09T05:34:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.129+0000 7efd037fe700 1 -- 172.21.15.115:0/1390561743 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7efd04029070 con 0x7efd141045e0 2024-05-09T05:34:19.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.130+0000 7efd037fe700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efcfc0b2880 0x7efcfc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:19.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.130+0000 7efd11d9b700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efcfc0b2880 0x7efcfc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:19.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.130+0000 7efd037fe700 1 -- 172.21.15.115:0/1390561743 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7efd040e3390 con 0x7efd141045e0 2024-05-09T05:34:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.130+0000 7efd11d9b700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efcfc0b2880 0x7efcfc0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efd14107860 tx=0x7efd08009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:19.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.134+0000 7efd037fe700 1 -- 172.21.15.115:0/1390561743 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd04030080 con 0x7efd141045e0 2024-05-09T05:34:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.339+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd1402e5f0 con 0x7efd141045e0 2024-05-09T05:34:19.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.340+0000 7efd037fe700 1 -- 172.21.15.115:0/1390561743 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24198 (secure 0 0 0) 0x7efd04013070 con 0x7efd141045e0 2024-05-09T05:34:19.335 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:34:19.335 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":85,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:09.810128+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:34:19.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efcfc0b2880 msgr2=0x7efcfc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:19.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efcfc0b2880 0x7efcfc0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efd14107860 tx=0x7efd08009190 comp rx=0 tx=0).stop 2024-05-09T05:34:19.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 msgr2=0x7efd1419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:19.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 0x7efd1419ae20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd04000c00 tx=0x7efd0400f690 comp rx=0 tx=0).stop 2024-05-09T05:34:19.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 shutdown_connections 2024-05-09T05:34:19.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7efcfc0b2880 0x7efcfc0b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd14109860 0x7efd1419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd141045e0 0x7efd1419ae20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 --2- 172.21.15.115:0/1390561743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd14108f10 0x7efd1419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:19.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 >> 172.21.15.115:0/1390561743 conn(0x7efd140ffe70 msgr2=0x7efd1410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:19.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 shutdown_connections 2024-05-09T05:34:19.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:19.343+0000 7efd18c27700 1 -- 172.21.15.115:0/1390561743 wait complete. 2024-05-09T05:34:20.111 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:19 smithi115 ceph-mon[30349]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:20.111 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:19 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1390561743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:20.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:19 smithi195 ceph-mon[33844]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:20.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:19 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1390561743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:20.153 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:34:20.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:19 smithi161 ceph-mon[33805]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:20.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:19 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1390561743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:20.395 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:22.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:21 smithi195 ceph-mon[33844]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:22.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:21 smithi115 ceph-mon[30349]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:22.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:21 smithi161 ceph-mon[33805]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:22.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.266+0000 7f550580c700 1 -- 172.21.15.115:0/2588213042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500108780 msgr2=0x7f55001100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:22.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.266+0000 7f550580c700 1 --2- 172.21.15.115:0/2588213042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500108780 0x7f55001100e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f54f80099c0 tx=0x7f54f8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.266+0000 7f550580c700 1 -- 172.21.15.115:0/2588213042 shutdown_connections 2024-05-09T05:34:22.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.266+0000 7f550580c700 1 --2- 172.21.15.115:0/2588213042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500108780 0x7f55001100e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.266+0000 7f550580c700 1 --2- 172.21.15.115:0/2588213042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5500103550 0x7f5500103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.266+0000 7f550580c700 1 --2- 172.21.15.115:0/2588213042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5500107e30 0x7f5500108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.266+0000 7f550580c700 1 -- 172.21.15.115:0/2588213042 >> 172.21.15.115:0/2588213042 conn(0x7f5500078e00 msgr2=0x7f5500079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:22.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.266+0000 7f550580c700 1 -- 172.21.15.115:0/2588213042 shutdown_connections 2024-05-09T05:34:22.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.266+0000 7f550580c700 1 -- 172.21.15.115:0/2588213042 wait complete. 2024-05-09T05:34:22.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f550580c700 1 Processor -- start 2024-05-09T05:34:22.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f550580c700 1 -- start start 2024-05-09T05:34:22.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f550580c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500103550 0x7f550019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:22.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f550580c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5500107e30 0x7f550019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f550580c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5500108780 0x7f5500199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f550580c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5500113260 con 0x7f5500103550 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f550580c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55001130e0 con 0x7f5500107e30 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f550580c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f55001133e0 con 0x7f5500108780 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f54feffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500103550 0x7f550019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f54feffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500103550 0x7f550019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34214/0 (socket says 172.21.15.115:34214) 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f54feffd700 1 -- 172.21.15.115:0/2621132241 learned_addr learned my addr 172.21.15.115:0/2621132241 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f54fe7fc700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5500107e30 0x7f550019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.267+0000 7f54ff7fe700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5500108780 0x7f5500199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f54feffd700 1 -- 172.21.15.115:0/2621132241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5500108780 msgr2=0x7f5500199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f54feffd700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5500108780 0x7f5500199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f54feffd700 1 -- 172.21.15.115:0/2621132241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5500107e30 msgr2=0x7f550019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f54feffd700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5500107e30 0x7f550019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f54feffd700 1 -- 172.21.15.115:0/2621132241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550019a0d0 con 0x7f5500103550 2024-05-09T05:34:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f54feffd700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500103550 0x7f550019f800 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f54f00060b0 tx=0x7f54f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f550480a700 1 -- 172.21.15.115:0/2621132241 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f001a070 con 0x7f5500103550 2024-05-09T05:34:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f550480a700 1 -- 172.21.15.115:0/2621132241 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54f000dd20 con 0x7f5500103550 2024-05-09T05:34:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550019a3c0 con 0x7f5500103550 2024-05-09T05:34:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f550480a700 1 -- 172.21.15.115:0/2621132241 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0014600 con 0x7f5500103550 2024-05-09T05:34:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.268+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55001a6690 con 0x7f5500103550 2024-05-09T05:34:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.269+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55000512c0 con 0x7f5500103550 2024-05-09T05:34:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.271+0000 7f550480a700 1 -- 172.21.15.115:0/2621132241 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f54f001d070 con 0x7f5500103550 2024-05-09T05:34:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.271+0000 7f550480a700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f54e80b2900 0x7f54e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.272+0000 7f54fe7fc700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f54e80b2900 0x7f54e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.272+0000 7f550480a700 1 -- 172.21.15.115:0/2621132241 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f54f00e1d70 con 0x7f5500103550 2024-05-09T05:34:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.272+0000 7f54fe7fc700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f54e80b2900 0x7f54e80b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f550006b9e0 tx=0x7f54f400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:22.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.275+0000 7f550480a700 1 -- 172.21.15.115:0/2621132241 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54f0010070 con 0x7f5500103550 2024-05-09T05:34:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.525+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f550019af50 con 0x7f5500103550 2024-05-09T05:34:22.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.526+0000 7f550480a700 1 -- 172.21.15.115:0/2621132241 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11818 (secure 0 0 0) 0x7f54f00a9200 con 0x7f5500103550 2024-05-09T05:34:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:34:22.521 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:34:09.810273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24469,"qdb_cluster":[24469]},"id":1}]} 2024-05-09T05:34:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f54e80b2900 msgr2=0x7f54e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f54e80b2900 0x7f54e80b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f550006b9e0 tx=0x7f54f400b410 comp rx=0 tx=0).stop 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500103550 msgr2=0x7f550019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500103550 0x7f550019f800 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f54f00060b0 tx=0x7f54f000d3b0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 shutdown_connections 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f54e80b2900 0x7f54e80b4e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5500108780 0x7f5500199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5500103550 0x7f550019f800 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 --2- 172.21.15.115:0/2621132241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5500107e30 0x7f550019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 >> 172.21.15.115:0/2621132241 conn(0x7f5500078e00 msgr2=0x7f5500100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 shutdown_connections 2024-05-09T05:34:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:22.529+0000 7f550580c700 1 -- 172.21.15.115:0/2621132241 wait complete. 2024-05-09T05:34:22.525 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 17 2024-05-09T05:34:23.116 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:22 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2621132241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:34:23.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:22 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2621132241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:34:23.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:22 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2621132241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:34:23.357 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:34:23.602 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:24.065 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:23 smithi115 ceph-mon[30349]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:24.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:23 smithi161 ceph-mon[33805]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:24.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:23 smithi195 ceph-mon[33844]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:25.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.321+0000 7f47ba70b700 1 -- 172.21.15.115:0/1083002259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 msgr2=0x7f47b4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:25.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.321+0000 7f47ba70b700 1 --2- 172.21.15.115:0/1083002259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 0x7f47b4108200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f47a80099c0 tx=0x7f47a8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.321+0000 7f47ba70b700 1 -- 172.21.15.115:0/1083002259 shutdown_connections 2024-05-09T05:34:25.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.321+0000 7f47ba70b700 1 --2- 172.21.15.115:0/1083002259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47b4108740 0x7f47b41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.321+0000 7f47ba70b700 1 --2- 172.21.15.115:0/1083002259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b41034c0 0x7f47b41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.321+0000 7f47ba70b700 1 --2- 172.21.15.115:0/1083002259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 0x7f47b4108200 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.321+0000 7f47ba70b700 1 -- 172.21.15.115:0/1083002259 >> 172.21.15.115:0/1083002259 conn(0x7f47b40fedc0 msgr2=0x7f47b41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:25.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.321+0000 7f47ba70b700 1 -- 172.21.15.115:0/1083002259 shutdown_connections 2024-05-09T05:34:25.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.321+0000 7f47ba70b700 1 -- 172.21.15.115:0/1083002259 wait complete. 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47ba70b700 1 Processor -- start 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47ba70b700 1 -- start start 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47ba70b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b41034c0 0x7f47b4100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47ba70b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 0x7f47b40ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47ba70b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47b4108740 0x7f47b40ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47ba70b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113290 con 0x7f47b4107df0 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47ba70b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113110 con 0x7f47b4108740 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47ba70b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113410 con 0x7f47b41034c0 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47b37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 0x7f47b40ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:25.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47b37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 0x7f47b40ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45436/0 (socket says 172.21.15.115:45436) 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.322+0000 7f47b37fe700 1 -- 172.21.15.115:0/630481023 learned_addr learned my addr 172.21.15.115:0/630481023 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b3fff700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b41034c0 0x7f47b4100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b8ca8700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47b4108740 0x7f47b40ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b37fe700 1 -- 172.21.15.115:0/630481023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b41034c0 msgr2=0x7f47b4100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b37fe700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b41034c0 0x7f47b4100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b37fe700 1 -- 172.21.15.115:0/630481023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47b4108740 msgr2=0x7f47b40ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b37fe700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47b4108740 0x7f47b40ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b37fe700 1 -- 172.21.15.115:0/630481023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b41000b0 con 0x7f47b4107df0 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b37fe700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 0x7f47b40ff2a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f47a800b580 tx=0x7f47a800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b17fa700 1 -- 172.21.15.115:0/630481023 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8026070 con 0x7f47b4107df0 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b17fa700 1 -- 172.21.15.115:0/630481023 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47a8005030 con 0x7f47b4107df0 2024-05-09T05:34:25.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47b17fa700 1 -- 172.21.15.115:0/630481023 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a80049f0 con 0x7f47b4107df0 2024-05-09T05:34:25.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b41ad840 con 0x7f47b4107df0 2024-05-09T05:34:25.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.323+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b41adc40 con 0x7f47b4107df0 2024-05-09T05:34:25.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.324+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b4104970 con 0x7f47b4107df0 2024-05-09T05:34:25.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.326+0000 7f47b17fa700 1 -- 172.21.15.115:0/630481023 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f47a8029070 con 0x7f47b4107df0 2024-05-09T05:34:25.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.327+0000 7f47b17fa700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f479c0b2880 0x7f479c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:25.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.327+0000 7f47b3fff700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f479c0b2880 0x7f479c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:25.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.327+0000 7f47b17fa700 1 -- 172.21.15.115:0/630481023 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f47a80e32b0 con 0x7f47b4107df0 2024-05-09T05:34:25.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.327+0000 7f47b3fff700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f479c0b2880 0x7f479c0b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f47a4005950 tx=0x7f47a40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:25.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.330+0000 7f47b17fa700 1 -- 172.21.15.115:0/630481023 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47a8030080 con 0x7f47b4107df0 2024-05-09T05:34:25.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.543+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47b41adf10 con 0x7f47b4107df0 2024-05-09T05:34:25.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.545+0000 7f47b17fa700 1 -- 172.21.15.115:0/630481023 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24198 (secure 0 0 0) 0x7f47a8013070 con 0x7f47b4107df0 2024-05-09T05:34:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:34:25.539 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":85,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:09.810128+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f479c0b2880 msgr2=0x7f479c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f479c0b2880 0x7f479c0b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f47a4005950 tx=0x7f47a40058e0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 msgr2=0x7f47b40ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 0x7f47b40ff2a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f47a800b580 tx=0x7f47a800f510 comp rx=0 tx=0).stop 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 shutdown_connections 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f479c0b2880 0x7f479c0b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47b4108740 0x7f47b40ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47b41034c0 0x7f47b4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 --2- 172.21.15.115:0/630481023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b4107df0 0x7f47b40ff2a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 >> 172.21.15.115:0/630481023 conn(0x7f47b40fedc0 msgr2=0x7f47b41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 shutdown_connections 2024-05-09T05:34:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:25.547+0000 7f47ba70b700 1 -- 172.21.15.115:0/630481023 wait complete. 2024-05-09T05:34:25.955 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:25 smithi115 ceph-mon[30349]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:25.955 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:25 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/630481023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:26.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:25 smithi161 ceph-mon[33805]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:26.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/630481023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:26.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T05:33:54.732251+0000', 'modified': '2024-05-09T05:34:09.810273+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.195:6839/1691623113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 1691623113}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 1691623113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24469, 'qdb_cluster': [24469]} 2024-05-09T05:34:26.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T05:34:26.284 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T05:34:26.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:25 smithi195 ceph-mon[33844]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:26.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:25 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/630481023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:26.528 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:27.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:27 smithi115 ceph-mon[30349]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:28.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:27 smithi161 ceph-mon[33805]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:28.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.227+0000 7faabd29a700 1 -- 172.21.15.115:0/3068561593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8107e30 msgr2=0x7faab8108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:28.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.227+0000 7faabd29a700 1 --2- 172.21.15.115:0/3068561593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8107e30 0x7faab8108240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faaa00099c0 tx=0x7faaa0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.227+0000 7faabd29a700 1 -- 172.21.15.115:0/3068561593 shutdown_connections 2024-05-09T05:34:28.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.227+0000 7faabd29a700 1 --2- 172.21.15.115:0/3068561593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faab8103550 0x7faab8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.227+0000 7faabd29a700 1 --2- 172.21.15.115:0/3068561593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab8108780 0x7faab81100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.227+0000 7faabd29a700 1 --2- 172.21.15.115:0/3068561593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8107e30 0x7faab8108240 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.227+0000 7faabd29a700 1 -- 172.21.15.115:0/3068561593 >> 172.21.15.115:0/3068561593 conn(0x7faab8078e00 msgr2=0x7faab8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:28.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.227+0000 7faabd29a700 1 -- 172.21.15.115:0/3068561593 shutdown_connections 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.227+0000 7faabd29a700 1 -- 172.21.15.115:0/3068561593 wait complete. 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.228+0000 7faabd29a700 1 Processor -- start 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.228+0000 7faabd29a700 1 -- start start 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.228+0000 7faabd29a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faab8103550 0x7faab819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.228+0000 7faabd29a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab8107e30 0x7faab819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.228+0000 7faabd29a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8108780 0x7faab819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.228+0000 7faabd29a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faab81132b0 con 0x7faab8108780 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.228+0000 7faabd29a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faab8113130 con 0x7faab8103550 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.228+0000 7faabd29a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faab8113430 con 0x7faab8107e30 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8108780 0x7faab819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8108780 0x7faab819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45450/0 (socket says 172.21.15.115:45450) 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab77fe700 1 -- 172.21.15.115:0/1284230003 learned_addr learned my addr 172.21.15.115:0/1284230003 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab6ffd700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faab8103550 0x7faab819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faaaffff700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab8107e30 0x7faab819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab77fe700 1 -- 172.21.15.115:0/1284230003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab8107e30 msgr2=0x7faab819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab77fe700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab8107e30 0x7faab819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab77fe700 1 -- 172.21.15.115:0/1284230003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faab8103550 msgr2=0x7faab819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab77fe700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faab8103550 0x7faab819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab77fe700 1 -- 172.21.15.115:0/1284230003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab819fc60 con 0x7faab8108780 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab77fe700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8108780 0x7faab819f670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faab000b970 tx=0x7faab000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab4ff9700 1 -- 172.21.15.115:0/1284230003 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab000d860 con 0x7faab8108780 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab4ff9700 1 -- 172.21.15.115:0/1284230003 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faab000e430 con 0x7faab8108780 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faabd29a700 1 -- 172.21.15.115:0/1284230003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab819ff50 con 0x7faab8108780 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faab4ff9700 1 -- 172.21.15.115:0/1284230003 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0017660 con 0x7faab8108780 2024-05-09T05:34:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.229+0000 7faabd29a700 1 -- 172.21.15.115:0/1284230003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab81a0410 con 0x7faab8108780 2024-05-09T05:34:28.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.233+0000 7faab4ff9700 1 -- 172.21.15.115:0/1284230003 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7faab000da00 con 0x7faab8108780 2024-05-09T05:34:28.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.233+0000 7faab4ff9700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faaa40b2880 0x7faaa40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faab6ffd700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faaa40b2880 0x7faaa40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faacbcb0b80 1 Processor -- start 2024-05-09T05:34:28.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faab4ff9700 1 -- 172.21.15.115:0/1284230003 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7faab00e2080 con 0x7faab8108780 2024-05-09T05:34:28.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faacbcb0b80 1 -- start start 2024-05-09T05:34:28.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faab6ffd700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faaa40b2880 0x7faaa40b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faaa800ac50 tx=0x7faaa800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:28.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238e82110 0x556238f1b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238d98450 0x556238f1b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f15a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556238e37fa0 con 0x556238f15650 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556238dd67a0 con 0x556238d98450 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556238e33f60 con 0x556238e82110 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faab77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f15a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faab77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f15a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45454/0 (socket says 172.21.15.115:45454) 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.234+0000 7faab77fe700 1 -- 172.21.15.115:0/4247770864 learned_addr learned my addr 172.21.15.115:0/4247770864 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faab6ffd700 1 --2- 172.21.15.115:0/4247770864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238e82110 0x556238f1b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faaaffff700 1 --2- 172.21.15.115:0/4247770864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238d98450 0x556238f1b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faab77fe700 1 -- 172.21.15.115:0/4247770864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238e82110 msgr2=0x556238f1b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:34:28.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faab77fe700 1 --2- 172.21.15.115:0/4247770864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238e82110 0x556238f1b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faab77fe700 1 -- 172.21.15.115:0/4247770864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238d98450 msgr2=0x556238f1b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faab77fe700 1 --2- 172.21.15.115:0/4247770864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238d98450 0x556238f1b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faab77fe700 1 -- 172.21.15.115:0/4247770864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556238dce320 con 0x556238f15650 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faab77fe700 1 --2- 172.21.15.115:0/4247770864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f15a80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faab00a93f0 tx=0x7faab00a9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbee0f2aa49737fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faaad7fa700 1 -- 172.21.15.115:0/4247770864 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00b22c0 con 0x556238f15650 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faaad7fa700 1 -- 172.21.15.115:0/4247770864 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faab00a5a50 con 0x556238f15650 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.235+0000 7faaad7fa700 1 -- 172.21.15.115:0/4247770864 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0015300 con 0x556238f15650 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4247770864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 msgr2=0x556238f15a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4247770864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f15a80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faab00a93f0 tx=0x7faab00a9b60 comp rx=0 tx=0).stop 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4247770864 shutdown_connections 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4247770864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f15a80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4247770864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238e82110 0x556238f1b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4247770864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238d98450 0x556238f1b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4247770864 >> 172.21.15.115:0/4247770864 conn(0x556238df8650 msgr2=0x556238f103b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4247770864 shutdown_connections 2024-05-09T05:34:28.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4247770864 wait complete. 2024-05-09T05:34:28.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.236+0000 7faacbcb0b80 1 -- start start 2024-05-09T05:34:28.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faacbcb0b80 1 Processor -- start 2024-05-09T05:34:28.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f1f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f1fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556238cb3b50 con 0x556238f15650 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556238df2100 con 0x556238e82110 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556238f22180 con 0x556238d98450 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faacbcb0b80 10 client.0 _ll_register_callbacks cb 0x556238ca9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f1fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f1fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45470/0 (socket says 172.21.15.115:45470) 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab77fe700 1 -- 172.21.15.115:0/458937690 learned_addr learned my addr 172.21.15.115:0/458937690 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faaaffff700 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f1f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab6ffd700 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab77fe700 1 -- 172.21.15.115:0/458937690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 msgr2=0x556238f1f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab77fe700 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f1f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab77fe700 1 -- 172.21.15.115:0/458937690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 msgr2=0x556238f1f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:34:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab77fe700 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f1f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab77fe700 1 -- 172.21.15.115:0/458937690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556238cb3440 con 0x556238f15650 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab6ffd700 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f1f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.237+0000 7faab77fe700 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f1fba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faab00a5490 tx=0x7faab0091120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6125046385f3db27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faa9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faa9e7fc700 1 -- 172.21.15.115:0/458937690 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab000dcf0 con 0x556238f15650 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faa9e7fc700 1 -- 172.21.15.115:0/458937690 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7faab0015300 con 0x556238f15650 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faacbcb0b80 1 -- 172.21.15.115:0/458937690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab00a5a50 con 0x556238f15650 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faa9e7fc700 1 -- 172.21.15.115:0/458937690 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7faab0015500 con 0x556238f15650 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faacbcb0b80 10 client.14727 fetch_fsmap learned FSMap version 17 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faacbcb0b80 1 -- 172.21.15.115:0/458937690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556238f2cbc0 con 0x556238f15650 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faa9e7fc700 1 -- 172.21.15.115:0/458937690 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faab00a5a50 con 0x556238f15650 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faa9e7fc700 1 -- 172.21.15.115:0/458937690 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7faab00b1ea0 con 0x556238f15650 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faacbcb0b80 10 client.14727 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faacbcb0b80 10 client.14727 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T05:34:28.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faacbcb0b80 20 client.14727 populate_metadata read hostname 'smithi115' 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556238f2df10 0x556238f30100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faacbcb0b80 4 client.14727 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.238+0000 7faacbcb0b80 1 -- 172.21.15.115:0/458937690 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556238e45980 con 0x556238f2df10 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.239+0000 7faab6ffd700 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556238f2df10 0x556238f30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.239+0000 7faa9e7fc700 10 client.14727 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.239+0000 7faab6ffd700 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556238f2df10 0x556238f30100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e713a4acd8f59e86 server_cookie=3ec2f2b6e68e9193 in_seq=0 out_seq=0 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faa9e7fc700 1 -- 172.21.15.115:0/458937690 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556238e45980 con 0x556238f2df10 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faa9e7fc700 10 client.14727 handle_command_reply: tid=0 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 client.14727 shutdown 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 -- 172.21.15.115:0/458937690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 msgr2=0x556238f1fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f1fba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faab00a5490 tx=0x7faab0091120 comp rx=0 tx=0).stop 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 -- 172.21.15.115:0/458937690 shutdown_connections 2024-05-09T05:34:28.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556238f2df10 0x556238f30100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faa9e7fc700 0 client.14727 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:34:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f1fba0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f1f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/458937690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f1f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 -- 172.21.15.115:0/458937690 >> 172.21.15.115:0/458937690 conn(0x556238df8650 msgr2=0x556238f10170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 -- 172.21.15.115:0/458937690 shutdown_connections 2024-05-09T05:34:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.240+0000 7faacbcb0b80 1 -- 172.21.15.115:0/458937690 wait complete. 2024-05-09T05:34:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.241+0000 7faacbcb0b80 20 client.14727 trim_cache size 0 max 16384 2024-05-09T05:34:28.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.261+0000 7faacbcb0b80 1 Processor -- start 2024-05-09T05:34:28.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.261+0000 7faacbcb0b80 1 -- start start 2024-05-09T05:34:28.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.261+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f2b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.261+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f1e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556238f2e440 con 0x556238f15650 2024-05-09T05:34:28.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556238f2eb10 con 0x556238e82110 2024-05-09T05:34:28.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556238f2f1e0 con 0x556238d98450 2024-05-09T05:34:28.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f194c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45474/0 (socket says 172.21.15.115:45474) 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab77fe700 1 -- 172.21.15.115:0/451283200 learned_addr learned my addr 172.21.15.115:0/451283200 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab6ffd700 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f2b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faaaffff700 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f1e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab77fe700 1 -- 172.21.15.115:0/451283200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 msgr2=0x556238f2b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab77fe700 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f2b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab77fe700 1 -- 172.21.15.115:0/451283200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 msgr2=0x556238f1e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab77fe700 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f1e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab77fe700 1 -- 172.21.15.115:0/451283200 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556238f11290 con 0x556238f15650 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faaaffff700 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f1e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:34:28.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab6ffd700 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f2b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:34:28.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faab77fe700 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f194c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faab00a3b80 tx=0x7faab0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2539867732a1a97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:28.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faaacff9700 1 -- 172.21.15.115:0/451283200 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00a86c0 con 0x556238f15650 2024-05-09T05:34:28.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faaacff9700 1 -- 172.21.15.115:0/451283200 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faab006d070 con 0x556238f15650 2024-05-09T05:34:28.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.262+0000 7faaacff9700 1 -- 172.21.15.115:0/451283200 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00090a0 con 0x556238f15650 2024-05-09T05:34:28.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.263+0000 7faacbcb0b80 1 -- 172.21.15.115:0/451283200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 msgr2=0x556238f194c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:28.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.263+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f194c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faab00a3b80 tx=0x7faab0009780 comp rx=0 tx=0).stop 2024-05-09T05:34:28.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.263+0000 7faacbcb0b80 1 -- 172.21.15.115:0/451283200 shutdown_connections 2024-05-09T05:34:28.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.263+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238f15650 0x556238f194c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.263+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238d98450 0x556238f2b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.263+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/451283200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f1e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.263+0000 7faacbcb0b80 1 -- 172.21.15.115:0/451283200 >> 172.21.15.115:0/451283200 conn(0x556238df8650 msgr2=0x556238f18db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.263+0000 7faacbcb0b80 1 -- 172.21.15.115:0/451283200 shutdown_connections 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.263+0000 7faacbcb0b80 1 -- 172.21.15.115:0/451283200 wait complete. 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faacbcb0b80 1 -- start start 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faacbcb0b80 1 Processor -- start 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238d98450 0x556238f39580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f39ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faacbcb0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238f15650 0x556238f34530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556238f2ff80 con 0x556238d98450 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556238f2f8b0 con 0x556238e82110 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faacbcb0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556238f30650 con 0x556238f15650 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faacbcb0b80 10 client.0 _ll_register_callbacks cb 0x556238ca9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faab6ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238d98450 0x556238f39580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faab6ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238d98450 0x556238f39580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45478/0 (socket says 172.21.15.115:45478) 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faab6ffd700 1 -- 172.21.15.115:0/4111006182 learned_addr learned my addr 172.21.15.115:0/4111006182 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faaaffff700 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f39ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faab6ffd700 1 -- 172.21.15.115:0/4111006182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238f15650 msgr2=0x556238f34530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.264+0000 7faab77fe700 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238f15650 0x556238f34530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faab6ffd700 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238f15650 0x556238f34530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faab6ffd700 1 -- 172.21.15.115:0/4111006182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 msgr2=0x556238f39ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faab6ffd700 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f39ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faab6ffd700 1 -- 172.21.15.115:0/4111006182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556238f34a70 con 0x556238d98450 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faaaffff700 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f39ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faab77fe700 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238f15650 0x556238f34530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faab6ffd700 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238d98450 0x556238f39580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faaa800aa30 tx=0x7faaa80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85ceaf2f719a141 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faa9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faa9e7fc700 1 -- 172.21.15.115:0/4111006182 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8016070 con 0x556238d98450 2024-05-09T05:34:28.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faa9e7fc700 1 -- 172.21.15.115:0/4111006182 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7faaa8027450 con 0x556238d98450 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4111006182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab006d070 con 0x556238d98450 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faa9e7fc700 1 -- 172.21.15.115:0/4111006182 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7faaa8035050 con 0x556238d98450 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faacbcb0b80 10 client.14733 fetch_fsmap learned FSMap version 17 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4111006182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556238f9a770 con 0x556238d98450 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faa9e7fc700 1 -- 172.21.15.115:0/4111006182 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faab006d070 con 0x556238d98450 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.265+0000 7faa9e7fc700 1 -- 172.21.15.115:0/4111006182 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7faaa8027ce0 con 0x556238d98450 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.266+0000 7faacbcb0b80 10 client.14733 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.266+0000 7faacbcb0b80 10 client.14733 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.266+0000 7faacbcb0b80 20 client.14733 populate_metadata read hostname 'smithi115' 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.266+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556238f9ccd0 0x556238f9e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.266+0000 7faacbcb0b80 4 client.14733 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.266+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4111006182 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "status"}) -- 0x556238e3fa20 con 0x556238f9ccd0 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.266+0000 7faaaffff700 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556238f9ccd0 0x556238f9e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:28.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.266+0000 7faa9e7fc700 10 client.14733 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.266+0000 7faaaffff700 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556238f9ccd0 0x556238f9e140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62954c44e530c6aa server_cookie=df47a372b35c8dc3 in_seq=0 out_seq=0 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faa9e7fc700 1 -- 172.21.15.115:0/4111006182 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556238e3fa20 con 0x556238f9ccd0 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faa9e7fc700 10 client.14733 handle_command_reply: tid=0 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 client.14733 shutdown 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4111006182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238d98450 msgr2=0x556238f39580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238d98450 0x556238f39580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faaa800aa30 tx=0x7faaa80047c0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4111006182 shutdown_connections 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556238f9ccd0 0x556238f9e140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556238f15650 0x556238f34530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556238d98450 0x556238f39580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 --2- 172.21.15.115:0/4111006182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556238e82110 0x556238f39ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4111006182 >> 172.21.15.115:0/4111006182 conn(0x556238df8650 msgr2=0x556238f14640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faa9e7fc700 0 client.14733 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:34:28.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4111006182 shutdown_connections 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 1 -- 172.21.15.115:0/4111006182 wait complete. 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.267+0000 7faacbcb0b80 20 client.14733 trim_cache size 0 max 16384 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24469, 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:34:28.267 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 33.491966316999999, 2024-05-09T05:34:28.268 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 17, 2024-05-09T05:34:28.268 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 85, 2024-05-09T05:34:28.268 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 84, 2024-05-09T05:34:28.268 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 59.005373216999999 2024-05-09T05:34:28.268 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:34:28.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 -- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faaa40b2880 msgr2=0x7faaa40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:28.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faaa40b2880 0x7faaa40b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faaa800ac50 tx=0x7faaa800a380 comp rx=0 tx=0).stop 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 -- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8108780 msgr2=0x7faab819f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8108780 0x7faab819f670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faab000b970 tx=0x7faab000bc80 comp rx=0 tx=0).stop 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 -- 172.21.15.115:0/1284230003 shutdown_connections 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7faaa40b2880 0x7faaa40b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faab8103550 0x7faab819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab8108780 0x7faab819f670 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 --2- 172.21.15.115:0/1284230003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab8107e30 0x7faab819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 -- 172.21.15.115:0/1284230003 >> 172.21.15.115:0/1284230003 conn(0x7faab8078e00 msgr2=0x7faab8106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 -- 172.21.15.115:0/1284230003 shutdown_connections 2024-05-09T05:34:28.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:28.268+0000 7faabd29a700 1 -- 172.21.15.115:0/1284230003 wait complete. 2024-05-09T05:34:28.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:27 smithi195 ceph-mon[33844]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:29.011 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T05:34:29.012 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T05:34:29.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T05:34:29.259 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:30.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:29 smithi115 ceph-mon[30349]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:30.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:29 smithi161 ceph-mon[33805]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:30.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:29 smithi195 ceph-mon[33844]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.025+0000 7feea1e3d700 1 -- 172.21.15.115:0/3447832834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c107df0 msgr2=0x7fee9c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.025+0000 7feea1e3d700 1 --2- 172.21.15.115:0/3447832834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c107df0 0x7fee9c108200 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee900099c0 tx=0x7fee90009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.025+0000 7feea1e3d700 1 -- 172.21.15.115:0/3447832834 shutdown_connections 2024-05-09T05:34:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.025+0000 7feea1e3d700 1 --2- 172.21.15.115:0/3447832834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee9c1034c0 0x7fee9c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.025+0000 7feea1e3d700 1 --2- 172.21.15.115:0/3447832834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee9c108740 0x7fee9c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.025+0000 7feea1e3d700 1 --2- 172.21.15.115:0/3447832834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c107df0 0x7fee9c108200 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.025+0000 7feea1e3d700 1 -- 172.21.15.115:0/3447832834 >> 172.21.15.115:0/3447832834 conn(0x7fee9c0fedc0 msgr2=0x7fee9c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.025+0000 7feea1e3d700 1 -- 172.21.15.115:0/3447832834 shutdown_connections 2024-05-09T05:34:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.025+0000 7feea1e3d700 1 -- 172.21.15.115:0/3447832834 wait complete. 2024-05-09T05:34:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7feea1e3d700 1 Processor -- start 2024-05-09T05:34:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7feea1e3d700 1 -- start start 2024-05-09T05:34:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7feea1e3d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee9c1034c0 0x7fee9c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7feea1e3d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee9c107df0 0x7fee9c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7feea1e3d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c108740 0x7fee9c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7feea1e3d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c113200 con 0x7fee9c108740 2024-05-09T05:34:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7feea1e3d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c113080 con 0x7fee9c1034c0 2024-05-09T05:34:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7feea1e3d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c113380 con 0x7fee9c107df0 2024-05-09T05:34:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7fee9bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c108740 0x7fee9c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7fee9bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c108740 0x7fee9c199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45484/0 (socket says 172.21.15.115:45484) 2024-05-09T05:34:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.026+0000 7fee9bfff700 1 -- 172.21.15.115:0/2833603303 learned_addr learned my addr 172.21.15.115:0/2833603303 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee9b7fe700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee9c1034c0 0x7fee9c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee9affd700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee9c107df0 0x7fee9c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee9bfff700 1 -- 172.21.15.115:0/2833603303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee9c107df0 msgr2=0x7fee9c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee9bfff700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee9c107df0 0x7fee9c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee9bfff700 1 -- 172.21.15.115:0/2833603303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee9c1034c0 msgr2=0x7fee9c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee9bfff700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee9c1034c0 0x7fee9c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee9bfff700 1 -- 172.21.15.115:0/2833603303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c19a0f0 con 0x7fee9c108740 2024-05-09T05:34:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee9affd700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee9c107df0 0x7fee9c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:34:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee9bfff700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c108740 0x7fee9c199a40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee94007f10 tx=0x7fee9400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee98ff9700 1 -- 172.21.15.115:0/2833603303 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94016030 con 0x7fee9c108740 2024-05-09T05:34:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee98ff9700 1 -- 172.21.15.115:0/2833603303 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee9400fde0 con 0x7fee9c108740 2024-05-09T05:34:31.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9c19a3e0 con 0x7fee9c108740 2024-05-09T05:34:31.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7fee98ff9700 1 -- 172.21.15.115:0/2833603303 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94012e40 con 0x7fee9c108740 2024-05-09T05:34:31.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.027+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9c06b610 con 0x7fee9c108740 2024-05-09T05:34:31.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.028+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9c104970 con 0x7fee9c108740 2024-05-09T05:34:31.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.030+0000 7fee98ff9700 1 -- 172.21.15.115:0/2833603303 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fee94012460 con 0x7fee9c108740 2024-05-09T05:34:31.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.030+0000 7fee98ff9700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fee840b2880 0x7fee840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:31.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.030+0000 7fee9b7fe700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fee840b2880 0x7fee840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:31.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.030+0000 7fee98ff9700 1 -- 172.21.15.115:0/2833603303 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fee940e4a90 con 0x7fee9c108740 2024-05-09T05:34:31.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.031+0000 7fee9b7fe700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fee840b2880 0x7fee840b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee9c19ab80 tx=0x7fee8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:31.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.034+0000 7fee98ff9700 1 -- 172.21.15.115:0/2833603303 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee940ac570 con 0x7fee9c108740 2024-05-09T05:34:31.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.286+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fee9c19af70 con 0x7fee9c108740 2024-05-09T05:34:31.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.286+0000 7fee98ff9700 1 -- 172.21.15.115:0/2833603303 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fee940abe70 con 0x7fee9c108740 2024-05-09T05:34:31.280 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T05:34:31.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fee840b2880 msgr2=0x7fee840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:31.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fee840b2880 0x7fee840b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee9c19ab80 tx=0x7fee8c00b410 comp rx=0 tx=0).stop 2024-05-09T05:34:31.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c108740 msgr2=0x7fee9c199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:31.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c108740 0x7fee9c199a40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee94007f10 tx=0x7fee9400f5c0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 shutdown_connections 2024-05-09T05:34:31.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fee840b2880 0x7fee840b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee9c1034c0 0x7fee9c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee9c108740 0x7fee9c199a40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 --2- 172.21.15.115:0/2833603303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee9c107df0 0x7fee9c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:31.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 >> 172.21.15.115:0/2833603303 conn(0x7fee9c0fedc0 msgr2=0x7fee9c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:31.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 shutdown_connections 2024-05-09T05:34:31.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:31.289+0000 7feea1e3d700 1 -- 172.21.15.115:0/2833603303 wait complete. 2024-05-09T05:34:31.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:31 smithi115 ceph-mon[30349]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:31.979 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:31 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2833603303' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T05:34:32.009 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T05:34:32.009 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T05:34:32.009 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2024-05-09T05:34:32.009 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-09T05:34:32.009 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T05:34:32.009 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T05:34:32.009 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T05:34:32.009 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T05:34:32.009 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T05:34:32.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:32.010 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2024-05-09T05:34:32.027 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T05:34:32.027 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T05:34:32.027 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T05:34:32.027 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2024-05-09T05:34:32.027 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2024-05-09T05:34:32.027 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2024-05-09T05:34:32.027 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T05:34:32.027 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 42230sec preferred_lft 42230sec 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2024-05-09T05:34:32.028 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2024-05-09T05:34:32.029 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T05:34:32.029 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:34:32.029 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:34:32.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add name ceph-brx type bridge 2024-05-09T05:34:32.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr flush dev ceph-brx 2024-05-09T05:34:32.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set ceph-brx up 2024-05-09T05:34:32.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T05:34:32.029 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:34:32.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:34:32.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:31 smithi161 ceph-mon[33805]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:32.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:31 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2833603303' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T05:34:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:34:32.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:32.238 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T05:34:32.263 INFO:teuthology.orchestra.run.smithi115.stdout:1 2024-05-09T05:34:32.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:32.264 DEBUG:teuthology.orchestra.run.smithi115:> route 2024-05-09T05:34:32.336 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2024-05-09T05:34:32.336 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T05:34:32.336 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T05:34:32.336 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T05:34:32.336 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T05:34:32.337 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:34:32.337 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:34:32.337 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T05:34:32.337 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T05:34:32.337 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T05:34:32.337 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:34:32.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:31 smithi195 ceph-mon[33844]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:32.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:31 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2833603303' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T05:34:32.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:34:32.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:34:32.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:32.623 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2024-05-09T05:34:32.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:32.639 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list-id 2024-05-09T05:34:32.695 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:34:32.695 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:34:32.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T05:34:32.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-09T05:34:32.695 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:34:32.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:34:32.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:34:32.814 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-09T05:34:32.814 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:34:32.814 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:34:32.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-09T05:34:32.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T05:34:32.815 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-09T05:34:32.815 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-09T05:34:32.815 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-09T05:34:32.815 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:34:32.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:34:32.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:34:32.943 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:34:32.943 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:34:32.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2024-05-09T05:34:32.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T05:34:32.943 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:34:32.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:34:33.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:34:33.020 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-09T05:34:33.020 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T05:34:33.020 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-09T05:34:33.035 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T05:34:33.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T05:34:33.035 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-09T05:34:33.090 DEBUG:teuthology.orchestra.run.smithi115:> sudo modprobe fuse 2024-05-09T05:34:33.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-09T05:34:33.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-09T05:34:33.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T05:34:33.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-09T05:34:33.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-09T05:34:33.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-09T05:34:33.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-09T05:34:33.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]: dispatch 2024-05-09T05:34:33.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-09T05:34:33.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:32 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-09T05:34:33.157 DEBUG:teuthology.orchestra.run.smithi115:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T05:34:33.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-09T05:34:33.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-09T05:34:33.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T05:34:33.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-09T05:34:33.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-09T05:34:33.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-09T05:34:33.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-09T05:34:33.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]: dispatch 2024-05-09T05:34:33.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-09T05:34:33.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:32 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/proc 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/sys 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/dev 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/security 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/shm 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/pts 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/run 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/cgroup 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/pstore 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/bpf 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/ 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/selinux 2024-05-09T05:34:33.215 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/mqueue 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/hugepages 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/debug 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/tracing 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/fuse/connections 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/config 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/1000 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/ceph 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/13a2e22bb6052153ab7c86470b37bef8d0bdfc7083091867e931e28879b67b86/merged 2024-05-09T05:34:33.216 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/acd3ad335cfcee6ed0aa2a904af0b0960dadb67054786bc04b6dc799cc080a0a/merged 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/0 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/3fca301c4cc88f1303928810526de64ff814683cc20926b9033b93fd701806aa/merged 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/7e6b44b8f210ec9bbde40c7d64b8f6ce3dbb10bbc0fb21980fa9a03ab2b47c4b/merged 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/34a01ea06d6341033cabb42ced8f8049b48f28ded742a71da8c92423c8ea2ce2/merged 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6c0fe2d3f5e21d60d900b5e58abede3a279e865b4da38d63d7aef65a195c9f8d/merged 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f803c522de6ea34098ff189db5afa25cfbc08f325cdcd31c1e77c8ed40c44ed7/merged 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/cd6f5685d865a8044769d8ef3dbdb9da57e95198eae808f462bad7f5dd7ef90c/merged 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/aed36e62638f1a670ec5fd9f39ce551bb3ab57cf62f78a9841e1a6c8084cee2b/merged 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f1e281515338294c1c7e459159c0cad407353f82069f197b9c0c97d1772886ed/merged 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T05:34:33.217 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T05:34:33.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:33.218 DEBUG:teuthology.orchestra.run.smithi115:> ls /sys/fs/fuse/connections 2024-05-09T05:34:33.270 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T05:34:33.271 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-09T05:34:33.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo modprobe fuse 2024-05-09T05:34:33.337 DEBUG:teuthology.orchestra.run.smithi115:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T05:34:33.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-09T05:34:33.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-09T05:34:33.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T05:34:33.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-09T05:34:33.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-09T05:34:33.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-09T05:34:33.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-09T05:34:33.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]: dispatch 2024-05-09T05:34:33.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-05-09T05:34:33.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:32 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-05-09T05:34:33.394 INFO:teuthology.orchestra.run.smithi115.stdout:/proc 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/sys 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/dev 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/security 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/shm 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/pts 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/run 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/cgroup 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/pstore 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/bpf 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/ 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/selinux 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:34:33.395 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/mqueue 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/hugepages 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/debug 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/tracing 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/fuse/connections 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/config 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/1000 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/ceph 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/13a2e22bb6052153ab7c86470b37bef8d0bdfc7083091867e931e28879b67b86/merged 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/acd3ad335cfcee6ed0aa2a904af0b0960dadb67054786bc04b6dc799cc080a0a/merged 2024-05-09T05:34:33.396 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/0 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/3fca301c4cc88f1303928810526de64ff814683cc20926b9033b93fd701806aa/merged 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/7e6b44b8f210ec9bbde40c7d64b8f6ce3dbb10bbc0fb21980fa9a03ab2b47c4b/merged 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/34a01ea06d6341033cabb42ced8f8049b48f28ded742a71da8c92423c8ea2ce2/merged 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6c0fe2d3f5e21d60d900b5e58abede3a279e865b4da38d63d7aef65a195c9f8d/merged 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f803c522de6ea34098ff189db5afa25cfbc08f325cdcd31c1e77c8ed40c44ed7/merged 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/cd6f5685d865a8044769d8ef3dbdb9da57e95198eae808f462bad7f5dd7ef90c/merged 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/aed36e62638f1a670ec5fd9f39ce551bb3ab57cf62f78a9841e1a6c8084cee2b/merged 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f1e281515338294c1c7e459159c0cad407353f82069f197b9c0c97d1772886ed/merged 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T05:34:33.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:33.398 DEBUG:teuthology.orchestra.run.smithi115:> ls /sys/fs/fuse/connections 2024-05-09T05:34:33.433 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi115.stderr:2024-05-09T05:34:33.440+0000 7fd84f30e480 -1 init, newargv = 0x7fd8300046a0 newargc=15 2024-05-09T05:34:33.433 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi115.stderr:ceph-fuse[73473]: starting ceph client 2024-05-09T05:34:33.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi115.stderr:ceph-fuse[73473]: starting fuse 2024-05-09T05:34:33.449 INFO:teuthology.orchestra.run.smithi115.stdout:114 2024-05-09T05:34:33.449 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-09T05:34:33.449 DEBUG:teuthology.orchestra.run.smithi115:> sudo stdin-killer -- python3 -c ' 2024-05-09T05:34:33.449 DEBUG:teuthology.orchestra.run.smithi115:> import glob 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> import re 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> import os 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> import subprocess 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> def _find_admin_socket(client_name): 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> files = glob.glob(asok_path) 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> # Given a non-glob path, it better be there 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> if "*" not in asok_path: 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> assert(len(files) == 1) 2024-05-09T05:34:33.450 DEBUG:teuthology.orchestra.run.smithi115:> return files[0] 2024-05-09T05:34:33.451 DEBUG:teuthology.orchestra.run.smithi115:> 2024-05-09T05:34:33.451 DEBUG:teuthology.orchestra.run.smithi115:> for f in files: 2024-05-09T05:34:33.451 DEBUG:teuthology.orchestra.run.smithi115:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T05:34:33.451 DEBUG:teuthology.orchestra.run.smithi115:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T05:34:33.451 DEBUG:teuthology.orchestra.run.smithi115:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T05:34:33.452 DEBUG:teuthology.orchestra.run.smithi115:> contents = proc_f.read() 2024-05-09T05:34:33.452 DEBUG:teuthology.orchestra.run.smithi115:> if mountpoint in contents: 2024-05-09T05:34:33.452 DEBUG:teuthology.orchestra.run.smithi115:> return f 2024-05-09T05:34:33.452 DEBUG:teuthology.orchestra.run.smithi115:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T05:34:33.452 DEBUG:teuthology.orchestra.run.smithi115:> 2024-05-09T05:34:33.452 DEBUG:teuthology.orchestra.run.smithi115:> print(_find_admin_socket("client.admin")) 2024-05-09T05:34:33.452 DEBUG:teuthology.orchestra.run.smithi115:> ' 2024-05-09T05:34:33.569 INFO:teuthology.orchestra.run.smithi115.stdout:/var/run/ceph/ceph-client.admin.73473.asok 2024-05-09T05:34:33.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:34:33.580 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73473.asok 2024-05-09T05:34:33.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:33.581 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73473.asok status 2024-05-09T05:34:33.725 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:34:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "metadata": { 2024-05-09T05:34:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_sha1": "c9da4049f638c0f84bdd9b4e33c42e86046ac554", 2024-05-09T05:34:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T05:34:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "entity_id": "admin", 2024-05-09T05:34:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "hostname": "smithi115", 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "pid": "73473", 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "root": "/" 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "dentry_count": 0, 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "dentry_pinned_count": 0, 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 14745, 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "inst": { 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "name": { 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "client", 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 14745 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": { 2024-05-09T05:34:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "192.168.0.1:0", 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3558862283 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": { 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "192.168.0.1:0", 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3558862283 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "inst_str": "client.14745 192.168.0.1:0/3558862283", 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addr_str": "192.168.0.1:0/3558862283", 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "inode_count": 1, 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_epoch": 17, 2024-05-09T05:34:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_epoch": 86, 2024-05-09T05:34:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_epoch_barrier": 0, 2024-05-09T05:34:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "blocklisted": false, 2024-05-09T05:34:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs" 2024-05-09T05:34:33.728 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:34:33.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:33.734 DEBUG:teuthology.orchestra.run.smithi115:> stat --file-system '--printf=%T 2024-05-09T05:34:33.734 DEBUG:teuthology.orchestra.run.smithi115:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-09T05:34:33.750 INFO:teuthology.orchestra.run.smithi115.stdout:fuseblk 2024-05-09T05:34:33.750 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-09T05:34:33.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:33.750 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-09T05:34:33.816 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-09T05:34:33.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T05:34:33.934 INFO:teuthology.orchestra.run.smithi115.stdout:# file: . 2024-05-09T05:34:33.934 INFO:teuthology.orchestra.run.smithi115.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-09T05:34:33.934 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:34:33.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:34:33.940 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:34:33.940 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T05:34:33.958 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T05:34:34.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-09T05:34:34.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:33 smithi115 ceph-mon[30349]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T05:34:34.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:34.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-09T05:34:34.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-09T05:34:34.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T05:34:34.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-09T05:34:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-09T05:34:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-09T05:34:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-09T05:34:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]': finished 2024-05-09T05:34:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-09T05:34:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-09T05:34:34.161 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:33 smithi161 ceph-mon[33805]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T05:34:34.242 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:34.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:34.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-09T05:34:34.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-09T05:34:34.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T05:34:34.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-09T05:34:34.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-09T05:34:34.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-09T05:34:34.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-09T05:34:34.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 10]}]': finished 2024-05-09T05:34:34.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-05-09T05:34:34.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-05-09T05:34:34.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:33 smithi195 ceph-mon[33844]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T05:34:35.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:34 smithi115 ceph-mon[30349]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T05:34:35.160 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:34 smithi161 ceph-mon[33805]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T05:34:35.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:34 smithi195 ceph-mon[33844]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T05:34:36.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.099+0000 7f1f08d09700 1 -- 172.21.15.115:0/3597724699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 msgr2=0x7f1f041100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:36.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.099+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3597724699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 0x7f1f041100a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00b0d0 tx=0x7f1efc00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.099+0000 7f1f08d09700 1 -- 172.21.15.115:0/3597724699 shutdown_connections 2024-05-09T05:34:36.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.099+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3597724699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f041034c0 0x7f1f041038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.099+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3597724699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 0x7f1f041100a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.099+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3597724699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f04107df0 0x7f1f04108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.099+0000 7f1f08d09700 1 -- 172.21.15.115:0/3597724699 >> 172.21.15.115:0/3597724699 conn(0x7f1f040fedf0 msgr2=0x7f1f04101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:36.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.099+0000 7f1f08d09700 1 -- 172.21.15.115:0/3597724699 shutdown_connections 2024-05-09T05:34:36.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.099+0000 7f1f08d09700 1 -- 172.21.15.115:0/3597724699 wait complete. 2024-05-09T05:34:36.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f08d09700 1 Processor -- start 2024-05-09T05:34:36.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f08d09700 1 -- start start 2024-05-09T05:34:36.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f08d09700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f041034c0 0x7f1f0419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:36.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f08d09700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f04107df0 0x7f1f0419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f08d09700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 0x7f1f04199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f08d09700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1f04113220 con 0x7f1f04108740 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f08d09700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041130a0 con 0x7f1f04107df0 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f08d09700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041133a0 con 0x7f1f041034c0 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f02d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 0x7f1f04199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f02d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 0x7f1f04199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40582/0 (socket says 172.21.15.115:40582) 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f02d9d700 1 -- 172.21.15.115:0/3867340413 learned_addr learned my addr 172.21.15.115:0/3867340413 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.100+0000 7f1f0259c700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f041034c0 0x7f1f0419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1f02d9d700 1 -- 172.21.15.115:0/3867340413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f041034c0 msgr2=0x7f1f0419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1f02d9d700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f041034c0 0x7f1f0419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1f02d9d700 1 -- 172.21.15.115:0/3867340413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f04107df0 msgr2=0x7f1f0419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1f01d9b700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f04107df0 0x7f1f0419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1f02d9d700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f04107df0 0x7f1f0419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1f02d9d700 1 -- 172.21.15.115:0/3867340413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0419a0c0 con 0x7f1f04108740 2024-05-09T05:34:36.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1f02d9d700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 0x7f1f04199a80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009850 tx=0x7f1efc007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:36.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1ef37fe700 1 -- 172.21.15.115:0/3867340413 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc01f030 con 0x7f1f04108740 2024-05-09T05:34:36.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0419a350 con 0x7f1f04108740 2024-05-09T05:34:36.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1ef37fe700 1 -- 172.21.15.115:0/3867340413 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1efc01bad0 con 0x7f1f04108740 2024-05-09T05:34:36.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1ef37fe700 1 -- 172.21.15.115:0/3867340413 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc012bb0 con 0x7f1f04108740 2024-05-09T05:34:36.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.101+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f041a66f0 con 0x7f1f04108740 2024-05-09T05:34:36.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.102+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f04104970 con 0x7f1f04108740 2024-05-09T05:34:36.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.104+0000 7f1ef37fe700 1 -- 172.21.15.115:0/3867340413 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1efc022030 con 0x7f1f04108740 2024-05-09T05:34:36.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.105+0000 7f1ef37fe700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1eec0b2880 0x7f1eec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:36.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.105+0000 7f1f0259c700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1eec0b2880 0x7f1eec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:36.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.105+0000 7f1ef37fe700 1 -- 172.21.15.115:0/3867340413 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f1efc00f020 con 0x7f1f04108740 2024-05-09T05:34:36.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.106+0000 7f1f0259c700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1eec0b2880 0x7f1eec0b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4005ea0 tx=0x7f1ef4005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:36.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.109+0000 7f1ef37fe700 1 -- 172.21.15.115:0/3867340413 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1efc029020 con 0x7f1f04108740 2024-05-09T05:34:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.327+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f0419afb0 con 0x7f1f04108740 2024-05-09T05:34:36.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:35 smithi115 ceph-mon[30349]: pgmap v318: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.328+0000 7f1ef37fe700 1 -- 172.21.15.115:0/3867340413 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+600 (secure 0 0 0) 0x7f1efc0ab900 con 0x7f1f04108740 2024-05-09T05:34:36.323 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/792641138 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:36.323 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1072371912 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:36.323 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3822912073 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:36.323 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1329653367 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:36.324 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/1212404088 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:36.324 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/1212404088 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:36.324 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/792641138 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:36.324 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3376666830 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:36.324 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1620364593 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:36.324 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2102093122 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1eec0b2880 msgr2=0x7f1eec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1eec0b2880 0x7f1eec0b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4005ea0 tx=0x7f1ef4005e30 comp rx=0 tx=0).stop 2024-05-09T05:34:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 msgr2=0x7f1f04199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 0x7f1f04199a80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009850 tx=0x7f1efc007e40 comp rx=0 tx=0).stop 2024-05-09T05:34:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 shutdown_connections 2024-05-09T05:34:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1eec0b2880 0x7f1eec0b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f04108740 0x7f1f04199a80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f041034c0 0x7f1f0419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 --2- 172.21.15.115:0/3867340413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f04107df0 0x7f1f0419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 >> 172.21.15.115:0/3867340413 conn(0x7f1f040fedf0 msgr2=0x7f1f041096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 shutdown_connections 2024-05-09T05:34:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:36.331+0000 7f1f08d09700 1 -- 172.21.15.115:0/3867340413 wait complete. 2024-05-09T05:34:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:listed 10 entries 2024-05-09T05:34:36.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:35 smithi195 ceph-mon[33844]: pgmap v318: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:36.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:35 smithi161 ceph-mon[33805]: pgmap v318: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:37.035 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:34:37.036 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T05:34:37.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T05:34:37.340 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:37.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:36 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3867340413' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T05:34:37.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:36 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3867340413' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T05:34:37.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:36 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3867340413' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T05:34:38.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:37 smithi115 ceph-mon[30349]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:38.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:37 smithi195 ceph-mon[33844]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:38.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:37 smithi161 ceph-mon[33805]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.124+0000 7f562a391700 1 -- 172.21.15.115:0/3042819862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56241089f0 msgr2=0x7f5624108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.124+0000 7f562a391700 1 --2- 172.21.15.115:0/3042819862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56241089f0 0x7f5624108e00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5614009990 tx=0x7f5614009ca0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.124+0000 7f562a391700 1 -- 172.21.15.115:0/3042819862 shutdown_connections 2024-05-09T05:34:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.124+0000 7f562a391700 1 --2- 172.21.15.115:0/3042819862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5624103310 0x7f562410f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.124+0000 7f562a391700 1 --2- 172.21.15.115:0/3042819862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56241089f0 0x7f5624108e00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.124+0000 7f562a391700 1 --2- 172.21.15.115:0/3042819862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56241029c0 0x7f5624102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.124+0000 7f562a391700 1 -- 172.21.15.115:0/3042819862 >> 172.21.15.115:0/3042819862 conn(0x7f56240fe6f0 msgr2=0x7f5624100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.124+0000 7f562a391700 1 -- 172.21.15.115:0/3042819862 shutdown_connections 2024-05-09T05:34:39.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.124+0000 7f562a391700 1 -- 172.21.15.115:0/3042819862 wait complete. 2024-05-09T05:34:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f562a391700 1 Processor -- start 2024-05-09T05:34:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f562a391700 1 -- start start 2024-05-09T05:34:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f562a391700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56241029c0 0x7f562419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f562a391700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5624103310 0x7f562419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f562a391700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56241089f0 0x7f562419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f562a391700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f56241143d0 con 0x7f5624103310 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f562a391700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5624114250 con 0x7f56241029c0 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f562a391700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5624114550 con 0x7f56241089f0 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f56237fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5624103310 0x7f562419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f56237fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5624103310 0x7f562419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40600/0 (socket says 172.21.15.115:40600) 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f56237fe700 1 -- 172.21.15.115:0/4168567644 learned_addr learned my addr 172.21.15.115:0/4168567644 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.125+0000 7f5623fff700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56241029c0 0x7f562419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f562892e700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56241089f0 0x7f562419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f56237fe700 1 -- 172.21.15.115:0/4168567644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56241089f0 msgr2=0x7f562419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f56237fe700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56241089f0 0x7f562419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f56237fe700 1 -- 172.21.15.115:0/4168567644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56241029c0 msgr2=0x7f562419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f56237fe700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56241029c0 0x7f562419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f56237fe700 1 -- 172.21.15.115:0/4168567644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562419fc50 con 0x7f5624103310 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f56237fe700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5624103310 0x7f562419b320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f561800d980 tx=0x7f561800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f56217fa700 1 -- 172.21.15.115:0/4168567644 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618017070 con 0x7f5624103310 2024-05-09T05:34:39.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f56217fa700 1 -- 172.21.15.115:0/4168567644 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f561800fd20 con 0x7f5624103310 2024-05-09T05:34:39.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f56217fa700 1 -- 172.21.15.115:0/4168567644 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561800e3b0 con 0x7f5624103310 2024-05-09T05:34:39.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562419ff40 con 0x7f5624103310 2024-05-09T05:34:39.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.126+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56241a7860 con 0x7f5624103310 2024-05-09T05:34:39.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.127+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56240512c0 con 0x7f5624103310 2024-05-09T05:34:39.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.130+0000 7f56217fa700 1 -- 172.21.15.115:0/4168567644 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f561800f890 con 0x7f5624103310 2024-05-09T05:34:39.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.130+0000 7f56217fa700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f560c0b2900 0x7f560c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:39.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.130+0000 7f5623fff700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f560c0b2900 0x7f560c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:39.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.131+0000 7f56217fa700 1 -- 172.21.15.115:0/4168567644 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f56180e1f90 con 0x7f5624103310 2024-05-09T05:34:39.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.131+0000 7f5623fff700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f560c0b2900 0x7f560c0b4e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f561400b2e0 tx=0x7f5614005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:39.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.134+0000 7f56217fa700 1 -- 172.21.15.115:0/4168567644 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56180a99d0 con 0x7f5624103310 2024-05-09T05:34:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.351+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f562402e5f0 con 0x7f5624103310 2024-05-09T05:34:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.351+0000 7f56217fa700 1 -- 172.21.15.115:0/4168567644 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+600 (secure 0 0 0) 0x7f56180a92d0 con 0x7f5624103310 2024-05-09T05:34:39.344 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/792641138 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:39.345 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1072371912 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:39.345 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3822912073 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:39.345 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1329653367 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:39.345 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/1212404088 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:39.345 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/1212404088 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:39.345 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/792641138 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:39.345 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3376666830 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:39.345 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1620364593 2024-05-10T05:26:06.415053+0000 2024-05-09T05:34:39.345 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2102093122 2024-05-10T05:26:32.548125+0000 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f560c0b2900 msgr2=0x7f560c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f560c0b2900 0x7f560c0b4e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f561400b2e0 tx=0x7f5614005fb0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5624103310 msgr2=0x7f562419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5624103310 0x7f562419b320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f561800d980 tx=0x7f561800dc90 comp rx=0 tx=0).stop 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 shutdown_connections 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f560c0b2900 0x7f560c0b4e10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56241029c0 0x7f562419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56241089f0 0x7f562419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 --2- 172.21.15.115:0/4168567644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5624103310 0x7f562419b320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.353+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 >> 172.21.15.115:0/4168567644 conn(0x7f56240fe6f0 msgr2=0x7f5624100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.354+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 shutdown_connections 2024-05-09T05:34:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:39.354+0000 7f562a391700 1 -- 172.21.15.115:0/4168567644 wait complete. 2024-05-09T05:34:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:listed 10 entries 2024-05-09T05:34:40.041 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:39 smithi115 ceph-mon[30349]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:40.041 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:39 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4168567644' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T05:34:40.070 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi115.front.sepia.ceph.com... 2024-05-09T05:34:40.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:40.071 DEBUG:teuthology.orchestra.run.smithi115:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-09T05:34:40.097 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi115.stderr:ceph-fuse[73473]: fuse finished with error 0 and tester_r 0 2024-05-09T05:34:40.099 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T05:34:40.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:39 smithi195 ceph-mon[33844]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:40.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:39 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4168567644' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T05:34:40.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:39 smithi161 ceph-mon[33805]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:40.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:39 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4168567644' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T05:34:42.266 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:41 smithi115 ceph-mon[30349]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:41 smithi195 ceph-mon[33844]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:41 smithi161 ceph-mon[33805]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T05:34:44.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:43 smithi195 ceph-mon[33844]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 419 B/s wr, 0 op/s 2024-05-09T05:34:44.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:43 smithi115 ceph-mon[30349]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 419 B/s wr, 0 op/s 2024-05-09T05:34:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:43 smithi161 ceph-mon[33805]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 419 B/s wr, 0 op/s 2024-05-09T05:34:46.201 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T05:34:46.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:34:46.201 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-09T05:34:46.212 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:45 smithi161 ceph-mon[33805]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 475 B/s wr, 0 op/s 2024-05-09T05:34:46.230 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:34:46.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:45 smithi115 ceph-mon[30349]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 475 B/s wr, 0 op/s 2024-05-09T05:34:46.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:45 smithi195 ceph-mon[33844]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 475 B/s wr, 0 op/s 2024-05-09T05:34:46.525 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:48.286 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:47 smithi115 ceph-mon[30349]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:48.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.298+0000 7fed02551700 1 -- 172.21.15.115:0/3964443754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc103310 msgr2=0x7fecfc10f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:48.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.298+0000 7fed02551700 1 --2- 172.21.15.115:0/3964443754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc103310 0x7fecfc10f890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fecf400c210 tx=0x7fecf400c520 comp rx=0 tx=0).stop 2024-05-09T05:34:48.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.298+0000 7fed02551700 1 -- 172.21.15.115:0/3964443754 shutdown_connections 2024-05-09T05:34:48.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.298+0000 7fed02551700 1 --2- 172.21.15.115:0/3964443754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc103310 0x7fecfc10f890 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:48.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.298+0000 7fed02551700 1 --2- 172.21.15.115:0/3964443754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecfc1089f0 0x7fecfc108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:48.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.298+0000 7fed02551700 1 --2- 172.21.15.115:0/3964443754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecfc1029c0 0x7fecfc102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:48.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.298+0000 7fed02551700 1 -- 172.21.15.115:0/3964443754 >> 172.21.15.115:0/3964443754 conn(0x7fecfc0fe520 msgr2=0x7fecfc100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:48.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.298+0000 7fed02551700 1 -- 172.21.15.115:0/3964443754 shutdown_connections 2024-05-09T05:34:48.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.299+0000 7fed02551700 1 -- 172.21.15.115:0/3964443754 wait complete. 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.299+0000 7fed02551700 1 Processor -- start 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.299+0000 7fed02551700 1 -- start start 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.299+0000 7fed02551700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecfc1029c0 0x7fecfc19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed02551700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecfc103310 0x7fecfc19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed02551700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc1089f0 0x7fecfc19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed02551700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1121b0 con 0x7fecfc1089f0 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed02551700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc112030 con 0x7fecfc103310 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed02551700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc112330 con 0x7fecfc1029c0 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed00aee700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc1089f0 0x7fecfc19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed00aee700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc1089f0 0x7fecfc19f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46922/0 (socket says 172.21.15.115:46922) 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed00aee700 1 -- 172.21.15.115:0/3813973467 learned_addr learned my addr 172.21.15.115:0/3813973467 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fecf3fff700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecfc103310 0x7fecfc19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:48.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fecfbfff700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecfc1029c0 0x7fecfc19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed00aee700 1 -- 172.21.15.115:0/3813973467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecfc1029c0 msgr2=0x7fecfc19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed00aee700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecfc1029c0 0x7fecfc19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed00aee700 1 -- 172.21.15.115:0/3813973467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecfc103310 msgr2=0x7fecfc19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed00aee700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecfc103310 0x7fecfc19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed00aee700 1 -- 172.21.15.115:0/3813973467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecfc19fca0 con 0x7fecfc1089f0 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed00aee700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc1089f0 0x7fecfc19f600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fecf400a720 tx=0x7fecf4008560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fecf9ffb700 1 -- 172.21.15.115:0/3813973467 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4022070 con 0x7fecfc1089f0 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fecf9ffb700 1 -- 172.21.15.115:0/3813973467 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecf4009e20 con 0x7fecfc1089f0 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecfc19ff30 con 0x7fecfc1089f0 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fecf9ffb700 1 -- 172.21.15.115:0/3813973467 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4005260 con 0x7fecfc1089f0 2024-05-09T05:34:48.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.300+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecfc1a78b0 con 0x7fecfc1089f0 2024-05-09T05:34:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.301+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecfc104100 con 0x7fecfc1089f0 2024-05-09T05:34:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.303+0000 7fecf9ffb700 1 -- 172.21.15.115:0/3813973467 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fecf4025070 con 0x7fecfc1089f0 2024-05-09T05:34:48.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.303+0000 7fecf9ffb700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fece80b2880 0x7fece80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:48.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.304+0000 7fecf9ffb700 1 -- 172.21.15.115:0/3813973467 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fecf4010020 con 0x7fecfc1089f0 2024-05-09T05:34:48.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.304+0000 7fecfbfff700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fece80b2880 0x7fece80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:48.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.304+0000 7fecfbfff700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fece80b2880 0x7fece80b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fecec006fd0 tx=0x7fecec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:48.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.307+0000 7fecf9ffb700 1 -- 172.21.15.115:0/3813973467 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecf402c020 con 0x7fecfc1089f0 2024-05-09T05:34:48.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:47 smithi195 ceph-mon[33844]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:48.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:47 smithi161 ceph-mon[33805]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:48.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.511+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecfc19c130 con 0x7fecfc1089f0 2024-05-09T05:34:48.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.513+0000 7fecf9ffb700 1 -- 172.21.15.115:0/3813973467 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24724 (secure 0 0 0) 0x7fecf40abbc0 con 0x7fecfc1089f0 2024-05-09T05:34:48.507 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:34:48.507 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:33.901006+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:34:48.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fece80b2880 msgr2=0x7fece80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:48.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fece80b2880 0x7fece80b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fecec006fd0 tx=0x7fecec008040 comp rx=0 tx=0).stop 2024-05-09T05:34:48.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc1089f0 msgr2=0x7fecfc19f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc1089f0 0x7fecfc19f600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fecf400a720 tx=0x7fecf4008560 comp rx=0 tx=0).stop 2024-05-09T05:34:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 shutdown_connections 2024-05-09T05:34:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fece80b2880 0x7fece80b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecfc1089f0 0x7fecfc19f600 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecfc1029c0 0x7fecfc19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 --2- 172.21.15.115:0/3813973467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecfc103310 0x7fecfc19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 >> 172.21.15.115:0/3813973467 conn(0x7fecfc0fe520 msgr2=0x7fecfc107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 shutdown_connections 2024-05-09T05:34:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:48.516+0000 7fed02551700 1 -- 172.21.15.115:0/3813973467 wait complete. 2024-05-09T05:34:49.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:48 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:34:49.208 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:48 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3813973467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:49.234 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:34:49.291 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:48 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:34:49.291 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:48 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3813973467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:49.323 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:48 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:34:49.323 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:48 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3813973467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:50.270 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:50.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:49 smithi195 ceph-mon[33844]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:50.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:49 smithi115 ceph-mon[30349]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:50.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:49 smithi161 ceph-mon[33805]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:51.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:51 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:51 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:51.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:51 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:52.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 -- 172.21.15.115:0/819169357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 msgr2=0x7fbf30108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:52.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 --2- 172.21.15.115:0/819169357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 0x7fbf30108220 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf240099c0 tx=0x7fbf24009cd0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 -- 172.21.15.115:0/819169357 shutdown_connections 2024-05-09T05:34:52.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 --2- 172.21.15.115:0/819169357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf30103ca0 0x7fbf301040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 --2- 172.21.15.115:0/819169357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30108760 0x7fbf30110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 --2- 172.21.15.115:0/819169357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 0x7fbf30108220 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 -- 172.21.15.115:0/819169357 >> 172.21.15.115:0/819169357 conn(0x7fbf300fe6f0 msgr2=0x7fbf30100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:52.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 -- 172.21.15.115:0/819169357 shutdown_connections 2024-05-09T05:34:52.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 -- 172.21.15.115:0/819169357 wait complete. 2024-05-09T05:34:52.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.098+0000 7fbf35f34700 1 Processor -- start 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf35f34700 1 -- start start 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf35f34700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf30103ca0 0x7fbf30110b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf35f34700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 0x7fbf30111070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf35f34700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30108760 0x7fbf301115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf35f34700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbf30114390 con 0x7fbf30107e10 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf35f34700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbf30114210 con 0x7fbf30103ca0 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf35f34700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf30114510 con 0x7fbf30108760 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf2effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 0x7fbf30111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf2effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 0x7fbf30111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46940/0 (socket says 172.21.15.115:46940) 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf2effd700 1 -- 172.21.15.115:0/274524220 learned_addr learned my addr 172.21.15.115:0/274524220 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf2f7fe700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf30103ca0 0x7fbf30110b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf2ffff700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30108760 0x7fbf301115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:52.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.099+0000 7fbf2effd700 1 -- 172.21.15.115:0/274524220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30108760 msgr2=0x7fbf301115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf2effd700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30108760 0x7fbf301115b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf2effd700 1 -- 172.21.15.115:0/274524220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf30103ca0 msgr2=0x7fbf30110b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf2effd700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf30103ca0 0x7fbf30110b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf2effd700 1 -- 172.21.15.115:0/274524220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf301a91a0 con 0x7fbf30107e10 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf2effd700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 0x7fbf30111070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbf240098e0 tx=0x7fbf240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf2cff9700 1 -- 172.21.15.115:0/274524220 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24027070 con 0x7fbf30107e10 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf301a9430 con 0x7fbf30107e10 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf2cff9700 1 -- 172.21.15.115:0/274524220 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf2400fc00 con 0x7fbf30107e10 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf2cff9700 1 -- 172.21.15.115:0/274524220 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24022ca0 con 0x7fbf30107e10 2024-05-09T05:34:52.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.100+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf301a9880 con 0x7fbf30107e10 2024-05-09T05:34:52.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.101+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf300512c0 con 0x7fbf30107e10 2024-05-09T05:34:52.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.103+0000 7fbf2cff9700 1 -- 172.21.15.115:0/274524220 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fbf2402a070 con 0x7fbf30107e10 2024-05-09T05:34:52.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.103+0000 7fbf2cff9700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbf180b2880 0x7fbf180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:52.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.104+0000 7fbf2f7fe700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbf180b2880 0x7fbf180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:52.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.104+0000 7fbf2cff9700 1 -- 172.21.15.115:0/274524220 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fbf240e3a40 con 0x7fbf30107e10 2024-05-09T05:34:52.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.104+0000 7fbf2f7fe700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbf180b2880 0x7fbf180b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf20005950 tx=0x7fbf2000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.107+0000 7fbf2cff9700 1 -- 172.21.15.115:0/274524220 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf24031020 con 0x7fbf30107e10 2024-05-09T05:34:52.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.318+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf3002e5f0 con 0x7fbf30107e10 2024-05-09T05:34:52.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.319+0000 7fbf2cff9700 1 -- 172.21.15.115:0/274524220 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24724 (secure 0 0 0) 0x7fbf24013070 con 0x7fbf30107e10 2024-05-09T05:34:52.313 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:34:52.313 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:33.901006+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:34:52.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbf180b2880 msgr2=0x7fbf180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:52.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbf180b2880 0x7fbf180b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf20005950 tx=0x7fbf2000a300 comp rx=0 tx=0).stop 2024-05-09T05:34:52.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 msgr2=0x7fbf30111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:52.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 0x7fbf30111070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbf240098e0 tx=0x7fbf240047b0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 shutdown_connections 2024-05-09T05:34:52.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbf180b2880 0x7fbf180b4d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf30103ca0 0x7fbf30110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf30108760 0x7fbf301115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 --2- 172.21.15.115:0/274524220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf30107e10 0x7fbf30111070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:52.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 >> 172.21.15.115:0/274524220 conn(0x7fbf300fe6f0 msgr2=0x7fbf301075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:52.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 shutdown_connections 2024-05-09T05:34:52.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:52.322+0000 7fbf35f34700 1 -- 172.21.15.115:0/274524220 wait complete. 2024-05-09T05:34:52.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:52 smithi115 ceph-mon[30349]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:52.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:52.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:52 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:52.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:52 smithi195 ceph-mon[33844]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:52.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:52.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:52 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:52.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:52 smithi161 ceph-mon[33805]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:52.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:52.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:52 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:53.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-09T05:34:53.284 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:53.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:53 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/274524220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:53.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:53 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:34:53.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:53 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:34:53.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:53 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:53.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:53 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/274524220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:53.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:53 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:34:53.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:53 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:34:53.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:53 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:53.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:53 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/274524220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:34:53.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:53 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:34:53.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:53 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:34:53.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:53 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:34:54.471 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:54 smithi115 ceph-mon[30349]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:54 smithi195 ceph-mon[33844]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:54 smithi161 ceph-mon[33805]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:55.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 -- 172.21.15.115:0/2853056155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc01031d0 msgr2=0x7f9dc0111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:55.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 --2- 172.21.15.115:0/2853056155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc01031d0 0x7f9dc0111170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9db800b0a0 tx=0x7f9db800b3b0 comp rx=0 tx=0).stop 2024-05-09T05:34:55.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 -- 172.21.15.115:0/2853056155 shutdown_connections 2024-05-09T05:34:55.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 --2- 172.21.15.115:0/2853056155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc01031d0 0x7f9dc0111170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:55.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 --2- 172.21.15.115:0/2853056155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc0109300 0x7f9dc0109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:55.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 --2- 172.21.15.115:0/2853056155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0102880 0x7f9dc0102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 -- 172.21.15.115:0/2853056155 >> 172.21.15.115:0/2853056155 conn(0x7f9dc00fe6f0 msgr2=0x7f9dc0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 -- 172.21.15.115:0/2853056155 shutdown_connections 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 -- 172.21.15.115:0/2853056155 wait complete. 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 Processor -- start 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.090+0000 7f9dc5732700 1 -- start start 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dc5732700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc0102880 0x7f9dc0103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dc5732700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc01031d0 0x7f9dc0103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dc5732700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109300 0x7f9dc01044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dc5732700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0114450 con 0x7f9dc0102880 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dc5732700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01142d0 con 0x7f9dc0109300 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dc5732700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01145d0 con 0x7f9dc01031d0 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbeffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc0102880 0x7f9dc0103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbeffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc0102880 0x7f9dc0103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38636/0 (socket says 172.21.15.115:38636) 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbeffd700 1 -- 172.21.15.115:0/1992089831 learned_addr learned my addr 172.21.15.115:0/1992089831 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:55.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9db7fff700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc01031d0 0x7f9dc0103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbf7fe700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109300 0x7f9dc01044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbeffd700 1 -- 172.21.15.115:0/1992089831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc01031d0 msgr2=0x7f9dc0103fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbeffd700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc01031d0 0x7f9dc0103fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbeffd700 1 -- 172.21.15.115:0/1992089831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109300 msgr2=0x7f9dc01044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbeffd700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109300 0x7f9dc01044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbeffd700 1 -- 172.21.15.115:0/1992089831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc01a9140 con 0x7f9dc0102880 2024-05-09T05:34:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.091+0000 7f9dbf7fe700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109300 0x7f9dc01044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:34:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.092+0000 7f9dbeffd700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc0102880 0x7f9dc0103a60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9db000d950 tx=0x7f9db000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.092+0000 7f9dbcff9700 1 -- 172.21.15.115:0/1992089831 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0017070 con 0x7f9dc0102880 2024-05-09T05:34:55.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.092+0000 7f9dbcff9700 1 -- 172.21.15.115:0/1992089831 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9db0009730 con 0x7f9dc0102880 2024-05-09T05:34:55.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.092+0000 7f9dbcff9700 1 -- 172.21.15.115:0/1992089831 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0003a00 con 0x7f9dc0102880 2024-05-09T05:34:55.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.092+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc01a93d0 con 0x7f9dc0102880 2024-05-09T05:34:55.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.092+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc01a98c0 con 0x7f9dc0102880 2024-05-09T05:34:55.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.093+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc00512c0 con 0x7f9dc0102880 2024-05-09T05:34:55.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.095+0000 7f9dbcff9700 1 -- 172.21.15.115:0/1992089831 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9db0003ba0 con 0x7f9dc0102880 2024-05-09T05:34:55.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.096+0000 7f9dbcff9700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9dac0b2880 0x7f9dac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:55.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.096+0000 7f9db7fff700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9dac0b2880 0x7f9dac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:55.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.096+0000 7f9dbcff9700 1 -- 172.21.15.115:0/1992089831 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9db00e27e0 con 0x7f9dc0102880 2024-05-09T05:34:55.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.097+0000 7f9db7fff700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9dac0b2880 0x7f9dac0b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0107a10 tx=0x7f9da800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:55.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.100+0000 7f9dbcff9700 1 -- 172.21.15.115:0/1992089831 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9db0020030 con 0x7f9dc0102880 2024-05-09T05:34:55.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:55.361+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9dc002e7b0 con 0x7f9dc0102880 2024-05-09T05:34:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.239+0000 7f9dbcff9700 1 -- 172.21.15.115:0/1992089831 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f9db00aa1a0 con 0x7f9dc0102880 2024-05-09T05:34:56.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9dac0b2880 msgr2=0x7f9dac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:56.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9dac0b2880 0x7f9dac0b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0107a10 tx=0x7f9da800b410 comp rx=0 tx=0).stop 2024-05-09T05:34:56.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc0102880 msgr2=0x7f9dc0103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:56.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc0102880 0x7f9dc0103a60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9db000d950 tx=0x7f9db000dc60 comp rx=0 tx=0).stop 2024-05-09T05:34:56.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 shutdown_connections 2024-05-09T05:34:56.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9dac0b2880 0x7f9dac0b4d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:56.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dc0109300 0x7f9dc01044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:56.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc0102880 0x7f9dc0103a60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:56.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 --2- 172.21.15.115:0/1992089831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dc01031d0 0x7f9dc0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:56.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 >> 172.21.15.115:0/1992089831 conn(0x7f9dc00fe6f0 msgr2=0x7f9dc010a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:56.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 shutdown_connections 2024-05-09T05:34:56.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:56.241+0000 7f9dc5732700 1 -- 172.21.15.115:0/1992089831 wait complete. 2024-05-09T05:34:56.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:56 smithi195 ceph-mon[33844]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:56.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:56 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1992089831' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T05:34:56.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:56 smithi115 ceph-mon[30349]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:56.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:56 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1992089831' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T05:34:56.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:56 smithi161 ceph-mon[33805]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-09T05:34:56.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:56 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1992089831' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T05:34:56.975 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-09T05:34:56.975 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-09T05:34:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:34:57.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:57 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1992089831' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T05:34:57.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:57 smithi115 ceph-mon[30349]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:34:57.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:57 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1992089831' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T05:34:57.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:57 smithi195 ceph-mon[33844]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:34:57.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:57 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1992089831' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T05:34:57.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:57 smithi161 ceph-mon[33805]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:34:58.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:58 smithi115 ceph-mon[30349]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:58.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:58 smithi195 ceph-mon[33844]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:58.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:58 smithi161 ceph-mon[33805]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:34:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.965+0000 7f9c157a0700 1 -- 172.21.15.115:0/1491217250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 msgr2=0x7f9c1010fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.965+0000 7f9c157a0700 1 --2- 172.21.15.115:0/1491217250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 0x7f9c1010fed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800b0d0 tx=0x7f9c0800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:34:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.965+0000 7f9c157a0700 1 -- 172.21.15.115:0/1491217250 shutdown_connections 2024-05-09T05:34:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.965+0000 7f9c157a0700 1 --2- 172.21.15.115:0/1491217250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 0x7f9c1010fed0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.965+0000 7f9c157a0700 1 --2- 172.21.15.115:0/1491217250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c101032f0 0x7f9c10103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:58.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.965+0000 7f9c157a0700 1 --2- 172.21.15.115:0/1491217250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c10107c20 0x7f9c10108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:58.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.965+0000 7f9c157a0700 1 -- 172.21.15.115:0/1491217250 >> 172.21.15.115:0/1491217250 conn(0x7f9c10078b50 msgr2=0x7f9c10078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:58.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.965+0000 7f9c157a0700 1 -- 172.21.15.115:0/1491217250 shutdown_connections 2024-05-09T05:34:58.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.965+0000 7f9c157a0700 1 -- 172.21.15.115:0/1491217250 wait complete. 2024-05-09T05:34:58.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.966+0000 7f9c157a0700 1 Processor -- start 2024-05-09T05:34:58.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.966+0000 7f9c157a0700 1 -- start start 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.966+0000 7f9c157a0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c101032f0 0x7f9c1019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.966+0000 7f9c157a0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c10107c20 0x7f9c1019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.966+0000 7f9c157a0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 0x7f9c1019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.966+0000 7f9c157a0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10113050 con 0x7f9c10108570 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.966+0000 7f9c157a0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10112ed0 con 0x7f9c101032f0 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.966+0000 7f9c157a0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101131d0 con 0x7f9c10107c20 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.966+0000 7f9c14f9f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 0x7f9c1019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c14f9f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 0x7f9c1019c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38640/0 (socket says 172.21.15.115:38640) 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c14f9f700 1 -- 172.21.15.115:0/295989326 learned_addr learned my addr 172.21.15.115:0/295989326 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:34:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c07fff700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c10107c20 0x7f9c1019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c0ffff700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c101032f0 0x7f9c1019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c14f9f700 1 -- 172.21.15.115:0/295989326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c10107c20 msgr2=0x7f9c1019be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c14f9f700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c10107c20 0x7f9c1019be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c14f9f700 1 -- 172.21.15.115:0/295989326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c101032f0 msgr2=0x7f9c1019b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c14f9f700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c101032f0 0x7f9c1019b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c14f9f700 1 -- 172.21.15.115:0/295989326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c101a6c10 con 0x7f9c10108570 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c14f9f700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 0x7f9c1019c3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c08007d90 tx=0x7f9c08007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c0dffb700 1 -- 172.21.15.115:0/295989326 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0801f030 con 0x7f9c10108570 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c0dffb700 1 -- 172.21.15.115:0/295989326 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c0801bad0 con 0x7f9c10108570 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c0dffb700 1 -- 172.21.15.115:0/295989326 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08012b70 con 0x7f9c10108570 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c101a6ea0 con 0x7f9c10108570 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.967+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101a7240 con 0x7f9c10108570 2024-05-09T05:34:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.968+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c100512c0 con 0x7f9c10108570 2024-05-09T05:34:58.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.970+0000 7f9c0dffb700 1 -- 172.21.15.115:0/295989326 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9c08012d10 con 0x7f9c10108570 2024-05-09T05:34:58.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.970+0000 7f9c0dffb700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9bfc0b2900 0x7f9bfc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:34:58.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.971+0000 7f9c0ffff700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9bfc0b2900 0x7f9bfc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:34:58.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.971+0000 7f9c0dffb700 1 -- 172.21.15.115:0/295989326 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9c0800f020 con 0x7f9c10108570 2024-05-09T05:34:58.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.971+0000 7f9c0ffff700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9bfc0b2900 0x7f9bfc0b4e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c00005950 tx=0x7f9c0000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:34:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:58.974+0000 7f9c0dffb700 1 -- 172.21.15.115:0/295989326 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c08029020 con 0x7f9c10108570 2024-05-09T05:34:59.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.226+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f9c1002e690 con 0x7f9c10108570 2024-05-09T05:34:59.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.252+0000 7f9c0dffb700 1 -- 172.21.15.115:0/295989326 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f9c080ab8f0 con 0x7f9c10108570 2024-05-09T05:34:59.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9bfc0b2900 msgr2=0x7f9bfc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:59.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9bfc0b2900 0x7f9bfc0b4e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c00005950 tx=0x7f9c0000a300 comp rx=0 tx=0).stop 2024-05-09T05:34:59.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 msgr2=0x7f9c1019c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:34:59.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 0x7f9c1019c3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c08007d90 tx=0x7f9c08007dc0 comp rx=0 tx=0).stop 2024-05-09T05:34:59.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 shutdown_connections 2024-05-09T05:34:59.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9bfc0b2900 0x7f9bfc0b4e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:59.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c101032f0 0x7f9c1019b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:59.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c10108570 0x7f9c1019c3f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:59.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 --2- 172.21.15.115:0/295989326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c10107c20 0x7f9c1019be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:34:59.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 >> 172.21.15.115:0/295989326 conn(0x7f9c10078b50 msgr2=0x7f9c10100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:34:59.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 shutdown_connections 2024-05-09T05:34:59.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:34:59.255+0000 7f9c157a0700 1 -- 172.21.15.115:0/295989326 wait complete. 2024-05-09T05:34:59.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:34:59 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/295989326' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T05:34:59.621 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:34:59 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/295989326' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T05:34:59.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:34:59 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/295989326' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-09T05:34:59.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-09T05:35:00.222 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:00.494 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:00 smithi115 ceph-mon[30349]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:35:00.494 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:00 smithi115 ceph-mon[30349]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T05:35:00.494 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:00 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/295989326' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T05:35:00.494 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:00 smithi115 ceph-mon[30349]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T05:35:00.494 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:00 smithi115 ceph-mon[30349]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:35:00.494 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:00 smithi115 ceph-mon[30349]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-09T05:35:00.494 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:00 smithi115 ceph-mon[30349]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-09T05:35:00.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:00 smithi195 ceph-mon[33844]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:35:00.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:00 smithi195 ceph-mon[33844]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T05:35:00.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:00 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/295989326' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T05:35:00.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:00 smithi195 ceph-mon[33844]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T05:35:00.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:00 smithi195 ceph-mon[33844]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:35:00.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:00 smithi195 ceph-mon[33844]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-09T05:35:00.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:00 smithi195 ceph-mon[33844]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-09T05:35:00.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:00 smithi161 ceph-mon[33805]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:35:00.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:00 smithi161 ceph-mon[33805]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T05:35:00.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:00 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/295989326' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-09T05:35:00.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:00 smithi161 ceph-mon[33805]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T05:35:00.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:00 smithi161 ceph-mon[33805]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T05:35:00.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:00 smithi161 ceph-mon[33805]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-09T05:35:00.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:00 smithi161 ceph-mon[33805]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-09T05:35:01.566 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:01 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] up:active 2024-05-09T05:35:01.566 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:01 smithi115 ceph-mon[30349]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T05:35:01.566 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:01 smithi115 ceph-mon[30349]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-09T05:35:01.566 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:01 smithi115 ceph-mon[30349]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-05-09T05:35:01.566 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:01 smithi115 ceph-mon[30349]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-09T05:35:01.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:01 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] up:active 2024-05-09T05:35:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:01 smithi195 ceph-mon[33844]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T05:35:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:01 smithi195 ceph-mon[33844]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-09T05:35:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:01 smithi195 ceph-mon[33844]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-05-09T05:35:01.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:01 smithi195 ceph-mon[33844]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-09T05:35:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:01 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] up:active 2024-05-09T05:35:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:01 smithi161 ceph-mon[33805]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T05:35:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:01 smithi161 ceph-mon[33805]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-09T05:35:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:01 smithi161 ceph-mon[33805]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-05-09T05:35:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:01 smithi161 ceph-mon[33805]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.984+0000 7f6d2eae8700 1 -- 172.21.15.115:0/470173663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 msgr2=0x7f6d28104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.984+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/470173663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 0x7f6d28104a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0099c0 tx=0x7f6d1c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.984+0000 7f6d2eae8700 1 -- 172.21.15.115:0/470173663 shutdown_connections 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.984+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/470173663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d28109870 0x7f6d281111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.984+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/470173663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 0x7f6d28104a00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.984+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/470173663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d28108f20 0x7f6d28109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.984+0000 7f6d2eae8700 1 -- 172.21.15.115:0/470173663 >> 172.21.15.115:0/470173663 conn(0x7f6d28100040 msgr2=0x7f6d28102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.984+0000 7f6d2eae8700 1 -- 172.21.15.115:0/470173663 shutdown_connections 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.984+0000 7f6d2eae8700 1 -- 172.21.15.115:0/470173663 wait complete. 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2eae8700 1 Processor -- start 2024-05-09T05:35:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2eae8700 1 -- start start 2024-05-09T05:35:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2eae8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 0x7f6d2819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2eae8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d28108f20 0x7f6d2819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2eae8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d28109870 0x7f6d2819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2eae8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28114350 con 0x7f6d281045f0 2024-05-09T05:35:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2eae8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281141d0 con 0x7f6d28108f20 2024-05-09T05:35:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2eae8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281144d0 con 0x7f6d28109870 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2c884700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 0x7f6d2819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2c884700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 0x7f6d2819ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38658/0 (socket says 172.21.15.115:38658) 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2c884700 1 -- 172.21.15.115:0/2830997795 learned_addr learned my addr 172.21.15.115:0/2830997795 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2d085700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d28109870 0x7f6d2819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d27fff700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d28108f20 0x7f6d2819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2c884700 1 -- 172.21.15.115:0/2830997795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d28109870 msgr2=0x7f6d2819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2c884700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d28109870 0x7f6d2819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2c884700 1 -- 172.21.15.115:0/2830997795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d28108f20 msgr2=0x7f6d2819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2c884700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d28108f20 0x7f6d2819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d2c884700 1 -- 172.21.15.115:0/2830997795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2819fc50 con 0x7f6d281045f0 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.985+0000 7f6d27fff700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d28108f20 0x7f6d2819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.986+0000 7f6d2c884700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 0x7f6d2819ad80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c006010 tx=0x7f6d1c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.986+0000 7f6d25ffb700 1 -- 172.21.15.115:0/2830997795 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c026070 con 0x7f6d281045f0 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.986+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2819fee0 con 0x7f6d281045f0 2024-05-09T05:35:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.986+0000 7f6d25ffb700 1 -- 172.21.15.115:0/2830997795 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d1c00f460 con 0x7f6d281045f0 2024-05-09T05:35:01.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.986+0000 7f6d25ffb700 1 -- 172.21.15.115:0/2830997795 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c009e60 con 0x7f6d281045f0 2024-05-09T05:35:01.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.986+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d281a7860 con 0x7f6d281045f0 2024-05-09T05:35:01.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.987+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d280512c0 con 0x7f6d281045f0 2024-05-09T05:35:01.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.989+0000 7f6d25ffb700 1 -- 172.21.15.115:0/2830997795 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6d1c029070 con 0x7f6d281045f0 2024-05-09T05:35:01.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.989+0000 7f6d25ffb700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d180b2880 0x7f6d180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:01.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.989+0000 7f6d27fff700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d180b2880 0x7f6d180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:01.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.989+0000 7f6d25ffb700 1 -- 172.21.15.115:0/2830997795 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6d1c0e31a0 con 0x7f6d281045f0 2024-05-09T05:35:01.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.990+0000 7f6d27fff700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d180b2880 0x7f6d180b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d281075f0 tx=0x7f6d14009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:01.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:01.993+0000 7f6d25ffb700 1 -- 172.21.15.115:0/2830997795 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d1c05edf0 con 0x7f6d281045f0 2024-05-09T05:35:02.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.244+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6d2802e5f0 con 0x7f6d281045f0 2024-05-09T05:35:02.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.280+0000 7f6d25ffb700 1 -- 172.21.15.115:0/2830997795 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f6d1c013070 con 0x7f6d281045f0 2024-05-09T05:35:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.282+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d180b2880 msgr2=0x7f6d180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.282+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d180b2880 0x7f6d180b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d281075f0 tx=0x7f6d14009260 comp rx=0 tx=0).stop 2024-05-09T05:35:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.282+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 msgr2=0x7f6d2819ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.282+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 0x7f6d2819ad80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c006010 tx=0x7f6d1c0045e0 comp rx=0 tx=0).stop 2024-05-09T05:35:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.283+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 shutdown_connections 2024-05-09T05:35:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.283+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6d180b2880 0x7f6d180b4d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.283+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d28109870 0x7f6d2819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:02.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.283+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d281045f0 0x7f6d2819ad80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:02.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.283+0000 7f6d2eae8700 1 --2- 172.21.15.115:0/2830997795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d28108f20 0x7f6d2819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:02.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.283+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 >> 172.21.15.115:0/2830997795 conn(0x7f6d28100040 msgr2=0x7f6d28102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:02.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.283+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 shutdown_connections 2024-05-09T05:35:02.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:02.283+0000 7f6d2eae8700 1 -- 172.21.15.115:0/2830997795 wait complete. 2024-05-09T05:35:02.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:02 smithi195 ceph-mon[33844]: pgmap v331: 129 pgs: 129 active+clean; 582 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-09T05:35:02.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:02 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] up:active 2024-05-09T05:35:02.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:02 smithi195 ceph-mon[33844]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T05:35:02.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:02 smithi195 ceph-mon[33844]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-09T05:35:02.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:02 smithi195 ceph-mon[33844]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-05-09T05:35:02.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:02 smithi195 ceph-mon[33844]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-05-09T05:35:02.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:02 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2830997795' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T05:35:02.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:02 smithi115 ceph-mon[30349]: pgmap v331: 129 pgs: 129 active+clean; 582 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-09T05:35:02.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:02 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] up:active 2024-05-09T05:35:02.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:02 smithi115 ceph-mon[30349]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T05:35:02.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:02 smithi115 ceph-mon[30349]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-09T05:35:02.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:02 smithi115 ceph-mon[30349]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-05-09T05:35:02.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:02 smithi115 ceph-mon[30349]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-05-09T05:35:02.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:02 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2830997795' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T05:35:02.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:02 smithi161 ceph-mon[33805]: pgmap v331: 129 pgs: 129 active+clean; 582 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-09T05:35:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:02 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] up:active 2024-05-09T05:35:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:02 smithi161 ceph-mon[33805]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-09T05:35:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:02 smithi161 ceph-mon[33805]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-09T05:35:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:02 smithi161 ceph-mon[33805]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-05-09T05:35:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:02 smithi161 ceph-mon[33805]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-05-09T05:35:02.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:02 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2830997795' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T05:35:03.013 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-09T05:35:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:03.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:03 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2830997795' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T05:35:03.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:03 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] up:active 2024-05-09T05:35:03.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:03 smithi115 ceph-mon[30349]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T05:35:03.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:03 smithi115 ceph-mon[30349]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T05:35:03.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:03 smithi115 ceph-mon[30349]: Cluster is now healthy 2024-05-09T05:35:03.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:03 smithi115 ceph-mon[30349]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 8 up:standby 2024-05-09T05:35:03.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:03 smithi115 ceph-mon[30349]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T05:35:03.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:03 smithi115 ceph-mon[30349]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-09T05:35:03.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:03 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2830997795' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T05:35:03.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:03 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] up:active 2024-05-09T05:35:03.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:03 smithi195 ceph-mon[33844]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T05:35:03.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:03 smithi195 ceph-mon[33844]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T05:35:03.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:03 smithi195 ceph-mon[33844]: Cluster is now healthy 2024-05-09T05:35:03.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:03 smithi195 ceph-mon[33844]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 8 up:standby 2024-05-09T05:35:03.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:03 smithi195 ceph-mon[33844]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T05:35:03.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:03 smithi195 ceph-mon[33844]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-09T05:35:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:03 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2830997795' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T05:35:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:03 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] up:active 2024-05-09T05:35:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:03 smithi161 ceph-mon[33805]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-09T05:35:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:03 smithi161 ceph-mon[33805]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T05:35:03.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:03 smithi161 ceph-mon[33805]: Cluster is now healthy 2024-05-09T05:35:03.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:03 smithi161 ceph-mon[33805]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 8 up:standby 2024-05-09T05:35:03.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:03 smithi161 ceph-mon[33805]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-09T05:35:03.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:03 smithi161 ceph-mon[33805]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-09T05:35:04.393 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:04 smithi115 ceph-mon[30349]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-05-09T05:35:04.394 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:04 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] up:active 2024-05-09T05:35:04.394 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:04 smithi115 ceph-mon[30349]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T05:35:04.394 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:04 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:35:04.615 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:35:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T05:35:04.471+0000 7f9d66d1b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-09T05:35:04.616 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:35:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T05:35:04.471+0000 7f9d66d1b700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-09T05:35:04.616 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:35:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T05:35:04.471+0000 7f9d66d1b700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-09T05:35:04.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:04 smithi195 ceph-mon[33844]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-05-09T05:35:04.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:04 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] up:active 2024-05-09T05:35:04.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:04 smithi195 ceph-mon[33844]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T05:35:04.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:04 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:35:04.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:04 smithi161 ceph-mon[33805]: pgmap v332: 129 pgs: 129 active+clean; 582 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-05-09T05:35:04.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:04 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] up:active 2024-05-09T05:35:04.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:04 smithi161 ceph-mon[33805]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-09T05:35:04.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:04 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:35:04.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.952+0000 7f097bc08700 1 -- 172.21.15.115:0/2754250973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09741034e0 msgr2=0x7f09741038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:04.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.952+0000 7f097bc08700 1 --2- 172.21.15.115:0/2754250973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09741034e0 0x7f09741038f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09680099c0 tx=0x7f0968009cd0 comp rx=0 tx=0).stop 2024-05-09T05:35:04.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.952+0000 7f097bc08700 1 -- 172.21.15.115:0/2754250973 shutdown_connections 2024-05-09T05:35:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.952+0000 7f097bc08700 1 --2- 172.21.15.115:0/2754250973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0974108760 0x7f09741100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.952+0000 7f097bc08700 1 --2- 172.21.15.115:0/2754250973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09741034e0 0x7f09741038f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.952+0000 7f097bc08700 1 --2- 172.21.15.115:0/2754250973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0974107e10 0x7f0974108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.952+0000 7f097bc08700 1 -- 172.21.15.115:0/2754250973 >> 172.21.15.115:0/2754250973 conn(0x7f09740fef50 msgr2=0x7f0974101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.952+0000 7f097bc08700 1 -- 172.21.15.115:0/2754250973 shutdown_connections 2024-05-09T05:35:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.952+0000 7f097bc08700 1 -- 172.21.15.115:0/2754250973 wait complete. 2024-05-09T05:35:04.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f097bc08700 1 Processor -- start 2024-05-09T05:35:04.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f097bc08700 1 -- start start 2024-05-09T05:35:04.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f097bc08700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741034e0 0x7f097419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:04.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f097bc08700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0974107e10 0x7f097419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:04.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f097bc08700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0974108760 0x7f0974199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:04.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f097bc08700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0974113240 con 0x7f0974107e10 2024-05-09T05:35:04.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f097bc08700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09741130c0 con 0x7f0974108760 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f097bc08700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09741133c0 con 0x7f09741034e0 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f09791a3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0974107e10 0x7f097419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f09791a3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0974107e10 0x7f097419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33098/0 (socket says 172.21.15.115:33098) 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f09791a3700 1 -- 172.21.15.115:0/700115809 learned_addr learned my addr 172.21.15.115:0/700115809 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f09799a4700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741034e0 0x7f097419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f09791a3700 1 -- 172.21.15.115:0/700115809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741034e0 msgr2=0x7f097419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f09791a3700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741034e0 0x7f097419f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f097a1a5700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0974108760 0x7f0974199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f09791a3700 1 -- 172.21.15.115:0/700115809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0974108760 msgr2=0x7f0974199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f09791a3700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0974108760 0x7f0974199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.953+0000 7f09791a3700 1 -- 172.21.15.115:0/700115809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097419a150 con 0x7f0974107e10 2024-05-09T05:35:04.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.954+0000 7f09791a3700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0974107e10 0x7f097419fdc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f097000d980 tx=0x7f097000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.954+0000 7f0966ffd700 1 -- 172.21.15.115:0/700115809 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970017070 con 0x7f0974107e10 2024-05-09T05:35:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.954+0000 7f0966ffd700 1 -- 172.21.15.115:0/700115809 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0970010460 con 0x7f0974107e10 2024-05-09T05:35:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.954+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097419a440 con 0x7f0974107e10 2024-05-09T05:35:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.954+0000 7f0966ffd700 1 -- 172.21.15.115:0/700115809 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097000de60 con 0x7f0974107e10 2024-05-09T05:35:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.954+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09741a6710 con 0x7f0974107e10 2024-05-09T05:35:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.955+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09740512c0 con 0x7f0974107e10 2024-05-09T05:35:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.956+0000 7f0966ffd700 1 -- 172.21.15.115:0/700115809 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0970010610 con 0x7f0974107e10 2024-05-09T05:35:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.957+0000 7f0966ffd700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f09600b2900 0x7f09600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.957+0000 7f09799a4700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f09600b2900 0x7f09600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.957+0000 7f0966ffd700 1 -- 172.21.15.115:0/700115809 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f09700e2080 con 0x7f0974107e10 2024-05-09T05:35:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.958+0000 7f09799a4700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f09600b2900 0x7f09600b4e10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0968009fd0 tx=0x7f0968005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:04.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:04.962+0000 7f0966ffd700 1 -- 172.21.15.115:0/700115809 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09700a9a40 con 0x7f0974107e10 2024-05-09T05:35:05.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.213+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f097410b190 con 0x7f0974107e10 2024-05-09T05:35:05.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.298+0000 7f0966ffd700 1 -- 172.21.15.115:0/700115809 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f09700a9340 con 0x7f0974107e10 2024-05-09T05:35:05.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.301+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f09600b2900 msgr2=0x7f09600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:05.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.301+0000 7f097bc08700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f09600b2900 0x7f09600b4e10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0968009fd0 tx=0x7f0968005fb0 comp rx=0 tx=0).stop 2024-05-09T05:35:05.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.301+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0974107e10 msgr2=0x7f097419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:05.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.301+0000 7f097bc08700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0974107e10 0x7f097419fdc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f097000d980 tx=0x7f097000dd40 comp rx=0 tx=0).stop 2024-05-09T05:35:05.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.302+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 shutdown_connections 2024-05-09T05:35:05.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.302+0000 7f097bc08700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f09600b2900 0x7f09600b4e10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:05.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.302+0000 7f097bc08700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0974108760 0x7f0974199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:05.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.302+0000 7f097bc08700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09741034e0 0x7f097419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:05.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.302+0000 7f097bc08700 1 --2- 172.21.15.115:0/700115809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0974107e10 0x7f097419fdc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:05.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.302+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 >> 172.21.15.115:0/700115809 conn(0x7f09740fef50 msgr2=0x7f09741096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:05.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.302+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 shutdown_connections 2024-05-09T05:35:05.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:05.302+0000 7f097bc08700 1 -- 172.21.15.115:0/700115809 wait complete. 2024-05-09T05:35:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:05 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/700115809' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T05:35:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:05 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/700115809' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T05:35:05.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:05 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/700115809' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T05:35:06.126 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-09T05:35:06.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-09T05:35:06.370 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:06 smithi115 ceph-mon[30349]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-05-09T05:35:06.370 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:06 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/700115809' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T05:35:06.371 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:06 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:35:06.379 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:06.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:06 smithi195 ceph-mon[33844]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-05-09T05:35:06.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:06 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/700115809' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T05:35:06.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:06 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:35:06.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:06 smithi161 ceph-mon[33805]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-05-09T05:35:06.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:06 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/700115809' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T05:35:06.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:06 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:35:08.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.131+0000 7f3ea4feb700 1 -- 172.21.15.115:0/3913377735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea0107e20 msgr2=0x7f3ea0108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:08.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.131+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/3913377735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea0107e20 0x7f3ea0108230 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3e900099c0 tx=0x7f3e90009cd0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.131+0000 7f3ea4feb700 1 -- 172.21.15.115:0/3913377735 shutdown_connections 2024-05-09T05:35:08.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.131+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/3913377735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea0103c80 0x7f3ea0104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.131+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/3913377735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea0108770 0x7f3ea01100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.131+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/3913377735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea0107e20 0x7f3ea0108230 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.131+0000 7f3ea4feb700 1 -- 172.21.15.115:0/3913377735 >> 172.21.15.115:0/3913377735 conn(0x7f3ea00fe6f0 msgr2=0x7f3ea0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:08.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.131+0000 7f3ea4feb700 1 -- 172.21.15.115:0/3913377735 shutdown_connections 2024-05-09T05:35:08.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.131+0000 7f3ea4feb700 1 -- 172.21.15.115:0/3913377735 wait complete. 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.132+0000 7f3ea4feb700 1 Processor -- start 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3ea4feb700 1 -- start start 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3ea4feb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea0103c80 0x7f3ea019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3ea4feb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea0107e20 0x7f3ea019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3ea4feb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea0108770 0x7f3ea0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3ea4feb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0113260 con 0x7f3ea0108770 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3ea4feb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01130e0 con 0x7f3ea0103c80 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3ea4feb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01133e0 con 0x7f3ea0107e20 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3e9e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea0103c80 0x7f3ea019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3e9e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea0103c80 0x7f3ea019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:52290/0 (socket says 172.21.15.115:52290) 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3e9ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea0108770 0x7f3ea0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3e9e59c700 1 -- 172.21.15.115:0/126118276 learned_addr learned my addr 172.21.15.115:0/126118276 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:08.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.133+0000 7f3e9dd9b700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea0107e20 0x7f3ea019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3e9ed9d700 1 -- 172.21.15.115:0/126118276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea0107e20 msgr2=0x7f3ea019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3e9ed9d700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea0107e20 0x7f3ea019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3e9ed9d700 1 -- 172.21.15.115:0/126118276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea0103c80 msgr2=0x7f3ea019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3e9ed9d700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea0103c80 0x7f3ea019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3e9ed9d700 1 -- 172.21.15.115:0/126118276 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea019a150 con 0x7f3ea0108770 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3e9ed9d700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea0108770 0x7f3ea0199aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007f10 tx=0x7f3e9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3e8f7fe700 1 -- 172.21.15.115:0/126118276 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98004db0 con 0x7f3ea0108770 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3e8f7fe700 1 -- 172.21.15.115:0/126118276 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e98009e20 con 0x7f3ea0108770 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea019a440 con 0x7f3ea0108770 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3e8f7fe700 1 -- 172.21.15.115:0/126118276 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98015400 con 0x7f3ea0108770 2024-05-09T05:35:08.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.134+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea01028e0 con 0x7f3ea0108770 2024-05-09T05:35:08.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.135+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea0104a00 con 0x7f3ea0108770 2024-05-09T05:35:08.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.137+0000 7f3e8f7fe700 1 -- 172.21.15.115:0/126118276 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3e9800db20 con 0x7f3ea0108770 2024-05-09T05:35:08.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.137+0000 7f3e8f7fe700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e880b2880 0x7f3e880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:08.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.137+0000 7f3e9e59c700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e880b2880 0x7f3e880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:08.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.137+0000 7f3e8f7fe700 1 -- 172.21.15.115:0/126118276 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f3e980e1e50 con 0x7f3ea0108770 2024-05-09T05:35:08.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.138+0000 7f3e9e59c700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e880b2880 0x7f3e880b4d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ea019abe0 tx=0x7f3e9400d690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:08.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.142+0000 7f3e8f7fe700 1 -- 172.21.15.115:0/126118276 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e98010070 con 0x7f3ea0108770 2024-05-09T05:35:08.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.378+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3ea0102b10 con 0x7f3e880b2880 2024-05-09T05:35:08.379 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:08 smithi115 ceph-mon[30349]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-05-09T05:35:08.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.405+0000 7f3e8f7fe700 1 -- 172.21.15.115:0/126118276 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ea0102b10 con 0x7f3e880b2880 2024-05-09T05:35:08.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e880b2880 msgr2=0x7f3e880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:08.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e880b2880 0x7f3e880b4d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ea019abe0 tx=0x7f3e9400d690 comp rx=0 tx=0).stop 2024-05-09T05:35:08.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea0108770 msgr2=0x7f3ea0199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:08.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea0108770 0x7f3ea0199aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007f10 tx=0x7f3e9800d3b0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 shutdown_connections 2024-05-09T05:35:08.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3e880b2880 0x7f3e880b4d90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ea0103c80 0x7f3ea019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea0108770 0x7f3ea0199aa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 --2- 172.21.15.115:0/126118276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea0107e20 0x7f3ea019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:08.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 >> 172.21.15.115:0/126118276 conn(0x7f3ea00fe6f0 msgr2=0x7f3ea0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:08.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 shutdown_connections 2024-05-09T05:35:08.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:08.408+0000 7f3ea4feb700 1 -- 172.21.15.115:0/126118276 wait complete. 2024-05-09T05:35:08.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:08 smithi195 ceph-mon[33844]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-05-09T05:35:08.646 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:35:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:35:08.384+0000 7fd1076e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:35:08.646 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:35:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:35:08.384+0000 7fd1076e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:35:08.646 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:35:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:35:08.384+0000 7fd1076e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:35:08.647 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:35:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:35:08.384+0000 7fd1076e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:35:08.647 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:35:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:35:08.384+0000 7fd1076e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:35:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:08 smithi161 ceph-mon[33805]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-05-09T05:35:09.131 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T05:35:09.380 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:09.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:09 smithi115 ceph-mon[30349]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:09.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:09 smithi195 ceph-mon[33844]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:09.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:09 smithi161 ceph-mon[33805]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:10.562 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:10 smithi115 ceph-mon[30349]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-09T05:35:10.563 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:10 smithi115 ceph-mon[30349]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-09T05:35:10.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:10 smithi195 ceph-mon[33844]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-09T05:35:10.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:10 smithi195 ceph-mon[33844]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-09T05:35:10.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:10 smithi161 ceph-mon[33805]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-09T05:35:10.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:10 smithi161 ceph-mon[33805]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-09T05:35:11.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.132+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/2609078084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24107e10 msgr2=0x7fcd24108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:11.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.132+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/2609078084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24107e10 0x7fcd24108220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcd180099c0 tx=0x7fcd18009cd0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.132+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/2609078084 shutdown_connections 2024-05-09T05:35:11.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.132+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/2609078084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd241034e0 0x7fcd241038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.132+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/2609078084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd24108760 0x7fcd241100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.132+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/2609078084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24107e10 0x7fcd24108220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.132+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/2609078084 >> 172.21.15.115:0/2609078084 conn(0x7fcd240fef50 msgr2=0x7fcd24101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:11.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.132+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/2609078084 shutdown_connections 2024-05-09T05:35:11.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.132+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/2609078084 wait complete. 2024-05-09T05:35:11.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.133+0000 7fcd2a9c9700 1 Processor -- start 2024-05-09T05:35:11.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.133+0000 7fcd2a9c9700 1 -- start start 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd2a9c9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd241034e0 0x7fcd2419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd2a9c9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd24107e10 0x7fcd2419c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd2a9c9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd24108760 0x7fcd2419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd2a9c9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd24113220 con 0x7fcd241034e0 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd2a9c9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241130a0 con 0x7fcd24108760 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd2a9c9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241133a0 con 0x7fcd24107e10 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd23fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd241034e0 0x7fcd2419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd23fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd241034e0 0x7fcd2419bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33158/0 (socket says 172.21.15.115:33158) 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd23fff700 1 -- 172.21.15.115:0/3825976893 learned_addr learned my addr 172.21.15.115:0/3825976893 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd28f66700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd24108760 0x7fcd2419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd237fe700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd24107e10 0x7fcd2419c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd23fff700 1 -- 172.21.15.115:0/3825976893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd24107e10 msgr2=0x7fcd2419c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:11.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd23fff700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd24107e10 0x7fcd2419c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd23fff700 1 -- 172.21.15.115:0/3825976893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd24108760 msgr2=0x7fcd2419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:11.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd23fff700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd24108760 0x7fcd2419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd23fff700 1 -- 172.21.15.115:0/3825976893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241999f0 con 0x7fcd241034e0 2024-05-09T05:35:11.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.134+0000 7fcd23fff700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd241034e0 0x7fcd2419bc00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0060b0 tx=0x7fcd1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:11.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.135+0000 7fcd217fa700 1 -- 172.21.15.115:0/3825976893 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c01a070 con 0x7fcd241034e0 2024-05-09T05:35:11.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.135+0000 7fcd217fa700 1 -- 172.21.15.115:0/3825976893 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd1c00dd20 con 0x7fcd241034e0 2024-05-09T05:35:11.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.135+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd24199ce0 con 0x7fcd241034e0 2024-05-09T05:35:11.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.135+0000 7fcd217fa700 1 -- 172.21.15.115:0/3825976893 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c014600 con 0x7fcd241034e0 2024-05-09T05:35:11.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.135+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2419a1f0 con 0x7fcd241034e0 2024-05-09T05:35:11.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.136+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd240512c0 con 0x7fcd241034e0 2024-05-09T05:35:11.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.137+0000 7fcd217fa700 1 -- 172.21.15.115:0/3825976893 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fcd1c01d070 con 0x7fcd241034e0 2024-05-09T05:35:11.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.138+0000 7fcd217fa700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fcd0c0b2910 0x7fcd0c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:11.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.138+0000 7fcd237fe700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fcd0c0b2910 0x7fcd0c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:11.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.138+0000 7fcd217fa700 1 -- 172.21.15.115:0/3825976893 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fcd1c0e1e60 con 0x7fcd241034e0 2024-05-09T05:35:11.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.138+0000 7fcd237fe700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fcd0c0b2910 0x7fcd0c0b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcd18000c00 tx=0x7fcd1801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.143+0000 7fcd217fa700 1 -- 172.21.15.115:0/3825976893 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd1c010070 con 0x7fcd241034e0 2024-05-09T05:35:11.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.379+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcd24063c00 con 0x7fcd0c0b2910 2024-05-09T05:35:11.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.408+0000 7fcd217fa700 1 -- 172.21.15.115:0/3825976893 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd24063c00 con 0x7fcd0c0b2910 2024-05-09T05:35:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fcd0c0b2910 msgr2=0x7fcd0c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fcd0c0b2910 0x7fcd0c0b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcd18000c00 tx=0x7fcd1801a040 comp rx=0 tx=0).stop 2024-05-09T05:35:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd241034e0 msgr2=0x7fcd2419bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd241034e0 0x7fcd2419bc00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0060b0 tx=0x7fcd1c00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 shutdown_connections 2024-05-09T05:35:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fcd0c0b2910 0x7fcd0c0b4e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd24108760 0x7fcd2419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd241034e0 0x7fcd2419bc00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 --2- 172.21.15.115:0/3825976893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd24107e10 0x7fcd2419c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 >> 172.21.15.115:0/3825976893 conn(0x7fcd240fef50 msgr2=0x7fcd241096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 shutdown_connections 2024-05-09T05:35:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:11.411+0000 7fcd2a9c9700 1 -- 172.21.15.115:0/3825976893 wait complete. 2024-05-09T05:35:12.238 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T05:35:12.488 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:12.614 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:12 smithi115 ceph-mon[30349]: pgmap v336: 129 pgs: 129 active+clean; 593 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-05-09T05:35:12.614 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:12 smithi115 ceph-mon[30349]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:12.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:12 smithi195 ceph-mon[33844]: pgmap v336: 129 pgs: 129 active+clean; 593 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-05-09T05:35:12.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:12 smithi195 ceph-mon[33844]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:12.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:12 smithi161 ceph-mon[33805]: pgmap v336: 129 pgs: 129 active+clean; 593 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-05-09T05:35:12.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:12 smithi161 ceph-mon[33805]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.417+0000 7fbccfb2e700 1 -- 172.21.15.115:0/3640977556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc8108760 msgr2=0x7fbcc81100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.417+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/3640977556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc8108760 0x7fbcc81100c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000b0d0 tx=0x7fbcc000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.417+0000 7fbccfb2e700 1 -- 172.21.15.115:0/3640977556 shutdown_connections 2024-05-09T05:35:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.417+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/3640977556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc81034e0 0x7fbcc81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.417+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/3640977556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc8108760 0x7fbcc81100c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.417+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/3640977556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcc8107e10 0x7fbcc8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.417+0000 7fbccfb2e700 1 -- 172.21.15.115:0/3640977556 >> 172.21.15.115:0/3640977556 conn(0x7fbcc80fef50 msgr2=0x7fbcc8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.417+0000 7fbccfb2e700 1 -- 172.21.15.115:0/3640977556 shutdown_connections 2024-05-09T05:35:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.417+0000 7fbccfb2e700 1 -- 172.21.15.115:0/3640977556 wait complete. 2024-05-09T05:35:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:14 smithi115 ceph-mon[30349]: pgmap v337: 129 pgs: 129 active+clean; 593 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.4 KiB/s wr, 20 op/s 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccfb2e700 1 Processor -- start 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccfb2e700 1 -- start start 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccfb2e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc81034e0 0x7fbcc819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccfb2e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc8107e10 0x7fbcc819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccfb2e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcc8108760 0x7fbcc81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccfb2e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8113240 con 0x7fbcc81034e0 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccfb2e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81130c0 con 0x7fbcc8107e10 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccfb2e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81133c0 con 0x7fbcc8108760 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccd8ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc81034e0 0x7fbcc819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccd8ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc81034e0 0x7fbcc819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53170/0 (socket says 172.21.15.115:53170) 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccd8ca700 1 -- 172.21.15.115:0/1659216095 learned_addr learned my addr 172.21.15.115:0/1659216095 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbccd0c9700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc8107e10 0x7fbcc819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.418+0000 7fbcce0cb700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcc8108760 0x7fbcc81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbccd8ca700 1 -- 172.21.15.115:0/1659216095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcc8108760 msgr2=0x7fbcc81a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbccd8ca700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcc8108760 0x7fbcc81a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbccd8ca700 1 -- 172.21.15.115:0/1659216095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc8107e10 msgr2=0x7fbcc819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbccd8ca700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc8107e10 0x7fbcc819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbccd8ca700 1 -- 172.21.15.115:0/1659216095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc81a1d70 con 0x7fbcc81034e0 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbccd8ca700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc81034e0 0x7fbcc819cea0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00e9a0 tx=0x7fbcbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbcbaffd700 1 -- 172.21.15.115:0/1659216095 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00ca10 con 0x7fbcc81034e0 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbcbaffd700 1 -- 172.21.15.115:0/1659216095 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcbc00cbb0 con 0x7fbcc81034e0 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc81a2060 con 0x7fbcc81034e0 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbcbaffd700 1 -- 172.21.15.115:0/1659216095 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc018680 con 0x7fbcc81034e0 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.419+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc81a9980 con 0x7fbcc81034e0 2024-05-09T05:35:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.420+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc80512c0 con 0x7fbcc81034e0 2024-05-09T05:35:14.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.422+0000 7fbcbaffd700 1 -- 172.21.15.115:0/1659216095 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fbcbc010430 con 0x7fbcc81034e0 2024-05-09T05:35:14.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.423+0000 7fbcbaffd700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcb40b2910 0x7fbcb40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:14.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.423+0000 7fbccd0c9700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcb40b2910 0x7fbcb40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:14.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.423+0000 7fbcbaffd700 1 -- 172.21.15.115:0/1659216095 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fbcbc0e2c50 con 0x7fbcc81034e0 2024-05-09T05:35:14.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.424+0000 7fbccd0c9700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcb40b2910 0x7fbcb40b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbcc81098c0 tx=0x7fbcc400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:14.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.426+0000 7fbcbaffd700 1 -- 172.21.15.115:0/1659216095 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcbc0aa4b0 con 0x7fbcc81034e0 2024-05-09T05:35:14.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:14 smithi195 ceph-mon[33844]: pgmap v337: 129 pgs: 129 active+clean; 593 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.4 KiB/s wr, 20 op/s 2024-05-09T05:35:14.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.664+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fbcc8106c80 con 0x7fbcb40b2910 2024-05-09T05:35:14.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:14 smithi161 ceph-mon[33805]: pgmap v337: 129 pgs: 129 active+clean; 593 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.4 KiB/s wr, 20 op/s 2024-05-09T05:35:14.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.669+0000 7fbcbaffd700 1 -- 172.21.15.115:0/1659216095 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbcc8106c80 con 0x7fbcb40b2910 2024-05-09T05:35:14.663 INFO:teuthology.orchestra.run.smithi115.stdout:/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00 2024-05-09T05:35:14.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcb40b2910 msgr2=0x7fbcb40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:14.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcb40b2910 0x7fbcb40b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbcc81098c0 tx=0x7fbcc400b410 comp rx=0 tx=0).stop 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc81034e0 msgr2=0x7fbcc819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc81034e0 0x7fbcc819cea0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00e9a0 tx=0x7fbcbc00ecb0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 shutdown_connections 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbcb40b2910 0x7fbcb40b4e20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcc8108760 0x7fbcc81a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc81034e0 0x7fbcc819cea0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 --2- 172.21.15.115:0/1659216095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc8107e10 0x7fbcc819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 >> 172.21.15.115:0/1659216095 conn(0x7fbcc80fef50 msgr2=0x7fbcc8075bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 shutdown_connections 2024-05-09T05:35:14.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:14.671+0000 7fbccfb2e700 1 -- 172.21.15.115:0/1659216095 wait complete. 2024-05-09T05:35:15.498 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T05:35:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:16.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:16 smithi115 ceph-mon[30349]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:16.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:16 smithi115 ceph-mon[30349]: pgmap v338: 129 pgs: 129 active+clean; 613 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 24 op/s 2024-05-09T05:35:16.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:16 smithi195 ceph-mon[33844]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:16.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:16 smithi195 ceph-mon[33844]: pgmap v338: 129 pgs: 129 active+clean; 613 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 24 op/s 2024-05-09T05:35:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:16 smithi161 ceph-mon[33805]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:16 smithi161 ceph-mon[33805]: pgmap v338: 129 pgs: 129 active+clean; 613 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 24 op/s 2024-05-09T05:35:17.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.586+0000 7f9555d39700 1 -- 172.21.15.115:0/133934948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9550109850 msgr2=0x7f95501111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:17.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.586+0000 7f9555d39700 1 --2- 172.21.15.115:0/133934948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9550109850 0x7f95501111b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f954800b0d0 tx=0x7f954800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.586+0000 7f9555d39700 1 -- 172.21.15.115:0/133934948 shutdown_connections 2024-05-09T05:35:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.586+0000 7f9555d39700 1 --2- 172.21.15.115:0/133934948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9550109850 0x7f95501111b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.586+0000 7f9555d39700 1 --2- 172.21.15.115:0/133934948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95501045d0 0x7f95501049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.586+0000 7f9555d39700 1 --2- 172.21.15.115:0/133934948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9550108f00 0x7f9550109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.586+0000 7f9555d39700 1 -- 172.21.15.115:0/133934948 >> 172.21.15.115:0/133934948 conn(0x7f95500ffe80 msgr2=0x7f95501022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.586+0000 7f9555d39700 1 -- 172.21.15.115:0/133934948 shutdown_connections 2024-05-09T05:35:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.586+0000 7f9555d39700 1 -- 172.21.15.115:0/133934948 wait complete. 2024-05-09T05:35:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.587+0000 7f9555d39700 1 Processor -- start 2024-05-09T05:35:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.587+0000 7f9555d39700 1 -- start start 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f9555d39700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95501045d0 0x7f95501a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f9555d39700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9550108f00 0x7f95501a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f9555d39700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9550109850 0x7f955019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f9555d39700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9550114330 con 0x7f95501045d0 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f9555d39700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95501141b0 con 0x7f9550108f00 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f9555d39700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95501144b0 con 0x7f9550109850 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95501045d0 0x7f95501a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95501045d0 0x7f95501a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53188/0 (socket says 172.21.15.115:53188) 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954f7fe700 1 -- 172.21.15.115:0/1677420308 learned_addr learned my addr 172.21.15.115:0/1677420308 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:17.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954ffff700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9550109850 0x7f955019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f9546ffd700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9550108f00 0x7f95501a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954f7fe700 1 -- 172.21.15.115:0/1677420308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9550109850 msgr2=0x7f955019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954f7fe700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9550109850 0x7f955019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954f7fe700 1 -- 172.21.15.115:0/1677420308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9550108f00 msgr2=0x7f95501a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954f7fe700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9550108f00 0x7f95501a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954f7fe700 1 -- 172.21.15.115:0/1677420308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955019b1d0 con 0x7f95501045d0 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f9546ffd700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9550108f00 0x7f95501a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954f7fe700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95501045d0 0x7f95501a08b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f954000e9a0 tx=0x7f954000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954d7fa700 1 -- 172.21.15.115:0/1677420308 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954000cbd0 con 0x7f95501045d0 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955019b4c0 con 0x7f95501045d0 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954d7fa700 1 -- 172.21.15.115:0/1677420308 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f954000cd70 con 0x7f95501045d0 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.588+0000 7f954d7fa700 1 -- 172.21.15.115:0/1677420308 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540018680 con 0x7f95501045d0 2024-05-09T05:35:17.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.589+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95501a7760 con 0x7f95501045d0 2024-05-09T05:35:17.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.589+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95500512c0 con 0x7f95501045d0 2024-05-09T05:35:17.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.591+0000 7f954d7fa700 1 -- 172.21.15.115:0/1677420308 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f9540010430 con 0x7f95501045d0 2024-05-09T05:35:17.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.592+0000 7f954d7fa700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f953c0b2910 0x7f953c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:17.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.592+0000 7f9546ffd700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f953c0b2910 0x7f953c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:17.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.592+0000 7f954d7fa700 1 -- 172.21.15.115:0/1677420308 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f95400e2bb0 con 0x7f95501045d0 2024-05-09T05:35:17.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.592+0000 7f9546ffd700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f953c0b2910 0x7f953c0b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9538005e10 tx=0x7f9538005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:17.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.595+0000 7f954d7fa700 1 -- 172.21.15.115:0/1677420308 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95400aa520 con 0x7f95501045d0 2024-05-09T05:35:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.830+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9550063c00 con 0x7f953c0b2910 2024-05-09T05:35:17.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.855+0000 7f954d7fa700 1 -- 172.21.15.115:0/1677420308 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9550063c00 con 0x7f953c0b2910 2024-05-09T05:35:17.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f953c0b2910 msgr2=0x7f953c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:17.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f953c0b2910 0x7f953c0b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9538005e10 tx=0x7f9538005d60 comp rx=0 tx=0).stop 2024-05-09T05:35:17.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95501045d0 msgr2=0x7f95501a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:17.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95501045d0 0x7f95501a08b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f954000e9a0 tx=0x7f954000ecb0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 shutdown_connections 2024-05-09T05:35:17.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f953c0b2910 0x7f953c0b4e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9550109850 0x7f955019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95501045d0 0x7f95501a08b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 --2- 172.21.15.115:0/1677420308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9550108f00 0x7f95501a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:17.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 >> 172.21.15.115:0/1677420308 conn(0x7f95500ffe80 msgr2=0x7f95501086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:17.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 shutdown_connections 2024-05-09T05:35:17.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:17.857+0000 7f9555d39700 1 -- 172.21.15.115:0/1677420308 wait complete. 2024-05-09T05:35:18.511 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:18 smithi115 ceph-mon[30349]: pgmap v339: 129 pgs: 129 active+clean; 613 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-09T05:35:18.538 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T05:35:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:18 smithi195 ceph-mon[33844]: pgmap v339: 129 pgs: 129 active+clean; 613 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-09T05:35:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:18 smithi161 ceph-mon[33805]: pgmap v339: 129 pgs: 129 active+clean; 613 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-09T05:35:18.781 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:19.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:19 smithi195 ceph-mon[33844]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:19.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:19 smithi115 ceph-mon[30349]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:19.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:19 smithi161 ceph-mon[33805]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.557+0000 7f034f869700 1 -- 172.21.15.115:0/1211762899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 msgr2=0x7f0348109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.557+0000 7f034f869700 1 --2- 172.21.15.115:0/1211762899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 0x7f0348109710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0344009990 tx=0x7f0344009ca0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.557+0000 7f034f869700 1 -- 172.21.15.115:0/1211762899 shutdown_connections 2024-05-09T05:35:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.557+0000 7f034f869700 1 --2- 172.21.15.115:0/1211762899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03481030e0 0x7f0348111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.557+0000 7f034f869700 1 --2- 172.21.15.115:0/1211762899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 0x7f0348109710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.557+0000 7f034f869700 1 --2- 172.21.15.115:0/1211762899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0348102790 0x7f0348102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.557+0000 7f034f869700 1 -- 172.21.15.115:0/1211762899 >> 172.21.15.115:0/1211762899 conn(0x7f03480fe520 msgr2=0x7f0348100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.557+0000 7f034f869700 1 -- 172.21.15.115:0/1211762899 shutdown_connections 2024-05-09T05:35:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.557+0000 7f034f869700 1 -- 172.21.15.115:0/1211762899 wait complete. 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.558+0000 7f034f869700 1 Processor -- start 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.558+0000 7f034f869700 1 -- start start 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.558+0000 7f034f869700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0348102790 0x7f034819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.558+0000 7f034f869700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03481030e0 0x7f034819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.558+0000 7f034f869700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 0x7f034819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.558+0000 7f034f869700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0348114440 con 0x7f0348109300 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.558+0000 7f034f869700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f03481142c0 con 0x7f0348102790 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.558+0000 7f034f869700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03481145c0 con 0x7f03481030e0 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034de06700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 0x7f034819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034de06700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 0x7f034819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53204/0 (socket says 172.21.15.115:53204) 2024-05-09T05:35:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034de06700 1 -- 172.21.15.115:0/2974853843 learned_addr learned my addr 172.21.15.115:0/2974853843 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:20.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034d605700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0348102790 0x7f034819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:20.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034de06700 1 -- 172.21.15.115:0/2974853843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03481030e0 msgr2=0x7f034819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:20.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034de06700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03481030e0 0x7f034819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034de06700 1 -- 172.21.15.115:0/2974853843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0348102790 msgr2=0x7f034819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034de06700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0348102790 0x7f034819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034de06700 1 -- 172.21.15.115:0/2974853843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034819fce0 con 0x7f0348109300 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034d605700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0348102790 0x7f034819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034de06700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 0x7f034819f640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0340007ee0 tx=0x7f034000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f033e7fc700 1 -- 172.21.15.115:0/2974853843 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034000dbd0 con 0x7f0348109300 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f033e7fc700 1 -- 172.21.15.115:0/2974853843 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f034000dd70 con 0x7f0348109300 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034819ffd0 con 0x7f0348109300 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f033e7fc700 1 -- 172.21.15.115:0/2974853843 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400058d0 con 0x7f0348109300 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.559+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03481a78f0 con 0x7f0348109300 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.560+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03480512c0 con 0x7f0348109300 2024-05-09T05:35:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.562+0000 7f033e7fc700 1 -- 172.21.15.115:0/2974853843 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f034001a020 con 0x7f0348109300 2024-05-09T05:35:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.563+0000 7f033e7fc700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f03340b2990 0x7f03340b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.563+0000 7f034d605700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f03340b2990 0x7f03340b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.563+0000 7f033e7fc700 1 -- 172.21.15.115:0/2974853843 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f03400e25b0 con 0x7f0348109300 2024-05-09T05:35:20.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.563+0000 7f034d605700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f03340b2990 0x7f03340b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f034400b5c0 tx=0x7f0344005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.566+0000 7f033e7fc700 1 -- 172.21.15.115:0/2974853843 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0340010070 con 0x7f0348109300 2024-05-09T05:35:20.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:20 smithi195 ceph-mon[33844]: pgmap v340: 129 pgs: 129 active+clean; 613 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-09T05:35:20.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:20 smithi115 ceph-mon[30349]: pgmap v340: 129 pgs: 129 active+clean; 613 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-09T05:35:20.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:20 smithi161 ceph-mon[33805]: pgmap v340: 129 pgs: 129 active+clean; 613 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-09T05:35:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.807+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f034819c170 con 0x7f03340b2990 2024-05-09T05:35:20.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.813+0000 7f033e7fc700 1 -- 172.21.15.115:0/2974853843 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f034819c170 con 0x7f03340b2990 2024-05-09T05:35:20.806 INFO:teuthology.orchestra.run.smithi115.stdout:/volumes/qa/sv_1/de3cc593-6d74-4635-9a46-25bb3934746c 2024-05-09T05:35:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f03340b2990 msgr2=0x7f03340b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f03340b2990 0x7f03340b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f034400b5c0 tx=0x7f0344005960 comp rx=0 tx=0).stop 2024-05-09T05:35:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 msgr2=0x7f034819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 0x7f034819f640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0340007ee0 tx=0x7f034000d3b0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 shutdown_connections 2024-05-09T05:35:20.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f03340b2990 0x7f03340b4ea0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0348102790 0x7f034819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0348109300 0x7f034819f640 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 --2- 172.21.15.115:0/2974853843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03481030e0 0x7f034819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:20.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 >> 172.21.15.115:0/2974853843 conn(0x7f03480fe520 msgr2=0x7f0348107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:20.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 shutdown_connections 2024-05-09T05:35:20.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:20.815+0000 7f034f869700 1 -- 172.21.15.115:0/2974853843 wait complete. 2024-05-09T05:35:21.515 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:35:21.758 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:22.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:22 smithi115 ceph-mon[30349]: pgmap v341: 129 pgs: 129 active+clean; 635 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 14 op/s 2024-05-09T05:35:22.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:22 smithi115 ceph-mon[30349]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:22.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:22 smithi195 ceph-mon[33844]: pgmap v341: 129 pgs: 129 active+clean; 635 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 14 op/s 2024-05-09T05:35:22.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:22 smithi195 ceph-mon[33844]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:22.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:22 smithi161 ceph-mon[33805]: pgmap v341: 129 pgs: 129 active+clean; 635 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 14 op/s 2024-05-09T05:35:22.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:22 smithi161 ceph-mon[33805]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.570+0000 7fde3fc09700 1 -- 172.21.15.115:0/2994597462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381089f0 msgr2=0x7fde38108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.570+0000 7fde3fc09700 1 --2- 172.21.15.115:0/2994597462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381089f0 0x7fde38108e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fde34009990 tx=0x7fde34009ca0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.570+0000 7fde3fc09700 1 -- 172.21.15.115:0/2994597462 shutdown_connections 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.570+0000 7fde3fc09700 1 --2- 172.21.15.115:0/2994597462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde38103310 0x7fde3810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.570+0000 7fde3fc09700 1 --2- 172.21.15.115:0/2994597462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381089f0 0x7fde38108e00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.570+0000 7fde3fc09700 1 --2- 172.21.15.115:0/2994597462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde381029c0 0x7fde38102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.570+0000 7fde3fc09700 1 -- 172.21.15.115:0/2994597462 >> 172.21.15.115:0/2994597462 conn(0x7fde380fe520 msgr2=0x7fde38100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.570+0000 7fde3fc09700 1 -- 172.21.15.115:0/2994597462 shutdown_connections 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.570+0000 7fde3fc09700 1 -- 172.21.15.115:0/2994597462 wait complete. 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3fc09700 1 Processor -- start 2024-05-09T05:35:23.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3fc09700 1 -- start start 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3fc09700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381029c0 0x7fde3819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3fc09700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde38103310 0x7fde3819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3fc09700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde381089f0 0x7fde3819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3fc09700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fde381143a0 con 0x7fde381029c0 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3fc09700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fde38114220 con 0x7fde381089f0 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3fc09700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fde38114520 con 0x7fde38103310 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3d9a5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381029c0 0x7fde3819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3d9a5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381029c0 0x7fde3819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60784/0 (socket says 172.21.15.115:60784) 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3d9a5700 1 -- 172.21.15.115:0/1829919795 learned_addr learned my addr 172.21.15.115:0/1829919795 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.571+0000 7fde3d1a4700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde38103310 0x7fde3819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3e1a6700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde381089f0 0x7fde3819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3d9a5700 1 -- 172.21.15.115:0/1829919795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde38103310 msgr2=0x7fde3819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3d9a5700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde38103310 0x7fde3819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3d9a5700 1 -- 172.21.15.115:0/1829919795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde381089f0 msgr2=0x7fde3819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:23.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3d9a5700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde381089f0 0x7fde3819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3d9a5700 1 -- 172.21.15.115:0/1829919795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3819fbe0 con 0x7fde381029c0 2024-05-09T05:35:23.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3e1a6700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde381089f0 0x7fde3819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:35:23.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3d9a5700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381029c0 0x7fde3819ad10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fde34000c00 tx=0x7fde340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:23.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde2effd700 1 -- 172.21.15.115:0/1829919795 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34027070 con 0x7fde381029c0 2024-05-09T05:35:23.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde2effd700 1 -- 172.21.15.115:0/1829919795 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde3400fb80 con 0x7fde381029c0 2024-05-09T05:35:23.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde2effd700 1 -- 172.21.15.115:0/1829919795 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34022d00 con 0x7fde381029c0 2024-05-09T05:35:23.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3819fe70 con 0x7fde381029c0 2024-05-09T05:35:23.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.572+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde381a77f0 con 0x7fde381029c0 2024-05-09T05:35:23.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.573+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde380512c0 con 0x7fde381029c0 2024-05-09T05:35:23.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.575+0000 7fde2effd700 1 -- 172.21.15.115:0/1829919795 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fde3402a070 con 0x7fde381029c0 2024-05-09T05:35:23.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.575+0000 7fde2effd700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fde240b2990 0x7fde240b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:23.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.575+0000 7fde3d1a4700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fde240b2990 0x7fde240b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:23.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.575+0000 7fde2effd700 1 -- 172.21.15.115:0/1829919795 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fde340e3900 con 0x7fde381029c0 2024-05-09T05:35:23.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.576+0000 7fde3d1a4700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fde240b2990 0x7fde240b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde38100170 tx=0x7fde2800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:23.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.579+0000 7fde2effd700 1 -- 172.21.15.115:0/1829919795 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fde34031080 con 0x7fde381029c0 2024-05-09T05:35:23.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.829+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde3819c070 con 0x7fde381029c0 2024-05-09T05:35:23.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.830+0000 7fde2effd700 1 -- 172.21.15.115:0/1829919795 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7fde34013070 con 0x7fde381029c0 2024-05-09T05:35:23.823 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:35:23.823 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:35:05.215676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:35:23.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fde240b2990 msgr2=0x7fde240b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:23.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fde240b2990 0x7fde240b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde38100170 tx=0x7fde2800a300 comp rx=0 tx=0).stop 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381029c0 msgr2=0x7fde3819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381029c0 0x7fde3819ad10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fde34000c00 tx=0x7fde340047b0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 shutdown_connections 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fde240b2990 0x7fde240b4ea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde381089f0 0x7fde3819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde381029c0 0x7fde3819ad10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 --2- 172.21.15.115:0/1829919795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde38103310 0x7fde3819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 >> 172.21.15.115:0/1829919795 conn(0x7fde380fe520 msgr2=0x7fde381067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 shutdown_connections 2024-05-09T05:35:23.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:23.832+0000 7fde3fc09700 1 -- 172.21.15.115:0/1829919795 wait complete. 2024-05-09T05:35:23.827 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 29 2024-05-09T05:35:24.573 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:24 smithi115 ceph-mon[30349]: pgmap v342: 129 pgs: 129 active+clean; 635 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-05-09T05:35:24.573 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:24 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1829919795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:35:24.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:35:24.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:24 smithi195 ceph-mon[33844]: pgmap v342: 129 pgs: 129 active+clean; 635 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-05-09T05:35:24.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:24 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1829919795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:35:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:24 smithi161 ceph-mon[33805]: pgmap v342: 129 pgs: 129 active+clean; 635 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-05-09T05:35:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:24 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1829919795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:35:24.848 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:26.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:26 smithi115 ceph-mon[30349]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2024-05-09T05:35:26.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:26 smithi195 ceph-mon[33844]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2024-05-09T05:35:26.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:26 smithi161 ceph-mon[33805]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2024-05-09T05:35:26.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.707+0000 7f2d25d07700 1 -- 172.21.15.115:0/3942828695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201089f0 msgr2=0x7f2d20108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.707+0000 7f2d25d07700 1 --2- 172.21.15.115:0/3942828695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201089f0 0x7f2d20108e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800c210 tx=0x7f2d1800c520 comp rx=0 tx=0).stop 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.708+0000 7f2d25d07700 1 -- 172.21.15.115:0/3942828695 shutdown_connections 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.708+0000 7f2d25d07700 1 --2- 172.21.15.115:0/3942828695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d20103310 0x7f2d2010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.708+0000 7f2d25d07700 1 --2- 172.21.15.115:0/3942828695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201089f0 0x7f2d20108e00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.708+0000 7f2d25d07700 1 --2- 172.21.15.115:0/3942828695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d201029c0 0x7f2d20102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.708+0000 7f2d25d07700 1 -- 172.21.15.115:0/3942828695 >> 172.21.15.115:0/3942828695 conn(0x7f2d200fe6f0 msgr2=0x7f2d20100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.708+0000 7f2d25d07700 1 -- 172.21.15.115:0/3942828695 shutdown_connections 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.708+0000 7f2d25d07700 1 -- 172.21.15.115:0/3942828695 wait complete. 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.708+0000 7f2d25d07700 1 Processor -- start 2024-05-09T05:35:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d25d07700 1 -- start start 2024-05-09T05:35:26.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d25d07700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201029c0 0x7f2d2019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:26.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d25d07700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d20103310 0x7f2d2019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:26.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d25d07700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d201089f0 0x7f2d2019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:26.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d25d07700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112190 con 0x7f2d201029c0 2024-05-09T05:35:26.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d25d07700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112010 con 0x7f2d20103310 2024-05-09T05:35:26.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d25d07700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112310 con 0x7f2d201089f0 2024-05-09T05:35:26.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201029c0 0x7f2d2019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201029c0 0x7f2d2019ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60798/0 (socket says 172.21.15.115:60798) 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1f7fe700 1 -- 172.21.15.115:0/4105270332 learned_addr learned my addr 172.21.15.115:0/4105270332 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1effd700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d20103310 0x7f2d2019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1ffff700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d201089f0 0x7f2d2019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1f7fe700 1 -- 172.21.15.115:0/4105270332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d201089f0 msgr2=0x7f2d2019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1f7fe700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d201089f0 0x7f2d2019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1f7fe700 1 -- 172.21.15.115:0/4105270332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d20103310 msgr2=0x7f2d2019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1f7fe700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d20103310 0x7f2d2019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1f7fe700 1 -- 172.21.15.115:0/4105270332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2019fc50 con 0x7f2d201029c0 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1effd700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d20103310 0x7f2d2019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.709+0000 7f2d1f7fe700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201029c0 0x7f2d2019ad80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d18008300 tx=0x7f2d180083e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.710+0000 7f2d1cff9700 1 -- 172.21.15.115:0/4105270332 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18022070 con 0x7f2d201029c0 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.710+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2019fee0 con 0x7f2d201029c0 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.710+0000 7f2d1cff9700 1 -- 172.21.15.115:0/4105270332 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d18008ae0 con 0x7f2d201029c0 2024-05-09T05:35:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.710+0000 7f2d1cff9700 1 -- 172.21.15.115:0/4105270332 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1801e7b0 con 0x7f2d201029c0 2024-05-09T05:35:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.710+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d201a7860 con 0x7f2d201029c0 2024-05-09T05:35:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.711+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d20104100 con 0x7f2d201029c0 2024-05-09T05:35:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.712+0000 7f2d1cff9700 1 -- 172.21.15.115:0/4105270332 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2d18025070 con 0x7f2d201029c0 2024-05-09T05:35:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.713+0000 7f2d1cff9700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2d0c0b2910 0x7f2d0c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.713+0000 7f2d1effd700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2d0c0b2910 0x7f2d0c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.713+0000 7f2d1cff9700 1 -- 172.21.15.115:0/4105270332 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2d18010020 con 0x7f2d201029c0 2024-05-09T05:35:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.713+0000 7f2d1effd700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2d0c0b2910 0x7f2d0c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d201006e0 tx=0x7f2d08005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:26.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.718+0000 7f2d1cff9700 1 -- 172.21.15.115:0/4105270332 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d180e7030 con 0x7f2d201029c0 2024-05-09T05:35:26.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.922+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d2019c0e0 con 0x7f2d201029c0 2024-05-09T05:35:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.924+0000 7f2d1cff9700 1 -- 172.21.15.115:0/4105270332 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24724 (secure 0 0 0) 0x7f2d1802c020 con 0x7f2d201029c0 2024-05-09T05:35:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:35:26.918 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:33.901006+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2d0c0b2910 msgr2=0x7f2d0c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2d0c0b2910 0x7f2d0c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d201006e0 tx=0x7f2d08005f50 comp rx=0 tx=0).stop 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201029c0 msgr2=0x7f2d2019ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201029c0 0x7f2d2019ad80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d18008300 tx=0x7f2d180083e0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 shutdown_connections 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2d0c0b2910 0x7f2d0c0b4e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d201089f0 0x7f2d2019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d201029c0 0x7f2d2019ad80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 --2- 172.21.15.115:0/4105270332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d20103310 0x7f2d2019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.926+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 >> 172.21.15.115:0/4105270332 conn(0x7f2d200fe6f0 msgr2=0x7f2d20106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.927+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 shutdown_connections 2024-05-09T05:35:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:26.927+0000 7f2d25d07700 1 -- 172.21.15.115:0/4105270332 wait complete. 2024-05-09T05:35:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:27 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4105270332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:35:27.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:27 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4105270332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:35:27.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:27 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4105270332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:35:27.766 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T05:35:27.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T05:35:28.015 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:28.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:28 smithi115 ceph-mon[30349]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-05-09T05:35:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:28 smithi195 ceph-mon[33844]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-05-09T05:35:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:28 smithi161 ceph-mon[33805]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.851+0000 7feb1509c700 1 -- 172.21.15.115:0/2442971540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb101045e0 msgr2=0x7feb101049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.851+0000 7feb1509c700 1 --2- 172.21.15.115:0/2442971540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb101045e0 0x7feb101049f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb000099c0 tx=0x7feb00009cd0 comp rx=0 tx=0).stop 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.851+0000 7feb1509c700 1 -- 172.21.15.115:0/2442971540 shutdown_connections 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.851+0000 7feb1509c700 1 --2- 172.21.15.115:0/2442971540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb10109860 0x7feb101111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.851+0000 7feb1509c700 1 --2- 172.21.15.115:0/2442971540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb101045e0 0x7feb101049f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.851+0000 7feb1509c700 1 --2- 172.21.15.115:0/2442971540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb10108f10 0x7feb10109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.851+0000 7feb1509c700 1 -- 172.21.15.115:0/2442971540 >> 172.21.15.115:0/2442971540 conn(0x7feb100ffef0 msgr2=0x7feb10102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.851+0000 7feb1509c700 1 -- 172.21.15.115:0/2442971540 shutdown_connections 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.851+0000 7feb1509c700 1 -- 172.21.15.115:0/2442971540 wait complete. 2024-05-09T05:35:29.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.852+0000 7feb1509c700 1 Processor -- start 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.852+0000 7feb1509c700 1 -- start start 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.852+0000 7feb1509c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb101045e0 0x7feb1019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.852+0000 7feb1509c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb10108f10 0x7feb1019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.852+0000 7feb1509c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb10109860 0x7feb1019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.852+0000 7feb1509c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb10114340 con 0x7feb10108f10 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.852+0000 7feb1509c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb101141c0 con 0x7feb101045e0 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.852+0000 7feb1509c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb101144c0 con 0x7feb10109860 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.852+0000 7feb0e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb10108f10 0x7feb1019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb10108f10 0x7feb1019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60812/0 (socket says 172.21.15.115:60812) 2024-05-09T05:35:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0e59c700 1 -- 172.21.15.115:0/608463643 learned_addr learned my addr 172.21.15.115:0/608463643 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0f59e700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb10109860 0x7feb1019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0ed9d700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb101045e0 0x7feb1019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0e59c700 1 -- 172.21.15.115:0/608463643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb10109860 msgr2=0x7feb1019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0e59c700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb10109860 0x7feb1019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0e59c700 1 -- 172.21.15.115:0/608463643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb101045e0 msgr2=0x7feb1019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0e59c700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb101045e0 0x7feb1019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0e59c700 1 -- 172.21.15.115:0/608463643 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1019fbe0 con 0x7feb10108f10 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb0e59c700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb10108f10 0x7feb1019b2b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb0400e970 tx=0x7feb0400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feaf7fff700 1 -- 172.21.15.115:0/608463643 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0400cbd0 con 0x7feb10108f10 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1019fed0 con 0x7feb10108f10 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feaf7fff700 1 -- 172.21.15.115:0/608463643 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb0400cd70 con 0x7feb10108f10 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feaf7fff700 1 -- 172.21.15.115:0/608463643 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04005570 con 0x7feb10108f10 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.853+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101a77f0 con 0x7feb10108f10 2024-05-09T05:35:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.854+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb100512c0 con 0x7feb10108f10 2024-05-09T05:35:29.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.856+0000 7feaf7fff700 1 -- 172.21.15.115:0/608463643 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7feb04004750 con 0x7feb10108f10 2024-05-09T05:35:29.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.857+0000 7feaf7fff700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaf80b2910 0x7feaf80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:29.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.857+0000 7feb0ed9d700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaf80b2910 0x7feaf80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.857+0000 7feaf7fff700 1 -- 172.21.15.115:0/608463643 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7feb040e2b10 con 0x7feb10108f10 2024-05-09T05:35:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.857+0000 7feb0ed9d700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaf80b2910 0x7feaf80b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb00003760 tx=0x7feb0001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:29.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:29.860+0000 7feaf7fff700 1 -- 172.21.15.115:0/608463643 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb04054df0 con 0x7feb10108f10 2024-05-09T05:35:30.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.058+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb1019c070 con 0x7feaf80b2910 2024-05-09T05:35:30.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.068+0000 7feaf7fff700 1 -- 172.21.15.115:0/608463643 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340618 (secure 0 0 0) 0x7feb1019c070 con 0x7feaf80b2910 2024-05-09T05:35:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:35:30.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaf80b2910 msgr2=0x7feaf80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaf80b2910 0x7feaf80b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb00003760 tx=0x7feb0001a040 comp rx=0 tx=0).stop 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb10108f10 msgr2=0x7feb1019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb10108f10 0x7feb1019b2b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb0400e970 tx=0x7feb0400ec80 comp rx=0 tx=0).stop 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 shutdown_connections 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7feaf80b2910 0x7feaf80b4e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb101045e0 0x7feb1019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb10109860 0x7feb1019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 --2- 172.21.15.115:0/608463643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb10108f10 0x7feb1019b2b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 >> 172.21.15.115:0/608463643 conn(0x7feb100ffef0 msgr2=0x7feb10102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 shutdown_connections 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:30.071+0000 7feb1509c700 1 -- 172.21.15.115:0/608463643 wait complete. 2024-05-09T05:35:30.070 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:35:30.436 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:30 smithi115 ceph-mon[30349]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-05-09T05:35:30.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:30 smithi195 ceph-mon[33844]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-05-09T05:35:30.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:30 smithi161 ceph-mon[33805]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-05-09T05:35:30.858 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-09T05:35:28.680322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655298,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":348,"num_read_kb":274,"num_write":373,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344308,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597516,"statfs":{"total":1151940427776,"available":1151587856384,"internally_reserved":0,"allocated":12079104,"data_stored":7885706,"data_compressed":426138,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27655,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":118,"num_read_kb":83,"num_write":25,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003717"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.908270+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:34:33.908270+0000","last_peered":"2024-05-09T05:34:33.908270+0000","last_clean":"2024-05-09T05:34:33.908270+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:34:33.908270+0000","last_undegraded":"2024-05-09T05:34:33.908270+0000","last_fullsized":"2024-05-09T05:34:33.908270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904243+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:34:33.904243+0000","last_peered":"2024-05-09T05:34:33.904243+0000","last_clean":"2024-05-09T05:34:33.904243+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:34:33.904243+0000","last_undegraded":"2024-05-09T05:34:33.904243+0000","last_fullsized":"2024-05-09T05:34:33.904243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904227+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:34:33.904227+0000","last_peered":"2024-05-09T05:34:33.904227+0000","last_clean":"2024-05-09T05:34:33.904227+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:34:33.904227+0000","last_undegraded":"2024-05-09T05:34:33.904227+0000","last_fullsized":"2024-05-09T05:34:33.904227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049163+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:34:34.049163+0000","last_peered":"2024-05-09T05:34:34.049163+0000","last_clean":"2024-05-09T05:34:34.049163+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:34:34.049163+0000","last_undegraded":"2024-05-09T05:34:34.049163+0000","last_fullsized":"2024-05-09T05:34:34.049163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904724+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:34:33.904724+0000","last_peered":"2024-05-09T05:34:33.904724+0000","last_clean":"2024-05-09T05:34:33.904724+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:34:33.904724+0000","last_undegraded":"2024-05-09T05:34:33.904724+0000","last_fullsized":"2024-05-09T05:34:33.904724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906840+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:34:33.906840+0000","last_peered":"2024-05-09T05:34:33.906840+0000","last_clean":"2024-05-09T05:34:33.906840+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:34:33.906840+0000","last_undegraded":"2024-05-09T05:34:33.906840+0000","last_fullsized":"2024-05-09T05:34:33.906840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904786+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:34:33.904786+0000","last_peered":"2024-05-09T05:34:33.904786+0000","last_clean":"2024-05-09T05:34:33.904786+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:34:33.904786+0000","last_undegraded":"2024-05-09T05:34:33.904786+0000","last_fullsized":"2024-05-09T05:34:33.904786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048405+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:34:34.048405+0000","last_peered":"2024-05-09T05:34:34.048405+0000","last_clean":"2024-05-09T05:34:34.048405+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:34:34.048405+0000","last_undegraded":"2024-05-09T05:34:34.048405+0000","last_fullsized":"2024-05-09T05:34:34.048405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.941003+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:34:33.941003+0000","last_peered":"2024-05-09T05:34:33.941003+0000","last_clean":"2024-05-09T05:34:33.941003+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:34:33.941003+0000","last_undegraded":"2024-05-09T05:34:33.941003+0000","last_fullsized":"2024-05-09T05:34:33.941003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.941852+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:34:33.941852+0000","last_peered":"2024-05-09T05:34:33.941852+0000","last_clean":"2024-05-09T05:34:33.941852+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:34:33.941852+0000","last_undegraded":"2024-05-09T05:34:33.941852+0000","last_fullsized":"2024-05-09T05:34:33.941852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905343+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:34:33.905343+0000","last_peered":"2024-05-09T05:34:33.905343+0000","last_clean":"2024-05-09T05:34:33.905343+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:34:33.905343+0000","last_undegraded":"2024-05-09T05:34:33.905343+0000","last_fullsized":"2024-05-09T05:34:33.905343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049091+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:34:34.049091+0000","last_peered":"2024-05-09T05:34:34.049091+0000","last_clean":"2024-05-09T05:34:34.049091+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:34:34.049091+0000","last_undegraded":"2024-05-09T05:34:34.049091+0000","last_fullsized":"2024-05-09T05:34:34.049091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048174+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:34:34.048174+0000","last_peered":"2024-05-09T05:34:34.048174+0000","last_clean":"2024-05-09T05:34:34.048174+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:34:34.048174+0000","last_undegraded":"2024-05-09T05:34:34.048174+0000","last_fullsized":"2024-05-09T05:34:34.048174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904098+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:34:33.904098+0000","last_peered":"2024-05-09T05:34:33.904098+0000","last_clean":"2024-05-09T05:34:33.904098+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:34:33.904098+0000","last_undegraded":"2024-05-09T05:34:33.904098+0000","last_fullsized":"2024-05-09T05:34:33.904098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904027+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:34:33.904027+0000","last_peered":"2024-05-09T05:34:33.904027+0000","last_clean":"2024-05-09T05:34:33.904027+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:34:33.904027+0000","last_undegraded":"2024-05-09T05:34:33.904027+0000","last_fullsized":"2024-05-09T05:34:33.904027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939791+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:34:33.939791+0000","last_peered":"2024-05-09T05:34:33.939791+0000","last_clean":"2024-05-09T05:34:33.939791+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:34:33.939791+0000","last_undegraded":"2024-05-09T05:34:33.939791+0000","last_fullsized":"2024-05-09T05:34:33.939791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907780+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:34:33.907780+0000","last_peered":"2024-05-09T05:34:33.907780+0000","last_clean":"2024-05-09T05:34:33.907780+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:34:33.907780+0000","last_undegraded":"2024-05-09T05:34:33.907780+0000","last_fullsized":"2024-05-09T05:34:33.907780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906558+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:34:33.906558+0000","last_peered":"2024-05-09T05:34:33.906558+0000","last_clean":"2024-05-09T05:34:33.906558+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:34:33.906558+0000","last_undegraded":"2024-05-09T05:34:33.906558+0000","last_fullsized":"2024-05-09T05:34:33.906558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940328+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:34:33.940328+0000","last_peered":"2024-05-09T05:34:33.940328+0000","last_clean":"2024-05-09T05:34:33.940328+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:34:33.940328+0000","last_undegraded":"2024-05-09T05:34:33.940328+0000","last_fullsized":"2024-05-09T05:34:33.940328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939639+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:34:33.939639+0000","last_peered":"2024-05-09T05:34:33.939639+0000","last_clean":"2024-05-09T05:34:33.939639+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:34:33.939639+0000","last_undegraded":"2024-05-09T05:34:33.939639+0000","last_fullsized":"2024-05-09T05:34:33.939639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:27.337262+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:35:27.337262+0000","last_peered":"2024-05-09T05:35:27.337262+0000","last_clean":"2024-05-09T05:35:27.337262+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:35:27.337262+0000","last_undegraded":"2024-05-09T05:35:27.337262+0000","last_fullsized":"2024-05-09T05:35:27.337262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049103+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:34:34.049103+0000","last_peered":"2024-05-09T05:34:34.049103+0000","last_clean":"2024-05-09T05:34:34.049103+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:34:34.049103+0000","last_undegraded":"2024-05-09T05:34:34.049103+0000","last_fullsized":"2024-05-09T05:34:34.049103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048353+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:34:34.048353+0000","last_peered":"2024-05-09T05:34:34.048353+0000","last_clean":"2024-05-09T05:34:34.048353+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:34:34.048353+0000","last_undegraded":"2024-05-09T05:34:34.048353+0000","last_fullsized":"2024-05-09T05:34:34.048353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049329+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:34:34.049329+0000","last_peered":"2024-05-09T05:34:34.049329+0000","last_clean":"2024-05-09T05:34:34.049329+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:34:34.049329+0000","last_undegraded":"2024-05-09T05:34:34.049329+0000","last_fullsized":"2024-05-09T05:34:34.049329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904216+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:34:33.904216+0000","last_peered":"2024-05-09T05:34:33.904216+0000","last_clean":"2024-05-09T05:34:33.904216+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:34:33.904216+0000","last_undegraded":"2024-05-09T05:34:33.904216+0000","last_fullsized":"2024-05-09T05:34:33.904216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048820+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:34:34.048820+0000","last_peered":"2024-05-09T05:34:34.048820+0000","last_clean":"2024-05-09T05:34:34.048820+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:34:34.048820+0000","last_undegraded":"2024-05-09T05:34:34.048820+0000","last_fullsized":"2024-05-09T05:34:34.048820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301610+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:35:03.301610+0000","last_peered":"2024-05-09T05:35:03.301610+0000","last_clean":"2024-05-09T05:35:03.301610+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:35:03.301610+0000","last_undegraded":"2024-05-09T05:35:03.301610+0000","last_fullsized":"2024-05-09T05:35:03.301610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940113+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:34:33.940113+0000","last_peered":"2024-05-09T05:34:33.940113+0000","last_clean":"2024-05-09T05:34:33.940113+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:34:33.940113+0000","last_undegraded":"2024-05-09T05:34:33.940113+0000","last_fullsized":"2024-05-09T05:34:33.940113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048328+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:34:34.048328+0000","last_peered":"2024-05-09T05:34:34.048328+0000","last_clean":"2024-05-09T05:34:34.048328+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:34:34.048328+0000","last_undegraded":"2024-05-09T05:34:34.048328+0000","last_fullsized":"2024-05-09T05:34:34.048328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296672+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:35:02.296672+0000","last_peered":"2024-05-09T05:35:02.296672+0000","last_clean":"2024-05-09T05:35:02.296672+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:35:02.296672+0000","last_undegraded":"2024-05-09T05:35:02.296672+0000","last_fullsized":"2024-05-09T05:35:02.296672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.941216+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:34:33.941216+0000","last_peered":"2024-05-09T05:34:33.941216+0000","last_clean":"2024-05-09T05:34:33.941216+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:34:33.941216+0000","last_undegraded":"2024-05-09T05:34:33.941216+0000","last_fullsized":"2024-05-09T05:34:33.941216+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939991+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:34:33.939991+0000","last_peered":"2024-05-09T05:34:33.939991+0000","last_clean":"2024-05-09T05:34:33.939991+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:34:33.939991+0000","last_undegraded":"2024-05-09T05:34:33.939991+0000","last_fullsized":"2024-05-09T05:34:33.939991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:22.332197+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:35:22.332197+0000","last_peered":"2024-05-09T05:35:22.332197+0000","last_clean":"2024-05-09T05:35:22.332197+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:35:22.332197+0000","last_undegraded":"2024-05-09T05:35:22.332197+0000","last_fullsized":"2024-05-09T05:35:22.332197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904664+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:34:33.904664+0000","last_peered":"2024-05-09T05:34:33.904664+0000","last_clean":"2024-05-09T05:34:33.904664+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:34:33.904664+0000","last_undegraded":"2024-05-09T05:34:33.904664+0000","last_fullsized":"2024-05-09T05:34:33.904664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940116+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:34:33.940116+0000","last_peered":"2024-05-09T05:34:33.940116+0000","last_clean":"2024-05-09T05:34:33.940116+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:34:33.940116+0000","last_undegraded":"2024-05-09T05:34:33.940116+0000","last_fullsized":"2024-05-09T05:34:33.940116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.292778+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:35:01.292778+0000","last_peered":"2024-05-09T05:35:01.292778+0000","last_clean":"2024-05-09T05:35:01.292778+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:35:01.292778+0000","last_undegraded":"2024-05-09T05:35:01.292778+0000","last_fullsized":"2024-05-09T05:35:01.292778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.910937+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:34:33.910937+0000","last_peered":"2024-05-09T05:34:33.910937+0000","last_clean":"2024-05-09T05:34:33.910937+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:34:33.910937+0000","last_undegraded":"2024-05-09T05:34:33.910937+0000","last_fullsized":"2024-05-09T05:34:33.910937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.051017+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:34:34.051017+0000","last_peered":"2024-05-09T05:34:34.051017+0000","last_clean":"2024-05-09T05:34:34.051017+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:34:34.051017+0000","last_undegraded":"2024-05-09T05:34:34.051017+0000","last_fullsized":"2024-05-09T05:34:34.051017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:00.275070+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:35:00.275070+0000","last_peered":"2024-05-09T05:35:00.275070+0000","last_clean":"2024-05-09T05:35:00.275070+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:35:00.275070+0000","last_undegraded":"2024-05-09T05:35:00.275070+0000","last_fullsized":"2024-05-09T05:35:00.275070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048513+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:34:34.048513+0000","last_peered":"2024-05-09T05:34:34.048513+0000","last_clean":"2024-05-09T05:34:34.048513+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:34:34.048513+0000","last_undegraded":"2024-05-09T05:34:34.048513+0000","last_fullsized":"2024-05-09T05:34:34.048513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904125+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:34:33.904125+0000","last_peered":"2024-05-09T05:34:33.904125+0000","last_clean":"2024-05-09T05:34:33.904125+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:34:33.904125+0000","last_undegraded":"2024-05-09T05:34:33.904125+0000","last_fullsized":"2024-05-09T05:34:33.904125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.297801+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:35:02.297801+0000","last_peered":"2024-05-09T05:35:02.297801+0000","last_clean":"2024-05-09T05:35:02.297801+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:35:02.297801+0000","last_undegraded":"2024-05-09T05:35:02.297801+0000","last_fullsized":"2024-05-09T05:35:02.297801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907202+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:34:33.907202+0000","last_peered":"2024-05-09T05:34:33.907202+0000","last_clean":"2024-05-09T05:34:33.907202+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:34:33.907202+0000","last_undegraded":"2024-05-09T05:34:33.907202+0000","last_fullsized":"2024-05-09T05:34:33.907202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048650+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:34:34.048650+0000","last_peered":"2024-05-09T05:34:34.048650+0000","last_clean":"2024-05-09T05:34:34.048650+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:34:34.048650+0000","last_undegraded":"2024-05-09T05:34:34.048650+0000","last_fullsized":"2024-05-09T05:34:34.048650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:24.333771+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:35:24.333771+0000","last_peered":"2024-05-09T05:35:24.333771+0000","last_clean":"2024-05-09T05:35:24.333771+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:35:24.333771+0000","last_undegraded":"2024-05-09T05:35:24.333771+0000","last_fullsized":"2024-05-09T05:35:24.333771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940740+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:34:33.940740+0000","last_peered":"2024-05-09T05:34:33.940740+0000","last_clean":"2024-05-09T05:34:33.940740+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:34:33.940740+0000","last_undegraded":"2024-05-09T05:34:33.940740+0000","last_fullsized":"2024-05-09T05:34:33.940740+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907283+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:34:33.907283+0000","last_peered":"2024-05-09T05:34:33.907283+0000","last_clean":"2024-05-09T05:34:33.907283+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:34:33.907283+0000","last_undegraded":"2024-05-09T05:34:33.907283+0000","last_fullsized":"2024-05-09T05:34:33.907283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.297309+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:35:02.297309+0000","last_peered":"2024-05-09T05:35:02.297309+0000","last_clean":"2024-05-09T05:35:02.297309+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:35:02.297309+0000","last_undegraded":"2024-05-09T05:35:02.297309+0000","last_fullsized":"2024-05-09T05:35:02.297309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904125+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:34:33.904125+0000","last_peered":"2024-05-09T05:34:33.904125+0000","last_clean":"2024-05-09T05:34:33.904125+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:34:33.904125+0000","last_undegraded":"2024-05-09T05:34:33.904125+0000","last_fullsized":"2024-05-09T05:34:33.904125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049012+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:34:34.049012+0000","last_peered":"2024-05-09T05:34:34.049012+0000","last_clean":"2024-05-09T05:34:34.049012+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:34:34.049012+0000","last_undegraded":"2024-05-09T05:34:34.049012+0000","last_fullsized":"2024-05-09T05:34:34.049012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.292960+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:35:01.292960+0000","last_peered":"2024-05-09T05:35:01.292960+0000","last_clean":"2024-05-09T05:35:01.292960+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:35:01.292960+0000","last_undegraded":"2024-05-09T05:35:01.292960+0000","last_fullsized":"2024-05-09T05:35:01.292960+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940268+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:34:33.940268+0000","last_peered":"2024-05-09T05:34:33.940268+0000","last_clean":"2024-05-09T05:34:33.940268+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:34:33.940268+0000","last_undegraded":"2024-05-09T05:34:33.940268+0000","last_fullsized":"2024-05-09T05:34:33.940268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906314+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:34:33.906314+0000","last_peered":"2024-05-09T05:34:33.906314+0000","last_clean":"2024-05-09T05:34:33.906314+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:34:33.906314+0000","last_undegraded":"2024-05-09T05:34:33.906314+0000","last_fullsized":"2024-05-09T05:34:33.906314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'34","reported_seq":125,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:24.338338+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:35:24.338338+0000","last_peered":"2024-05-09T05:35:24.338338+0000","last_clean":"2024-05-09T05:35:24.338338+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:35:24.338338+0000","last_undegraded":"2024-05-09T05:35:24.338338+0000","last_fullsized":"2024-05-09T05:35:24.338338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57883,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":82,"num_write":36,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905715+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:34:33.905715+0000","last_peered":"2024-05-09T05:34:33.905715+0000","last_clean":"2024-05-09T05:34:33.905715+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:34:33.905715+0000","last_undegraded":"2024-05-09T05:34:33.905715+0000","last_fullsized":"2024-05-09T05:34:33.905715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.911509+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:34:33.911509+0000","last_peered":"2024-05-09T05:34:33.911509+0000","last_clean":"2024-05-09T05:34:33.911509+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:34:33.911509+0000","last_undegraded":"2024-05-09T05:34:33.911509+0000","last_fullsized":"2024-05-09T05:34:33.911509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301044+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:35:03.301044+0000","last_peered":"2024-05-09T05:35:03.301044+0000","last_clean":"2024-05-09T05:35:03.301044+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:35:03.301044+0000","last_undegraded":"2024-05-09T05:35:03.301044+0000","last_fullsized":"2024-05-09T05:35:03.301044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.047970+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:34:34.047970+0000","last_peered":"2024-05-09T05:34:34.047970+0000","last_clean":"2024-05-09T05:34:34.047970+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:34:34.047970+0000","last_undegraded":"2024-05-09T05:34:34.047970+0000","last_fullsized":"2024-05-09T05:34:34.047970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907808+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:34:33.907808+0000","last_peered":"2024-05-09T05:34:33.907808+0000","last_clean":"2024-05-09T05:34:33.907808+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:34:33.907808+0000","last_undegraded":"2024-05-09T05:34:33.907808+0000","last_fullsized":"2024-05-09T05:34:33.907808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296993+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:35:02.296993+0000","last_peered":"2024-05-09T05:35:02.296993+0000","last_clean":"2024-05-09T05:35:02.296993+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:35:02.296993+0000","last_undegraded":"2024-05-09T05:35:02.296993+0000","last_fullsized":"2024-05-09T05:35:02.296993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904986+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:34:33.904986+0000","last_peered":"2024-05-09T05:34:33.904986+0000","last_clean":"2024-05-09T05:34:33.904986+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:34:33.904986+0000","last_undegraded":"2024-05-09T05:34:33.904986+0000","last_fullsized":"2024-05-09T05:34:33.904986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.051709+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:34:34.051709+0000","last_peered":"2024-05-09T05:34:34.051709+0000","last_clean":"2024-05-09T05:34:34.051709+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:34:34.051709+0000","last_undegraded":"2024-05-09T05:34:34.051709+0000","last_fullsized":"2024-05-09T05:34:34.051709+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:26.336788+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:35:26.336788+0000","last_peered":"2024-05-09T05:35:26.336788+0000","last_clean":"2024-05-09T05:35:26.336788+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:35:26.336788+0000","last_undegraded":"2024-05-09T05:35:26.336788+0000","last_fullsized":"2024-05-09T05:35:26.336788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905083+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:34:33.905083+0000","last_peered":"2024-05-09T05:34:33.905083+0000","last_clean":"2024-05-09T05:34:33.905083+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:34:33.905083+0000","last_undegraded":"2024-05-09T05:34:33.905083+0000","last_fullsized":"2024-05-09T05:34:33.905083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048527+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:34:34.048527+0000","last_peered":"2024-05-09T05:34:34.048527+0000","last_clean":"2024-05-09T05:34:34.048527+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:34:34.048527+0000","last_undegraded":"2024-05-09T05:34:34.048527+0000","last_fullsized":"2024-05-09T05:34:34.048527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.295954+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:35:02.295954+0000","last_peered":"2024-05-09T05:35:02.295954+0000","last_clean":"2024-05-09T05:35:02.295954+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:35:02.295954+0000","last_undegraded":"2024-05-09T05:35:02.295954+0000","last_fullsized":"2024-05-09T05:35:02.295954+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049898+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:34:34.049898+0000","last_peered":"2024-05-09T05:34:34.049898+0000","last_clean":"2024-05-09T05:34:34.049898+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:34:34.049898+0000","last_undegraded":"2024-05-09T05:34:34.049898+0000","last_fullsized":"2024-05-09T05:34:34.049898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.910567+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:34:33.910567+0000","last_peered":"2024-05-09T05:34:33.910567+0000","last_clean":"2024-05-09T05:34:33.910567+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:34:33.910567+0000","last_undegraded":"2024-05-09T05:34:33.910567+0000","last_fullsized":"2024-05-09T05:34:33.910567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.270702+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:35:02.270702+0000","last_peered":"2024-05-09T05:35:02.270702+0000","last_clean":"2024-05-09T05:35:02.270702+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:35:02.270702+0000","last_undegraded":"2024-05-09T05:35:02.270702+0000","last_fullsized":"2024-05-09T05:35:02.270702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907804+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:34:33.907804+0000","last_peered":"2024-05-09T05:34:33.907804+0000","last_clean":"2024-05-09T05:34:33.907804+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:34:33.907804+0000","last_undegraded":"2024-05-09T05:34:33.907804+0000","last_fullsized":"2024-05-09T05:34:33.907804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.942337+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:34:33.942337+0000","last_peered":"2024-05-09T05:34:33.942337+0000","last_clean":"2024-05-09T05:34:33.942337+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:34:33.942337+0000","last_undegraded":"2024-05-09T05:34:33.942337+0000","last_fullsized":"2024-05-09T05:34:33.942337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.912172+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:34:33.912172+0000","last_peered":"2024-05-09T05:34:33.912172+0000","last_clean":"2024-05-09T05:34:33.912172+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:34:33.912172+0000","last_undegraded":"2024-05-09T05:34:33.912172+0000","last_fullsized":"2024-05-09T05:34:33.912172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907623+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:34:33.907623+0000","last_peered":"2024-05-09T05:34:33.907623+0000","last_clean":"2024-05-09T05:34:33.907623+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:34:33.907623+0000","last_undegraded":"2024-05-09T05:34:33.907623+0000","last_fullsized":"2024-05-09T05:34:33.907623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048006+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:34:34.048006+0000","last_peered":"2024-05-09T05:34:34.048006+0000","last_clean":"2024-05-09T05:34:34.048006+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:34:34.048006+0000","last_undegraded":"2024-05-09T05:34:34.048006+0000","last_fullsized":"2024-05-09T05:34:34.048006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:22.330993+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:35:22.330993+0000","last_peered":"2024-05-09T05:35:22.330993+0000","last_clean":"2024-05-09T05:35:22.330993+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:35:22.330993+0000","last_undegraded":"2024-05-09T05:35:22.330993+0000","last_fullsized":"2024-05-09T05:35:22.330993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.913062+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:34:33.913062+0000","last_peered":"2024-05-09T05:34:33.913062+0000","last_clean":"2024-05-09T05:34:33.913062+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:34:33.913062+0000","last_undegraded":"2024-05-09T05:34:33.913062+0000","last_fullsized":"2024-05-09T05:34:33.913062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048401+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:34:34.048401+0000","last_peered":"2024-05-09T05:34:34.048401+0000","last_clean":"2024-05-09T05:34:34.048401+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:34:34.048401+0000","last_undegraded":"2024-05-09T05:34:34.048401+0000","last_fullsized":"2024-05-09T05:34:34.048401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.910013+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:34:33.910013+0000","last_peered":"2024-05-09T05:34:33.910013+0000","last_clean":"2024-05-09T05:34:33.910013+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:34:33.910013+0000","last_undegraded":"2024-05-09T05:34:33.910013+0000","last_fullsized":"2024-05-09T05:34:33.910013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:23.331745+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:35:23.331745+0000","last_peered":"2024-05-09T05:35:23.331745+0000","last_clean":"2024-05-09T05:35:23.331745+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:35:23.331745+0000","last_undegraded":"2024-05-09T05:35:23.331745+0000","last_fullsized":"2024-05-09T05:35:23.331745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905633+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:34:33.905633+0000","last_peered":"2024-05-09T05:34:33.905633+0000","last_clean":"2024-05-09T05:34:33.905633+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:34:33.905633+0000","last_undegraded":"2024-05-09T05:34:33.905633+0000","last_fullsized":"2024-05-09T05:34:33.905633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048889+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:34:34.048889+0000","last_peered":"2024-05-09T05:34:34.048889+0000","last_clean":"2024-05-09T05:34:34.048889+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:34:34.048889+0000","last_undegraded":"2024-05-09T05:34:34.048889+0000","last_fullsized":"2024-05-09T05:34:34.048889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.302304+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:35:03.302304+0000","last_peered":"2024-05-09T05:35:03.302304+0000","last_clean":"2024-05-09T05:35:03.302304+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:35:03.302304+0000","last_undegraded":"2024-05-09T05:35:03.302304+0000","last_fullsized":"2024-05-09T05:35:03.302304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904811+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:34:33.904811+0000","last_peered":"2024-05-09T05:34:33.904811+0000","last_clean":"2024-05-09T05:34:33.904811+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:34:33.904811+0000","last_undegraded":"2024-05-09T05:34:33.904811+0000","last_fullsized":"2024-05-09T05:34:33.904811+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.911099+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:34:33.911099+0000","last_peered":"2024-05-09T05:34:33.911099+0000","last_clean":"2024-05-09T05:34:33.911099+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:34:33.911099+0000","last_undegraded":"2024-05-09T05:34:33.911099+0000","last_fullsized":"2024-05-09T05:34:33.911099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048107+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:34:34.048107+0000","last_peered":"2024-05-09T05:34:34.048107+0000","last_clean":"2024-05-09T05:34:34.048107+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:34:34.048107+0000","last_undegraded":"2024-05-09T05:34:34.048107+0000","last_fullsized":"2024-05-09T05:34:34.048107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:00.276507+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:35:00.276507+0000","last_peered":"2024-05-09T05:35:00.276507+0000","last_clean":"2024-05-09T05:35:00.276507+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:35:00.276507+0000","last_undegraded":"2024-05-09T05:35:00.276507+0000","last_fullsized":"2024-05-09T05:35:00.276507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907526+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:34:33.907526+0000","last_peered":"2024-05-09T05:34:33.907526+0000","last_clean":"2024-05-09T05:34:33.907526+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:34:33.907526+0000","last_undegraded":"2024-05-09T05:34:33.907526+0000","last_fullsized":"2024-05-09T05:34:33.907526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.295356+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:35:02.295356+0000","last_peered":"2024-05-09T05:35:02.295356+0000","last_clean":"2024-05-09T05:35:02.295356+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:35:02.295356+0000","last_undegraded":"2024-05-09T05:35:02.295356+0000","last_fullsized":"2024-05-09T05:35:02.295356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048307+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:34:34.048307+0000","last_peered":"2024-05-09T05:34:34.048307+0000","last_clean":"2024-05-09T05:34:34.048307+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:34:34.048307+0000","last_undegraded":"2024-05-09T05:34:34.048307+0000","last_fullsized":"2024-05-09T05:34:34.048307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048421+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:34:34.048421+0000","last_peered":"2024-05-09T05:34:34.048421+0000","last_clean":"2024-05-09T05:34:34.048421+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:34:34.048421+0000","last_undegraded":"2024-05-09T05:34:34.048421+0000","last_fullsized":"2024-05-09T05:34:34.048421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'16","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:23.793428+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:35:23.793428+0000","last_peered":"2024-05-09T05:35:23.793428+0000","last_clean":"2024-05-09T05:35:23.793428+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:35:23.793428+0000","last_undegraded":"2024-05-09T05:35:23.793428+0000","last_fullsized":"2024-05-09T05:35:23.793428+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049225+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:34:34.049225+0000","last_peered":"2024-05-09T05:34:34.049225+0000","last_clean":"2024-05-09T05:34:34.049225+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:34:34.049225+0000","last_undegraded":"2024-05-09T05:34:34.049225+0000","last_fullsized":"2024-05-09T05:34:34.049225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939294+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:34:33.939294+0000","last_peered":"2024-05-09T05:34:33.939294+0000","last_clean":"2024-05-09T05:34:33.939294+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:34:33.939294+0000","last_undegraded":"2024-05-09T05:34:33.939294+0000","last_fullsized":"2024-05-09T05:34:33.939294+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.295838+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:35:02.295838+0000","last_peered":"2024-05-09T05:35:02.295838+0000","last_clean":"2024-05-09T05:35:02.295838+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:35:02.295838+0000","last_undegraded":"2024-05-09T05:35:02.295838+0000","last_fullsized":"2024-05-09T05:35:02.295838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904345+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:34:33.904345+0000","last_peered":"2024-05-09T05:34:33.904345+0000","last_clean":"2024-05-09T05:34:33.904345+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:34:33.904345+0000","last_undegraded":"2024-05-09T05:34:33.904345+0000","last_fullsized":"2024-05-09T05:34:33.904345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049243+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:34:34.049243+0000","last_peered":"2024-05-09T05:34:34.049243+0000","last_clean":"2024-05-09T05:34:34.049243+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:34:34.049243+0000","last_undegraded":"2024-05-09T05:34:34.049243+0000","last_fullsized":"2024-05-09T05:34:34.049243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:59.269115+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:34:59.269115+0000","last_peered":"2024-05-09T05:34:59.269115+0000","last_clean":"2024-05-09T05:34:59.269115+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:34:59.269115+0000","last_undegraded":"2024-05-09T05:34:59.269115+0000","last_fullsized":"2024-05-09T05:34:59.269115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904547+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:34:33.904547+0000","last_peered":"2024-05-09T05:34:33.904547+0000","last_clean":"2024-05-09T05:34:33.904547+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:34:33.904547+0000","last_undegraded":"2024-05-09T05:34:33.904547+0000","last_fullsized":"2024-05-09T05:34:33.904547+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906445+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:34:33.906445+0000","last_peered":"2024-05-09T05:34:33.906445+0000","last_clean":"2024-05-09T05:34:33.906445+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:34:33.906445+0000","last_undegraded":"2024-05-09T05:34:33.906445+0000","last_fullsized":"2024-05-09T05:34:33.906445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296261+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:35:02.296261+0000","last_peered":"2024-05-09T05:35:02.296261+0000","last_clean":"2024-05-09T05:35:02.296261+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:35:02.296261+0000","last_undegraded":"2024-05-09T05:35:02.296261+0000","last_fullsized":"2024-05-09T05:35:02.296261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907641+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:34:33.907641+0000","last_peered":"2024-05-09T05:34:33.907641+0000","last_clean":"2024-05-09T05:34:33.907641+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:34:33.907641+0000","last_undegraded":"2024-05-09T05:34:33.907641+0000","last_fullsized":"2024-05-09T05:34:33.907641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048860+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:34:34.048860+0000","last_peered":"2024-05-09T05:34:34.048860+0000","last_clean":"2024-05-09T05:34:34.048860+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:34:34.048860+0000","last_undegraded":"2024-05-09T05:34:34.048860+0000","last_fullsized":"2024-05-09T05:34:34.048860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.293923+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:35:01.293923+0000","last_peered":"2024-05-09T05:35:01.293923+0000","last_clean":"2024-05-09T05:35:01.293923+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:35:01.293923+0000","last_undegraded":"2024-05-09T05:35:01.293923+0000","last_fullsized":"2024-05-09T05:35:01.293923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049020+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:34:34.049020+0000","last_peered":"2024-05-09T05:34:34.049020+0000","last_clean":"2024-05-09T05:34:34.049020+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:34:34.049020+0000","last_undegraded":"2024-05-09T05:34:34.049020+0000","last_fullsized":"2024-05-09T05:34:34.049020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049070+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:34:34.049070+0000","last_peered":"2024-05-09T05:34:34.049070+0000","last_clean":"2024-05-09T05:34:34.049070+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:34:34.049070+0000","last_undegraded":"2024-05-09T05:34:34.049070+0000","last_fullsized":"2024-05-09T05:34:34.049070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:59.269932+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:34:59.269932+0000","last_peered":"2024-05-09T05:34:59.269932+0000","last_clean":"2024-05-09T05:34:59.269932+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:34:59.269932+0000","last_undegraded":"2024-05-09T05:34:59.269932+0000","last_fullsized":"2024-05-09T05:34:59.269932+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905116+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:34:33.905116+0000","last_peered":"2024-05-09T05:34:33.905116+0000","last_clean":"2024-05-09T05:34:33.905116+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:34:33.905116+0000","last_undegraded":"2024-05-09T05:34:33.905116+0000","last_fullsized":"2024-05-09T05:34:33.905116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940110+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:34:33.940110+0000","last_peered":"2024-05-09T05:34:33.940110+0000","last_clean":"2024-05-09T05:34:33.940110+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:34:33.940110+0000","last_undegraded":"2024-05-09T05:34:33.940110+0000","last_fullsized":"2024-05-09T05:34:33.940110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:26.337152+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:35:26.337152+0000","last_peered":"2024-05-09T05:35:26.337152+0000","last_clean":"2024-05-09T05:35:26.337152+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:35:26.337152+0000","last_undegraded":"2024-05-09T05:35:26.337152+0000","last_fullsized":"2024-05-09T05:35:26.337152+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905564+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:34:33.905564+0000","last_peered":"2024-05-09T05:34:33.905564+0000","last_clean":"2024-05-09T05:34:33.905564+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:34:33.905564+0000","last_undegraded":"2024-05-09T05:34:33.905564+0000","last_fullsized":"2024-05-09T05:34:33.905564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904141+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:34:33.904141+0000","last_peered":"2024-05-09T05:34:33.904141+0000","last_clean":"2024-05-09T05:34:33.904141+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:34:33.904141+0000","last_undegraded":"2024-05-09T05:34:33.904141+0000","last_fullsized":"2024-05-09T05:34:33.904141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296831+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:35:02.296831+0000","last_peered":"2024-05-09T05:35:02.296831+0000","last_clean":"2024-05-09T05:35:02.296831+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:35:02.296831+0000","last_undegraded":"2024-05-09T05:35:02.296831+0000","last_fullsized":"2024-05-09T05:35:02.296831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905074+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:34:33.905074+0000","last_peered":"2024-05-09T05:34:33.905074+0000","last_clean":"2024-05-09T05:34:33.905074+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:34:33.905074+0000","last_undegraded":"2024-05-09T05:34:33.905074+0000","last_fullsized":"2024-05-09T05:34:33.905074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.942331+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:34:33.942331+0000","last_peered":"2024-05-09T05:34:33.942331+0000","last_clean":"2024-05-09T05:34:33.942331+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:34:33.942331+0000","last_undegraded":"2024-05-09T05:34:33.942331+0000","last_fullsized":"2024-05-09T05:34:33.942331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296507+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:35:02.296507+0000","last_peered":"2024-05-09T05:35:02.296507+0000","last_clean":"2024-05-09T05:35:02.296507+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:35:02.296507+0000","last_undegraded":"2024-05-09T05:35:02.296507+0000","last_fullsized":"2024-05-09T05:35:02.296507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905803+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:34:33.905803+0000","last_peered":"2024-05-09T05:34:33.905803+0000","last_clean":"2024-05-09T05:34:33.905803+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:34:33.905803+0000","last_undegraded":"2024-05-09T05:34:33.905803+0000","last_fullsized":"2024-05-09T05:34:33.905803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907549+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:34:33.907549+0000","last_peered":"2024-05-09T05:34:33.907549+0000","last_clean":"2024-05-09T05:34:33.907549+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:34:33.907549+0000","last_undegraded":"2024-05-09T05:34:33.907549+0000","last_fullsized":"2024-05-09T05:34:33.907549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939983+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:34:33.939983+0000","last_peered":"2024-05-09T05:34:33.939983+0000","last_clean":"2024-05-09T05:34:33.939983+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:34:33.939983+0000","last_undegraded":"2024-05-09T05:34:33.939983+0000","last_fullsized":"2024-05-09T05:34:33.939983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939482+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:34:33.939482+0000","last_peered":"2024-05-09T05:34:33.939482+0000","last_clean":"2024-05-09T05:34:33.939482+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:34:33.939482+0000","last_undegraded":"2024-05-09T05:34:33.939482+0000","last_fullsized":"2024-05-09T05:34:33.939482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904585+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:34:33.904585+0000","last_peered":"2024-05-09T05:34:33.904585+0000","last_clean":"2024-05-09T05:34:33.904585+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:34:33.904585+0000","last_undegraded":"2024-05-09T05:34:33.904585+0000","last_fullsized":"2024-05-09T05:34:33.904585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048073+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:34:34.048073+0000","last_peered":"2024-05-09T05:34:34.048073+0000","last_clean":"2024-05-09T05:34:34.048073+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:34:34.048073+0000","last_undegraded":"2024-05-09T05:34:34.048073+0000","last_fullsized":"2024-05-09T05:34:34.048073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.908251+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:34:33.908251+0000","last_peered":"2024-05-09T05:34:33.908251+0000","last_clean":"2024-05-09T05:34:33.908251+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:34:33.908251+0000","last_undegraded":"2024-05-09T05:34:33.908251+0000","last_fullsized":"2024-05-09T05:34:33.908251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940181+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:34:33.940181+0000","last_peered":"2024-05-09T05:34:33.940181+0000","last_clean":"2024-05-09T05:34:33.940181+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:34:33.940181+0000","last_undegraded":"2024-05-09T05:34:33.940181+0000","last_fullsized":"2024-05-09T05:34:33.940181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049464+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:34:34.049464+0000","last_peered":"2024-05-09T05:34:34.049464+0000","last_clean":"2024-05-09T05:34:34.049464+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:34:34.049464+0000","last_undegraded":"2024-05-09T05:34:34.049464+0000","last_fullsized":"2024-05-09T05:34:34.049464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.909834+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:34:33.909834+0000","last_peered":"2024-05-09T05:34:33.909834+0000","last_clean":"2024-05-09T05:34:33.909834+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:34:33.909834+0000","last_undegraded":"2024-05-09T05:34:33.909834+0000","last_fullsized":"2024-05-09T05:34:33.909834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048743+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:34:34.048743+0000","last_peered":"2024-05-09T05:34:34.048743+0000","last_clean":"2024-05-09T05:34:34.048743+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:34:34.048743+0000","last_undegraded":"2024-05-09T05:34:34.048743+0000","last_fullsized":"2024-05-09T05:34:34.048743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940296+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:34:33.940296+0000","last_peered":"2024-05-09T05:34:33.940296+0000","last_clean":"2024-05-09T05:34:33.940296+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:34:33.940296+0000","last_undegraded":"2024-05-09T05:34:33.940296+0000","last_fullsized":"2024-05-09T05:34:33.940296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.047967+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:34:34.047967+0000","last_peered":"2024-05-09T05:34:34.047967+0000","last_clean":"2024-05-09T05:34:34.047967+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:34:34.047967+0000","last_undegraded":"2024-05-09T05:34:34.047967+0000","last_fullsized":"2024-05-09T05:34:34.047967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.908055+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:34:33.908055+0000","last_peered":"2024-05-09T05:34:33.908055+0000","last_clean":"2024-05-09T05:34:33.908055+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:34:33.908055+0000","last_undegraded":"2024-05-09T05:34:33.908055+0000","last_fullsized":"2024-05-09T05:34:33.908055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213909,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64696,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":267,"num_read_kb":205,"num_write":248,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":188685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710753,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":496946,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906983,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":1136130,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070509,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":500860,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266738,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":497030,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462966,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1093639,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":492268,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":545778,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":493724,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":551792,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":491976,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":496342,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1089221,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:35:30.867 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T05:35:31.110 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:31.490 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:31 smithi115 ceph-mon[30349]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:35:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:31 smithi195 ceph-mon[33844]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:35:31.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:31 smithi161 ceph-mon[33805]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:35:32.556 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:32 smithi115 ceph-mon[30349]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-05-09T05:35:32.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:32 smithi195 ceph-mon[33844]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-05-09T05:35:32.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:32 smithi161 ceph-mon[33805]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-05-09T05:35:32.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.944+0000 7f2279058700 1 -- 172.21.15.115:0/525935003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2274107df0 msgr2=0x7f2274108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:32.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.944+0000 7f2279058700 1 --2- 172.21.15.115:0/525935003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2274107df0 0x7f2274108200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f22680099c0 tx=0x7f2268009cd0 comp rx=0 tx=0).stop 2024-05-09T05:35:32.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.944+0000 7f2279058700 1 -- 172.21.15.115:0/525935003 shutdown_connections 2024-05-09T05:35:32.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.944+0000 7f2279058700 1 --2- 172.21.15.115:0/525935003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2274108740 0x7f22741100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:32.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.944+0000 7f2279058700 1 --2- 172.21.15.115:0/525935003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22741034c0 0x7f22741038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:32.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.944+0000 7f2279058700 1 --2- 172.21.15.115:0/525935003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2274107df0 0x7f2274108200 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:32.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.944+0000 7f2279058700 1 -- 172.21.15.115:0/525935003 >> 172.21.15.115:0/525935003 conn(0x7f22740fedf0 msgr2=0x7f2274101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:32.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.944+0000 7f2279058700 1 -- 172.21.15.115:0/525935003 shutdown_connections 2024-05-09T05:35:32.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.944+0000 7f2279058700 1 -- 172.21.15.115:0/525935003 wait complete. 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.945+0000 7f2279058700 1 Processor -- start 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.945+0000 7f2279058700 1 -- start start 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.945+0000 7f2279058700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22741034c0 0x7f227419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.945+0000 7f2279058700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2274107df0 0x7f227419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.945+0000 7f2279058700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274108740 0x7f227419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.945+0000 7f2279058700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2274113200 con 0x7f22741034c0 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.945+0000 7f2279058700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2274113080 con 0x7f2274107df0 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.945+0000 7f2279058700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2274113380 con 0x7f2274108740 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.945+0000 7f2272d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22741034c0 0x7f227419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2272d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22741034c0 0x7f227419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36880/0 (socket says 172.21.15.115:36880) 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2272d9d700 1 -- 172.21.15.115:0/599144849 learned_addr learned my addr 172.21.15.115:0/599144849 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f227259c700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2274107df0 0x7f227419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f227359e700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274108740 0x7f227419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2272d9d700 1 -- 172.21.15.115:0/599144849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274108740 msgr2=0x7f227419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2272d9d700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274108740 0x7f227419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2272d9d700 1 -- 172.21.15.115:0/599144849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2274107df0 msgr2=0x7f227419c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2272d9d700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2274107df0 0x7f227419c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2272d9d700 1 -- 172.21.15.115:0/599144849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22741a6f10 con 0x7f22741034c0 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f227259c700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2274107df0 0x7f227419c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2272d9d700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22741034c0 0x7f227419bba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f226400c920 tx=0x7f226400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f225bfff700 1 -- 172.21.15.115:0/599144849 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264007b00 con 0x7f22741034c0 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22741a7200 con 0x7f22741034c0 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f225bfff700 1 -- 172.21.15.115:0/599144849 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2264004cc0 con 0x7f22741034c0 2024-05-09T05:35:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f225bfff700 1 -- 172.21.15.115:0/599144849 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640054a0 con 0x7f22741034c0 2024-05-09T05:35:32.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.946+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22741a7740 con 0x7f22741034c0 2024-05-09T05:35:32.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.947+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22740512c0 con 0x7f22741034c0 2024-05-09T05:35:32.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.949+0000 7f225bfff700 1 -- 172.21.15.115:0/599144849 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2264004750 con 0x7f22741034c0 2024-05-09T05:35:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.949+0000 7f225bfff700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f225c0b2910 0x7f225c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.950+0000 7f225bfff700 1 -- 172.21.15.115:0/599144849 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f22640e2730 con 0x7f22741034c0 2024-05-09T05:35:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.950+0000 7f227259c700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f225c0b2910 0x7f225c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.950+0000 7f227259c700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f225c0b2910 0x7f225c0b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2268006010 tx=0x7f2268003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:32.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:32.953+0000 7f225bfff700 1 -- 172.21.15.115:0/599144849 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2264014030 con 0x7f22741034c0 2024-05-09T05:35:33.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.153+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2274063c00 con 0x7f225c0b2910 2024-05-09T05:35:33.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.163+0000 7f225bfff700 1 -- 172.21.15.115:0/599144849 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340619 (secure 0 0 0) 0x7f2274063c00 con 0x7f225c0b2910 2024-05-09T05:35:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:35:33.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f225c0b2910 msgr2=0x7f225c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:33.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f225c0b2910 0x7f225c0b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2268006010 tx=0x7f2268003680 comp rx=0 tx=0).stop 2024-05-09T05:35:33.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22741034c0 msgr2=0x7f227419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:33.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22741034c0 0x7f227419bba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f226400c920 tx=0x7f226400cce0 comp rx=0 tx=0).stop 2024-05-09T05:35:33.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 shutdown_connections 2024-05-09T05:35:33.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f225c0b2910 0x7f225c0b4e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:33.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2274108740 0x7f227419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:33.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22741034c0 0x7f227419bba0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 --2- 172.21.15.115:0/599144849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2274107df0 0x7f227419c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 >> 172.21.15.115:0/599144849 conn(0x7f22740fedf0 msgr2=0x7f2274101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.166+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 shutdown_connections 2024-05-09T05:35:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:33.167+0000 7f2279058700 1 -- 172.21.15.115:0/599144849 wait complete. 2024-05-09T05:35:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:35:33.925 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-05-09T05:35:32.681655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661551,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":384,"num_read_kb":301,"num_write":375,"num_write_kb":2295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344324,"kb_used_data":11812,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597500,"statfs":{"total":1151940427776,"available":1151587840000,"internally_reserved":0,"allocated":12095488,"data_stored":7902765,"data_compressed":426138,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11656,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":63,"num_write":15,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003950"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.908270+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:34:33.908270+0000","last_peered":"2024-05-09T05:34:33.908270+0000","last_clean":"2024-05-09T05:34:33.908270+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:34:33.908270+0000","last_undegraded":"2024-05-09T05:34:33.908270+0000","last_fullsized":"2024-05-09T05:34:33.908270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904243+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:34:33.904243+0000","last_peered":"2024-05-09T05:34:33.904243+0000","last_clean":"2024-05-09T05:34:33.904243+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:34:33.904243+0000","last_undegraded":"2024-05-09T05:34:33.904243+0000","last_fullsized":"2024-05-09T05:34:33.904243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904227+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:34:33.904227+0000","last_peered":"2024-05-09T05:34:33.904227+0000","last_clean":"2024-05-09T05:34:33.904227+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:34:33.904227+0000","last_undegraded":"2024-05-09T05:34:33.904227+0000","last_fullsized":"2024-05-09T05:34:33.904227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049163+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:34:34.049163+0000","last_peered":"2024-05-09T05:34:34.049163+0000","last_clean":"2024-05-09T05:34:34.049163+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:34:34.049163+0000","last_undegraded":"2024-05-09T05:34:34.049163+0000","last_fullsized":"2024-05-09T05:34:34.049163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904724+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:34:33.904724+0000","last_peered":"2024-05-09T05:34:33.904724+0000","last_clean":"2024-05-09T05:34:33.904724+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:34:33.904724+0000","last_undegraded":"2024-05-09T05:34:33.904724+0000","last_fullsized":"2024-05-09T05:34:33.904724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906840+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:34:33.906840+0000","last_peered":"2024-05-09T05:34:33.906840+0000","last_clean":"2024-05-09T05:34:33.906840+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:34:33.906840+0000","last_undegraded":"2024-05-09T05:34:33.906840+0000","last_fullsized":"2024-05-09T05:34:33.906840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904786+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:34:33.904786+0000","last_peered":"2024-05-09T05:34:33.904786+0000","last_clean":"2024-05-09T05:34:33.904786+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:34:33.904786+0000","last_undegraded":"2024-05-09T05:34:33.904786+0000","last_fullsized":"2024-05-09T05:34:33.904786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048405+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:34:34.048405+0000","last_peered":"2024-05-09T05:34:34.048405+0000","last_clean":"2024-05-09T05:34:34.048405+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:34:34.048405+0000","last_undegraded":"2024-05-09T05:34:34.048405+0000","last_fullsized":"2024-05-09T05:34:34.048405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.941003+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:34:33.941003+0000","last_peered":"2024-05-09T05:34:33.941003+0000","last_clean":"2024-05-09T05:34:33.941003+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:34:33.941003+0000","last_undegraded":"2024-05-09T05:34:33.941003+0000","last_fullsized":"2024-05-09T05:34:33.941003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.941852+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:34:33.941852+0000","last_peered":"2024-05-09T05:34:33.941852+0000","last_clean":"2024-05-09T05:34:33.941852+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:34:33.941852+0000","last_undegraded":"2024-05-09T05:34:33.941852+0000","last_fullsized":"2024-05-09T05:34:33.941852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905343+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:34:33.905343+0000","last_peered":"2024-05-09T05:34:33.905343+0000","last_clean":"2024-05-09T05:34:33.905343+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:34:33.905343+0000","last_undegraded":"2024-05-09T05:34:33.905343+0000","last_fullsized":"2024-05-09T05:34:33.905343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049091+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:34:34.049091+0000","last_peered":"2024-05-09T05:34:34.049091+0000","last_clean":"2024-05-09T05:34:34.049091+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:34:34.049091+0000","last_undegraded":"2024-05-09T05:34:34.049091+0000","last_fullsized":"2024-05-09T05:34:34.049091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048174+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:34:34.048174+0000","last_peered":"2024-05-09T05:34:34.048174+0000","last_clean":"2024-05-09T05:34:34.048174+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:34:34.048174+0000","last_undegraded":"2024-05-09T05:34:34.048174+0000","last_fullsized":"2024-05-09T05:34:34.048174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904098+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:34:33.904098+0000","last_peered":"2024-05-09T05:34:33.904098+0000","last_clean":"2024-05-09T05:34:33.904098+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:34:33.904098+0000","last_undegraded":"2024-05-09T05:34:33.904098+0000","last_fullsized":"2024-05-09T05:34:33.904098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904027+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:34:33.904027+0000","last_peered":"2024-05-09T05:34:33.904027+0000","last_clean":"2024-05-09T05:34:33.904027+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:34:33.904027+0000","last_undegraded":"2024-05-09T05:34:33.904027+0000","last_fullsized":"2024-05-09T05:34:33.904027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939791+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:34:33.939791+0000","last_peered":"2024-05-09T05:34:33.939791+0000","last_clean":"2024-05-09T05:34:33.939791+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:34:33.939791+0000","last_undegraded":"2024-05-09T05:34:33.939791+0000","last_fullsized":"2024-05-09T05:34:33.939791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907780+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:34:33.907780+0000","last_peered":"2024-05-09T05:34:33.907780+0000","last_clean":"2024-05-09T05:34:33.907780+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:34:33.907780+0000","last_undegraded":"2024-05-09T05:34:33.907780+0000","last_fullsized":"2024-05-09T05:34:33.907780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906558+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:34:33.906558+0000","last_peered":"2024-05-09T05:34:33.906558+0000","last_clean":"2024-05-09T05:34:33.906558+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:34:33.906558+0000","last_undegraded":"2024-05-09T05:34:33.906558+0000","last_fullsized":"2024-05-09T05:34:33.906558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940328+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:34:33.940328+0000","last_peered":"2024-05-09T05:34:33.940328+0000","last_clean":"2024-05-09T05:34:33.940328+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:34:33.940328+0000","last_undegraded":"2024-05-09T05:34:33.940328+0000","last_fullsized":"2024-05-09T05:34:33.940328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939639+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:34:33.939639+0000","last_peered":"2024-05-09T05:34:33.939639+0000","last_clean":"2024-05-09T05:34:33.939639+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:34:33.939639+0000","last_undegraded":"2024-05-09T05:34:33.939639+0000","last_fullsized":"2024-05-09T05:34:33.939639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:27.337262+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:35:27.337262+0000","last_peered":"2024-05-09T05:35:27.337262+0000","last_clean":"2024-05-09T05:35:27.337262+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:35:27.337262+0000","last_undegraded":"2024-05-09T05:35:27.337262+0000","last_fullsized":"2024-05-09T05:35:27.337262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049103+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:34:34.049103+0000","last_peered":"2024-05-09T05:34:34.049103+0000","last_clean":"2024-05-09T05:34:34.049103+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:34:34.049103+0000","last_undegraded":"2024-05-09T05:34:34.049103+0000","last_fullsized":"2024-05-09T05:34:34.049103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048353+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:34:34.048353+0000","last_peered":"2024-05-09T05:34:34.048353+0000","last_clean":"2024-05-09T05:34:34.048353+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:34:34.048353+0000","last_undegraded":"2024-05-09T05:34:34.048353+0000","last_fullsized":"2024-05-09T05:34:34.048353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049329+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:34:34.049329+0000","last_peered":"2024-05-09T05:34:34.049329+0000","last_clean":"2024-05-09T05:34:34.049329+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:34:34.049329+0000","last_undegraded":"2024-05-09T05:34:34.049329+0000","last_fullsized":"2024-05-09T05:34:34.049329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904216+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:34:33.904216+0000","last_peered":"2024-05-09T05:34:33.904216+0000","last_clean":"2024-05-09T05:34:33.904216+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:34:33.904216+0000","last_undegraded":"2024-05-09T05:34:33.904216+0000","last_fullsized":"2024-05-09T05:34:33.904216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048820+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:34:34.048820+0000","last_peered":"2024-05-09T05:34:34.048820+0000","last_clean":"2024-05-09T05:34:34.048820+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:34:34.048820+0000","last_undegraded":"2024-05-09T05:34:34.048820+0000","last_fullsized":"2024-05-09T05:34:34.048820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301610+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:35:03.301610+0000","last_peered":"2024-05-09T05:35:03.301610+0000","last_clean":"2024-05-09T05:35:03.301610+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:35:03.301610+0000","last_undegraded":"2024-05-09T05:35:03.301610+0000","last_fullsized":"2024-05-09T05:35:03.301610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940113+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:34:33.940113+0000","last_peered":"2024-05-09T05:34:33.940113+0000","last_clean":"2024-05-09T05:34:33.940113+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:34:33.940113+0000","last_undegraded":"2024-05-09T05:34:33.940113+0000","last_fullsized":"2024-05-09T05:34:33.940113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048328+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:34:34.048328+0000","last_peered":"2024-05-09T05:34:34.048328+0000","last_clean":"2024-05-09T05:34:34.048328+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:34:34.048328+0000","last_undegraded":"2024-05-09T05:34:34.048328+0000","last_fullsized":"2024-05-09T05:34:34.048328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296672+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:35:02.296672+0000","last_peered":"2024-05-09T05:35:02.296672+0000","last_clean":"2024-05-09T05:35:02.296672+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:35:02.296672+0000","last_undegraded":"2024-05-09T05:35:02.296672+0000","last_fullsized":"2024-05-09T05:35:02.296672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.941216+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:34:33.941216+0000","last_peered":"2024-05-09T05:34:33.941216+0000","last_clean":"2024-05-09T05:34:33.941216+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:34:33.941216+0000","last_undegraded":"2024-05-09T05:34:33.941216+0000","last_fullsized":"2024-05-09T05:34:33.941216+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939991+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:34:33.939991+0000","last_peered":"2024-05-09T05:34:33.939991+0000","last_clean":"2024-05-09T05:34:33.939991+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:34:33.939991+0000","last_undegraded":"2024-05-09T05:34:33.939991+0000","last_fullsized":"2024-05-09T05:34:33.939991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:27.338755+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:35:27.338755+0000","last_peered":"2024-05-09T05:35:27.338755+0000","last_clean":"2024-05-09T05:35:27.338755+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:35:27.338755+0000","last_undegraded":"2024-05-09T05:35:27.338755+0000","last_fullsized":"2024-05-09T05:35:27.338755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904664+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:34:33.904664+0000","last_peered":"2024-05-09T05:34:33.904664+0000","last_clean":"2024-05-09T05:34:33.904664+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:34:33.904664+0000","last_undegraded":"2024-05-09T05:34:33.904664+0000","last_fullsized":"2024-05-09T05:34:33.904664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940116+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:34:33.940116+0000","last_peered":"2024-05-09T05:34:33.940116+0000","last_clean":"2024-05-09T05:34:33.940116+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:34:33.940116+0000","last_undegraded":"2024-05-09T05:34:33.940116+0000","last_fullsized":"2024-05-09T05:34:33.940116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.292778+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:35:01.292778+0000","last_peered":"2024-05-09T05:35:01.292778+0000","last_clean":"2024-05-09T05:35:01.292778+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:35:01.292778+0000","last_undegraded":"2024-05-09T05:35:01.292778+0000","last_fullsized":"2024-05-09T05:35:01.292778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.910937+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:34:33.910937+0000","last_peered":"2024-05-09T05:34:33.910937+0000","last_clean":"2024-05-09T05:34:33.910937+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:34:33.910937+0000","last_undegraded":"2024-05-09T05:34:33.910937+0000","last_fullsized":"2024-05-09T05:34:33.910937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.051017+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:34:34.051017+0000","last_peered":"2024-05-09T05:34:34.051017+0000","last_clean":"2024-05-09T05:34:34.051017+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:34:34.051017+0000","last_undegraded":"2024-05-09T05:34:34.051017+0000","last_fullsized":"2024-05-09T05:34:34.051017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:00.275070+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:35:00.275070+0000","last_peered":"2024-05-09T05:35:00.275070+0000","last_clean":"2024-05-09T05:35:00.275070+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:35:00.275070+0000","last_undegraded":"2024-05-09T05:35:00.275070+0000","last_fullsized":"2024-05-09T05:35:00.275070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048513+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:34:34.048513+0000","last_peered":"2024-05-09T05:34:34.048513+0000","last_clean":"2024-05-09T05:34:34.048513+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:34:34.048513+0000","last_undegraded":"2024-05-09T05:34:34.048513+0000","last_fullsized":"2024-05-09T05:34:34.048513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904125+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:34:33.904125+0000","last_peered":"2024-05-09T05:34:33.904125+0000","last_clean":"2024-05-09T05:34:33.904125+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:34:33.904125+0000","last_undegraded":"2024-05-09T05:34:33.904125+0000","last_fullsized":"2024-05-09T05:34:33.904125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.297801+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:35:02.297801+0000","last_peered":"2024-05-09T05:35:02.297801+0000","last_clean":"2024-05-09T05:35:02.297801+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:35:02.297801+0000","last_undegraded":"2024-05-09T05:35:02.297801+0000","last_fullsized":"2024-05-09T05:35:02.297801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907202+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:34:33.907202+0000","last_peered":"2024-05-09T05:34:33.907202+0000","last_clean":"2024-05-09T05:34:33.907202+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:34:33.907202+0000","last_undegraded":"2024-05-09T05:34:33.907202+0000","last_fullsized":"2024-05-09T05:34:33.907202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048650+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:34:34.048650+0000","last_peered":"2024-05-09T05:34:34.048650+0000","last_clean":"2024-05-09T05:34:34.048650+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:34:34.048650+0000","last_undegraded":"2024-05-09T05:34:34.048650+0000","last_fullsized":"2024-05-09T05:34:34.048650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:29.340768+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:35:29.340768+0000","last_peered":"2024-05-09T05:35:29.340768+0000","last_clean":"2024-05-09T05:35:29.340768+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:35:29.340768+0000","last_undegraded":"2024-05-09T05:35:29.340768+0000","last_fullsized":"2024-05-09T05:35:29.340768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940740+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:34:33.940740+0000","last_peered":"2024-05-09T05:34:33.940740+0000","last_clean":"2024-05-09T05:34:33.940740+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:34:33.940740+0000","last_undegraded":"2024-05-09T05:34:33.940740+0000","last_fullsized":"2024-05-09T05:34:33.940740+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907283+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:34:33.907283+0000","last_peered":"2024-05-09T05:34:33.907283+0000","last_clean":"2024-05-09T05:34:33.907283+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:34:33.907283+0000","last_undegraded":"2024-05-09T05:34:33.907283+0000","last_fullsized":"2024-05-09T05:34:33.907283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.297309+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:35:02.297309+0000","last_peered":"2024-05-09T05:35:02.297309+0000","last_clean":"2024-05-09T05:35:02.297309+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:35:02.297309+0000","last_undegraded":"2024-05-09T05:35:02.297309+0000","last_fullsized":"2024-05-09T05:35:02.297309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904125+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:34:33.904125+0000","last_peered":"2024-05-09T05:34:33.904125+0000","last_clean":"2024-05-09T05:34:33.904125+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:34:33.904125+0000","last_undegraded":"2024-05-09T05:34:33.904125+0000","last_fullsized":"2024-05-09T05:34:33.904125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049012+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:34:34.049012+0000","last_peered":"2024-05-09T05:34:34.049012+0000","last_clean":"2024-05-09T05:34:34.049012+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:34:34.049012+0000","last_undegraded":"2024-05-09T05:34:34.049012+0000","last_fullsized":"2024-05-09T05:34:34.049012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.292960+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:35:01.292960+0000","last_peered":"2024-05-09T05:35:01.292960+0000","last_clean":"2024-05-09T05:35:01.292960+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:35:01.292960+0000","last_undegraded":"2024-05-09T05:35:01.292960+0000","last_fullsized":"2024-05-09T05:35:01.292960+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940268+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:34:33.940268+0000","last_peered":"2024-05-09T05:34:33.940268+0000","last_clean":"2024-05-09T05:34:33.940268+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:34:33.940268+0000","last_undegraded":"2024-05-09T05:34:33.940268+0000","last_fullsized":"2024-05-09T05:34:33.940268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906314+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:34:33.906314+0000","last_peered":"2024-05-09T05:34:33.906314+0000","last_clean":"2024-05-09T05:34:33.906314+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:34:33.906314+0000","last_undegraded":"2024-05-09T05:34:33.906314+0000","last_fullsized":"2024-05-09T05:34:33.906314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:29.346301+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:35:29.346301+0000","last_peered":"2024-05-09T05:35:29.346301+0000","last_clean":"2024-05-09T05:35:29.346301+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:35:29.346301+0000","last_undegraded":"2024-05-09T05:35:29.346301+0000","last_fullsized":"2024-05-09T05:35:29.346301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64136,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":63,"num_read_kb":94,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905715+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:34:33.905715+0000","last_peered":"2024-05-09T05:34:33.905715+0000","last_clean":"2024-05-09T05:34:33.905715+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:34:33.905715+0000","last_undegraded":"2024-05-09T05:34:33.905715+0000","last_fullsized":"2024-05-09T05:34:33.905715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.911509+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:34:33.911509+0000","last_peered":"2024-05-09T05:34:33.911509+0000","last_clean":"2024-05-09T05:34:33.911509+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:34:33.911509+0000","last_undegraded":"2024-05-09T05:34:33.911509+0000","last_fullsized":"2024-05-09T05:34:33.911509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301044+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:35:03.301044+0000","last_peered":"2024-05-09T05:35:03.301044+0000","last_clean":"2024-05-09T05:35:03.301044+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:35:03.301044+0000","last_undegraded":"2024-05-09T05:35:03.301044+0000","last_fullsized":"2024-05-09T05:35:03.301044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.047970+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:34:34.047970+0000","last_peered":"2024-05-09T05:34:34.047970+0000","last_clean":"2024-05-09T05:34:34.047970+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:34:34.047970+0000","last_undegraded":"2024-05-09T05:34:34.047970+0000","last_fullsized":"2024-05-09T05:34:34.047970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907808+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:34:33.907808+0000","last_peered":"2024-05-09T05:34:33.907808+0000","last_clean":"2024-05-09T05:34:33.907808+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:34:33.907808+0000","last_undegraded":"2024-05-09T05:34:33.907808+0000","last_fullsized":"2024-05-09T05:34:33.907808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296993+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:35:02.296993+0000","last_peered":"2024-05-09T05:35:02.296993+0000","last_clean":"2024-05-09T05:35:02.296993+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:35:02.296993+0000","last_undegraded":"2024-05-09T05:35:02.296993+0000","last_fullsized":"2024-05-09T05:35:02.296993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904986+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:34:33.904986+0000","last_peered":"2024-05-09T05:34:33.904986+0000","last_clean":"2024-05-09T05:34:33.904986+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:34:33.904986+0000","last_undegraded":"2024-05-09T05:34:33.904986+0000","last_fullsized":"2024-05-09T05:34:33.904986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.051709+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:34:34.051709+0000","last_peered":"2024-05-09T05:34:34.051709+0000","last_clean":"2024-05-09T05:34:34.051709+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:34:34.051709+0000","last_undegraded":"2024-05-09T05:34:34.051709+0000","last_fullsized":"2024-05-09T05:34:34.051709+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:26.336788+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:35:26.336788+0000","last_peered":"2024-05-09T05:35:26.336788+0000","last_clean":"2024-05-09T05:35:26.336788+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:35:26.336788+0000","last_undegraded":"2024-05-09T05:35:26.336788+0000","last_fullsized":"2024-05-09T05:35:26.336788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905083+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:34:33.905083+0000","last_peered":"2024-05-09T05:34:33.905083+0000","last_clean":"2024-05-09T05:34:33.905083+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:34:33.905083+0000","last_undegraded":"2024-05-09T05:34:33.905083+0000","last_fullsized":"2024-05-09T05:34:33.905083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048527+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:34:34.048527+0000","last_peered":"2024-05-09T05:34:34.048527+0000","last_clean":"2024-05-09T05:34:34.048527+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:34:34.048527+0000","last_undegraded":"2024-05-09T05:34:34.048527+0000","last_fullsized":"2024-05-09T05:34:34.048527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.295954+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:35:02.295954+0000","last_peered":"2024-05-09T05:35:02.295954+0000","last_clean":"2024-05-09T05:35:02.295954+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:35:02.295954+0000","last_undegraded":"2024-05-09T05:35:02.295954+0000","last_fullsized":"2024-05-09T05:35:02.295954+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049898+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:34:34.049898+0000","last_peered":"2024-05-09T05:34:34.049898+0000","last_clean":"2024-05-09T05:34:34.049898+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:34:34.049898+0000","last_undegraded":"2024-05-09T05:34:34.049898+0000","last_fullsized":"2024-05-09T05:34:34.049898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.910567+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:34:33.910567+0000","last_peered":"2024-05-09T05:34:33.910567+0000","last_clean":"2024-05-09T05:34:33.910567+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:34:33.910567+0000","last_undegraded":"2024-05-09T05:34:33.910567+0000","last_fullsized":"2024-05-09T05:34:33.910567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.270702+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:35:02.270702+0000","last_peered":"2024-05-09T05:35:02.270702+0000","last_clean":"2024-05-09T05:35:02.270702+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:35:02.270702+0000","last_undegraded":"2024-05-09T05:35:02.270702+0000","last_fullsized":"2024-05-09T05:35:02.270702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907804+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:34:33.907804+0000","last_peered":"2024-05-09T05:34:33.907804+0000","last_clean":"2024-05-09T05:34:33.907804+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:34:33.907804+0000","last_undegraded":"2024-05-09T05:34:33.907804+0000","last_fullsized":"2024-05-09T05:34:33.907804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.942337+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:34:33.942337+0000","last_peered":"2024-05-09T05:34:33.942337+0000","last_clean":"2024-05-09T05:34:33.942337+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:34:33.942337+0000","last_undegraded":"2024-05-09T05:34:33.942337+0000","last_fullsized":"2024-05-09T05:34:33.942337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.912172+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:34:33.912172+0000","last_peered":"2024-05-09T05:34:33.912172+0000","last_clean":"2024-05-09T05:34:33.912172+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:34:33.912172+0000","last_undegraded":"2024-05-09T05:34:33.912172+0000","last_fullsized":"2024-05-09T05:34:33.912172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907623+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:34:33.907623+0000","last_peered":"2024-05-09T05:34:33.907623+0000","last_clean":"2024-05-09T05:34:33.907623+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:34:33.907623+0000","last_undegraded":"2024-05-09T05:34:33.907623+0000","last_fullsized":"2024-05-09T05:34:33.907623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048006+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:34:34.048006+0000","last_peered":"2024-05-09T05:34:34.048006+0000","last_clean":"2024-05-09T05:34:34.048006+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:34:34.048006+0000","last_undegraded":"2024-05-09T05:34:34.048006+0000","last_fullsized":"2024-05-09T05:34:34.048006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:27.337973+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:35:27.337973+0000","last_peered":"2024-05-09T05:35:27.337973+0000","last_clean":"2024-05-09T05:35:27.337973+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:35:27.337973+0000","last_undegraded":"2024-05-09T05:35:27.337973+0000","last_fullsized":"2024-05-09T05:35:27.337973+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.913062+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:34:33.913062+0000","last_peered":"2024-05-09T05:34:33.913062+0000","last_clean":"2024-05-09T05:34:33.913062+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:34:33.913062+0000","last_undegraded":"2024-05-09T05:34:33.913062+0000","last_fullsized":"2024-05-09T05:34:33.913062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048401+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:34:34.048401+0000","last_peered":"2024-05-09T05:34:34.048401+0000","last_clean":"2024-05-09T05:34:34.048401+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:34:34.048401+0000","last_undegraded":"2024-05-09T05:34:34.048401+0000","last_fullsized":"2024-05-09T05:34:34.048401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.910013+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:34:33.910013+0000","last_peered":"2024-05-09T05:34:33.910013+0000","last_clean":"2024-05-09T05:34:33.910013+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:34:33.910013+0000","last_undegraded":"2024-05-09T05:34:33.910013+0000","last_fullsized":"2024-05-09T05:34:33.910013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:28.338751+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:35:28.338751+0000","last_peered":"2024-05-09T05:35:28.338751+0000","last_clean":"2024-05-09T05:35:28.338751+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:35:28.338751+0000","last_undegraded":"2024-05-09T05:35:28.338751+0000","last_fullsized":"2024-05-09T05:35:28.338751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905633+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:34:33.905633+0000","last_peered":"2024-05-09T05:34:33.905633+0000","last_clean":"2024-05-09T05:34:33.905633+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:34:33.905633+0000","last_undegraded":"2024-05-09T05:34:33.905633+0000","last_fullsized":"2024-05-09T05:34:33.905633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048889+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:34:34.048889+0000","last_peered":"2024-05-09T05:34:34.048889+0000","last_clean":"2024-05-09T05:34:34.048889+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:34:34.048889+0000","last_undegraded":"2024-05-09T05:34:34.048889+0000","last_fullsized":"2024-05-09T05:34:34.048889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.302304+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:35:03.302304+0000","last_peered":"2024-05-09T05:35:03.302304+0000","last_clean":"2024-05-09T05:35:03.302304+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:35:03.302304+0000","last_undegraded":"2024-05-09T05:35:03.302304+0000","last_fullsized":"2024-05-09T05:35:03.302304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904811+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:34:33.904811+0000","last_peered":"2024-05-09T05:34:33.904811+0000","last_clean":"2024-05-09T05:34:33.904811+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:34:33.904811+0000","last_undegraded":"2024-05-09T05:34:33.904811+0000","last_fullsized":"2024-05-09T05:34:33.904811+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.911099+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:34:33.911099+0000","last_peered":"2024-05-09T05:34:33.911099+0000","last_clean":"2024-05-09T05:34:33.911099+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:34:33.911099+0000","last_undegraded":"2024-05-09T05:34:33.911099+0000","last_fullsized":"2024-05-09T05:34:33.911099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048107+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:34:34.048107+0000","last_peered":"2024-05-09T05:34:34.048107+0000","last_clean":"2024-05-09T05:34:34.048107+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:34:34.048107+0000","last_undegraded":"2024-05-09T05:34:34.048107+0000","last_fullsized":"2024-05-09T05:34:34.048107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:00.276507+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:35:00.276507+0000","last_peered":"2024-05-09T05:35:00.276507+0000","last_clean":"2024-05-09T05:35:00.276507+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:35:00.276507+0000","last_undegraded":"2024-05-09T05:35:00.276507+0000","last_fullsized":"2024-05-09T05:35:00.276507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907526+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:34:33.907526+0000","last_peered":"2024-05-09T05:34:33.907526+0000","last_clean":"2024-05-09T05:34:33.907526+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:34:33.907526+0000","last_undegraded":"2024-05-09T05:34:33.907526+0000","last_fullsized":"2024-05-09T05:34:33.907526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.295356+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:35:02.295356+0000","last_peered":"2024-05-09T05:35:02.295356+0000","last_clean":"2024-05-09T05:35:02.295356+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:35:02.295356+0000","last_undegraded":"2024-05-09T05:35:02.295356+0000","last_fullsized":"2024-05-09T05:35:02.295356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048307+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:34:34.048307+0000","last_peered":"2024-05-09T05:34:34.048307+0000","last_clean":"2024-05-09T05:34:34.048307+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:34:34.048307+0000","last_undegraded":"2024-05-09T05:34:34.048307+0000","last_fullsized":"2024-05-09T05:34:34.048307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048421+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:34:34.048421+0000","last_peered":"2024-05-09T05:34:34.048421+0000","last_clean":"2024-05-09T05:34:34.048421+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:34:34.048421+0000","last_undegraded":"2024-05-09T05:34:34.048421+0000","last_fullsized":"2024-05-09T05:34:34.048421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'16","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:23.793428+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:35:23.793428+0000","last_peered":"2024-05-09T05:35:23.793428+0000","last_clean":"2024-05-09T05:35:23.793428+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:35:23.793428+0000","last_undegraded":"2024-05-09T05:35:23.793428+0000","last_fullsized":"2024-05-09T05:35:23.793428+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049225+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:34:34.049225+0000","last_peered":"2024-05-09T05:34:34.049225+0000","last_clean":"2024-05-09T05:34:34.049225+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:34:34.049225+0000","last_undegraded":"2024-05-09T05:34:34.049225+0000","last_fullsized":"2024-05-09T05:34:34.049225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939294+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:34:33.939294+0000","last_peered":"2024-05-09T05:34:33.939294+0000","last_clean":"2024-05-09T05:34:33.939294+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:34:33.939294+0000","last_undegraded":"2024-05-09T05:34:33.939294+0000","last_fullsized":"2024-05-09T05:34:33.939294+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.295838+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:35:02.295838+0000","last_peered":"2024-05-09T05:35:02.295838+0000","last_clean":"2024-05-09T05:35:02.295838+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:35:02.295838+0000","last_undegraded":"2024-05-09T05:35:02.295838+0000","last_fullsized":"2024-05-09T05:35:02.295838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904345+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:34:33.904345+0000","last_peered":"2024-05-09T05:34:33.904345+0000","last_clean":"2024-05-09T05:34:33.904345+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:34:33.904345+0000","last_undegraded":"2024-05-09T05:34:33.904345+0000","last_fullsized":"2024-05-09T05:34:33.904345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049243+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:34:34.049243+0000","last_peered":"2024-05-09T05:34:34.049243+0000","last_clean":"2024-05-09T05:34:34.049243+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:34:34.049243+0000","last_undegraded":"2024-05-09T05:34:34.049243+0000","last_fullsized":"2024-05-09T05:34:34.049243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:59.269115+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:34:59.269115+0000","last_peered":"2024-05-09T05:34:59.269115+0000","last_clean":"2024-05-09T05:34:59.269115+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:34:59.269115+0000","last_undegraded":"2024-05-09T05:34:59.269115+0000","last_fullsized":"2024-05-09T05:34:59.269115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904547+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:34:33.904547+0000","last_peered":"2024-05-09T05:34:33.904547+0000","last_clean":"2024-05-09T05:34:33.904547+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:34:33.904547+0000","last_undegraded":"2024-05-09T05:34:33.904547+0000","last_fullsized":"2024-05-09T05:34:33.904547+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906445+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:34:33.906445+0000","last_peered":"2024-05-09T05:34:33.906445+0000","last_clean":"2024-05-09T05:34:33.906445+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:34:33.906445+0000","last_undegraded":"2024-05-09T05:34:33.906445+0000","last_fullsized":"2024-05-09T05:34:33.906445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296261+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:35:02.296261+0000","last_peered":"2024-05-09T05:35:02.296261+0000","last_clean":"2024-05-09T05:35:02.296261+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:35:02.296261+0000","last_undegraded":"2024-05-09T05:35:02.296261+0000","last_fullsized":"2024-05-09T05:35:02.296261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907641+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:34:33.907641+0000","last_peered":"2024-05-09T05:34:33.907641+0000","last_clean":"2024-05-09T05:34:33.907641+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:34:33.907641+0000","last_undegraded":"2024-05-09T05:34:33.907641+0000","last_fullsized":"2024-05-09T05:34:33.907641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048860+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:34:34.048860+0000","last_peered":"2024-05-09T05:34:34.048860+0000","last_clean":"2024-05-09T05:34:34.048860+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:34:34.048860+0000","last_undegraded":"2024-05-09T05:34:34.048860+0000","last_fullsized":"2024-05-09T05:34:34.048860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.293923+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:35:01.293923+0000","last_peered":"2024-05-09T05:35:01.293923+0000","last_clean":"2024-05-09T05:35:01.293923+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:35:01.293923+0000","last_undegraded":"2024-05-09T05:35:01.293923+0000","last_fullsized":"2024-05-09T05:35:01.293923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049020+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:34:34.049020+0000","last_peered":"2024-05-09T05:34:34.049020+0000","last_clean":"2024-05-09T05:34:34.049020+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:34:34.049020+0000","last_undegraded":"2024-05-09T05:34:34.049020+0000","last_fullsized":"2024-05-09T05:34:34.049020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049070+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:34:34.049070+0000","last_peered":"2024-05-09T05:34:34.049070+0000","last_clean":"2024-05-09T05:34:34.049070+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:34:34.049070+0000","last_undegraded":"2024-05-09T05:34:34.049070+0000","last_fullsized":"2024-05-09T05:34:34.049070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:59.269932+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:34:59.269932+0000","last_peered":"2024-05-09T05:34:59.269932+0000","last_clean":"2024-05-09T05:34:59.269932+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:34:59.269932+0000","last_undegraded":"2024-05-09T05:34:59.269932+0000","last_fullsized":"2024-05-09T05:34:59.269932+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905116+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:34:33.905116+0000","last_peered":"2024-05-09T05:34:33.905116+0000","last_clean":"2024-05-09T05:34:33.905116+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:34:33.905116+0000","last_undegraded":"2024-05-09T05:34:33.905116+0000","last_fullsized":"2024-05-09T05:34:33.905116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940110+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:34:33.940110+0000","last_peered":"2024-05-09T05:34:33.940110+0000","last_clean":"2024-05-09T05:34:33.940110+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:34:33.940110+0000","last_undegraded":"2024-05-09T05:34:33.940110+0000","last_fullsized":"2024-05-09T05:34:33.940110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:26.337152+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:35:26.337152+0000","last_peered":"2024-05-09T05:35:26.337152+0000","last_clean":"2024-05-09T05:35:26.337152+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:35:26.337152+0000","last_undegraded":"2024-05-09T05:35:26.337152+0000","last_fullsized":"2024-05-09T05:35:26.337152+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905564+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:34:33.905564+0000","last_peered":"2024-05-09T05:34:33.905564+0000","last_clean":"2024-05-09T05:34:33.905564+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:34:33.905564+0000","last_undegraded":"2024-05-09T05:34:33.905564+0000","last_fullsized":"2024-05-09T05:34:33.905564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904141+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:34:33.904141+0000","last_peered":"2024-05-09T05:34:33.904141+0000","last_clean":"2024-05-09T05:34:33.904141+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:34:33.904141+0000","last_undegraded":"2024-05-09T05:34:33.904141+0000","last_fullsized":"2024-05-09T05:34:33.904141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296831+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:35:02.296831+0000","last_peered":"2024-05-09T05:35:02.296831+0000","last_clean":"2024-05-09T05:35:02.296831+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:35:02.296831+0000","last_undegraded":"2024-05-09T05:35:02.296831+0000","last_fullsized":"2024-05-09T05:35:02.296831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905074+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:34:33.905074+0000","last_peered":"2024-05-09T05:34:33.905074+0000","last_clean":"2024-05-09T05:34:33.905074+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:34:33.905074+0000","last_undegraded":"2024-05-09T05:34:33.905074+0000","last_fullsized":"2024-05-09T05:34:33.905074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.942331+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:34:33.942331+0000","last_peered":"2024-05-09T05:34:33.942331+0000","last_clean":"2024-05-09T05:34:33.942331+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:34:33.942331+0000","last_undegraded":"2024-05-09T05:34:33.942331+0000","last_fullsized":"2024-05-09T05:34:33.942331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296507+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:35:02.296507+0000","last_peered":"2024-05-09T05:35:02.296507+0000","last_clean":"2024-05-09T05:35:02.296507+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:35:02.296507+0000","last_undegraded":"2024-05-09T05:35:02.296507+0000","last_fullsized":"2024-05-09T05:35:02.296507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.905803+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:34:33.905803+0000","last_peered":"2024-05-09T05:34:33.905803+0000","last_clean":"2024-05-09T05:34:33.905803+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:34:33.905803+0000","last_undegraded":"2024-05-09T05:34:33.905803+0000","last_fullsized":"2024-05-09T05:34:33.905803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907549+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:34:33.907549+0000","last_peered":"2024-05-09T05:34:33.907549+0000","last_clean":"2024-05-09T05:34:33.907549+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:34:33.907549+0000","last_undegraded":"2024-05-09T05:34:33.907549+0000","last_fullsized":"2024-05-09T05:34:33.907549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939983+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:34:33.939983+0000","last_peered":"2024-05-09T05:34:33.939983+0000","last_clean":"2024-05-09T05:34:33.939983+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:34:33.939983+0000","last_undegraded":"2024-05-09T05:34:33.939983+0000","last_fullsized":"2024-05-09T05:34:33.939983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939482+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:34:33.939482+0000","last_peered":"2024-05-09T05:34:33.939482+0000","last_clean":"2024-05-09T05:34:33.939482+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:34:33.939482+0000","last_undegraded":"2024-05-09T05:34:33.939482+0000","last_fullsized":"2024-05-09T05:34:33.939482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904585+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:34:33.904585+0000","last_peered":"2024-05-09T05:34:33.904585+0000","last_clean":"2024-05-09T05:34:33.904585+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:34:33.904585+0000","last_undegraded":"2024-05-09T05:34:33.904585+0000","last_fullsized":"2024-05-09T05:34:33.904585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048073+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:34:34.048073+0000","last_peered":"2024-05-09T05:34:34.048073+0000","last_clean":"2024-05-09T05:34:34.048073+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:34:34.048073+0000","last_undegraded":"2024-05-09T05:34:34.048073+0000","last_fullsized":"2024-05-09T05:34:34.048073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.908251+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:34:33.908251+0000","last_peered":"2024-05-09T05:34:33.908251+0000","last_clean":"2024-05-09T05:34:33.908251+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:34:33.908251+0000","last_undegraded":"2024-05-09T05:34:33.908251+0000","last_fullsized":"2024-05-09T05:34:33.908251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940181+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:34:33.940181+0000","last_peered":"2024-05-09T05:34:33.940181+0000","last_clean":"2024-05-09T05:34:33.940181+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:34:33.940181+0000","last_undegraded":"2024-05-09T05:34:33.940181+0000","last_fullsized":"2024-05-09T05:34:33.940181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049464+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:34:34.049464+0000","last_peered":"2024-05-09T05:34:34.049464+0000","last_clean":"2024-05-09T05:34:34.049464+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:34:34.049464+0000","last_undegraded":"2024-05-09T05:34:34.049464+0000","last_fullsized":"2024-05-09T05:34:34.049464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.909834+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:34:33.909834+0000","last_peered":"2024-05-09T05:34:33.909834+0000","last_clean":"2024-05-09T05:34:33.909834+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:34:33.909834+0000","last_undegraded":"2024-05-09T05:34:33.909834+0000","last_fullsized":"2024-05-09T05:34:33.909834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048743+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:34:34.048743+0000","last_peered":"2024-05-09T05:34:34.048743+0000","last_clean":"2024-05-09T05:34:34.048743+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:34:34.048743+0000","last_undegraded":"2024-05-09T05:34:34.048743+0000","last_fullsized":"2024-05-09T05:34:34.048743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940296+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:34:33.940296+0000","last_peered":"2024-05-09T05:34:33.940296+0000","last_clean":"2024-05-09T05:34:33.940296+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:34:33.940296+0000","last_undegraded":"2024-05-09T05:34:33.940296+0000","last_fullsized":"2024-05-09T05:34:33.940296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.047967+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:34:34.047967+0000","last_peered":"2024-05-09T05:34:34.047967+0000","last_clean":"2024-05-09T05:34:34.047967+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:34:34.047967+0000","last_undegraded":"2024-05-09T05:34:34.047967+0000","last_fullsized":"2024-05-09T05:34:34.047967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.908055+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:34:33.908055+0000","last_peered":"2024-05-09T05:34:33.908055+0000","last_clean":"2024-05-09T05:34:33.908055+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:34:33.908055+0000","last_undegraded":"2024-05-09T05:34:33.908055+0000","last_fullsized":"2024-05-09T05:34:33.908055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213909,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70949,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":303,"num_read_kb":232,"num_write":250,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":200341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":496946,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906984,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":1147786,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070509,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":500860,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":497030,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1093639,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":492268,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":551181,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":493724,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":551792,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":491976,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":496342,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1089221,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:35:33.930 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T05:35:33.931 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T05:35:33.931 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T05:35:34.174 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:34.585 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:34 smithi115 ceph-mon[30349]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-09T05:35:34.585 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:34 smithi115 ceph-mon[30349]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:35:34.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:34 smithi195 ceph-mon[33844]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-09T05:35:34.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:34 smithi195 ceph-mon[33844]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:35:34.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:34 smithi161 ceph-mon[33805]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-09T05:35:34.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:34 smithi161 ceph-mon[33805]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:35:36.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.038+0000 7f3ecda47700 1 -- 172.21.15.115:0/3081050165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104d20 msgr2=0x7f3ec810b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:36.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.038+0000 7f3ecda47700 1 --2- 172.21.15.115:0/3081050165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104d20 0x7f3ec810b5d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b0a0 tx=0x7f3ec000b3b0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.038+0000 7f3ecda47700 1 -- 172.21.15.115:0/3081050165 shutdown_connections 2024-05-09T05:35:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.038+0000 7f3ecda47700 1 --2- 172.21.15.115:0/3081050165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8103250 0x7f3ec8103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.038+0000 7f3ecda47700 1 --2- 172.21.15.115:0/3081050165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104d20 0x7f3ec810b5d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.038+0000 7f3ecda47700 1 --2- 172.21.15.115:0/3081050165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8104380 0x7f3ec8104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.038+0000 7f3ecda47700 1 -- 172.21.15.115:0/3081050165 >> 172.21.15.115:0/3081050165 conn(0x7f3ec80fe890 msgr2=0x7f3ec8100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.038+0000 7f3ecda47700 1 -- 172.21.15.115:0/3081050165 shutdown_connections 2024-05-09T05:35:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.038+0000 7f3ecda47700 1 -- 172.21.15.115:0/3081050165 wait complete. 2024-05-09T05:35:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ecda47700 1 Processor -- start 2024-05-09T05:35:36.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ecda47700 1 -- start start 2024-05-09T05:35:36.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ecda47700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8103250 0x7f3ec8076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:36.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ecda47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104380 0x7f3ec819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:36.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ecda47700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8104d20 0x7f3ec81a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:36.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ecda47700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810ec60 con 0x7f3ec8104380 2024-05-09T05:35:36.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ecda47700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810eae0 con 0x7f3ec8103250 2024-05-09T05:35:36.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ecda47700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810ede0 con 0x7f3ec8104d20 2024-05-09T05:35:36.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ebffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104380 0x7f3ec819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:36.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ebffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104380 0x7f3ec819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36896/0 (socket says 172.21.15.115:36896) 2024-05-09T05:35:36.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.039+0000 7f3ebffff700 1 -- 172.21.15.115:0/294326460 learned_addr learned my addr 172.21.15.115:0/294326460 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:36.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ec77fe700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8104d20 0x7f3ec81a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:36.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ec6ffd700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8103250 0x7f3ec8076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ebffff700 1 -- 172.21.15.115:0/294326460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8104d20 msgr2=0x7f3ec81a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ebffff700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8104d20 0x7f3ec81a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ebffff700 1 -- 172.21.15.115:0/294326460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8103250 msgr2=0x7f3ec8076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ebffff700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8103250 0x7f3ec8076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ebffff700 1 -- 172.21.15.115:0/294326460 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec81a24a0 con 0x7f3ec8104380 2024-05-09T05:35:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ec77fe700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8104d20 0x7f3ec81a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:35:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ebffff700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104380 0x7f3ec819f700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000b530 tx=0x7f3eb000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:36.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ecca45700 1 -- 172.21.15.115:0/294326460 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0015020 con 0x7f3ec8104380 2024-05-09T05:35:36.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec81a2790 con 0x7f3ec8104380 2024-05-09T05:35:36.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ecca45700 1 -- 172.21.15.115:0/294326460 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3eb0011cc0 con 0x7f3ec8104380 2024-05-09T05:35:36.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ecca45700 1 -- 172.21.15.115:0/294326460 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0010d30 con 0x7f3ec8104380 2024-05-09T05:35:36.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.040+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec81a2d50 con 0x7f3ec8104380 2024-05-09T05:35:36.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.041+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec80512c0 con 0x7f3ec8104380 2024-05-09T05:35:36.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.043+0000 7f3ecca45700 1 -- 172.21.15.115:0/294326460 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f3eb0018020 con 0x7f3ec8104380 2024-05-09T05:35:36.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.044+0000 7f3ecca45700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3eb40b2910 0x7f3eb40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:36.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.044+0000 7f3ec6ffd700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3eb40b2910 0x7f3eb40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:36.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.044+0000 7f3ecca45700 1 -- 172.21.15.115:0/294326460 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f3eb00e1e80 con 0x7f3ec8104380 2024-05-09T05:35:36.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.044+0000 7f3ec6ffd700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3eb40b2910 0x7f3eb40b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006fd0 tx=0x7f3eb8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:36.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.049+0000 7f3ecca45700 1 -- 172.21.15.115:0/294326460 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3eb001f020 con 0x7f3ec8104380 2024-05-09T05:35:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.339+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3ec80643d0 con 0x7f3ec8104380 2024-05-09T05:35:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.342+0000 7f3ecca45700 1 -- 172.21.15.115:0/294326460 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f3eb00a97f0 con 0x7f3ec8104380 2024-05-09T05:35:36.336 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-05-09T05:35:36.336 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCoYDxmf9RSFBAA6ZBUz5tfznjXY8yPi99qCg== 2024-05-09T05:35:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.344+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3eb40b2910 msgr2=0x7f3eb40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.344+0000 7f3ecda47700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3eb40b2910 0x7f3eb40b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006fd0 tx=0x7f3eb8008040 comp rx=0 tx=0).stop 2024-05-09T05:35:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.344+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104380 msgr2=0x7f3ec819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.344+0000 7f3ecda47700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104380 0x7f3ec819f700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000b530 tx=0x7f3eb000b8f0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.345+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 shutdown_connections 2024-05-09T05:35:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.345+0000 7f3ecda47700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3eb40b2910 0x7f3eb40b4e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.345+0000 7f3ecda47700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8103250 0x7f3ec8076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.345+0000 7f3ecda47700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8104d20 0x7f3ec81a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.345+0000 7f3ecda47700 1 --2- 172.21.15.115:0/294326460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec8104380 0x7f3ec819f700 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.345+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 >> 172.21.15.115:0/294326460 conn(0x7f3ec80fe890 msgr2=0x7f3ec81075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.345+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 shutdown_connections 2024-05-09T05:35:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:36.345+0000 7f3ecda47700 1 -- 172.21.15.115:0/294326460 wait complete. 2024-05-09T05:35:36.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:36 smithi195 ceph-mon[33844]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-05-09T05:35:36.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:36 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/294326460' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T05:35:36.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:36 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/294326460' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T05:35:36.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:36 smithi115 ceph-mon[30349]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-05-09T05:35:36.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:36 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/294326460' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T05:35:36.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:36 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/294326460' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T05:35:36.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:36 smithi161 ceph-mon[33805]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-05-09T05:35:36.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:36 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/294326460' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T05:35:36.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:36 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/294326460' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T05:35:37.055 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:35:37.055 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T05:35:37.055 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T05:35:37.093 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T05:35:37.093 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T05:35:37.094 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T05:35:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:38.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:38 smithi195 ceph-mon[33844]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:35:38.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:38 smithi115 ceph-mon[30349]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:35:38.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:38 smithi161 ceph-mon[33805]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:35:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.353+0000 7f2c596c7700 1 -- 172.21.15.115:0/87067826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 msgr2=0x7f2c541100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.353+0000 7f2c596c7700 1 --2- 172.21.15.115:0/87067826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 0x7f2c541100a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c440099c0 tx=0x7f2c44009cd0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.353+0000 7f2c596c7700 1 -- 172.21.15.115:0/87067826 shutdown_connections 2024-05-09T05:35:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.353+0000 7f2c596c7700 1 --2- 172.21.15.115:0/87067826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 0x7f2c541100a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.353+0000 7f2c596c7700 1 --2- 172.21.15.115:0/87067826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c541034c0 0x7f2c541038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.353+0000 7f2c596c7700 1 --2- 172.21.15.115:0/87067826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54107df0 0x7f2c54108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.353+0000 7f2c596c7700 1 -- 172.21.15.115:0/87067826 >> 172.21.15.115:0/87067826 conn(0x7f2c540fedf0 msgr2=0x7f2c54101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.353+0000 7f2c596c7700 1 -- 172.21.15.115:0/87067826 shutdown_connections 2024-05-09T05:35:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.353+0000 7f2c596c7700 1 -- 172.21.15.115:0/87067826 wait complete. 2024-05-09T05:35:39.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.354+0000 7f2c596c7700 1 Processor -- start 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c596c7700 1 -- start start 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c596c7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c541034c0 0x7f2c5419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c596c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54107df0 0x7f2c5419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c596c7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 0x7f2c5419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c596c7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54113290 con 0x7f2c54108740 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c596c7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54113110 con 0x7f2c54107df0 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c596c7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54113410 con 0x7f2c541034c0 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c537fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 0x7f2c5419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c537fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 0x7f2c5419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36912/0 (socket says 172.21.15.115:36912) 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c537fe700 1 -- 172.21.15.115:0/4165097120 learned_addr learned my addr 172.21.15.115:0/4165097120 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c52ffd700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c541034c0 0x7f2c5419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c527fc700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54107df0 0x7f2c5419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:39.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c537fe700 1 -- 172.21.15.115:0/4165097120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c541034c0 msgr2=0x7f2c5419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c537fe700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c541034c0 0x7f2c5419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c537fe700 1 -- 172.21.15.115:0/4165097120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54107df0 msgr2=0x7f2c5419c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c537fe700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54107df0 0x7f2c5419c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.355+0000 7f2c537fe700 1 -- 172.21.15.115:0/4165097120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c541a6e70 con 0x7f2c54108740 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.356+0000 7f2c537fe700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 0x7f2c5419c650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b580 tx=0x7f2c4400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.356+0000 7f2c4bfff700 1 -- 172.21.15.115:0/4165097120 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44026070 con 0x7f2c54108740 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.356+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c541a7100 con 0x7f2c54108740 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.356+0000 7f2c4bfff700 1 -- 172.21.15.115:0/4165097120 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c44005030 con 0x7f2c54108740 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.356+0000 7f2c4bfff700 1 -- 172.21.15.115:0/4165097120 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44004a30 con 0x7f2c54108740 2024-05-09T05:35:39.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.356+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c541a7560 con 0x7f2c54108740 2024-05-09T05:35:39.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.357+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c540512c0 con 0x7f2c54108740 2024-05-09T05:35:39.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.358+0000 7f2c4bfff700 1 -- 172.21.15.115:0/4165097120 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2c44029070 con 0x7f2c54108740 2024-05-09T05:35:39.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.359+0000 7f2c4bfff700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c400b2910 0x7f2c400b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:39.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.359+0000 7f2c52ffd700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c400b2910 0x7f2c400b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:39.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.359+0000 7f2c4bfff700 1 -- 172.21.15.115:0/4165097120 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2c440e3660 con 0x7f2c54108740 2024-05-09T05:35:39.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.360+0000 7f2c52ffd700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c400b2910 0x7f2c400b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00f440 tx=0x7f2c4c0079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.363+0000 7f2c4bfff700 1 -- 172.21.15.115:0/4165097120 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c44030080 con 0x7f2c54108740 2024-05-09T05:35:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.625+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2c5402e7b0 con 0x7f2c54108740 2024-05-09T05:35:39.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.630+0000 7f2c4bfff700 1 -- 172.21.15.115:0/4165097120 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288654 (secure 0 0 0) 0x7f2c44013070 con 0x7f2c54108740 2024-05-09T05:35:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:35:39.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.633+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c400b2910 msgr2=0x7f2c400b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:39.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.633+0000 7f2c596c7700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c400b2910 0x7f2c400b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00f440 tx=0x7f2c4c0079e0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.633+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 msgr2=0x7f2c5419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:39.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.633+0000 7f2c596c7700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 0x7f2c5419c650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b580 tx=0x7f2c4400f690 comp rx=0 tx=0).stop 2024-05-09T05:35:39.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.634+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 shutdown_connections 2024-05-09T05:35:39.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.634+0000 7f2c596c7700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2c400b2910 0x7f2c400b4e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.634+0000 7f2c596c7700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c54108740 0x7f2c5419c650 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.634+0000 7f2c596c7700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c541034c0 0x7f2c5419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.634+0000 7f2c596c7700 1 --2- 172.21.15.115:0/4165097120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c54107df0 0x7f2c5419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:39.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.634+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 >> 172.21.15.115:0/4165097120 conn(0x7f2c540fedf0 msgr2=0x7f2c541096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:39.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.634+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 shutdown_connections 2024-05-09T05:35:39.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:39.634+0000 7f2c596c7700 1 -- 172.21.15.115:0/4165097120 wait complete. 2024-05-09T05:35:40.466 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2041699318},{"type":"v1","addr":"172.21.15.115:6801","nonce":2041699318}]},"active_addr":"172.21.15.115:6801/2041699318","active_change":"2024-05-09T05:26:32.548286+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.115:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":1352570099}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":1853670702}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":4237548909}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":617068092}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":339248430}]}]} 2024-05-09T05:35:40.472 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T05:35:40.472 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T05:35:40.472 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:35:40.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:40 smithi195 ceph-mon[33844]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T05:35:40.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:40 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4165097120' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T05:35:40.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:40 smithi115 ceph-mon[30349]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T05:35:40.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:40 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4165097120' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T05:35:40.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:40 smithi161 ceph-mon[33805]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-09T05:35:40.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:40 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4165097120' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T05:35:40.714 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.466+0000 7fc2084e6700 1 -- 172.21.15.115:0/425607751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 msgr2=0x7fc2001100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.466+0000 7fc2084e6700 1 --2- 172.21.15.115:0/425607751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 0x7fc2001100a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800b0d0 tx=0x7fc1f800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.466+0000 7fc2084e6700 1 -- 172.21.15.115:0/425607751 shutdown_connections 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.466+0000 7fc2084e6700 1 --2- 172.21.15.115:0/425607751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 0x7fc2001100a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.466+0000 7fc2084e6700 1 --2- 172.21.15.115:0/425607751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2001034c0 0x7fc2001038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.466+0000 7fc2084e6700 1 --2- 172.21.15.115:0/425607751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200107df0 0x7fc200108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.466+0000 7fc2084e6700 1 -- 172.21.15.115:0/425607751 >> 172.21.15.115:0/425607751 conn(0x7fc2000fedc0 msgr2=0x7fc2001011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.466+0000 7fc2084e6700 1 -- 172.21.15.115:0/425607751 shutdown_connections 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.466+0000 7fc2084e6700 1 -- 172.21.15.115:0/425607751 wait complete. 2024-05-09T05:35:42.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc2084e6700 1 Processor -- start 2024-05-09T05:35:42.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc2084e6700 1 -- start start 2024-05-09T05:35:42.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc2084e6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2001034c0 0x7fc20019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:42.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc2084e6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200107df0 0x7fc20019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc2084e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 0x7fc2001a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc2084e6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc200113220 con 0x7fc200108740 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc2084e6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc2001130a0 con 0x7fc200107df0 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc2084e6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2001133a0 con 0x7fc2001034c0 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc206a83700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 0x7fc2001a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc206a83700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 0x7fc2001a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36930/0 (socket says 172.21.15.115:36930) 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc206a83700 1 -- 172.21.15.115:0/2271265077 learned_addr learned my addr 172.21.15.115:0/2271265077 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc206282700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2001034c0 0x7fc20019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc205a81700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200107df0 0x7fc20019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc206a83700 1 -- 172.21.15.115:0/2271265077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2001034c0 msgr2=0x7fc20019bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc206a83700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2001034c0 0x7fc20019bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc206a83700 1 -- 172.21.15.115:0/2271265077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200107df0 msgr2=0x7fc20019c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc206a83700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200107df0 0x7fc20019c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.467+0000 7fc206a83700 1 -- 172.21.15.115:0/2271265077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20019c8d0 con 0x7fc200108740 2024-05-09T05:35:42.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.468+0000 7fc206a83700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 0x7fc2001a6b60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009850 tx=0x7fc1f8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:42.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.468+0000 7fc1f77fe700 1 -- 172.21.15.115:0/2271265077 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f801f070 con 0x7fc200108740 2024-05-09T05:35:42.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.468+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2001a7160 con 0x7fc200108740 2024-05-09T05:35:42.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.468+0000 7fc1f77fe700 1 -- 172.21.15.115:0/2271265077 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1f8005500 con 0x7fc200108740 2024-05-09T05:35:42.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.468+0000 7fc1f77fe700 1 -- 172.21.15.115:0/2271265077 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f801b9a0 con 0x7fc200108740 2024-05-09T05:35:42.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.468+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2001a75b0 con 0x7fc200108740 2024-05-09T05:35:42.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.469+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2000512c0 con 0x7fc200108740 2024-05-09T05:35:42.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.471+0000 7fc1f77fe700 1 -- 172.21.15.115:0/2271265077 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc1f8022070 con 0x7fc200108740 2024-05-09T05:35:42.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.471+0000 7fc1f77fe700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc1ec0b2910 0x7fc1ec0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:42.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.471+0000 7fc206282700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc1ec0b2910 0x7fc1ec0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:42.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.471+0000 7fc1f77fe700 1 -- 172.21.15.115:0/2271265077 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fc1f800f020 con 0x7fc200108740 2024-05-09T05:35:42.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.472+0000 7fc206282700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc1ec0b2910 0x7fc1ec0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc005950 tx=0x7fc1fc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:42.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.475+0000 7fc1f77fe700 1 -- 172.21.15.115:0/2271265077 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1f8029020 con 0x7fc200108740 2024-05-09T05:35:42.468 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:42 smithi115 ceph-mon[30349]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T05:35:42.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:42 smithi195 ceph-mon[33844]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T05:35:42.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:42 smithi161 ceph-mon[33805]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-09T05:35:42.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.682+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc20002e5f0 con 0x7fc200108740 2024-05-09T05:35:42.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.683+0000 7fc1f77fe700 1 -- 172.21.15.115:0/2271265077 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24724 (secure 0 0 0) 0x7fc1f80ab8a0 con 0x7fc200108740 2024-05-09T05:35:42.677 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:35:42.677 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:33.901006+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:35:42.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc1ec0b2910 msgr2=0x7fc1ec0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:42.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc1ec0b2910 0x7fc1ec0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc005950 tx=0x7fc1fc00b410 comp rx=0 tx=0).stop 2024-05-09T05:35:42.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 msgr2=0x7fc2001a6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:42.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 0x7fc2001a6b60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009850 tx=0x7fc1f8007d90 comp rx=0 tx=0).stop 2024-05-09T05:35:42.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 shutdown_connections 2024-05-09T05:35:42.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc1ec0b2910 0x7fc1ec0b4e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc200108740 0x7fc2001a6b60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2001034c0 0x7fc20019bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 --2- 172.21.15.115:0/2271265077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200107df0 0x7fc20019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:42.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 >> 172.21.15.115:0/2271265077 conn(0x7fc2000fedc0 msgr2=0x7fc2001075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:42.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 shutdown_connections 2024-05-09T05:35:42.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:42.686+0000 7fc2084e6700 1 -- 172.21.15.115:0/2271265077 wait complete. 2024-05-09T05:35:43.403 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T05:35:43.403 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:35:43.644 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:43.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:43 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2271265077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:35:43.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:43 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2271265077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:35:43.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:43 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2271265077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:35:44.372 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:44 smithi115 ceph-mon[30349]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:44.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:44 smithi161 ceph-mon[33805]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:44.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:44 smithi195 ceph-mon[33844]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:45.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 -- 172.21.15.115:0/684875416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b0c105e60 msgr2=0x7f9b0c10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 --2- 172.21.15.115:0/684875416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b0c105e60 0x7f9b0c10e320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009990 tx=0x7f9b00009ca0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 -- 172.21.15.115:0/684875416 shutdown_connections 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 --2- 172.21.15.115:0/684875416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b0c105480 0x7f9b0c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 --2- 172.21.15.115:0/684875416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0c10e940 0x7f9b0c110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 --2- 172.21.15.115:0/684875416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b0c105e60 0x7f9b0c10e320 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 -- 172.21.15.115:0/684875416 >> 172.21.15.115:0/684875416 conn(0x7f9b0c0fef50 msgr2=0x7f9b0c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 -- 172.21.15.115:0/684875416 shutdown_connections 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 -- 172.21.15.115:0/684875416 wait complete. 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.394+0000 7f9b116f8700 1 Processor -- start 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b116f8700 1 -- start start 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b116f8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b0c105480 0x7f9b0c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b116f8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b0c105e60 0x7f9b0c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:45.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b116f8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0c10e940 0x7f9b0c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b116f8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c114330 con 0x7f9b0c105e60 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b116f8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1141b0 con 0x7f9b0c105480 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b116f8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1144b0 con 0x7f9b0c10e940 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b0a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b0c105e60 0x7f9b0c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b0affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b0c105480 0x7f9b0c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b0affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b0c105480 0x7f9b0c19ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:57896/0 (socket says 172.21.15.115:57896) 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b0affd700 1 -- 172.21.15.115:0/1748232500 learned_addr learned my addr 172.21.15.115:0/1748232500 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b0b7fe700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0c10e940 0x7f9b0c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b0a7fc700 1 -- 172.21.15.115:0/1748232500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0c10e940 msgr2=0x7f9b0c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b0a7fc700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0c10e940 0x7f9b0c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b0a7fc700 1 -- 172.21.15.115:0/1748232500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b0c105480 msgr2=0x7f9b0c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.395+0000 7f9b0a7fc700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b0c105480 0x7f9b0c19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.396+0000 7f9b0a7fc700 1 -- 172.21.15.115:0/1748232500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c19fbf0 con 0x7f9b0c105e60 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.396+0000 7f9b0affd700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b0c105480 0x7f9b0c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:35:45.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.396+0000 7f9b0a7fc700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b0c105e60 0x7f9b0c19b2c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9b000098e0 tx=0x7f9b000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:45.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.396+0000 7f9af3fff700 1 -- 172.21.15.115:0/1748232500 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00004830 con 0x7f9b0c105e60 2024-05-09T05:35:45.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.396+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c19fe80 con 0x7f9b0c105e60 2024-05-09T05:35:45.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.396+0000 7f9af3fff700 1 -- 172.21.15.115:0/1748232500 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b00005020 con 0x7f9b0c105e60 2024-05-09T05:35:45.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.396+0000 7f9af3fff700 1 -- 172.21.15.115:0/1748232500 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000f8d0 con 0x7f9b0c105e60 2024-05-09T05:35:45.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.396+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c1a02d0 con 0x7f9b0c105e60 2024-05-09T05:35:45.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.397+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af8005360 con 0x7f9b0c105e60 2024-05-09T05:35:45.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.399+0000 7f9af3fff700 1 -- 172.21.15.115:0/1748232500 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f9b00029060 con 0x7f9b0c105e60 2024-05-09T05:35:45.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.400+0000 7f9af3fff700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9af40b2910 0x7f9af40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:45.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.400+0000 7f9b0affd700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9af40b2910 0x7f9af40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:45.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.400+0000 7f9af3fff700 1 -- 172.21.15.115:0/1748232500 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9b000e3390 con 0x7f9b0c105e60 2024-05-09T05:35:45.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.400+0000 7f9b0affd700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9af40b2910 0x7f9af40b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9afc005950 tx=0x7f9afc00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:45.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.403+0000 7f9af3fff700 1 -- 172.21.15.115:0/1748232500 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b00013070 con 0x7f9b0c105e60 2024-05-09T05:35:45.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.607+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9af8005920 con 0x7f9b0c105e60 2024-05-09T05:35:45.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.608+0000 7f9af3fff700 1 -- 172.21.15.115:0/1748232500 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24724 (secure 0 0 0) 0x7f9b000aac80 con 0x7f9b0c105e60 2024-05-09T05:35:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:35:45.602 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:33.901006+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:35:45.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9af40b2910 msgr2=0x7f9af40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9af40b2910 0x7f9af40b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9afc005950 tx=0x7f9afc00b500 comp rx=0 tx=0).stop 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b0c105e60 msgr2=0x7f9b0c19b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b0c105e60 0x7f9b0c19b2c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9b000098e0 tx=0x7f9b000047b0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 shutdown_connections 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9af40b2910 0x7f9af40b4e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b0c105480 0x7f9b0c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0c10e940 0x7f9b0c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 --2- 172.21.15.115:0/1748232500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b0c105e60 0x7f9b0c19b2c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 >> 172.21.15.115:0/1748232500 conn(0x7f9b0c0fef50 msgr2=0x7f9b0c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 shutdown_connections 2024-05-09T05:35:45.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:45.611+0000 7f9b116f8700 1 -- 172.21.15.115:0/1748232500 wait complete. 2024-05-09T05:35:46.331 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T05:35:46.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T05:35:46.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T05:35:46.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T05:35:46.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T05:35:46.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T05:35:46.334 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T05:35:46.334 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T05:35:46.334 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T05:35:46.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T05:35:46.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T05:35:46.336 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T05:35:46.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:46 smithi115 ceph-mon[30349]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:35:46.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:46 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1748232500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:35:46.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:46 smithi161 ceph-mon[33805]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:35:46.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:46 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1748232500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:35:46.668 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:46 smithi195 ceph-mon[33844]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:35:46.668 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:46 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1748232500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:35:47.035 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:47.039 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:47.039 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:47.044 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:48.497 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:48 smithi115 ceph-mon[30349]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:48 smithi161 ceph-mon[33805]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:48.736 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:48.737 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:48.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:48 smithi195 ceph-mon[33844]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:50.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:50 smithi115 ceph-mon[30349]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:35:50.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:50 smithi161 ceph-mon[33805]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:35:50.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:50 smithi195 ceph-mon[33844]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:35:51.370 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:52.577 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:52 smithi115 ceph-mon[30349]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:35:52.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:52 smithi161 ceph-mon[33805]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:35:52.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:52 smithi195 ceph-mon[33844]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:35:53.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:53 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:35:53.647 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:53 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:35:53.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:53 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:35:53.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 -- 172.21.15.115:0/1296966649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4804108780 msgr2=0x7f48041100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:53.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 --2- 172.21.15.115:0/1296966649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4804108780 0x7f48041100e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00b0d0 tx=0x7f47fc00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:35:53.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 -- 172.21.15.115:0/1296966649 shutdown_connections 2024-05-09T05:35:53.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 --2- 172.21.15.115:0/1296966649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4804103550 0x7f4804103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:53.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 --2- 172.21.15.115:0/1296966649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4804108780 0x7f48041100e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:53.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 --2- 172.21.15.115:0/1296966649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4804107e30 0x7f4804108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:53.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 -- 172.21.15.115:0/1296966649 >> 172.21.15.115:0/1296966649 conn(0x7f4804078e00 msgr2=0x7f4804079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:53.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 -- 172.21.15.115:0/1296966649 shutdown_connections 2024-05-09T05:35:53.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 -- 172.21.15.115:0/1296966649 wait complete. 2024-05-09T05:35:53.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.936+0000 7f480a39f700 1 Processor -- start 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f480a39f700 1 -- start start 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f480a39f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4804103550 0x7f480419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f480a39f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4804107e30 0x7f480419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f480a39f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4804108780 0x7f480419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f480a39f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4804113260 con 0x7f4804103550 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f480a39f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f48041130e0 con 0x7f4804108780 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f480a39f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48041133e0 con 0x7f4804107e30 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f4803fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4804103550 0x7f480419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f4803fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4804103550 0x7f480419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44090/0 (socket says 172.21.15.115:44090) 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f4803fff700 1 -- 172.21.15.115:0/344081938 learned_addr learned my addr 172.21.15.115:0/344081938 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f480893c700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4804108780 0x7f480419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.937+0000 7f48037fe700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4804107e30 0x7f480419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f4803fff700 1 -- 172.21.15.115:0/344081938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4804107e30 msgr2=0x7f480419c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f4803fff700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4804107e30 0x7f480419c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f4803fff700 1 -- 172.21.15.115:0/344081938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4804108780 msgr2=0x7f480419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f4803fff700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4804108780 0x7f480419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f4803fff700 1 -- 172.21.15.115:0/344081938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48041a6f30 con 0x7f4804103550 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f480893c700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4804108780 0x7f480419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f4803fff700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4804103550 0x7f480419bbe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00e9a0 tx=0x7f47ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f48017fa700 1 -- 172.21.15.115:0/344081938 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00ca10 con 0x7f4804103550 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48041a7220 con 0x7f4804103550 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f48017fa700 1 -- 172.21.15.115:0/344081938 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47ec00cbb0 con 0x7f4804103550 2024-05-09T05:35:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.938+0000 7f48017fa700 1 -- 172.21.15.115:0/344081938 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec018680 con 0x7f4804103550 2024-05-09T05:35:53.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.939+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48041a77b0 con 0x7f4804103550 2024-05-09T05:35:53.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.940+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4804063ed0 con 0x7f4804103550 2024-05-09T05:35:53.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.941+0000 7f48017fa700 1 -- 172.21.15.115:0/344081938 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f47ec010430 con 0x7f4804103550 2024-05-09T05:35:53.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.942+0000 7f48017fa700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47f00b2990 0x7f47f00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:53.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.943+0000 7f48037fe700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47f00b2990 0x7f47f00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:53.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.943+0000 7f48017fa700 1 -- 172.21.15.115:0/344081938 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f47ec0e28a0 con 0x7f4804103550 2024-05-09T05:35:53.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.943+0000 7f48037fe700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47f00b2990 0x7f47f00b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4804079350 tx=0x7f47f40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:53.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.943+0000 7f48017fa700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] conn(0x7f47f00bdb40 0x7f47f00c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:53.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.943+0000 7f48017fa700 1 -- 172.21.15.115:0/344081938 --> [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47f00c0700 con 0x7f47f00bdb40 2024-05-09T05:35:53.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.943+0000 7f48017fa700 1 -- 172.21.15.115:0/344081938 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f47ec0e72c0 con 0x7f4804103550 2024-05-09T05:35:53.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.943+0000 7f480893c700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] conn(0x7f47f00bdb40 0x7f47f00c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:53.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.944+0000 7f480893c700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] conn(0x7f47f00bdb40 0x7f47f00c0030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:53.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.945+0000 7f48017fa700 1 -- 172.21.15.115:0/344081938 <== osd.3 v2:172.21.15.115:6810/1326312717 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47f00c0700 con 0x7f47f00bdb40 2024-05-09T05:35:53.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.961+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 --> [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f48040642d0 con 0x7f47f00bdb40 2024-05-09T05:35:53.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f48017fa700 1 -- 172.21.15.115:0/344081938 <== osd.3 v2:172.21.15.115:6810/1326312717 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f48040642d0 con 0x7f47f00bdb40 2024-05-09T05:35:53.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] conn(0x7f47f00bdb40 msgr2=0x7f47f00c0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] conn(0x7f47f00bdb40 0x7f47f00c0030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47f00b2990 msgr2=0x7f47f00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47f00b2990 0x7f47f00b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4804079350 tx=0x7f47f40073d0 comp rx=0 tx=0).stop 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4804103550 msgr2=0x7f480419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 --2- 172.21.15.115:0/344081938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4804103550 0x7f480419bbe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00e9a0 tx=0x7f47ec00ecb0 comp rx=0 tx=0).stop 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480893c700 1 -- 172.21.15.115:0/344081938 reap_dead start 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 shutdown_connections 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 >> 172.21.15.115:0/344081938 conn(0x7f4804078e00 msgr2=0x7f4804100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 shutdown_connections 2024-05-09T05:35:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:53.962+0000 7f480a39f700 1 -- 172.21.15.115:0/344081938 wait complete. 2024-05-09T05:35:54.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:54 smithi115 ceph-mon[30349]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:54 smithi161 ceph-mon[33805]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:54.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:54 smithi195 ceph-mon[33844]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:55.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.637 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:55 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:55 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:55.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:55 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:35:56.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.250+0000 7f8b34d45700 1 -- 172.21.15.115:0/1007009600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b30105570 msgr2=0x7f8b30105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:56.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.250+0000 7f8b34d45700 1 --2- 172.21.15.115:0/1007009600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b30105570 0x7f8b30105980 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009990 tx=0x7f8b24009ca0 comp rx=0 tx=0).stop 2024-05-09T05:35:56.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.251+0000 7f8b34d45700 1 -- 172.21.15.115:0/1007009600 shutdown_connections 2024-05-09T05:35:56.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.251+0000 7f8b34d45700 1 --2- 172.21.15.115:0/1007009600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3010e950 0x7f8b30110e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:56.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.251+0000 7f8b34d45700 1 --2- 172.21.15.115:0/1007009600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b30105570 0x7f8b30105980 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.251+0000 7f8b34d45700 1 --2- 172.21.15.115:0/1007009600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b30105ec0 0x7f8b3010e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.251+0000 7f8b34d45700 1 -- 172.21.15.115:0/1007009600 >> 172.21.15.115:0/1007009600 conn(0x7f8b30078e00 msgr2=0x7f8b30079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.251+0000 7f8b34d45700 1 -- 172.21.15.115:0/1007009600 shutdown_connections 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.251+0000 7f8b34d45700 1 -- 172.21.15.115:0/1007009600 wait complete. 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b34d45700 1 Processor -- start 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b34d45700 1 -- start start 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b34d45700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b30105570 0x7f8b3019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b34d45700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b30105ec0 0x7f8b3019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b34d45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3010e950 0x7f8b3019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b34d45700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30114320 con 0x7f8b3010e950 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b34d45700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301141a0 con 0x7f8b30105ec0 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b34d45700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b301144a0 con 0x7f8b30105570 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3010e950 0x7f8b3019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:56.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3010e950 0x7f8b3019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44124/0 (socket says 172.21.15.115:44124) 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2ed9d700 1 -- 172.21.15.115:0/669716815 learned_addr learned my addr 172.21.15.115:0/669716815 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2dd9b700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b30105ec0 0x7f8b3019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2e59c700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b30105570 0x7f8b3019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2ed9d700 1 -- 172.21.15.115:0/669716815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b30105570 msgr2=0x7f8b3019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2ed9d700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b30105570 0x7f8b3019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2ed9d700 1 -- 172.21.15.115:0/669716815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b30105ec0 msgr2=0x7f8b3019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2ed9d700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b30105ec0 0x7f8b3019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.252+0000 7f8b2ed9d700 1 -- 172.21.15.115:0/669716815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3019fc40 con 0x7f8b3010e950 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.253+0000 7f8b2ed9d700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3010e950 0x7f8b3019f5a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800ef30 tx=0x7f8b2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.253+0000 7f8b1f7fe700 1 -- 172.21.15.115:0/669716815 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b280092e0 con 0x7f8b3010e950 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.253+0000 7f8b34d45700 1 -- 172.21.15.115:0/669716815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3019fed0 con 0x7f8b3010e950 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.253+0000 7f8b1f7fe700 1 -- 172.21.15.115:0/669716815 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b28004bb0 con 0x7f8b3010e950 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.253+0000 7f8b1f7fe700 1 -- 172.21.15.115:0/669716815 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28004020 con 0x7f8b3010e950 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.253+0000 7f8b34d45700 1 -- 172.21.15.115:0/669716815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b301a7850 con 0x7f8b3010e950 2024-05-09T05:35:56.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.254+0000 7f8b1d7fa700 1 -- 172.21.15.115:0/669716815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b30063e90 con 0x7f8b3010e950 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.255+0000 7f8b1f7fe700 1 -- 172.21.15.115:0/669716815 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8b2801b030 con 0x7f8b3010e950 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.256+0000 7f8b1f7fe700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8b180b2910 0x7f8b180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.256+0000 7f8b2e59c700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8b180b2910 0x7f8b180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.256+0000 7f8b1f7fe700 1 -- 172.21.15.115:0/669716815 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f8b280e2ee0 con 0x7f8b3010e950 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.256+0000 7f8b1f7fe700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] conn(0x7f8b180bdac0 0x7f8b180bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.256+0000 7f8b1f7fe700 1 -- 172.21.15.115:0/669716815 --> [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b28008ec0 con 0x7f8b180bdac0 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.256+0000 7f8b1f7fe700 1 -- 172.21.15.115:0/669716815 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f8b280e3330 con 0x7f8b3010e950 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.257+0000 7f8b2dd9b700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] conn(0x7f8b180bdac0 0x7f8b180bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.257+0000 7f8b2e59c700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8b180b2910 0x7f8b180b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400b2e0 tx=0x7f8b24005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.257+0000 7f8b2dd9b700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] conn(0x7f8b180bdac0 0x7f8b180bffb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:56.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.258+0000 7f8b1f7fe700 1 -- 172.21.15.115:0/669716815 <== osd.2 v2:172.21.15.195:6800/960731357 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b28008ec0 con 0x7f8b180bdac0 2024-05-09T05:35:56.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.274+0000 7f8b1d7fa700 1 -- 172.21.15.115:0/669716815 --> [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b30064290 con 0x7f8b180bdac0 2024-05-09T05:35:56.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.275+0000 7f8b1f7fe700 1 -- 172.21.15.115:0/669716815 <== osd.2 v2:172.21.15.195:6800/960731357 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8b30064290 con 0x7f8b180bdac0 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.275+0000 7f8b1d7fa700 1 -- 172.21.15.115:0/669716815 >> [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] conn(0x7f8b180bdac0 msgr2=0x7f8b180bffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.275+0000 7f8b1d7fa700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] conn(0x7f8b180bdac0 0x7f8b180bffb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.275+0000 7f8b1d7fa700 1 -- 172.21.15.115:0/669716815 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8b180b2910 msgr2=0x7f8b180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.275+0000 7f8b1d7fa700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8b180b2910 0x7f8b180b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400b2e0 tx=0x7f8b24005fb0 comp rx=0 tx=0).stop 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.275+0000 7f8b1d7fa700 1 -- 172.21.15.115:0/669716815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3010e950 msgr2=0x7f8b3019f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.275+0000 7f8b1d7fa700 1 --2- 172.21.15.115:0/669716815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3010e950 0x7f8b3019f5a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800ef30 tx=0x7f8b2800c3a0 comp rx=0 tx=0).stop 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.275+0000 7f8b2ed9d700 1 -- 172.21.15.115:0/669716815 reap_dead start 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.276+0000 7f8b1d7fa700 1 -- 172.21.15.115:0/669716815 shutdown_connections 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.276+0000 7f8b1d7fa700 1 -- 172.21.15.115:0/669716815 >> 172.21.15.115:0/669716815 conn(0x7f8b30078e00 msgr2=0x7f8b301012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:56.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.276+0000 7f8b1d7fa700 1 -- 172.21.15.115:0/669716815 shutdown_connections 2024-05-09T05:35:56.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:56.276+0000 7f8b1d7fa700 1 -- 172.21.15.115:0/669716815 wait complete. 2024-05-09T05:35:56.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.642+0000 7fac60597700 1 -- 172.21.15.115:0/992009732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac58108760 msgr2=0x7fac581100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:57.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.642+0000 7fac60597700 1 --2- 172.21.15.115:0/992009732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac58108760 0x7fac581100c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fac5000b0d0 tx=0x7fac5000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:35:57.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.642+0000 7fac60597700 1 -- 172.21.15.115:0/992009732 shutdown_connections 2024-05-09T05:35:57.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.642+0000 7fac60597700 1 --2- 172.21.15.115:0/992009732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac581034e0 0x7fac581038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.642+0000 7fac60597700 1 --2- 172.21.15.115:0/992009732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac58108760 0x7fac581100c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.642+0000 7fac60597700 1 --2- 172.21.15.115:0/992009732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac58107e10 0x7fac58108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.642+0000 7fac60597700 1 -- 172.21.15.115:0/992009732 >> 172.21.15.115:0/992009732 conn(0x7fac580fee50 msgr2=0x7fac58101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.642+0000 7fac60597700 1 -- 172.21.15.115:0/992009732 shutdown_connections 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.642+0000 7fac60597700 1 -- 172.21.15.115:0/992009732 wait complete. 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac60597700 1 Processor -- start 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac60597700 1 -- start start 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac60597700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac581034e0 0x7fac5819bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac60597700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac58107e10 0x7fac5819c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac60597700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac58108760 0x7fac5819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac60597700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac58113260 con 0x7fac58107e10 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac60597700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac581130e0 con 0x7fac58108760 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac60597700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fac581133e0 con 0x7fac581034e0 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac5db32700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac58107e10 0x7fac5819c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:57.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac5e333700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac581034e0 0x7fac5819bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:57.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac5db32700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac58107e10 0x7fac5819c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44106/0 (socket says 172.21.15.115:44106) 2024-05-09T05:35:57.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.643+0000 7fac5db32700 1 -- 172.21.15.115:0/2310593596 learned_addr learned my addr 172.21.15.115:0/2310593596 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:35:57.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac5eb34700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac58108760 0x7fac5819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:57.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac5db32700 1 -- 172.21.15.115:0/2310593596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac581034e0 msgr2=0x7fac5819bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:57.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac5db32700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac581034e0 0x7fac5819bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:57.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac5db32700 1 -- 172.21.15.115:0/2310593596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac58108760 msgr2=0x7fac5819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:57.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac5db32700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac58108760 0x7fac5819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:57.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac5db32700 1 -- 172.21.15.115:0/2310593596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac581a6e80 con 0x7fac58107e10 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac5db32700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac58107e10 0x7fac5819c0a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fac4800c970 tx=0x7fac4800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac4f7fe700 1 -- 172.21.15.115:0/2310593596 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48004a30 con 0x7fac58107e10 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac581a7170 con 0x7fac58107e10 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac4f7fe700 1 -- 172.21.15.115:0/2310593596 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac480054b0 con 0x7fac58107e10 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac4f7fe700 1 -- 172.21.15.115:0/2310593596 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48007af0 con 0x7fac58107e10 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.644+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac581a76b0 con 0x7fac58107e10 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.645+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac58063ed0 con 0x7fac58107e10 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.647+0000 7fac4f7fe700 1 -- 172.21.15.115:0/2310593596 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fac48007c90 con 0x7fac58107e10 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.648+0000 7fac4f7fe700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac440b2910 0x7fac440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.648+0000 7fac5e333700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac440b2910 0x7fac440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.648+0000 7fac4f7fe700 1 -- 172.21.15.115:0/2310593596 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fac480e2700 con 0x7fac58107e10 2024-05-09T05:35:57.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.648+0000 7fac4f7fe700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] conn(0x7fac440bdac0 0x7fac440bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.648+0000 7fac4f7fe700 1 -- 172.21.15.115:0/2310593596 --> [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac48012ec0 con 0x7fac440bdac0 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.648+0000 7fac4f7fe700 1 -- 172.21.15.115:0/2310593596 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fac480e2b50 con 0x7fac58107e10 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.648+0000 7fac5e333700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac440b2910 0x7fac440b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac54009c60 tx=0x7fac54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.648+0000 7fac5eb34700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] conn(0x7fac440bdac0 0x7fac440bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.649+0000 7fac5eb34700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] conn(0x7fac440bdac0 0x7fac440bffb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.650+0000 7fac4f7fe700 1 -- 172.21.15.115:0/2310593596 <== osd.7 v2:172.21.15.161:6816/2577292327 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac48012ec0 con 0x7fac440bdac0 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.666+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 --> [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac580642d0 con 0x7fac440bdac0 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.667+0000 7fac4f7fe700 1 -- 172.21.15.115:0/2310593596 <== osd.7 v2:172.21.15.161:6816/2577292327 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fac580642d0 con 0x7fac440bdac0 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 >> [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] conn(0x7fac440bdac0 msgr2=0x7fac440bffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] conn(0x7fac440bdac0 0x7fac440bffb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac440b2910 msgr2=0x7fac440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:57.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac440b2910 0x7fac440b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac54009c60 tx=0x7fac54009260 comp rx=0 tx=0).stop 2024-05-09T05:35:57.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac58107e10 msgr2=0x7fac5819c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:35:57.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 --2- 172.21.15.115:0/2310593596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac58107e10 0x7fac5819c0a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fac4800c970 tx=0x7fac4800cc80 comp rx=0 tx=0).stop 2024-05-09T05:35:57.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac5eb34700 1 -- 172.21.15.115:0/2310593596 reap_dead start 2024-05-09T05:35:57.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 shutdown_connections 2024-05-09T05:35:57.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 >> 172.21.15.115:0/2310593596 conn(0x7fac580fee50 msgr2=0x7fac58107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:35:57.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 shutdown_connections 2024-05-09T05:35:57.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:35:55.668+0000 7fac60597700 1 -- 172.21.15.115:0/2310593596 wait complete. 2024-05-09T05:35:57.056 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:56 smithi115 ceph-mon[30349]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:35:57.057 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:56 smithi161 ceph-mon[33805]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:35:57.057 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:56 smithi195 ceph-mon[33844]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:35:57.319 INFO:teuthology.orchestra.run.smithi115.stdout:201863462973 2024-05-09T05:35:57.320 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T05:35:57.951 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:57.951 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:35:58.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:35:58 smithi195 ceph-mon[33844]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:58.886 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:35:58 smithi115 ceph-mon[30349]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:58.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:35:58 smithi161 ceph-mon[33805]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:35:59.237 INFO:teuthology.orchestra.run.smithi115.stdout:77309411414 2024-05-09T05:35:59.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T05:36:00.687 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:00 smithi115 ceph-mon[30349]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:36:00.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:00 smithi195 ceph-mon[33844]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:36:00.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:00 smithi161 ceph-mon[33805]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:36:01.023 INFO:teuthology.orchestra.run.smithi115.stdout:107374182481 2024-05-09T05:36:01.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T05:36:01.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.232+0000 7f139e9fb700 1 -- 172.21.15.115:0/1362977997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1398108760 msgr2=0x7f13981100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:01.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.232+0000 7f139e9fb700 1 --2- 172.21.15.115:0/1362977997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1398108760 0x7f13981100c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f139000b0d0 tx=0x7f139000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:36:01.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.232+0000 7f139e9fb700 1 -- 172.21.15.115:0/1362977997 shutdown_connections 2024-05-09T05:36:01.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.232+0000 7f139e9fb700 1 --2- 172.21.15.115:0/1362977997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1398108760 0x7f13981100c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:01.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.232+0000 7f139e9fb700 1 --2- 172.21.15.115:0/1362977997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13981034e0 0x7f13981038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.232+0000 7f139e9fb700 1 --2- 172.21.15.115:0/1362977997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1398107e10 0x7f1398108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.232+0000 7f139e9fb700 1 -- 172.21.15.115:0/1362977997 >> 172.21.15.115:0/1362977997 conn(0x7f13980fef30 msgr2=0x7f1398101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.232+0000 7f139e9fb700 1 -- 172.21.15.115:0/1362977997 shutdown_connections 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.232+0000 7f139e9fb700 1 -- 172.21.15.115:0/1362977997 wait complete. 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.233+0000 7f139e9fb700 1 Processor -- start 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.233+0000 7f139e9fb700 1 -- start start 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.233+0000 7f139e9fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13981034e0 0x7f139819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.233+0000 7f139e9fb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1398107e10 0x7f139819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.233+0000 7f139e9fb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1398108760 0x7f139819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.233+0000 7f139e9fb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1398113240 con 0x7f1398107e10 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.233+0000 7f139e9fb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13981130c0 con 0x7f13981034e0 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.233+0000 7f139e9fb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f13981133c0 con 0x7f1398108760 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13977fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1398107e10 0x7f139819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13977fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1398107e10 0x7f139819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44134/0 (socket says 172.21.15.115:44134) 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13977fe700 1 -- 172.21.15.115:0/2254272413 learned_addr learned my addr 172.21.15.115:0/2254272413 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:01.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f139cf98700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1398108760 0x7f139819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:01.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f1397fff700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13981034e0 0x7f139819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:01.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13977fe700 1 -- 172.21.15.115:0/2254272413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1398108760 msgr2=0x7f139819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:01.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13977fe700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1398108760 0x7f139819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:01.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13977fe700 1 -- 172.21.15.115:0/2254272413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13981034e0 msgr2=0x7f139819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:01.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13977fe700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13981034e0 0x7f139819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:01.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13977fe700 1 -- 172.21.15.115:0/2254272413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139819fc30 con 0x7f1398107e10 2024-05-09T05:36:01.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13977fe700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1398107e10 0x7f139819b300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f138c00c8f0 tx=0x7f138c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:01.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13957fa700 1 -- 172.21.15.115:0/2254272413 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c007b00 con 0x7f1398107e10 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13957fa700 1 -- 172.21.15.115:0/2254272413 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f138c007ca0 con 0x7f1398107e10 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f13957fa700 1 -- 172.21.15.115:0/2254272413 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c018660 con 0x7f1398107e10 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.234+0000 7f139e9fb700 1 -- 172.21.15.115:0/2254272413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139819ff20 con 0x7f1398107e10 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.235+0000 7f139e9fb700 1 -- 172.21.15.115:0/2254272413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13981a7840 con 0x7f1398107e10 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.235+0000 7f137effd700 1 -- 172.21.15.115:0/2254272413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1378001030 con 0x7f1398107e10 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.237+0000 7f13957fa700 1 -- 172.21.15.115:0/2254272413 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f138c00f450 con 0x7f1398107e10 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.238+0000 7f13957fa700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f13800b2990 0x7f13800b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.238+0000 7f1397fff700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f13800b2990 0x7f13800b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.238+0000 7f13957fa700 1 -- 172.21.15.115:0/2254272413 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f138c0e27f0 con 0x7f1398107e10 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.238+0000 7f13957fa700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] conn(0x7f13800bdb40 0x7f13800c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.238+0000 7f13957fa700 1 -- 172.21.15.115:0/2254272413 --> [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f138c015e70 con 0x7f13800bdb40 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.238+0000 7f13957fa700 1 -- 172.21.15.115:0/2254272413 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f138c0e2c40 con 0x7f1398107e10 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.239+0000 7f1397fff700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f13800b2990 0x7f13800b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1388005fd0 tx=0x7f138800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.239+0000 7f139cf98700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] conn(0x7f13800bdb40 0x7f13800c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:01.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.239+0000 7f139cf98700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] conn(0x7f13800bdb40 0x7f13800c0030 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.240+0000 7f13957fa700 1 -- 172.21.15.115:0/2254272413 <== osd.5 v2:172.21.15.195:6808/3142086553 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f138c015e70 con 0x7f13800bdb40 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.256+0000 7f137effd700 1 -- 172.21.15.115:0/2254272413 --> [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1378002e60 con 0x7f13800bdb40 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f13957fa700 1 -- 172.21.15.115:0/2254272413 <== osd.5 v2:172.21.15.195:6808/3142086553 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1378002e60 con 0x7f13800bdb40 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f137effd700 1 -- 172.21.15.115:0/2254272413 >> [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] conn(0x7f13800bdb40 msgr2=0x7f13800c0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f137effd700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] conn(0x7f13800bdb40 0x7f13800c0030 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f137effd700 1 -- 172.21.15.115:0/2254272413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f13800b2990 msgr2=0x7f13800b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f137effd700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f13800b2990 0x7f13800b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1388005fd0 tx=0x7f138800a560 comp rx=0 tx=0).stop 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f137effd700 1 -- 172.21.15.115:0/2254272413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1398107e10 msgr2=0x7f139819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f137effd700 1 --2- 172.21.15.115:0/2254272413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1398107e10 0x7f139819b300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f138c00c8f0 tx=0x7f138c00ccb0 comp rx=0 tx=0).stop 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f139cf98700 1 -- 172.21.15.115:0/2254272413 reap_dead start 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f137effd700 1 -- 172.21.15.115:0/2254272413 shutdown_connections 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f137effd700 1 -- 172.21.15.115:0/2254272413 >> 172.21.15.115:0/2254272413 conn(0x7f13980fef30 msgr2=0x7f139806c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.257+0000 7f137effd700 1 -- 172.21.15.115:0/2254272413 shutdown_connections 2024-05-09T05:36:01.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:01.258+0000 7f137effd700 1 -- 172.21.15.115:0/2254272413 wait complete. 2024-05-09T05:36:02.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:02 smithi115 ceph-mon[30349]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:36:02.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:02 smithi195 ceph-mon[33844]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:36:02.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:02 smithi161 ceph-mon[33805]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:36:04.516 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:04 smithi115 ceph-mon[30349]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:04.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:04 smithi195 ceph-mon[33844]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:04.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:04 smithi161 ceph-mon[33805]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:04.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.969+0000 7fb4e919a700 1 -- 172.21.15.115:0/2095843230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4108740 msgr2=0x7fb4e41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:04.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.969+0000 7fb4e919a700 1 --2- 172.21.15.115:0/2095843230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4108740 0x7fb4e41100a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009f90 tx=0x7fb4dc00b330 comp rx=0 tx=0).stop 2024-05-09T05:36:04.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.969+0000 7fb4e919a700 1 -- 172.21.15.115:0/2095843230 shutdown_connections 2024-05-09T05:36:04.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.969+0000 7fb4e919a700 1 --2- 172.21.15.115:0/2095843230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4108740 0x7fb4e41100a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:04.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.969+0000 7fb4e919a700 1 --2- 172.21.15.115:0/2095843230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4e41034c0 0x7fb4e41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.969+0000 7fb4e919a700 1 --2- 172.21.15.115:0/2095843230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e4107df0 0x7fb4e4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.969+0000 7fb4e919a700 1 -- 172.21.15.115:0/2095843230 >> 172.21.15.115:0/2095843230 conn(0x7fb4e40fedc0 msgr2=0x7fb4e41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.969+0000 7fb4e919a700 1 -- 172.21.15.115:0/2095843230 shutdown_connections 2024-05-09T05:36:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.969+0000 7fb4e919a700 1 -- 172.21.15.115:0/2095843230 wait complete. 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e919a700 1 Processor -- start 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e919a700 1 -- start start 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e919a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e41034c0 0x7fb4e41a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e919a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4e4107df0 0x7fb4e41a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e919a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4108740 0x7fb4e419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e919a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e4114370 con 0x7fb4e4107df0 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e919a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e41141f0 con 0x7fb4e4108740 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e919a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e41144f0 con 0x7fb4e41034c0 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4e4107df0 0x7fb4e41a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4e4107df0 0x7fb4e41a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35192/0 (socket says 172.21.15.115:35192) 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e259c700 1 -- 172.21.15.115:0/1209502487 learned_addr learned my addr 172.21.15.115:0/1209502487 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e2d9d700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e41034c0 0x7fb4e41a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.970+0000 7fb4e359e700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4108740 0x7fb4e419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4e259c700 1 -- 172.21.15.115:0/1209502487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e41034c0 msgr2=0x7fb4e41a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4e259c700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e41034c0 0x7fb4e41a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4e259c700 1 -- 172.21.15.115:0/1209502487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4108740 msgr2=0x7fb4e419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4e259c700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4108740 0x7fb4e419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4e259c700 1 -- 172.21.15.115:0/1209502487 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e419b280 con 0x7fb4e4107df0 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4e259c700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4e4107df0 0x7fb4e41a0f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800e9f0 tx=0x7fb4d800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4cbfff700 1 -- 172.21.15.115:0/1209502487 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8004a30 con 0x7fb4e4107df0 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e419b570 con 0x7fb4e4107df0 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4cbfff700 1 -- 172.21.15.115:0/1209502487 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4d800cc50 con 0x7fb4e4107df0 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4cbfff700 1 -- 172.21.15.115:0/1209502487 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80054e0 con 0x7fb4e4107df0 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.971+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e41a7770 con 0x7fb4e4107df0 2024-05-09T05:36:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.972+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4e4063f50 con 0x7fb4e4107df0 2024-05-09T05:36:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.974+0000 7fb4cbfff700 1 -- 172.21.15.115:0/1209502487 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb4d800c7c0 con 0x7fb4e4107df0 2024-05-09T05:36:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.974+0000 7fb4cbfff700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb4cc0b2910 0x7fb4cc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.975+0000 7fb4e2d9d700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb4cc0b2910 0x7fb4cc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.975+0000 7fb4cbfff700 1 -- 172.21.15.115:0/1209502487 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fb4d80e30a0 con 0x7fb4e4107df0 2024-05-09T05:36:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.975+0000 7fb4cbfff700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] conn(0x7fb4cc0bdac0 0x7fb4cc0bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.975+0000 7fb4cbfff700 1 -- 172.21.15.115:0/1209502487 --> [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4d801aec0 con 0x7fb4cc0bdac0 2024-05-09T05:36:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.975+0000 7fb4cbfff700 1 -- 172.21.15.115:0/1209502487 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb4d80e34f0 con 0x7fb4e4107df0 2024-05-09T05:36:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.975+0000 7fb4e2d9d700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb4cc0b2910 0x7fb4cc0b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4005fd0 tx=0x7fb4d4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.975+0000 7fb4e359e700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] conn(0x7fb4cc0bdac0 0x7fb4cc0bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:04.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.976+0000 7fb4e359e700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] conn(0x7fb4cc0bdac0 0x7fb4cc0bffb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:04.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.976+0000 7fb4cbfff700 1 -- 172.21.15.115:0/1209502487 <== osd.11 v2:172.21.15.195:6824/2489992228 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4d801aec0 con 0x7fb4cc0bdac0 2024-05-09T05:36:04.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.993+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 --> [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4e4064350 con 0x7fb4cc0bdac0 2024-05-09T05:36:04.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.993+0000 7fb4cbfff700 1 -- 172.21.15.115:0/1209502487 <== osd.11 v2:172.21.15.195:6824/2489992228 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4e4064350 con 0x7fb4cc0bdac0 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 >> [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] conn(0x7fb4cc0bdac0 msgr2=0x7fb4cc0bffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] conn(0x7fb4cc0bdac0 0x7fb4cc0bffb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb4cc0b2910 msgr2=0x7fb4cc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb4cc0b2910 0x7fb4cc0b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4005fd0 tx=0x7fb4d4009310 comp rx=0 tx=0).stop 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4e4107df0 msgr2=0x7fb4e41a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 --2- 172.21.15.115:0/1209502487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4e4107df0 0x7fb4e41a0f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800e9f0 tx=0x7fb4d800ed00 comp rx=0 tx=0).stop 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e359e700 1 -- 172.21.15.115:0/1209502487 reap_dead start 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 shutdown_connections 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 >> 172.21.15.115:0/1209502487 conn(0x7fb4e40fedc0 msgr2=0x7fb4e41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 shutdown_connections 2024-05-09T05:36:04.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:04.994+0000 7fb4e919a700 1 -- 172.21.15.115:0/1209502487 wait complete. 2024-05-09T05:36:05.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.178+0000 7f68f0f03700 1 -- 172.21.15.115:0/3473999671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68ec108f30 msgr2=0x7f68ec109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.178+0000 7f68f0f03700 1 --2- 172.21.15.115:0/3473999671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68ec108f30 0x7f68ec109340 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f68e00099c0 tx=0x7f68e0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.178+0000 7f68f0f03700 1 -- 172.21.15.115:0/3473999671 shutdown_connections 2024-05-09T05:36:05.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.178+0000 7f68f0f03700 1 --2- 172.21.15.115:0/3473999671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ec109880 0x7f68ec1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.178+0000 7f68f0f03700 1 --2- 172.21.15.115:0/3473999671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec104650 0x7f68ec104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.178+0000 7f68f0f03700 1 --2- 172.21.15.115:0/3473999671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68ec108f30 0x7f68ec109340 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.178+0000 7f68f0f03700 1 -- 172.21.15.115:0/3473999671 >> 172.21.15.115:0/3473999671 conn(0x7f68ec079f00 msgr2=0x7f68ec07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:05.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.178+0000 7f68f0f03700 1 -- 172.21.15.115:0/3473999671 shutdown_connections 2024-05-09T05:36:05.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.178+0000 7f68f0f03700 1 -- 172.21.15.115:0/3473999671 wait complete. 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68f0f03700 1 Processor -- start 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68f0f03700 1 -- start start 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68f0f03700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68ec104650 0x7f68ec19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68f0f03700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ec108f30 0x7f68ec19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68f0f03700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec109880 0x7f68ec19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68f0f03700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec114340 con 0x7f68ec104650 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68f0f03700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec1141c0 con 0x7f68ec108f30 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68f0f03700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec1144c0 con 0x7f68ec109880 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ea59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68ec104650 0x7f68ec19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ea59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68ec104650 0x7f68ec19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35200/0 (socket says 172.21.15.115:35200) 2024-05-09T05:36:05.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ea59c700 1 -- 172.21.15.115:0/3102208636 learned_addr learned my addr 172.21.15.115:0/3102208636 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68e9d9b700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ec108f30 0x7f68ec19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ead9d700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec109880 0x7f68ec19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ea59c700 1 -- 172.21.15.115:0/3102208636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec109880 msgr2=0x7f68ec19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ea59c700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec109880 0x7f68ec19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ea59c700 1 -- 172.21.15.115:0/3102208636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ec108f30 msgr2=0x7f68ec19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ea59c700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68ec108f30 0x7f68ec19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ea59c700 1 -- 172.21.15.115:0/3102208636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec19fbb0 con 0x7f68ec104650 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.179+0000 7f68ead9d700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec109880 0x7f68ec19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.180+0000 7f68ea59c700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68ec104650 0x7f68ec19ad90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c920 tx=0x7f68dc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.180+0000 7f68db7fe700 1 -- 172.21.15.115:0/3102208636 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc007b00 con 0x7f68ec104650 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.180+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ec19fea0 con 0x7f68ec104650 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.180+0000 7f68db7fe700 1 -- 172.21.15.115:0/3102208636 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68dc004cc0 con 0x7f68ec104650 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.180+0000 7f68db7fe700 1 -- 172.21.15.115:0/3102208636 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc0053f0 con 0x7f68ec104650 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.180+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ec1a0400 con 0x7f68ec104650 2024-05-09T05:36:05.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.180+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68ec063f10 con 0x7f68ec104650 2024-05-09T05:36:05.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.183+0000 7f68db7fe700 1 -- 172.21.15.115:0/3102208636 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f68dc004750 con 0x7f68ec104650 2024-05-09T05:36:05.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.183+0000 7f68db7fe700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f68d40b2910 0x7f68d40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.183+0000 7f68e9d9b700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f68d40b2910 0x7f68d40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.183+0000 7f68db7fe700 1 -- 172.21.15.115:0/3102208636 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f68dc0e2800 con 0x7f68ec104650 2024-05-09T05:36:05.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.184+0000 7f68db7fe700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] conn(0x7f68d40bdac0 0x7f68d40bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.184+0000 7f68db7fe700 1 -- 172.21.15.115:0/3102208636 --> [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68dc012ec0 con 0x7f68d40bdac0 2024-05-09T05:36:05.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.184+0000 7f68db7fe700 1 -- 172.21.15.115:0/3102208636 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f68dc0e2c50 con 0x7f68ec104650 2024-05-09T05:36:05.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.184+0000 7f68ead9d700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] conn(0x7f68d40bdac0 0x7f68d40bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.184+0000 7f68e9d9b700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f68d40b2910 0x7f68d40b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009990 tx=0x7f68e0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:05.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.184+0000 7f68ead9d700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] conn(0x7f68d40bdac0 0x7f68d40bffb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:05.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.185+0000 7f68db7fe700 1 -- 172.21.15.115:0/3102208636 <== osd.9 v2:172.21.15.115:6826/4143175430 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68dc012ec0 con 0x7f68d40bdac0 2024-05-09T05:36:05.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.201+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 --> [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68ec064310 con 0x7f68d40bdac0 2024-05-09T05:36:05.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.201+0000 7f68db7fe700 1 -- 172.21.15.115:0/3102208636 <== osd.9 v2:172.21.15.115:6826/4143175430 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f68ec064310 con 0x7f68d40bdac0 2024-05-09T05:36:05.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] conn(0x7f68d40bdac0 msgr2=0x7f68d40bffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] conn(0x7f68d40bdac0 0x7f68d40bffb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f68d40b2910 msgr2=0x7f68d40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f68d40b2910 0x7f68d40b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009990 tx=0x7f68e0005fb0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68ec104650 msgr2=0x7f68ec19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 --2- 172.21.15.115:0/3102208636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68ec104650 0x7f68ec19ad90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c920 tx=0x7f68dc00cc30 comp rx=0 tx=0).stop 2024-05-09T05:36:05.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68ead9d700 1 -- 172.21.15.115:0/3102208636 reap_dead start 2024-05-09T05:36:05.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 shutdown_connections 2024-05-09T05:36:05.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 >> 172.21.15.115:0/3102208636 conn(0x7f68ec079f00 msgr2=0x7f68ec1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:05.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 shutdown_connections 2024-05-09T05:36:05.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.202+0000 7f68f0f03700 1 -- 172.21.15.115:0/3102208636 wait complete. 2024-05-09T05:36:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.394+0000 7fc8f3634700 1 -- 172.21.15.115:0/3182622456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8ec108740 msgr2=0x7fc8ec1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.394+0000 7fc8f3634700 1 --2- 172.21.15.115:0/3182622456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8ec108740 0x7fc8ec1100a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400b0d0 tx=0x7fc8e400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.394+0000 7fc8f3634700 1 -- 172.21.15.115:0/3182622456 shutdown_connections 2024-05-09T05:36:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.394+0000 7fc8f3634700 1 --2- 172.21.15.115:0/3182622456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8ec108740 0x7fc8ec1100a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.394+0000 7fc8f3634700 1 --2- 172.21.15.115:0/3182622456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8ec1034c0 0x7fc8ec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.394+0000 7fc8f3634700 1 --2- 172.21.15.115:0/3182622456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ec107df0 0x7fc8ec108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.394+0000 7fc8f3634700 1 -- 172.21.15.115:0/3182622456 >> 172.21.15.115:0/3182622456 conn(0x7fc8ec0fedc0 msgr2=0x7fc8ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.395+0000 7fc8f3634700 1 -- 172.21.15.115:0/3182622456 shutdown_connections 2024-05-09T05:36:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.395+0000 7fc8f3634700 1 -- 172.21.15.115:0/3182622456 wait complete. 2024-05-09T05:36:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.395+0000 7fc8f3634700 1 Processor -- start 2024-05-09T05:36:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.395+0000 7fc8f3634700 1 -- start start 2024-05-09T05:36:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f3634700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ec1034c0 0x7fc8ec1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f3634700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8ec107df0 0x7fc8ec1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f3634700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8ec108740 0x7fc8ec19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f3634700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec114370 con 0x7fc8ec108740 2024-05-09T05:36:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f3634700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1141f0 con 0x7fc8ec1034c0 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f3634700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1144f0 con 0x7fc8ec107df0 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f1bd1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8ec108740 0x7fc8ec19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f1bd1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8ec108740 0x7fc8ec19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35216/0 (socket says 172.21.15.115:35216) 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f1bd1700 1 -- 172.21.15.115:0/1578346239 learned_addr learned my addr 172.21.15.115:0/1578346239 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f13d0700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ec1034c0 0x7fc8ec1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f0bcf700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8ec107df0 0x7fc8ec1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f1bd1700 1 -- 172.21.15.115:0/1578346239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8ec107df0 msgr2=0x7fc8ec1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f1bd1700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8ec107df0 0x7fc8ec1a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f1bd1700 1 -- 172.21.15.115:0/1578346239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ec1034c0 msgr2=0x7fc8ec1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f1bd1700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ec1034c0 0x7fc8ec1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f1bd1700 1 -- 172.21.15.115:0/1578346239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec19b260 con 0x7fc8ec108740 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f1bd1700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8ec108740 0x7fc8ec19ab30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009850 tx=0x7fc8e4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8e27fc700 1 -- 172.21.15.115:0/1578346239 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e401f070 con 0x7fc8ec108740 2024-05-09T05:36:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8e27fc700 1 -- 172.21.15.115:0/1578346239 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8e4005500 con 0x7fc8ec108740 2024-05-09T05:36:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec19b4f0 con 0x7fc8ec108740 2024-05-09T05:36:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.396+0000 7fc8e27fc700 1 -- 172.21.15.115:0/1578346239 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e401b8f0 con 0x7fc8ec108740 2024-05-09T05:36:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.397+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec1a7750 con 0x7fc8ec108740 2024-05-09T05:36:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.397+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc8ec063f50 con 0x7fc8ec108740 2024-05-09T05:36:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.399+0000 7fc8e27fc700 1 -- 172.21.15.115:0/1578346239 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc8e4022070 con 0x7fc8ec108740 2024-05-09T05:36:05.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.400+0000 7fc8e27fc700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8d80b2910 0x7fc8d80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.400+0000 7fc8f13d0700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8d80b2910 0x7fc8d80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.400+0000 7fc8e27fc700 1 -- 172.21.15.115:0/1578346239 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fc8e400f020 con 0x7fc8ec108740 2024-05-09T05:36:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.400+0000 7fc8e27fc700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] conn(0x7fc8d80bdac0 0x7fc8d80bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.400+0000 7fc8e27fc700 1 -- 172.21.15.115:0/1578346239 --> [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8e40075c0 con 0x7fc8d80bdac0 2024-05-09T05:36:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.400+0000 7fc8e27fc700 1 -- 172.21.15.115:0/1578346239 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc8e40e4220 con 0x7fc8ec108740 2024-05-09T05:36:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.400+0000 7fc8f0bcf700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] conn(0x7fc8d80bdac0 0x7fc8d80bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.400+0000 7fc8f13d0700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8d80b2910 0x7fc8d80b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8004630 tx=0x7fc8e800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.400+0000 7fc8f0bcf700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] conn(0x7fc8d80bdac0 0x7fc8d80bffb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.401+0000 7fc8e27fc700 1 -- 172.21.15.115:0/1578346239 <== osd.0 v2:172.21.15.115:6802/2561526848 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8e40075c0 con 0x7fc8d80bdac0 2024-05-09T05:36:05.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.417+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 --> [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc8ec0643d0 con 0x7fc8d80bdac0 2024-05-09T05:36:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8e27fc700 1 -- 172.21.15.115:0/1578346239 <== osd.0 v2:172.21.15.115:6802/2561526848 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc8ec0643d0 con 0x7fc8d80bdac0 2024-05-09T05:36:05.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] conn(0x7fc8d80bdac0 msgr2=0x7fc8d80bffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] conn(0x7fc8d80bdac0 0x7fc8d80bffb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8d80b2910 msgr2=0x7fc8d80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8d80b2910 0x7fc8d80b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8004630 tx=0x7fc8e800b3c0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8ec108740 msgr2=0x7fc8ec19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 --2- 172.21.15.115:0/1578346239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8ec108740 0x7fc8ec19ab30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009850 tx=0x7fc8e4007d90 comp rx=0 tx=0).stop 2024-05-09T05:36:05.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f1bd1700 1 -- 172.21.15.115:0/1578346239 reap_dead start 2024-05-09T05:36:05.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 shutdown_connections 2024-05-09T05:36:05.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 >> 172.21.15.115:0/1578346239 conn(0x7fc8ec0fedc0 msgr2=0x7fc8ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:05.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 shutdown_connections 2024-05-09T05:36:05.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.418+0000 7fc8f3634700 1 -- 172.21.15.115:0/1578346239 wait complete. 2024-05-09T05:36:05.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.840+0000 7fc8b0716700 1 -- 172.21.15.115:0/3351662169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a8107df0 msgr2=0x7fc8a8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.840+0000 7fc8b0716700 1 --2- 172.21.15.115:0/3351662169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a8107df0 0x7fc8a8108200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc8980099c0 tx=0x7fc898009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.840+0000 7fc8b0716700 1 -- 172.21.15.115:0/3351662169 shutdown_connections 2024-05-09T05:36:05.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.840+0000 7fc8b0716700 1 --2- 172.21.15.115:0/3351662169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a81034c0 0x7fc8a81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.840+0000 7fc8b0716700 1 --2- 172.21.15.115:0/3351662169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a8108740 0x7fc8a81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.841+0000 7fc8b0716700 1 --2- 172.21.15.115:0/3351662169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a8107df0 0x7fc8a8108200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.841+0000 7fc8b0716700 1 -- 172.21.15.115:0/3351662169 >> 172.21.15.115:0/3351662169 conn(0x7fc8a80fedc0 msgr2=0x7fc8a81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:05.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.841+0000 7fc8b0716700 1 -- 172.21.15.115:0/3351662169 shutdown_connections 2024-05-09T05:36:05.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.841+0000 7fc8b0716700 1 -- 172.21.15.115:0/3351662169 wait complete. 2024-05-09T05:36:05.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.841+0000 7fc8b0716700 1 Processor -- start 2024-05-09T05:36:05.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.841+0000 7fc8b0716700 1 -- start start 2024-05-09T05:36:05.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.841+0000 7fc8b0716700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a81034c0 0x7fc8a81a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.841+0000 7fc8b0716700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a8107df0 0x7fc8a81a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8b0716700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a8108740 0x7fc8a819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8b0716700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a8113200 con 0x7fc8a8108740 2024-05-09T05:36:05.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8b0716700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a8113080 con 0x7fc8a8107df0 2024-05-09T05:36:05.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8b0716700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a8113380 con 0x7fc8a81034c0 2024-05-09T05:36:05.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8aecb3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a8108740 0x7fc8a819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8aecb3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a8108740 0x7fc8a819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35228/0 (socket says 172.21.15.115:35228) 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8aecb3700 1 -- 172.21.15.115:0/2305812513 learned_addr learned my addr 172.21.15.115:0/2305812513 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8ae4b2700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a81034c0 0x7fc8a81a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8adcb1700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a8107df0 0x7fc8a81a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8aecb3700 1 -- 172.21.15.115:0/2305812513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a81034c0 msgr2=0x7fc8a81a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8aecb3700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8a81034c0 0x7fc8a81a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8aecb3700 1 -- 172.21.15.115:0/2305812513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a8107df0 msgr2=0x7fc8a81a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8aecb3700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a8107df0 0x7fc8a81a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8aecb3700 1 -- 172.21.15.115:0/2305812513 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a819b160 con 0x7fc8a8108740 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8aecb3700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a8108740 0x7fc8a819aaf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0007ee0 tx=0x7fc8a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc89f7fe700 1 -- 172.21.15.115:0/2305812513 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000dbd0 con 0x7fc8a8108740 2024-05-09T05:36:05.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc89f7fe700 1 -- 172.21.15.115:0/2305812513 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8a000dd70 con 0x7fc8a8108740 2024-05-09T05:36:05.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a819b450 con 0x7fc8a8108740 2024-05-09T05:36:05.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.842+0000 7fc89f7fe700 1 -- 172.21.15.115:0/2305812513 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a00058d0 con 0x7fc8a8108740 2024-05-09T05:36:05.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.843+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a819b910 con 0x7fc8a8108740 2024-05-09T05:36:05.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.843+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc8a8063f10 con 0x7fc8a8108740 2024-05-09T05:36:05.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.845+0000 7fc89f7fe700 1 -- 172.21.15.115:0/2305812513 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc8a001a020 con 0x7fc8a8108740 2024-05-09T05:36:05.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.846+0000 7fc89f7fe700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8940b2910 0x7fc8940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.846+0000 7fc8ae4b2700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8940b2910 0x7fc8940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.846+0000 7fc89f7fe700 1 -- 172.21.15.115:0/2305812513 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fc8a00e28c0 con 0x7fc8a8108740 2024-05-09T05:36:05.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.846+0000 7fc89f7fe700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] conn(0x7fc8940bdac0 0x7fc8940bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:05.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.846+0000 7fc89f7fe700 1 -- 172.21.15.115:0/2305812513 --> [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8a0012ec0 con 0x7fc8940bdac0 2024-05-09T05:36:05.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.846+0000 7fc89f7fe700 1 -- 172.21.15.115:0/2305812513 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc8a00e2d10 con 0x7fc8a8108740 2024-05-09T05:36:05.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.846+0000 7fc8ae4b2700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8940b2910 0x7fc8940b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc8a819bc30 tx=0x7fc8a400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:05.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.846+0000 7fc8adcb1700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] conn(0x7fc8940bdac0 0x7fc8940bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:05.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.847+0000 7fc8adcb1700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] conn(0x7fc8940bdac0 0x7fc8940bffb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:05.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.848+0000 7fc89f7fe700 1 -- 172.21.15.115:0/2305812513 <== osd.10 v2:172.21.15.161:6824/1493802614 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8a0012ec0 con 0x7fc8940bdac0 2024-05-09T05:36:05.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.864+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 --> [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc8a8064310 con 0x7fc8940bdac0 2024-05-09T05:36:05.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.864+0000 7fc89f7fe700 1 -- 172.21.15.115:0/2305812513 <== osd.10 v2:172.21.15.161:6824/1493802614 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc8a8064310 con 0x7fc8940bdac0 2024-05-09T05:36:05.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.864+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 >> [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] conn(0x7fc8940bdac0 msgr2=0x7fc8940bffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.864+0000 7fc8b0716700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] conn(0x7fc8940bdac0 0x7fc8940bffb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.865+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8940b2910 msgr2=0x7fc8940b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.865+0000 7fc8b0716700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc8940b2910 0x7fc8940b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc8a819bc30 tx=0x7fc8a400a380 comp rx=0 tx=0).stop 2024-05-09T05:36:05.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.865+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a8108740 msgr2=0x7fc8a819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:05.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.865+0000 7fc8b0716700 1 --2- 172.21.15.115:0/2305812513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a8108740 0x7fc8a819aaf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0007ee0 tx=0x7fc8a000d3b0 comp rx=0 tx=0).stop 2024-05-09T05:36:05.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.865+0000 7fc8aecb3700 1 -- 172.21.15.115:0/2305812513 reap_dead start 2024-05-09T05:36:05.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.865+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 shutdown_connections 2024-05-09T05:36:05.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.865+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 >> 172.21.15.115:0/2305812513 conn(0x7fc8a80fedc0 msgr2=0x7fc8a806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:05.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.865+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 shutdown_connections 2024-05-09T05:36:05.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:05.865+0000 7fc8b0716700 1 -- 172.21.15.115:0/2305812513 wait complete. 2024-05-09T05:36:06.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:06 smithi115 ceph-mon[30349]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:36:06.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:06 smithi195 ceph-mon[33844]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:36:06.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:06 smithi161 ceph-mon[33805]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:36:07.234 INFO:teuthology.orchestra.run.smithi115.stdout:300647710761 2024-05-09T05:36:07.234 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T05:36:08.522 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:08 smithi115 ceph-mon[30349]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T05:36:08.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:08 smithi195 ceph-mon[33844]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T05:36:08.873 INFO:teuthology.orchestra.run.smithi115.stdout:274877906992 2024-05-09T05:36:08.874 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T05:36:08.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:08 smithi161 ceph-mon[33805]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T05:36:10.162 INFO:teuthology.orchestra.run.smithi115.stdout:154618822729 2024-05-09T05:36:10.163 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T05:36:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.922+0000 7f08a297a700 1 -- 172.21.15.115:0/3380444730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c1086c0 msgr2=0x7f089c108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.922+0000 7f08a297a700 1 --2- 172.21.15.115:0/3380444730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c1086c0 0x7f089c108b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0890009990 tx=0x7f0890009ca0 comp rx=0 tx=0).stop 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.922+0000 7f08a297a700 1 -- 172.21.15.115:0/3380444730 shutdown_connections 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.922+0000 7f08a297a700 1 --2- 172.21.15.115:0/3380444730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c1086c0 0x7f089c108b90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.922+0000 7f08a297a700 1 --2- 172.21.15.115:0/3380444730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f089c103270 0x7f089c103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.922+0000 7f08a297a700 1 --2- 172.21.15.115:0/3380444730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f089c103bc0 0x7f089c1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.922+0000 7f08a297a700 1 -- 172.21.15.115:0/3380444730 >> 172.21.15.115:0/3380444730 conn(0x7f089c0fef30 msgr2=0x7f089c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.922+0000 7f08a297a700 1 -- 172.21.15.115:0/3380444730 shutdown_connections 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.923+0000 7f08a297a700 1 -- 172.21.15.115:0/3380444730 wait complete. 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f08a297a700 1 Processor -- start 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f08a297a700 1 -- start start 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f08a297a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f089c103270 0x7f089c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f08a297a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103bc0 0x7f089c1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f08a297a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f089c1086c0 0x7f089c1a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f08a297a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f089c11a910 con 0x7f089c103bc0 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f08a297a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f089c11a790 con 0x7f089c103270 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f08a297a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f089c11aa90 con 0x7f089c1086c0 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f089b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103bc0 0x7f089c1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.924+0000 7f089b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103bc0 0x7f089c1a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35248/0 (socket says 172.21.15.115:35248) 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f089b7fe700 1 -- 172.21.15.115:0/143366014 learned_addr learned my addr 172.21.15.115:0/143366014 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f089bfff700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f089c103270 0x7f089c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f08a0f17700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f089c1086c0 0x7f089c1a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f089b7fe700 1 -- 172.21.15.115:0/143366014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f089c1086c0 msgr2=0x7f089c1a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f089b7fe700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f089c1086c0 0x7f089c1a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f089b7fe700 1 -- 172.21.15.115:0/143366014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f089c103270 msgr2=0x7f089c1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f089b7fe700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f089c103270 0x7f089c1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f089b7fe700 1 -- 172.21.15.115:0/143366014 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c1a6170 con 0x7f089c103bc0 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f089b7fe700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103bc0 0x7f089c1a1840 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f089400ef10 tx=0x7f089400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f08997fa700 1 -- 172.21.15.115:0/143366014 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089400cad0 con 0x7f089c103bc0 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c1a6400 con 0x7f089c103bc0 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f08997fa700 1 -- 172.21.15.115:0/143366014 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0894019070 con 0x7f089c103bc0 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f08997fa700 1 -- 172.21.15.115:0/143366014 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894014480 con 0x7f089c103bc0 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.925+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c1add80 con 0x7f089c103bc0 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.926+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f089c107a90 con 0x7f089c103bc0 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.928+0000 7f08997fa700 1 -- 172.21.15.115:0/143366014 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f0894012050 con 0x7f089c103bc0 2024-05-09T05:36:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.929+0000 7f08997fa700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08840b2910 0x7f08840b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.929+0000 7f089bfff700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08840b2910 0x7f08840b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.929+0000 7f08997fa700 1 -- 172.21.15.115:0/143366014 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f08940e3190 con 0x7f089c103bc0 2024-05-09T05:36:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.929+0000 7f08997fa700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] conn(0x7f08840bdac0 0x7f08840bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.929+0000 7f08997fa700 1 -- 172.21.15.115:0/143366014 --> [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0894008ec0 con 0x7f08840bdac0 2024-05-09T05:36:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.929+0000 7f08997fa700 1 -- 172.21.15.115:0/143366014 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f08940e35e0 con 0x7f089c103bc0 2024-05-09T05:36:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.929+0000 7f089bfff700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08840b2910 0x7f08840b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f088c005950 tx=0x7f088c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.929+0000 7f08a0f17700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] conn(0x7f08840bdac0 0x7f08840bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.930+0000 7f08a0f17700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] conn(0x7f08840bdac0 0x7f08840bffb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.931+0000 7f08997fa700 1 -- 172.21.15.115:0/143366014 <== osd.8 v2:172.21.15.195:6816/818250056 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0894008ec0 con 0x7f08840bdac0 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.948+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 --> [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f089c06c920 con 0x7f08840bdac0 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.949+0000 7f08997fa700 1 -- 172.21.15.115:0/143366014 <== osd.8 v2:172.21.15.195:6816/818250056 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f089c06c920 con 0x7f08840bdac0 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 >> [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] conn(0x7f08840bdac0 msgr2=0x7f08840bffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] conn(0x7f08840bdac0 0x7f08840bffb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08840b2910 msgr2=0x7f08840b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08840b2910 0x7f08840b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f088c005950 tx=0x7f088c0058e0 comp rx=0 tx=0).stop 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103bc0 msgr2=0x7f089c1a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 --2- 172.21.15.115:0/143366014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f089c103bc0 0x7f089c1a1840 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f089400ef10 tx=0x7f089400c3a0 comp rx=0 tx=0).stop 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a0f17700 1 -- 172.21.15.115:0/143366014 reap_dead start 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 shutdown_connections 2024-05-09T05:36:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 >> 172.21.15.115:0/143366014 conn(0x7f089c0fef30 msgr2=0x7f089c106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 shutdown_connections 2024-05-09T05:36:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:07.950+0000 7f08a297a700 1 -- 172.21.15.115:0/143366014 wait complete. 2024-05-09T05:36:10.383 INFO:teuthology.orchestra.run.smithi115.stdout:253403070517 2024-05-09T05:36:10.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T05:36:10.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:10 smithi115 ceph-mon[30349]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T05:36:10.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:10 smithi195 ceph-mon[33844]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T05:36:10.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:10 smithi161 ceph-mon[33805]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T05:36:11.023 INFO:teuthology.orchestra.run.smithi115.stdout:227633266747 2024-05-09T05:36:11.024 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T05:36:11.447 INFO:teuthology.orchestra.run.smithi115.stdout:34359738466 2024-05-09T05:36:11.448 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T05:36:11.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.681+0000 7fc4120d6700 1 -- 172.21.15.115:0/142118925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc40c108740 msgr2=0x7fc40c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:11.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.681+0000 7fc4120d6700 1 --2- 172.21.15.115:0/142118925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc40c108740 0x7fc40c1100a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b0d0 tx=0x7fc40400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:36:11.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.681+0000 7fc4120d6700 1 -- 172.21.15.115:0/142118925 shutdown_connections 2024-05-09T05:36:11.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.681+0000 7fc4120d6700 1 --2- 172.21.15.115:0/142118925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc40c1034c0 0x7fc40c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:11.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.681+0000 7fc4120d6700 1 --2- 172.21.15.115:0/142118925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc40c108740 0x7fc40c1100a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:11.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.681+0000 7fc4120d6700 1 --2- 172.21.15.115:0/142118925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc40c107df0 0x7fc40c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:11.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.681+0000 7fc4120d6700 1 -- 172.21.15.115:0/142118925 >> 172.21.15.115:0/142118925 conn(0x7fc40c0fedc0 msgr2=0x7fc40c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:11.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.682+0000 7fc4120d6700 1 -- 172.21.15.115:0/142118925 shutdown_connections 2024-05-09T05:36:11.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.683+0000 7fc4120d6700 1 -- 172.21.15.115:0/142118925 wait complete. 2024-05-09T05:36:11.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.684+0000 7fc4120d6700 1 Processor -- start 2024-05-09T05:36:11.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.684+0000 7fc4120d6700 1 -- start start 2024-05-09T05:36:11.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.684+0000 7fc4120d6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc40c1034c0 0x7fc40c075390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:11.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.684+0000 7fc4120d6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc40c107df0 0x7fc40c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:11.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.684+0000 7fc4120d6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc40c108740 0x7fc40c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:11.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.684+0000 7fc4120d6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c1121d0 con 0x7fc40c107df0 2024-05-09T05:36:11.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.684+0000 7fc4120d6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c112050 con 0x7fc40c1034c0 2024-05-09T05:36:11.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.684+0000 7fc4120d6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c112350 con 0x7fc40c108740 2024-05-09T05:36:11.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc40c108740 0x7fc40c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:11.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc40c108740 0x7fc40c06eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:39126/0 (socket says 172.21.15.115:39126) 2024-05-09T05:36:11.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40bfff700 1 -- 172.21.15.115:0/1283112787 learned_addr learned my addr 172.21.15.115:0/1283112787 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:11.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc403fff700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc40c107df0 0x7fc40c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40b7fe700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc40c1034c0 0x7fc40c075390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40bfff700 1 -- 172.21.15.115:0/1283112787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc40c1034c0 msgr2=0x7fc40c075390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40bfff700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc40c1034c0 0x7fc40c075390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40bfff700 1 -- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc40c107df0 msgr2=0x7fc40c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40bfff700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc40c107df0 0x7fc40c06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40bfff700 1 -- 172.21.15.115:0/1283112787 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c072ec0 con 0x7fc40c108740 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.685+0000 7fc40bfff700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc40c108740 0x7fc40c06eac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4040079a0 tx=0x7fc4040079d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.686+0000 7fc4097fa700 1 -- 172.21.15.115:0/1283112787 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40401f070 con 0x7fc40c108740 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.686+0000 7fc4097fa700 1 -- 172.21.15.115:0/1283112787 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc404003c00 con 0x7fc40c108740 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.686+0000 7fc4097fa700 1 -- 172.21.15.115:0/1283112787 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40401b430 con 0x7fc40c108740 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.686+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c073150 con 0x7fc40c108740 2024-05-09T05:36:11.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.686+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c0735e0 con 0x7fc40c108740 2024-05-09T05:36:11.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.688+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc40c06f320 con 0x7fc40c108740 2024-05-09T05:36:11.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.690+0000 7fc4097fa700 1 -- 172.21.15.115:0/1283112787 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc404022070 con 0x7fc40c108740 2024-05-09T05:36:11.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.691+0000 7fc4097fa700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc3f80b2a10 0x7fc3f80b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:11.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.692+0000 7fc4097fa700 1 -- 172.21.15.115:0/1283112787 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fc40400f020 con 0x7fc40c108740 2024-05-09T05:36:11.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.692+0000 7fc40b7fe700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc3f80b2a10 0x7fc3f80b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:11.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.692+0000 7fc4097fa700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] conn(0x7fc3f80bdbc0 0x7fc3f80c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:11.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.693+0000 7fc4097fa700 1 -- 172.21.15.115:0/1283112787 --> [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc404022e50 con 0x7fc3f80bdbc0 2024-05-09T05:36:11.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.693+0000 7fc4097fa700 1 -- 172.21.15.115:0/1283112787 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc4040ea050 con 0x7fc40c108740 2024-05-09T05:36:11.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.693+0000 7fc40b7fe700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc3f80b2a10 0x7fc3f80b4f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00f640 tx=0x7fc3fc011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:11.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.693+0000 7fc403fff700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] conn(0x7fc3f80bdbc0 0x7fc3f80c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:11.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.694+0000 7fc403fff700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] conn(0x7fc3f80bdbc0 0x7fc3f80c00b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:11.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.695+0000 7fc4097fa700 1 -- 172.21.15.115:0/1283112787 <== osd.6 v2:172.21.15.115:6818/3657552371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc404022e50 con 0x7fc3f80bdbc0 2024-05-09T05:36:11.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.719+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 --> [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc40c104970 con 0x7fc3f80bdbc0 2024-05-09T05:36:11.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.720+0000 7fc4097fa700 1 -- 172.21.15.115:0/1283112787 <== osd.6 v2:172.21.15.115:6818/3657552371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc40c104970 con 0x7fc3f80bdbc0 2024-05-09T05:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.720+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] conn(0x7fc3f80bdbc0 msgr2=0x7fc3f80c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.720+0000 7fc4120d6700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] conn(0x7fc3f80bdbc0 0x7fc3f80c00b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.721+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc3f80b2a10 msgr2=0x7fc3f80b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.721+0000 7fc4120d6700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc3f80b2a10 0x7fc3f80b4f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00f640 tx=0x7fc3fc011040 comp rx=0 tx=0).stop 2024-05-09T05:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.721+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc40c108740 msgr2=0x7fc40c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.721+0000 7fc4120d6700 1 --2- 172.21.15.115:0/1283112787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc40c108740 0x7fc40c06eac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4040079a0 tx=0x7fc4040079d0 comp rx=0 tx=0).stop 2024-05-09T05:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.721+0000 7fc40bfff700 1 -- 172.21.15.115:0/1283112787 reap_dead start 2024-05-09T05:36:11.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.721+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 shutdown_connections 2024-05-09T05:36:11.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.721+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 >> 172.21.15.115:0/1283112787 conn(0x7fc40c0fedc0 msgr2=0x7fc40c1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:11.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.721+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 shutdown_connections 2024-05-09T05:36:11.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:11.721+0000 7fc4120d6700 1 -- 172.21.15.115:0/1283112787 wait complete. 2024-05-09T05:36:12.583 INFO:teuthology.orchestra.run.smithi115.stdout:180388626502 2024-05-09T05:36:12.583 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T05:36:12.710 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:12 smithi115 ceph-mon[30349]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:36:12.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:12 smithi195 ceph-mon[33844]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:36:12.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:12 smithi161 ceph-mon[33805]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:36:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.094+0000 7ffa4cf46700 1 -- 172.21.15.115:0/1024783782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa48103360 msgr2=0x7ffa48112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.094+0000 7ffa4cf46700 1 --2- 172.21.15.115:0/1024783782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa48103360 0x7ffa48112fe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000b0a0 tx=0x7ffa4000b3b0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.094+0000 7ffa4cf46700 1 -- 172.21.15.115:0/1024783782 shutdown_connections 2024-05-09T05:36:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.094+0000 7ffa4cf46700 1 --2- 172.21.15.115:0/1024783782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa48105210 0x7ffa48102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.094+0000 7ffa4cf46700 1 --2- 172.21.15.115:0/1024783782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa48103360 0x7ffa48112fe0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.094+0000 7ffa4cf46700 1 --2- 172.21.15.115:0/1024783782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa48105620 0x7ffa48102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.094+0000 7ffa4cf46700 1 -- 172.21.15.115:0/1024783782 >> 172.21.15.115:0/1024783782 conn(0x7ffa480fe520 msgr2=0x7ffa48100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.094+0000 7ffa4cf46700 1 -- 172.21.15.115:0/1024783782 shutdown_connections 2024-05-09T05:36:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.094+0000 7ffa4cf46700 1 -- 172.21.15.115:0/1024783782 wait complete. 2024-05-09T05:36:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.095+0000 7ffa4cf46700 1 Processor -- start 2024-05-09T05:36:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.095+0000 7ffa4cf46700 1 -- start start 2024-05-09T05:36:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa4cf46700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa48103360 0x7ffa4819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa4cf46700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa48105210 0x7ffa4819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa4cf46700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa48105620 0x7ffa481a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa46d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa48105620 0x7ffa481a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa4cf46700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffa48116670 con 0x7ffa48105620 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa4cf46700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481164f0 con 0x7ffa48105210 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa4cf46700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481167f0 con 0x7ffa48103360 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa46d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa48105620 0x7ffa481a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38126/0 (socket says 172.21.15.115:38126) 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa46d9d700 1 -- 172.21.15.115:0/2195143496 learned_addr learned my addr 172.21.15.115:0/2195143496 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa45d9b700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa48105210 0x7ffa4819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa4659c700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa48103360 0x7ffa4819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa46d9d700 1 -- 172.21.15.115:0/2195143496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa48103360 msgr2=0x7ffa4819d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa46d9d700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa48103360 0x7ffa4819d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa46d9d700 1 -- 172.21.15.115:0/2195143496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa48105210 msgr2=0x7ffa4819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa46d9d700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa48105210 0x7ffa4819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.096+0000 7ffa46d9d700 1 -- 172.21.15.115:0/2195143496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa481a1f00 con 0x7ffa48105620 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.097+0000 7ffa46d9d700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa48105620 0x7ffa481a1860 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffa40004dc0 tx=0x7ffa40004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.097+0000 7ffa377fe700 1 -- 172.21.15.115:0/2195143496 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40021070 con 0x7ffa48105620 2024-05-09T05:36:13.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.097+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa481a2190 con 0x7ffa48105620 2024-05-09T05:36:13.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.097+0000 7ffa377fe700 1 -- 172.21.15.115:0/2195143496 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa40007b80 con 0x7ffa48105620 2024-05-09T05:36:13.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.097+0000 7ffa377fe700 1 -- 172.21.15.115:0/2195143496 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400052a0 con 0x7ffa48105620 2024-05-09T05:36:13.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.097+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa481a9b10 con 0x7ffa48105620 2024-05-09T05:36:13.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.098+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa48063f50 con 0x7ffa48105620 2024-05-09T05:36:13.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.099+0000 7ffa377fe700 1 -- 172.21.15.115:0/2195143496 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7ffa40024070 con 0x7ffa48105620 2024-05-09T05:36:13.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.100+0000 7ffa377fe700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ffa300b2990 0x7ffa300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.101+0000 7ffa4659c700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ffa300b2990 0x7ffa300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.101+0000 7ffa377fe700 1 -- 172.21.15.115:0/2195143496 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ffa400e3db0 con 0x7ffa48105620 2024-05-09T05:36:13.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.101+0000 7ffa377fe700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] conn(0x7ffa300bdb40 0x7ffa300c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.101+0000 7ffa377fe700 1 -- 172.21.15.115:0/2195143496 --> [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa400075c0 con 0x7ffa300bdb40 2024-05-09T05:36:13.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.101+0000 7ffa377fe700 1 -- 172.21.15.115:0/2195143496 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ffa400e4160 con 0x7ffa48105620 2024-05-09T05:36:13.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.101+0000 7ffa4659c700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ffa300b2990 0x7ffa300b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa4806c000 tx=0x7ffa38009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:13.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.101+0000 7ffa45d9b700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] conn(0x7ffa300bdb40 0x7ffa300c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.101+0000 7ffa45d9b700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] conn(0x7ffa300bdb40 0x7ffa300c0030 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:13.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.102+0000 7ffa377fe700 1 -- 172.21.15.115:0/2195143496 <== osd.4 v2:172.21.15.161:6808/3719283900 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa400075c0 con 0x7ffa300bdb40 2024-05-09T05:36:13.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.119+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 --> [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa480643d0 con 0x7ffa300bdb40 2024-05-09T05:36:13.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa377fe700 1 -- 172.21.15.115:0/2195143496 <== osd.4 v2:172.21.15.161:6808/3719283900 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa480643d0 con 0x7ffa300bdb40 2024-05-09T05:36:13.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 >> [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] conn(0x7ffa300bdb40 msgr2=0x7ffa300c0030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] conn(0x7ffa300bdb40 0x7ffa300c0030 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ffa300b2990 msgr2=0x7ffa300b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ffa300b2990 0x7ffa300b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa4806c000 tx=0x7ffa38009190 comp rx=0 tx=0).stop 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa48105620 msgr2=0x7ffa481a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 --2- 172.21.15.115:0/2195143496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa48105620 0x7ffa481a1860 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffa40004dc0 tx=0x7ffa40004df0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa46d9d700 1 -- 172.21.15.115:0/2195143496 reap_dead start 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 shutdown_connections 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 >> 172.21.15.115:0/2195143496 conn(0x7ffa480fe520 msgr2=0x7ffa480ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 shutdown_connections 2024-05-09T05:36:13.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.120+0000 7ffa4cf46700 1 -- 172.21.15.115:0/2195143496 wait complete. 2024-05-09T05:36:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.328+0000 7f1e4c645700 1 -- 172.21.15.115:0/989414485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e44107df0 msgr2=0x7f1e44108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.328+0000 7f1e4c645700 1 --2- 172.21.15.115:0/989414485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e44107df0 0x7f1e44108200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e340099c0 tx=0x7f1e34009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.328+0000 7f1e4c645700 1 -- 172.21.15.115:0/989414485 shutdown_connections 2024-05-09T05:36:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.328+0000 7f1e4c645700 1 --2- 172.21.15.115:0/989414485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e44108740 0x7f1e441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.328+0000 7f1e4c645700 1 --2- 172.21.15.115:0/989414485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e441034c0 0x7f1e441038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.328+0000 7f1e4c645700 1 --2- 172.21.15.115:0/989414485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e44107df0 0x7f1e44108200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.328+0000 7f1e4c645700 1 -- 172.21.15.115:0/989414485 >> 172.21.15.115:0/989414485 conn(0x7f1e440fedf0 msgr2=0x7f1e44101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.328+0000 7f1e4c645700 1 -- 172.21.15.115:0/989414485 shutdown_connections 2024-05-09T05:36:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.328+0000 7f1e4c645700 1 -- 172.21.15.115:0/989414485 wait complete. 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4c645700 1 Processor -- start 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4c645700 1 -- start start 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4c645700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e441034c0 0x7f1e4419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4c645700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e44107df0 0x7f1e4419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4c645700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e44108740 0x7f1e4419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4c645700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44113200 con 0x7f1e44107df0 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4c645700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44113080 con 0x7f1e441034c0 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4c645700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44113380 con 0x7f1e44108740 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e49be0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e44107df0 0x7f1e4419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e49be0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e44107df0 0x7f1e4419c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38130/0 (socket says 172.21.15.115:38130) 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e49be0700 1 -- 172.21.15.115:0/3030527491 learned_addr learned my addr 172.21.15.115:0/3030527491 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4a3e1700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e441034c0 0x7f1e4419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e4abe2700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e44108740 0x7f1e4419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e49be0700 1 -- 172.21.15.115:0/3030527491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e44108740 msgr2=0x7f1e4419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e49be0700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e44108740 0x7f1e4419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e49be0700 1 -- 172.21.15.115:0/3030527491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e441034c0 msgr2=0x7f1e4419bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e49be0700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e441034c0 0x7f1e4419bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.329+0000 7f1e49be0700 1 -- 172.21.15.115:0/3030527491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e441a6f30 con 0x7f1e44107df0 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.330+0000 7f1e49be0700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e44107df0 0x7f1e4419c100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400b580 tx=0x7f1e34004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.330+0000 7f1e3b7fe700 1 -- 172.21.15.115:0/3030527491 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34027070 con 0x7f1e44107df0 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.330+0000 7f1e3b7fe700 1 -- 172.21.15.115:0/3030527491 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e3400fc00 con 0x7f1e44107df0 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.330+0000 7f1e3b7fe700 1 -- 172.21.15.115:0/3030527491 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34022cb0 con 0x7f1e44107df0 2024-05-09T05:36:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.330+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e441a71c0 con 0x7f1e44107df0 2024-05-09T05:36:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.330+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e441a75c0 con 0x7f1e44107df0 2024-05-09T05:36:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.331+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e44063e90 con 0x7f1e44107df0 2024-05-09T05:36:13.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.333+0000 7f1e3b7fe700 1 -- 172.21.15.115:0/3030527491 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f1e3402a070 con 0x7f1e44107df0 2024-05-09T05:36:13.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.333+0000 7f1e3b7fe700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e300b2990 0x7f1e300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.334+0000 7f1e4a3e1700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e300b2990 0x7f1e300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.334+0000 7f1e3b7fe700 1 -- 172.21.15.115:0/3030527491 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f1e340e3e50 con 0x7f1e44107df0 2024-05-09T05:36:13.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.334+0000 7f1e3b7fe700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] conn(0x7f1e300bda10 0x7f1e300bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:13.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.334+0000 7f1e3b7fe700 1 -- 172.21.15.115:0/3030527491 --> [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e300c05d0 con 0x7f1e300bda10 2024-05-09T05:36:13.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.334+0000 7f1e3b7fe700 1 -- 172.21.15.115:0/3030527491 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f1e340e42a0 con 0x7f1e44107df0 2024-05-09T05:36:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.334+0000 7f1e4a3e1700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e300b2990 0x7f1e300b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005950 tx=0x7f1e4000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.334+0000 7f1e4abe2700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] conn(0x7f1e300bda10 0x7f1e300bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.334+0000 7f1e4abe2700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] conn(0x7f1e300bda10 0x7f1e300bff00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.335+0000 7f1e3b7fe700 1 -- 172.21.15.115:0/3030527491 <== osd.1 v2:172.21.15.161:6800/3600915673 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e300c05d0 con 0x7f1e300bda10 2024-05-09T05:36:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.351+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 --> [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e44064360 con 0x7f1e300bda10 2024-05-09T05:36:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.352+0000 7f1e3b7fe700 1 -- 172.21.15.115:0/3030527491 <== osd.1 v2:172.21.15.161:6800/3600915673 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1e44064360 con 0x7f1e300bda10 2024-05-09T05:36:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.352+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 >> [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] conn(0x7f1e300bda10 msgr2=0x7f1e300bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.352+0000 7f1e4c645700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] conn(0x7f1e300bda10 0x7f1e300bff00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.353+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e300b2990 msgr2=0x7f1e300b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.353+0000 7f1e4c645700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e300b2990 0x7f1e300b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005950 tx=0x7f1e4000b410 comp rx=0 tx=0).stop 2024-05-09T05:36:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.353+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e44107df0 msgr2=0x7f1e4419c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.353+0000 7f1e4c645700 1 --2- 172.21.15.115:0/3030527491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e44107df0 0x7f1e4419c100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400b580 tx=0x7f1e34004790 comp rx=0 tx=0).stop 2024-05-09T05:36:13.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.353+0000 7f1e4abe2700 1 -- 172.21.15.115:0/3030527491 reap_dead start 2024-05-09T05:36:13.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.353+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 shutdown_connections 2024-05-09T05:36:13.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.353+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 >> 172.21.15.115:0/3030527491 conn(0x7f1e440fedf0 msgr2=0x7f1e44101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:13.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.353+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 shutdown_connections 2024-05-09T05:36:13.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:13.353+0000 7f1e4c645700 1 -- 172.21.15.115:0/3030527491 wait complete. 2024-05-09T05:36:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:13.782 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:14.227 INFO:teuthology.orchestra.run.smithi115.stdout:133143986254 2024-05-09T05:36:14.228 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T05:36:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:14.545 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:14 smithi115 ceph-mon[30349]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-09T05:36:14.705 INFO:teuthology.orchestra.run.smithi115.stdout:55834574941 2024-05-09T05:36:14.705 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T05:36:14.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:14 smithi195 ceph-mon[33844]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-09T05:36:14.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:14 smithi161 ceph-mon[33805]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-09T05:36:16.712 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:16 smithi161 ceph-mon[33805]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:36:16.787 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:16 smithi115 ceph-mon[30349]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:36:16.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:16 smithi195 ceph-mon[33844]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:36:18.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:18 smithi115 ceph-mon[30349]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:18.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:18 smithi195 ceph-mon[33844]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:18.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:18 smithi161 ceph-mon[33805]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:19.119 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:19.120 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:19.120 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:19.865 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:20.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.763+0000 7fb70c852700 1 -- 172.21.15.115:0/877586321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb708103130 msgr2=0x7fb708111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:20.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.763+0000 7fb70c852700 1 --2- 172.21.15.115:0/877586321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb708103130 0x7fb708111120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009990 tx=0x7fb6fc009ca0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.763+0000 7fb70c852700 1 -- 172.21.15.115:0/877586321 shutdown_connections 2024-05-09T05:36:20.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.763+0000 7fb70c852700 1 --2- 172.21.15.115:0/877586321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb70810a3e0 0x7fb70810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.763+0000 7fb70c852700 1 --2- 172.21.15.115:0/877586321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb708103130 0x7fb708111120 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.763+0000 7fb70c852700 1 --2- 172.21.15.115:0/877586321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7081027e0 0x7fb708102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.763+0000 7fb70c852700 1 -- 172.21.15.115:0/877586321 >> 172.21.15.115:0/877586321 conn(0x7fb7080fe6f0 msgr2=0x7fb708100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:20.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.764+0000 7fb70c852700 1 -- 172.21.15.115:0/877586321 shutdown_connections 2024-05-09T05:36:20.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.764+0000 7fb70c852700 1 -- 172.21.15.115:0/877586321 wait complete. 2024-05-09T05:36:20.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.764+0000 7fb70c852700 1 Processor -- start 2024-05-09T05:36:20.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.764+0000 7fb70c852700 1 -- start start 2024-05-09T05:36:20.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70c852700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7081027e0 0x7fb7081039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:20.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70c852700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb708103130 0x7fb708103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:20.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70c852700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb70810a3e0 0x7fb708104440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:20.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70c852700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb708114330 con 0x7fb7081027e0 2024-05-09T05:36:20.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70c852700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081141b0 con 0x7fb708103130 2024-05-09T05:36:20.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70c852700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081144b0 con 0x7fb70810a3e0 2024-05-09T05:36:20.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7081027e0 0x7fb7081039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:20.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7081027e0 0x7fb7081039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38138/0 (socket says 172.21.15.115:38138) 2024-05-09T05:36:20.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70659c700 1 -- 172.21.15.115:0/643229171 learned_addr learned my addr 172.21.15.115:0/643229171 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:20.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb705d9b700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb708103130 0x7fb708103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:20.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb706d9d700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb70810a3e0 0x7fb708104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:20.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70659c700 1 -- 172.21.15.115:0/643229171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb70810a3e0 msgr2=0x7fb708104440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:20.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70659c700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb70810a3e0 0x7fb708104440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.013 INFO:teuthology.orchestra.run.smithi115.stdout:274877906994 2024-05-09T05:36:20.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70659c700 1 -- 172.21.15.115:0/643229171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb708103130 msgr2=0x7fb708103f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70659c700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb708103130 0x7fb708103f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70659c700 1 -- 172.21.15.115:0/643229171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7081a9060 con 0x7fb7081027e0 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb706d9d700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb70810a3e0 0x7fb708104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.765+0000 7fb70659c700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7081027e0 0x7fb7081039c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800e9a0 tx=0x7fb6f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.766+0000 7fb6f77fe700 1 -- 172.21.15.115:0/643229171 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f800cbd0 con 0x7fb7081027e0 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.766+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081a9350 con 0x7fb7081027e0 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.766+0000 7fb6f77fe700 1 -- 172.21.15.115:0/643229171 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6f800cd70 con 0x7fb7081027e0 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.766+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a98e0 con 0x7fb7081027e0 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.766+0000 7fb6f77fe700 1 -- 172.21.15.115:0/643229171 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f800cbd0 con 0x7fb7081027e0 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.767+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7080512c0 con 0x7fb7081027e0 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.769+0000 7fb6f77fe700 1 -- 172.21.15.115:0/643229171 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb6f8004020 con 0x7fb7081027e0 2024-05-09T05:36:20.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.769+0000 7fb6f77fe700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb6f00b2b10 0x7fb6f00b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.770+0000 7fb6f77fe700 1 -- 172.21.15.115:0/643229171 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fb6f80e2570 con 0x7fb7081027e0 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.770+0000 7fb705d9b700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb6f00b2b10 0x7fb6f00b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.770+0000 7fb705d9b700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb6f00b2b10 0x7fb6f00b5020 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb708100370 tx=0x7fb70000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:19.774+0000 7fb6f77fe700 1 -- 172.21.15.115:0/643229171 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb6f80a9dd0 con 0x7fb7081027e0 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.005+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb708104ef0 con 0x7fb7081027e0 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.006+0000 7fb6f77fe700 1 -- 172.21.15.115:0/643229171 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb6f80a96d0 con 0x7fb7081027e0 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb6f00b2b10 msgr2=0x7fb6f00b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb6f00b2b10 0x7fb6f00b5020 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb708100370 tx=0x7fb70000b040 comp rx=0 tx=0).stop 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7081027e0 msgr2=0x7fb7081039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7081027e0 0x7fb7081039c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800e9a0 tx=0x7fb6f800ecb0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 shutdown_connections 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb6f00b2b10 0x7fb6f00b5020 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb70810a3e0 0x7fb708104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7081027e0 0x7fb7081039c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 --2- 172.21.15.115:0/643229171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb708103130 0x7fb708103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:20.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 >> 172.21.15.115:0/643229171 conn(0x7fb7080fe6f0 msgr2=0x7fb708100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:20.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 shutdown_connections 2024-05-09T05:36:20.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:20.008+0000 7fb70c852700 1 -- 172.21.15.115:0/643229171 wait complete. 2024-05-09T05:36:20.616 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:20 smithi115 ceph-mon[30349]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T05:36:20.616 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:20 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/643229171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T05:36:20.644 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906992 got 274877906994 for osd.10 2024-05-09T05:36:20.644 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:20.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:20 smithi195 ceph-mon[33844]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T05:36:20.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:20 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/643229171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T05:36:20.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:20 smithi161 ceph-mon[33805]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-09T05:36:20.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:20 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/643229171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.147+0000 7f0486700700 1 -- 172.21.15.115:0/2041066461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 msgr2=0x7f048010f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.147+0000 7f0486700700 1 --2- 172.21.15.115:0/2041066461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 0x7f048010f890 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f047800b0a0 tx=0x7f047800b3b0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.147+0000 7f0486700700 1 -- 172.21.15.115:0/2041066461 shutdown_connections 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.147+0000 7f0486700700 1 --2- 172.21.15.115:0/2041066461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 0x7f048010f890 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.147+0000 7f0486700700 1 --2- 172.21.15.115:0/2041066461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04801089f0 0x7f0480108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.147+0000 7f0486700700 1 --2- 172.21.15.115:0/2041066461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04801029c0 0x7f0480102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.147+0000 7f0486700700 1 -- 172.21.15.115:0/2041066461 >> 172.21.15.115:0/2041066461 conn(0x7f04800fe520 msgr2=0x7f0480100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.147+0000 7f0486700700 1 -- 172.21.15.115:0/2041066461 shutdown_connections 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.147+0000 7f0486700700 1 -- 172.21.15.115:0/2041066461 wait complete. 2024-05-09T05:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f0486700700 1 Processor -- start 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f0486700700 1 -- start start 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f0486700700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04801029c0 0x7f048019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f0486700700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 0x7f048019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f0486700700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04801089f0 0x7f048019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f0486700700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0480112a30 con 0x7f0480103310 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f0486700700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f04801128b0 con 0x7f04801029c0 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f0486700700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0480112bb0 con 0x7f04801089f0 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f047f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 0x7f048019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f047f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 0x7f048019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38160/0 (socket says 172.21.15.115:38160) 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f047f7fe700 1 -- 172.21.15.115:0/707226505 learned_addr learned my addr 172.21.15.115:0/707226505 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f047ffff700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04801029c0 0x7f048019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.148+0000 7f0484c9d700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04801089f0 0x7f048019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f047f7fe700 1 -- 172.21.15.115:0/707226505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04801089f0 msgr2=0x7f048019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f047f7fe700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04801089f0 0x7f048019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f047f7fe700 1 -- 172.21.15.115:0/707226505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04801029c0 msgr2=0x7f048019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f047f7fe700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04801029c0 0x7f048019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.381 INFO:teuthology.orchestra.run.smithi115.stdout:201863462978 2024-05-09T05:36:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f047f7fe700 1 -- 172.21.15.115:0/707226505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048019fc00 con 0x7f0480103310 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f047f7fe700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 0x7f048019b2d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f047400d980 tx=0x7f047400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f047d7fa700 1 -- 172.21.15.115:0/707226505 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474017070 con 0x7f0480103310 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f047d7fa700 1 -- 172.21.15.115:0/707226505 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f047400fd20 con 0x7f0480103310 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048019fe90 con 0x7f0480103310 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.149+0000 7f047d7fa700 1 -- 172.21.15.115:0/707226505 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047400e3b0 con 0x7f0480103310 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.150+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04800fe930 con 0x7f0480103310 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.151+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0480104100 con 0x7f0480103310 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.152+0000 7f047d7fa700 1 -- 172.21.15.115:0/707226505 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f047400f890 con 0x7f0480103310 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.153+0000 7f047d7fa700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f04680b2910 0x7f04680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.153+0000 7f047ffff700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f04680b2910 0x7f04680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.153+0000 7f047d7fa700 1 -- 172.21.15.115:0/707226505 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f04740e2180 con 0x7f0480103310 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.153+0000 7f047ffff700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f04680b2910 0x7f04680b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0480103b00 tx=0x7f047000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.157+0000 7f047d7fa700 1 -- 172.21.15.115:0/707226505 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04740a99e0 con 0x7f0480103310 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.369+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f048019c090 con 0x7f0480103310 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.369+0000 7f047d7fa700 1 -- 172.21.15.115:0/707226505 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f04740a92e0 con 0x7f0480103310 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f04680b2910 msgr2=0x7f04680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f04680b2910 0x7f04680b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0480103b00 tx=0x7f047000b3c0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 msgr2=0x7f048019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 0x7f048019b2d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f047400d980 tx=0x7f047400dc90 comp rx=0 tx=0).stop 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 shutdown_connections 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f04680b2910 0x7f04680b4e20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04801029c0 0x7f048019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04801089f0 0x7f048019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 --2- 172.21.15.115:0/707226505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0480103310 0x7f048019b2d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 >> 172.21.15.115:0/707226505 conn(0x7f04800fe520 msgr2=0x7f0480109820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 shutdown_connections 2024-05-09T05:36:22.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:22.372+0000 7f0486700700 1 -- 172.21.15.115:0/707226505 wait complete. 2024-05-09T05:36:22.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:22 smithi115 ceph-mon[30349]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-09T05:36:22.650 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:22 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/707226505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T05:36:22.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:22 smithi195 ceph-mon[33844]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-09T05:36:22.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:22 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/707226505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T05:36:22.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:22 smithi161 ceph-mon[33805]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-09T05:36:22.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:22 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/707226505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T05:36:23.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462973 got 201863462978 for osd.7 2024-05-09T05:36:23.041 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:24.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.340+0000 7f3d0395e700 1 -- 172.21.15.115:0/3486449779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc102790 msgr2=0x7f3cfc102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:24.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.340+0000 7f3d0395e700 1 --2- 172.21.15.115:0/3486449779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc102790 0x7f3cfc102ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009990 tx=0x7f3cec009ca0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.340+0000 7f3d0395e700 1 -- 172.21.15.115:0/3486449779 shutdown_connections 2024-05-09T05:36:24.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.340+0000 7f3d0395e700 1 --2- 172.21.15.115:0/3486449779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cfc109300 0x7f3cfc109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.340+0000 7f3d0395e700 1 --2- 172.21.15.115:0/3486449779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cfc1030e0 0x7f3cfc111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.340+0000 7f3d0395e700 1 --2- 172.21.15.115:0/3486449779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc102790 0x7f3cfc102ba0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.340+0000 7f3d0395e700 1 -- 172.21.15.115:0/3486449779 >> 172.21.15.115:0/3486449779 conn(0x7f3cfc0fe520 msgr2=0x7f3cfc100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:24.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.340+0000 7f3d0395e700 1 -- 172.21.15.115:0/3486449779 shutdown_connections 2024-05-09T05:36:24.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.340+0000 7f3d0395e700 1 -- 172.21.15.115:0/3486449779 wait complete. 2024-05-09T05:36:24.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d0395e700 1 Processor -- start 2024-05-09T05:36:24.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d0395e700 1 -- start start 2024-05-09T05:36:24.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d0395e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cfc102790 0x7f3cfc19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:24.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d0395e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cfc1030e0 0x7f3cfc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:24.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d0395e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc109300 0x7f3cfc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:24.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d0395e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc114450 con 0x7f3cfc109300 2024-05-09T05:36:24.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d0395e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1142d0 con 0x7f3cfc1030e0 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d0395e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1145d0 con 0x7f3cfc102790 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d01efb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc109300 0x7f3cfc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d01efb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc109300 0x7f3cfc19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45904/0 (socket says 172.21.15.115:45904) 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d016fa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cfc102790 0x7f3cfc19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.341+0000 7f3d01efb700 1 -- 172.21.15.115:0/1048857294 learned_addr learned my addr 172.21.15.115:0/1048857294 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d00ef9700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cfc1030e0 0x7f3cfc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d01efb700 1 -- 172.21.15.115:0/1048857294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cfc102790 msgr2=0x7f3cfc19aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d01efb700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cfc102790 0x7f3cfc19aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d01efb700 1 -- 172.21.15.115:0/1048857294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cfc1030e0 msgr2=0x7f3cfc19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d01efb700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cfc1030e0 0x7f3cfc19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d01efb700 1 -- 172.21.15.115:0/1048857294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc19fd70 con 0x7f3cfc109300 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d00ef9700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cfc1030e0 0x7f3cfc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d01efb700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc109300 0x7f3cfc19f6d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4007ee0 tx=0x7f3cf400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:24.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3cf27fc700 1 -- 172.21.15.115:0/1048857294 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400dbd0 con 0x7f3cfc109300 2024-05-09T05:36:24.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3cf27fc700 1 -- 172.21.15.115:0/1048857294 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cf400dd70 con 0x7f3cfc109300 2024-05-09T05:36:24.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cfc1a0060 con 0x7f3cfc109300 2024-05-09T05:36:24.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3cf27fc700 1 -- 172.21.15.115:0/1048857294 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40058d0 con 0x7f3cfc109300 2024-05-09T05:36:24.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.342+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cfc1a7980 con 0x7f3cfc109300 2024-05-09T05:36:24.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.343+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cfc0512c0 con 0x7f3cfc109300 2024-05-09T05:36:24.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.345+0000 7f3cf27fc700 1 -- 172.21.15.115:0/1048857294 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f3cf401a020 con 0x7f3cfc109300 2024-05-09T05:36:24.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.346+0000 7f3cf27fc700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3ce80b2990 0x7f3ce80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:24.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.346+0000 7f3d016fa700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3ce80b2990 0x7f3ce80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:24.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.346+0000 7f3cf27fc700 1 -- 172.21.15.115:0/1048857294 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f3cf40e1e60 con 0x7f3cfc109300 2024-05-09T05:36:24.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.346+0000 7f3d016fa700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3ce80b2990 0x7f3ce80b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc06ca70 tx=0x7f3cf800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:24.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.351+0000 7f3cf27fc700 1 -- 172.21.15.115:0/1048857294 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3cf4010070 con 0x7f3cfc109300 2024-05-09T05:36:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.566+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3cfc19c200 con 0x7f3cfc109300 2024-05-09T05:36:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.567+0000 7f3cf27fc700 1 -- 172.21.15.115:0/1048857294 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3cf40a96c0 con 0x7f3cfc109300 2024-05-09T05:36:24.560 INFO:teuthology.orchestra.run.smithi115.stdout:77309411419 2024-05-09T05:36:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3ce80b2990 msgr2=0x7f3ce80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3ce80b2990 0x7f3ce80b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc06ca70 tx=0x7f3cf800a380 comp rx=0 tx=0).stop 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc109300 msgr2=0x7f3cfc19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc109300 0x7f3cfc19f6d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4007ee0 tx=0x7f3cf400d3b0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 shutdown_connections 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f3ce80b2990 0x7f3ce80b4ea0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cfc109300 0x7f3cfc19f6d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3cfc102790 0x7f3cfc19aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 --2- 172.21.15.115:0/1048857294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cfc1030e0 0x7f3cfc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 >> 172.21.15.115:0/1048857294 conn(0x7f3cfc0fe520 msgr2=0x7f3cfc107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 shutdown_connections 2024-05-09T05:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:24.569+0000 7f3d0395e700 1 -- 172.21.15.115:0/1048857294 wait complete. 2024-05-09T05:36:24.774 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:24 smithi115 ceph-mon[30349]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:24.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:24 smithi195 ceph-mon[33844]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:24.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:24 smithi161 ceph-mon[33805]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:25.487 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411414 got 77309411419 for osd.2 2024-05-09T05:36:25.487 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:25.746 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:25 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1048857294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T05:36:25.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:25 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1048857294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T05:36:25.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1048857294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T05:36:26.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:26 smithi115 ceph-mon[30349]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-05-09T05:36:26.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:26 smithi195 ceph-mon[33844]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-05-09T05:36:26.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:26 smithi161 ceph-mon[33805]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-05-09T05:36:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:28 smithi115 ceph-mon[30349]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:28 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3263288432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T05:36:28.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:28 smithi195 ceph-mon[33844]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:28.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:28 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3263288432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T05:36:28.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:28 smithi161 ceph-mon[33805]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:28.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:28 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3263288432' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T05:36:29.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.823+0000 7fd677ce7700 1 -- 172.21.15.115:0/4091753254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 msgr2=0x7fd6701100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.823+0000 7fd677ce7700 1 --2- 172.21.15.115:0/4091753254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 0x7fd6701100e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b0d0 tx=0x7fd66800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.823+0000 7fd677ce7700 1 -- 172.21.15.115:0/4091753254 shutdown_connections 2024-05-09T05:36:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.823+0000 7fd677ce7700 1 --2- 172.21.15.115:0/4091753254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd670103550 0x7fd670103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.823+0000 7fd677ce7700 1 --2- 172.21.15.115:0/4091753254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 0x7fd6701100e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.823+0000 7fd677ce7700 1 --2- 172.21.15.115:0/4091753254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd670107e30 0x7fd670108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.823+0000 7fd677ce7700 1 -- 172.21.15.115:0/4091753254 >> 172.21.15.115:0/4091753254 conn(0x7fd670078e00 msgr2=0x7fd670079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.823+0000 7fd677ce7700 1 -- 172.21.15.115:0/4091753254 shutdown_connections 2024-05-09T05:36:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.823+0000 7fd677ce7700 1 -- 172.21.15.115:0/4091753254 wait complete. 2024-05-09T05:36:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd677ce7700 1 Processor -- start 2024-05-09T05:36:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd677ce7700 1 -- start start 2024-05-09T05:36:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd677ce7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd670103550 0x7fd67019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd677ce7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd670107e30 0x7fd67019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd677ce7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 0x7fd670199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:29.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd677ce7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd670113260 con 0x7fd670108780 2024-05-09T05:36:29.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd677ce7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701130e0 con 0x7fd670103550 2024-05-09T05:36:29.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd677ce7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701133e0 con 0x7fd670107e30 2024-05-09T05:36:29.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd676284700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 0x7fd670199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:29.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd676284700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 0x7fd670199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45924/0 (socket says 172.21.15.115:45924) 2024-05-09T05:36:29.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd676284700 1 -- 172.21.15.115:0/3263288432 learned_addr learned my addr 172.21.15.115:0/3263288432 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:29.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd675a83700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd670103550 0x7fd67019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:29.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd675282700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd670107e30 0x7fd67019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:29.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd676284700 1 -- 172.21.15.115:0/3263288432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd670107e30 msgr2=0x7fd67019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd676284700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd670107e30 0x7fd67019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd676284700 1 -- 172.21.15.115:0/3263288432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd670103550 msgr2=0x7fd67019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd676284700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd670103550 0x7fd67019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd676284700 1 -- 172.21.15.115:0/3263288432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67019a0e0 con 0x7fd670108780 2024-05-09T05:36:29.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd675282700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd670107e30 0x7fd67019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:36:29.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.824+0000 7fd675a83700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd670103550 0x7fd67019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:36:29.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.825+0000 7fd676284700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 0x7fd670199aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd668007d90 tx=0x7fd668007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:29.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.825+0000 7fd666ffd700 1 -- 172.21.15.115:0/3263288432 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66801f030 con 0x7fd670108780 2024-05-09T05:36:29.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.825+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67019a370 con 0x7fd670108780 2024-05-09T05:36:29.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.825+0000 7fd666ffd700 1 -- 172.21.15.115:0/3263288432 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd66801bb90 con 0x7fd670108780 2024-05-09T05:36:29.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.825+0000 7fd666ffd700 1 -- 172.21.15.115:0/3263288432 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668012c90 con 0x7fd670108780 2024-05-09T05:36:29.141 INFO:teuthology.orchestra.run.smithi115.stdout:180388626505 2024-05-09T05:36:29.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.825+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67019a8b0 con 0x7fd670108780 2024-05-09T05:36:29.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.826+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6700512c0 con 0x7fd670108780 2024-05-09T05:36:29.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.827+0000 7fd666ffd700 1 -- 172.21.15.115:0/3263288432 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd668022030 con 0x7fd670108780 2024-05-09T05:36:29.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.828+0000 7fd666ffd700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd65c0b2910 0x7fd65c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:29.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.829+0000 7fd675a83700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd65c0b2910 0x7fd65c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:29.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.829+0000 7fd666ffd700 1 -- 172.21.15.115:0/3263288432 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd66800f020 con 0x7fd670108780 2024-05-09T05:36:29.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.829+0000 7fd675a83700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd65c0b2910 0x7fd65c0b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd660005950 tx=0x7fd660005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:29.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:27.833+0000 7fd666ffd700 1 -- 172.21.15.115:0/3263288432 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd668029020 con 0x7fd670108780 2024-05-09T05:36:29.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.060+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd67002e5f0 con 0x7fd670108780 2024-05-09T05:36:29.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.060+0000 7fd666ffd700 1 -- 172.21.15.115:0/3263288432 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd6680ab3b0 con 0x7fd670108780 2024-05-09T05:36:29.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd65c0b2910 msgr2=0x7fd65c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd65c0b2910 0x7fd65c0b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd660005950 tx=0x7fd660005f50 comp rx=0 tx=0).stop 2024-05-09T05:36:29.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 msgr2=0x7fd670199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 0x7fd670199aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd668007d90 tx=0x7fd668007e70 comp rx=0 tx=0).stop 2024-05-09T05:36:29.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 shutdown_connections 2024-05-09T05:36:29.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd65c0b2910 0x7fd65c0b4e20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd670103550 0x7fd67019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd670108780 0x7fd670199aa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 --2- 172.21.15.115:0/3263288432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd670107e30 0x7fd67019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.063+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 >> 172.21.15.115:0/3263288432 conn(0x7fd670078e00 msgr2=0x7fd670100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:29.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.064+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 shutdown_connections 2024-05-09T05:36:29.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.064+0000 7fd677ce7700 1 -- 172.21.15.115:0/3263288432 wait complete. 2024-05-09T05:36:29.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.747+0000 7f08bd107700 1 -- 172.21.15.115:0/3731447008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b81034e0 msgr2=0x7f08b81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.747+0000 7f08bd107700 1 --2- 172.21.15.115:0/3731447008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b81034e0 0x7f08b81038f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f08a80099c0 tx=0x7f08a8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.747+0000 7f08bd107700 1 -- 172.21.15.115:0/3731447008 shutdown_connections 2024-05-09T05:36:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.747+0000 7f08bd107700 1 --2- 172.21.15.115:0/3731447008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08b8108760 0x7f08b81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.747+0000 7f08bd107700 1 --2- 172.21.15.115:0/3731447008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b81034e0 0x7f08b81038f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.747+0000 7f08bd107700 1 --2- 172.21.15.115:0/3731447008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8107e10 0x7f08b8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.747+0000 7f08bd107700 1 -- 172.21.15.115:0/3731447008 >> 172.21.15.115:0/3731447008 conn(0x7f08b80fef50 msgr2=0x7f08b8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.747+0000 7f08bd107700 1 -- 172.21.15.115:0/3731447008 shutdown_connections 2024-05-09T05:36:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.747+0000 7f08bd107700 1 -- 172.21.15.115:0/3731447008 wait complete. 2024-05-09T05:36:29.741 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:29 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2261265169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T05:36:29.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08bd107700 1 Processor -- start 2024-05-09T05:36:29.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08bd107700 1 -- start start 2024-05-09T05:36:29.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08bd107700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08b81034e0 0x7f08b819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:29.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08bd107700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b8107e10 0x7f08b819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08bd107700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8108760 0x7f08b8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08bd107700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8113240 con 0x7f08b8107e10 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08bd107700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81130c0 con 0x7f08b81034e0 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08bd107700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81133c0 con 0x7f08b8108760 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08b659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b8107e10 0x7f08b819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08b659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b8107e10 0x7f08b819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45948/0 (socket says 172.21.15.115:45948) 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.748+0000 7f08b659c700 1 -- 172.21.15.115:0/3674583837 learned_addr learned my addr 172.21.15.115:0/3674583837 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08b759e700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8108760 0x7f08b8199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08b6d9d700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08b81034e0 0x7f08b819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08b659c700 1 -- 172.21.15.115:0/3674583837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8108760 msgr2=0x7f08b8199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08b659c700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8108760 0x7f08b8199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08b659c700 1 -- 172.21.15.115:0/3674583837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08b81034e0 msgr2=0x7f08b819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08b659c700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08b81034e0 0x7f08b819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08b659c700 1 -- 172.21.15.115:0/3674583837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b819a100 con 0x7f08b8107e10 2024-05-09T05:36:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08b659c700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b8107e10 0x7f08b819fde0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00d980 tx=0x7f08ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f089ffff700 1 -- 172.21.15.115:0/3674583837 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac017070 con 0x7f08b8107e10 2024-05-09T05:36:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f089ffff700 1 -- 172.21.15.115:0/3674583837 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08ac00fd20 con 0x7f08b8107e10 2024-05-09T05:36:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b819a3f0 con 0x7f08b8107e10 2024-05-09T05:36:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f089ffff700 1 -- 172.21.15.115:0/3674583837 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac00e3b0 con 0x7f08b8107e10 2024-05-09T05:36:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.749+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b81a6730 con 0x7f08b8107e10 2024-05-09T05:36:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.751+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b80512c0 con 0x7f08b8107e10 2024-05-09T05:36:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.752+0000 7f089ffff700 1 -- 172.21.15.115:0/3674583837 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f08ac00f890 con 0x7f08b8107e10 2024-05-09T05:36:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.753+0000 7f089ffff700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08a00b2910 0x7f08a00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.753+0000 7f08b6d9d700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08a00b2910 0x7f08a00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.753+0000 7f089ffff700 1 -- 172.21.15.115:0/3674583837 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f08ac0e2190 con 0x7f08b8107e10 2024-05-09T05:36:29.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.754+0000 7f08b6d9d700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08a00b2910 0x7f08a00b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009990 tx=0x7f08a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:29.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.757+0000 7f089ffff700 1 -- 172.21.15.115:0/3674583837 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08ac0a99f0 con 0x7f08b8107e10 2024-05-09T05:36:29.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:29 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2261265169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T05:36:29.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:29 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2261265169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T05:36:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.968+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f08b802e5f0 con 0x7f08b8107e10 2024-05-09T05:36:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.968+0000 7f089ffff700 1 -- 172.21.15.115:0/3674583837 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f08ac0a92f0 con 0x7f08b8107e10 2024-05-09T05:36:29.962 INFO:teuthology.orchestra.run.smithi115.stdout:107374182488 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08a00b2910 msgr2=0x7f08a00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08a00b2910 0x7f08a00b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009990 tx=0x7f08a8005fb0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b8107e10 msgr2=0x7f08b819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b8107e10 0x7f08b819fde0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00d980 tx=0x7f08ac00dc90 comp rx=0 tx=0).stop 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 shutdown_connections 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f08a00b2910 0x7f08a00b4e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08b81034e0 0x7f08b819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b8108760 0x7f08b8199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 --2- 172.21.15.115:0/3674583837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08b8107e10 0x7f08b819fde0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 >> 172.21.15.115:0/3674583837 conn(0x7f08b80fef50 msgr2=0x7f08b8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 shutdown_connections 2024-05-09T05:36:29.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:29.970+0000 7f08bd107700 1 -- 172.21.15.115:0/3674583837 wait complete. 2024-05-09T05:36:30.328 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626502 got 180388626505 for osd.6 2024-05-09T05:36:30.328 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:30.638 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:30 smithi115 ceph-mon[30349]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:36:30.638 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:30 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3674583837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T05:36:30.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:30 smithi195 ceph-mon[33844]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:36:30.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:30 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3674583837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T05:36:30.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:30 smithi161 ceph-mon[33805]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:36:30.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:30 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3674583837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T05:36:30.976 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182481 got 107374182488 for osd.3 2024-05-09T05:36:30.976 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.477+0000 7f98e0b10700 1 -- 172.21.15.115:0/462027979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc103c50 msgr2=0x7f98dc104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.477+0000 7f98e0b10700 1 --2- 172.21.15.115:0/462027979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc103c50 0x7f98dc104060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0099c0 tx=0x7f98cc009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.477+0000 7f98e0b10700 1 -- 172.21.15.115:0/462027979 shutdown_connections 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.477+0000 7f98e0b10700 1 --2- 172.21.15.115:0/462027979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc1087a0 0x7f98dc110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.477+0000 7f98e0b10700 1 --2- 172.21.15.115:0/462027979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc103c50 0x7f98dc104060 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.477+0000 7f98e0b10700 1 --2- 172.21.15.115:0/462027979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98dc107e50 0x7f98dc108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.477+0000 7f98e0b10700 1 -- 172.21.15.115:0/462027979 >> 172.21.15.115:0/462027979 conn(0x7f98dc0fe4e0 msgr2=0x7f98dc100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.477+0000 7f98e0b10700 1 -- 172.21.15.115:0/462027979 shutdown_connections 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.477+0000 7f98e0b10700 1 -- 172.21.15.115:0/462027979 wait complete. 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.478+0000 7f98e0b10700 1 Processor -- start 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.478+0000 7f98e0b10700 1 -- start start 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.478+0000 7f98e0b10700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98dc103c50 0x7f98dc19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.478+0000 7f98e0b10700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc107e50 0x7f98dc19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.478+0000 7f98e0b10700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc1087a0 0x7f98dc199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.478+0000 7f98e0b10700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1132c0 con 0x7f98dc1087a0 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.478+0000 7f98e0b10700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc113140 con 0x7f98dc103c50 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98e0b10700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc113440 con 0x7f98dc107e50 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98dad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc1087a0 0x7f98dc199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98dad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc1087a0 0x7f98dc199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45932/0 (socket says 172.21.15.115:45932) 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98dad9d700 1 -- 172.21.15.115:0/2261265169 learned_addr learned my addr 172.21.15.115:0/2261265169 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98d9d9b700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc107e50 0x7f98dc19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98da59c700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98dc103c50 0x7f98dc19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98dad9d700 1 -- 172.21.15.115:0/2261265169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc107e50 msgr2=0x7f98dc19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98dad9d700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc107e50 0x7f98dc19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98dad9d700 1 -- 172.21.15.115:0/2261265169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98dc103c50 msgr2=0x7f98dc19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98dad9d700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98dc103c50 0x7f98dc19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98dad9d700 1 -- 172.21.15.115:0/2261265169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc19a170 con 0x7f98dc1087a0 2024-05-09T05:36:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98da59c700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98dc103c50 0x7f98dc19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:36:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98dad9d700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc1087a0 0x7f98dc199ac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007ee0 tx=0x7f98d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98cb7fe700 1 -- 172.21.15.115:0/2261265169 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d400dbd0 con 0x7f98dc1087a0 2024-05-09T05:36:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98cb7fe700 1 -- 172.21.15.115:0/2261265169 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98d400dd70 con 0x7f98dc1087a0 2024-05-09T05:36:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc19a460 con 0x7f98dc1087a0 2024-05-09T05:36:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98cb7fe700 1 -- 172.21.15.115:0/2261265169 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40058d0 con 0x7f98dc1087a0 2024-05-09T05:36:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.479+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc1028b0 con 0x7f98dc1087a0 2024-05-09T05:36:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.480+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98dc1049d0 con 0x7f98dc1087a0 2024-05-09T05:36:31.202 INFO:teuthology.orchestra.run.smithi115.stdout:154618822734 2024-05-09T05:36:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.483+0000 7f98cb7fe700 1 -- 172.21.15.115:0/2261265169 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f98d401a020 con 0x7f98dc1087a0 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.484+0000 7f98cb7fe700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f98c40b2910 0x7f98c40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.484+0000 7f98da59c700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f98c40b2910 0x7f98c40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.484+0000 7f98cb7fe700 1 -- 172.21.15.115:0/2261265169 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f98d40e25c0 con 0x7f98dc1087a0 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.484+0000 7f98da59c700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f98c40b2910 0x7f98c40b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009990 tx=0x7f98cc005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.487+0000 7f98cb7fe700 1 -- 172.21.15.115:0/2261265169 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98d4010070 con 0x7f98dc1087a0 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.699+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f98dc19aff0 con 0x7f98dc1087a0 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.700+0000 7f98cb7fe700 1 -- 172.21.15.115:0/2261265169 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f98d40a9e20 con 0x7f98dc1087a0 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f98c40b2910 msgr2=0x7f98c40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f98c40b2910 0x7f98c40b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009990 tx=0x7f98cc005940 comp rx=0 tx=0).stop 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc1087a0 msgr2=0x7f98dc199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc1087a0 0x7f98dc199ac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007ee0 tx=0x7f98d400d3b0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 shutdown_connections 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f98c40b2910 0x7f98c40b4e20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98dc103c50 0x7f98dc19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc1087a0 0x7f98dc199ac0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 --2- 172.21.15.115:0/2261265169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc107e50 0x7f98dc19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 >> 172.21.15.115:0/2261265169 conn(0x7f98dc0fe4e0 msgr2=0x7f98dc1096d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 shutdown_connections 2024-05-09T05:36:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:28.702+0000 7f98e0b10700 1 -- 172.21.15.115:0/2261265169 wait complete. 2024-05-09T05:36:31.964 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822729 got 154618822734 for osd.5 2024-05-09T05:36:31.964 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:32.008 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:32.474 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:32 smithi115 ceph-mon[30349]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:36:32.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:32 smithi195 ceph-mon[33844]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:36:32.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:32 smithi161 ceph-mon[33805]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:36:33.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:33 smithi115 ceph-mon[30349]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:33.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:33 smithi195 ceph-mon[33844]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:33.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:33 smithi161 ceph-mon[33805]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:34 smithi115 ceph-mon[30349]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-09T05:36:34.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.852+0000 7f1e2d474700 1 -- 172.21.15.115:0/767167300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281089f0 msgr2=0x7f1e28108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:34.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.852+0000 7f1e2d474700 1 --2- 172.21.15.115:0/767167300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281089f0 0x7f1e28108e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009990 tx=0x7f1e18009ca0 comp rx=0 tx=0).stop 2024-05-09T05:36:34.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.852+0000 7f1e2d474700 1 -- 172.21.15.115:0/767167300 shutdown_connections 2024-05-09T05:36:34.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.852+0000 7f1e2d474700 1 --2- 172.21.15.115:0/767167300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e28103310 0x7f1e2810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:34.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.852+0000 7f1e2d474700 1 --2- 172.21.15.115:0/767167300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281089f0 0x7f1e28108e00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:34.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.852+0000 7f1e2d474700 1 --2- 172.21.15.115:0/767167300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e281029c0 0x7f1e28102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:34.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.852+0000 7f1e2d474700 1 -- 172.21.15.115:0/767167300 >> 172.21.15.115:0/767167300 conn(0x7f1e280fe6f0 msgr2=0x7f1e28100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:34.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.852+0000 7f1e2d474700 1 -- 172.21.15.115:0/767167300 shutdown_connections 2024-05-09T05:36:34.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.852+0000 7f1e2d474700 1 -- 172.21.15.115:0/767167300 wait complete. 2024-05-09T05:36:34.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.853+0000 7f1e2d474700 1 Processor -- start 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.853+0000 7f1e2d474700 1 -- start start 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e2d474700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281029c0 0x7f1e2819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e2d474700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e28103310 0x7f1e2819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e2d474700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e281089f0 0x7f1e2819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e2d474700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281129b0 con 0x7f1e281029c0 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e2d474700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28112830 con 0x7f1e281089f0 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e2d474700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28112b30 con 0x7f1e28103310 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e26ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281029c0 0x7f1e2819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e26ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281029c0 0x7f1e2819ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45158/0 (socket says 172.21.15.115:45158) 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e26ffd700 1 -- 172.21.15.115:0/152738757 learned_addr learned my addr 172.21.15.115:0/152738757 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e267fc700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e28103310 0x7f1e2819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.854+0000 7f1e277fe700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e281089f0 0x7f1e2819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e26ffd700 1 -- 172.21.15.115:0/152738757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e28103310 msgr2=0x7f1e2819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e26ffd700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e28103310 0x7f1e2819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e26ffd700 1 -- 172.21.15.115:0/152738757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e281089f0 msgr2=0x7f1e2819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e26ffd700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e281089f0 0x7f1e2819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e26ffd700 1 -- 172.21.15.115:0/152738757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2819fc50 con 0x7f1e281029c0 2024-05-09T05:36:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e26ffd700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281029c0 0x7f1e2819ad80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1e18000c00 tx=0x7f1e180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e0ffff700 1 -- 172.21.15.115:0/152738757 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18027070 con 0x7f1e281029c0 2024-05-09T05:36:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2819fee0 con 0x7f1e281029c0 2024-05-09T05:36:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e0ffff700 1 -- 172.21.15.115:0/152738757 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e1800fb00 con 0x7f1e281029c0 2024-05-09T05:36:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e0ffff700 1 -- 172.21.15.115:0/152738757 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18022c40 con 0x7f1e281029c0 2024-05-09T05:36:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.855+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e281a7860 con 0x7f1e281029c0 2024-05-09T05:36:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.856+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e28104100 con 0x7f1e281029c0 2024-05-09T05:36:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.858+0000 7f1e0ffff700 1 -- 172.21.15.115:0/152738757 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1e1802a070 con 0x7f1e281029c0 2024-05-09T05:36:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.858+0000 7f1e0ffff700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e100b2880 0x7f1e100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.858+0000 7f1e267fc700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e100b2880 0x7f1e100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.858+0000 7f1e0ffff700 1 -- 172.21.15.115:0/152738757 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f1e180e3860 con 0x7f1e281029c0 2024-05-09T05:36:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.859+0000 7f1e267fc700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e100b2880 0x7f1e100b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e281015d0 tx=0x7f1e1c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:34.862+0000 7f1e0ffff700 1 -- 172.21.15.115:0/152738757 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e18031080 con 0x7f1e281029c0 2024-05-09T05:36:34.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:34 smithi195 ceph-mon[33844]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-09T05:36:34.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:34 smithi161 ceph-mon[33805]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-09T05:36:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.078+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f1e2819c0e0 con 0x7f1e281029c0 2024-05-09T05:36:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.078+0000 7f1e0ffff700 1 -- 172.21.15.115:0/152738757 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1e18013070 con 0x7f1e281029c0 2024-05-09T05:36:35.074 INFO:teuthology.orchestra.run.smithi115.stdout:227633266752 2024-05-09T05:36:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e100b2880 msgr2=0x7f1e100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e100b2880 0x7f1e100b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e281015d0 tx=0x7f1e1c00b410 comp rx=0 tx=0).stop 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281029c0 msgr2=0x7f1e2819ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281029c0 0x7f1e2819ad80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1e18000c00 tx=0x7f1e180047b0 comp rx=0 tx=0).stop 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 shutdown_connections 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1e100b2880 0x7f1e100b4d90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e281089f0 0x7f1e2819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e281029c0 0x7f1e2819ad80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 --2- 172.21.15.115:0/152738757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e28103310 0x7f1e2819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 >> 172.21.15.115:0/152738757 conn(0x7f1e280fe6f0 msgr2=0x7f1e281066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 shutdown_connections 2024-05-09T05:36:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.081+0000 7f1e2d474700 1 -- 172.21.15.115:0/152738757 wait complete. 2024-05-09T05:36:35.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:35 smithi115 ceph-mon[30349]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-05-09T05:36:35.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:35 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/152738757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T05:36:35.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:35 smithi195 ceph-mon[33844]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-05-09T05:36:35.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:35 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/152738757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T05:36:35.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:35 smithi161 ceph-mon[33805]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-05-09T05:36:35.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:35 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/152738757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T05:36:36.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.936+0000 7f36cee22700 1 -- 172.21.15.115:0/1662973556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 msgr2=0x7f36c81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.936+0000 7f36cee22700 1 --2- 172.21.15.115:0/1662973556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 0x7f36c81100a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f36c000b0d0 tx=0x7f36c000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.936+0000 7f36cee22700 1 -- 172.21.15.115:0/1662973556 shutdown_connections 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.936+0000 7f36cee22700 1 --2- 172.21.15.115:0/1662973556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 0x7f36c81100a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.936+0000 7f36cee22700 1 --2- 172.21.15.115:0/1662973556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c81034c0 0x7f36c81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.936+0000 7f36cee22700 1 --2- 172.21.15.115:0/1662973556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c8107df0 0x7f36c8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.936+0000 7f36cee22700 1 -- 172.21.15.115:0/1662973556 >> 172.21.15.115:0/1662973556 conn(0x7f36c80fedc0 msgr2=0x7f36c81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.936+0000 7f36cee22700 1 -- 172.21.15.115:0/1662973556 shutdown_connections 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.936+0000 7f36cee22700 1 -- 172.21.15.115:0/1662973556 wait complete. 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cee22700 1 Processor -- start 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cee22700 1 -- start start 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cee22700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c81034c0 0x7f36c81a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:36.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cee22700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c8107df0 0x7f36c81a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cee22700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 0x7f36c819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cee22700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8113220 con 0x7f36c8108740 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cee22700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81130a0 con 0x7f36c8107df0 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cee22700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81133a0 con 0x7f36c81034c0 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cd3bf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 0x7f36c819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cd3bf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 0x7f36c819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45176/0 (socket says 172.21.15.115:45176) 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cd3bf700 1 -- 172.21.15.115:0/2730614776 learned_addr learned my addr 172.21.15.115:0/2730614776 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36ccbbe700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c81034c0 0x7f36c81a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36c7fff700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c8107df0 0x7f36c81a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cd3bf700 1 -- 172.21.15.115:0/2730614776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c81034c0 msgr2=0x7f36c81a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cd3bf700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c81034c0 0x7f36c81a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cd3bf700 1 -- 172.21.15.115:0/2730614776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c8107df0 msgr2=0x7f36c81a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cd3bf700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c8107df0 0x7f36c81a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.937+0000 7f36cd3bf700 1 -- 172.21.15.115:0/2730614776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c819b1d0 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.938+0000 7f36cd3bf700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 0x7f36c819aaf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009850 tx=0x7f36c0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.938+0000 7f36c5ffb700 1 -- 172.21.15.115:0/2730614776 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c001f070 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.938+0000 7f36c5ffb700 1 -- 172.21.15.115:0/2730614776 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36c0004c50 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.938+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c819b460 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.938+0000 7f36c5ffb700 1 -- 172.21.15.115:0/2730614776 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c001b690 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.938+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c81a7760 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.940+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c8104970 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.940+0000 7f36c5ffb700 1 -- 172.21.15.115:0/2730614776 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f36c0022070 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.941+0000 7f36c5ffb700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36b80b2900 0x7f36b80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.941+0000 7f36ccbbe700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36b80b2900 0x7f36b80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.941+0000 7f36c5ffb700 1 -- 172.21.15.115:0/2730614776 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f36c000f020 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.942+0000 7f36ccbbe700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36b80b2900 0x7f36b80b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36c819bc30 tx=0x7f36bc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.946+0000 7f36c5ffb700 1 -- 172.21.15.115:0/2730614776 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36c0029020 con 0x7f36c8108740 2024-05-09T05:36:36.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.161+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f36c819bfe0 con 0x7f36c8108740 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.162+0000 7f36c5ffb700 1 -- 172.21.15.115:0/2730614776 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f36c00aaff0 con 0x7f36c8108740 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36b80b2900 msgr2=0x7f36b80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36b80b2900 0x7f36b80b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36c819bc30 tx=0x7f36bc00b410 comp rx=0 tx=0).stop 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 msgr2=0x7f36c819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 0x7f36c819aaf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009850 tx=0x7f36c0007a20 comp rx=0 tx=0).stop 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 shutdown_connections 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36b80b2900 0x7f36b80b4e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36c8108740 0x7f36c819aaf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36c81034c0 0x7f36c81a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 --2- 172.21.15.115:0/2730614776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36c8107df0 0x7f36c81a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 >> 172.21.15.115:0/2730614776 conn(0x7f36c80fedc0 msgr2=0x7f36c81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 shutdown_connections 2024-05-09T05:36:36.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.164+0000 7f36cee22700 1 -- 172.21.15.115:0/2730614776 wait complete. 2024-05-09T05:36:36.555 INFO:teuthology.orchestra.run.smithi115.stdout:34359738472 2024-05-09T05:36:36.828 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:36 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1672227573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T05:36:36.829 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:36 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2730614776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T05:36:36.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:36 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1672227573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T05:36:36.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:36 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2730614776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T05:36:36.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:36 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1672227573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T05:36:36.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:36 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2730614776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T05:36:37.273 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738466 got 34359738472 for osd.0 2024-05-09T05:36:37.273 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.839+0000 7f47ec083700 1 -- 172.21.15.115:0/1541792347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e4108760 msgr2=0x7f47e41100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.839+0000 7f47ec083700 1 --2- 172.21.15.115:0/1541792347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e4108760 0x7f47e41100c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00b0d0 tx=0x7f47dc00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.839+0000 7f47ec083700 1 -- 172.21.15.115:0/1541792347 shutdown_connections 2024-05-09T05:36:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.839+0000 7f47ec083700 1 --2- 172.21.15.115:0/1541792347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e4108760 0x7f47e41100c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.839+0000 7f47ec083700 1 --2- 172.21.15.115:0/1541792347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e41034e0 0x7f47e41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.839+0000 7f47ec083700 1 --2- 172.21.15.115:0/1541792347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4107e10 0x7f47e4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.839+0000 7f47ec083700 1 -- 172.21.15.115:0/1541792347 >> 172.21.15.115:0/1541792347 conn(0x7f47e40fef30 msgr2=0x7f47e4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.839+0000 7f47ec083700 1 -- 172.21.15.115:0/1541792347 shutdown_connections 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.839+0000 7f47ec083700 1 -- 172.21.15.115:0/1541792347 wait complete. 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47ec083700 1 Processor -- start 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47ec083700 1 -- start start 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47ec083700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e41034e0 0x7f47e419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47ec083700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4107e10 0x7f47e419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47ec083700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4108760 0x7f47e4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47ec083700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4113240 con 0x7f47e41034e0 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47ec083700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41130c0 con 0x7f47e4107e10 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47ec083700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41133c0 con 0x7f47e4108760 2024-05-09T05:36:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47e9e1f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e41034e0 0x7f47e419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47e9e1f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e41034e0 0x7f47e419f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45162/0 (socket says 172.21.15.115:45162) 2024-05-09T05:36:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47e9e1f700 1 -- 172.21.15.115:0/1672227573 learned_addr learned my addr 172.21.15.115:0/1672227573 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.840+0000 7f47e961e700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4107e10 0x7f47e419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47ea620700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4108760 0x7f47e4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47e9e1f700 1 -- 172.21.15.115:0/1672227573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4108760 msgr2=0x7f47e4199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47e9e1f700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4108760 0x7f47e4199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47e9e1f700 1 -- 172.21.15.115:0/1672227573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4107e10 msgr2=0x7f47e419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47e9e1f700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4107e10 0x7f47e419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47e9e1f700 1 -- 172.21.15.115:0/1672227573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e419a0a0 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47e9e1f700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e41034e0 0x7f47e419f840 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f47e000c9b0 tx=0x7f47e000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47daffd700 1 -- 172.21.15.115:0/1672227573 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0007b00 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47daffd700 1 -- 172.21.15.115:0/1672227573 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47e0007ca0 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47daffd700 1 -- 172.21.15.115:0/1672227573 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0018660 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e419a390 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.841+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e406c040 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.842+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e4104990 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.844+0000 7f47daffd700 1 -- 172.21.15.115:0/1672227573 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f47e000f450 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.845+0000 7f47daffd700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47d00b2900 0x7f47d00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.845+0000 7f47e961e700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47d00b2900 0x7f47d00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.845+0000 7f47daffd700 1 -- 172.21.15.115:0/1672227573 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f47e00e2740 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.845+0000 7f47e961e700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47d00b2900 0x7f47d00b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f47e4102a20 tx=0x7f47d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:35.848+0000 7f47daffd700 1 -- 172.21.15.115:0/1672227573 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47e0014030 con 0x7f47e41034e0 2024-05-09T05:36:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.064+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f47e419af90 con 0x7f47e41034e0 2024-05-09T05:36:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.065+0000 7f47daffd700 1 -- 172.21.15.115:0/1672227573 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f47e00b2020 con 0x7f47e41034e0 2024-05-09T05:36:37.460 INFO:teuthology.orchestra.run.smithi115.stdout:300647710767 2024-05-09T05:36:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47d00b2900 msgr2=0x7f47d00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47d00b2900 0x7f47d00b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f47e4102a20 tx=0x7f47d4009260 comp rx=0 tx=0).stop 2024-05-09T05:36:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e41034e0 msgr2=0x7f47e419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e41034e0 0x7f47e419f840 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f47e000c9b0 tx=0x7f47e000ccc0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 shutdown_connections 2024-05-09T05:36:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f47d00b2900 0x7f47d00b4e10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e4108760 0x7f47e4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47e41034e0 0x7f47e419f840 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 --2- 172.21.15.115:0/1672227573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e4107e10 0x7f47e419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 >> 172.21.15.115:0/1672227573 conn(0x7f47e40fef30 msgr2=0x7f47e41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 shutdown_connections 2024-05-09T05:36:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:36.067+0000 7f47ec083700 1 -- 172.21.15.115:0/1672227573 wait complete. 2024-05-09T05:36:37.860 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:37 smithi115 ceph-mon[30349]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-09T05:36:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:37 smithi195 ceph-mon[33844]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-09T05:36:37.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:37 smithi161 ceph-mon[33805]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-09T05:36:38.179 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710761 got 300647710767 for osd.11 2024-05-09T05:36:38.179 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:38.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.374+0000 7f83e499f700 1 -- 172.21.15.115:0/2405355985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e01025e0 msgr2=0x7f83e01029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:38.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.374+0000 7f83e499f700 1 --2- 172.21.15.115:0/2405355985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e01025e0 0x7f83e01029f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009990 tx=0x7f83c8009ca0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.374+0000 7f83e499f700 1 -- 172.21.15.115:0/2405355985 shutdown_connections 2024-05-09T05:36:38.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.374+0000 7f83e499f700 1 --2- 172.21.15.115:0/2405355985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83e0109150 0x7f83e0109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.374+0000 7f83e499f700 1 --2- 172.21.15.115:0/2405355985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e0102f30 0x7f83e0110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.374+0000 7f83e499f700 1 --2- 172.21.15.115:0/2405355985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e01025e0 0x7f83e01029f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.374+0000 7f83e499f700 1 -- 172.21.15.115:0/2405355985 >> 172.21.15.115:0/2405355985 conn(0x7f83e00783f0 msgr2=0x7f83e0078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:38.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.374+0000 7f83e499f700 1 -- 172.21.15.115:0/2405355985 shutdown_connections 2024-05-09T05:36:38.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.374+0000 7f83e499f700 1 -- 172.21.15.115:0/2405355985 wait complete. 2024-05-09T05:36:38.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.375+0000 7f83e499f700 1 Processor -- start 2024-05-09T05:36:38.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83e499f700 1 -- start start 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83e499f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e01025e0 0x7f83e01a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83e499f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83e0102f30 0x7f83e01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83e499f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e0109150 0x7f83e019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83e499f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01142a0 con 0x7f83e0109150 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83e499f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0114120 con 0x7f83e0102f30 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83e499f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0114420 con 0x7f83e01025e0 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83dffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e0109150 0x7f83e019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83dffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e0109150 0x7f83e019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45196/0 (socket says 172.21.15.115:45196) 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83dffff700 1 -- 172.21.15.115:0/3186056766 learned_addr learned my addr 172.21.15.115:0/3186056766 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83deffd700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83e0102f30 0x7f83e01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.376+0000 7f83df7fe700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e01025e0 0x7f83e01a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83dffff700 1 -- 172.21.15.115:0/3186056766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e01025e0 msgr2=0x7f83e01a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83dffff700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e01025e0 0x7f83e01a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83dffff700 1 -- 172.21.15.115:0/3186056766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83e0102f30 msgr2=0x7f83e01a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83dffff700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83e0102f30 0x7f83e01a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83dffff700 1 -- 172.21.15.115:0/3186056766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e019b0a0 con 0x7f83e0109150 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83dffff700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e0109150 0x7f83e019aa60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f83d800c900 tx=0x7f83d800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83dcff9700 1 -- 172.21.15.115:0/3186056766 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8007b00 con 0x7f83e0109150 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83dcff9700 1 -- 172.21.15.115:0/3186056766 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83d800ce50 con 0x7f83e0109150 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e019b390 con 0x7f83e0109150 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83dcff9700 1 -- 172.21.15.115:0/3186056766 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8018660 con 0x7f83e0109150 2024-05-09T05:36:38.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.377+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e019b850 con 0x7f83e0109150 2024-05-09T05:36:38.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.378+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e0105900 con 0x7f83e0109150 2024-05-09T05:36:38.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.380+0000 7f83dcff9700 1 -- 172.21.15.115:0/3186056766 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f83d800f450 con 0x7f83e0109150 2024-05-09T05:36:38.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.380+0000 7f83dcff9700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f83cc0b2880 0x7f83cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:38.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.380+0000 7f83df7fe700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f83cc0b2880 0x7f83cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:38.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.380+0000 7f83dcff9700 1 -- 172.21.15.115:0/3186056766 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f83d80e2490 con 0x7f83e0109150 2024-05-09T05:36:38.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.381+0000 7f83df7fe700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f83cc0b2880 0x7f83cc0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83e019bba0 tx=0x7f83d0009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:38.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.384+0000 7f83dcff9700 1 -- 172.21.15.115:0/3186056766 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83d8014030 con 0x7f83e0109150 2024-05-09T05:36:38.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.596+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f83e019bf90 con 0x7f83e0109150 2024-05-09T05:36:38.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.597+0000 7f83dcff9700 1 -- 172.21.15.115:0/3186056766 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f83d80a9ee0 con 0x7f83e0109150 2024-05-09T05:36:38.590 INFO:teuthology.orchestra.run.smithi115.stdout:253403070523 2024-05-09T05:36:38.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f83cc0b2880 msgr2=0x7f83cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:38.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f83cc0b2880 0x7f83cc0b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83e019bba0 tx=0x7f83d0009040 comp rx=0 tx=0).stop 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e0109150 msgr2=0x7f83e019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e0109150 0x7f83e019aa60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f83d800c900 tx=0x7f83d800cc10 comp rx=0 tx=0).stop 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 shutdown_connections 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f83cc0b2880 0x7f83cc0b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83e0109150 0x7f83e019aa60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83e01025e0 0x7f83e01a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 --2- 172.21.15.115:0/3186056766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83e0102f30 0x7f83e01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 >> 172.21.15.115:0/3186056766 conn(0x7f83e00783f0 msgr2=0x7f83e00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 shutdown_connections 2024-05-09T05:36:38.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:38.599+0000 7f83e499f700 1 -- 172.21.15.115:0/3186056766 wait complete. 2024-05-09T05:36:38.816 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:38 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3186056766' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T05:36:38.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:38 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3186056766' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T05:36:39.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:38 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3186056766' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T05:36:39.994 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266747 got 227633266752 for osd.8 2024-05-09T05:36:39.994 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:40.337 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:40 smithi115 ceph-mon[30349]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T05:36:40.337 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:36:40.337 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:36:40.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:40 smithi161 ceph-mon[33805]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T05:36:40.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:36:40.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:36:40.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:40 smithi195 ceph-mon[33844]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-09T05:36:40.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:36:40.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:36:40.794 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070517 got 253403070523 for osd.9 2024-05-09T05:36:40.794 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:41.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:36:41.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:36:41.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:36:41.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:36:41.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:36:41.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:36:41.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:36:41.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:36:41.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:36:42.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.148+0000 7ff802359700 1 -- 172.21.15.115:0/3420595873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc107df0 msgr2=0x7ff7fc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.148+0000 7ff802359700 1 --2- 172.21.15.115:0/3420595873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc107df0 0x7ff7fc108200 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0099c0 tx=0x7ff7ec009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.148+0000 7ff802359700 1 -- 172.21.15.115:0/3420595873 shutdown_connections 2024-05-09T05:36:42.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.148+0000 7ff802359700 1 --2- 172.21.15.115:0/3420595873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7fc1034c0 0x7ff7fc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.148+0000 7ff802359700 1 --2- 172.21.15.115:0/3420595873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7fc108740 0x7ff7fc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.148+0000 7ff802359700 1 --2- 172.21.15.115:0/3420595873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc107df0 0x7ff7fc108200 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.148+0000 7ff802359700 1 -- 172.21.15.115:0/3420595873 >> 172.21.15.115:0/3420595873 conn(0x7ff7fc0fedc0 msgr2=0x7ff7fc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.148+0000 7ff802359700 1 -- 172.21.15.115:0/3420595873 shutdown_connections 2024-05-09T05:36:42.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.149+0000 7ff802359700 1 -- 172.21.15.115:0/3420595873 wait complete. 2024-05-09T05:36:42.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.149+0000 7ff802359700 1 Processor -- start 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.149+0000 7ff802359700 1 -- start start 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff802359700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7fc1034c0 0x7ff7fc19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff802359700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7fc107df0 0x7ff7fc19c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff802359700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc108740 0x7ff7fc19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff802359700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc113220 con 0x7ff7fc108740 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff802359700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1130a0 con 0x7ff7fc107df0 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff802359700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc1133a0 con 0x7ff7fc1034c0 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff8008f6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc108740 0x7ff7fc19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff7fb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7fc107df0 0x7ff7fc19c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff8008f6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc108740 0x7ff7fc19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45230/0 (socket says 172.21.15.115:45230) 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff8008f6700 1 -- 172.21.15.115:0/3114602896 learned_addr learned my addr 172.21.15.115:0/3114602896 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff7fbfff700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7fc1034c0 0x7ff7fc19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff8008f6700 1 -- 172.21.15.115:0/3114602896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7fc1034c0 msgr2=0x7ff7fc19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff8008f6700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7fc1034c0 0x7ff7fc19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff8008f6700 1 -- 172.21.15.115:0/3114602896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7fc107df0 msgr2=0x7ff7fc19c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff8008f6700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7fc107df0 0x7ff7fc19c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff8008f6700 1 -- 172.21.15.115:0/3114602896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc1a6ed0 con 0x7ff7fc108740 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.150+0000 7ff7fb7fe700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7fc107df0 0x7ff7fc19c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.151+0000 7ff8008f6700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc108740 0x7ff7fc19c620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400ef30 tx=0x7ff7f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.151+0000 7ff7f97fa700 1 -- 172.21.15.115:0/3114602896 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40092e0 con 0x7ff7fc108740 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.151+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7fc1a7160 con 0x7ff7fc108740 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.151+0000 7ff7f97fa700 1 -- 172.21.15.115:0/3114602896 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7f4004bb0 con 0x7ff7fc108740 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.151+0000 7ff7f97fa700 1 -- 172.21.15.115:0/3114602896 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4004020 con 0x7ff7fc108740 2024-05-09T05:36:42.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.151+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7fc1a7670 con 0x7ff7fc108740 2024-05-09T05:36:42.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.152+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7fc104970 con 0x7ff7fc108740 2024-05-09T05:36:42.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.153+0000 7ff7f97fa700 1 -- 172.21.15.115:0/3114602896 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff7f401b030 con 0x7ff7fc108740 2024-05-09T05:36:42.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.154+0000 7ff7f97fa700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff7e40b2880 0x7ff7e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:42.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.154+0000 7ff7fbfff700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff7e40b2880 0x7ff7e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:42.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.154+0000 7ff7f97fa700 1 -- 172.21.15.115:0/3114602896 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ff7f40e2060 con 0x7ff7fc108740 2024-05-09T05:36:42.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.155+0000 7ff7fbfff700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff7e40b2880 0x7ff7e40b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009bf0 tx=0x7ff7f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:42.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.159+0000 7ff7f97fa700 1 -- 172.21.15.115:0/3114602896 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7f4022020 con 0x7ff7fc108740 2024-05-09T05:36:42.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.046+0000 7f0e95a8e700 1 -- 172.21.15.115:0/23787844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e90107df0 msgr2=0x7f0e90108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.046+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/23787844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e90107df0 0x7f0e90108200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e800099c0 tx=0x7f0e80009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.046+0000 7f0e95a8e700 1 -- 172.21.15.115:0/23787844 shutdown_connections 2024-05-09T05:36:42.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.046+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/23787844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e901034c0 0x7f0e901038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.046+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/23787844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e90108740 0x7f0e901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.046+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/23787844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e90107df0 0x7f0e90108200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.046+0000 7f0e95a8e700 1 -- 172.21.15.115:0/23787844 >> 172.21.15.115:0/23787844 conn(0x7f0e900fedc0 msgr2=0x7f0e901011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.046+0000 7f0e95a8e700 1 -- 172.21.15.115:0/23787844 shutdown_connections 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.046+0000 7f0e95a8e700 1 -- 172.21.15.115:0/23787844 wait complete. 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.047+0000 7f0e95a8e700 1 Processor -- start 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.047+0000 7f0e95a8e700 1 -- start start 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.047+0000 7f0e95a8e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e901034c0 0x7f0e9019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.047+0000 7f0e95a8e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90107df0 0x7f0e9019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.047+0000 7f0e95a8e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e90108740 0x7f0e90199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.047+0000 7f0e95a8e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113200 con 0x7f0e901034c0 2024-05-09T05:36:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.047+0000 7f0e95a8e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113080 con 0x7f0e90107df0 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.047+0000 7f0e95a8e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113380 con 0x7f0e90108740 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e901034c0 0x7f0e9019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e901034c0 0x7f0e9019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45212/0 (socket says 172.21.15.115:45212) 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8f7fe700 1 -- 172.21.15.115:0/3714067152 learned_addr learned my addr 172.21.15.115:0/3714067152 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8effd700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90107df0 0x7f0e9019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8ffff700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e90108740 0x7f0e90199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8f7fe700 1 -- 172.21.15.115:0/3714067152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e90108740 msgr2=0x7f0e90199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8f7fe700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e90108740 0x7f0e90199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8f7fe700 1 -- 172.21.15.115:0/3714067152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90107df0 msgr2=0x7f0e9019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8f7fe700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90107df0 0x7f0e9019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8f7fe700 1 -- 172.21.15.115:0/3714067152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9019a0f0 con 0x7f0e901034c0 2024-05-09T05:36:42.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8f7fe700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e901034c0 0x7f0e9019f820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e84007b50 tx=0x7f0e84007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8cff9700 1 -- 172.21.15.115:0/3714067152 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84019020 con 0x7f0e901034c0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9019a3e0 con 0x7f0e901034c0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8cff9700 1 -- 172.21.15.115:0/3714067152 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e84004cc0 con 0x7f0e901034c0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e8cff9700 1 -- 172.21.15.115:0/3714067152 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400e500 con 0x7f0e901034c0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.048+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9006b610 con 0x7f0e901034c0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.050+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e90104970 con 0x7f0e901034c0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.051+0000 7f0e8cff9700 1 -- 172.21.15.115:0/3714067152 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0e84004830 con 0x7f0e901034c0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.051+0000 7f0e8cff9700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0e780b2880 0x7f0e780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.052+0000 7f0e8effd700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0e780b2880 0x7f0e780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.052+0000 7f0e8cff9700 1 -- 172.21.15.115:0/3714067152 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f0e840e3a00 con 0x7f0e901034c0 2024-05-09T05:36:42.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.052+0000 7f0e8effd700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0e780b2880 0x7f0e780b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009990 tx=0x7f0e80005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:42.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.056+0000 7f0e8cff9700 1 -- 172.21.15.115:0/3714067152 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e84023020 con 0x7f0e901034c0 2024-05-09T05:36:42.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.277+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0e9019af70 con 0x7f0e901034c0 2024-05-09T05:36:42.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.277+0000 7f0e8cff9700 1 -- 172.21.15.115:0/3714067152 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e840ab3f0 con 0x7f0e901034c0 2024-05-09T05:36:42.270 INFO:teuthology.orchestra.run.smithi115.stdout:55834574947 2024-05-09T05:36:42.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0e780b2880 msgr2=0x7f0e780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0e780b2880 0x7f0e780b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009990 tx=0x7f0e80005fb0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e901034c0 msgr2=0x7f0e9019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e901034c0 0x7f0e9019f820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e84007b50 tx=0x7f0e84007e60 comp rx=0 tx=0).stop 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 shutdown_connections 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f0e780b2880 0x7f0e780b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e90108740 0x7f0e90199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e901034c0 0x7f0e9019f820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 --2- 172.21.15.115:0/3714067152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e90107df0 0x7f0e9019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 >> 172.21.15.115:0/3714067152 conn(0x7f0e900fedc0 msgr2=0x7f0e901011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:42.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 shutdown_connections 2024-05-09T05:36:42.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.279+0000 7f0e95a8e700 1 -- 172.21.15.115:0/3714067152 wait complete. 2024-05-09T05:36:42.284 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:42 smithi115 ceph-mon[30349]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T05:36:42.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.375+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff7fc02e5f0 con 0x7ff7fc108740 2024-05-09T05:36:42.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.375+0000 7ff7f97fa700 1 -- 172.21.15.115:0/3114602896 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff7f4010040 con 0x7ff7fc108740 2024-05-09T05:36:42.369 INFO:teuthology.orchestra.run.smithi115.stdout:133143986260 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.377+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff7e40b2880 msgr2=0x7ff7e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.377+0000 7ff802359700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff7e40b2880 0x7ff7e40b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009bf0 tx=0x7ff7f0009260 comp rx=0 tx=0).stop 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.377+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc108740 msgr2=0x7ff7fc19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.377+0000 7ff802359700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc108740 0x7ff7fc19c620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400ef30 tx=0x7ff7f400c3a0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.378+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 shutdown_connections 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.378+0000 7ff802359700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7ff7e40b2880 0x7ff7e40b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.378+0000 7ff802359700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7fc108740 0x7ff7fc19c620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.378+0000 7ff802359700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7fc1034c0 0x7ff7fc19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.378+0000 7ff802359700 1 --2- 172.21.15.115:0/3114602896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7fc107df0 0x7ff7fc19c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:42.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.378+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 >> 172.21.15.115:0/3114602896 conn(0x7ff7fc0fedc0 msgr2=0x7ff7fc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:42.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.378+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 shutdown_connections 2024-05-09T05:36:42.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:42.378+0000 7ff802359700 1 -- 172.21.15.115:0/3114602896 wait complete. 2024-05-09T05:36:42.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:42 smithi195 ceph-mon[33844]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T05:36:42.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:42 smithi161 ceph-mon[33805]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-09T05:36:43.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574947 for osd.1 2024-05-09T05:36:43.109 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:43.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:43 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3714067152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T05:36:43.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:43 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3114602896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T05:36:43.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:43 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3714067152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T05:36:43.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:43 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3114602896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T05:36:43.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:43 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3714067152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T05:36:43.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:43 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3114602896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T05:36:43.904 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986254 got 133143986260 for osd.4 2024-05-09T05:36:43.904 DEBUG:teuthology.parallel:result is None 2024-05-09T05:36:43.904 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T05:36:43.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T05:36:44.152 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:44.550 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:44 smithi115 ceph-mon[30349]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-09T05:36:44.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:44 smithi195 ceph-mon[33844]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-09T05:36:44.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:44 smithi161 ceph-mon[33805]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-09T05:36:46.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.021+0000 7fec2ed6e700 1 -- 172.21.15.115:0/1968017106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec281034c0 msgr2=0x7fec281038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:46.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.021+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/1968017106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec281034c0 0x7fec281038d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0099c0 tx=0x7fec1c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.021+0000 7fec2ed6e700 1 -- 172.21.15.115:0/1968017106 shutdown_connections 2024-05-09T05:36:46.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.021+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/1968017106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec28108740 0x7fec281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.021+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/1968017106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec281034c0 0x7fec281038d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.021+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/1968017106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec28107df0 0x7fec28108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.021+0000 7fec2ed6e700 1 -- 172.21.15.115:0/1968017106 >> 172.21.15.115:0/1968017106 conn(0x7fec280fedc0 msgr2=0x7fec281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.021+0000 7fec2ed6e700 1 -- 172.21.15.115:0/1968017106 shutdown_connections 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.021+0000 7fec2ed6e700 1 -- 172.21.15.115:0/1968017106 wait complete. 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2ed6e700 1 Processor -- start 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2ed6e700 1 -- start start 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2ed6e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec281034c0 0x7fec2819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2ed6e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec28107df0 0x7fec2819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2ed6e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec28108740 0x7fec2819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2ed6e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fec28113220 con 0x7fec28108740 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2ed6e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec281130a0 con 0x7fec281034c0 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2ed6e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fec281133a0 con 0x7fec28107df0 2024-05-09T05:36:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2d30b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec28108740 0x7fec2819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2d30b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec28108740 0x7fec2819f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55376/0 (socket says 172.21.15.115:55376) 2024-05-09T05:36:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2d30b700 1 -- 172.21.15.115:0/371613388 learned_addr learned my addr 172.21.15.115:0/371613388 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2cb0a700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec281034c0 0x7fec2819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec27fff700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec28107df0 0x7fec2819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2d30b700 1 -- 172.21.15.115:0/371613388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec28107df0 msgr2=0x7fec2819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2d30b700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec28107df0 0x7fec2819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2d30b700 1 -- 172.21.15.115:0/371613388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec281034c0 msgr2=0x7fec2819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2d30b700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec281034c0 0x7fec2819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec2d30b700 1 -- 172.21.15.115:0/371613388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2819fbf0 con 0x7fec28108740 2024-05-09T05:36:46.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.022+0000 7fec27fff700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec28107df0 0x7fec2819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:36:46.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.023+0000 7fec2d30b700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec28108740 0x7fec2819f550 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec2000ada0 tx=0x7fec2000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:46.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.023+0000 7fec25ffb700 1 -- 172.21.15.115:0/371613388 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20017020 con 0x7fec28108740 2024-05-09T05:36:46.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.023+0000 7fec25ffb700 1 -- 172.21.15.115:0/371613388 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec2000ce10 con 0x7fec28108740 2024-05-09T05:36:46.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.023+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2819fee0 con 0x7fec28108740 2024-05-09T05:36:46.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.023+0000 7fec25ffb700 1 -- 172.21.15.115:0/371613388 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20005280 con 0x7fec28108740 2024-05-09T05:36:46.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.023+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec281a7800 con 0x7fec28108740 2024-05-09T05:36:46.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.024+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec280512c0 con 0x7fec28108740 2024-05-09T05:36:46.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.025+0000 7fec25ffb700 1 -- 172.21.15.115:0/371613388 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fec2000c980 con 0x7fec28108740 2024-05-09T05:36:46.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.026+0000 7fec25ffb700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fec180b2880 0x7fec180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:46.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.026+0000 7fec2cb0a700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fec180b2880 0x7fec180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:46.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.026+0000 7fec25ffb700 1 -- 172.21.15.115:0/371613388 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fec200e1c60 con 0x7fec28108740 2024-05-09T05:36:46.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.027+0000 7fec2cb0a700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fec180b2880 0x7fec180b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fec1c000c00 tx=0x7fec1c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:46.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.031+0000 7fec25ffb700 1 -- 172.21.15.115:0/371613388 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec20021020 con 0x7fec28108740 2024-05-09T05:36:46.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.227+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec2819c080 con 0x7fec180b2880 2024-05-09T05:36:46.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.236+0000 7fec25ffb700 1 -- 172.21.15.115:0/371613388 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340744 (secure 0 0 0) 0x7fec2819c080 con 0x7fec180b2880 2024-05-09T05:36:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fec180b2880 msgr2=0x7fec180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fec180b2880 0x7fec180b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fec1c000c00 tx=0x7fec1c01a040 comp rx=0 tx=0).stop 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec28108740 msgr2=0x7fec2819f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec28108740 0x7fec2819f550 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec2000ada0 tx=0x7fec2000c330 comp rx=0 tx=0).stop 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 shutdown_connections 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fec180b2880 0x7fec180b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec281034c0 0x7fec2819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec28108740 0x7fec2819f550 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 --2- 172.21.15.115:0/371613388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec28107df0 0x7fec2819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 >> 172.21.15.115:0/371613388 conn(0x7fec280fedc0 msgr2=0x7fec28101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 shutdown_connections 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:46.240+0000 7fec2ed6e700 1 -- 172.21.15.115:0/371613388 wait complete. 2024-05-09T05:36:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:36:46.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:46 smithi115 ceph-mon[30349]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-05-09T05:36:46.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:46 smithi195 ceph-mon[33844]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-05-09T05:36:46.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:46 smithi161 ceph-mon[33805]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-05-09T05:36:47.076 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-05-09T05:36:44.703952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661941,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1134,"num_read_kb":677,"num_write":382,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":815,"ondisk_log_size":815,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344340,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597484,"statfs":{"total":1151940427776,"available":1151587823616,"internally_reserved":0,"allocated":12111872,"data_stored":7916441,"data_compressed":426138,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":76,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003161"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673632+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:36:32.673632+0000","last_peered":"2024-05-09T05:36:32.673632+0000","last_clean":"2024-05-09T05:36:32.673632+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:36:32.673632+0000","last_undegraded":"2024-05-09T05:36:32.673632+0000","last_fullsized":"2024-05-09T05:36:32.673632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.668405+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:36:32.668405+0000","last_peered":"2024-05-09T05:36:32.668405+0000","last_clean":"2024-05-09T05:36:32.668405+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:36:32.668405+0000","last_undegraded":"2024-05-09T05:36:32.668405+0000","last_fullsized":"2024-05-09T05:36:32.668405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674286+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:36:32.674286+0000","last_peered":"2024-05-09T05:36:32.674286+0000","last_clean":"2024-05-09T05:36:32.674286+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:36:32.674286+0000","last_undegraded":"2024-05-09T05:36:32.674286+0000","last_fullsized":"2024-05-09T05:36:32.674286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.662136+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:36:32.662136+0000","last_peered":"2024-05-09T05:36:32.662136+0000","last_clean":"2024-05-09T05:36:32.662136+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:36:32.662136+0000","last_undegraded":"2024-05-09T05:36:32.662136+0000","last_fullsized":"2024-05-09T05:36:32.662136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.656869+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:36:32.656869+0000","last_peered":"2024-05-09T05:36:32.656869+0000","last_clean":"2024-05-09T05:36:32.656869+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:36:32.656869+0000","last_undegraded":"2024-05-09T05:36:32.656869+0000","last_fullsized":"2024-05-09T05:36:32.656869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.661485+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:36:32.661485+0000","last_peered":"2024-05-09T05:36:32.661485+0000","last_clean":"2024-05-09T05:36:32.661485+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:36:32.661485+0000","last_undegraded":"2024-05-09T05:36:32.661485+0000","last_fullsized":"2024-05-09T05:36:32.661485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.658544+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:36:32.658544+0000","last_peered":"2024-05-09T05:36:32.658544+0000","last_clean":"2024-05-09T05:36:32.658544+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:36:32.658544+0000","last_undegraded":"2024-05-09T05:36:32.658544+0000","last_fullsized":"2024-05-09T05:36:32.658544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.662921+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:36:32.662921+0000","last_peered":"2024-05-09T05:36:32.662921+0000","last_clean":"2024-05-09T05:36:32.662921+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:36:32.662921+0000","last_undegraded":"2024-05-09T05:36:32.662921+0000","last_fullsized":"2024-05-09T05:36:32.662921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674927+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:36:32.674927+0000","last_peered":"2024-05-09T05:36:32.674927+0000","last_clean":"2024-05-09T05:36:32.674927+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:36:32.674927+0000","last_undegraded":"2024-05-09T05:36:32.674927+0000","last_fullsized":"2024-05-09T05:36:32.674927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.663034+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:36:32.663034+0000","last_peered":"2024-05-09T05:36:32.663034+0000","last_clean":"2024-05-09T05:36:32.663034+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:36:32.663034+0000","last_undegraded":"2024-05-09T05:36:32.663034+0000","last_fullsized":"2024-05-09T05:36:32.663034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673050+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:36:32.673050+0000","last_peered":"2024-05-09T05:36:32.673050+0000","last_clean":"2024-05-09T05:36:32.673050+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:36:32.673050+0000","last_undegraded":"2024-05-09T05:36:32.673050+0000","last_fullsized":"2024-05-09T05:36:32.673050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.644278+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:36:32.644278+0000","last_peered":"2024-05-09T05:36:32.644278+0000","last_clean":"2024-05-09T05:36:32.644278+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:36:32.644278+0000","last_undegraded":"2024-05-09T05:36:32.644278+0000","last_fullsized":"2024-05-09T05:36:32.644278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.652656+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:36:32.652656+0000","last_peered":"2024-05-09T05:36:32.652656+0000","last_clean":"2024-05-09T05:36:32.652656+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:36:32.652656+0000","last_undegraded":"2024-05-09T05:36:32.652656+0000","last_fullsized":"2024-05-09T05:36:32.652656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670497+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:36:32.670497+0000","last_peered":"2024-05-09T05:36:32.670497+0000","last_clean":"2024-05-09T05:36:32.670497+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:36:32.670497+0000","last_undegraded":"2024-05-09T05:36:32.670497+0000","last_fullsized":"2024-05-09T05:36:32.670497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671339+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:36:32.671339+0000","last_peered":"2024-05-09T05:36:32.671339+0000","last_clean":"2024-05-09T05:36:32.671339+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:36:32.671339+0000","last_undegraded":"2024-05-09T05:36:32.671339+0000","last_fullsized":"2024-05-09T05:36:32.671339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673857+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:36:32.673857+0000","last_peered":"2024-05-09T05:36:32.673857+0000","last_clean":"2024-05-09T05:36:32.673857+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:36:32.673857+0000","last_undegraded":"2024-05-09T05:36:32.673857+0000","last_fullsized":"2024-05-09T05:36:32.673857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673738+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:36:32.673738+0000","last_peered":"2024-05-09T05:36:32.673738+0000","last_clean":"2024-05-09T05:36:32.673738+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:36:32.673738+0000","last_undegraded":"2024-05-09T05:36:32.673738+0000","last_fullsized":"2024-05-09T05:36:32.673738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.660165+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:36:32.660165+0000","last_peered":"2024-05-09T05:36:32.660165+0000","last_clean":"2024-05-09T05:36:32.660165+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:36:32.660165+0000","last_undegraded":"2024-05-09T05:36:32.660165+0000","last_fullsized":"2024-05-09T05:36:32.660165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670233+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:36:32.670233+0000","last_peered":"2024-05-09T05:36:32.670233+0000","last_clean":"2024-05-09T05:36:32.670233+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:36:32.670233+0000","last_undegraded":"2024-05-09T05:36:32.670233+0000","last_fullsized":"2024-05-09T05:36:32.670233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939702+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:34:33.939702+0000","last_peered":"2024-05-09T05:34:33.939702+0000","last_clean":"2024-05-09T05:34:33.939702+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:34:33.939702+0000","last_undegraded":"2024-05-09T05:34:33.939702+0000","last_fullsized":"2024-05-09T05:34:33.939702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":156,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:42.440023+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:36:42.440023+0000","last_peered":"2024-05-09T05:36:42.440023+0000","last_clean":"2024-05-09T05:36:42.440023+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:36:42.440023+0000","last_undegraded":"2024-05-09T05:36:42.440023+0000","last_fullsized":"2024-05-09T05:36:42.440023+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.657390+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:36:32.657390+0000","last_peered":"2024-05-09T05:36:32.657390+0000","last_clean":"2024-05-09T05:36:32.657390+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:36:32.657390+0000","last_undegraded":"2024-05-09T05:36:32.657390+0000","last_fullsized":"2024-05-09T05:36:32.657390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048456+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:34:34.048456+0000","last_peered":"2024-05-09T05:34:34.048456+0000","last_clean":"2024-05-09T05:34:34.048456+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:34:34.048456+0000","last_undegraded":"2024-05-09T05:34:34.048456+0000","last_fullsized":"2024-05-09T05:34:34.048456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:00.277761+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:35:00.277761+0000","last_peered":"2024-05-09T05:35:00.277761+0000","last_clean":"2024-05-09T05:35:00.277761+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:35:00.277761+0000","last_undegraded":"2024-05-09T05:35:00.277761+0000","last_fullsized":"2024-05-09T05:35:00.277761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674300+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:36:32.674300+0000","last_peered":"2024-05-09T05:36:32.674300+0000","last_clean":"2024-05-09T05:36:32.674300+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:36:32.674300+0000","last_undegraded":"2024-05-09T05:36:32.674300+0000","last_fullsized":"2024-05-09T05:36:32.674300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048916+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:34:34.048916+0000","last_peered":"2024-05-09T05:34:34.048916+0000","last_clean":"2024-05-09T05:34:34.048916+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:34:34.048916+0000","last_undegraded":"2024-05-09T05:34:34.048916+0000","last_fullsized":"2024-05-09T05:34:34.048916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:04.292236+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:35:04.292236+0000","last_peered":"2024-05-09T05:35:04.292236+0000","last_clean":"2024-05-09T05:35:04.292236+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:35:04.292236+0000","last_undegraded":"2024-05-09T05:35:04.292236+0000","last_fullsized":"2024-05-09T05:35:04.292236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.672949+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:36:32.672949+0000","last_peered":"2024-05-09T05:36:32.672949+0000","last_clean":"2024-05-09T05:36:32.672949+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:36:32.672949+0000","last_undegraded":"2024-05-09T05:36:32.672949+0000","last_fullsized":"2024-05-09T05:36:32.672949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048403+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:34:34.048403+0000","last_peered":"2024-05-09T05:34:34.048403+0000","last_clean":"2024-05-09T05:34:34.048403+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:34:34.048403+0000","last_undegraded":"2024-05-09T05:34:34.048403+0000","last_fullsized":"2024-05-09T05:34:34.048403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.304728+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:35:03.304728+0000","last_peered":"2024-05-09T05:35:03.304728+0000","last_clean":"2024-05-09T05:35:03.304728+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:35:03.304728+0000","last_undegraded":"2024-05-09T05:35:03.304728+0000","last_fullsized":"2024-05-09T05:35:03.304728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.657952+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:36:32.657952+0000","last_peered":"2024-05-09T05:36:32.657952+0000","last_clean":"2024-05-09T05:36:32.657952+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:36:32.657952+0000","last_undegraded":"2024-05-09T05:36:32.657952+0000","last_fullsized":"2024-05-09T05:36:32.657952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940095+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:34:33.940095+0000","last_peered":"2024-05-09T05:34:33.940095+0000","last_clean":"2024-05-09T05:34:33.940095+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:34:33.940095+0000","last_undegraded":"2024-05-09T05:34:33.940095+0000","last_fullsized":"2024-05-09T05:34:33.940095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:42.446183+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:36:42.446183+0000","last_peered":"2024-05-09T05:36:42.446183+0000","last_clean":"2024-05-09T05:36:42.446183+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:36:42.446183+0000","last_undegraded":"2024-05-09T05:36:42.446183+0000","last_fullsized":"2024-05-09T05:36:42.446183+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.657342+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:36:32.657342+0000","last_peered":"2024-05-09T05:36:32.657342+0000","last_clean":"2024-05-09T05:36:32.657342+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:36:32.657342+0000","last_undegraded":"2024-05-09T05:36:32.657342+0000","last_fullsized":"2024-05-09T05:36:32.657342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940225+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:34:33.940225+0000","last_peered":"2024-05-09T05:34:33.940225+0000","last_clean":"2024-05-09T05:34:33.940225+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:34:33.940225+0000","last_undegraded":"2024-05-09T05:34:33.940225+0000","last_fullsized":"2024-05-09T05:34:33.940225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.293766+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:35:01.293766+0000","last_peered":"2024-05-09T05:35:01.293766+0000","last_clean":"2024-05-09T05:35:01.293766+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:35:01.293766+0000","last_undegraded":"2024-05-09T05:35:01.293766+0000","last_fullsized":"2024-05-09T05:35:01.293766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.661859+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:36:32.661859+0000","last_peered":"2024-05-09T05:36:32.661859+0000","last_clean":"2024-05-09T05:36:32.661859+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:36:32.661859+0000","last_undegraded":"2024-05-09T05:36:32.661859+0000","last_fullsized":"2024-05-09T05:36:32.661859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.051637+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:34:34.051637+0000","last_peered":"2024-05-09T05:34:34.051637+0000","last_clean":"2024-05-09T05:34:34.051637+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:34:34.051637+0000","last_undegraded":"2024-05-09T05:34:34.051637+0000","last_fullsized":"2024-05-09T05:34:34.051637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296327+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:35:02.296327+0000","last_peered":"2024-05-09T05:35:02.296327+0000","last_clean":"2024-05-09T05:35:02.296327+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:35:02.296327+0000","last_undegraded":"2024-05-09T05:35:02.296327+0000","last_fullsized":"2024-05-09T05:35:02.296327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.665685+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:36:32.665685+0000","last_peered":"2024-05-09T05:36:32.665685+0000","last_clean":"2024-05-09T05:36:32.665685+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:36:32.665685+0000","last_undegraded":"2024-05-09T05:36:32.665685+0000","last_fullsized":"2024-05-09T05:36:32.665685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904186+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:34:33.904186+0000","last_peered":"2024-05-09T05:34:33.904186+0000","last_clean":"2024-05-09T05:34:33.904186+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:34:33.904186+0000","last_undegraded":"2024-05-09T05:34:33.904186+0000","last_fullsized":"2024-05-09T05:34:33.904186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.305110+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:35:03.305110+0000","last_peered":"2024-05-09T05:35:03.305110+0000","last_clean":"2024-05-09T05:35:03.305110+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:35:03.305110+0000","last_undegraded":"2024-05-09T05:35:03.305110+0000","last_fullsized":"2024-05-09T05:35:03.305110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.659743+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:36:32.659743+0000","last_peered":"2024-05-09T05:36:32.659743+0000","last_clean":"2024-05-09T05:36:32.659743+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:36:32.659743+0000","last_undegraded":"2024-05-09T05:36:32.659743+0000","last_fullsized":"2024-05-09T05:36:32.659743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048695+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:34:34.048695+0000","last_peered":"2024-05-09T05:34:34.048695+0000","last_clean":"2024-05-09T05:34:34.048695+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:34:34.048695+0000","last_undegraded":"2024-05-09T05:34:34.048695+0000","last_fullsized":"2024-05-09T05:34:34.048695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":249,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:44.449123+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:36:44.449123+0000","last_peered":"2024-05-09T05:36:44.449123+0000","last_clean":"2024-05-09T05:36:44.449123+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:36:44.449123+0000","last_undegraded":"2024-05-09T05:36:44.449123+0000","last_fullsized":"2024-05-09T05:36:44.449123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673353+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:36:32.673353+0000","last_peered":"2024-05-09T05:36:32.673353+0000","last_clean":"2024-05-09T05:36:32.673353+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:36:32.673353+0000","last_undegraded":"2024-05-09T05:36:32.673353+0000","last_fullsized":"2024-05-09T05:36:32.673353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907348+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:34:33.907348+0000","last_peered":"2024-05-09T05:34:33.907348+0000","last_clean":"2024-05-09T05:34:33.907348+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:34:33.907348+0000","last_undegraded":"2024-05-09T05:34:33.907348+0000","last_fullsized":"2024-05-09T05:34:33.907348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.298403+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:35:02.298403+0000","last_peered":"2024-05-09T05:35:02.298403+0000","last_clean":"2024-05-09T05:35:02.298403+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:35:02.298403+0000","last_undegraded":"2024-05-09T05:35:02.298403+0000","last_fullsized":"2024-05-09T05:35:02.298403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.666783+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:36:32.666783+0000","last_peered":"2024-05-09T05:36:32.666783+0000","last_clean":"2024-05-09T05:36:32.666783+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:36:32.666783+0000","last_undegraded":"2024-05-09T05:36:32.666783+0000","last_fullsized":"2024-05-09T05:36:32.666783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049137+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:34:34.049137+0000","last_peered":"2024-05-09T05:34:34.049137+0000","last_clean":"2024-05-09T05:34:34.049137+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:34:34.049137+0000","last_undegraded":"2024-05-09T05:34:34.049137+0000","last_fullsized":"2024-05-09T05:34:34.049137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.304949+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:35:03.304949+0000","last_peered":"2024-05-09T05:35:03.304949+0000","last_clean":"2024-05-09T05:35:03.304949+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:35:03.304949+0000","last_undegraded":"2024-05-09T05:35:03.304949+0000","last_fullsized":"2024-05-09T05:35:03.304949+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674668+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:36:32.674668+0000","last_peered":"2024-05-09T05:36:32.674668+0000","last_clean":"2024-05-09T05:36:32.674668+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:36:32.674668+0000","last_undegraded":"2024-05-09T05:36:32.674668+0000","last_fullsized":"2024-05-09T05:36:32.674668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906379+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:34:33.906379+0000","last_peered":"2024-05-09T05:34:33.906379+0000","last_clean":"2024-05-09T05:34:33.906379+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:34:33.906379+0000","last_undegraded":"2024-05-09T05:34:33.906379+0000","last_fullsized":"2024-05-09T05:34:33.906379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'39","reported_seq":291,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:43.463973+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:36:43.463973+0000","last_peered":"2024-05-09T05:36:43.463973+0000","last_clean":"2024-05-09T05:36:43.463973+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:36:43.463973+0000","last_undegraded":"2024-05-09T05:36:43.463973+0000","last_fullsized":"2024-05-09T05:36:43.463973+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":169,"num_write":41,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670390+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:36:32.670390+0000","last_peered":"2024-05-09T05:36:32.670390+0000","last_clean":"2024-05-09T05:36:32.670390+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:36:32.670390+0000","last_undegraded":"2024-05-09T05:36:32.670390+0000","last_fullsized":"2024-05-09T05:36:32.670390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.911562+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:34:33.911562+0000","last_peered":"2024-05-09T05:34:33.911562+0000","last_clean":"2024-05-09T05:34:33.911562+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:34:33.911562+0000","last_undegraded":"2024-05-09T05:34:33.911562+0000","last_fullsized":"2024-05-09T05:34:33.911562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301529+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:35:03.301529+0000","last_peered":"2024-05-09T05:35:03.301529+0000","last_clean":"2024-05-09T05:35:03.301529+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:35:03.301529+0000","last_undegraded":"2024-05-09T05:35:03.301529+0000","last_fullsized":"2024-05-09T05:35:03.301529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.667519+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:36:32.667519+0000","last_peered":"2024-05-09T05:36:32.667519+0000","last_clean":"2024-05-09T05:36:32.667519+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:36:32.667519+0000","last_undegraded":"2024-05-09T05:36:32.667519+0000","last_fullsized":"2024-05-09T05:36:32.667519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907888+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:34:33.907888+0000","last_peered":"2024-05-09T05:34:33.907888+0000","last_clean":"2024-05-09T05:34:33.907888+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:34:33.907888+0000","last_undegraded":"2024-05-09T05:34:33.907888+0000","last_fullsized":"2024-05-09T05:34:33.907888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301445+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:35:03.301445+0000","last_peered":"2024-05-09T05:35:03.301445+0000","last_clean":"2024-05-09T05:35:03.301445+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:35:03.301445+0000","last_undegraded":"2024-05-09T05:35:03.301445+0000","last_fullsized":"2024-05-09T05:35:03.301445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670024+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:36:32.670024+0000","last_peered":"2024-05-09T05:36:32.670024+0000","last_clean":"2024-05-09T05:36:32.670024+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:36:32.670024+0000","last_undegraded":"2024-05-09T05:36:32.670024+0000","last_fullsized":"2024-05-09T05:36:32.670024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.051709+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:34:34.051709+0000","last_peered":"2024-05-09T05:34:34.051709+0000","last_clean":"2024-05-09T05:34:34.051709+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:34:34.051709+0000","last_undegraded":"2024-05-09T05:34:34.051709+0000","last_fullsized":"2024-05-09T05:34:34.051709+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:41.439418+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:36:41.439418+0000","last_peered":"2024-05-09T05:36:41.439418+0000","last_clean":"2024-05-09T05:36:41.439418+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:36:41.439418+0000","last_undegraded":"2024-05-09T05:36:41.439418+0000","last_fullsized":"2024-05-09T05:36:41.439418+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.656977+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:36:32.656977+0000","last_peered":"2024-05-09T05:36:32.656977+0000","last_clean":"2024-05-09T05:36:32.656977+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:36:32.656977+0000","last_undegraded":"2024-05-09T05:36:32.656977+0000","last_fullsized":"2024-05-09T05:36:32.656977+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048584+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:34:34.048584+0000","last_peered":"2024-05-09T05:34:34.048584+0000","last_clean":"2024-05-09T05:34:34.048584+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:34:34.048584+0000","last_undegraded":"2024-05-09T05:34:34.048584+0000","last_fullsized":"2024-05-09T05:34:34.048584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.302358+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:35:03.302358+0000","last_peered":"2024-05-09T05:35:03.302358+0000","last_clean":"2024-05-09T05:35:03.302358+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:35:03.302358+0000","last_undegraded":"2024-05-09T05:35:03.302358+0000","last_fullsized":"2024-05-09T05:35:03.302358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.652733+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:36:32.652733+0000","last_peered":"2024-05-09T05:36:32.652733+0000","last_clean":"2024-05-09T05:36:32.652733+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:36:32.652733+0000","last_undegraded":"2024-05-09T05:36:32.652733+0000","last_fullsized":"2024-05-09T05:36:32.652733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.910622+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:34:33.910622+0000","last_peered":"2024-05-09T05:34:33.910622+0000","last_clean":"2024-05-09T05:34:33.910622+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:34:33.910622+0000","last_undegraded":"2024-05-09T05:34:33.910622+0000","last_fullsized":"2024-05-09T05:34:33.910622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301008+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:35:03.301008+0000","last_peered":"2024-05-09T05:35:03.301008+0000","last_clean":"2024-05-09T05:35:03.301008+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:35:03.301008+0000","last_undegraded":"2024-05-09T05:35:03.301008+0000","last_fullsized":"2024-05-09T05:35:03.301008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.667110+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:36:32.667110+0000","last_peered":"2024-05-09T05:36:32.667110+0000","last_clean":"2024-05-09T05:36:32.667110+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:36:32.667110+0000","last_undegraded":"2024-05-09T05:36:32.667110+0000","last_fullsized":"2024-05-09T05:36:32.667110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.943171+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:34:33.943171+0000","last_peered":"2024-05-09T05:34:33.943171+0000","last_clean":"2024-05-09T05:34:33.943171+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:34:33.943171+0000","last_undegraded":"2024-05-09T05:34:33.943171+0000","last_fullsized":"2024-05-09T05:34:33.943171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.294078+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:35:01.294078+0000","last_peered":"2024-05-09T05:35:01.294078+0000","last_clean":"2024-05-09T05:35:01.294078+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:35:01.294078+0000","last_undegraded":"2024-05-09T05:35:01.294078+0000","last_fullsized":"2024-05-09T05:35:01.294078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.650831+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:36:32.650831+0000","last_peered":"2024-05-09T05:36:32.650831+0000","last_clean":"2024-05-09T05:36:32.650831+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:36:32.650831+0000","last_undegraded":"2024-05-09T05:36:32.650831+0000","last_fullsized":"2024-05-09T05:36:32.650831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048059+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:34:34.048059+0000","last_peered":"2024-05-09T05:34:34.048059+0000","last_clean":"2024-05-09T05:34:34.048059+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:34:34.048059+0000","last_undegraded":"2024-05-09T05:34:34.048059+0000","last_fullsized":"2024-05-09T05:34:34.048059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:42.440626+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:36:42.440626+0000","last_peered":"2024-05-09T05:36:42.440626+0000","last_clean":"2024-05-09T05:36:42.440626+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:36:42.440626+0000","last_undegraded":"2024-05-09T05:36:42.440626+0000","last_fullsized":"2024-05-09T05:36:42.440626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.660957+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:36:32.660957+0000","last_peered":"2024-05-09T05:36:32.660957+0000","last_clean":"2024-05-09T05:36:32.660957+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:36:32.660957+0000","last_undegraded":"2024-05-09T05:36:32.660957+0000","last_fullsized":"2024-05-09T05:36:32.660957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048446+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:34:34.048446+0000","last_peered":"2024-05-09T05:34:34.048446+0000","last_clean":"2024-05-09T05:34:34.048446+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:34:34.048446+0000","last_undegraded":"2024-05-09T05:34:34.048446+0000","last_fullsized":"2024-05-09T05:34:34.048446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671054+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:36:32.671054+0000","last_peered":"2024-05-09T05:36:32.671054+0000","last_clean":"2024-05-09T05:36:32.671054+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:36:32.671054+0000","last_undegraded":"2024-05-09T05:36:32.671054+0000","last_fullsized":"2024-05-09T05:36:32.671054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:43.441298+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:36:43.441298+0000","last_peered":"2024-05-09T05:36:43.441298+0000","last_clean":"2024-05-09T05:36:43.441298+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:36:43.441298+0000","last_undegraded":"2024-05-09T05:36:43.441298+0000","last_fullsized":"2024-05-09T05:36:43.441298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:11.405828+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:35:11.405828+0000","last_peered":"2024-05-09T05:35:11.405828+0000","last_clean":"2024-05-09T05:35:11.405828+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:35:11.405828+0000","last_undegraded":"2024-05-09T05:35:11.405828+0000","last_fullsized":"2024-05-09T05:35:11.405828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.658675+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:36:32.658675+0000","last_peered":"2024-05-09T05:36:32.658675+0000","last_clean":"2024-05-09T05:36:32.658675+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:36:32.658675+0000","last_undegraded":"2024-05-09T05:36:32.658675+0000","last_fullsized":"2024-05-09T05:36:32.658675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.302837+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:35:03.302837+0000","last_peered":"2024-05-09T05:35:03.302837+0000","last_clean":"2024-05-09T05:35:03.302837+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:35:03.302837+0000","last_undegraded":"2024-05-09T05:35:03.302837+0000","last_fullsized":"2024-05-09T05:35:03.302837+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904811+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:34:33.904811+0000","last_peered":"2024-05-09T05:34:33.904811+0000","last_clean":"2024-05-09T05:34:33.904811+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:34:33.904811+0000","last_undegraded":"2024-05-09T05:34:33.904811+0000","last_fullsized":"2024-05-09T05:34:33.904811+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.911155+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:34:33.911155+0000","last_peered":"2024-05-09T05:34:33.911155+0000","last_clean":"2024-05-09T05:34:33.911155+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:34:33.911155+0000","last_undegraded":"2024-05-09T05:34:33.911155+0000","last_fullsized":"2024-05-09T05:34:33.911155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.666413+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:36:32.666413+0000","last_peered":"2024-05-09T05:36:32.666413+0000","last_clean":"2024-05-09T05:36:32.666413+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:36:32.666413+0000","last_undegraded":"2024-05-09T05:36:32.666413+0000","last_fullsized":"2024-05-09T05:36:32.666413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.297586+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:35:02.297586+0000","last_peered":"2024-05-09T05:35:02.297586+0000","last_clean":"2024-05-09T05:35:02.297586+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:35:02.297586+0000","last_undegraded":"2024-05-09T05:35:02.297586+0000","last_fullsized":"2024-05-09T05:35:02.297586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907587+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:34:33.907587+0000","last_peered":"2024-05-09T05:34:33.907587+0000","last_clean":"2024-05-09T05:34:33.907587+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:34:33.907587+0000","last_undegraded":"2024-05-09T05:34:33.907587+0000","last_fullsized":"2024-05-09T05:34:33.907587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.302434+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:35:03.302434+0000","last_peered":"2024-05-09T05:35:03.302434+0000","last_clean":"2024-05-09T05:35:03.302434+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:35:03.302434+0000","last_undegraded":"2024-05-09T05:35:03.302434+0000","last_fullsized":"2024-05-09T05:35:03.302434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.660776+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:36:32.660776+0000","last_peered":"2024-05-09T05:36:32.660776+0000","last_clean":"2024-05-09T05:36:32.660776+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:36:32.660776+0000","last_undegraded":"2024-05-09T05:36:32.660776+0000","last_fullsized":"2024-05-09T05:36:32.660776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048464+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:34:34.048464+0000","last_peered":"2024-05-09T05:34:34.048464+0000","last_clean":"2024-05-09T05:34:34.048464+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:34:34.048464+0000","last_undegraded":"2024-05-09T05:34:34.048464+0000","last_fullsized":"2024-05-09T05:34:34.048464+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'17","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:03.595473+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:36:03.595473+0000","last_peered":"2024-05-09T05:36:03.595473+0000","last_clean":"2024-05-09T05:36:03.595473+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:36:03.595473+0000","last_undegraded":"2024-05-09T05:36:03.595473+0000","last_fullsized":"2024-05-09T05:36:03.595473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.667526+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:36:32.667526+0000","last_peered":"2024-05-09T05:36:32.667526+0000","last_clean":"2024-05-09T05:36:32.667526+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:36:32.667526+0000","last_undegraded":"2024-05-09T05:36:32.667526+0000","last_fullsized":"2024-05-09T05:36:32.667526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939375+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:34:33.939375+0000","last_peered":"2024-05-09T05:34:33.939375+0000","last_clean":"2024-05-09T05:34:33.939375+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:34:33.939375+0000","last_undegraded":"2024-05-09T05:34:33.939375+0000","last_fullsized":"2024-05-09T05:34:33.939375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301938+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:35:03.301938+0000","last_peered":"2024-05-09T05:35:03.301938+0000","last_clean":"2024-05-09T05:35:03.301938+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:35:03.301938+0000","last_undegraded":"2024-05-09T05:35:03.301938+0000","last_fullsized":"2024-05-09T05:35:03.301938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.664081+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:36:32.664081+0000","last_peered":"2024-05-09T05:36:32.664081+0000","last_clean":"2024-05-09T05:36:32.664081+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:36:32.664081+0000","last_undegraded":"2024-05-09T05:36:32.664081+0000","last_fullsized":"2024-05-09T05:36:32.664081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049332+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:34:34.049332+0000","last_peered":"2024-05-09T05:34:34.049332+0000","last_clean":"2024-05-09T05:34:34.049332+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:34:34.049332+0000","last_undegraded":"2024-05-09T05:34:34.049332+0000","last_fullsized":"2024-05-09T05:34:34.049332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:59.271069+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:34:59.271069+0000","last_peered":"2024-05-09T05:34:59.271069+0000","last_clean":"2024-05-09T05:34:59.271069+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:34:59.271069+0000","last_undegraded":"2024-05-09T05:34:59.271069+0000","last_fullsized":"2024-05-09T05:34:59.271069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.625581+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:36:32.625581+0000","last_peered":"2024-05-09T05:36:32.625581+0000","last_clean":"2024-05-09T05:36:32.625581+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:36:32.625581+0000","last_undegraded":"2024-05-09T05:36:32.625581+0000","last_fullsized":"2024-05-09T05:36:32.625581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906493+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:34:33.906493+0000","last_peered":"2024-05-09T05:34:33.906493+0000","last_clean":"2024-05-09T05:34:33.906493+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:34:33.906493+0000","last_undegraded":"2024-05-09T05:34:33.906493+0000","last_fullsized":"2024-05-09T05:34:33.906493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.297494+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:35:02.297494+0000","last_peered":"2024-05-09T05:35:02.297494+0000","last_clean":"2024-05-09T05:35:02.297494+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:35:02.297494+0000","last_undegraded":"2024-05-09T05:35:02.297494+0000","last_fullsized":"2024-05-09T05:35:02.297494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673393+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:36:32.673393+0000","last_peered":"2024-05-09T05:36:32.673393+0000","last_clean":"2024-05-09T05:36:32.673393+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:36:32.673393+0000","last_undegraded":"2024-05-09T05:36:32.673393+0000","last_fullsized":"2024-05-09T05:36:32.673393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:17.852895+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:35:17.852895+0000","last_peered":"2024-05-09T05:35:17.852895+0000","last_clean":"2024-05-09T05:35:17.852895+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:35:17.852895+0000","last_undegraded":"2024-05-09T05:35:17.852895+0000","last_fullsized":"2024-05-09T05:35:17.852895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.288715+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:35:02.288715+0000","last_peered":"2024-05-09T05:35:02.288715+0000","last_clean":"2024-05-09T05:35:02.288715+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:35:02.288715+0000","last_undegraded":"2024-05-09T05:35:02.288715+0000","last_fullsized":"2024-05-09T05:35:02.288715+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.664661+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:36:32.664661+0000","last_peered":"2024-05-09T05:36:32.664661+0000","last_clean":"2024-05-09T05:36:32.664661+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:36:32.664661+0000","last_undegraded":"2024-05-09T05:36:32.664661+0000","last_fullsized":"2024-05-09T05:36:32.664661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049130+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:34:34.049130+0000","last_peered":"2024-05-09T05:34:34.049130+0000","last_clean":"2024-05-09T05:34:34.049130+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:34:34.049130+0000","last_undegraded":"2024-05-09T05:34:34.049130+0000","last_fullsized":"2024-05-09T05:34:34.049130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.300961+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:35:03.300961+0000","last_peered":"2024-05-09T05:35:03.300961+0000","last_clean":"2024-05-09T05:35:03.300961+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:35:03.300961+0000","last_undegraded":"2024-05-09T05:35:03.300961+0000","last_fullsized":"2024-05-09T05:35:03.300961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671748+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:36:32.671748+0000","last_peered":"2024-05-09T05:36:32.671748+0000","last_clean":"2024-05-09T05:36:32.671748+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:36:32.671748+0000","last_undegraded":"2024-05-09T05:36:32.671748+0000","last_fullsized":"2024-05-09T05:36:32.671748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940189+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:34:33.940189+0000","last_peered":"2024-05-09T05:34:33.940189+0000","last_clean":"2024-05-09T05:34:33.940189+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:34:33.940189+0000","last_undegraded":"2024-05-09T05:34:33.940189+0000","last_fullsized":"2024-05-09T05:34:33.940189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":147,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:41.445062+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:36:41.445062+0000","last_peered":"2024-05-09T05:36:41.445062+0000","last_clean":"2024-05-09T05:36:41.445062+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:36:41.445062+0000","last_undegraded":"2024-05-09T05:36:41.445062+0000","last_fullsized":"2024-05-09T05:36:41.445062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.672363+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:36:32.672363+0000","last_peered":"2024-05-09T05:36:32.672363+0000","last_clean":"2024-05-09T05:36:32.672363+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:36:32.672363+0000","last_undegraded":"2024-05-09T05:36:32.672363+0000","last_fullsized":"2024-05-09T05:36:32.672363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904191+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:34:33.904191+0000","last_peered":"2024-05-09T05:34:33.904191+0000","last_clean":"2024-05-09T05:34:33.904191+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:34:33.904191+0000","last_undegraded":"2024-05-09T05:34:33.904191+0000","last_fullsized":"2024-05-09T05:34:33.904191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301029+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:35:03.301029+0000","last_peered":"2024-05-09T05:35:03.301029+0000","last_clean":"2024-05-09T05:35:03.301029+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:35:03.301029+0000","last_undegraded":"2024-05-09T05:35:03.301029+0000","last_fullsized":"2024-05-09T05:35:03.301029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.652652+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:36:32.652652+0000","last_peered":"2024-05-09T05:36:32.652652+0000","last_clean":"2024-05-09T05:36:32.652652+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:36:32.652652+0000","last_undegraded":"2024-05-09T05:36:32.652652+0000","last_fullsized":"2024-05-09T05:36:32.652652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.435700+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:34:34.435700+0000","last_peered":"2024-05-09T05:34:34.435700+0000","last_clean":"2024-05-09T05:34:34.435700+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:34:34.435700+0000","last_undegraded":"2024-05-09T05:34:34.435700+0000","last_fullsized":"2024-05-09T05:34:34.435700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296675+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:35:02.296675+0000","last_peered":"2024-05-09T05:35:02.296675+0000","last_clean":"2024-05-09T05:35:02.296675+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:35:02.296675+0000","last_undegraded":"2024-05-09T05:35:02.296675+0000","last_fullsized":"2024-05-09T05:35:02.296675+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673360+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:36:32.673360+0000","last_peered":"2024-05-09T05:36:32.673360+0000","last_clean":"2024-05-09T05:36:32.673360+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:36:32.673360+0000","last_undegraded":"2024-05-09T05:36:32.673360+0000","last_fullsized":"2024-05-09T05:36:32.673360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673905+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:36:32.673905+0000","last_peered":"2024-05-09T05:36:32.673905+0000","last_clean":"2024-05-09T05:36:32.673905+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:36:32.673905+0000","last_undegraded":"2024-05-09T05:36:32.673905+0000","last_fullsized":"2024-05-09T05:36:32.673905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671679+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:36:32.671679+0000","last_peered":"2024-05-09T05:36:32.671679+0000","last_clean":"2024-05-09T05:36:32.671679+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:36:32.671679+0000","last_undegraded":"2024-05-09T05:36:32.671679+0000","last_fullsized":"2024-05-09T05:36:32.671679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670751+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:36:32.670751+0000","last_peered":"2024-05-09T05:36:32.670751+0000","last_clean":"2024-05-09T05:36:32.670751+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:36:32.670751+0000","last_undegraded":"2024-05-09T05:36:32.670751+0000","last_fullsized":"2024-05-09T05:36:32.670751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.662209+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:36:32.662209+0000","last_peered":"2024-05-09T05:36:32.662209+0000","last_clean":"2024-05-09T05:36:32.662209+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:36:32.662209+0000","last_undegraded":"2024-05-09T05:36:32.662209+0000","last_fullsized":"2024-05-09T05:36:32.662209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.655309+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:36:32.655309+0000","last_peered":"2024-05-09T05:36:32.655309+0000","last_clean":"2024-05-09T05:36:32.655309+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:36:32.655309+0000","last_undegraded":"2024-05-09T05:36:32.655309+0000","last_fullsized":"2024-05-09T05:36:32.655309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.668675+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:36:32.668675+0000","last_peered":"2024-05-09T05:36:32.668675+0000","last_clean":"2024-05-09T05:36:32.668675+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:36:32.668675+0000","last_undegraded":"2024-05-09T05:36:32.668675+0000","last_fullsized":"2024-05-09T05:36:32.668675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.669619+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:36:32.669619+0000","last_peered":"2024-05-09T05:36:32.669619+0000","last_clean":"2024-05-09T05:36:32.669619+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:36:32.669619+0000","last_undegraded":"2024-05-09T05:36:32.669619+0000","last_fullsized":"2024-05-09T05:36:32.669619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.650073+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:36:32.650073+0000","last_peered":"2024-05-09T05:36:32.650073+0000","last_clean":"2024-05-09T05:36:32.650073+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:36:32.650073+0000","last_undegraded":"2024-05-09T05:36:32.650073+0000","last_fullsized":"2024-05-09T05:36:32.650073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.662497+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:36:32.662497+0000","last_peered":"2024-05-09T05:36:32.662497+0000","last_clean":"2024-05-09T05:36:32.662497+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:36:32.662497+0000","last_undegraded":"2024-05-09T05:36:32.662497+0000","last_fullsized":"2024-05-09T05:36:32.662497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670717+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:36:32.670717+0000","last_peered":"2024-05-09T05:36:32.670717+0000","last_clean":"2024-05-09T05:36:32.670717+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:36:32.670717+0000","last_undegraded":"2024-05-09T05:36:32.670717+0000","last_fullsized":"2024-05-09T05:36:32.670717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674120+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:36:32.674120+0000","last_peered":"2024-05-09T05:36:32.674120+0000","last_clean":"2024-05-09T05:36:32.674120+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:36:32.674120+0000","last_undegraded":"2024-05-09T05:36:32.674120+0000","last_fullsized":"2024-05-09T05:36:32.674120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.667614+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:36:32.667614+0000","last_peered":"2024-05-09T05:36:32.667614+0000","last_clean":"2024-05-09T05:36:32.667614+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:36:32.667614+0000","last_undegraded":"2024-05-09T05:36:32.667614+0000","last_fullsized":"2024-05-09T05:36:32.667614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671820+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:36:32.671820+0000","last_peered":"2024-05-09T05:36:32.671820+0000","last_clean":"2024-05-09T05:36:32.671820+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:36:32.671820+0000","last_undegraded":"2024-05-09T05:36:32.671820+0000","last_fullsized":"2024-05-09T05:36:32.671820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213909,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71339,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1053,"num_read_kb":608,"num_write":257,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710769,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":496946,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907000,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":1148035,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070525,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":500860,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266754,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":497030,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462983,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1093780,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626509,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":492268,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":557683,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":493724,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":558435,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":491976,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986261,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":496342,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822738,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1089362,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:36:47.082 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T05:36:47.359 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:47.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:47 smithi115 ceph-mon[30349]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:36:47.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:47 smithi195 ceph-mon[33844]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:36:47.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:47 smithi161 ceph-mon[33805]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:36:48.559 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:48 smithi115 ceph-mon[30349]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:48.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:48 smithi195 ceph-mon[33844]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:48 smithi161 ceph-mon[33805]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:36:49.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.158+0000 7fc046b35700 1 -- 172.21.15.115:0/2680836872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 msgr2=0x7fc040104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:49.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.158+0000 7fc046b35700 1 --2- 172.21.15.115:0/2680836872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 0x7fc040104030 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc0380099c0 tx=0x7fc038009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.158+0000 7fc046b35700 1 -- 172.21.15.115:0/2680836872 shutdown_connections 2024-05-09T05:36:49.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.158+0000 7fc046b35700 1 --2- 172.21.15.115:0/2680836872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc040108770 0x7fc0401100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.158+0000 7fc046b35700 1 --2- 172.21.15.115:0/2680836872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 0x7fc040104030 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.158+0000 7fc046b35700 1 --2- 172.21.15.115:0/2680836872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc040107e20 0x7fc040108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.158+0000 7fc046b35700 1 -- 172.21.15.115:0/2680836872 >> 172.21.15.115:0/2680836872 conn(0x7fc0400fe6f0 msgr2=0x7fc040100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:49.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.158+0000 7fc046b35700 1 -- 172.21.15.115:0/2680836872 shutdown_connections 2024-05-09T05:36:49.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.158+0000 7fc046b35700 1 -- 172.21.15.115:0/2680836872 wait complete. 2024-05-09T05:36:49.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.159+0000 7fc046b35700 1 Processor -- start 2024-05-09T05:36:49.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.159+0000 7fc046b35700 1 -- start start 2024-05-09T05:36:49.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.159+0000 7fc046b35700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 0x7fc04019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:49.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.159+0000 7fc046b35700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc040107e20 0x7fc04019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:49.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.159+0000 7fc046b35700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc040108770 0x7fc04019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:49.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.159+0000 7fc046b35700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc040113250 con 0x7fc040103c20 2024-05-09T05:36:49.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.159+0000 7fc046b35700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc0401130d0 con 0x7fc040107e20 2024-05-09T05:36:49.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.159+0000 7fc046b35700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc0401133d0 con 0x7fc040108770 2024-05-09T05:36:49.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0448d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 0x7fc04019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:49.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0448d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 0x7fc04019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55396/0 (socket says 172.21.15.115:55396) 2024-05-09T05:36:49.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0448d1700 1 -- 172.21.15.115:0/184265312 learned_addr learned my addr 172.21.15.115:0/184265312 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0450d2700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc040108770 0x7fc04019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc03ffff700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc040107e20 0x7fc04019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0448d1700 1 -- 172.21.15.115:0/184265312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc040108770 msgr2=0x7fc04019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0448d1700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc040108770 0x7fc04019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0448d1700 1 -- 172.21.15.115:0/184265312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc040107e20 msgr2=0x7fc04019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0448d1700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc040107e20 0x7fc04019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0448d1700 1 -- 172.21.15.115:0/184265312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0401a6f30 con 0x7fc040103c20 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc03ffff700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc040107e20 0x7fc04019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0450d2700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc040108770 0x7fc04019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc0448d1700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 0x7fc04019bc10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc03800b580 tx=0x7fc03800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc03dffb700 1 -- 172.21.15.115:0/184265312 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038026070 con 0x7fc040103c20 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0401a71c0 con 0x7fc040103c20 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc03dffb700 1 -- 172.21.15.115:0/184265312 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc038005030 con 0x7fc040103c20 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.160+0000 7fc03dffb700 1 -- 172.21.15.115:0/184265312 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0380049f0 con 0x7fc040103c20 2024-05-09T05:36:49.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.161+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0401a7610 con 0x7fc040103c20 2024-05-09T05:36:49.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.162+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0400512c0 con 0x7fc040103c20 2024-05-09T05:36:49.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.163+0000 7fc03dffb700 1 -- 172.21.15.115:0/184265312 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc038029070 con 0x7fc040103c20 2024-05-09T05:36:49.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.164+0000 7fc03dffb700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc02c0b2880 0x7fc02c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:49.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.164+0000 7fc03ffff700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc02c0b2880 0x7fc02c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:49.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.165+0000 7fc03dffb700 1 -- 172.21.15.115:0/184265312 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fc0380e3370 con 0x7fc040103c20 2024-05-09T05:36:49.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.165+0000 7fc03ffff700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc02c0b2880 0x7fc02c0b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc040106b00 tx=0x7fc0300076a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:49.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.168+0000 7fc03dffb700 1 -- 172.21.15.115:0/184265312 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc038030080 con 0x7fc040103c20 2024-05-09T05:36:49.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.369+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc040063c00 con 0x7fc02c0b2880 2024-05-09T05:36:49.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.378+0000 7fc03dffb700 1 -- 172.21.15.115:0/184265312 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340742 (secure 0 0 0) 0x7fc040063c00 con 0x7fc02c0b2880 2024-05-09T05:36:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:36:49.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc02c0b2880 msgr2=0x7fc02c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:49.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc02c0b2880 0x7fc02c0b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc040106b00 tx=0x7fc0300076a0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 msgr2=0x7fc04019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:49.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 0x7fc04019bc10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc03800b580 tx=0x7fc03800f510 comp rx=0 tx=0).stop 2024-05-09T05:36:49.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 shutdown_connections 2024-05-09T05:36:49.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fc02c0b2880 0x7fc02c0b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc040108770 0x7fc04019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc040103c20 0x7fc04019bc10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 --2- 172.21.15.115:0/184265312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc040107e20 0x7fc04019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.382+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 >> 172.21.15.115:0/184265312 conn(0x7fc0400fe6f0 msgr2=0x7fc040100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.383+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 shutdown_connections 2024-05-09T05:36:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:49.383+0000 7fc046b35700 1 -- 172.21.15.115:0/184265312 wait complete. 2024-05-09T05:36:49.388 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:36:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-09T05:36:48.705539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661941,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1149,"num_read_kb":682,"num_write":382,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":815,"ondisk_log_size":815,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344340,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597484,"statfs":{"total":1151940427776,"available":1151587823616,"internally_reserved":0,"allocated":12111872,"data_stored":7916441,"data_compressed":426138,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003634"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673632+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:36:32.673632+0000","last_peered":"2024-05-09T05:36:32.673632+0000","last_clean":"2024-05-09T05:36:32.673632+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:36:32.673632+0000","last_undegraded":"2024-05-09T05:36:32.673632+0000","last_fullsized":"2024-05-09T05:36:32.673632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.668405+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:36:32.668405+0000","last_peered":"2024-05-09T05:36:32.668405+0000","last_clean":"2024-05-09T05:36:32.668405+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:36:32.668405+0000","last_undegraded":"2024-05-09T05:36:32.668405+0000","last_fullsized":"2024-05-09T05:36:32.668405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674286+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:36:32.674286+0000","last_peered":"2024-05-09T05:36:32.674286+0000","last_clean":"2024-05-09T05:36:32.674286+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:36:32.674286+0000","last_undegraded":"2024-05-09T05:36:32.674286+0000","last_fullsized":"2024-05-09T05:36:32.674286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.662136+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:36:32.662136+0000","last_peered":"2024-05-09T05:36:32.662136+0000","last_clean":"2024-05-09T05:36:32.662136+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:36:32.662136+0000","last_undegraded":"2024-05-09T05:36:32.662136+0000","last_fullsized":"2024-05-09T05:36:32.662136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.656869+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:36:32.656869+0000","last_peered":"2024-05-09T05:36:32.656869+0000","last_clean":"2024-05-09T05:36:32.656869+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:36:32.656869+0000","last_undegraded":"2024-05-09T05:36:32.656869+0000","last_fullsized":"2024-05-09T05:36:32.656869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.661485+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:36:32.661485+0000","last_peered":"2024-05-09T05:36:32.661485+0000","last_clean":"2024-05-09T05:36:32.661485+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:36:32.661485+0000","last_undegraded":"2024-05-09T05:36:32.661485+0000","last_fullsized":"2024-05-09T05:36:32.661485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.658544+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:36:32.658544+0000","last_peered":"2024-05-09T05:36:32.658544+0000","last_clean":"2024-05-09T05:36:32.658544+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:36:32.658544+0000","last_undegraded":"2024-05-09T05:36:32.658544+0000","last_fullsized":"2024-05-09T05:36:32.658544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.662921+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:36:32.662921+0000","last_peered":"2024-05-09T05:36:32.662921+0000","last_clean":"2024-05-09T05:36:32.662921+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:36:32.662921+0000","last_undegraded":"2024-05-09T05:36:32.662921+0000","last_fullsized":"2024-05-09T05:36:32.662921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674927+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:36:32.674927+0000","last_peered":"2024-05-09T05:36:32.674927+0000","last_clean":"2024-05-09T05:36:32.674927+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:36:32.674927+0000","last_undegraded":"2024-05-09T05:36:32.674927+0000","last_fullsized":"2024-05-09T05:36:32.674927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.663034+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:36:32.663034+0000","last_peered":"2024-05-09T05:36:32.663034+0000","last_clean":"2024-05-09T05:36:32.663034+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:36:32.663034+0000","last_undegraded":"2024-05-09T05:36:32.663034+0000","last_fullsized":"2024-05-09T05:36:32.663034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673050+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:36:32.673050+0000","last_peered":"2024-05-09T05:36:32.673050+0000","last_clean":"2024-05-09T05:36:32.673050+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:36:32.673050+0000","last_undegraded":"2024-05-09T05:36:32.673050+0000","last_fullsized":"2024-05-09T05:36:32.673050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.644278+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:36:32.644278+0000","last_peered":"2024-05-09T05:36:32.644278+0000","last_clean":"2024-05-09T05:36:32.644278+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:36:32.644278+0000","last_undegraded":"2024-05-09T05:36:32.644278+0000","last_fullsized":"2024-05-09T05:36:32.644278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.652656+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:36:32.652656+0000","last_peered":"2024-05-09T05:36:32.652656+0000","last_clean":"2024-05-09T05:36:32.652656+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:36:32.652656+0000","last_undegraded":"2024-05-09T05:36:32.652656+0000","last_fullsized":"2024-05-09T05:36:32.652656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670497+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:36:32.670497+0000","last_peered":"2024-05-09T05:36:32.670497+0000","last_clean":"2024-05-09T05:36:32.670497+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:36:32.670497+0000","last_undegraded":"2024-05-09T05:36:32.670497+0000","last_fullsized":"2024-05-09T05:36:32.670497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671339+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:36:32.671339+0000","last_peered":"2024-05-09T05:36:32.671339+0000","last_clean":"2024-05-09T05:36:32.671339+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:36:32.671339+0000","last_undegraded":"2024-05-09T05:36:32.671339+0000","last_fullsized":"2024-05-09T05:36:32.671339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673857+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:36:32.673857+0000","last_peered":"2024-05-09T05:36:32.673857+0000","last_clean":"2024-05-09T05:36:32.673857+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:36:32.673857+0000","last_undegraded":"2024-05-09T05:36:32.673857+0000","last_fullsized":"2024-05-09T05:36:32.673857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673738+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:36:32.673738+0000","last_peered":"2024-05-09T05:36:32.673738+0000","last_clean":"2024-05-09T05:36:32.673738+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:36:32.673738+0000","last_undegraded":"2024-05-09T05:36:32.673738+0000","last_fullsized":"2024-05-09T05:36:32.673738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.660165+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:36:32.660165+0000","last_peered":"2024-05-09T05:36:32.660165+0000","last_clean":"2024-05-09T05:36:32.660165+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:36:32.660165+0000","last_undegraded":"2024-05-09T05:36:32.660165+0000","last_fullsized":"2024-05-09T05:36:32.660165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670233+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:36:32.670233+0000","last_peered":"2024-05-09T05:36:32.670233+0000","last_clean":"2024-05-09T05:36:32.670233+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:36:32.670233+0000","last_undegraded":"2024-05-09T05:36:32.670233+0000","last_fullsized":"2024-05-09T05:36:32.670233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939702+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:34:33.939702+0000","last_peered":"2024-05-09T05:34:33.939702+0000","last_clean":"2024-05-09T05:34:33.939702+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:34:33.939702+0000","last_undegraded":"2024-05-09T05:34:33.939702+0000","last_fullsized":"2024-05-09T05:34:33.939702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":161,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:47.447202+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:36:47.447202+0000","last_peered":"2024-05-09T05:36:47.447202+0000","last_clean":"2024-05-09T05:36:47.447202+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:36:47.447202+0000","last_undegraded":"2024-05-09T05:36:47.447202+0000","last_fullsized":"2024-05-09T05:36:47.447202+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.657390+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:36:32.657390+0000","last_peered":"2024-05-09T05:36:32.657390+0000","last_clean":"2024-05-09T05:36:32.657390+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:36:32.657390+0000","last_undegraded":"2024-05-09T05:36:32.657390+0000","last_fullsized":"2024-05-09T05:36:32.657390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048456+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:34:34.048456+0000","last_peered":"2024-05-09T05:34:34.048456+0000","last_clean":"2024-05-09T05:34:34.048456+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:34:34.048456+0000","last_undegraded":"2024-05-09T05:34:34.048456+0000","last_fullsized":"2024-05-09T05:34:34.048456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:00.277761+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:35:00.277761+0000","last_peered":"2024-05-09T05:35:00.277761+0000","last_clean":"2024-05-09T05:35:00.277761+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:35:00.277761+0000","last_undegraded":"2024-05-09T05:35:00.277761+0000","last_fullsized":"2024-05-09T05:35:00.277761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674300+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:36:32.674300+0000","last_peered":"2024-05-09T05:36:32.674300+0000","last_clean":"2024-05-09T05:36:32.674300+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:36:32.674300+0000","last_undegraded":"2024-05-09T05:36:32.674300+0000","last_fullsized":"2024-05-09T05:36:32.674300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048916+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:34:34.048916+0000","last_peered":"2024-05-09T05:34:34.048916+0000","last_clean":"2024-05-09T05:34:34.048916+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:34:34.048916+0000","last_undegraded":"2024-05-09T05:34:34.048916+0000","last_fullsized":"2024-05-09T05:34:34.048916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:04.292236+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:35:04.292236+0000","last_peered":"2024-05-09T05:35:04.292236+0000","last_clean":"2024-05-09T05:35:04.292236+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:35:04.292236+0000","last_undegraded":"2024-05-09T05:35:04.292236+0000","last_fullsized":"2024-05-09T05:35:04.292236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.672949+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:36:32.672949+0000","last_peered":"2024-05-09T05:36:32.672949+0000","last_clean":"2024-05-09T05:36:32.672949+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:36:32.672949+0000","last_undegraded":"2024-05-09T05:36:32.672949+0000","last_fullsized":"2024-05-09T05:36:32.672949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048403+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:34:34.048403+0000","last_peered":"2024-05-09T05:34:34.048403+0000","last_clean":"2024-05-09T05:34:34.048403+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:34:34.048403+0000","last_undegraded":"2024-05-09T05:34:34.048403+0000","last_fullsized":"2024-05-09T05:34:34.048403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.304728+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:35:03.304728+0000","last_peered":"2024-05-09T05:35:03.304728+0000","last_clean":"2024-05-09T05:35:03.304728+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:35:03.304728+0000","last_undegraded":"2024-05-09T05:35:03.304728+0000","last_fullsized":"2024-05-09T05:35:03.304728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.657952+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:36:32.657952+0000","last_peered":"2024-05-09T05:36:32.657952+0000","last_clean":"2024-05-09T05:36:32.657952+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:36:32.657952+0000","last_undegraded":"2024-05-09T05:36:32.657952+0000","last_fullsized":"2024-05-09T05:36:32.657952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940095+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:34:33.940095+0000","last_peered":"2024-05-09T05:34:33.940095+0000","last_clean":"2024-05-09T05:34:33.940095+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:34:33.940095+0000","last_undegraded":"2024-05-09T05:34:33.940095+0000","last_fullsized":"2024-05-09T05:34:33.940095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:42.446183+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:36:42.446183+0000","last_peered":"2024-05-09T05:36:42.446183+0000","last_clean":"2024-05-09T05:36:42.446183+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:36:42.446183+0000","last_undegraded":"2024-05-09T05:36:42.446183+0000","last_fullsized":"2024-05-09T05:36:42.446183+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.657342+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:36:32.657342+0000","last_peered":"2024-05-09T05:36:32.657342+0000","last_clean":"2024-05-09T05:36:32.657342+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:36:32.657342+0000","last_undegraded":"2024-05-09T05:36:32.657342+0000","last_fullsized":"2024-05-09T05:36:32.657342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940225+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:34:33.940225+0000","last_peered":"2024-05-09T05:34:33.940225+0000","last_clean":"2024-05-09T05:34:33.940225+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:34:33.940225+0000","last_undegraded":"2024-05-09T05:34:33.940225+0000","last_fullsized":"2024-05-09T05:34:33.940225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.293766+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:35:01.293766+0000","last_peered":"2024-05-09T05:35:01.293766+0000","last_clean":"2024-05-09T05:35:01.293766+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:35:01.293766+0000","last_undegraded":"2024-05-09T05:35:01.293766+0000","last_fullsized":"2024-05-09T05:35:01.293766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.661859+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:36:32.661859+0000","last_peered":"2024-05-09T05:36:32.661859+0000","last_clean":"2024-05-09T05:36:32.661859+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:36:32.661859+0000","last_undegraded":"2024-05-09T05:36:32.661859+0000","last_fullsized":"2024-05-09T05:36:32.661859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.051637+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:34:34.051637+0000","last_peered":"2024-05-09T05:34:34.051637+0000","last_clean":"2024-05-09T05:34:34.051637+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:34:34.051637+0000","last_undegraded":"2024-05-09T05:34:34.051637+0000","last_fullsized":"2024-05-09T05:34:34.051637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296327+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:35:02.296327+0000","last_peered":"2024-05-09T05:35:02.296327+0000","last_clean":"2024-05-09T05:35:02.296327+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:35:02.296327+0000","last_undegraded":"2024-05-09T05:35:02.296327+0000","last_fullsized":"2024-05-09T05:35:02.296327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.665685+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:36:32.665685+0000","last_peered":"2024-05-09T05:36:32.665685+0000","last_clean":"2024-05-09T05:36:32.665685+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:36:32.665685+0000","last_undegraded":"2024-05-09T05:36:32.665685+0000","last_fullsized":"2024-05-09T05:36:32.665685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904186+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:34:33.904186+0000","last_peered":"2024-05-09T05:34:33.904186+0000","last_clean":"2024-05-09T05:34:33.904186+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:34:33.904186+0000","last_undegraded":"2024-05-09T05:34:33.904186+0000","last_fullsized":"2024-05-09T05:34:33.904186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.305110+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:35:03.305110+0000","last_peered":"2024-05-09T05:35:03.305110+0000","last_clean":"2024-05-09T05:35:03.305110+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:35:03.305110+0000","last_undegraded":"2024-05-09T05:35:03.305110+0000","last_fullsized":"2024-05-09T05:35:03.305110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.659743+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:36:32.659743+0000","last_peered":"2024-05-09T05:36:32.659743+0000","last_clean":"2024-05-09T05:36:32.659743+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:36:32.659743+0000","last_undegraded":"2024-05-09T05:36:32.659743+0000","last_fullsized":"2024-05-09T05:36:32.659743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048695+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:34:34.048695+0000","last_peered":"2024-05-09T05:34:34.048695+0000","last_clean":"2024-05-09T05:34:34.048695+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:34:34.048695+0000","last_undegraded":"2024-05-09T05:34:34.048695+0000","last_fullsized":"2024-05-09T05:34:34.048695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":249,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:44.449123+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:36:44.449123+0000","last_peered":"2024-05-09T05:36:44.449123+0000","last_clean":"2024-05-09T05:36:44.449123+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:36:44.449123+0000","last_undegraded":"2024-05-09T05:36:44.449123+0000","last_fullsized":"2024-05-09T05:36:44.449123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673353+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:36:32.673353+0000","last_peered":"2024-05-09T05:36:32.673353+0000","last_clean":"2024-05-09T05:36:32.673353+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:36:32.673353+0000","last_undegraded":"2024-05-09T05:36:32.673353+0000","last_fullsized":"2024-05-09T05:36:32.673353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907348+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:34:33.907348+0000","last_peered":"2024-05-09T05:34:33.907348+0000","last_clean":"2024-05-09T05:34:33.907348+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:34:33.907348+0000","last_undegraded":"2024-05-09T05:34:33.907348+0000","last_fullsized":"2024-05-09T05:34:33.907348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.298403+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:35:02.298403+0000","last_peered":"2024-05-09T05:35:02.298403+0000","last_clean":"2024-05-09T05:35:02.298403+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:35:02.298403+0000","last_undegraded":"2024-05-09T05:35:02.298403+0000","last_fullsized":"2024-05-09T05:35:02.298403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.666783+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:36:32.666783+0000","last_peered":"2024-05-09T05:36:32.666783+0000","last_clean":"2024-05-09T05:36:32.666783+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:36:32.666783+0000","last_undegraded":"2024-05-09T05:36:32.666783+0000","last_fullsized":"2024-05-09T05:36:32.666783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049137+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:34:34.049137+0000","last_peered":"2024-05-09T05:34:34.049137+0000","last_clean":"2024-05-09T05:34:34.049137+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:34:34.049137+0000","last_undegraded":"2024-05-09T05:34:34.049137+0000","last_fullsized":"2024-05-09T05:34:34.049137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.304949+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:35:03.304949+0000","last_peered":"2024-05-09T05:35:03.304949+0000","last_clean":"2024-05-09T05:35:03.304949+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:35:03.304949+0000","last_undegraded":"2024-05-09T05:35:03.304949+0000","last_fullsized":"2024-05-09T05:35:03.304949+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674668+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:36:32.674668+0000","last_peered":"2024-05-09T05:36:32.674668+0000","last_clean":"2024-05-09T05:36:32.674668+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:36:32.674668+0000","last_undegraded":"2024-05-09T05:36:32.674668+0000","last_fullsized":"2024-05-09T05:36:32.674668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906379+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:34:33.906379+0000","last_peered":"2024-05-09T05:34:33.906379+0000","last_clean":"2024-05-09T05:34:33.906379+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:34:33.906379+0000","last_undegraded":"2024-05-09T05:34:33.906379+0000","last_fullsized":"2024-05-09T05:34:33.906379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'39","reported_seq":291,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:43.463973+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:36:43.463973+0000","last_peered":"2024-05-09T05:36:43.463973+0000","last_clean":"2024-05-09T05:36:43.463973+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:36:43.463973+0000","last_undegraded":"2024-05-09T05:36:43.463973+0000","last_fullsized":"2024-05-09T05:36:43.463973+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":169,"num_write":41,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670390+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:36:32.670390+0000","last_peered":"2024-05-09T05:36:32.670390+0000","last_clean":"2024-05-09T05:36:32.670390+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:36:32.670390+0000","last_undegraded":"2024-05-09T05:36:32.670390+0000","last_fullsized":"2024-05-09T05:36:32.670390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.911562+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:34:33.911562+0000","last_peered":"2024-05-09T05:34:33.911562+0000","last_clean":"2024-05-09T05:34:33.911562+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:34:33.911562+0000","last_undegraded":"2024-05-09T05:34:33.911562+0000","last_fullsized":"2024-05-09T05:34:33.911562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301529+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:35:03.301529+0000","last_peered":"2024-05-09T05:35:03.301529+0000","last_clean":"2024-05-09T05:35:03.301529+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:35:03.301529+0000","last_undegraded":"2024-05-09T05:35:03.301529+0000","last_fullsized":"2024-05-09T05:35:03.301529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.667519+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:36:32.667519+0000","last_peered":"2024-05-09T05:36:32.667519+0000","last_clean":"2024-05-09T05:36:32.667519+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:36:32.667519+0000","last_undegraded":"2024-05-09T05:36:32.667519+0000","last_fullsized":"2024-05-09T05:36:32.667519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907888+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:34:33.907888+0000","last_peered":"2024-05-09T05:34:33.907888+0000","last_clean":"2024-05-09T05:34:33.907888+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:34:33.907888+0000","last_undegraded":"2024-05-09T05:34:33.907888+0000","last_fullsized":"2024-05-09T05:34:33.907888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301445+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:35:03.301445+0000","last_peered":"2024-05-09T05:35:03.301445+0000","last_clean":"2024-05-09T05:35:03.301445+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:35:03.301445+0000","last_undegraded":"2024-05-09T05:35:03.301445+0000","last_fullsized":"2024-05-09T05:35:03.301445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670024+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:36:32.670024+0000","last_peered":"2024-05-09T05:36:32.670024+0000","last_clean":"2024-05-09T05:36:32.670024+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:36:32.670024+0000","last_undegraded":"2024-05-09T05:36:32.670024+0000","last_fullsized":"2024-05-09T05:36:32.670024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.051709+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:34:34.051709+0000","last_peered":"2024-05-09T05:34:34.051709+0000","last_clean":"2024-05-09T05:34:34.051709+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:34:34.051709+0000","last_undegraded":"2024-05-09T05:34:34.051709+0000","last_fullsized":"2024-05-09T05:34:34.051709+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":153,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:46.446565+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:36:46.446565+0000","last_peered":"2024-05-09T05:36:46.446565+0000","last_clean":"2024-05-09T05:36:46.446565+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:36:46.446565+0000","last_undegraded":"2024-05-09T05:36:46.446565+0000","last_fullsized":"2024-05-09T05:36:46.446565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.656977+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:36:32.656977+0000","last_peered":"2024-05-09T05:36:32.656977+0000","last_clean":"2024-05-09T05:36:32.656977+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:36:32.656977+0000","last_undegraded":"2024-05-09T05:36:32.656977+0000","last_fullsized":"2024-05-09T05:36:32.656977+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048584+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:34:34.048584+0000","last_peered":"2024-05-09T05:34:34.048584+0000","last_clean":"2024-05-09T05:34:34.048584+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:34:34.048584+0000","last_undegraded":"2024-05-09T05:34:34.048584+0000","last_fullsized":"2024-05-09T05:34:34.048584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.302358+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:35:03.302358+0000","last_peered":"2024-05-09T05:35:03.302358+0000","last_clean":"2024-05-09T05:35:03.302358+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:35:03.302358+0000","last_undegraded":"2024-05-09T05:35:03.302358+0000","last_fullsized":"2024-05-09T05:35:03.302358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.652733+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:36:32.652733+0000","last_peered":"2024-05-09T05:36:32.652733+0000","last_clean":"2024-05-09T05:36:32.652733+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:36:32.652733+0000","last_undegraded":"2024-05-09T05:36:32.652733+0000","last_fullsized":"2024-05-09T05:36:32.652733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.910622+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:34:33.910622+0000","last_peered":"2024-05-09T05:34:33.910622+0000","last_clean":"2024-05-09T05:34:33.910622+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:34:33.910622+0000","last_undegraded":"2024-05-09T05:34:33.910622+0000","last_fullsized":"2024-05-09T05:34:33.910622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301008+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:35:03.301008+0000","last_peered":"2024-05-09T05:35:03.301008+0000","last_clean":"2024-05-09T05:35:03.301008+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:35:03.301008+0000","last_undegraded":"2024-05-09T05:35:03.301008+0000","last_fullsized":"2024-05-09T05:35:03.301008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.667110+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:36:32.667110+0000","last_peered":"2024-05-09T05:36:32.667110+0000","last_clean":"2024-05-09T05:36:32.667110+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:36:32.667110+0000","last_undegraded":"2024-05-09T05:36:32.667110+0000","last_fullsized":"2024-05-09T05:36:32.667110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.943171+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:34:33.943171+0000","last_peered":"2024-05-09T05:34:33.943171+0000","last_clean":"2024-05-09T05:34:33.943171+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:34:33.943171+0000","last_undegraded":"2024-05-09T05:34:33.943171+0000","last_fullsized":"2024-05-09T05:34:33.943171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:01.294078+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:35:01.294078+0000","last_peered":"2024-05-09T05:35:01.294078+0000","last_clean":"2024-05-09T05:35:01.294078+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:35:01.294078+0000","last_undegraded":"2024-05-09T05:35:01.294078+0000","last_fullsized":"2024-05-09T05:35:01.294078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.650831+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:36:32.650831+0000","last_peered":"2024-05-09T05:36:32.650831+0000","last_clean":"2024-05-09T05:36:32.650831+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:36:32.650831+0000","last_undegraded":"2024-05-09T05:36:32.650831+0000","last_fullsized":"2024-05-09T05:36:32.650831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048059+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:34:34.048059+0000","last_peered":"2024-05-09T05:34:34.048059+0000","last_clean":"2024-05-09T05:34:34.048059+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:34:34.048059+0000","last_undegraded":"2024-05-09T05:34:34.048059+0000","last_fullsized":"2024-05-09T05:34:34.048059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:42.440626+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:36:42.440626+0000","last_peered":"2024-05-09T05:36:42.440626+0000","last_clean":"2024-05-09T05:36:42.440626+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:36:42.440626+0000","last_undegraded":"2024-05-09T05:36:42.440626+0000","last_fullsized":"2024-05-09T05:36:42.440626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.660957+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:36:32.660957+0000","last_peered":"2024-05-09T05:36:32.660957+0000","last_clean":"2024-05-09T05:36:32.660957+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:36:32.660957+0000","last_undegraded":"2024-05-09T05:36:32.660957+0000","last_fullsized":"2024-05-09T05:36:32.660957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048446+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:34:34.048446+0000","last_peered":"2024-05-09T05:34:34.048446+0000","last_clean":"2024-05-09T05:34:34.048446+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:34:34.048446+0000","last_undegraded":"2024-05-09T05:34:34.048446+0000","last_fullsized":"2024-05-09T05:34:34.048446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671054+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:36:32.671054+0000","last_peered":"2024-05-09T05:36:32.671054+0000","last_clean":"2024-05-09T05:36:32.671054+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:36:32.671054+0000","last_undegraded":"2024-05-09T05:36:32.671054+0000","last_fullsized":"2024-05-09T05:36:32.671054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:43.441298+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:36:43.441298+0000","last_peered":"2024-05-09T05:36:43.441298+0000","last_clean":"2024-05-09T05:36:43.441298+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:36:43.441298+0000","last_undegraded":"2024-05-09T05:36:43.441298+0000","last_fullsized":"2024-05-09T05:36:43.441298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:11.405828+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:35:11.405828+0000","last_peered":"2024-05-09T05:35:11.405828+0000","last_clean":"2024-05-09T05:35:11.405828+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:35:11.405828+0000","last_undegraded":"2024-05-09T05:35:11.405828+0000","last_fullsized":"2024-05-09T05:35:11.405828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.658675+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:36:32.658675+0000","last_peered":"2024-05-09T05:36:32.658675+0000","last_clean":"2024-05-09T05:36:32.658675+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:36:32.658675+0000","last_undegraded":"2024-05-09T05:36:32.658675+0000","last_fullsized":"2024-05-09T05:36:32.658675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.302837+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:35:03.302837+0000","last_peered":"2024-05-09T05:35:03.302837+0000","last_clean":"2024-05-09T05:35:03.302837+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:35:03.302837+0000","last_undegraded":"2024-05-09T05:35:03.302837+0000","last_fullsized":"2024-05-09T05:35:03.302837+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904811+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:34:33.904811+0000","last_peered":"2024-05-09T05:34:33.904811+0000","last_clean":"2024-05-09T05:34:33.904811+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:34:33.904811+0000","last_undegraded":"2024-05-09T05:34:33.904811+0000","last_fullsized":"2024-05-09T05:34:33.904811+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.911155+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:34:33.911155+0000","last_peered":"2024-05-09T05:34:33.911155+0000","last_clean":"2024-05-09T05:34:33.911155+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:34:33.911155+0000","last_undegraded":"2024-05-09T05:34:33.911155+0000","last_fullsized":"2024-05-09T05:34:33.911155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.666413+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:36:32.666413+0000","last_peered":"2024-05-09T05:36:32.666413+0000","last_clean":"2024-05-09T05:36:32.666413+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:36:32.666413+0000","last_undegraded":"2024-05-09T05:36:32.666413+0000","last_fullsized":"2024-05-09T05:36:32.666413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.297586+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:35:02.297586+0000","last_peered":"2024-05-09T05:35:02.297586+0000","last_clean":"2024-05-09T05:35:02.297586+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:35:02.297586+0000","last_undegraded":"2024-05-09T05:35:02.297586+0000","last_fullsized":"2024-05-09T05:35:02.297586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.907587+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:34:33.907587+0000","last_peered":"2024-05-09T05:34:33.907587+0000","last_clean":"2024-05-09T05:34:33.907587+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:34:33.907587+0000","last_undegraded":"2024-05-09T05:34:33.907587+0000","last_fullsized":"2024-05-09T05:34:33.907587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.302434+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:35:03.302434+0000","last_peered":"2024-05-09T05:35:03.302434+0000","last_clean":"2024-05-09T05:35:03.302434+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:35:03.302434+0000","last_undegraded":"2024-05-09T05:35:03.302434+0000","last_fullsized":"2024-05-09T05:35:03.302434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.660776+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:36:32.660776+0000","last_peered":"2024-05-09T05:36:32.660776+0000","last_clean":"2024-05-09T05:36:32.660776+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:36:32.660776+0000","last_undegraded":"2024-05-09T05:36:32.660776+0000","last_fullsized":"2024-05-09T05:36:32.660776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.048464+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:34:34.048464+0000","last_peered":"2024-05-09T05:34:34.048464+0000","last_clean":"2024-05-09T05:34:34.048464+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:34:34.048464+0000","last_undegraded":"2024-05-09T05:34:34.048464+0000","last_fullsized":"2024-05-09T05:34:34.048464+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'17","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:03.595473+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:36:03.595473+0000","last_peered":"2024-05-09T05:36:03.595473+0000","last_clean":"2024-05-09T05:36:03.595473+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:36:03.595473+0000","last_undegraded":"2024-05-09T05:36:03.595473+0000","last_fullsized":"2024-05-09T05:36:03.595473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.667526+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:36:32.667526+0000","last_peered":"2024-05-09T05:36:32.667526+0000","last_clean":"2024-05-09T05:36:32.667526+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:36:32.667526+0000","last_undegraded":"2024-05-09T05:36:32.667526+0000","last_fullsized":"2024-05-09T05:36:32.667526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.939375+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:34:33.939375+0000","last_peered":"2024-05-09T05:34:33.939375+0000","last_clean":"2024-05-09T05:34:33.939375+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:34:33.939375+0000","last_undegraded":"2024-05-09T05:34:33.939375+0000","last_fullsized":"2024-05-09T05:34:33.939375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301938+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:35:03.301938+0000","last_peered":"2024-05-09T05:35:03.301938+0000","last_clean":"2024-05-09T05:35:03.301938+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:35:03.301938+0000","last_undegraded":"2024-05-09T05:35:03.301938+0000","last_fullsized":"2024-05-09T05:35:03.301938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.664081+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:36:32.664081+0000","last_peered":"2024-05-09T05:36:32.664081+0000","last_clean":"2024-05-09T05:36:32.664081+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:36:32.664081+0000","last_undegraded":"2024-05-09T05:36:32.664081+0000","last_fullsized":"2024-05-09T05:36:32.664081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049332+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:34:34.049332+0000","last_peered":"2024-05-09T05:34:34.049332+0000","last_clean":"2024-05-09T05:34:34.049332+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:34:34.049332+0000","last_undegraded":"2024-05-09T05:34:34.049332+0000","last_fullsized":"2024-05-09T05:34:34.049332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:59.271069+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:34:59.271069+0000","last_peered":"2024-05-09T05:34:59.271069+0000","last_clean":"2024-05-09T05:34:59.271069+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:34:59.271069+0000","last_undegraded":"2024-05-09T05:34:59.271069+0000","last_fullsized":"2024-05-09T05:34:59.271069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.625581+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:36:32.625581+0000","last_peered":"2024-05-09T05:36:32.625581+0000","last_clean":"2024-05-09T05:36:32.625581+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:36:32.625581+0000","last_undegraded":"2024-05-09T05:36:32.625581+0000","last_fullsized":"2024-05-09T05:36:32.625581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.906493+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:34:33.906493+0000","last_peered":"2024-05-09T05:34:33.906493+0000","last_clean":"2024-05-09T05:34:33.906493+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:34:33.906493+0000","last_undegraded":"2024-05-09T05:34:33.906493+0000","last_fullsized":"2024-05-09T05:34:33.906493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.297494+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:35:02.297494+0000","last_peered":"2024-05-09T05:35:02.297494+0000","last_clean":"2024-05-09T05:35:02.297494+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:35:02.297494+0000","last_undegraded":"2024-05-09T05:35:02.297494+0000","last_fullsized":"2024-05-09T05:35:02.297494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673393+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:36:32.673393+0000","last_peered":"2024-05-09T05:36:32.673393+0000","last_clean":"2024-05-09T05:36:32.673393+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:36:32.673393+0000","last_undegraded":"2024-05-09T05:36:32.673393+0000","last_fullsized":"2024-05-09T05:36:32.673393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:17.852895+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:35:17.852895+0000","last_peered":"2024-05-09T05:35:17.852895+0000","last_clean":"2024-05-09T05:35:17.852895+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:35:17.852895+0000","last_undegraded":"2024-05-09T05:35:17.852895+0000","last_fullsized":"2024-05-09T05:35:17.852895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.288715+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:35:02.288715+0000","last_peered":"2024-05-09T05:35:02.288715+0000","last_clean":"2024-05-09T05:35:02.288715+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:35:02.288715+0000","last_undegraded":"2024-05-09T05:35:02.288715+0000","last_fullsized":"2024-05-09T05:35:02.288715+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.664661+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:36:32.664661+0000","last_peered":"2024-05-09T05:36:32.664661+0000","last_clean":"2024-05-09T05:36:32.664661+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:36:32.664661+0000","last_undegraded":"2024-05-09T05:36:32.664661+0000","last_fullsized":"2024-05-09T05:36:32.664661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.049130+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:34:34.049130+0000","last_peered":"2024-05-09T05:34:34.049130+0000","last_clean":"2024-05-09T05:34:34.049130+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:34:34.049130+0000","last_undegraded":"2024-05-09T05:34:34.049130+0000","last_fullsized":"2024-05-09T05:34:34.049130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.300961+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:35:03.300961+0000","last_peered":"2024-05-09T05:35:03.300961+0000","last_clean":"2024-05-09T05:35:03.300961+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:35:03.300961+0000","last_undegraded":"2024-05-09T05:35:03.300961+0000","last_fullsized":"2024-05-09T05:35:03.300961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671748+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:36:32.671748+0000","last_peered":"2024-05-09T05:36:32.671748+0000","last_clean":"2024-05-09T05:36:32.671748+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:36:32.671748+0000","last_undegraded":"2024-05-09T05:36:32.671748+0000","last_fullsized":"2024-05-09T05:36:32.671748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.940189+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:34:33.940189+0000","last_peered":"2024-05-09T05:34:33.940189+0000","last_clean":"2024-05-09T05:34:33.940189+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:34:33.940189+0000","last_undegraded":"2024-05-09T05:34:33.940189+0000","last_fullsized":"2024-05-09T05:34:33.940189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:46.453103+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:36:46.453103+0000","last_peered":"2024-05-09T05:36:46.453103+0000","last_clean":"2024-05-09T05:36:46.453103+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:36:46.453103+0000","last_undegraded":"2024-05-09T05:36:46.453103+0000","last_fullsized":"2024-05-09T05:36:46.453103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.672363+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:36:32.672363+0000","last_peered":"2024-05-09T05:36:32.672363+0000","last_clean":"2024-05-09T05:36:32.672363+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:36:32.672363+0000","last_undegraded":"2024-05-09T05:36:32.672363+0000","last_fullsized":"2024-05-09T05:36:32.672363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:33.904191+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:34:33.904191+0000","last_peered":"2024-05-09T05:34:33.904191+0000","last_clean":"2024-05-09T05:34:33.904191+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:34:33.904191+0000","last_undegraded":"2024-05-09T05:34:33.904191+0000","last_fullsized":"2024-05-09T05:34:33.904191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:03.301029+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:35:03.301029+0000","last_peered":"2024-05-09T05:35:03.301029+0000","last_clean":"2024-05-09T05:35:03.301029+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:35:03.301029+0000","last_undegraded":"2024-05-09T05:35:03.301029+0000","last_fullsized":"2024-05-09T05:35:03.301029+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.652652+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:36:32.652652+0000","last_peered":"2024-05-09T05:36:32.652652+0000","last_clean":"2024-05-09T05:36:32.652652+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:36:32.652652+0000","last_undegraded":"2024-05-09T05:36:32.652652+0000","last_fullsized":"2024-05-09T05:36:32.652652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:34:34.435700+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:34:34.435700+0000","last_peered":"2024-05-09T05:34:34.435700+0000","last_clean":"2024-05-09T05:34:34.435700+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:34:34.435700+0000","last_undegraded":"2024-05-09T05:34:34.435700+0000","last_fullsized":"2024-05-09T05:34:34.435700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:35:02.296675+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:35:02.296675+0000","last_peered":"2024-05-09T05:35:02.296675+0000","last_clean":"2024-05-09T05:35:02.296675+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:35:02.296675+0000","last_undegraded":"2024-05-09T05:35:02.296675+0000","last_fullsized":"2024-05-09T05:35:02.296675+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673360+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:36:32.673360+0000","last_peered":"2024-05-09T05:36:32.673360+0000","last_clean":"2024-05-09T05:36:32.673360+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:36:32.673360+0000","last_undegraded":"2024-05-09T05:36:32.673360+0000","last_fullsized":"2024-05-09T05:36:32.673360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.673905+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:36:32.673905+0000","last_peered":"2024-05-09T05:36:32.673905+0000","last_clean":"2024-05-09T05:36:32.673905+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:36:32.673905+0000","last_undegraded":"2024-05-09T05:36:32.673905+0000","last_fullsized":"2024-05-09T05:36:32.673905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671679+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:36:32.671679+0000","last_peered":"2024-05-09T05:36:32.671679+0000","last_clean":"2024-05-09T05:36:32.671679+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:36:32.671679+0000","last_undegraded":"2024-05-09T05:36:32.671679+0000","last_fullsized":"2024-05-09T05:36:32.671679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670751+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:36:32.670751+0000","last_peered":"2024-05-09T05:36:32.670751+0000","last_clean":"2024-05-09T05:36:32.670751+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:36:32.670751+0000","last_undegraded":"2024-05-09T05:36:32.670751+0000","last_fullsized":"2024-05-09T05:36:32.670751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.662209+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:36:32.662209+0000","last_peered":"2024-05-09T05:36:32.662209+0000","last_clean":"2024-05-09T05:36:32.662209+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:36:32.662209+0000","last_undegraded":"2024-05-09T05:36:32.662209+0000","last_fullsized":"2024-05-09T05:36:32.662209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.655309+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:36:32.655309+0000","last_peered":"2024-05-09T05:36:32.655309+0000","last_clean":"2024-05-09T05:36:32.655309+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:36:32.655309+0000","last_undegraded":"2024-05-09T05:36:32.655309+0000","last_fullsized":"2024-05-09T05:36:32.655309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.668675+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:36:32.668675+0000","last_peered":"2024-05-09T05:36:32.668675+0000","last_clean":"2024-05-09T05:36:32.668675+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:36:32.668675+0000","last_undegraded":"2024-05-09T05:36:32.668675+0000","last_fullsized":"2024-05-09T05:36:32.668675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.669619+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:36:32.669619+0000","last_peered":"2024-05-09T05:36:32.669619+0000","last_clean":"2024-05-09T05:36:32.669619+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:36:32.669619+0000","last_undegraded":"2024-05-09T05:36:32.669619+0000","last_fullsized":"2024-05-09T05:36:32.669619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.650073+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:36:32.650073+0000","last_peered":"2024-05-09T05:36:32.650073+0000","last_clean":"2024-05-09T05:36:32.650073+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:36:32.650073+0000","last_undegraded":"2024-05-09T05:36:32.650073+0000","last_fullsized":"2024-05-09T05:36:32.650073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.662497+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:36:32.662497+0000","last_peered":"2024-05-09T05:36:32.662497+0000","last_clean":"2024-05-09T05:36:32.662497+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:36:32.662497+0000","last_undegraded":"2024-05-09T05:36:32.662497+0000","last_fullsized":"2024-05-09T05:36:32.662497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.670717+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:36:32.670717+0000","last_peered":"2024-05-09T05:36:32.670717+0000","last_clean":"2024-05-09T05:36:32.670717+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:36:32.670717+0000","last_undegraded":"2024-05-09T05:36:32.670717+0000","last_fullsized":"2024-05-09T05:36:32.670717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.674120+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:36:32.674120+0000","last_peered":"2024-05-09T05:36:32.674120+0000","last_clean":"2024-05-09T05:36:32.674120+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:36:32.674120+0000","last_undegraded":"2024-05-09T05:36:32.674120+0000","last_fullsized":"2024-05-09T05:36:32.674120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.667614+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:36:32.667614+0000","last_peered":"2024-05-09T05:36:32.667614+0000","last_clean":"2024-05-09T05:36:32.667614+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:36:32.667614+0000","last_undegraded":"2024-05-09T05:36:32.667614+0000","last_fullsized":"2024-05-09T05:36:32.667614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-09T05:36:32.671820+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:36:32.671820+0000","last_peered":"2024-05-09T05:36:32.671820+0000","last_clean":"2024-05-09T05:36:32.671820+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:36:32.671820+0000","last_undegraded":"2024-05-09T05:36:32.671820+0000","last_fullsized":"2024-05-09T05:36:32.671820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213909,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71339,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1068,"num_read_kb":613,"num_write":257,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710770,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":496946,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907000,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":1148035,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070526,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":500860,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266755,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":497030,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462983,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1093780,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626509,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":492268,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":557683,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":493724,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":558435,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":491976,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986262,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":496342,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822738,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1089362,"data_compressed":62453,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34299,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:36:50.212 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T05:36:50.212 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T05:36:50.212 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T05:36:50.212 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-09T05:36:50.458 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:50.580 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:50 smithi115 ceph-mon[30349]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:36:50.581 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:50 smithi115 ceph-mon[30349]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:36:50.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:50 smithi195 ceph-mon[33844]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:36:50.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:50 smithi195 ceph-mon[33844]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:36:50.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:50 smithi161 ceph-mon[33805]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:36:50.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:50 smithi161 ceph-mon[33805]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:36:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.326+0000 7f22159d5700 1 -- 172.21.15.115:0/1016284901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 msgr2=0x7f2210104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.326+0000 7f22159d5700 1 --2- 172.21.15.115:0/1016284901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 0x7f2210104070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f220000b0d0 tx=0x7f220000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.326+0000 7f22159d5700 1 -- 172.21.15.115:0/1016284901 shutdown_connections 2024-05-09T05:36:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.326+0000 7f22159d5700 1 --2- 172.21.15.115:0/1016284901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2210108750 0x7f22101100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.326+0000 7f22159d5700 1 --2- 172.21.15.115:0/1016284901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 0x7f2210104070 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.326+0000 7f22159d5700 1 --2- 172.21.15.115:0/1016284901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2210107e00 0x7f2210108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.326+0000 7f22159d5700 1 -- 172.21.15.115:0/1016284901 >> 172.21.15.115:0/1016284901 conn(0x7f22100fe6f0 msgr2=0x7f2210100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.326+0000 7f22159d5700 1 -- 172.21.15.115:0/1016284901 shutdown_connections 2024-05-09T05:36:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.326+0000 7f22159d5700 1 -- 172.21.15.115:0/1016284901 wait complete. 2024-05-09T05:36:52.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.327+0000 7f22159d5700 1 Processor -- start 2024-05-09T05:36:52.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.327+0000 7f22159d5700 1 -- start start 2024-05-09T05:36:52.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f22159d5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 0x7f221019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:52.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f22159d5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2210107e00 0x7f221019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:52.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f22159d5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2210108750 0x7f221019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:52.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f22159d5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f22101132a0 con 0x7f2210103c60 2024-05-09T05:36:52.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f22159d5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2210113120 con 0x7f2210107e00 2024-05-09T05:36:52.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f22159d5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2210113420 con 0x7f2210108750 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 0x7f221019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 0x7f221019ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55414/0 (socket says 172.21.15.115:55414) 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220effd700 1 -- 172.21.15.115:0/3758942284 learned_addr learned my addr 172.21.15.115:0/3758942284 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220e7fc700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2210107e00 0x7f221019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220f7fe700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2210108750 0x7f221019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220effd700 1 -- 172.21.15.115:0/3758942284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2210108750 msgr2=0x7f221019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220effd700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2210108750 0x7f221019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220effd700 1 -- 172.21.15.115:0/3758942284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2210107e00 msgr2=0x7f221019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220effd700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2210107e00 0x7f221019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220effd700 1 -- 172.21.15.115:0/3758942284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221019fc50 con 0x7f2210103c60 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f220effd700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 0x7f221019ad80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2200007a20 tx=0x7f2200007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.328+0000 7f22149d3700 1 -- 172.21.15.115:0/3758942284 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220001f070 con 0x7f2210103c60 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.329+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221019fee0 con 0x7f2210103c60 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.329+0000 7f22149d3700 1 -- 172.21.15.115:0/3758942284 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2200004c50 con 0x7f2210103c60 2024-05-09T05:36:52.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.329+0000 7f22149d3700 1 -- 172.21.15.115:0/3758942284 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220001b690 con 0x7f2210103c60 2024-05-09T05:36:52.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.329+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22101a03c0 con 0x7f2210103c60 2024-05-09T05:36:52.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.330+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22100512c0 con 0x7f2210103c60 2024-05-09T05:36:52.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.331+0000 7f22149d3700 1 -- 172.21.15.115:0/3758942284 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2200022070 con 0x7f2210103c60 2024-05-09T05:36:52.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.332+0000 7f22149d3700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f21fc0b2880 0x7f21fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:52.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.332+0000 7f220e7fc700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f21fc0b2880 0x7f21fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:52.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.332+0000 7f22149d3700 1 -- 172.21.15.115:0/3758942284 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f220000f020 con 0x7f2210103c60 2024-05-09T05:36:52.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.332+0000 7f220e7fc700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f21fc0b2880 0x7f21fc0b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2210109a60 tx=0x7f21f800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:52.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.336+0000 7f22149d3700 1 -- 172.21.15.115:0/3758942284 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2200029020 con 0x7f2210103c60 2024-05-09T05:36:52.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.585+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f221002e5f0 con 0x7f2210103c60 2024-05-09T05:36:52.580 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:52 smithi115 ceph-mon[30349]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T05:36:52.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.586+0000 7f22149d3700 1 -- 172.21.15.115:0/3758942284 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f22000ab4b0 con 0x7f2210103c60 2024-05-09T05:36:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:36:52.581 INFO:teuthology.orchestra.run.smithi115.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T05:36:52.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f21fc0b2880 msgr2=0x7f21fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:52.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f21fc0b2880 0x7f21fc0b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2210109a60 tx=0x7f21f800a300 comp rx=0 tx=0).stop 2024-05-09T05:36:52.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 msgr2=0x7f221019ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 0x7f221019ad80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2200007a20 tx=0x7f2200007a50 comp rx=0 tx=0).stop 2024-05-09T05:36:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 shutdown_connections 2024-05-09T05:36:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f21fc0b2880 0x7f21fc0b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2210108750 0x7f221019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2210103c60 0x7f221019ad80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 --2- 172.21.15.115:0/3758942284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2210107e00 0x7f221019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 >> 172.21.15.115:0/3758942284 conn(0x7f22100fe6f0 msgr2=0x7f2210102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 shutdown_connections 2024-05-09T05:36:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:52.589+0000 7f22159d5700 1 -- 172.21.15.115:0/3758942284 wait complete. 2024-05-09T05:36:52.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:52 smithi195 ceph-mon[33844]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T05:36:52.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:52 smithi161 ceph-mon[33805]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T05:36:53.394 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T05:36:53.395 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:36:53.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:53 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3758942284' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T05:36:53.639 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:53.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:53 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3758942284' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T05:36:53.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:53 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3758942284' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T05:36:54.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:54 smithi115 ceph-mon[30349]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-09T05:36:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:54 smithi195 ceph-mon[33844]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-09T05:36:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:54 smithi161 ceph-mon[33805]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-09T05:36:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.505+0000 7f8a683b9700 1 -- 172.21.15.115:0/984304236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a601045d0 msgr2=0x7f8a601049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.505+0000 7f8a683b9700 1 --2- 172.21.15.115:0/984304236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a601045d0 0x7f8a601049e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c0099c0 tx=0x7f8a5c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.505+0000 7f8a683b9700 1 -- 172.21.15.115:0/984304236 shutdown_connections 2024-05-09T05:36:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.505+0000 7f8a683b9700 1 --2- 172.21.15.115:0/984304236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a60109850 0x7f8a601111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.505+0000 7f8a683b9700 1 --2- 172.21.15.115:0/984304236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a601045d0 0x7f8a601049e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.505+0000 7f8a683b9700 1 --2- 172.21.15.115:0/984304236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a60108f00 0x7f8a60109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.505+0000 7f8a683b9700 1 -- 172.21.15.115:0/984304236 >> 172.21.15.115:0/984304236 conn(0x7f8a600ffea0 msgr2=0x7f8a601022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.505+0000 7f8a683b9700 1 -- 172.21.15.115:0/984304236 shutdown_connections 2024-05-09T05:36:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.505+0000 7f8a683b9700 1 -- 172.21.15.115:0/984304236 wait complete. 2024-05-09T05:36:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.506+0000 7f8a683b9700 1 Processor -- start 2024-05-09T05:36:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.506+0000 7f8a683b9700 1 -- start start 2024-05-09T05:36:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.506+0000 7f8a683b9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a601045d0 0x7f8a6019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.506+0000 7f8a683b9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a60108f00 0x7f8a6019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.506+0000 7f8a683b9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a60109850 0x7f8a6019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.506+0000 7f8a683b9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60114330 con 0x7f8a60109850 2024-05-09T05:36:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.506+0000 7f8a683b9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601141b0 con 0x7f8a601045d0 2024-05-09T05:36:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.506+0000 7f8a683b9700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601144b0 con 0x7f8a60108f00 2024-05-09T05:36:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.506+0000 7f8a66956700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a60109850 0x7f8a6019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a66956700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a60109850 0x7f8a6019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55870/0 (socket says 172.21.15.115:55870) 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a66956700 1 -- 172.21.15.115:0/1246116313 learned_addr learned my addr 172.21.15.115:0/1246116313 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a66155700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a601045d0 0x7f8a6019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a65954700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a60108f00 0x7f8a6019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a66956700 1 -- 172.21.15.115:0/1246116313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a60108f00 msgr2=0x7f8a6019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a66956700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a60108f00 0x7f8a6019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a66956700 1 -- 172.21.15.115:0/1246116313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a601045d0 msgr2=0x7f8a6019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a66956700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a601045d0 0x7f8a6019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a66956700 1 -- 172.21.15.115:0/1246116313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6019fb60 con 0x7f8a60109850 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a66956700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a60109850 0x7f8a6019f570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8a58007d70 tx=0x7f8a5800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a577fe700 1 -- 172.21.15.115:0/1246116313 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58016020 con 0x7f8a60109850 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a577fe700 1 -- 172.21.15.115:0/1246116313 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a5800ad40 con 0x7f8a60109850 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6019fe50 con 0x7f8a60109850 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a577fe700 1 -- 172.21.15.115:0/1246116313 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a580115e0 con 0x7f8a60109850 2024-05-09T05:36:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.507+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a601a0340 con 0x7f8a60109850 2024-05-09T05:36:55.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.508+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a600512c0 con 0x7f8a60109850 2024-05-09T05:36:55.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.510+0000 7f8a577fe700 1 -- 172.21.15.115:0/1246116313 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8a58019020 con 0x7f8a60109850 2024-05-09T05:36:55.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.510+0000 7f8a577fe700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8a4c0b2880 0x7f8a4c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:55.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.511+0000 7f8a66155700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8a4c0b2880 0x7f8a4c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:55.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.511+0000 7f8a577fe700 1 -- 172.21.15.115:0/1246116313 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f8a580e24a0 con 0x7f8a60109850 2024-05-09T05:36:55.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.511+0000 7f8a66155700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8a4c0b2880 0x7f8a4c0b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:55.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.515+0000 7f8a577fe700 1 -- 172.21.15.115:0/1246116313 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a58020020 con 0x7f8a60109850 2024-05-09T05:36:55.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.762+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a6002e5f0 con 0x7f8a60109850 2024-05-09T05:36:55.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.762+0000 7f8a577fe700 1 -- 172.21.15.115:0/1246116313 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7f8a580a9e60 con 0x7f8a60109850 2024-05-09T05:36:55.757 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:36:55.757 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:35:05.215676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.764+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8a4c0b2880 msgr2=0x7f8a4c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.764+0000 7f8a683b9700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8a4c0b2880 0x7f8a4c0b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c01a040 comp rx=0 tx=0).stop 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.764+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a60109850 msgr2=0x7f8a6019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.764+0000 7f8a683b9700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a60109850 0x7f8a6019f570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8a58007d70 tx=0x7f8a5800a340 comp rx=0 tx=0).stop 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.765+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 shutdown_connections 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.765+0000 7f8a683b9700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f8a4c0b2880 0x7f8a4c0b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.765+0000 7f8a683b9700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a601045d0 0x7f8a6019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.765+0000 7f8a683b9700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a60109850 0x7f8a6019f570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.765+0000 7f8a683b9700 1 --2- 172.21.15.115:0/1246116313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a60108f00 0x7f8a6019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.765+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 >> 172.21.15.115:0/1246116313 conn(0x7f8a600ffea0 msgr2=0x7f8a60102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:55.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.765+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 shutdown_connections 2024-05-09T05:36:55.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:55.765+0000 7f8a683b9700 1 -- 172.21.15.115:0/1246116313 wait complete. 2024-05-09T05:36:55.760 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 29 2024-05-09T05:36:56.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:56 smithi115 ceph-mon[30349]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-05-09T05:36:56.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:56 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1246116313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:36:56.572 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:36:56.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:56 smithi195 ceph-mon[33844]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-05-09T05:36:56.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:56 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1246116313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:36:56.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:56 smithi161 ceph-mon[33805]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-05-09T05:36:56.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:56 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1246116313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:36:56.824 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:36:58.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:58 smithi115 ceph-mon[30349]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-09T05:36:58.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:58 smithi195 ceph-mon[33844]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-09T05:36:58.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:58 smithi161 ceph-mon[33805]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-09T05:36:58.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.701+0000 7f69a4d78700 1 -- 172.21.15.115:0/2716539394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 msgr2=0x7f69a0109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:58.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.701+0000 7f69a4d78700 1 --2- 172.21.15.115:0/2716539394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 0x7f69a0109310 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f69900099c0 tx=0x7f6990009cd0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.701+0000 7f69a4d78700 1 -- 172.21.15.115:0/2716539394 shutdown_connections 2024-05-09T05:36:58.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.701+0000 7f69a4d78700 1 --2- 172.21.15.115:0/2716539394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69a0109850 0x7f69a01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.701+0000 7f69a4d78700 1 --2- 172.21.15.115:0/2716539394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69a01045d0 0x7f69a01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.701+0000 7f69a4d78700 1 --2- 172.21.15.115:0/2716539394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 0x7f69a0109310 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.701+0000 7f69a4d78700 1 -- 172.21.15.115:0/2716539394 >> 172.21.15.115:0/2716539394 conn(0x7f69a00ffea0 msgr2=0x7f69a01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:58.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.701+0000 7f69a4d78700 1 -- 172.21.15.115:0/2716539394 shutdown_connections 2024-05-09T05:36:58.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.701+0000 7f69a4d78700 1 -- 172.21.15.115:0/2716539394 wait complete. 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f69a4d78700 1 Processor -- start 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f69a4d78700 1 -- start start 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f69a4d78700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69a01045d0 0x7f69a019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f69a4d78700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 0x7f69a019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f69a4d78700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69a0109850 0x7f69a019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f69a4d78700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f69a0114310 con 0x7f69a0108f00 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f69a4d78700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69a0114190 con 0x7f69a0109850 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f69a4d78700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69a0114490 con 0x7f69a01045d0 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f699dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 0x7f69a019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f699dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 0x7f69a019b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55878/0 (socket says 172.21.15.115:55878) 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f699dd9b700 1 -- 172.21.15.115:0/3275424486 learned_addr learned my addr 172.21.15.115:0/3275424486 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:36:58.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f699e59c700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69a01045d0 0x7f69a019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.702+0000 7f699ed9d700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69a0109850 0x7f69a019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f699dd9b700 1 -- 172.21.15.115:0/3275424486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69a01045d0 msgr2=0x7f69a019ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f699dd9b700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69a01045d0 0x7f69a019ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f699dd9b700 1 -- 172.21.15.115:0/3275424486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69a0109850 msgr2=0x7f69a019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f699dd9b700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69a0109850 0x7f69a019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f699dd9b700 1 -- 172.21.15.115:0/3275424486 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a019fb40 con 0x7f69a0108f00 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f699ed9d700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69a0109850 0x7f69a019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f699dd9b700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 0x7f69a019b2c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f699000b580 tx=0x7f699000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f698f7fe700 1 -- 172.21.15.115:0/3275424486 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990027070 con 0x7f69a0108f00 2024-05-09T05:36:58.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a019fdd0 con 0x7f69a0108f00 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.703+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a01a0300 con 0x7f69a0108f00 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.704+0000 7f698f7fe700 1 -- 172.21.15.115:0/3275424486 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69900056d0 con 0x7f69a0108f00 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.704+0000 7f698f7fe700 1 -- 172.21.15.115:0/3275424486 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69900173f0 con 0x7f69a0108f00 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.705+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a00512c0 con 0x7f69a0108f00 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.707+0000 7f698f7fe700 1 -- 172.21.15.115:0/3275424486 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f699002a070 con 0x7f69a0108f00 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.708+0000 7f698f7fe700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f69880b2900 0x7f69880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.708+0000 7f699e59c700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f69880b2900 0x7f69880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.708+0000 7f698f7fe700 1 -- 172.21.15.115:0/3275424486 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f69900e31d0 con 0x7f69a0108f00 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.708+0000 7f699e59c700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f69880b2900 0x7f69880b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f699400ac50 tx=0x7f699400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:36:58.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.713+0000 7f698f7fe700 1 -- 172.21.15.115:0/3275424486 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6990013070 con 0x7f69a0108f00 2024-05-09T05:36:58.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.962+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69a019c080 con 0x7f69a0108f00 2024-05-09T05:36:58.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.963+0000 7f698f7fe700 1 -- 172.21.15.115:0/3275424486 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7f6990022470 con 0x7f69a0108f00 2024-05-09T05:36:58.958 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:36:58.958 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:35:05.215676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.965+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f69880b2900 msgr2=0x7f69880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.965+0000 7f69a4d78700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f69880b2900 0x7f69880b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f699400ac50 tx=0x7f699400a380 comp rx=0 tx=0).stop 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.965+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 msgr2=0x7f69a019b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.965+0000 7f69a4d78700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 0x7f69a019b2c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f699000b580 tx=0x7f699000f690 comp rx=0 tx=0).stop 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.965+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 shutdown_connections 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.965+0000 7f69a4d78700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f69880b2900 0x7f69880b4e10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.965+0000 7f69a4d78700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69a0109850 0x7f69a019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.966+0000 7f69a4d78700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69a01045d0 0x7f69a019ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.966+0000 7f69a4d78700 1 --2- 172.21.15.115:0/3275424486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69a0108f00 0x7f69a019b2c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.966+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 >> 172.21.15.115:0/3275424486 conn(0x7f69a00ffea0 msgr2=0x7f69a01022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.966+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 shutdown_connections 2024-05-09T05:36:58.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:36:58.966+0000 7f69a4d78700 1 -- 172.21.15.115:0/3275424486 wait complete. 2024-05-09T05:36:58.961 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 29 2024-05-09T05:36:59.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:36:59 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3275424486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:36:59.638 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:36:59 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3275424486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:36:59.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:36:59 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3275424486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:36:59.668 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:36:59.913 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:00.301 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:00 smithi115 ceph-mon[30349]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T05:37:00.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:00 smithi195 ceph-mon[33844]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T05:37:00.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:00 smithi161 ceph-mon[33805]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-09T05:37:01.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.830+0000 7f877c6fb700 1 -- 172.21.15.115:0/2254029946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8774107e10 msgr2=0x7f8774108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:01.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.830+0000 7f877c6fb700 1 --2- 172.21.15.115:0/2254029946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8774107e10 0x7f8774108220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f87640099c0 tx=0x7f8764009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.830+0000 7f877c6fb700 1 -- 172.21.15.115:0/2254029946 shutdown_connections 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.830+0000 7f877c6fb700 1 --2- 172.21.15.115:0/2254029946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87741034e0 0x7f87741038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.830+0000 7f877c6fb700 1 --2- 172.21.15.115:0/2254029946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8774108760 0x7f87741100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.830+0000 7f877c6fb700 1 --2- 172.21.15.115:0/2254029946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8774107e10 0x7f8774108220 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.830+0000 7f877c6fb700 1 -- 172.21.15.115:0/2254029946 >> 172.21.15.115:0/2254029946 conn(0x7f87740fef30 msgr2=0x7f8774101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.830+0000 7f877c6fb700 1 -- 172.21.15.115:0/2254029946 shutdown_connections 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.830+0000 7f877c6fb700 1 -- 172.21.15.115:0/2254029946 wait complete. 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877c6fb700 1 Processor -- start 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877c6fb700 1 -- start start 2024-05-09T05:37:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877c6fb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87741034e0 0x7f877419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877c6fb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8774107e10 0x7f877419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877c6fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8774108760 0x7f877419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877c6fb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8774114360 con 0x7f87741034e0 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877c6fb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f87741141e0 con 0x7f8774108760 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877c6fb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87741144e0 con 0x7f8774107e10 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877a497700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87741034e0 0x7f877419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877a497700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87741034e0 0x7f877419ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55906/0 (socket says 172.21.15.115:55906) 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877a497700 1 -- 172.21.15.115:0/537952201 learned_addr learned my addr 172.21.15.115:0/537952201 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f877ac98700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8774108760 0x7f877419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.831+0000 7f8779c96700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8774107e10 0x7f877419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:01.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f877a497700 1 -- 172.21.15.115:0/537952201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8774107e10 msgr2=0x7f877419b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:01.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f877a497700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8774107e10 0x7f877419b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f877a497700 1 -- 172.21.15.115:0/537952201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8774108760 msgr2=0x7f877419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f877a497700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8774108760 0x7f877419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f877a497700 1 -- 172.21.15.115:0/537952201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877419fc40 con 0x7f87741034e0 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f877ac98700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8774108760 0x7f877419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f877a497700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87741034e0 0x7f877419ad70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f877000c920 tx=0x7f877000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f876b7fe700 1 -- 172.21.15.115:0/537952201 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770004a30 con 0x7f87741034e0 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f876b7fe700 1 -- 172.21.15.115:0/537952201 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87700054b0 con 0x7f87741034e0 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877419ff30 con 0x7f87741034e0 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f876b7fe700 1 -- 172.21.15.115:0/537952201 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770007b70 con 0x7f87741034e0 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.832+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87741a7850 con 0x7f87741034e0 2024-05-09T05:37:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.833+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87740512c0 con 0x7f87741034e0 2024-05-09T05:37:01.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.836+0000 7f876b7fe700 1 -- 172.21.15.115:0/537952201 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8770004bd0 con 0x7f87741034e0 2024-05-09T05:37:01.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.836+0000 7f876b7fe700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f87600b2900 0x7f87600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:01.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.837+0000 7f8779c96700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f87600b2900 0x7f87600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:01.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.837+0000 7f876b7fe700 1 -- 172.21.15.115:0/537952201 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f87700e2320 con 0x7f87741034e0 2024-05-09T05:37:01.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.837+0000 7f8779c96700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f87600b2900 0x7f87600b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8764009990 tx=0x7f8764005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:01.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:01.840+0000 7f876b7fe700 1 -- 172.21.15.115:0/537952201 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8770014030 con 0x7f87741034e0 2024-05-09T05:37:02.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.046+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f877419c100 con 0x7f87741034e0 2024-05-09T05:37:02.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.047+0000 7f876b7fe700 1 -- 172.21.15.115:0/537952201 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24724 (secure 0 0 0) 0x7f877419c100 con 0x7f87741034e0 2024-05-09T05:37:02.042 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:37:02.042 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:33.901006+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:37:02.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f87600b2900 msgr2=0x7f87600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:02.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f87600b2900 0x7f87600b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8764009990 tx=0x7f8764005fb0 comp rx=0 tx=0).stop 2024-05-09T05:37:02.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87741034e0 msgr2=0x7f877419ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:02.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87741034e0 0x7f877419ad70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f877000c920 tx=0x7f877000cc30 comp rx=0 tx=0).stop 2024-05-09T05:37:02.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 shutdown_connections 2024-05-09T05:37:02.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f87600b2900 0x7f87600b4e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:02.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8774108760 0x7f877419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:02.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87741034e0 0x7f877419ad70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:02.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 --2- 172.21.15.115:0/537952201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8774107e10 0x7f877419b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:02.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 >> 172.21.15.115:0/537952201 conn(0x7f87740fef30 msgr2=0x7f87741096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:02.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 shutdown_connections 2024-05-09T05:37:02.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:02.050+0000 7f877c6fb700 1 -- 172.21.15.115:0/537952201 wait complete. 2024-05-09T05:37:02.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:02 smithi195 ceph-mon[33844]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T05:37:02.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:02 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/537952201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:37:02.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:02 smithi115 ceph-mon[30349]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T05:37:02.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:02 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/537952201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:37:02.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:02 smithi161 ceph-mon[33805]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T05:37:02.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:02 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/537952201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:37:02.752 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:37:02.995 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:04.515 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:04 smithi115 ceph-mon[30349]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:04.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:04 smithi195 ceph-mon[33844]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:04.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:04 smithi161 ceph-mon[33805]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:04.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.782+0000 7f9d5c92e700 1 -- 172.21.15.115:0/1175839383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d58108f40 msgr2=0x7f9d58109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:04.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.782+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/1175839383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d58108f40 0x7f9d58109350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0099c0 tx=0x7f9d4c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:04.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.782+0000 7f9d5c92e700 1 -- 172.21.15.115:0/1175839383 shutdown_connections 2024-05-09T05:37:04.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.782+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/1175839383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d58104610 0x7f9d58104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:04.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.782+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/1175839383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d58109890 0x7f9d581111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:04.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.782+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/1175839383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d58108f40 0x7f9d58109350 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:04.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.782+0000 7f9d5c92e700 1 -- 172.21.15.115:0/1175839383 >> 172.21.15.115:0/1175839383 conn(0x7f9d5807af70 msgr2=0x7f9d5807b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:04.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.782+0000 7f9d5c92e700 1 -- 172.21.15.115:0/1175839383 shutdown_connections 2024-05-09T05:37:04.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.782+0000 7f9d5c92e700 1 -- 172.21.15.115:0/1175839383 wait complete. 2024-05-09T05:37:04.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d5c92e700 1 Processor -- start 2024-05-09T05:37:04.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d5c92e700 1 -- start start 2024-05-09T05:37:04.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d5c92e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d58104610 0x7f9d58075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:04.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d5c92e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d58108f40 0x7f9d58076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:04.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d5c92e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d58109890 0x7f9d58076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:04.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d5c92e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58114350 con 0x7f9d58104610 2024-05-09T05:37:04.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d5c92e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581141d0 con 0x7f9d58109890 2024-05-09T05:37:04.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d5c92e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581144d0 con 0x7f9d58108f40 2024-05-09T05:37:04.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d55d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d58108f40 0x7f9d58076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:04.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d5659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d58104610 0x7f9d58075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:04.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d55d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d58108f40 0x7f9d58076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:37276/0 (socket says 172.21.15.115:37276) 2024-05-09T05:37:04.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d55d9b700 1 -- 172.21.15.115:0/2760098143 learned_addr learned my addr 172.21.15.115:0/2760098143 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:04.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.783+0000 7f9d56d9d700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d58109890 0x7f9d58076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:04.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.784+0000 7f9d5659c700 1 -- 172.21.15.115:0/2760098143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d58108f40 msgr2=0x7f9d58076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:04.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.784+0000 7f9d5659c700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d58108f40 0x7f9d58076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:04.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.784+0000 7f9d5659c700 1 -- 172.21.15.115:0/2760098143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d58109890 msgr2=0x7f9d58076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:04.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.784+0000 7f9d5659c700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d58109890 0x7f9d58076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:04.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.784+0000 7f9d5659c700 1 -- 172.21.15.115:0/2760098143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d581a9100 con 0x7f9d58104610 2024-05-09T05:37:04.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.784+0000 7f9d5659c700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d58104610 0x7f9d58075ac0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800d750 tx=0x7f9d4800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:04.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.785+0000 7f9d477fe700 1 -- 172.21.15.115:0/2760098143 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48014030 con 0x7f9d58104610 2024-05-09T05:37:04.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.785+0000 7f9d477fe700 1 -- 172.21.15.115:0/2760098143 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d480044b0 con 0x7f9d58104610 2024-05-09T05:37:04.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.785+0000 7f9d477fe700 1 -- 172.21.15.115:0/2760098143 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48005240 con 0x7f9d58104610 2024-05-09T05:37:04.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.785+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5819ab20 con 0x7f9d58104610 2024-05-09T05:37:04.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.786+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5819b060 con 0x7f9d58104610 2024-05-09T05:37:04.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.786+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d580512c0 con 0x7f9d58104610 2024-05-09T05:37:04.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.789+0000 7f9d477fe700 1 -- 172.21.15.115:0/2760098143 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9d480049e0 con 0x7f9d58104610 2024-05-09T05:37:04.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.790+0000 7f9d477fe700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9d400b29d0 0x7f9d400b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:04.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.790+0000 7f9d55d9b700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9d400b29d0 0x7f9d400b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:04.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.790+0000 7f9d477fe700 1 -- 172.21.15.115:0/2760098143 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9d480e2050 con 0x7f9d58104610 2024-05-09T05:37:04.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.790+0000 7f9d55d9b700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9d400b29d0 0x7f9d400b4ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9d58101010 tx=0x7f9d4c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:04.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:04.793+0000 7f9d477fe700 1 -- 172.21.15.115:0/2760098143 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d480a9a90 con 0x7f9d58104610 2024-05-09T05:37:05.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.043+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d5802e710 con 0x7f9d58104610 2024-05-09T05:37:05.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.044+0000 7f9d477fe700 1 -- 172.21.15.115:0/2760098143 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7f9d480a9390 con 0x7f9d58104610 2024-05-09T05:37:05.038 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:37:05.038 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:35:05.215676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:37:05.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9d400b29d0 msgr2=0x7f9d400b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:05.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9d400b29d0 0x7f9d400b4ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9d58101010 tx=0x7f9d4c01a040 comp rx=0 tx=0).stop 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d58104610 msgr2=0x7f9d58075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d58104610 0x7f9d58075ac0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800d750 tx=0x7f9d4800da60 comp rx=0 tx=0).stop 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 shutdown_connections 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9d400b29d0 0x7f9d400b4ee0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d58109890 0x7f9d58076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d58104610 0x7f9d58075ac0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 --2- 172.21.15.115:0/2760098143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d58108f40 0x7f9d58076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 >> 172.21.15.115:0/2760098143 conn(0x7f9d5807af70 msgr2=0x7f9d58101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 shutdown_connections 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:05.046+0000 7f9d5c92e700 1 -- 172.21.15.115:0/2760098143 wait complete. 2024-05-09T05:37:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 29 2024-05-09T05:37:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:05 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2760098143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:37:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:05 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2760098143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:37:05.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:05 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2760098143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:37:05.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T05:33:54.732251+0000', 'modified': '2024-05-09T05:35:05.215676+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24469, 'mds_1': 24457, 'mds_2': 24451, 'mds_3': 24440, 'mds_4': 24425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6839/3801478957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 3801478957}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 3801478957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6841/1081505697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6840', 'nonce': 1081505697}, {'type': 'v1', 'addr': '172.21.15.115:6841', 'nonce': 1081505697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24386': {'gid': 24386, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6833/634985009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 634985009}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 634985009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/1779202002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 1779202002}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 1779202002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6833/2784864553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6832', 'nonce': 2784864553}, {'type': 'v1', 'addr': '172.21.15.195:6833', 'nonce': 2784864553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.161:6837/3591780613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3591780613}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3591780613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'b', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.161:6839/234220396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 234220396}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 234220396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.195:6835/969435997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 969435997}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 969435997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.195:6837/4026198088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 4026198088}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 4026198088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.195:6839/1691623113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 1691623113}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 1691623113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24469, 'qdb_cluster': [24425, 24440, 24451, 24457, 24469]} 2024-05-09T05:37:05.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T05:37:05.755 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T05:37:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:06.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:06 smithi115 ceph-mon[30349]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:06.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:06 smithi195 ceph-mon[33844]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:06.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:06 smithi161 ceph-mon[33805]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:07.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.741+0000 7fd566431700 1 -- 172.21.15.115:0/4190201383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5601034c0 msgr2=0x7fd5601038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:07.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.741+0000 7fd566431700 1 --2- 172.21.15.115:0/4190201383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5601034c0 0x7fd5601038d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd5500099c0 tx=0x7fd550009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.741+0000 7fd566431700 1 -- 172.21.15.115:0/4190201383 shutdown_connections 2024-05-09T05:37:07.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.741+0000 7fd566431700 1 --2- 172.21.15.115:0/4190201383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560108740 0x7fd5601100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.741+0000 7fd566431700 1 --2- 172.21.15.115:0/4190201383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5601034c0 0x7fd5601038d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.741+0000 7fd566431700 1 --2- 172.21.15.115:0/4190201383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd560107df0 0x7fd560108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.741+0000 7fd566431700 1 -- 172.21.15.115:0/4190201383 >> 172.21.15.115:0/4190201383 conn(0x7fd5600fedf0 msgr2=0x7fd560101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:07.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.741+0000 7fd566431700 1 -- 172.21.15.115:0/4190201383 shutdown_connections 2024-05-09T05:37:07.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.741+0000 7fd566431700 1 -- 172.21.15.115:0/4190201383 wait complete. 2024-05-09T05:37:07.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.742+0000 7fd566431700 1 Processor -- start 2024-05-09T05:37:07.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.742+0000 7fd566431700 1 -- start start 2024-05-09T05:37:07.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.742+0000 7fd566431700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5601034c0 0x7fd56019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.742+0000 7fd566431700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560107df0 0x7fd56019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.742+0000 7fd566431700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd560108740 0x7fd560199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.742+0000 7fd566431700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601132a0 con 0x7fd560108740 2024-05-09T05:37:07.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.742+0000 7fd566431700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd560113120 con 0x7fd560107df0 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.742+0000 7fd566431700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd560113420 con 0x7fd5601034c0 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.742+0000 7fd5649ce700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd560108740 0x7fd560199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd5649ce700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd560108740 0x7fd560199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40078/0 (socket says 172.21.15.115:40078) 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd5649ce700 1 -- 172.21.15.115:0/3237837553 learned_addr learned my addr 172.21.15.115:0/3237837553 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd55ffff700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5601034c0 0x7fd56019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560107df0 0x7fd56019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd5649ce700 1 -- 172.21.15.115:0/3237837553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5601034c0 msgr2=0x7fd56019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd5649ce700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5601034c0 0x7fd56019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd5649ce700 1 -- 172.21.15.115:0/3237837553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560107df0 msgr2=0x7fd56019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd5649ce700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560107df0 0x7fd56019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd5649ce700 1 -- 172.21.15.115:0/3237837553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56019a1c0 con 0x7fd560108740 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd55ffff700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5601034c0 0x7fd56019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560107df0 0x7fd56019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:07.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd5649ce700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd560108740 0x7fd560199ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd55800b970 tx=0x7fd55800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:07.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd55d7fa700 1 -- 172.21.15.115:0/3237837553 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558019070 con 0x7fd560108740 2024-05-09T05:37:07.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd55d7fa700 1 -- 172.21.15.115:0/3237837553 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd55800ce40 con 0x7fd560108740 2024-05-09T05:37:07.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd566431700 1 -- 172.21.15.115:0/3237837553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56019a4b0 con 0x7fd560108740 2024-05-09T05:37:07.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd55d7fa700 1 -- 172.21.15.115:0/3237837553 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558007370 con 0x7fd560108740 2024-05-09T05:37:07.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.743+0000 7fd566431700 1 -- 172.21.15.115:0/3237837553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5600759f0 con 0x7fd560108740 2024-05-09T05:37:07.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.746+0000 7fd55d7fa700 1 -- 172.21.15.115:0/3237837553 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd55801c070 con 0x7fd560108740 2024-05-09T05:37:07.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.746+0000 7fd55d7fa700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd5480b2880 0x7fd5480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd55ffff700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd5480b2880 0x7fd5480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd55d7fa700 1 -- 172.21.15.115:0/3237837553 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd5580e1b70 con 0x7fd560108740 2024-05-09T05:37:07.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd55ffff700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd5480b2880 0x7fd5480b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd55001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:07.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd574e47b80 1 Processor -- start 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd574e47b80 1 -- start start 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c9646d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c96472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c964b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd574e47b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b4c9564fa0 con 0x55b4c94c5450 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd574e47b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4c95037a0 con 0x55b4c95af110 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.747+0000 7fd574e47b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b4c9560f60 con 0x55b4c964b3b0 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c96472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c96472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40090/0 (socket says 172.21.15.115:40090) 2024-05-09T05:37:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55f7fe700 1 -- 172.21.15.115:0/3047906456 learned_addr learned my addr 172.21.15.115:0/3047906456 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd5649ce700 1 --2- 172.21.15.115:0/3047906456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c964b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55ffff700 1 --2- 172.21.15.115:0/3047906456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c9646d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55f7fe700 1 -- 172.21.15.115:0/3047906456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 msgr2=0x55b4c964b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/3047906456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c964b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55f7fe700 1 -- 172.21.15.115:0/3047906456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 msgr2=0x55b4c9646d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/3047906456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c9646d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55f7fe700 1 -- 172.21.15.115:0/3047906456 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c94fb320 con 0x55b4c94c5450 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/3047906456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c96472a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd55400bd30 tx=0x7fd55400f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b059cff8f01f78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd5467fc700 1 -- 172.21.15.115:0/3047906456 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55400fda0 con 0x55b4c94c5450 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd5467fc700 1 -- 172.21.15.115:0/3047906456 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5540118d0 con 0x55b4c94c5450 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.748+0000 7fd5467fc700 1 -- 172.21.15.115:0/3047906456 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55401b650 con 0x55b4c94c5450 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.749+0000 7fd574e47b80 1 -- 172.21.15.115:0/3047906456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 msgr2=0x55b4c96472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.749+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3047906456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c96472a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd55400bd30 tx=0x7fd55400f640 comp rx=0 tx=0).stop 2024-05-09T05:37:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.749+0000 7fd574e47b80 1 -- 172.21.15.115:0/3047906456 shutdown_connections 2024-05-09T05:37:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.749+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3047906456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c9646d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.749+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3047906456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c964b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.749+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3047906456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c96472a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.749+0000 7fd574e47b80 1 -- 172.21.15.115:0/3047906456 >> 172.21.15.115:0/3047906456 conn(0x55b4c9525650 msgr2=0x55b4c963d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.749+0000 7fd574e47b80 1 -- 172.21.15.115:0/3047906456 shutdown_connections 2024-05-09T05:37:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.749+0000 7fd574e47b80 1 -- 172.21.15.115:0/3047906456 wait complete. 2024-05-09T05:37:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd574e47b80 1 -- start start 2024-05-09T05:37:07.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd574e47b80 1 Processor -- start 2024-05-09T05:37:07.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c94c5450 0x55b4c96505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c95af110 0x55b4c9650b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c964b3b0 0x55b4c9651070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd574e47b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b4c93e0b50 con 0x55b4c95af110 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd574e47b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4c951f100 con 0x55b4c964b3b0 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd574e47b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b4c964c160 con 0x55b4c94c5450 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd574e47b80 10 client.0 _ll_register_callbacks cb 0x55b4c93d6670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c95af110 0x55b4c9650b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c95af110 0x55b4c9650b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40098/0 (socket says 172.21.15.115:40098) 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.750+0000 7fd55f7fe700 1 -- 172.21.15.115:0/2994758985 learned_addr learned my addr 172.21.15.115:0/2994758985 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd55ffff700 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c94c5450 0x55b4c96505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd5649ce700 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c964b3b0 0x55b4c9651070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd55f7fe700 1 -- 172.21.15.115:0/2994758985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c94c5450 msgr2=0x55b4c96505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c94c5450 0x55b4c96505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd55f7fe700 1 -- 172.21.15.115:0/2994758985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c964b3b0 msgr2=0x55b4c9651070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c964b3b0 0x55b4c9651070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd55f7fe700 1 -- 172.21.15.115:0/2994758985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c93e0440 con 0x55b4c95af110 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd5649ce700 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c964b3b0 0x55b4c9651070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c95af110 0x55b4c9650b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd554005c50 tx=0x7fd554011840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdb3a4cdc380553 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd53f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd53f7fe700 1 -- 172.21.15.115:0/2994758985 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55400fd50 con 0x55b4c95af110 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd53f7fe700 1 -- 172.21.15.115:0/2994758985 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd554011a80 con 0x55b4c95af110 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd574e47b80 1 -- 172.21.15.115:0/2994758985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5540118d0 con 0x55b4c95af110 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd53f7fe700 1 -- 172.21.15.115:0/2994758985 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd554031050 con 0x55b4c95af110 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd574e47b80 10 client.15075 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.751+0000 7fd574e47b80 1 -- 172.21.15.115:0/2994758985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4c965dee0 con 0x55b4c95af110 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.752+0000 7fd53f7fe700 1 -- 172.21.15.115:0/2994758985 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5540118d0 con 0x55b4c95af110 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.752+0000 7fd53f7fe700 1 -- 172.21.15.115:0/2994758985 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd554004830 con 0x55b4c95af110 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.752+0000 7fd574e47b80 10 client.15075 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.752+0000 7fd574e47b80 10 client.15075 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.752+0000 7fd574e47b80 20 client.15075 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.752+0000 7fd574e47b80 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b4c96600e0 0x55b4c9661550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.752+0000 7fd574e47b80 4 client.15075 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.752+0000 7fd574e47b80 1 -- 172.21.15.115:0/2994758985 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4c9572980 con 0x55b4c96600e0 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.752+0000 7fd55ffff700 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b4c96600e0 0x55b4c9661550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.753+0000 7fd53f7fe700 10 client.15075 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.753+0000 7fd55ffff700 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b4c96600e0 0x55b4c9661550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bfdc4a1faf9e9f3e server_cookie=571248f786a11df4 in_seq=0 out_seq=0 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd53f7fe700 1 -- 172.21.15.115:0/2994758985 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b4c9572980 con 0x55b4c96600e0 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd53f7fe700 10 client.15075 handle_command_reply: tid=0 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 client.15075 shutdown 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 -- 172.21.15.115:0/2994758985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c95af110 msgr2=0x55b4c9650b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c95af110 0x55b4c9650b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd554005c50 tx=0x7fd554011840 comp rx=0 tx=0).stop 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 -- 172.21.15.115:0/2994758985 shutdown_connections 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c964b3b0 0x55b4c9651070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b4c96600e0 0x55b4c9661550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c94c5450 0x55b4c96505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 --2- 172.21.15.115:0/2994758985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c95af110 0x55b4c9650b30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 -- 172.21.15.115:0/2994758985 >> 172.21.15.115:0/2994758985 conn(0x55b4c9525650 msgr2=0x55b4c963d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:07.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd53f7fe700 0 client.15075 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:37:07.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 -- 172.21.15.115:0/2994758985 shutdown_connections 2024-05-09T05:37:07.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 1 -- 172.21.15.115:0/2994758985 wait complete. 2024-05-09T05:37:07.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.754+0000 7fd574e47b80 20 client.15075 trim_cache size 0 max 16384 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.775+0000 7fd574e47b80 1 Processor -- start 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.775+0000 7fd574e47b80 1 -- start start 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.775+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c965c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.775+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c964c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.775+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c964c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.775+0000 7fd574e47b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b4c965f890 con 0x55b4c94c5450 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.775+0000 7fd574e47b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4c965ff60 con 0x55b4c95af110 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.775+0000 7fd574e47b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b4c9660630 con 0x55b4c964b3b0 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c965c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c965c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40114/0 (socket says 172.21.15.115:40114) 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55ffff700 1 -- 172.21.15.115:0/432379640 learned_addr learned my addr 172.21.15.115:0/432379640 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c964c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55ffff700 1 -- 172.21.15.115:0/432379640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 msgr2=0x55b4c964c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd5649ce700 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c964c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55ffff700 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c964c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55ffff700 1 -- 172.21.15.115:0/432379640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 msgr2=0x55b4c964c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55ffff700 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c964c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55ffff700 1 -- 172.21.15.115:0/432379640 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c96618c0 con 0x55b4c94c5450 2024-05-09T05:37:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c964c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd55ffff700 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c965c740 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd550006010 tx=0x7fd55001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52170a98051632bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd545ffb700 1 -- 172.21.15.115:0/432379640 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550035100 con 0x55b4c94c5450 2024-05-09T05:37:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd545ffb700 1 -- 172.21.15.115:0/432379640 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd550035730 con 0x55b4c94c5450 2024-05-09T05:37:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.776+0000 7fd545ffb700 1 -- 172.21.15.115:0/432379640 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550023890 con 0x55b4c94c5450 2024-05-09T05:37:07.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.777+0000 7fd574e47b80 1 -- 172.21.15.115:0/432379640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 msgr2=0x55b4c965c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:07.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.777+0000 7fd574e47b80 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c965c740 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd550006010 tx=0x7fd55001e000 comp rx=0 tx=0).stop 2024-05-09T05:37:07.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.777+0000 7fd574e47b80 1 -- 172.21.15.115:0/432379640 shutdown_connections 2024-05-09T05:37:07.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.777+0000 7fd574e47b80 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c964c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.777+0000 7fd574e47b80 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c965c740 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.777+0000 7fd574e47b80 1 --2- 172.21.15.115:0/432379640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c964c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.777+0000 7fd574e47b80 1 -- 172.21.15.115:0/432379640 >> 172.21.15.115:0/432379640 conn(0x55b4c9525650 msgr2=0x55b4c964eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.777+0000 7fd574e47b80 1 -- 172.21.15.115:0/432379640 shutdown_connections 2024-05-09T05:37:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.777+0000 7fd574e47b80 1 -- 172.21.15.115:0/432379640 wait complete. 2024-05-09T05:37:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd574e47b80 1 -- start start 2024-05-09T05:37:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd574e47b80 1 Processor -- start 2024-05-09T05:37:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c9649b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c966a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd574e47b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c966af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd574e47b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b4c96613d0 con 0x55b4c94c5450 2024-05-09T05:37:07.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd574e47b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4c9660d00 con 0x55b4c95af110 2024-05-09T05:37:07.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd574e47b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b4c9661aa0 con 0x55b4c964b3b0 2024-05-09T05:37:07.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd574e47b80 10 client.0 _ll_register_callbacks cb 0x55b4c93d6670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:07.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c9649b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c9649b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40120/0 (socket says 172.21.15.115:40120) 2024-05-09T05:37:07.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd55ffff700 1 -- 172.21.15.115:0/3038143623 learned_addr learned my addr 172.21.15.115:0/3038143623 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:07.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.778+0000 7fd5649ce700 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c966af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c966a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd55ffff700 1 -- 172.21.15.115:0/3038143623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 msgr2=0x55b4c966af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:07.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd55ffff700 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c966af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd55ffff700 1 -- 172.21.15.115:0/3038143623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 msgr2=0x55b4c966a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:07.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd55ffff700 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c966a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd55ffff700 1 -- 172.21.15.115:0/3038143623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c965ba30 con 0x55b4c94c5450 2024-05-09T05:37:07.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd55ffff700 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c9649b00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd5500098e0 tx=0x7fd550009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9cdc0495ee95cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:07.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd53f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:07.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd53f7fe700 1 -- 172.21.15.115:0/3038143623 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550035300 con 0x55b4c94c5450 2024-05-09T05:37:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd53f7fe700 1 -- 172.21.15.115:0/3038143623 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd5500354a0 con 0x55b4c94c5450 2024-05-09T05:37:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd574e47b80 1 -- 172.21.15.115:0/3038143623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd550035730 con 0x55b4c94c5450 2024-05-09T05:37:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd53f7fe700 1 -- 172.21.15.115:0/3038143623 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd550023510 con 0x55b4c94c5450 2024-05-09T05:37:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd574e47b80 10 client.15081 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd574e47b80 1 -- 172.21.15.115:0/3038143623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4c96cb810 con 0x55b4c94c5450 2024-05-09T05:37:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.779+0000 7fd53f7fe700 1 -- 172.21.15.115:0/3038143623 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd550035730 con 0x55b4c94c5450 2024-05-09T05:37:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.780+0000 7fd53f7fe700 1 -- 172.21.15.115:0/3038143623 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd550022070 con 0x55b4c94c5450 2024-05-09T05:37:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.780+0000 7fd574e47b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.780+0000 7fd574e47b80 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T05:37:07.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.780+0000 7fd574e47b80 20 client.15081 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:07.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.780+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b4c96cd670 0x55b4c96ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:07.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.780+0000 7fd574e47b80 4 client.15081 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:37:07.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.780+0000 7fd574e47b80 1 -- 172.21.15.115:0/3038143623 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "status"}) -- 0x55b4c956ca20 con 0x55b4c96cd670 2024-05-09T05:37:07.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.780+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b4c96cd670 0x55b4c96ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:07.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd53f7fe700 10 client.15081 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:37:07.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd55f7fe700 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b4c96cd670 0x55b4c96ceae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d772a6e5f2a2580 server_cookie=13aca5f108d2a07f in_seq=0 out_seq=0 2024-05-09T05:37:07.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd53f7fe700 1 -- 172.21.15.115:0/3038143623 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b4c956ca20 con 0x55b4c96cd670 2024-05-09T05:37:07.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd53f7fe700 10 client.15081 handle_command_reply: tid=0 2024-05-09T05:37:07.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 client.15081 shutdown 2024-05-09T05:37:07.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 -- 172.21.15.115:0/3038143623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 msgr2=0x55b4c9649b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:07.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c9649b00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd5500098e0 tx=0x7fd550009310 comp rx=0 tx=0).stop 2024-05-09T05:37:07.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 -- 172.21.15.115:0/3038143623 shutdown_connections 2024-05-09T05:37:07.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b4c96cd670 0x55b4c96ceae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4c964b3b0 0x55b4c966af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4c94c5450 0x55b4c9649b00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 --2- 172.21.15.115:0/3038143623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4c95af110 0x55b4c966a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 -- 172.21.15.115:0/3038143623 >> 172.21.15.115:0/3038143623 conn(0x55b4c9525650 msgr2=0x55b4c9641640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd53f7fe700 0 client.15081 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 -- 172.21.15.115:0/3038143623 shutdown_connections 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 1 -- 172.21.15.115:0/3038143623 wait complete. 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.781+0000 7fd574e47b80 20 client.15081 trim_cache size 0 max 16384 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:37:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 4, 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24425, 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 125.495003231, 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 221.46572121599999 2024-05-09T05:37:07.788 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 -- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd5480b2880 msgr2=0x7fd5480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd5480b2880 0x7fd5480b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd55001a040 comp rx=0 tx=0).stop 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 -- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd560108740 msgr2=0x7fd560199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd560108740 0x7fd560199ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd55800b970 tx=0x7fd55800bd30 comp rx=0 tx=0).stop 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 -- 172.21.15.115:0/3237837553 shutdown_connections 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd5480b2880 0x7fd5480b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd560108740 0x7fd560199ae0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5601034c0 0x7fd56019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 --2- 172.21.15.115:0/3237837553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd560107df0 0x7fd56019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 -- 172.21.15.115:0/3237837553 >> 172.21.15.115:0/3237837553 conn(0x7fd5600fedf0 msgr2=0x7fd560101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:07.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 -- 172.21.15.115:0/3237837553 shutdown_connections 2024-05-09T05:37:07.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:07.782+0000 7fd566431700 1 -- 172.21.15.115:0/3237837553 wait complete. 2024-05-09T05:37:08.511 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:08 smithi115 ceph-mon[30349]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:08.546 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T05:37:08.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:08 smithi195 ceph-mon[33844]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:08.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:08 smithi161 ceph-mon[33805]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:08.796 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:10.542 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:10 smithi115 ceph-mon[30349]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.553+0000 7f2ec2e72700 1 -- 172.21.15.115:0/1729274295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc103bc0 msgr2=0x7f2ebc1080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.553+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/1729274295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc103bc0 0x7f2ebc1080f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009990 tx=0x7f2ea8009ca0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.553+0000 7f2ec2e72700 1 -- 172.21.15.115:0/1729274295 shutdown_connections 2024-05-09T05:37:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.553+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/1729274295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc1086c0 0x7f2ebc108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.553+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/1729274295 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc103270 0x7f2ebc103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.553+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/1729274295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc103bc0 0x7f2ebc1080f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.553+0000 7f2ec2e72700 1 -- 172.21.15.115:0/1729274295 >> 172.21.15.115:0/1729274295 conn(0x7f2ebc0fef30 msgr2=0x7f2ebc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.553+0000 7f2ec2e72700 1 -- 172.21.15.115:0/1729274295 shutdown_connections 2024-05-09T05:37:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.553+0000 7f2ec2e72700 1 -- 172.21.15.115:0/1729274295 wait complete. 2024-05-09T05:37:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.554+0000 7f2ec2e72700 1 Processor -- start 2024-05-09T05:37:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.554+0000 7f2ec2e72700 1 -- start start 2024-05-09T05:37:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.554+0000 7f2ec2e72700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc103270 0x7f2ebc1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.554+0000 7f2ec2e72700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc103bc0 0x7f2ebc1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.554+0000 7f2ec2e72700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc1086c0 0x7f2ebc1a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.554+0000 7f2ec2e72700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc11a970 con 0x7f2ebc1086c0 2024-05-09T05:37:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.554+0000 7f2ec2e72700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc11a7f0 con 0x7f2ebc103270 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.554+0000 7f2ec2e72700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc11aaf0 con 0x7f2ebc103bc0 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.554+0000 7f2ec140f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc1086c0 0x7f2ebc1a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec140f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc1086c0 0x7f2ebc1a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40150/0 (socket says 172.21.15.115:40150) 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec140f700 1 -- 172.21.15.115:0/3372081840 learned_addr learned my addr 172.21.15.115:0/3372081840 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc103270 0x7f2ebc1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ebbfff700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc103bc0 0x7f2ebc1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec140f700 1 -- 172.21.15.115:0/3372081840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc103bc0 msgr2=0x7f2ebc1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec140f700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc103bc0 0x7f2ebc1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec140f700 1 -- 172.21.15.115:0/3372081840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc103270 msgr2=0x7f2ebc1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec140f700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc103270 0x7f2ebc1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec140f700 1 -- 172.21.15.115:0/3372081840 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc1a6290 con 0x7f2ebc1086c0 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec140f700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc1086c0 0x7f2ebc1a5bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400b970 tx=0x7f2eb400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2eb9ffb700 1 -- 172.21.15.115:0/3372081840 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400d860 con 0x7f2ebc1086c0 2024-05-09T05:37:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2eb9ffb700 1 -- 172.21.15.115:0/3372081840 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2eb400e430 con 0x7f2ebc1086c0 2024-05-09T05:37:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2ec2e72700 1 -- 172.21.15.115:0/3372081840 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ebc1a6580 con 0x7f2ebc1086c0 2024-05-09T05:37:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.555+0000 7f2eb9ffb700 1 -- 172.21.15.115:0/3372081840 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4017660 con 0x7f2ebc1086c0 2024-05-09T05:37:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.556+0000 7f2ec2e72700 1 -- 172.21.15.115:0/3372081840 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ebc1adea0 con 0x7f2ebc1086c0 2024-05-09T05:37:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.558+0000 7f2eb9ffb700 1 -- 172.21.15.115:0/3372081840 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2eb400da00 con 0x7f2ebc1086c0 2024-05-09T05:37:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.559+0000 7f2eb9ffb700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2eac0b2900 0x7f2eac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.559+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2eac0b2900 0x7f2eac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.559+0000 7f2eb9ffb700 1 -- 172.21.15.115:0/3372081840 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2eb40e2170 con 0x7f2ebc1086c0 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.559+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2eac0b2900 0x7f2eac0b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0005950 tx=0x7f2eb000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.559+0000 7f2ed1888b80 1 Processor -- start 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ed1888b80 1 -- start start 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 0x56363b2a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56363b1c3050 con 0x56363b19dcb0 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56363b168950 con 0x56363b2a03b0 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56363b1bf010 con 0x56363b123450 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec0c0e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec0c0e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2a61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40154/0 (socket says 172.21.15.115:40154) 2024-05-09T05:37:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec0c0e700 1 -- 172.21.15.115:0/1112357890 learned_addr learned my addr 172.21.15.115:0/1112357890 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec140f700 1 --2- 172.21.15.115:0/1112357890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 0x56363b2a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ebbfff700 1 --2- 172.21.15.115:0/1112357890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec0c0e700 1 -- 172.21.15.115:0/1112357890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 msgr2=0x56363b2a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/1112357890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec0c0e700 1 -- 172.21.15.115:0/1112357890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 msgr2=0x56363b2a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/1112357890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 0x56363b2a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec0c0e700 1 -- 172.21.15.115:0/1112357890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363b1594b0 con 0x56363b19dcb0 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.560+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/1112357890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2a61f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0010fa0 tx=0x7f2eb0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59088f2f1dc01428 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.561+0000 7f2ea6ffd700 1 -- 172.21.15.115:0/1112357890 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb000be60 con 0x56363b19dcb0 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.561+0000 7f2ea6ffd700 1 -- 172.21.15.115:0/1112357890 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2eb001eab0 con 0x56363b19dcb0 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.561+0000 7f2ea6ffd700 1 -- 172.21.15.115:0/1112357890 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb001ca90 con 0x56363b19dcb0 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 -- 172.21.15.115:0/1112357890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 msgr2=0x56363b2a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/1112357890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2a61f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0010fa0 tx=0x7f2eb0013000 comp rx=0 tx=0).stop 2024-05-09T05:37:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 -- 172.21.15.115:0/1112357890 shutdown_connections 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/1112357890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 0x56363b2a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/1112357890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2a61f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/1112357890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 -- 172.21.15.115:0/1112357890 >> 172.21.15.115:0/1112357890 conn(0x56363b126f20 msgr2=0x56363b29b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 -- 172.21.15.115:0/1112357890 shutdown_connections 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 -- 172.21.15.115:0/1112357890 wait complete. 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 -- start start 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.562+0000 7f2ed1888b80 1 Processor -- start 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 0x56363b2aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56363b2a1110 con 0x56363b2a03b0 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56363b15dad0 con 0x56363b19dcb0 2024-05-09T05:37:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56363b2acfd0 con 0x56363b123450 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ed1888b80 10 client.0 _ll_register_callbacks cb 0x56363b034670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec140f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec140f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40160/0 (socket says 172.21.15.115:40160) 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec140f700 1 -- 172.21.15.115:0/2869049756 learned_addr learned my addr 172.21.15.115:0/2869049756 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ebbfff700 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 0x56363b2aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec140f700 1 -- 172.21.15.115:0/2869049756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 msgr2=0x56363b2a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec140f700 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec140f700 1 -- 172.21.15.115:0/2869049756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 msgr2=0x56363b2aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec140f700 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 0x56363b2aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec140f700 1 -- 172.21.15.115:0/2869049756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56363b03e440 con 0x56363b2a03b0 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ebbfff700 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 0x56363b2aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ec140f700 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2aaa70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40df8b0 tx=0x7f2eb40a9b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b870daa7122252 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2e9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2869049756 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40b2230 con 0x56363b2a03b0 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2869049756 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2eb400dcf0 con 0x56363b2a03b0 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.563+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2869049756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb001eab0 con 0x56363b2a03b0 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2869049756 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2eb40a9630 con 0x56363b2a03b0 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2ed1888b80 10 client.15093 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2869049756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56363b2b79c0 con 0x56363b2a03b0 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2869049756 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2eb001eab0 con 0x56363b2a03b0 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2869049756 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2eb40ca120 con 0x56363b2a03b0 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2ed1888b80 10 client.15093 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2ed1888b80 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2ed1888b80 20 client.15093 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x56363b2b9f70 0x56363b2bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2ed1888b80 4 client.15093 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.564+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2869049756 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363b1d0a30 con 0x56363b2b9f70 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.565+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x56363b2b9f70 0x56363b2bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.565+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x56363b2b9f70 0x56363b2bb3e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=681f50cf1f68fbe2 server_cookie=548ca57be957a72b in_seq=0 out_seq=0 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.565+0000 7f2e9bfff700 10 client.15093 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2869049756 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56363b1d0a30 con 0x56363b2b9f70 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2e9bfff700 10 client.15093 handle_command_reply: tid=0 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 client.15093 shutdown 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2869049756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 msgr2=0x56363b2aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2aaa70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40df8b0 tx=0x7f2eb40a9b80 comp rx=0 tx=0).stop 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2869049756 shutdown_connections 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x56363b2b9f70 0x56363b2bb3e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2aaa70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2869049756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 0x56363b2aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2869049756 >> 172.21.15.115:0/2869049756 conn(0x56363b126f20 msgr2=0x56363b294fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2e9bfff700 0 client.15093 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:37:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2869049756 shutdown_connections 2024-05-09T05:37:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2869049756 wait complete. 2024-05-09T05:37:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.566+0000 7f2ed1888b80 20 client.15093 trim_cache size 0 max 16384 2024-05-09T05:37:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.587+0000 7f2ed1888b80 1 Processor -- start 2024-05-09T05:37:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.587+0000 7f2ed1888b80 1 -- start start 2024-05-09T05:37:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 0x56363b2aa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56363b2b9720 con 0x56363b19dcb0 2024-05-09T05:37:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56363b2b9df0 con 0x56363b2a03b0 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56363b2ba4c0 con 0x56363b123450 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ebbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ebbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40164/0 (socket says 172.21.15.115:40164) 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ebbfff700 1 -- 172.21.15.115:0/2547508990 learned_addr learned my addr 172.21.15.115:0/2547508990 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/2547508990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ec140f700 1 --2- 172.21.15.115:0/2547508990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 0x56363b2aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ebbfff700 1 -- 172.21.15.115:0/2547508990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 msgr2=0x56363b2a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ebbfff700 1 --2- 172.21.15.115:0/2547508990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ebbfff700 1 -- 172.21.15.115:0/2547508990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 msgr2=0x56363b2aa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ebbfff700 1 --2- 172.21.15.115:0/2547508990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 0x56363b2aa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ebbfff700 1 -- 172.21.15.115:0/2547508990 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363b297980 con 0x56363b19dcb0 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ebbfff700 1 --2- 172.21.15.115:0/2547508990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2ac490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8000c00 tx=0x7f2ea8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f0944d4972023a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ea67fc700 1 -- 172.21.15.115:0/2547508990 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8028070 con 0x56363b19dcb0 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.588+0000 7f2ea67fc700 1 -- 172.21.15.115:0/2547508990 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ea8005040 con 0x56363b19dcb0 2024-05-09T05:37:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ea67fc700 1 -- 172.21.15.115:0/2547508990 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800bb70 con 0x56363b19dcb0 2024-05-09T05:37:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2547508990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 msgr2=0x56363b2ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2547508990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2ac490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8000c00 tx=0x7f2ea8009f90 comp rx=0 tx=0).stop 2024-05-09T05:37:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2547508990 shutdown_connections 2024-05-09T05:37:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2547508990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b2a03b0 0x56363b2aa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2547508990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2547508990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b19dcb0 0x56363b2ac490 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2547508990 >> 172.21.15.115:0/2547508990 conn(0x56363b126f20 msgr2=0x56363b2a5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2547508990 shutdown_connections 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.589+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2547508990 wait complete. 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ed1888b80 1 -- start start 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ed1888b80 1 Processor -- start 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2c4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 0x56363b2c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ed1888b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2bf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56363b2bb260 con 0x56363b2a03b0 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56363b2bab90 con 0x56363b19dcb0 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ed1888b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56363b2bb930 con 0x56363b123450 2024-05-09T05:37:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ed1888b80 10 client.0 _ll_register_callbacks cb 0x56363b034670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ec140f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ec140f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2bf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40176/0 (socket says 172.21.15.115:40176) 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ec140f700 1 -- 172.21.15.115:0/2648509420 learned_addr learned my addr 172.21.15.115:0/2648509420 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.590+0000 7f2ec0c0e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ebbfff700 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 0x56363b2c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ec140f700 1 -- 172.21.15.115:0/2648509420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 msgr2=0x56363b2c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ec140f700 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ec140f700 1 -- 172.21.15.115:0/2648509420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 msgr2=0x56363b2c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ec140f700 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 0x56363b2c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ec140f700 1 -- 172.21.15.115:0/2648509420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56363b29ce90 con 0x56363b2a03b0 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ec140f700 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2bf810 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40a5450 tx=0x7f2eb40b20f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85052fb01aeb7bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2e9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2648509420 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40a91a0 con 0x56363b2a03b0 2024-05-09T05:37:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2648509420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ea8005040 con 0x56363b2a03b0 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2648509420 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2eb400dcf0 con 0x56363b2a03b0 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2648509420 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2eb40b2230 con 0x56363b2a03b0 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ed1888b80 10 client.15099 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2648509420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56363b325880 con 0x56363b2a03b0 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.591+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2648509420 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ea8005040 con 0x56363b2a03b0 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2648509420 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2eb4083ae0 con 0x56363b2a03b0 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2ed1888b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2ed1888b80 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2ed1888b80 20 client.15099 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x56363b327b30 0x56363b328fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2ed1888b80 4 client.15099 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2648509420 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "status"}) -- 0x56363b1caad0 con 0x56363b327b30 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x56363b327b30 0x56363b328fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2e9bfff700 10 client.15099 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.592+0000 7f2ec0c0e700 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x56363b327b30 0x56363b328fa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68fef7c6d27fe6d2 server_cookie=cbe2aaeeed0117b6 in_seq=0 out_seq=0 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2e9bfff700 1 -- 172.21.15.115:0/2648509420 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56363b1caad0 con 0x56363b327b30 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2e9bfff700 10 client.15099 handle_command_reply: tid=0 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 client.15099 shutdown 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2648509420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 msgr2=0x56363b2bf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2bf810 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40a5450 tx=0x7f2eb40b20f0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2648509420 shutdown_connections 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x56363b327b30 0x56363b328fa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56363b2a03b0 0x56363b2bf810 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56363b123450 0x56363b2c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 --2- 172.21.15.115:0/2648509420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56363b19dcb0 0x56363b2c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2648509420 >> 172.21.15.115:0/2648509420 conn(0x56363b126f20 msgr2=0x56363b294fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2e9bfff700 0 client.15099 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2648509420 shutdown_connections 2024-05-09T05:37:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 1 -- 172.21.15.115:0/2648509420 wait complete. 2024-05-09T05:37:10.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.593+0000 7f2ed1888b80 20 client.15099 trim_cache size 0 max 16384 2024-05-09T05:37:10.594 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:37:10.594 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:37:10.594 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 3, 2024-05-09T05:37:10.594 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24440, 2024-05-09T05:37:10.594 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:37:10.595 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:37:10.595 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:37:10.595 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 129.310107827, 2024-05-09T05:37:10.595 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:37:10.595 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:37:10.595 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:37:10.595 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 210.680657349 2024-05-09T05:37:10.595 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 -- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2eac0b2900 msgr2=0x7f2eac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2eac0b2900 0x7f2eac0b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0005950 tx=0x7f2eb000b410 comp rx=0 tx=0).stop 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 -- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc1086c0 msgr2=0x7f2ebc1a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc1086c0 0x7f2ebc1a5bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400b970 tx=0x7f2eb400bc80 comp rx=0 tx=0).stop 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 -- 172.21.15.115:0/3372081840 shutdown_connections 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2eac0b2900 0x7f2eac0b4e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ebc103270 0x7f2ebc1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc1086c0 0x7f2ebc1a5bf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 --2- 172.21.15.115:0/3372081840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc103bc0 0x7f2ebc1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 -- 172.21.15.115:0/3372081840 >> 172.21.15.115:0/3372081840 conn(0x7f2ebc0fef30 msgr2=0x7f2ebc1060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 -- 172.21.15.115:0/3372081840 shutdown_connections 2024-05-09T05:37:10.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:10.594+0000 7f2ec2e72700 1 -- 172.21.15.115:0/3372081840 wait complete. 2024-05-09T05:37:10.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:10 smithi195 ceph-mon[33844]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:10.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:10 smithi161 ceph-mon[33805]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:11.300 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-09T05:37:11.543 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:12.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:12 smithi195 ceph-mon[33844]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:12.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:12 smithi115 ceph-mon[30349]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:12.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:12 smithi161 ceph-mon[33805]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.347+0000 7fbe51cb7700 1 -- 172.21.15.115:0/3808719700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c103310 msgr2=0x7fbe4c10f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.347+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/3808719700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c103310 0x7fbe4c10f890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009990 tx=0x7fbe44009ca0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.347+0000 7fbe51cb7700 1 -- 172.21.15.115:0/3808719700 shutdown_connections 2024-05-09T05:37:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.347+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/3808719700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c103310 0x7fbe4c10f890 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.347+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/3808719700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c1089f0 0x7fbe4c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.347+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/3808719700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c1029c0 0x7fbe4c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.347+0000 7fbe51cb7700 1 -- 172.21.15.115:0/3808719700 >> 172.21.15.115:0/3808719700 conn(0x7fbe4c0fe4e0 msgr2=0x7fbe4c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.347+0000 7fbe51cb7700 1 -- 172.21.15.115:0/3808719700 shutdown_connections 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.347+0000 7fbe51cb7700 1 -- 172.21.15.115:0/3808719700 wait complete. 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.348+0000 7fbe51cb7700 1 Processor -- start 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.348+0000 7fbe51cb7700 1 -- start start 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.348+0000 7fbe51cb7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c1029c0 0x7fbe4c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.348+0000 7fbe51cb7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c103310 0x7fbe4c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe51cb7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c1089f0 0x7fbe4c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe51cb7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1143d0 con 0x7fbe4c1029c0 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe51cb7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c114250 con 0x7fbe4c1089f0 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe51cb7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c114550 con 0x7fbe4c103310 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c1029c0 0x7fbe4c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c1029c0 0x7fbe4c19ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51786/0 (socket says 172.21.15.115:51786) 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4b7fe700 1 -- 172.21.15.115:0/1722828056 learned_addr learned my addr 172.21.15.115:0/1722828056 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c1089f0 0x7fbe4c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4affd700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c103310 0x7fbe4c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4b7fe700 1 -- 172.21.15.115:0/1722828056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c103310 msgr2=0x7fbe4c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c103310 0x7fbe4c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4b7fe700 1 -- 172.21.15.115:0/1722828056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c1089f0 msgr2=0x7fbe4c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c1089f0 0x7fbe4c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4b7fe700 1 -- 172.21.15.115:0/1722828056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4c19fc00 con 0x7fbe4c1029c0 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c1029c0 0x7fbe4c19ade0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00b9c0 tx=0x7fbe3c00bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe48ff9700 1 -- 172.21.15.115:0/1722828056 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c019070 con 0x7fbe4c1029c0 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe48ff9700 1 -- 172.21.15.115:0/1722828056 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe3c00cde0 con 0x7fbe4c1029c0 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe51cb7700 1 -- 172.21.15.115:0/1722828056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4c19fef0 con 0x7fbe4c1029c0 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe48ff9700 1 -- 172.21.15.115:0/1722828056 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c012340 con 0x7fbe4c1029c0 2024-05-09T05:37:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.349+0000 7fbe51cb7700 1 -- 172.21.15.115:0/1722828056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe4c1a0430 con 0x7fbe4c1029c0 2024-05-09T05:37:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.352+0000 7fbe48ff9700 1 -- 172.21.15.115:0/1722828056 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbe3c01c070 con 0x7fbe4c1029c0 2024-05-09T05:37:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.353+0000 7fbe48ff9700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe340b2880 0x7fbe340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.353+0000 7fbe4affd700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe340b2880 0x7fbe340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.353+0000 7fbe48ff9700 1 -- 172.21.15.115:0/1722828056 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fbe3c0e1be0 con 0x7fbe4c1029c0 2024-05-09T05:37:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.353+0000 7fbe606cdb80 1 Processor -- start 2024-05-09T05:37:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.353+0000 7fbe606cdb80 1 -- start start 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4affd700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe340b2880 0x7fbe340b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbe40005ea0 tx=0x7fbe40005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb95e110 0x5569cb9f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb874450 0x5569cb9f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cb9f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5569cb913fa0 con 0x5569cb9f1c30 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5569cb8b27a0 con 0x5569cb874450 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5569cb90ff60 con 0x5569cb95e110 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cb9f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cb9f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51788/0 (socket says 172.21.15.115:51788) 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4bfff700 1 -- 172.21.15.115:0/3204665941 learned_addr learned my addr 172.21.15.115:0/3204665941 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/3204665941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb95e110 0x5569cb9f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4affd700 1 --2- 172.21.15.115:0/3204665941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb874450 0x5569cb9f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4bfff700 1 -- 172.21.15.115:0/3204665941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb95e110 msgr2=0x5569cb9f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/3204665941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb95e110 0x5569cb9f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4bfff700 1 -- 172.21.15.115:0/3204665941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb874450 msgr2=0x5569cb9f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/3204665941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb874450 0x5569cb9f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4bfff700 1 -- 172.21.15.115:0/3204665941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cb8aa320 con 0x5569cb9f1c30 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.354+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/3204665941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cb9f2100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbe44015000 tx=0x7fbe44003720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f455235e31dc41d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.355+0000 7fbe31ffb700 1 -- 172.21.15.115:0/3204665941 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44027070 con 0x5569cb9f1c30 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.355+0000 7fbe31ffb700 1 -- 172.21.15.115:0/3204665941 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe44003c50 con 0x5569cb9f1c30 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.355+0000 7fbe31ffb700 1 -- 172.21.15.115:0/3204665941 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400bb40 con 0x5569cb9f1c30 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.355+0000 7fbe606cdb80 1 -- 172.21.15.115:0/3204665941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 msgr2=0x5569cb9f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.355+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/3204665941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cb9f2100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbe44015000 tx=0x7fbe44003720 comp rx=0 tx=0).stop 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.356+0000 7fbe606cdb80 1 -- 172.21.15.115:0/3204665941 shutdown_connections 2024-05-09T05:37:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.356+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/3204665941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cb9f2100 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.356+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/3204665941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb95e110 0x5569cb9f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.356+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/3204665941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb874450 0x5569cb9f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.356+0000 7fbe606cdb80 1 -- 172.21.15.115:0/3204665941 >> 172.21.15.115:0/3204665941 conn(0x5569cb8d4650 msgr2=0x5569cb9ec3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.356+0000 7fbe606cdb80 1 -- 172.21.15.115:0/3204665941 shutdown_connections 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.356+0000 7fbe606cdb80 1 -- 172.21.15.115:0/3204665941 wait complete. 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.356+0000 7fbe606cdb80 1 -- start start 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.356+0000 7fbe606cdb80 1 Processor -- start 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba03380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cba038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 0x5569cba03e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5569cb78fb50 con 0x5569cb95e110 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5569cb8ce100 con 0x5569cb9f1c30 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5569cba067a0 con 0x5569cb874450 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe606cdb80 10 client.0 _ll_register_callbacks cb 0x5569cb785670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cba038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cba038c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51792/0 (socket says 172.21.15.115:51792) 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4affd700 1 -- 172.21.15.115:0/1309832702 learned_addr learned my addr 172.21.15.115:0/1309832702 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 0x5569cba03e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba03380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4affd700 1 -- 172.21.15.115:0/1309832702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 msgr2=0x5569cba03380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4affd700 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba03380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4affd700 1 -- 172.21.15.115:0/1309832702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 msgr2=0x5569cba03e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4affd700 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 0x5569cba03e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4affd700 1 -- 172.21.15.115:0/1309832702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cb78f440 con 0x5569cb95e110 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba03380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe4affd700 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cba038c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe400109e0 tx=0x7fbe400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4821df20aef9717b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe2affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe2affd700 1 -- 172.21.15.115:0/1309832702 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4001da50 con 0x5569cb95e110 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe2affd700 1 -- 172.21.15.115:0/1309832702 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fbe4001dbf0 con 0x5569cb95e110 2024-05-09T05:37:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.357+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1309832702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe44003c50 con 0x5569cb95e110 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe2affd700 1 -- 172.21.15.115:0/1309832702 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe40031050 con 0x5569cb95e110 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe606cdb80 10 client.15111 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1309832702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569cba11330 con 0x5569cb95e110 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe2affd700 1 -- 172.21.15.115:0/1309832702 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe44003c50 con 0x5569cb95e110 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe2affd700 1 -- 172.21.15.115:0/1309832702 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbe400263f0 con 0x5569cb95e110 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe606cdb80 10 client.15111 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe606cdb80 10 client.15111 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe606cdb80 20 client.15111 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5569cba13a00 0x5569cba14e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe606cdb80 4 client.15111 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1309832702 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569cb921980 con 0x5569cba13a00 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.358+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5569cba13a00 0x5569cba14e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.359+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5569cba13a00 0x5569cba14e70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1fd4134cb92e5d8f server_cookie=8580d63504dd5ece in_seq=0 out_seq=0 2024-05-09T05:37:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.359+0000 7fbe2affd700 10 client.15111 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe2affd700 1 -- 172.21.15.115:0/1309832702 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569cb921980 con 0x5569cba13a00 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe2affd700 10 client.15111 handle_command_reply: tid=0 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 client.15111 shutdown 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1309832702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 msgr2=0x5569cba038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cba038c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe400109e0 tx=0x7fbe400047c0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1309832702 shutdown_connections 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 0x5569cba03e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5569cba13a00 0x5569cba14e70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba03380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe2affd700 0 client.15111 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1309832702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cba038c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1309832702 >> 172.21.15.115:0/1309832702 conn(0x5569cb8d4650 msgr2=0x5569cb9ec170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1309832702 shutdown_connections 2024-05-09T05:37:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1309832702 wait complete. 2024-05-09T05:37:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.360+0000 7fbe606cdb80 20 client.15111 trim_cache size 0 max 16384 2024-05-09T05:37:13.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe606cdb80 1 Processor -- start 2024-05-09T05:37:13.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe606cdb80 1 -- start start 2024-05-09T05:37:13.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba03c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cb9f0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 0x5569cb9f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5569cba131b0 con 0x5569cb95e110 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5569cba13880 con 0x5569cb9f1c30 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5569cba13f50 con 0x5569cb874450 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe4affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cb9f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe4affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cb9f0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51806/0 (socket says 172.21.15.115:51806) 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe4affd700 1 -- 172.21.15.115:0/2658212025 learned_addr learned my addr 172.21.15.115:0/2658212025 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba03c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.381+0000 7fbe4affd700 1 -- 172.21.15.115:0/2658212025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 msgr2=0x5569cba03c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe4affd700 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba03c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe4affd700 1 -- 172.21.15.115:0/2658212025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 msgr2=0x5569cb9f0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 0x5569cb9f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe4affd700 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 0x5569cb9f0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe4affd700 1 -- 172.21.15.115:0/2658212025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cb9f5020 con 0x5569cb95e110 2024-05-09T05:37:13.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 0x5569cb9f0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:13.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe4affd700 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cb9f0640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbe40004790 tx=0x7fbe4000a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8313f9a01c855a3f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:13.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe317fa700 1 -- 172.21.15.115:0/2658212025 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4003b070 con 0x5569cb95e110 2024-05-09T05:37:13.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe317fa700 1 -- 172.21.15.115:0/2658212025 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe400269d0 con 0x5569cb95e110 2024-05-09T05:37:13.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.382+0000 7fbe317fa700 1 -- 172.21.15.115:0/2658212025 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4001e430 con 0x5569cb95e110 2024-05-09T05:37:13.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 -- 172.21.15.115:0/2658212025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 msgr2=0x5569cb9f0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:13.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cb9f0640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbe40004790 tx=0x7fbe4000a4f0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 -- 172.21.15.115:0/2658212025 shutdown_connections 2024-05-09T05:37:13.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb9f1c30 0x5569cb9f0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba03c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/2658212025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb95e110 0x5569cb9f0640 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 -- 172.21.15.115:0/2658212025 >> 172.21.15.115:0/2658212025 conn(0x5569cb8d4650 msgr2=0x5569cb9ec040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 -- 172.21.15.115:0/2658212025 shutdown_connections 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 -- 172.21.15.115:0/2658212025 wait complete. 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 -- start start 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.383+0000 7fbe606cdb80 1 Processor -- start 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb95e110 0x5569cba18a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe606cdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cba1ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5569cba14cf0 con 0x5569cb9f1c30 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5569cba14620 con 0x5569cb95e110 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe606cdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5569cba153c0 con 0x5569cb874450 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe606cdb80 10 client.0 _ll_register_callbacks cb 0x5569cb785670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cba1ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cba1ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51808/0 (socket says 172.21.15.115:51808) 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4bfff700 1 -- 172.21.15.115:0/1267907494 learned_addr learned my addr 172.21.15.115:0/1267907494 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4affd700 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb95e110 0x5569cba18a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4bfff700 1 -- 172.21.15.115:0/1267907494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 msgr2=0x5569cba184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba184d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4bfff700 1 -- 172.21.15.115:0/1267907494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb95e110 msgr2=0x5569cba18a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb95e110 0x5569cba18a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4bfff700 1 -- 172.21.15.115:0/1267907494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cba1cfd0 con 0x5569cb9f1c30 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe4bfff700 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cba1ca90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbe44025040 tx=0x7fbe440059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835dca5077a5ff10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.384+0000 7fbe2affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe2affd700 1 -- 172.21.15.115:0/1267907494 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44027040 con 0x5569cb9f1c30 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe2affd700 1 -- 172.21.15.115:0/1267907494 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fbe440037c0 con 0x5569cb9f1c30 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1267907494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe400269d0 con 0x5569cb9f1c30 2024-05-09T05:37:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe2affd700 1 -- 172.21.15.115:0/1267907494 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe4403a050 con 0x5569cb9f1c30 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe606cdb80 10 client.15117 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1267907494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569cba7f4c0 con 0x5569cb9f1c30 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe2affd700 1 -- 172.21.15.115:0/1267907494 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe400269d0 con 0x5569cb9f1c30 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe2affd700 1 -- 172.21.15.115:0/1267907494 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbe44035030 con 0x5569cb9f1c30 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe606cdb80 10 client.15117 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe606cdb80 10 client.15117 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe606cdb80 20 client.15117 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5569cba813d0 0x5569cba82840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe606cdb80 4 client.15117 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.385+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1267907494 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "status"}) -- 0x5569cb91ba20 con 0x5569cba813d0 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5569cba813d0 0x5569cba82840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe2affd700 10 client.15117 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe4b7fe700 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5569cba813d0 0x5569cba82840 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aae82c17b5af81ca server_cookie=31673f081d1a0649 in_seq=0 out_seq=0 2024-05-09T05:37:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe2affd700 1 -- 172.21.15.115:0/1267907494 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5569cb91ba20 con 0x5569cba813d0 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe2affd700 10 client.15117 handle_command_reply: tid=0 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe606cdb80 1 client.15117 shutdown 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1267907494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 msgr2=0x5569cba1ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cba1ca90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbe44025040 tx=0x7fbe440059f0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1267907494 shutdown_connections 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5569cba813d0 0x5569cba82840 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5569cb9f1c30 0x5569cba1ca90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5569cb874450 0x5569cba184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe606cdb80 1 --2- 172.21.15.115:0/1267907494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569cb95e110 0x5569cba18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.386+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1267907494 >> 172.21.15.115:0/1267907494 conn(0x5569cb8d4650 msgr2=0x5569cb9fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe2affd700 0 client.15117 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1267907494 shutdown_connections 2024-05-09T05:37:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe606cdb80 1 -- 172.21.15.115:0/1267907494 wait complete. 2024-05-09T05:37:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe606cdb80 20 client.15117 trim_cache size 0 max 16384 2024-05-09T05:37:13.386 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:37:13.386 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:37:13.386 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 2, 2024-05-09T05:37:13.386 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24451, 2024-05-09T05:37:13.386 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:37:13.386 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:37:13.386 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:37:13.386 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 133.12156751500001, 2024-05-09T05:37:13.387 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:37:13.387 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:37:13.387 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:37:13.387 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 232.838193233 2024-05-09T05:37:13.387 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:37:13.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 -- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe340b2880 msgr2=0x7fbe340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe340b2880 0x7fbe340b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbe40005ea0 tx=0x7fbe40005e30 comp rx=0 tx=0).stop 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 -- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c1029c0 msgr2=0x7fbe4c19ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c1029c0 0x7fbe4c19ade0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00b9c0 tx=0x7fbe3c00bcd0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 -- 172.21.15.115:0/1722828056 shutdown_connections 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fbe340b2880 0x7fbe340b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c1089f0 0x7fbe4c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe4c1029c0 0x7fbe4c19ade0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 --2- 172.21.15.115:0/1722828056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c103310 0x7fbe4c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.387+0000 7fbe51cb7700 1 -- 172.21.15.115:0/1722828056 >> 172.21.15.115:0/1722828056 conn(0x7fbe4c0fe4e0 msgr2=0x7fbe4c106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.388+0000 7fbe51cb7700 1 -- 172.21.15.115:0/1722828056 shutdown_connections 2024-05-09T05:37:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:13.388+0000 7fbe51cb7700 1 -- 172.21.15.115:0/1722828056 wait complete. 2024-05-09T05:37:14.085 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T05:37:14.326 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:14.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:14 smithi115 ceph-mon[30349]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:14.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:14 smithi195 ceph-mon[33844]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:14.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:14 smithi161 ceph-mon[33805]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.090+0000 7fd844ed5700 1 -- 172.21.15.115:0/4187849852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8401034c0 msgr2=0x7fd8401038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.090+0000 7fd844ed5700 1 --2- 172.21.15.115:0/4187849852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8401034c0 0x7fd8401038d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd8300099c0 tx=0x7fd830009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.090+0000 7fd844ed5700 1 -- 172.21.15.115:0/4187849852 shutdown_connections 2024-05-09T05:37:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.090+0000 7fd844ed5700 1 --2- 172.21.15.115:0/4187849852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd840108740 0x7fd8401100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.090+0000 7fd844ed5700 1 --2- 172.21.15.115:0/4187849852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8401034c0 0x7fd8401038d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.090+0000 7fd844ed5700 1 --2- 172.21.15.115:0/4187849852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd840107df0 0x7fd840108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.090+0000 7fd844ed5700 1 -- 172.21.15.115:0/4187849852 >> 172.21.15.115:0/4187849852 conn(0x7fd8400fedf0 msgr2=0x7fd840101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.090+0000 7fd844ed5700 1 -- 172.21.15.115:0/4187849852 shutdown_connections 2024-05-09T05:37:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.090+0000 7fd844ed5700 1 -- 172.21.15.115:0/4187849852 wait complete. 2024-05-09T05:37:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.091+0000 7fd844ed5700 1 Processor -- start 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.091+0000 7fd844ed5700 1 -- start start 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd844ed5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8401034c0 0x7fd84019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd844ed5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd840107df0 0x7fd84019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd844ed5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd840108740 0x7fd840199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd844ed5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd840113220 con 0x7fd840108740 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd844ed5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401130a0 con 0x7fd8401034c0 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd844ed5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401133a0 con 0x7fd840107df0 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd840108740 0x7fd840199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd840108740 0x7fd840199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51816/0 (socket says 172.21.15.115:51816) 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83ed9d700 1 -- 172.21.15.115:0/887391086 learned_addr learned my addr 172.21.15.115:0/887391086 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83dd9b700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd840107df0 0x7fd84019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83e59c700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8401034c0 0x7fd84019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83ed9d700 1 -- 172.21.15.115:0/887391086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd840107df0 msgr2=0x7fd84019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83ed9d700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd840107df0 0x7fd84019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83ed9d700 1 -- 172.21.15.115:0/887391086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8401034c0 msgr2=0x7fd84019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83ed9d700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8401034c0 0x7fd84019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83ed9d700 1 -- 172.21.15.115:0/887391086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84019a110 con 0x7fd840108740 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83e59c700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8401034c0 0x7fd84019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd83ed9d700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd840108740 0x7fd840199a60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd838007f10 tx=0x7fd83800f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd82f7fe700 1 -- 172.21.15.115:0/887391086 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838016030 con 0x7fd840108740 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd82f7fe700 1 -- 172.21.15.115:0/887391086 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd838004bb0 con 0x7fd840108740 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd82f7fe700 1 -- 172.21.15.115:0/887391086 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838005020 con 0x7fd840108740 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.092+0000 7fd844ed5700 1 -- 172.21.15.115:0/887391086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84019a400 con 0x7fd840108740 2024-05-09T05:37:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.093+0000 7fd844ed5700 1 -- 172.21.15.115:0/887391086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8400759f0 con 0x7fd840108740 2024-05-09T05:37:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.095+0000 7fd82f7fe700 1 -- 172.21.15.115:0/887391086 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd8380051c0 con 0x7fd840108740 2024-05-09T05:37:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.096+0000 7fd82f7fe700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd8280b2880 0x7fd8280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.096+0000 7fd8538ebb80 1 Processor -- start 2024-05-09T05:37:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.096+0000 7fd83e59c700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd8280b2880 0x7fd8280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd82f7fe700 1 -- 172.21.15.115:0/887391086 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd8380e4980 con 0x7fd840108740 2024-05-09T05:37:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd8538ebb80 1 -- start start 2024-05-09T05:37:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd83e59c700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd8280b2880 0x7fd8280b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd83000b580 tx=0x7fd830005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 0x559e305849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 0x559e3057ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559e304a0fa0 con 0x559e304eb110 2024-05-09T05:37:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e3043f7a0 con 0x559e30401450 2024-05-09T05:37:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e3049cf60 con 0x559e3057e650 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd83e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd83e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305844a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51818/0 (socket says 172.21.15.115:51818) 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd83e59c700 1 -- 172.21.15.115:0/2213311428 learned_addr learned my addr 172.21.15.115:0/2213311428 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd83dd9b700 1 --2- 172.21.15.115:0/2213311428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 0x559e305849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.097+0000 7fd83ed9d700 1 --2- 172.21.15.115:0/2213311428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 0x559e3057ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.098+0000 7fd83e59c700 1 -- 172.21.15.115:0/2213311428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 msgr2=0x559e3057ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.098+0000 7fd83e59c700 1 --2- 172.21.15.115:0/2213311428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 0x559e3057ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.098+0000 7fd83e59c700 1 -- 172.21.15.115:0/2213311428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 msgr2=0x559e305849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.098+0000 7fd83e59c700 1 --2- 172.21.15.115:0/2213311428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 0x559e305849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.098+0000 7fd83e59c700 1 -- 172.21.15.115:0/2213311428 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e30437320 con 0x559e304eb110 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.098+0000 7fd83e59c700 1 --2- 172.21.15.115:0/2213311428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305844a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd830015000 tx=0x7fd83001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f5d7d264826bc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.098+0000 7fd82cff9700 1 -- 172.21.15.115:0/2213311428 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830003bd0 con 0x559e304eb110 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.098+0000 7fd82cff9700 1 -- 172.21.15.115:0/2213311428 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd830003d70 con 0x559e304eb110 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.098+0000 7fd82cff9700 1 -- 172.21.15.115:0/2213311428 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83002e6f0 con 0x559e304eb110 2024-05-09T05:37:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 -- 172.21.15.115:0/2213311428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 msgr2=0x559e305844a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/2213311428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305844a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd830015000 tx=0x7fd83001d000 comp rx=0 tx=0).stop 2024-05-09T05:37:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 -- 172.21.15.115:0/2213311428 shutdown_connections 2024-05-09T05:37:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/2213311428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 0x559e3057ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/2213311428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305844a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/2213311428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 0x559e305849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 -- 172.21.15.115:0/2213311428 >> 172.21.15.115:0/2213311428 conn(0x559e30461650 msgr2=0x559e305793b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 -- 172.21.15.115:0/2213311428 shutdown_connections 2024-05-09T05:37:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 -- 172.21.15.115:0/2213311428 wait complete. 2024-05-09T05:37:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.099+0000 7fd8538ebb80 1 -- start start 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd8538ebb80 1 Processor -- start 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e30401450 0x559e30588190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e304eb110 0x559e305886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e3057e650 0x559e30588c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559e3031cb50 con 0x559e30401450 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e3045b100 con 0x559e3057e650 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e3058b180 con 0x559e304eb110 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd8538ebb80 10 client.0 _ll_register_callbacks cb 0x559e30312670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e30401450 0x559e30588190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e30401450 0x559e30588190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51830/0 (socket says 172.21.15.115:51830) 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83e59c700 1 -- 172.21.15.115:0/3017371559 learned_addr learned my addr 172.21.15.115:0/3017371559 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83ed9d700 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e3057e650 0x559e30588c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83dd9b700 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e304eb110 0x559e305886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83e59c700 1 -- 172.21.15.115:0/3017371559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e304eb110 msgr2=0x559e305886d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83e59c700 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e304eb110 0x559e305886d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83e59c700 1 -- 172.21.15.115:0/3017371559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e3057e650 msgr2=0x559e30588c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83e59c700 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e3057e650 0x559e30588c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.100+0000 7fd83e59c700 1 -- 172.21.15.115:0/3017371559 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3031c440 con 0x559e30401450 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd83e59c700 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e30401450 0x559e30588190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd830003760 tx=0x7fd83002fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d0426dea354d1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd825ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd825ffb700 1 -- 172.21.15.115:0/3017371559 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83002fec0 con 0x559e30401450 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3017371559 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd830003d70 con 0x559e30401450 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd825ffb700 1 -- 172.21.15.115:0/3017371559 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd83000b9a0 con 0x559e30401450 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd825ffb700 1 -- 172.21.15.115:0/3017371559 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd83003e050 con 0x559e30401450 2024-05-09T05:37:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd8538ebb80 10 client.15129 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3017371559 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e30595b30 con 0x559e30401450 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd825ffb700 1 -- 172.21.15.115:0/3017371559 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd830003d70 con 0x559e30401450 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd825ffb700 1 -- 172.21.15.115:0/3017371559 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd830010c70 con 0x559e30401450 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd8538ebb80 10 client.15129 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd8538ebb80 10 client.15129 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd8538ebb80 20 client.15129 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x559e30597ce0 0x559e30599150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd8538ebb80 4 client.15129 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.101+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3017371559 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e304ae980 con 0x559e30597ce0 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.102+0000 7fd83dd9b700 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x559e30597ce0 0x559e30599150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.102+0000 7fd825ffb700 10 client.15129 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.102+0000 7fd83dd9b700 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x559e30597ce0 0x559e30599150 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cc90c021344da88 server_cookie=986598128a31d4b6 in_seq=0 out_seq=0 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd825ffb700 1 -- 172.21.15.115:0/3017371559 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e304ae980 con 0x559e30597ce0 2024-05-09T05:37:16.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd825ffb700 10 client.15129 handle_command_reply: tid=0 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 client.15129 shutdown 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3017371559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e30401450 msgr2=0x559e30588190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e30401450 0x559e30588190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd830003760 tx=0x7fd83002fbf0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3017371559 shutdown_connections 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x559e30597ce0 0x559e30599150 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e3057e650 0x559e30588c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e30401450 0x559e30588190 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd825ffb700 0 client.15129 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3017371559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e304eb110 0x559e305886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3017371559 >> 172.21.15.115:0/3017371559 conn(0x559e30461650 msgr2=0x559e30579170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3017371559 shutdown_connections 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3017371559 wait complete. 2024-05-09T05:37:16.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.103+0000 7fd8538ebb80 20 client.15129 trim_cache size 0 max 16384 2024-05-09T05:37:16.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.124+0000 7fd8538ebb80 1 Processor -- start 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.124+0000 7fd8538ebb80 1 -- start start 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 0x559e30585dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305941d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 0x559e305a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559e30597490 con 0x559e304eb110 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e30597b60 con 0x559e30401450 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e30598230 con 0x559e3057e650 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305941d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51846/0 (socket says 172.21.15.115:51846) 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83dd9b700 1 -- 172.21.15.115:0/3669332667 learned_addr learned my addr 172.21.15.115:0/3669332667 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83ed9d700 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 0x559e305a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83e59c700 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 0x559e30585dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83dd9b700 1 -- 172.21.15.115:0/3669332667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 msgr2=0x559e305a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:16.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83dd9b700 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 0x559e305a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83dd9b700 1 -- 172.21.15.115:0/3669332667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 msgr2=0x559e30585dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83dd9b700 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 0x559e30585dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83dd9b700 1 -- 172.21.15.115:0/3669332667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3057acb0 con 0x559e304eb110 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83e59c700 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 0x559e30585dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd83dd9b700 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305941d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd840106a50 tx=0x7fd834009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324297fb1b061668 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.125+0000 7fd827fff700 1 -- 172.21.15.115:0/3669332667 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834012bc0 con 0x559e304eb110 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.126+0000 7fd827fff700 1 -- 172.21.15.115:0/3669332667 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd834024020 con 0x559e304eb110 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.126+0000 7fd827fff700 1 -- 172.21.15.115:0/3669332667 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340116a0 con 0x559e304eb110 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.126+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3669332667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 msgr2=0x559e305941d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.126+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305941d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd840106a50 tx=0x7fd834009260 comp rx=0 tx=0).stop 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.126+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3669332667 shutdown_connections 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.126+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e30401450 0x559e30585dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.126+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e3057e650 0x559e305a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.126+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/3669332667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e304eb110 0x559e305941d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.126+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3669332667 >> 172.21.15.115:0/3669332667 conn(0x559e30461650 msgr2=0x559e30579130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3669332667 shutdown_connections 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 -- 172.21.15.115:0/3669332667 wait complete. 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 -- start start 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 Processor -- start 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e30401450 0x559e305935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e304eb110 0x559e30593b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e3057e650 0x559e30594040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559e30598fd0 con 0x559e3057e650 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559e30598900 con 0x559e304eb110 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559e305996a0 con 0x559e30401450 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd8538ebb80 10 client.0 _ll_register_callbacks cb 0x559e30312670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd83ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e3057e650 0x559e30594040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd83ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e3057e650 0x559e30594040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51856/0 (socket says 172.21.15.115:51856) 2024-05-09T05:37:16.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.127+0000 7fd83ed9d700 1 -- 172.21.15.115:0/4294130669 learned_addr learned my addr 172.21.15.115:0/4294130669 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:16.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd83e59c700 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e30401450 0x559e305935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd83dd9b700 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e304eb110 0x559e30593b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd83ed9d700 1 -- 172.21.15.115:0/4294130669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e30401450 msgr2=0x559e305935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd83ed9d700 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e30401450 0x559e305935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd83ed9d700 1 -- 172.21.15.115:0/4294130669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e304eb110 msgr2=0x559e30593b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd83ed9d700 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e304eb110 0x559e30593b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd83ed9d700 1 -- 172.21.15.115:0/4294130669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e305a07e0 con 0x559e3057e650 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd83ed9d700 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e3057e650 0x559e30594040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8380e2060 tx=0x7fd838004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b3525046f8197b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd825ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd825ffb700 1 -- 172.21.15.115:0/4294130669 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838020040 con 0x559e3057e650 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd825ffb700 1 -- 172.21.15.115:0/4294130669 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd8380a81d0 con 0x559e3057e650 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd8538ebb80 1 -- 172.21.15.115:0/4294130669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd834024020 con 0x559e3057e650 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd825ffb700 1 -- 172.21.15.115:0/4294130669 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8380aff90 con 0x559e3057e650 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd8538ebb80 10 client.15135 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd8538ebb80 1 -- 172.21.15.115:0/4294130669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e306031b0 con 0x559e3057e650 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.128+0000 7fd825ffb700 1 -- 172.21.15.115:0/4294130669 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd834024020 con 0x559e3057e650 2024-05-09T05:37:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd825ffb700 1 -- 172.21.15.115:0/4294130669 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd8380b4620 con 0x559e3057e650 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd8538ebb80 10 client.15135 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd8538ebb80 10 client.15135 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd8538ebb80 20 client.15135 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x559e30605680 0x559e30606af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd8538ebb80 4 client.15135 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd8538ebb80 1 -- 172.21.15.115:0/4294130669 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "status"}) -- 0x559e304a8a20 con 0x559e30605680 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd83e59c700 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x559e30605680 0x559e30606af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd825ffb700 10 client.15135 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.129+0000 7fd83e59c700 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x559e30605680 0x559e30606af0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee3ccac1795766f1 server_cookie=5b2d93beadc8d656 in_seq=0 out_seq=0 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd825ffb700 1 -- 172.21.15.115:0/4294130669 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559e304a8a20 con 0x559e30605680 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd825ffb700 10 client.15135 handle_command_reply: tid=0 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 client.15135 shutdown 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 -- 172.21.15.115:0/4294130669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e3057e650 msgr2=0x559e30594040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e3057e650 0x559e30594040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8380e2060 tx=0x7fd838004f90 comp rx=0 tx=0).stop 2024-05-09T05:37:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 -- 172.21.15.115:0/4294130669 shutdown_connections 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x559e30605680 0x559e30606af0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559e3057e650 0x559e30594040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559e30401450 0x559e305935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd825ffb700 0 client.15135 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 --2- 172.21.15.115:0/4294130669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e304eb110 0x559e30593b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 -- 172.21.15.115:0/4294130669 >> 172.21.15.115:0/4294130669 conn(0x559e30461650 msgr2=0x559e30581db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 -- 172.21.15.115:0/4294130669 shutdown_connections 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 1 -- 172.21.15.115:0/4294130669 wait complete. 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd8538ebb80 20 client.15135 trim_cache size 0 max 16384 2024-05-09T05:37:16.132 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 1, 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24457, 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 136.87088628000001, 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 232.78362302900001 2024-05-09T05:37:16.133 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:37:16.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd844ed5700 1 -- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd8280b2880 msgr2=0x7fd8280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:16.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd844ed5700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd8280b2880 0x7fd8280b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd83000b580 tx=0x7fd830005fb0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd844ed5700 1 -- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd840108740 msgr2=0x7fd840199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:16.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd844ed5700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd840108740 0x7fd840199a60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd838007f10 tx=0x7fd83800f5c0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.130+0000 7fd844ed5700 1 -- 172.21.15.115:0/887391086 shutdown_connections 2024-05-09T05:37:16.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.131+0000 7fd844ed5700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fd8280b2880 0x7fd8280b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.131+0000 7fd844ed5700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8401034c0 0x7fd84019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.131+0000 7fd844ed5700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd840108740 0x7fd840199a60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.131+0000 7fd844ed5700 1 --2- 172.21.15.115:0/887391086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd840107df0 0x7fd84019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:16.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.131+0000 7fd844ed5700 1 -- 172.21.15.115:0/887391086 >> 172.21.15.115:0/887391086 conn(0x7fd8400fedf0 msgr2=0x7fd840101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:16.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.131+0000 7fd844ed5700 1 -- 172.21.15.115:0/887391086 shutdown_connections 2024-05-09T05:37:16.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:16.131+0000 7fd844ed5700 1 -- 172.21.15.115:0/887391086 wait complete. 2024-05-09T05:37:16.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:16 smithi115 ceph-mon[30349]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:16.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:16 smithi195 ceph-mon[33844]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:16 smithi161 ceph-mon[33805]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:16.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T05:37:17.139 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:18 smithi195 ceph-mon[33844]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:18.624 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:18 smithi115 ceph-mon[30349]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:18 smithi161 ceph-mon[33805]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:18.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.896+0000 7fa764593700 1 -- 172.21.15.115:0/3372205327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c107df0 msgr2=0x7fa75c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.896+0000 7fa764593700 1 --2- 172.21.15.115:0/3372205327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c107df0 0x7fa75c108200 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0099c0 tx=0x7fa74c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.896+0000 7fa764593700 1 -- 172.21.15.115:0/3372205327 shutdown_connections 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.896+0000 7fa764593700 1 --2- 172.21.15.115:0/3372205327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c108740 0x7fa75c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.896+0000 7fa764593700 1 --2- 172.21.15.115:0/3372205327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa75c1034c0 0x7fa75c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.896+0000 7fa764593700 1 --2- 172.21.15.115:0/3372205327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c107df0 0x7fa75c108200 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.896+0000 7fa764593700 1 -- 172.21.15.115:0/3372205327 >> 172.21.15.115:0/3372205327 conn(0x7fa75c0fedc0 msgr2=0x7fa75c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.896+0000 7fa764593700 1 -- 172.21.15.115:0/3372205327 shutdown_connections 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.896+0000 7fa764593700 1 -- 172.21.15.115:0/3372205327 wait complete. 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.897+0000 7fa764593700 1 Processor -- start 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.897+0000 7fa764593700 1 -- start start 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.897+0000 7fa764593700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c1034c0 0x7fa75c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.897+0000 7fa764593700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c107df0 0x7fa75c19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.897+0000 7fa764593700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa75c108740 0x7fa75c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.897+0000 7fa764593700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c113270 con 0x7fa75c1034c0 2024-05-09T05:37:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.897+0000 7fa764593700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1130f0 con 0x7fa75c107df0 2024-05-09T05:37:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.897+0000 7fa764593700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c1133f0 con 0x7fa75c108740 2024-05-09T05:37:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa76232f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c1034c0 0x7fa75c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa761b2e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c107df0 0x7fa75c19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa76232f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c1034c0 0x7fa75c19bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51880/0 (socket says 172.21.15.115:51880) 2024-05-09T05:37:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa76232f700 1 -- 172.21.15.115:0/2896582685 learned_addr learned my addr 172.21.15.115:0/2896582685 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa762b30700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa75c108740 0x7fa75c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa76232f700 1 -- 172.21.15.115:0/2896582685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa75c108740 msgr2=0x7fa75c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa76232f700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa75c108740 0x7fa75c19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa76232f700 1 -- 172.21.15.115:0/2896582685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c107df0 msgr2=0x7fa75c19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa76232f700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c107df0 0x7fa75c19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa76232f700 1 -- 172.21.15.115:0/2896582685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c1a6f40 con 0x7fa75c1034c0 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa76232f700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c1034c0 0x7fa75c19bbf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa75800b530 tx=0x7fa75800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa7537fe700 1 -- 172.21.15.115:0/2896582685 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758015020 con 0x7fa75c1034c0 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa764593700 1 -- 172.21.15.115:0/2896582685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75c1a7230 con 0x7fa75c1034c0 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa7537fe700 1 -- 172.21.15.115:0/2896582685 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa758011cc0 con 0x7fa75c1034c0 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.898+0000 7fa7537fe700 1 -- 172.21.15.115:0/2896582685 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758010c80 con 0x7fa75c1034c0 2024-05-09T05:37:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.899+0000 7fa764593700 1 -- 172.21.15.115:0/2896582685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75c1a7790 con 0x7fa75c1034c0 2024-05-09T05:37:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.901+0000 7fa7537fe700 1 -- 172.21.15.115:0/2896582685 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa758018020 con 0x7fa75c1034c0 2024-05-09T05:37:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.902+0000 7fa7537fe700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa7480b2880 0x7fa7480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.902+0000 7fa761b2e700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa7480b2880 0x7fa7480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.902+0000 7fa7537fe700 1 -- 172.21.15.115:0/2896582685 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa7580e1c30 con 0x7fa75c1034c0 2024-05-09T05:37:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.902+0000 7fa772fa9b80 1 Processor -- start 2024-05-09T05:37:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.902+0000 7fa772fa9b80 1 -- start start 2024-05-09T05:37:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa761b2e700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa7480b2880 0x7fa7480b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009990 tx=0x7fa74c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e23acb0 0x55ef4e3431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 0x55ef4e343730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e33d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e260050 con 0x55ef4e23acb0 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e205950 con 0x55ef4e1c0450 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e25c010 con 0x55ef4e33d3b0 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa76232f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e23acb0 0x55ef4e3431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa76232f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e23acb0 0x55ef4e3431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51882/0 (socket says 172.21.15.115:51882) 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa76232f700 1 -- 172.21.15.115:0/77327401 learned_addr learned my addr 172.21.15.115:0/77327401 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa762b30700 1 --2- 172.21.15.115:0/77327401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e33d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa761b2e700 1 --2- 172.21.15.115:0/77327401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 0x55ef4e343730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa76232f700 1 -- 172.21.15.115:0/77327401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 msgr2=0x55ef4e33d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa76232f700 1 --2- 172.21.15.115:0/77327401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e33d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa76232f700 1 -- 172.21.15.115:0/77327401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 msgr2=0x55ef4e343730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa76232f700 1 --2- 172.21.15.115:0/77327401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 0x55ef4e343730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.903+0000 7fa76232f700 1 -- 172.21.15.115:0/77327401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef4e1f64b0 con 0x55ef4e23acb0 2024-05-09T05:37:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.904+0000 7fa76232f700 1 --2- 172.21.15.115:0/77327401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e23acb0 0x55ef4e3431f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa7580a8bf0 tx=0x7fa758004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c44cae1ffbebab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.904+0000 7fa750ff9700 1 -- 172.21.15.115:0/77327401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7580a8ef0 con 0x55ef4e23acb0 2024-05-09T05:37:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.904+0000 7fa750ff9700 1 -- 172.21.15.115:0/77327401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7580ad2a0 con 0x55ef4e23acb0 2024-05-09T05:37:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.904+0000 7fa750ff9700 1 -- 172.21.15.115:0/77327401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7580b1b20 con 0x55ef4e23acb0 2024-05-09T05:37:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 -- 172.21.15.115:0/77327401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e23acb0 msgr2=0x55ef4e3431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/77327401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e23acb0 0x55ef4e3431f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa7580a8bf0 tx=0x7fa758004860 comp rx=0 tx=0).stop 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 -- 172.21.15.115:0/77327401 shutdown_connections 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/77327401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e33d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/77327401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e23acb0 0x55ef4e3431f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/77327401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 0x55ef4e343730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 -- 172.21.15.115:0/77327401 >> 172.21.15.115:0/77327401 conn(0x55ef4e1c3f20 msgr2=0x55ef4e3381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 -- 172.21.15.115:0/77327401 shutdown_connections 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 -- 172.21.15.115:0/77327401 wait complete. 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 -- start start 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.905+0000 7fa772fa9b80 1 Processor -- start 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 0x55ef4e346ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e23acb0 0x55ef4e347530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e33d3b0 0x55ef4e347a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e33e0b0 con 0x55ef4e33d3b0 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e1faad0 con 0x55ef4e1c0450 2024-05-09T05:37:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e353180 con 0x55ef4e23acb0 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa772fa9b80 10 client.0 _ll_register_callbacks cb 0x55ef4e0d1670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa762b30700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e33d3b0 0x55ef4e347a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa762b30700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e33d3b0 0x55ef4e347a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51898/0 (socket says 172.21.15.115:51898) 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa762b30700 1 -- 172.21.15.115:0/2654072658 learned_addr learned my addr 172.21.15.115:0/2654072658 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa76232f700 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 0x55ef4e346ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa761b2e700 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e23acb0 0x55ef4e347530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa762b30700 1 -- 172.21.15.115:0/2654072658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e23acb0 msgr2=0x55ef4e347530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa762b30700 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e23acb0 0x55ef4e347530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa762b30700 1 -- 172.21.15.115:0/2654072658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 msgr2=0x55ef4e346ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa762b30700 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 0x55ef4e346ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa762b30700 1 -- 172.21.15.115:0/2654072658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef4e0db440 con 0x55ef4e33d3b0 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa761b2e700 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e23acb0 0x55ef4e347530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa762b30700 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e33d3b0 0x55ef4e347a70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7540042a0 tx=0x7fa7540042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f221a5e1380e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa745ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa745ffb700 1 -- 172.21.15.115:0/2654072658 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75401a070 con 0x55ef4e33d3b0 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa745ffb700 1 -- 172.21.15.115:0/2654072658 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa754007710 con 0x55ef4e33d3b0 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.906+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2654072658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7580ad2a0 con 0x55ef4e33d3b0 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa745ffb700 1 -- 172.21.15.115:0/2654072658 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa754026050 con 0x55ef4e33d3b0 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa772fa9b80 10 client.15147 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2654072658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef4e354b10 con 0x55ef4e33d3b0 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa745ffb700 1 -- 172.21.15.115:0/2654072658 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7580ad2a0 con 0x55ef4e33d3b0 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa745ffb700 1 -- 172.21.15.115:0/2654072658 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa7540169e0 con 0x55ef4e33d3b0 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa772fa9b80 10 client.15147 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa772fa9b80 10 client.15147 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa772fa9b80 20 client.15147 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ef4e357140 0x55ef4e3585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa772fa9b80 4 client.15147 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.907+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2654072658 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef4e26da30 con 0x55ef4e357140 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.908+0000 7fa76232f700 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ef4e357140 0x55ef4e3585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.908+0000 7fa745ffb700 10 client.15147 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:37:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.908+0000 7fa76232f700 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ef4e357140 0x55ef4e3585b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8de09772f06dfd51 server_cookie=9c49bd3b6928918e in_seq=0 out_seq=0 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa745ffb700 1 -- 172.21.15.115:0/2654072658 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef4e26da30 con 0x55ef4e357140 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa745ffb700 10 client.15147 handle_command_reply: tid=0 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 client.15147 shutdown 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2654072658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e33d3b0 msgr2=0x55ef4e347a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e33d3b0 0x55ef4e347a70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7540042a0 tx=0x7fa7540042d0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2654072658 shutdown_connections 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e1c0450 0x55ef4e346ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ef4e357140 0x55ef4e3585b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e33d3b0 0x55ef4e347a70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2654072658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e23acb0 0x55ef4e347530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2654072658 >> 172.21.15.115:0/2654072658 conn(0x55ef4e1c3f20 msgr2=0x55ef4e331fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa745ffb700 0 client.15147 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2654072658 shutdown_connections 2024-05-09T05:37:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2654072658 wait complete. 2024-05-09T05:37:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.909+0000 7fa772fa9b80 20 client.15147 trim_cache size 0 max 16384 2024-05-09T05:37:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa772fa9b80 1 Processor -- start 2024-05-09T05:37:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa772fa9b80 1 -- start start 2024-05-09T05:37:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e3326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 0x55ef4e33c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e33c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e3568f0 con 0x55ef4e1c0450 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e356fc0 con 0x55ef4e23acb0 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e357690 con 0x55ef4e33d3b0 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa76232f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e3326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa76232f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e3326b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51912/0 (socket says 172.21.15.115:51912) 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa76232f700 1 -- 172.21.15.115:0/785780561 learned_addr learned my addr 172.21.15.115:0/785780561 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.930+0000 7fa761b2e700 1 --2- 172.21.15.115:0/785780561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 0x55ef4e33c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa762b30700 1 --2- 172.21.15.115:0/785780561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e33c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa76232f700 1 -- 172.21.15.115:0/785780561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 msgr2=0x55ef4e33c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa76232f700 1 --2- 172.21.15.115:0/785780561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e33c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa76232f700 1 -- 172.21.15.115:0/785780561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 msgr2=0x55ef4e33c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa76232f700 1 --2- 172.21.15.115:0/785780561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 0x55ef4e33c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa76232f700 1 -- 172.21.15.115:0/785780561 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef4e33db80 con 0x55ef4e1c0450 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa76232f700 1 --2- 172.21.15.115:0/785780561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e3326b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa7580a6160 tx=0x7fa758000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba48f7b7afdd8f76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa747fff700 1 -- 172.21.15.115:0/785780561 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7580a9150 con 0x55ef4e1c0450 2024-05-09T05:37:18.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa747fff700 1 -- 172.21.15.115:0/785780561 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7580a92f0 con 0x55ef4e1c0450 2024-05-09T05:37:18.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.931+0000 7fa747fff700 1 -- 172.21.15.115:0/785780561 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7580a8150 con 0x55ef4e1c0450 2024-05-09T05:37:18.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 -- 172.21.15.115:0/785780561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 msgr2=0x55ef4e3326b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:18.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/785780561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e3326b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa7580a6160 tx=0x7fa758000bc0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 -- 172.21.15.115:0/785780561 shutdown_connections 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/785780561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e33c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/785780561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e3326b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/785780561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 0x55ef4e33c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 -- 172.21.15.115:0/785780561 >> 172.21.15.115:0/785780561 conn(0x55ef4e1c3f20 msgr2=0x55ef4e331fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 -- 172.21.15.115:0/785780561 shutdown_connections 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 -- 172.21.15.115:0/785780561 wait complete. 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 -- start start 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.932+0000 7fa772fa9b80 1 Processor -- start 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e35f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 0x55ef4e35fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa772fa9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e35ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e358430 con 0x55ef4e1c0450 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e357d60 con 0x55ef4e23acb0 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa772fa9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ef4e358b00 con 0x55ef4e33d3b0 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa772fa9b80 10 client.0 _ll_register_callbacks cb 0x55ef4e0d1670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa76232f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e35f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa76232f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e35f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51918/0 (socket says 172.21.15.115:51918) 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa76232f700 1 -- 172.21.15.115:0/2605743000 learned_addr learned my addr 172.21.15.115:0/2605743000 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa761b2e700 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 0x55ef4e35fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa762b30700 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e35ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa76232f700 1 -- 172.21.15.115:0/2605743000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 msgr2=0x55ef4e35ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa76232f700 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e35ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa76232f700 1 -- 172.21.15.115:0/2605743000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 msgr2=0x55ef4e35fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa76232f700 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 0x55ef4e35fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa76232f700 1 -- 172.21.15.115:0/2605743000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef4e3c15c0 con 0x55ef4e1c0450 2024-05-09T05:37:18.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa762b30700 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e35ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa76232f700 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e35f520 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa7580a8910 tx=0x7fa758010fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332992e3102f5665 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa745ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa745ffb700 1 -- 172.21.15.115:0/2605743000 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7580b5070 con 0x55ef4e1c0450 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2605743000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7580a92f0 con 0x55ef4e1c0450 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.933+0000 7fa745ffb700 1 -- 172.21.15.115:0/2605743000 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa7580b1ad0 con 0x55ef4e1c0450 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa745ffb700 1 -- 172.21.15.115:0/2605743000 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa7580be0d0 con 0x55ef4e1c0450 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa772fa9b80 10 client.15153 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2605743000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef4e3c26b0 con 0x55ef4e1c0450 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa745ffb700 1 -- 172.21.15.115:0/2605743000 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7580a92f0 con 0x55ef4e1c0450 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa745ffb700 1 -- 172.21.15.115:0/2605743000 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa7580a4980 con 0x55ef4e1c0450 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa772fa9b80 10 client.15153 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa772fa9b80 10 client.15153 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa772fa9b80 20 client.15153 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ef4e3c4380 0x55ef4e3c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa772fa9b80 4 client.15153 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2605743000 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "status"}) -- 0x55ef4e267ad0 con 0x55ef4e3c4380 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.934+0000 7fa761b2e700 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ef4e3c4380 0x55ef4e3c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa745ffb700 10 client.15153 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa761b2e700 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ef4e3c4380 0x55ef4e3c57f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58107397dfe992de server_cookie=63c850ccd941af35 in_seq=0 out_seq=0 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa745ffb700 1 -- 172.21.15.115:0/2605743000 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ef4e267ad0 con 0x55ef4e3c4380 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa745ffb700 10 client.15153 handle_command_reply: tid=0 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 client.15153 shutdown 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2605743000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 msgr2=0x55ef4e35f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e35f520 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa7580a8910 tx=0x7fa758010fb0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2605743000 shutdown_connections 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ef4e3c4380 0x55ef4e3c57f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ef4e33d3b0 0x55ef4e35ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ef4e1c0450 0x55ef4e35f520 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 --2- 172.21.15.115:0/2605743000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef4e23acb0 0x55ef4e35fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2605743000 >> 172.21.15.115:0/2605743000 conn(0x55ef4e1c3f20 msgr2=0x55ef4e33c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa745ffb700 0 client.15153 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2605743000 shutdown_connections 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.935+0000 7fa772fa9b80 1 -- 172.21.15.115:0/2605743000 wait complete. 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa772fa9b80 20 client.15153 trim_cache size 0 max 16384 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24469, 2024-05-09T05:37:18.934 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:37:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:37:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:37:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 204.16044546500001, 2024-05-09T05:37:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:37:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:37:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:37:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 229.67385160000001 2024-05-09T05:37:18.935 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 -- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa7480b2880 msgr2=0x7fa7480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa7480b2880 0x7fa7480b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009990 tx=0x7fa74c005fb0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 -- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c1034c0 msgr2=0x7fa75c19bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c1034c0 0x7fa75c19bbf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa75800b530 tx=0x7fa75800b840 comp rx=0 tx=0).stop 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 -- 172.21.15.115:0/2896582685 shutdown_connections 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa7480b2880 0x7fa7480b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa75c108740 0x7fa75c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa75c1034c0 0x7fa75c19bbf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 --2- 172.21.15.115:0/2896582685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c107df0 0x7fa75c19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 -- 172.21.15.115:0/2896582685 >> 172.21.15.115:0/2896582685 conn(0x7fa75c0fedc0 msgr2=0x7fa75c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 -- 172.21.15.115:0/2896582685 shutdown_connections 2024-05-09T05:37:18.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:18.936+0000 7fa764593700 1 -- 172.21.15.115:0/2896582685 wait complete. 2024-05-09T05:37:19.627 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T05:37:19.628 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T05:37:19.643 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T05:37:19.643 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T05:37:19.889 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:20.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:20 smithi195 ceph-mon[33844]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:20.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:20 smithi115 ceph-mon[30349]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:20.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:20 smithi161 ceph-mon[33805]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:21.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.614+0000 7f2fd8791700 1 -- 172.21.15.115:0/1258083391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 msgr2=0x7f2fd0109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:21.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.614+0000 7f2fd8791700 1 --2- 172.21.15.115:0/1258083391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 0x7f2fd0109960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009990 tx=0x7f2fc0009ca0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.614+0000 7f2fd8791700 1 -- 172.21.15.115:0/1258083391 shutdown_connections 2024-05-09T05:37:21.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.614+0000 7f2fd8791700 1 --2- 172.21.15.115:0/1258083391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd0109ff0 0x7f2fd010c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.614+0000 7f2fd8791700 1 --2- 172.21.15.115:0/1258083391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fd006c2d0 0x7f2fd0106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.614+0000 7f2fd8791700 1 --2- 172.21.15.115:0/1258083391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 0x7f2fd0109960 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.614+0000 7f2fd8791700 1 -- 172.21.15.115:0/1258083391 >> 172.21.15.115:0/1258083391 conn(0x7f2fd00fe880 msgr2=0x7f2fd0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:21.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.615+0000 7f2fd8791700 1 -- 172.21.15.115:0/1258083391 shutdown_connections 2024-05-09T05:37:21.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.615+0000 7f2fd8791700 1 -- 172.21.15.115:0/1258083391 wait complete. 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.615+0000 7f2fd8791700 1 Processor -- start 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.615+0000 7f2fd8791700 1 -- start start 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd8791700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fd006c2d0 0x7f2fd019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd8791700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 0x7f2fd019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd8791700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd0109ff0 0x7f2fd01a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd8791700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd010fc30 con 0x7f2fd0107480 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd8791700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd010fab0 con 0x7f2fd0109ff0 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd8791700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd010fdb0 con 0x7f2fd006c2d0 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd5d2c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 0x7f2fd019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd5d2c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 0x7f2fd019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51932/0 (socket says 172.21.15.115:51932) 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd5d2c700 1 -- 172.21.15.115:0/3111106262 learned_addr learned my addr 172.21.15.115:0/3111106262 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd652d700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fd006c2d0 0x7f2fd019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd6d2e700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd0109ff0 0x7f2fd01a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd5d2c700 1 -- 172.21.15.115:0/3111106262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fd006c2d0 msgr2=0x7f2fd019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd5d2c700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fd006c2d0 0x7f2fd019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd5d2c700 1 -- 172.21.15.115:0/3111106262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd0109ff0 msgr2=0x7f2fd01a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd5d2c700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd0109ff0 0x7f2fd01a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd5d2c700 1 -- 172.21.15.115:0/3111106262 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd01a2370 con 0x7f2fd0107480 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd6d2e700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd0109ff0 0x7f2fd01a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fd5d2c700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 0x7f2fd019ad50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00098e0 tx=0x7f2fc00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.616+0000 7f2fc77fe700 1 -- 172.21.15.115:0/3111106262 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0004830 con 0x7f2fd0107480 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.617+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd01a2600 con 0x7f2fd0107480 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.617+0000 7f2fc77fe700 1 -- 172.21.15.115:0/3111106262 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fc0003b90 con 0x7f2fd0107480 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.617+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd01a2a50 con 0x7f2fd0107480 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.617+0000 7f2fc77fe700 1 -- 172.21.15.115:0/3111106262 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0005280 con 0x7f2fd0107480 2024-05-09T05:37:21.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.618+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd00512c0 con 0x7f2fd0107480 2024-05-09T05:37:21.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.619+0000 7f2fc77fe700 1 -- 172.21.15.115:0/3111106262 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2fc0005420 con 0x7f2fd0107480 2024-05-09T05:37:21.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.620+0000 7f2fc77fe700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2fbc0b2880 0x7f2fbc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:21.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.620+0000 7f2fd652d700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2fbc0b2880 0x7f2fbc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:21.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.620+0000 7f2fc77fe700 1 -- 172.21.15.115:0/3111106262 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2fc00e38f0 con 0x7f2fd0107480 2024-05-09T05:37:21.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.620+0000 7f2fd652d700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2fbc0b2880 0x7f2fbc0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009550 tx=0x7f2fcc0091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.625+0000 7f2fc77fe700 1 -- 172.21.15.115:0/3111106262 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2fc0013070 con 0x7f2fd0107480 2024-05-09T05:37:21.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.844+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f2fd0063c00 con 0x7f2fbc0b2880 2024-05-09T05:37:21.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.845+0000 7f2fc77fe700 1 -- 172.21.15.115:0/3111106262 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f2fd0063c00 con 0x7f2fbc0b2880 2024-05-09T05:37:21.838 INFO:teuthology.orchestra.run.smithi115.stdout:Backend: cephadm 2024-05-09T05:37:21.839 INFO:teuthology.orchestra.run.smithi115.stdout:Available: Yes 2024-05-09T05:37:21.839 INFO:teuthology.orchestra.run.smithi115.stdout:Paused: No 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.847+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2fbc0b2880 msgr2=0x7f2fbc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.847+0000 7f2fd8791700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2fbc0b2880 0x7f2fbc0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009550 tx=0x7f2fcc0091b0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.847+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 msgr2=0x7f2fd019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.847+0000 7f2fd8791700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 0x7f2fd019ad50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00098e0 tx=0x7f2fc00047b0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.847+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 shutdown_connections 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.847+0000 7f2fd8791700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2fbc0b2880 0x7f2fbc0b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.847+0000 7f2fd8791700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd0109ff0 0x7f2fd01a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.848+0000 7f2fd8791700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fd006c2d0 0x7f2fd019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.848+0000 7f2fd8791700 1 --2- 172.21.15.115:0/3111106262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fd0107480 0x7f2fd019ad50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.848+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 >> 172.21.15.115:0/3111106262 conn(0x7f2fd00fe880 msgr2=0x7f2fd01058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:21.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.848+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 shutdown_connections 2024-05-09T05:37:21.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:21.848+0000 7f2fd8791700 1 -- 172.21.15.115:0/3111106262 wait complete. 2024-05-09T05:37:22.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:22 smithi115 ceph-mon[30349]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:22.540 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T05:37:22.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:22 smithi195 ceph-mon[33844]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:22.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:22 smithi161 ceph-mon[33805]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:23.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:23 smithi195 ceph-mon[33844]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:23.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:23 smithi115 ceph-mon[30349]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:23.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:23 smithi161 ceph-mon[33805]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 -- 172.21.15.115:0/3406362415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 msgr2=0x7f9fac10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/3406362415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 0x7f9fac10ad60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400c210 tx=0x7f9fa400c520 comp rx=0 tx=0).stop 2024-05-09T05:37:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 -- 172.21.15.115:0/3406362415 shutdown_connections 2024-05-09T05:37:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/3406362415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fac1029e0 0x7f9fac102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/3406362415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 0x7f9fac10ad60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/3406362415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fac103b10 0x7f9fac103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:24.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 -- 172.21.15.115:0/3406362415 >> 172.21.15.115:0/3406362415 conn(0x7f9fac0fe020 msgr2=0x7f9fac1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:24.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 -- 172.21.15.115:0/3406362415 shutdown_connections 2024-05-09T05:37:24.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 -- 172.21.15.115:0/3406362415 wait complete. 2024-05-09T05:37:24.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.484+0000 7f9fb11b7700 1 Processor -- start 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fb11b7700 1 -- start start 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fb11b7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fac1029e0 0x7f9fac19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fb11b7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fac103b10 0x7f9fac19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fb11b7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 0x7f9fac1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fb11b7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10db00 con 0x7f9fac1044b0 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fb11b7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10d980 con 0x7f9fac1029e0 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fb11b7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10dc80 con 0x7f9fac103b10 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fab59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 0x7f9fac1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fab59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 0x7f9fac1a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55046/0 (socket says 172.21.15.115:55046) 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fab59e700 1 -- 172.21.15.115:0/2215908109 learned_addr learned my addr 172.21.15.115:0/2215908109 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9faad9d700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fac1029e0 0x7f9fac19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9faa59c700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fac103b10 0x7f9fac19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fab59e700 1 -- 172.21.15.115:0/2215908109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fac103b10 msgr2=0x7f9fac19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fab59e700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fac103b10 0x7f9fac19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fab59e700 1 -- 172.21.15.115:0/2215908109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fac1029e0 msgr2=0x7f9fac19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fab59e700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fac1029e0 0x7f9fac19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.485+0000 7f9fab59e700 1 -- 172.21.15.115:0/2215908109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac1a2390 con 0x7f9fac1044b0 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.486+0000 7f9fab59e700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 0x7f9fac1a1d10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4004fd0 tx=0x7f9fa400a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.486+0000 7f9f93fff700 1 -- 172.21.15.115:0/2215908109 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4022020 con 0x7f9fac1044b0 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.486+0000 7f9f93fff700 1 -- 172.21.15.115:0/2215908109 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fa4008d50 con 0x7f9fac1044b0 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.486+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac1a2620 con 0x7f9fac1044b0 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.486+0000 7f9f93fff700 1 -- 172.21.15.115:0/2215908109 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa401c740 con 0x7f9fac1044b0 2024-05-09T05:37:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.486+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac1a2b50 con 0x7f9fac1044b0 2024-05-09T05:37:24.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.487+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac0512c0 con 0x7f9fac1044b0 2024-05-09T05:37:24.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.489+0000 7f9f93fff700 1 -- 172.21.15.115:0/2215908109 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9fa4025020 con 0x7f9fac1044b0 2024-05-09T05:37:24.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.489+0000 7f9f93fff700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9f940b2880 0x7f9f940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:24.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.489+0000 7f9faad9d700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9f940b2880 0x7f9f940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:24.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.489+0000 7f9f93fff700 1 -- 172.21.15.115:0/2215908109 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9fa4010020 con 0x7f9fac1044b0 2024-05-09T05:37:24.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.490+0000 7f9faad9d700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9f940b2880 0x7f9f940b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9fac103a10 tx=0x7f9f9c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:24.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.493+0000 7f9f93fff700 1 -- 172.21.15.115:0/2215908109 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9fa40abb90 con 0x7f9fac1044b0 2024-05-09T05:37:24.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:24 smithi195 ceph-mon[33844]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:24.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:24 smithi115 ceph-mon[30349]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:24.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:24 smithi161 ceph-mon[33805]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:24.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.704+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9fac063c00 con 0x7f9f940b2880 2024-05-09T05:37:24.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.718+0000 7f9f93fff700 1 -- 172.21.15.115:0/2215908109 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f9fac063c00 con 0x7f9f940b2880 2024-05-09T05:37:24.711 INFO:teuthology.orchestra.run.smithi115.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T05:37:24.711 INFO:teuthology.orchestra.run.smithi115.stdout:mds.a smithi115 running (3m) 45s ago 3m 85.0M - 19.0.0-3594-gc9da4049 db794d706f49 215309d10b0c 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.b smithi161 running (3m) 90s ago 3m 85.5M - 19.0.0-3594-gc9da4049 db794d706f49 937d4c8c6dd8 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.c smithi195 running (4m) 90s ago 4m 85.1M - 19.0.0-3594-gc9da4049 db794d706f49 c411b27886fd 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.d smithi115 running (4m) 45s ago 4m 82.2M - 19.0.0-3594-gc9da4049 db794d706f49 3299ca8f441b 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.e smithi161 running (4m) 90s ago 4m 84.7M - 19.0.0-3594-gc9da4049 db794d706f49 5bd2dbf594bf 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.f smithi195 running (4m) 90s ago 4m 88.1M - 19.0.0-3594-gc9da4049 db794d706f49 993232dbb6ea 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.g smithi115 running (3m) 45s ago 3m 85.0M - 19.0.0-3594-gc9da4049 db794d706f49 94da5797eb46 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.h smithi161 running (3m) 90s ago 3m 85.4M - 19.0.0-3594-gc9da4049 db794d706f49 460ce5a078fc 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.i smithi195 running (3m) 90s ago 3m 93.6M - 19.0.0-3594-gc9da4049 db794d706f49 c7a2a7b85874 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.j smithi115 running (4m) 45s ago 4m 83.7M - 19.0.0-3594-gc9da4049 db794d706f49 d8fa90235cf3 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.k smithi161 running (4m) 90s ago 4m 83.9M - 19.0.0-3594-gc9da4049 db794d706f49 f22c6e34e52f 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mds.l smithi195 running (4m) 90s ago 4m 88.6M - 19.0.0-3594-gc9da4049 db794d706f49 4ab77f7ac162 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.x smithi115 *:9283,8765 running (11m) 45s ago 11m 600M - 19.0.0-3594-gc9da4049 db794d706f49 1b4a7b30461e 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.y smithi161 *:8443,8765 running (9m) 90s ago 9m 534M - 19.0.0-3594-gc9da4049 db794d706f49 550fab3ac83c 2024-05-09T05:37:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:mgr.z smithi195 *:8443,8765 running (9m) 90s ago 9m 534M - 19.0.0-3594-gc9da4049 db794d706f49 25037a9333af 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:mon.a smithi115 running (11m) 45s ago 11m 122M 2048M 19.0.0-3594-gc9da4049 db794d706f49 8cbc383f681f 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:mon.b smithi161 running (9m) 90s ago 9m 106M 2048M 19.0.0-3594-gc9da4049 db794d706f49 72441f6aa892 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:mon.c smithi195 running (10m) 90s ago 10m 110M 2048M 19.0.0-3594-gc9da4049 db794d706f49 2e0df25d1365 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 smithi115 running (9m) 45s ago 9m 114M 4272M 19.0.0-3594-gc9da4049 db794d706f49 7b821fadca5e 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 smithi161 running (8m) 90s ago 8m 113M 4272M 19.0.0-3594-gc9da4049 db794d706f49 3189ba1ef780 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 smithi195 running (8m) 90s ago 8m 114M 4272M 19.0.0-3594-gc9da4049 db794d706f49 27bcae66ce23 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 smithi115 running (8m) 45s ago 8m 115M 4272M 19.0.0-3594-gc9da4049 db794d706f49 222c8a418787 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 smithi161 running (7m) 90s ago 7m 110M 4272M 19.0.0-3594-gc9da4049 db794d706f49 0389c2e2d85c 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 smithi195 running (7m) 90s ago 7m 114M 4272M 19.0.0-3594-gc9da4049 db794d706f49 19406b5b5727 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6 smithi115 running (6m) 45s ago 6m 109M 4272M 19.0.0-3594-gc9da4049 db794d706f49 3c1b714fe547 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7 smithi161 running (6m) 90s ago 6m 114M 4272M 19.0.0-3594-gc9da4049 db794d706f49 9ff86490d7c2 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.8 smithi195 running (6m) 90s ago 6m 114M 4272M 19.0.0-3594-gc9da4049 db794d706f49 d380bb6ecde0 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.9 smithi115 running (5m) 45s ago 5m 123M 4272M 19.0.0-3594-gc9da4049 db794d706f49 d7fd055542de 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.10 smithi161 running (5m) 90s ago 5m 123M 4272M 19.0.0-3594-gc9da4049 db794d706f49 f53fbaf2a392 2024-05-09T05:37:24.713 INFO:teuthology.orchestra.run.smithi115.stdout:osd.11 smithi195 running (4m) 90s ago 4m 109M 4272M 19.0.0-3594-gc9da4049 db794d706f49 5e0565e2fe3b 2024-05-09T05:37:24.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9f940b2880 msgr2=0x7f9f940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:24.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9f940b2880 0x7f9f940b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9fac103a10 tx=0x7f9f9c009190 comp rx=0 tx=0).stop 2024-05-09T05:37:24.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 msgr2=0x7f9fac1a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:24.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 0x7f9fac1a1d10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4004fd0 tx=0x7f9fa400a670 comp rx=0 tx=0).stop 2024-05-09T05:37:24.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 shutdown_connections 2024-05-09T05:37:24.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f9f940b2880 0x7f9f940b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:24.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fac1029e0 0x7f9fac19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:24.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fac1044b0 0x7f9fac1a1d10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:24.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 --2- 172.21.15.115:0/2215908109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fac103b10 0x7f9fac19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:24.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.720+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 >> 172.21.15.115:0/2215908109 conn(0x7f9fac0fe020 msgr2=0x7f9fac106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:24.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.721+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 shutdown_connections 2024-05-09T05:37:24.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:24.721+0000 7f9fb11b7700 1 -- 172.21.15.115:0/2215908109 wait complete. 2024-05-09T05:37:25.437 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T05:37:25.684 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:26.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:26 smithi195 ceph-mon[33844]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:26.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:26 smithi195 ceph-mon[33844]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:26.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:26 smithi115 ceph-mon[30349]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:26.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:26 smithi115 ceph-mon[30349]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:26.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:26 smithi161 ceph-mon[33805]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:26.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:26 smithi161 ceph-mon[33805]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:27.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 -- 172.21.15.115:0/662944864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e410a110 msgr2=0x7f24e410c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:27.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/662944864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e410a110 0x7f24e410c530 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00b0a0 tx=0x7f24dc00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 -- 172.21.15.115:0/662944864 shutdown_connections 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/662944864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e410a110 0x7f24e410c530 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/662944864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e4104e50 0x7f24e4107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/662944864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24e41077b0 0x7f24e4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 -- 172.21.15.115:0/662944864 >> 172.21.15.115:0/662944864 conn(0x7f24e40fe890 msgr2=0x7f24e4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 -- 172.21.15.115:0/662944864 shutdown_connections 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 -- 172.21.15.115:0/662944864 wait complete. 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 Processor -- start 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.425+0000 7f24ec5e2700 1 -- start start 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24ec5e2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e4104e50 0x7f24e419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24ec5e2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e41077b0 0x7f24e419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24ec5e2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24e410a110 0x7f24e41a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24ec5e2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f24e410fc60 con 0x7f24e41077b0 2024-05-09T05:37:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24ec5e2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f24e410fae0 con 0x7f24e410a110 2024-05-09T05:37:27.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24ec5e2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24e410fde0 con 0x7f24e4104e50 2024-05-09T05:37:27.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24e9b7d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e41077b0 0x7f24e419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:27.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24e9b7d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e41077b0 0x7f24e419ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55062/0 (socket says 172.21.15.115:55062) 2024-05-09T05:37:27.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24e9b7d700 1 -- 172.21.15.115:0/57188679 learned_addr learned my addr 172.21.15.115:0/57188679 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:27.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24eab7f700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24e410a110 0x7f24e41a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:27.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24ea37e700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e4104e50 0x7f24e419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:27.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24e9b7d700 1 -- 172.21.15.115:0/57188679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e4104e50 msgr2=0x7f24e419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:27.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24e9b7d700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e4104e50 0x7f24e419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24e9b7d700 1 -- 172.21.15.115:0/57188679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24e410a110 msgr2=0x7f24e41a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:27.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24e9b7d700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24e410a110 0x7f24e41a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24e9b7d700 1 -- 172.21.15.115:0/57188679 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e41a2270 con 0x7f24e41077b0 2024-05-09T05:37:27.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24eab7f700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24e410a110 0x7f24e41a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:27.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24ea37e700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e4104e50 0x7f24e419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:27.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24e9b7d700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e41077b0 0x7f24e419ace0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24e000c9b0 tx=0x7f24e000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:27.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24d77fe700 1 -- 172.21.15.115:0/57188679 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0007b00 con 0x7f24e41077b0 2024-05-09T05:37:27.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.426+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e41a2560 con 0x7f24e41077b0 2024-05-09T05:37:27.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.427+0000 7f24d77fe700 1 -- 172.21.15.115:0/57188679 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24e0007ca0 con 0x7f24e41077b0 2024-05-09T05:37:27.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.427+0000 7f24d77fe700 1 -- 172.21.15.115:0/57188679 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0018660 con 0x7f24e41077b0 2024-05-09T05:37:27.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.427+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e41a2af0 con 0x7f24e41077b0 2024-05-09T05:37:27.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.427+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e40512c0 con 0x7f24e41077b0 2024-05-09T05:37:27.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.430+0000 7f24d77fe700 1 -- 172.21.15.115:0/57188679 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f24e000f450 con 0x7f24e41077b0 2024-05-09T05:37:27.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.431+0000 7f24d77fe700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f24d00b2880 0x7f24d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:27.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.431+0000 7f24ea37e700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f24d00b2880 0x7f24d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:27.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.431+0000 7f24d77fe700 1 -- 172.21.15.115:0/57188679 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f24e00e2900 con 0x7f24e41077b0 2024-05-09T05:37:27.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.431+0000 7f24ea37e700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f24d00b2880 0x7f24d00b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f24d8005fd0 tx=0x7f24d8005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:27.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.435+0000 7f24d77fe700 1 -- 172.21.15.115:0/57188679 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24e0014030 con 0x7f24e41077b0 2024-05-09T05:37:27.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.646+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f24e4063c00 con 0x7f24d00b2880 2024-05-09T05:37:27.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.652+0000 7f24d77fe700 1 -- 172.21.15.115:0/57188679 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f24e4063c00 con 0x7f24d00b2880 2024-05-09T05:37:27.646 INFO:teuthology.orchestra.run.smithi115.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T05:37:27.646 INFO:teuthology.orchestra.run.smithi115.stdout:mds.all 12/12 93s ago 4m smithi115=a;smithi115=d;smithi115=g;smithi115=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-05-09T05:37:27.646 INFO:teuthology.orchestra.run.smithi115.stdout:mgr 3/3 93s ago 9m smithi115=x;smithi161=y;smithi195=z;count:3 2024-05-09T05:37:27.646 INFO:teuthology.orchestra.run.smithi115.stdout:mon 3/3 93s ago 10m smithi115:172.21.15.115=a;smithi161:172.21.15.161=b;smithi195:172.21.15.195=c;count:3 2024-05-09T05:37:27.646 INFO:teuthology.orchestra.run.smithi115.stdout:osd 12 93s ago - 2024-05-09T05:37:27.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f24d00b2880 msgr2=0x7f24d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:27.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f24d00b2880 0x7f24d00b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f24d8005fd0 tx=0x7f24d8005dc0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e41077b0 msgr2=0x7f24e419ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:27.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e41077b0 0x7f24e419ace0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24e000c9b0 tx=0x7f24e000cd70 comp rx=0 tx=0).stop 2024-05-09T05:37:27.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 shutdown_connections 2024-05-09T05:37:27.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f24d00b2880 0x7f24d00b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24e410a110 0x7f24e41a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e4104e50 0x7f24e419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 --2- 172.21.15.115:0/57188679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24e41077b0 0x7f24e419ace0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:27.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 >> 172.21.15.115:0/57188679 conn(0x7f24e40fe890 msgr2=0x7f24e4105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:27.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 shutdown_connections 2024-05-09T05:37:27.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:27.655+0000 7f24ec5e2700 1 -- 172.21.15.115:0/57188679 wait complete. 2024-05-09T05:37:28.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T05:37:28.525 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:28 smithi195 ceph-mon[33844]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:28 smithi115 ceph-mon[30349]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:28 smithi161 ceph-mon[33805]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:29.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:29 smithi195 ceph-mon[33844]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:29.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:29 smithi115 ceph-mon[30349]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:29.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:29 smithi161 ceph-mon[33805]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.281+0000 7f2b4b725700 1 -- 172.21.15.115:0/1756843783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102ca0 msgr2=0x7f2b4410c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.281+0000 7f2b4b725700 1 --2- 172.21.15.115:0/1756843783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102ca0 0x7f2b4410c8c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00b0a0 tx=0x7f2b3c00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.281+0000 7f2b4b725700 1 -- 172.21.15.115:0/1756843783 shutdown_connections 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.281+0000 7f2b4b725700 1 --2- 172.21.15.115:0/1756843783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102ca0 0x7f2b4410c8c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.281+0000 7f2b4b725700 1 --2- 172.21.15.115:0/1756843783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b44105900 0x7f2b44105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.281+0000 7f2b4b725700 1 --2- 172.21.15.115:0/1756843783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b44102350 0x7f2b44102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.281+0000 7f2b4b725700 1 -- 172.21.15.115:0/1756843783 >> 172.21.15.115:0/1756843783 conn(0x7f2b440fe020 msgr2=0x7f2b441004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.281+0000 7f2b4b725700 1 -- 172.21.15.115:0/1756843783 shutdown_connections 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.281+0000 7f2b4b725700 1 -- 172.21.15.115:0/1756843783 wait complete. 2024-05-09T05:37:30.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.282+0000 7f2b4b725700 1 Processor -- start 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.282+0000 7f2b4b725700 1 -- start start 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.282+0000 7f2b4b725700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102350 0x7f2b4419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.282+0000 7f2b4b725700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b44102ca0 0x7f2b4419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.282+0000 7f2b4b725700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b44105900 0x7f2b441a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.282+0000 7f2b4b725700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4410fd70 con 0x7f2b44102350 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.282+0000 7f2b4b725700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4410fbf0 con 0x7f2b44105900 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.282+0000 7f2b4b725700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4410fef0 con 0x7f2b44102ca0 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.282+0000 7f2b494c1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102350 0x7f2b4419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b494c1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102350 0x7f2b4419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55088/0 (socket says 172.21.15.115:55088) 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b494c1700 1 -- 172.21.15.115:0/2504876695 learned_addr learned my addr 172.21.15.115:0/2504876695 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:30.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b49cc2700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b44105900 0x7f2b441a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b48cc0700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b44102ca0 0x7f2b4419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b494c1700 1 -- 172.21.15.115:0/2504876695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b44102ca0 msgr2=0x7f2b4419af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b494c1700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b44102ca0 0x7f2b4419af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b494c1700 1 -- 172.21.15.115:0/2504876695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b44105900 msgr2=0x7f2b441a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b494c1700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b44105900 0x7f2b441a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b494c1700 1 -- 172.21.15.115:0/2504876695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b441a2530 con 0x7f2b44102350 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b494c1700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102350 0x7f2b4419a9d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000b4d0 tx=0x7f2b4000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b3a7fc700 1 -- 172.21.15.115:0/2504876695 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40015020 con 0x7f2b44102350 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b3a7fc700 1 -- 172.21.15.115:0/2504876695 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b40011cc0 con 0x7f2b44102350 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b441a2820 con 0x7f2b44102350 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b3a7fc700 1 -- 172.21.15.115:0/2504876695 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40010c30 con 0x7f2b44102350 2024-05-09T05:37:30.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.283+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b441a2d10 con 0x7f2b44102350 2024-05-09T05:37:30.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.284+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b440512c0 con 0x7f2b44102350 2024-05-09T05:37:30.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.286+0000 7f2b3a7fc700 1 -- 172.21.15.115:0/2504876695 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2b40018020 con 0x7f2b44102350 2024-05-09T05:37:30.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.286+0000 7f2b3a7fc700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2b300b2880 0x7f2b300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:30.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.287+0000 7f2b48cc0700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2b300b2880 0x7f2b300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:30.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.287+0000 7f2b3a7fc700 1 -- 172.21.15.115:0/2504876695 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2b400e1d20 con 0x7f2b44102350 2024-05-09T05:37:30.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.287+0000 7f2b48cc0700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2b300b2880 0x7f2b300b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2b34006fd0 tx=0x7f2b34008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:30.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.291+0000 7f2b3a7fc700 1 -- 172.21.15.115:0/2504876695 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2b4001f020 con 0x7f2b44102350 2024-05-09T05:37:30.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.505+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2b44063c00 con 0x7f2b300b2880 2024-05-09T05:37:30.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.506+0000 7f2b3a7fc700 1 -- 172.21.15.115:0/2504876695 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2b44063c00 con 0x7f2b300b2880 2024-05-09T05:37:30.499 INFO:teuthology.orchestra.run.smithi115.stdout:HOST ADDR LABELS STATUS 2024-05-09T05:37:30.500 INFO:teuthology.orchestra.run.smithi115.stdout:smithi115 172.21.15.115 2024-05-09T05:37:30.500 INFO:teuthology.orchestra.run.smithi115.stdout:smithi161 172.21.15.161 2024-05-09T05:37:30.500 INFO:teuthology.orchestra.run.smithi115.stdout:smithi195 172.21.15.195 2024-05-09T05:37:30.500 INFO:teuthology.orchestra.run.smithi115.stdout:3 hosts in cluster 2024-05-09T05:37:30.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2b300b2880 msgr2=0x7f2b300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:30.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2b300b2880 0x7f2b300b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2b34006fd0 tx=0x7f2b34008040 comp rx=0 tx=0).stop 2024-05-09T05:37:30.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102350 msgr2=0x7f2b4419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:30.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102350 0x7f2b4419a9d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000b4d0 tx=0x7f2b4000b7e0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 shutdown_connections 2024-05-09T05:37:30.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f2b300b2880 0x7f2b300b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b44105900 0x7f2b441a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b44102350 0x7f2b4419a9d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 --2- 172.21.15.115:0/2504876695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b44102ca0 0x7f2b4419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:30.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 >> 172.21.15.115:0/2504876695 conn(0x7f2b440fe020 msgr2=0x7f2b441088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:30.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 shutdown_connections 2024-05-09T05:37:30.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:30.509+0000 7f2b4b725700 1 -- 172.21.15.115:0/2504876695 wait complete. 2024-05-09T05:37:30.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:30 smithi195 ceph-mon[33844]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:30.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:30 smithi115 ceph-mon[30349]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:30.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:30 smithi161 ceph-mon[33805]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:31.391 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T05:37:31.640 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:32.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:32 smithi195 ceph-mon[33844]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:32.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:32 smithi195 ceph-mon[33844]: pgmap v406: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:32.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:32 smithi115 ceph-mon[30349]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:32.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:32 smithi115 ceph-mon[30349]: pgmap v406: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:32.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:32 smithi161 ceph-mon[33805]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:32.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:32 smithi161 ceph-mon[33805]: pgmap v406: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:33.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.342+0000 7f6ab27b7700 1 -- 172.21.15.115:0/3715679752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac104d10 msgr2=0x7f6aac10b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:33.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.342+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/3715679752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac104d10 0x7f6aac10b5c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400b0a0 tx=0x7f6aa400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.342+0000 7f6ab27b7700 1 -- 172.21.15.115:0/3715679752 shutdown_connections 2024-05-09T05:37:33.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.342+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/3715679752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac104d10 0x7f6aac10b5c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.342+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/3715679752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac103240 0x7f6aac103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.342+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/3715679752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac104370 0x7f6aac104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.342+0000 7f6ab27b7700 1 -- 172.21.15.115:0/3715679752 >> 172.21.15.115:0/3715679752 conn(0x7f6aac0fe880 msgr2=0x7f6aac100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:33.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.343+0000 7f6ab27b7700 1 -- 172.21.15.115:0/3715679752 shutdown_connections 2024-05-09T05:37:33.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.343+0000 7f6ab27b7700 1 -- 172.21.15.115:0/3715679752 wait complete. 2024-05-09T05:37:33.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.343+0000 7f6ab27b7700 1 Processor -- start 2024-05-09T05:37:33.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.343+0000 7f6ab27b7700 1 -- start start 2024-05-09T05:37:33.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.343+0000 7f6ab27b7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac103240 0x7f6aac1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.343+0000 7f6ab27b7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac104370 0x7f6aac199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6ab27b7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac104d10 0x7f6aac1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6ab27b7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac10eb50 con 0x7f6aac103240 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6ab27b7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac10e9d0 con 0x7f6aac104370 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6ab27b7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac10ecd0 con 0x7f6aac104d10 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aabfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac103240 0x7f6aac1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aabfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac103240 0x7f6aac1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59356/0 (socket says 172.21.15.115:59356) 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aabfff700 1 -- 172.21.15.115:0/1299327785 learned_addr learned my addr 172.21.15.115:0/1299327785 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6ab0d54700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac104d10 0x7f6aac1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aa3fff700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac104370 0x7f6aac199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aabfff700 1 -- 172.21.15.115:0/1299327785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac104d10 msgr2=0x7f6aac1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aabfff700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac104d10 0x7f6aac1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aabfff700 1 -- 172.21.15.115:0/1299327785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac104370 msgr2=0x7f6aac199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aabfff700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac104370 0x7f6aac199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aabfff700 1 -- 172.21.15.115:0/1299327785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac1a1200 con 0x7f6aac103240 2024-05-09T05:37:33.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aa3fff700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac104370 0x7f6aac199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:33.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aabfff700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac103240 0x7f6aac1996a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c007b50 tx=0x7f6a9c007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:33.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aa9ffb700 1 -- 172.21.15.115:0/1299327785 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c019020 con 0x7f6aac103240 2024-05-09T05:37:33.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aa9ffb700 1 -- 172.21.15.115:0/1299327785 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a9c0044b0 con 0x7f6aac103240 2024-05-09T05:37:33.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac06b670 con 0x7f6aac103240 2024-05-09T05:37:33.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.344+0000 7f6aa9ffb700 1 -- 172.21.15.115:0/1299327785 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c003e60 con 0x7f6aac103240 2024-05-09T05:37:33.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.345+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac06bbb0 con 0x7f6aac103240 2024-05-09T05:37:33.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.345+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac0512c0 con 0x7f6aac103240 2024-05-09T05:37:33.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.347+0000 7f6aa9ffb700 1 -- 172.21.15.115:0/1299327785 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6a9c004000 con 0x7f6aac103240 2024-05-09T05:37:33.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.348+0000 7f6aa9ffb700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6a980b2880 0x7f6a980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:33.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.348+0000 7f6aa3fff700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6a980b2880 0x7f6a980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:33.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.348+0000 7f6aa9ffb700 1 -- 172.21.15.115:0/1299327785 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6a9c0e1b00 con 0x7f6aac103240 2024-05-09T05:37:33.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.348+0000 7f6aa3fff700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6a980b2880 0x7f6a980b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a94005950 tx=0x7f6a940058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:33.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.351+0000 7f6aa9ffb700 1 -- 172.21.15.115:0/1299327785 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a9c0a9550 con 0x7f6aac103240 2024-05-09T05:37:33.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.566+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 --> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6aac06bed0 con 0x7f6a980b2880 2024-05-09T05:37:33.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.568+0000 7f6aa9ffb700 1 -- 172.21.15.115:0/1299327785 <== mgr.14150 v2:172.21.15.115:6800/2041699318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6aac06bed0 con 0x7f6a980b2880 2024-05-09T05:37:33.562 INFO:teuthology.orchestra.run.smithi115.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T05:37:33.562 INFO:teuthology.orchestra.run.smithi115.stdout:smithi115 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UU400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-09T05:37:33.562 INFO:teuthology.orchestra.run.smithi115.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-09T05:37:33.562 INFO:teuthology.orchestra.run.smithi115.stdout:smithi195 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300FS480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-09T05:37:33.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6a980b2880 msgr2=0x7f6a980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:33.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6a980b2880 0x7f6a980b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a94005950 tx=0x7f6a940058e0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac103240 msgr2=0x7f6aac1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:33.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac103240 0x7f6aac1996a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c007b50 tx=0x7f6a9c007e60 comp rx=0 tx=0).stop 2024-05-09T05:37:33.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 shutdown_connections 2024-05-09T05:37:33.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6a980b2880 0x7f6a980b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aac104d10 0x7f6aac1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aac103240 0x7f6aac1996a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 --2- 172.21.15.115:0/1299327785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac104370 0x7f6aac199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:33.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 >> 172.21.15.115:0/1299327785 conn(0x7f6aac0fe880 msgr2=0x7f6aac1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:33.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 shutdown_connections 2024-05-09T05:37:33.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:33.571+0000 7f6ab27b7700 1 -- 172.21.15.115:0/1299327785 wait complete. 2024-05-09T05:37:34.281 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T05:37:34.290 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T05:37:34.290 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T05:37:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:34.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:34 smithi195 ceph-mon[33844]: pgmap v407: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:34 smithi115 ceph-mon[30349]: pgmap v407: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:34.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:34 smithi161 ceph-mon[33805]: pgmap v407: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:35.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:35 smithi195 ceph-mon[33844]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:35.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:35 smithi115 ceph-mon[30349]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:35.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:35 smithi161 ceph-mon[33805]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:37:36.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.284+0000 7f7b78e39700 1 -- 172.21.15.115:0/694081726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b74104380 msgr2=0x7f7b74104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:36.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.284+0000 7f7b78e39700 1 --2- 172.21.15.115:0/694081726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b74104380 0x7f7b74104790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009990 tx=0x7f7b68009ca0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.284+0000 7f7b78e39700 1 -- 172.21.15.115:0/694081726 shutdown_connections 2024-05-09T05:37:36.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.284+0000 7f7b78e39700 1 --2- 172.21.15.115:0/694081726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74103250 0x7f7b74103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.284+0000 7f7b78e39700 1 --2- 172.21.15.115:0/694081726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b74104d20 0x7f7b7410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.284+0000 7f7b78e39700 1 --2- 172.21.15.115:0/694081726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b74104380 0x7f7b74104790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.284+0000 7f7b78e39700 1 -- 172.21.15.115:0/694081726 >> 172.21.15.115:0/694081726 conn(0x7f7b740fe890 msgr2=0x7f7b74100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.284+0000 7f7b78e39700 1 -- 172.21.15.115:0/694081726 shutdown_connections 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.284+0000 7f7b78e39700 1 -- 172.21.15.115:0/694081726 wait complete. 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b78e39700 1 Processor -- start 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b78e39700 1 -- start start 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b78e39700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74103250 0x7f7b74199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b78e39700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b74104380 0x7f7b74199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b78e39700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b74104d20 0x7f7b741a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b78e39700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410eb60 con 0x7f7b74104380 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b78e39700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410e9e0 con 0x7f7b74103250 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b78e39700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410ece0 con 0x7f7b74104d20 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b7259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74103250 0x7f7b74199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b71d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b74104380 0x7f7b74199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b7259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74103250 0x7f7b74199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:57488/0 (socket says 172.21.15.115:57488) 2024-05-09T05:37:36.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.285+0000 7f7b7259c700 1 -- 172.21.15.115:0/3490606388 learned_addr learned my addr 172.21.15.115:0/3490606388 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b72d9d700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b74104d20 0x7f7b741a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b71d9b700 1 -- 172.21.15.115:0/3490606388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b74104d20 msgr2=0x7f7b741a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b71d9b700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b74104d20 0x7f7b741a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b71d9b700 1 -- 172.21.15.115:0/3490606388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74103250 msgr2=0x7f7b74199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b71d9b700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74103250 0x7f7b74199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b71d9b700 1 -- 172.21.15.115:0/3490606388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b74075aa0 con 0x7f7b74104380 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b7259c700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74103250 0x7f7b74199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b71d9b700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b74104380 0x7f7b74199c70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7b68006010 tx=0x7f7b680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b637fe700 1 -- 172.21.15.115:0/3490606388 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68027070 con 0x7f7b74104380 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b637fe700 1 -- 172.21.15.115:0/3490606388 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b68004020 con 0x7f7b74104380 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b637fe700 1 -- 172.21.15.115:0/3490606388 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68022c80 con 0x7f7b74104380 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b74075d30 con 0x7f7b74104380 2024-05-09T05:37:36.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.286+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b740761c0 con 0x7f7b74104380 2024-05-09T05:37:36.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.287+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b740512c0 con 0x7f7b74104380 2024-05-09T05:37:36.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.290+0000 7f7b637fe700 1 -- 172.21.15.115:0/3490606388 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7b6802a070 con 0x7f7b74104380 2024-05-09T05:37:36.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.291+0000 7f7b637fe700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7b5c0b2900 0x7f7b5c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:36.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.291+0000 7f7b7259c700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7b5c0b2900 0x7f7b5c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:36.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.291+0000 7f7b637fe700 1 -- 172.21.15.115:0/3490606388 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f7b680e3b50 con 0x7f7b74104380 2024-05-09T05:37:36.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.291+0000 7f7b7259c700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7b5c0b2900 0x7f7b5c0b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b64005d90 tx=0x7f7b6400b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:36.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.294+0000 7f7b637fe700 1 -- 172.21.15.115:0/3490606388 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7b68031020 con 0x7f7b74104380 2024-05-09T05:37:36.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.542+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7b7402e5f0 con 0x7f7b74104380 2024-05-09T05:37:36.535 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:36 smithi115 ceph-mon[30349]: pgmap v408: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:36.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.542+0000 7f7b637fe700 1 -- 172.21.15.115:0/3490606388 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3106 (secure 0 0 0) 0x7f7b68022e20 con 0x7f7b74104380 2024-05-09T05:37:36.536 INFO:teuthology.orchestra.run.smithi115.stdout:e29 2024-05-09T05:37:36.536 INFO:teuthology.orchestra.run.smithi115.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T05:37:36.536 INFO:teuthology.orchestra.run.smithi115.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T05:37:36.536 INFO:teuthology.orchestra.run.smithi115.stdout:legacy client fscid: 1 2024-05-09T05:37:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:37:36.536 INFO:teuthology.orchestra.run.smithi115.stdout:Filesystem 'cephfs' (1) 2024-05-09T05:37:36.536 INFO:teuthology.orchestra.run.smithi115.stdout:fs_name cephfs 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 29 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-05-09T05:33:54.732251+0000 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-05-09T05:35:05.215676+0000 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:tableserver 0 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:root 0 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:session_timeout 300 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:session_autoclose 300 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:max_file_size 1099511627776 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:max_xattr_size 65536 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:required_client_features {} 2024-05-09T05:37:36.537 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure 0 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:last_failure_osd_epoch 0 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:max_mds 5 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:in 0,1,2,3,4 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:up {0=24469,1=24457,2=24451,3=24440,4=24425} 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:failed 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:damaged 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:stopped 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:data_pools [3,4] 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:metadata_pool 2 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:inline_data disabled 2024-05-09T05:37:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:balancer 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:bal_rank_mask -1 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:standby_count_wanted 0 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:qdb_cluster leader: 24469 members: 24425,24440,24451,24457,24469 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.i{0:24469} state up:active seq 8 addr [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.l{0:24412} state up:standby-replay seq 1 addr [v2:172.21.15.195:6832/2784864553,v1:172.21.15.195:6833/2784864553] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.c{1:24457} state up:active seq 25 addr [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.e{1:24404} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/1779202002,v1:172.21.15.161:6835/1779202002] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.f{2:24451} state up:active seq 26 addr [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.k{2:24386} state up:standby-replay seq 1 addr [v2:172.21.15.161:6832/634985009,v1:172.21.15.161:6833/634985009] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.b{3:24440} state up:active seq 22 addr [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.a{3:14625} state up:standby-replay seq 1 addr [v2:172.21.15.115:6840/1081505697,v1:172.21.15.115:6841/1081505697] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.539 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.h{4:24425} state up:active seq 25 addr [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.540 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.115:6838/3801478957,v1:172.21.15.115:6839/3801478957] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:37:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:37:36.540 INFO:teuthology.orchestra.run.smithi115.stdout:Standby daemons: 2024-05-09T05:37:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:37:36.540 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.115:6834/1419658869,v1:172.21.15.115:6835/1419658869] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.540 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.115:6836/2746290424,v1:172.21.15.115:6837/2746290424] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T05:37:36.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.544+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7b5c0b2900 msgr2=0x7f7b5c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:36.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.544+0000 7f7b78e39700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7b5c0b2900 0x7f7b5c0b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b64005d90 tx=0x7f7b6400b480 comp rx=0 tx=0).stop 2024-05-09T05:37:36.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.544+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b74104380 msgr2=0x7f7b74199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:36.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.544+0000 7f7b78e39700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b74104380 0x7f7b74199c70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7b68006010 tx=0x7f7b680047b0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.545+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 shutdown_connections 2024-05-09T05:37:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.545+0000 7f7b78e39700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f7b5c0b2900 0x7f7b5c0b4e10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.545+0000 7f7b78e39700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b74103250 0x7f7b74199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.545+0000 7f7b78e39700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b74104d20 0x7f7b741a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.545+0000 7f7b78e39700 1 --2- 172.21.15.115:0/3490606388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b74104380 0x7f7b74199c70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.545+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 >> 172.21.15.115:0/3490606388 conn(0x7f7b740fe890 msgr2=0x7f7b741075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.545+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 shutdown_connections 2024-05-09T05:37:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:36.545+0000 7f7b78e39700 1 -- 172.21.15.115:0/3490606388 wait complete. 2024-05-09T05:37:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 29 2024-05-09T05:37:36.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:36 smithi195 ceph-mon[33844]: pgmap v408: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:36.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:36 smithi161 ceph-mon[33805]: pgmap v408: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:37.178 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T05:37:37.422 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:37.544 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:37 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3490606388' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T05:37:37.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:37 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3490606388' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T05:37:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:37 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3490606388' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T05:37:38.526 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:38 smithi115 ceph-mon[30349]: pgmap v409: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:38.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:38 smithi161 ceph-mon[33805]: pgmap v409: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:38.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:38 smithi195 ceph-mon[33844]: pgmap v409: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:39.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 -- 172.21.15.115:0/2596795043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4105080 msgr2=0x7f08a410b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:39.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 --2- 172.21.15.115:0/2596795043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4105080 0x7f08a410b980 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f089c00b0d0 tx=0x7f089c00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 -- 172.21.15.115:0/2596795043 shutdown_connections 2024-05-09T05:37:39.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 --2- 172.21.15.115:0/2596795043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a4103830 0x7f08a4103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 --2- 172.21.15.115:0/2596795043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4105080 0x7f08a410b980 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 --2- 172.21.15.115:0/2596795043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08a4104730 0x7f08a4104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 -- 172.21.15.115:0/2596795043 >> 172.21.15.115:0/2596795043 conn(0x7f08a40fe020 msgr2=0x7f08a41004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 -- 172.21.15.115:0/2596795043 shutdown_connections 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 -- 172.21.15.115:0/2596795043 wait complete. 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 Processor -- start 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.133+0000 7f08aa20d700 1 -- start start 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08aa20d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4103830 0x7f08a4102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08aa20d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08a4104730 0x7f08a419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08aa20d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a4105080 0x7f08a41a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08aa20d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f08a410edb0 con 0x7f08a4103830 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08aa20d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08a410ec30 con 0x7f08a4105080 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08aa20d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f08a410ef30 con 0x7f08a4104730 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4103830 0x7f08a4102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4103830 0x7f08a4102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59388/0 (socket says 172.21.15.115:59388) 2024-05-09T05:37:39.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a37fe700 1 -- 172.21.15.115:0/1351346003 learned_addr learned my addr 172.21.15.115:0/1351346003 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a2ffd700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08a4104730 0x7f08a419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a3fff700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a4105080 0x7f08a41a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a37fe700 1 -- 172.21.15.115:0/1351346003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08a4104730 msgr2=0x7f08a419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a37fe700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08a4104730 0x7f08a419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a37fe700 1 -- 172.21.15.115:0/1351346003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a4105080 msgr2=0x7f08a41a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a37fe700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a4105080 0x7f08a41a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a37fe700 1 -- 172.21.15.115:0/1351346003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a41a2520 con 0x7f08a4103830 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a3fff700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a4105080 0x7f08a41a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.134+0000 7f08a37fe700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4103830 0x7f08a4102ef0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f089400d950 tx=0x7f089400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.135+0000 7f08a0ff9700 1 -- 172.21.15.115:0/1351346003 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894016070 con 0x7f08a4103830 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.135+0000 7f08a0ff9700 1 -- 172.21.15.115:0/1351346003 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0894004830 con 0x7f08a4103830 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.135+0000 7f08a0ff9700 1 -- 172.21.15.115:0/1351346003 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08940053e0 con 0x7f08a4103830 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.135+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a41a2810 con 0x7f08a4103830 2024-05-09T05:37:39.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.135+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a41a2d00 con 0x7f08a4103830 2024-05-09T05:37:39.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.136+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a40512c0 con 0x7f08a4103830 2024-05-09T05:37:39.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.138+0000 7f08a0ff9700 1 -- 172.21.15.115:0/1351346003 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0894005580 con 0x7f08a4103830 2024-05-09T05:37:39.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.138+0000 7f08a0ff9700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f088c0b2880 0x7f088c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:39.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.138+0000 7f08a2ffd700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f088c0b2880 0x7f088c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:39.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.138+0000 7f08a0ff9700 1 -- 172.21.15.115:0/1351346003 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f08940e2880 con 0x7f08a4103830 2024-05-09T05:37:39.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.139+0000 7f08a2ffd700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f088c0b2880 0x7f088c0b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0898005950 tx=0x7f089800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.142+0000 7f08a0ff9700 1 -- 172.21.15.115:0/1351346003 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f089401f020 con 0x7f08a4103830 2024-05-09T05:37:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.346+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f08a402e8b0 con 0x7f08a4103830 2024-05-09T05:37:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.347+0000 7f08a0ff9700 1 -- 172.21.15.115:0/1351346003 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5631 (secure 0 0 0) 0x7f08940aa240 con 0x7f08a4103830 2024-05-09T05:37:39.340 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 87 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-05-09T05:25:33.775456+0000 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-05-09T05:34:33.901006+0000 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:crush_version 27 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:full_ratio 0.95 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:backfillfull_ratio 0.9 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:nearfull_ratio 0.85 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:require_min_compat_client luminous 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:min_compat_client luminous 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:require_osd_release squid 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:stretch_mode_enabled false 2024-05-09T05:37:39.341 INFO:teuthology.orchestra.run.smithi115.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:max_osd 12 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6802/2561526848,v1:172.21.15.115:6803/2561526848] [v2:172.21.15.115:6804/2561526848,v1:172.21.15.115:6805/2561526848] exists,up 72834922-0c68-46a5-a24b-7bad43c395c4 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/3600915673,v1:172.21.15.161:6801/3600915673] [v2:172.21.15.161:6802/3600915673,v1:172.21.15.161:6803/3600915673] exists,up b6b5e817-6c25-4b93-8eed-13aff270aac2 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6800/960731357,v1:172.21.15.195:6801/960731357] [v2:172.21.15.195:6802/960731357,v1:172.21.15.195:6803/960731357] exists,up 77c0b77f-2058-4131-a82c-b8f42780f350 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6810/1326312717,v1:172.21.15.115:6811/1326312717] [v2:172.21.15.115:6812/1326312717,v1:172.21.15.115:6813/1326312717] exists,up 588aa4a2-6263-494a-93e3-a9ad906119eb 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 up in weight 1 up_from 31 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/3719283900,v1:172.21.15.161:6809/3719283900] [v2:172.21.15.161:6810/3719283900,v1:172.21.15.161:6811/3719283900] exists,up 70e4d345-0e86-4383-8051-be261d261fd0 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 up in weight 1 up_from 36 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6808/3142086553,v1:172.21.15.195:6809/3142086553] [v2:172.21.15.195:6810/3142086553,v1:172.21.15.195:6811/3142086553] exists,up 8883101f-256e-4fa9-aa1f-0b5167462282 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6 up in weight 1 up_from 42 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6818/3657552371,v1:172.21.15.115:6819/3657552371] [v2:172.21.15.115:6820/3657552371,v1:172.21.15.115:6821/3657552371] exists,up 7c3cc250-18ea-4a70-972e-3c58917ccced 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7 up in weight 1 up_from 47 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/2577292327,v1:172.21.15.161:6817/2577292327] [v2:172.21.15.161:6818/2577292327,v1:172.21.15.161:6819/2577292327] exists,up 6c95dabd-b86f-4ca6-be1c-a783b8716848 2024-05-09T05:37:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.8 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6816/818250056,v1:172.21.15.195:6817/818250056] [v2:172.21.15.195:6818/818250056,v1:172.21.15.195:6819/818250056] exists,up 92757b9e-da23-40c2-920c-2ed2a247b772 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:osd.9 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6826/4143175430,v1:172.21.15.115:6827/4143175430] [v2:172.21.15.115:6828/4143175430,v1:172.21.15.115:6829/4143175430] exists,up 7d893599-0268-4ed6-9ea7-b74b6a623b3d 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:osd.10 up in weight 1 up_from 64 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/1493802614,v1:172.21.15.161:6825/1493802614] [v2:172.21.15.161:6826/1493802614,v1:172.21.15.161:6827/1493802614] exists,up 777e8971-300a-4291-b81c-13794e9d8f02 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:osd.11 up in weight 1 up_from 70 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6824/2489992228,v1:172.21.15.195:6825/2489992228] [v2:172.21.15.195:6826/2489992228,v1:172.21.15.195:6827/2489992228] exists,up debd622f-268c-4946-af2a-e687f95c4d89 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 2.4 [2,11] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 2.5 [8,11] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 3.3 [4,9] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 3.11 [5,9] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 3.1d [4,8] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 3.36 [4,9] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 4.2 [9,1] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 4.8 [5,10] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 4.17 [9,8] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:pg_upmap_items 4.1f [9,8] 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:6800/792641138 expires 2024-05-10T05:26:32.548125+0000 2024-05-09T05:37:39.343 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:0/1072371912 expires 2024-05-10T05:26:32.548125+0000 2024-05-09T05:37:39.344 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:0/3822912073 expires 2024-05-10T05:26:32.548125+0000 2024-05-09T05:37:39.344 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:0/1329653367 expires 2024-05-10T05:26:06.415053+0000 2024-05-09T05:37:39.344 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:6800/1212404088 expires 2024-05-10T05:26:06.415053+0000 2024-05-09T05:37:39.344 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:6801/1212404088 expires 2024-05-10T05:26:06.415053+0000 2024-05-09T05:37:39.344 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:6801/792641138 expires 2024-05-10T05:26:32.548125+0000 2024-05-09T05:37:39.344 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:0/3376666830 expires 2024-05-10T05:26:06.415053+0000 2024-05-09T05:37:39.344 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:0/1620364593 expires 2024-05-10T05:26:06.415053+0000 2024-05-09T05:37:39.344 INFO:teuthology.orchestra.run.smithi115.stdout:blocklist 172.21.15.115:0/2102093122 expires 2024-05-10T05:26:32.548125+0000 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.349+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f088c0b2880 msgr2=0x7f088c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.349+0000 7f08aa20d700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f088c0b2880 0x7f088c0b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0898005950 tx=0x7f089800a300 comp rx=0 tx=0).stop 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.349+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4103830 msgr2=0x7f08a4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.349+0000 7f08aa20d700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4103830 0x7f08a4102ef0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f089400d950 tx=0x7f089400dc60 comp rx=0 tx=0).stop 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.350+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 shutdown_connections 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.350+0000 7f08aa20d700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f088c0b2880 0x7f088c0b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.350+0000 7f08aa20d700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a4105080 0x7f08a41a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.350+0000 7f08aa20d700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08a4103830 0x7f08a4102ef0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.350+0000 7f08aa20d700 1 --2- 172.21.15.115:0/1351346003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08a4104730 0x7f08a419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.350+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 >> 172.21.15.115:0/1351346003 conn(0x7f08a40fe020 msgr2=0x7f08a4107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.350+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 shutdown_connections 2024-05-09T05:37:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:39.350+0000 7f08aa20d700 1 -- 172.21.15.115:0/1351346003 wait complete. 2024-05-09T05:37:39.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:39 smithi115 ceph-mon[30349]: pgmap v410: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:39.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:39 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1351346003' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T05:37:39.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:39 smithi161 ceph-mon[33805]: pgmap v410: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:39.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:39 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1351346003' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T05:37:40.070 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T05:37:40.082 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:37:40.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:39 smithi195 ceph-mon[33844]: pgmap v410: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:40.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:39 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1351346003' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T05:37:40.324 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:40.757 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:40 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:37:40.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:40 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:37:41.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:40 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:37:42.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.090+0000 7fe11105b700 1 -- 172.21.15.115:0/1671725671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c1034c0 msgr2=0x7fe10c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:42.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.090+0000 7fe11105b700 1 --2- 172.21.15.115:0/1671725671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c1034c0 0x7fe10c1038d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0099c0 tx=0x7fe0fc009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.090+0000 7fe11105b700 1 -- 172.21.15.115:0/1671725671 shutdown_connections 2024-05-09T05:37:42.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.090+0000 7fe11105b700 1 --2- 172.21.15.115:0/1671725671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe10c108740 0x7fe10c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.090+0000 7fe11105b700 1 --2- 172.21.15.115:0/1671725671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c1034c0 0x7fe10c1038d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.090+0000 7fe11105b700 1 --2- 172.21.15.115:0/1671725671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe10c107df0 0x7fe10c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.090+0000 7fe11105b700 1 -- 172.21.15.115:0/1671725671 >> 172.21.15.115:0/1671725671 conn(0x7fe10c0fedc0 msgr2=0x7fe10c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:42.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.090+0000 7fe11105b700 1 -- 172.21.15.115:0/1671725671 shutdown_connections 2024-05-09T05:37:42.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.090+0000 7fe11105b700 1 -- 172.21.15.115:0/1671725671 wait complete. 2024-05-09T05:37:42.085 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:41 smithi115 ceph-mon[30349]: pgmap v411: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:42.085 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.085 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.085 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.085 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.085 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.085 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:41 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.091+0000 7fe11105b700 1 Processor -- start 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.091+0000 7fe11105b700 1 -- start start 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe11105b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe10c1034c0 0x7fe10c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe11105b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe10c107df0 0x7fe10c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe11105b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c108740 0x7fe10c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe11105b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c113220 con 0x7fe10c108740 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe11105b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c1130a0 con 0x7fe10c107df0 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe11105b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c1133a0 con 0x7fe10c1034c0 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10b59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c108740 0x7fe10c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:42.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10b59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c108740 0x7fe10c199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59398/0 (socket says 172.21.15.115:59398) 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10b59e700 1 -- 172.21.15.115:0/824935270 learned_addr learned my addr 172.21.15.115:0/824935270 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10a59c700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe10c107df0 0x7fe10c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10ad9d700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe10c1034c0 0x7fe10c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10b59e700 1 -- 172.21.15.115:0/824935270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe10c1034c0 msgr2=0x7fe10c19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10b59e700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe10c1034c0 0x7fe10c19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10b59e700 1 -- 172.21.15.115:0/824935270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe10c107df0 msgr2=0x7fe10c19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10b59e700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe10c107df0 0x7fe10c19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10b59e700 1 -- 172.21.15.115:0/824935270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c19a150 con 0x7fe10c108740 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.092+0000 7fe10b59e700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c108740 0x7fe10c199aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe104007ee0 tx=0x7fe10400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.093+0000 7fe0f3fff700 1 -- 172.21.15.115:0/824935270 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104004bb0 con 0x7fe10c108740 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.093+0000 7fe0f3fff700 1 -- 172.21.15.115:0/824935270 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe104004d50 con 0x7fe10c108740 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.093+0000 7fe0f3fff700 1 -- 172.21.15.115:0/824935270 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040058b0 con 0x7fe10c108740 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.093+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c19a440 con 0x7fe10c108740 2024-05-09T05:37:42.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.093+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c1a66c0 con 0x7fe10c108740 2024-05-09T05:37:42.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.094+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c0512c0 con 0x7fe10c108740 2024-05-09T05:37:42.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.096+0000 7fe0f3fff700 1 -- 172.21.15.115:0/824935270 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe10401a020 con 0x7fe10c108740 2024-05-09T05:37:42.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.097+0000 7fe0f3fff700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fe0f40b2900 0x7fe0f40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:42.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.098+0000 7fe10ad9d700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fe0f40b2900 0x7fe0f40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:42.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.098+0000 7fe0f3fff700 1 -- 172.21.15.115:0/824935270 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fe1040e21e0 con 0x7fe10c108740 2024-05-09T05:37:42.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.098+0000 7fe10ad9d700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fe0f40b2900 0x7fe0f40b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009990 tx=0x7fe0fc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:42.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.101+0000 7fe0f3fff700 1 -- 172.21.15.115:0/824935270 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe104010070 con 0x7fe10c108740 2024-05-09T05:37:42.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.362+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe10c02e5f0 con 0x7fe10c108740 2024-05-09T05:37:42.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.363+0000 7fe0f3fff700 1 -- 172.21.15.115:0/824935270 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7fe1040a9ba0 con 0x7fe10c108740 2024-05-09T05:37:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:37:42.356 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:35:05.215676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:37:42.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fe0f40b2900 msgr2=0x7fe0f40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:42.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fe0f40b2900 0x7fe0f40b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009990 tx=0x7fe0fc005fb0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c108740 msgr2=0x7fe10c199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:42.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c108740 0x7fe10c199aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe104007ee0 tx=0x7fe10400d3b0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 shutdown_connections 2024-05-09T05:37:42.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fe0f40b2900 0x7fe0f40b4e10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe10c108740 0x7fe10c199aa0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe10c1034c0 0x7fe10c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 --2- 172.21.15.115:0/824935270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe10c107df0 0x7fe10c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:42.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 >> 172.21.15.115:0/824935270 conn(0x7fe10c0fedc0 msgr2=0x7fe10c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:42.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 shutdown_connections 2024-05-09T05:37:42.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:42.366+0000 7fe11105b700 1 -- 172.21.15.115:0/824935270 wait complete. 2024-05-09T05:37:42.360 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 29 2024-05-09T05:37:42.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:41 smithi195 ceph-mon[33844]: pgmap v411: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:42.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:41 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:41 smithi161 ceph-mon[33805]: pgmap v411: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:42.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:41 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:43.046 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:37:43.046 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:37:43.046 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:42 smithi115 ceph-mon[30349]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:43.046 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:42 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/824935270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:37:43.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:37:43.316 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:43.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:37:43.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:37:43.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:42 smithi195 ceph-mon[33844]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:43.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:42 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/824935270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:37:43.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:37:43.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:37:43.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:42 smithi161 ceph-mon[33805]: from='mgr.14150 172.21.15.115:0/3501107835' entity='mgr.x' 2024-05-09T05:37:43.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:42 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/824935270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:37:44.428 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:44 smithi115 ceph-mon[30349]: pgmap v412: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:44.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:44 smithi195 ceph-mon[33844]: pgmap v412: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:44.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:44 smithi161 ceph-mon[33805]: pgmap v412: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:44.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.992+0000 7f61e6106700 1 -- 172.21.15.115:0/3796435029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0107e40 msgr2=0x7f61e0108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:44.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.992+0000 7f61e6106700 1 --2- 172.21.15.115:0/3796435029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0107e40 0x7f61e0108250 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f61d40099c0 tx=0x7f61d4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:44.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.992+0000 7f61e6106700 1 -- 172.21.15.115:0/3796435029 shutdown_connections 2024-05-09T05:37:44.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.992+0000 7f61e6106700 1 --2- 172.21.15.115:0/3796435029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e0103c40 0x7f61e0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:44.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.992+0000 7f61e6106700 1 --2- 172.21.15.115:0/3796435029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0108790 0x7f61e01100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:44.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.992+0000 7f61e6106700 1 --2- 172.21.15.115:0/3796435029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0107e40 0x7f61e0108250 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:44.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.992+0000 7f61e6106700 1 -- 172.21.15.115:0/3796435029 >> 172.21.15.115:0/3796435029 conn(0x7f61e00fe6f0 msgr2=0x7f61e0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:44.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.992+0000 7f61e6106700 1 -- 172.21.15.115:0/3796435029 shutdown_connections 2024-05-09T05:37:44.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.992+0000 7f61e6106700 1 -- 172.21.15.115:0/3796435029 wait complete. 2024-05-09T05:37:44.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.993+0000 7f61e6106700 1 Processor -- start 2024-05-09T05:37:44.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.993+0000 7f61e6106700 1 -- start start 2024-05-09T05:37:44.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61e6106700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0103c40 0x7f61e019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:44.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61e6106700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e0107e40 0x7f61e019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61e6106700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0108790 0x7f61e0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61e6106700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0113240 con 0x7f61e0103c40 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61e6106700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01130c0 con 0x7f61e0107e40 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61e6106700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01133c0 con 0x7f61e0108790 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61df7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0103c40 0x7f61e019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61df7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0103c40 0x7f61e019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35390/0 (socket says 172.21.15.115:35390) 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61df7fe700 1 -- 172.21.15.115:0/4275249236 learned_addr learned my addr 172.21.15.115:0/4275249236 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61deffd700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e0107e40 0x7f61e019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61dffff700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0108790 0x7f61e0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61df7fe700 1 -- 172.21.15.115:0/4275249236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0108790 msgr2=0x7f61e0199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61df7fe700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0108790 0x7f61e0199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:44.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61df7fe700 1 -- 172.21.15.115:0/4275249236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e0107e40 msgr2=0x7f61e019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61df7fe700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e0107e40 0x7f61e019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61df7fe700 1 -- 172.21.15.115:0/4275249236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e019a1c0 con 0x7f61e0103c40 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61dffff700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0108790 0x7f61e0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61df7fe700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0103c40 0x7f61e019f8a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f61d000d980 tx=0x7f61d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61dcff9700 1 -- 172.21.15.115:0/4275249236 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0017030 con 0x7f61e0103c40 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61dcff9700 1 -- 172.21.15.115:0/4275249236 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61d00044b0 con 0x7f61e0103c40 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e019a4b0 con 0x7f61e0103c40 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.994+0000 7f61dcff9700 1 -- 172.21.15.115:0/4275249236 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00038e0 con 0x7f61e0103c40 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.995+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01a6750 con 0x7f61e0103c40 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.995+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e00512c0 con 0x7f61e0103c40 2024-05-09T05:37:44.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.997+0000 7f61dcff9700 1 -- 172.21.15.115:0/4275249236 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f61d001a070 con 0x7f61e0103c40 2024-05-09T05:37:44.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.998+0000 7f61dcff9700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f61c80b2880 0x7f61c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:44.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.998+0000 7f61deffd700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f61c80b2880 0x7f61c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:44.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.998+0000 7f61dcff9700 1 -- 172.21.15.115:0/4275249236 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f61d00e2270 con 0x7f61e0103c40 2024-05-09T05:37:44.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:44.999+0000 7f61deffd700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f61c80b2880 0x7f61c80b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009990 tx=0x7f61d4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:44.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.002+0000 7f61dcff9700 1 -- 172.21.15.115:0/4275249236 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61d00a9c30 con 0x7f61e0103c40 2024-05-09T05:37:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.246+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61e019afd0 con 0x7f61e0103c40 2024-05-09T05:37:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.247+0000 7f61dcff9700 1 -- 172.21.15.115:0/4275249236 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11953 (secure 0 0 0) 0x7f61d00a9530 con 0x7f61e0103c40 2024-05-09T05:37:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:37:45.243 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:35:05.215676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:37:45.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f61c80b2880 msgr2=0x7f61c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f61c80b2880 0x7f61c80b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009990 tx=0x7f61d4005fb0 comp rx=0 tx=0).stop 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0103c40 msgr2=0x7f61e019f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0103c40 0x7f61e019f8a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f61d000d980 tx=0x7f61d000dc90 comp rx=0 tx=0).stop 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 shutdown_connections 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f61c80b2880 0x7f61c80b4d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61e0108790 0x7f61e0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e0103c40 0x7f61e019f8a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 --2- 172.21.15.115:0/4275249236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e0107e40 0x7f61e019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.249+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 >> 172.21.15.115:0/4275249236 conn(0x7f61e00fe6f0 msgr2=0x7f61e0109710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.250+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 shutdown_connections 2024-05-09T05:37:45.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:45.250+0000 7f61e6106700 1 -- 172.21.15.115:0/4275249236 wait complete. 2024-05-09T05:37:45.245 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 29 2024-05-09T05:37:45.554 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:45 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4275249236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:37:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:45 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4275249236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:37:45.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:45 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4275249236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:37:45.900 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:37:46.142 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:46.561 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:46 smithi161 ceph-mon[33805]: pgmap v413: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:46.600 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:46 smithi115 ceph-mon[30349]: pgmap v413: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:46 smithi195 ceph-mon[33844]: pgmap v413: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:47.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.842+0000 7fac7d0ef700 1 -- 172.21.15.115:0/263472152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac781034c0 msgr2=0x7fac781038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:47.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.842+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/263472152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac781034c0 0x7fac781038d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0099c0 tx=0x7fac6c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:47.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.842+0000 7fac7d0ef700 1 -- 172.21.15.115:0/263472152 shutdown_connections 2024-05-09T05:37:47.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.842+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/263472152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac78108740 0x7fac781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:47.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.842+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/263472152 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac781034c0 0x7fac781038d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:47.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.842+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/263472152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac78107df0 0x7fac78108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:47.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.842+0000 7fac7d0ef700 1 -- 172.21.15.115:0/263472152 >> 172.21.15.115:0/263472152 conn(0x7fac780fedc0 msgr2=0x7fac781011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:47.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.842+0000 7fac7d0ef700 1 -- 172.21.15.115:0/263472152 shutdown_connections 2024-05-09T05:37:47.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.842+0000 7fac7d0ef700 1 -- 172.21.15.115:0/263472152 wait complete. 2024-05-09T05:37:47.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.843+0000 7fac7d0ef700 1 Processor -- start 2024-05-09T05:37:47.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.843+0000 7fac7d0ef700 1 -- start start 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7d0ef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac781034c0 0x7fac780ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7d0ef700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac78107df0 0x7fac780ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7d0ef700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac78108740 0x7fac780ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7d0ef700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac78113220 con 0x7fac78107df0 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7d0ef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac781130a0 con 0x7fac781034c0 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7d0ef700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fac781133a0 con 0x7fac78108740 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac78107df0 0x7fac780ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac78107df0 0x7fac780ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35402/0 (socket says 172.21.15.115:35402) 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7659c700 1 -- 172.21.15.115:0/50061922 learned_addr learned my addr 172.21.15.115:0/50061922 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7759e700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac78108740 0x7fac780ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac76d9d700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac781034c0 0x7fac780ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7659c700 1 -- 172.21.15.115:0/50061922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac78108740 msgr2=0x7fac780ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7659c700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac78108740 0x7fac780ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:47.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7659c700 1 -- 172.21.15.115:0/50061922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac781034c0 msgr2=0x7fac780ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:47.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7659c700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac781034c0 0x7fac780ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:47.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.844+0000 7fac7659c700 1 -- 172.21.15.115:0/50061922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac781a90c0 con 0x7fac78107df0 2024-05-09T05:37:47.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.845+0000 7fac7659c700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac78107df0 0x7fac780ff7e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fac6800d980 tx=0x7fac6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:47.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.845+0000 7fac5ffff700 1 -- 172.21.15.115:0/50061922 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac68017070 con 0x7fac78107df0 2024-05-09T05:37:47.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.845+0000 7fac5ffff700 1 -- 172.21.15.115:0/50061922 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac6800fd20 con 0x7fac78107df0 2024-05-09T05:37:47.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.845+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac781a93b0 con 0x7fac78107df0 2024-05-09T05:37:47.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.845+0000 7fac5ffff700 1 -- 172.21.15.115:0/50061922 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6800e3b0 con 0x7fac78107df0 2024-05-09T05:37:47.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.845+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac781a98f0 con 0x7fac78107df0 2024-05-09T05:37:47.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.846+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac780512c0 con 0x7fac78107df0 2024-05-09T05:37:47.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.847+0000 7fac5ffff700 1 -- 172.21.15.115:0/50061922 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fac6800f890 con 0x7fac78107df0 2024-05-09T05:37:47.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.848+0000 7fac5ffff700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac600b2880 0x7fac600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:47.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.848+0000 7fac76d9d700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac600b2880 0x7fac600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:47.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.848+0000 7fac5ffff700 1 -- 172.21.15.115:0/50061922 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fac680e2380 con 0x7fac78107df0 2024-05-09T05:37:47.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.849+0000 7fac76d9d700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac600b2880 0x7fac600b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:47.853+0000 7fac5ffff700 1 -- 172.21.15.115:0/50061922 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac680a9d40 con 0x7fac78107df0 2024-05-09T05:37:48.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.062+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac7802e5f0 con 0x7fac78107df0 2024-05-09T05:37:48.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.063+0000 7fac5ffff700 1 -- 172.21.15.115:0/50061922 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24724 (secure 0 0 0) 0x7fac680a9640 con 0x7fac78107df0 2024-05-09T05:37:48.058 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:37:48.058 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:33.901006+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:37:48.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac600b2880 msgr2=0x7fac600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:48.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac600b2880 0x7fac600b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c005fb0 comp rx=0 tx=0).stop 2024-05-09T05:37:48.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac78107df0 msgr2=0x7fac780ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:48.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac78107df0 0x7fac780ff7e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fac6800d980 tx=0x7fac6800dc90 comp rx=0 tx=0).stop 2024-05-09T05:37:48.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 shutdown_connections 2024-05-09T05:37:48.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac600b2880 0x7fac600b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:48.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac781034c0 0x7fac780ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:48.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac78108740 0x7fac780ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:48.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 --2- 172.21.15.115:0/50061922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac78107df0 0x7fac780ff7e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:48.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 >> 172.21.15.115:0/50061922 conn(0x7fac780fedc0 msgr2=0x7fac781063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:48.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 shutdown_connections 2024-05-09T05:37:48.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:48.066+0000 7fac7d0ef700 1 -- 172.21.15.115:0/50061922 wait complete. 2024-05-09T05:37:48.429 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:48 smithi115 ceph-mon[30349]: pgmap v414: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:48.430 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:48 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/50061922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:37:48.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:48 smithi195 ceph-mon[33844]: pgmap v414: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:48.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:48 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/50061922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:37:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:48 smithi161 ceph-mon[33805]: pgmap v414: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:48.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:48 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/50061922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:37:48.762 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T05:33:54.732251+0000', 'modified': '2024-05-09T05:35:05.215676+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24469, 'mds_1': 24457, 'mds_2': 24451, 'mds_3': 24440, 'mds_4': 24425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6839/3801478957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 3801478957}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 3801478957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6841/1081505697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6840', 'nonce': 1081505697}, {'type': 'v1', 'addr': '172.21.15.115:6841', 'nonce': 1081505697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24386': {'gid': 24386, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6833/634985009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 634985009}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 634985009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/1779202002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 1779202002}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 1779202002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6833/2784864553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6832', 'nonce': 2784864553}, {'type': 'v1', 'addr': '172.21.15.195:6833', 'nonce': 2784864553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.161:6837/3591780613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3591780613}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3591780613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'b', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.161:6839/234220396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 234220396}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 234220396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.195:6835/969435997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 969435997}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 969435997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.195:6837/4026198088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 4026198088}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 4026198088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.195:6839/1691623113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 1691623113}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 1691623113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24469, 'qdb_cluster': [24425, 24440, 24451, 24457, 24469]} 2024-05-09T05:37:48.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T05:37:48.763 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T05:37:49.008 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:50.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:50 smithi195 ceph-mon[33844]: pgmap v415: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:50.618 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:50 smithi115 ceph-mon[30349]: pgmap v415: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:50.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:50 smithi161 ceph-mon[33805]: pgmap v415: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.888+0000 7fb972c6e700 1 -- 172.21.15.115:0/1012295465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c1089f0 msgr2=0x7fb96c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.888+0000 7fb972c6e700 1 --2- 172.21.15.115:0/1012295465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c1089f0 0x7fb96c108e00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb96000aaa0 tx=0x7fb96000adb0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.888+0000 7fb972c6e700 1 -- 172.21.15.115:0/1012295465 shutdown_connections 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.888+0000 7fb972c6e700 1 --2- 172.21.15.115:0/1012295465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c103310 0x7fb96c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.888+0000 7fb972c6e700 1 --2- 172.21.15.115:0/1012295465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c1089f0 0x7fb96c108e00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.888+0000 7fb972c6e700 1 --2- 172.21.15.115:0/1012295465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb96c1029c0 0x7fb96c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.888+0000 7fb972c6e700 1 -- 172.21.15.115:0/1012295465 >> 172.21.15.115:0/1012295465 conn(0x7fb96c0fe520 msgr2=0x7fb96c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.888+0000 7fb972c6e700 1 -- 172.21.15.115:0/1012295465 shutdown_connections 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.888+0000 7fb972c6e700 1 -- 172.21.15.115:0/1012295465 wait complete. 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb972c6e700 1 Processor -- start 2024-05-09T05:37:50.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb972c6e700 1 -- start start 2024-05-09T05:37:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb972c6e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c1029c0 0x7fb96c104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb972c6e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c103310 0x7fb96c104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb972c6e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb96c1089f0 0x7fb96c105010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb972c6e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c112190 con 0x7fb96c103310 2024-05-09T05:37:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb972c6e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c112010 con 0x7fb96c1029c0 2024-05-09T05:37:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb972c6e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c112310 con 0x7fb96c1089f0 2024-05-09T05:37:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb96bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c103310 0x7fb96c104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb96bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c103310 0x7fb96c104ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35420/0 (socket says 172.21.15.115:35420) 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb96bfff700 1 -- 172.21.15.115:0/450691051 learned_addr learned my addr 172.21.15.115:0/450691051 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb970a0a700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c1029c0 0x7fb96c104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.889+0000 7fb97120b700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb96c1089f0 0x7fb96c105010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb96bfff700 1 -- 172.21.15.115:0/450691051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb96c1089f0 msgr2=0x7fb96c105010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb96bfff700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb96c1089f0 0x7fb96c105010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb96bfff700 1 -- 172.21.15.115:0/450691051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c1029c0 msgr2=0x7fb96c104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb96bfff700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c1029c0 0x7fb96c104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb96bfff700 1 -- 172.21.15.115:0/450691051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c1a90e0 con 0x7fb96c103310 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb96bfff700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c103310 0x7fb96c104ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb95800d980 tx=0x7fb95800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb969ffb700 1 -- 172.21.15.115:0/450691051 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958017070 con 0x7fb96c103310 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb969ffb700 1 -- 172.21.15.115:0/450691051 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb95800fd20 con 0x7fb96c103310 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb972c6e700 1 -- 172.21.15.115:0/450691051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c1a93d0 con 0x7fb96c103310 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb969ffb700 1 -- 172.21.15.115:0/450691051 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95800e3b0 con 0x7fb96c103310 2024-05-09T05:37:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.890+0000 7fb972c6e700 1 -- 172.21.15.115:0/450691051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1a9910 con 0x7fb96c103310 2024-05-09T05:37:50.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.893+0000 7fb969ffb700 1 -- 172.21.15.115:0/450691051 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb95800f890 con 0x7fb96c103310 2024-05-09T05:37:50.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.893+0000 7fb969ffb700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb95c0b2880 0x7fb95c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb970a0a700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb95c0b2880 0x7fb95c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb969ffb700 1 -- 172.21.15.115:0/450691051 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fb9580e1ff0 con 0x7fb96c103310 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb981684b80 1 Processor -- start 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb981684b80 1 -- start start 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb970a0a700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb95c0b2880 0x7fb95c0b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb960006010 tx=0x7fb96000c690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb981684b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b10ccb0 0x556f1b2151f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb981684b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b215730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb981684b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 0x556f1b20f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb981684b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556f1b132050 con 0x556f1b10ccb0 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb981684b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556f1b0d7950 con 0x556f1b092450 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb981684b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f1b12e010 con 0x556f1b20f3b0 2024-05-09T05:37:50.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb970a0a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b10ccb0 0x556f1b2151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb970a0a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b10ccb0 0x556f1b2151f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35432/0 (socket says 172.21.15.115:35432) 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.894+0000 7fb970a0a700 1 -- 172.21.15.115:0/36819355 learned_addr learned my addr 172.21.15.115:0/36819355 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb97120b700 1 --2- 172.21.15.115:0/36819355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 0x556f1b20f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb96bfff700 1 --2- 172.21.15.115:0/36819355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b215730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb970a0a700 1 -- 172.21.15.115:0/36819355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 msgr2=0x556f1b20f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb970a0a700 1 --2- 172.21.15.115:0/36819355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 0x556f1b20f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb970a0a700 1 -- 172.21.15.115:0/36819355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 msgr2=0x556f1b215730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb970a0a700 1 --2- 172.21.15.115:0/36819355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b215730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb970a0a700 1 -- 172.21.15.115:0/36819355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f1b0c84b0 con 0x556f1b10ccb0 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb970a0a700 1 --2- 172.21.15.115:0/36819355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b10ccb0 0x556f1b2151f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb960016000 tx=0x7fb96001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a503caf4a772a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb956ffd700 1 -- 172.21.15.115:0/36819355 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960028050 con 0x556f1b10ccb0 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb956ffd700 1 -- 172.21.15.115:0/36819355 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb960032070 con 0x556f1b10ccb0 2024-05-09T05:37:50.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.895+0000 7fb956ffd700 1 -- 172.21.15.115:0/36819355 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960022970 con 0x556f1b10ccb0 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.896+0000 7fb981684b80 1 -- 172.21.15.115:0/36819355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b10ccb0 msgr2=0x556f1b2151f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.896+0000 7fb981684b80 1 --2- 172.21.15.115:0/36819355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b10ccb0 0x556f1b2151f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb960016000 tx=0x7fb96001f000 comp rx=0 tx=0).stop 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.896+0000 7fb981684b80 1 -- 172.21.15.115:0/36819355 shutdown_connections 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.896+0000 7fb981684b80 1 --2- 172.21.15.115:0/36819355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 0x556f1b20f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.896+0000 7fb981684b80 1 --2- 172.21.15.115:0/36819355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b10ccb0 0x556f1b2151f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.896+0000 7fb981684b80 1 --2- 172.21.15.115:0/36819355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b215730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.896+0000 7fb981684b80 1 -- 172.21.15.115:0/36819355 >> 172.21.15.115:0/36819355 conn(0x556f1b095f20 msgr2=0x556f1b20a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.896+0000 7fb981684b80 1 -- 172.21.15.115:0/36819355 shutdown_connections 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.896+0000 7fb981684b80 1 -- 172.21.15.115:0/36819355 wait complete. 2024-05-09T05:37:50.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb981684b80 1 -- start start 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb981684b80 1 Processor -- start 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb981684b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b218f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb981684b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 0x556f1b2194b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb981684b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b2199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb981684b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556f1b2100b0 con 0x556f1b20f3b0 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb981684b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556f1b0ccad0 con 0x556f1b092450 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb981684b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f1b2251b0 con 0x556f1b10ccb0 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb981684b80 10 client.0 _ll_register_callbacks cb 0x556f1afa3670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb97120b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b2199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb97120b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b2199f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35442/0 (socket says 172.21.15.115:35442) 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb97120b700 1 -- 172.21.15.115:0/1664126147 learned_addr learned my addr 172.21.15.115:0/1664126147 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:50.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.897+0000 7fb96bfff700 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 0x556f1b2194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb970a0a700 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b218f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb97120b700 1 -- 172.21.15.115:0/1664126147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 msgr2=0x556f1b2194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb97120b700 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 0x556f1b2194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb97120b700 1 -- 172.21.15.115:0/1664126147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 msgr2=0x556f1b218f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb97120b700 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b218f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb97120b700 1 -- 172.21.15.115:0/1664126147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f1afad440 con 0x556f1b20f3b0 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb970a0a700 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b218f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb97120b700 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b2199f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb9640042a0 tx=0x7fb9640042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52f34bc07a53ffa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb94bfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:50.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb94bfff700 1 -- 172.21.15.115:0/1664126147 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96401a070 con 0x556f1b20f3b0 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb94bfff700 1 -- 172.21.15.115:0/1664126147 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fb964004bb0 con 0x556f1b20f3b0 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb981684b80 1 -- 172.21.15.115:0/1664126147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb960032070 con 0x556f1b20f3b0 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb94bfff700 1 -- 172.21.15.115:0/1664126147 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb964025050 con 0x556f1b20f3b0 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb981684b80 10 client.15225 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb981684b80 1 -- 172.21.15.115:0/1664126147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f1b226ae0 con 0x556f1b20f3b0 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.898+0000 7fb94bfff700 1 -- 172.21.15.115:0/1664126147 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb960032070 con 0x556f1b20f3b0 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb94bfff700 1 -- 172.21.15.115:0/1664126147 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb964016860 con 0x556f1b20f3b0 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb981684b80 10 client.15225 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb981684b80 10 client.15225 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb981684b80 20 client.15225 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:50.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb981684b80 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556f1b2290f0 0x556f1b22a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb981684b80 4 client.15225 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb981684b80 1 -- 172.21.15.115:0/1664126147 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f1b13fa30 con 0x556f1b2290f0 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb970a0a700 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556f1b2290f0 0x556f1b22a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb94bfff700 10 client.15225 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.899+0000 7fb970a0a700 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556f1b2290f0 0x556f1b22a560 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6160f823c50390e4 server_cookie=9c00c392fc4f2d0e in_seq=0 out_seq=0 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.900+0000 7fb94bfff700 1 -- 172.21.15.115:0/1664126147 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f1b13fa30 con 0x556f1b2290f0 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.900+0000 7fb94bfff700 10 client.15225 handle_command_reply: tid=0 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.900+0000 7fb981684b80 1 client.15225 shutdown 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.900+0000 7fb981684b80 1 -- 172.21.15.115:0/1664126147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 msgr2=0x556f1b2199f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.900+0000 7fb981684b80 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b2199f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb9640042a0 tx=0x7fb9640042d0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.900+0000 7fb981684b80 1 -- 172.21.15.115:0/1664126147 shutdown_connections 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.901+0000 7fb981684b80 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b218f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.901+0000 7fb981684b80 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556f1b2290f0 0x556f1b22a560 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.901+0000 7fb94bfff700 0 client.15225 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:37:50.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.901+0000 7fb981684b80 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b2199f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.901+0000 7fb981684b80 1 --2- 172.21.15.115:0/1664126147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 0x556f1b2194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.901+0000 7fb981684b80 1 -- 172.21.15.115:0/1664126147 >> 172.21.15.115:0/1664126147 conn(0x556f1b095f20 msgr2=0x556f1b203fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:50.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.901+0000 7fb981684b80 1 -- 172.21.15.115:0/1664126147 shutdown_connections 2024-05-09T05:37:50.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.901+0000 7fb981684b80 1 -- 172.21.15.115:0/1664126147 wait complete. 2024-05-09T05:37:50.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.901+0000 7fb981684b80 20 client.15225 trim_cache size 0 max 16384 2024-05-09T05:37:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.921+0000 7fb981684b80 1 Processor -- start 2024-05-09T05:37:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.921+0000 7fb981684b80 1 -- start start 2024-05-09T05:37:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.921+0000 7fb981684b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b20a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.921+0000 7fb981684b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 0x556f1b214770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.921+0000 7fb981684b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b225120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.921+0000 7fb981684b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556f1b2288a0 con 0x556f1b20f3b0 2024-05-09T05:37:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.921+0000 7fb981684b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556f1b228f70 con 0x556f1b092450 2024-05-09T05:37:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.921+0000 7fb981684b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f1b229640 con 0x556f1b10ccb0 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb97120b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b225120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb97120b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b225120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35444/0 (socket says 172.21.15.115:35444) 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb97120b700 1 -- 172.21.15.115:0/355313387 learned_addr learned my addr 172.21.15.115:0/355313387 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb96bfff700 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 0x556f1b214770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb97120b700 1 -- 172.21.15.115:0/355313387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 msgr2=0x556f1b214770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb97120b700 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 0x556f1b214770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb97120b700 1 -- 172.21.15.115:0/355313387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 msgr2=0x556f1b20a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb970a0a700 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b20a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb97120b700 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b20a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb97120b700 1 -- 172.21.15.115:0/355313387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f1b206980 con 0x556f1b20f3b0 2024-05-09T05:37:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb96bfff700 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 0x556f1b214770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb97120b700 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b225120 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb964004270 tx=0x7fb964017f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0672ffe6b69c66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb9567fc700 1 -- 172.21.15.115:0/355313387 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96401a050 con 0x556f1b20f3b0 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb9567fc700 1 -- 172.21.15.115:0/355313387 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb964003db0 con 0x556f1b20f3b0 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.922+0000 7fb9567fc700 1 -- 172.21.15.115:0/355313387 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964016930 con 0x556f1b20f3b0 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 -- 172.21.15.115:0/355313387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 msgr2=0x556f1b225120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b225120 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb964004270 tx=0x7fb964017f90 comp rx=0 tx=0).stop 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 -- 172.21.15.115:0/355313387 shutdown_connections 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b092450 0x556f1b20a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b20f3b0 0x556f1b225120 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 --2- 172.21.15.115:0/355313387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b10ccb0 0x556f1b214770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 -- 172.21.15.115:0/355313387 >> 172.21.15.115:0/355313387 conn(0x556f1b095f20 msgr2=0x556f1b214060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 -- 172.21.15.115:0/355313387 shutdown_connections 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 -- 172.21.15.115:0/355313387 wait complete. 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.923+0000 7fb981684b80 1 -- start start 2024-05-09T05:37:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb981684b80 1 Processor -- start 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb981684b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b092450 0x556f1b224920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb981684b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b10ccb0 0x556f1b224e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb981684b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 0x556f1b232060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb981684b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556f1b22a3e0 con 0x556f1b092450 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb981684b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556f1b229d10 con 0x556f1b10ccb0 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb981684b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f1b22aab0 con 0x556f1b20f3b0 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb981684b80 10 client.0 _ll_register_callbacks cb 0x556f1b218e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb970a0a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b092450 0x556f1b224920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb970a0a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b092450 0x556f1b224920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35456/0 (socket says 172.21.15.115:35456) 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb970a0a700 1 -- 172.21.15.115:0/3620376159 learned_addr learned my addr 172.21.15.115:0/3620376159 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb96bfff700 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b10ccb0 0x556f1b224e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb970a0a700 1 -- 172.21.15.115:0/3620376159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 msgr2=0x556f1b232060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb97120b700 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 0x556f1b232060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb970a0a700 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 0x556f1b232060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb970a0a700 1 -- 172.21.15.115:0/3620376159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b10ccb0 msgr2=0x556f1b224e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb970a0a700 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b10ccb0 0x556f1b224e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb970a0a700 1 -- 172.21.15.115:0/3620376159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f1b225680 con 0x556f1b092450 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb97120b700 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 0x556f1b232060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.924+0000 7fb96bfff700 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b10ccb0 0x556f1b224e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb970a0a700 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b092450 0x556f1b224920 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb96000a830 tx=0x7fb960003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7548a6f2146fd880 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb94bfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb94bfff700 1 -- 172.21.15.115:0/3620376159 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960028030 con 0x556f1b092450 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb981684b80 1 -- 172.21.15.115:0/3620376159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb964003db0 con 0x556f1b092450 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb94bfff700 1 -- 172.21.15.115:0/3620376159 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fb960005490 con 0x556f1b092450 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb94bfff700 1 -- 172.21.15.115:0/3620376159 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb960022480 con 0x556f1b092450 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb981684b80 10 client.15231 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb981684b80 1 -- 172.21.15.115:0/3620376159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f1b2949d0 con 0x556f1b092450 2024-05-09T05:37:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb94bfff700 1 -- 172.21.15.115:0/3620376159 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb964003db0 con 0x556f1b092450 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb94bfff700 1 -- 172.21.15.115:0/3620376159 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb96000b040 con 0x556f1b092450 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb981684b80 10 client.15231 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb981684b80 10 client.15231 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.925+0000 7fb981684b80 20 client.15231 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.926+0000 7fb981684b80 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556f1b2290f0 0x556f1b297df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.926+0000 7fb981684b80 4 client.15231 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.926+0000 7fb981684b80 1 -- 172.21.15.115:0/3620376159 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "status"}) -- 0x556f1b139ad0 con 0x556f1b2290f0 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.926+0000 7fb96bfff700 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556f1b2290f0 0x556f1b297df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.926+0000 7fb96bfff700 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556f1b2290f0 0x556f1b297df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77cec4f3708f4c06 server_cookie=3684d703bc6dee61 in_seq=0 out_seq=0 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.926+0000 7fb94bfff700 10 client.15231 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb94bfff700 1 -- 172.21.15.115:0/3620376159 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556f1b139ad0 con 0x556f1b2290f0 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb94bfff700 10 client.15231 handle_command_reply: tid=0 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 client.15231 shutdown 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 -- 172.21.15.115:0/3620376159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b092450 msgr2=0x556f1b224920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:50.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b092450 0x556f1b224920 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb96000a830 tx=0x7fb960003eb0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 -- 172.21.15.115:0/3620376159 shutdown_connections 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556f1b2290f0 0x556f1b297df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f1b20f3b0 0x556f1b232060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f1b092450 0x556f1b224920 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 --2- 172.21.15.115:0/3620376159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f1b10ccb0 0x556f1b224e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 -- 172.21.15.115:0/3620376159 >> 172.21.15.115:0/3620376159 conn(0x556f1b095f20 msgr2=0x556f1b203fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb94bfff700 0 client.15231 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 -- 172.21.15.115:0/3620376159 shutdown_connections 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 1 -- 172.21.15.115:0/3620376159 wait complete. 2024-05-09T05:37:50.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.927+0000 7fb981684b80 20 client.15231 trim_cache size 0 max 16384 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 4, 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24425, 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 168.640860489, 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 264.61157841300002 2024-05-09T05:37:50.933 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:37:50.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 -- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb95c0b2880 msgr2=0x7fb95c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:50.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb95c0b2880 0x7fb95c0b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb960006010 tx=0x7fb96000c690 comp rx=0 tx=0).stop 2024-05-09T05:37:50.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 -- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c103310 msgr2=0x7fb96c104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:50.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c103310 0x7fb96c104ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb95800d980 tx=0x7fb95800dc90 comp rx=0 tx=0).stop 2024-05-09T05:37:50.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 -- 172.21.15.115:0/450691051 shutdown_connections 2024-05-09T05:37:50.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fb95c0b2880 0x7fb95c0b4d90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c1029c0 0x7fb96c104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb96c1089f0 0x7fb96c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 --2- 172.21.15.115:0/450691051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb96c103310 0x7fb96c104ad0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:50.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 -- 172.21.15.115:0/450691051 >> 172.21.15.115:0/450691051 conn(0x7fb96c0fe520 msgr2=0x7fb96c100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:50.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 -- 172.21.15.115:0/450691051 shutdown_connections 2024-05-09T05:37:50.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:50.928+0000 7fb972c6e700 1 -- 172.21.15.115:0/450691051 wait complete. 2024-05-09T05:37:51.736 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T05:37:51.980 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:52.383 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:52 smithi115 ceph-mon[30349]: pgmap v416: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:52.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:52 smithi195 ceph-mon[33844]: pgmap v416: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:52.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:52 smithi161 ceph-mon[33805]: pgmap v416: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-09T05:37:53.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.815+0000 7f5942352700 1 -- 172.21.15.115:0/4275950718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 msgr2=0x7f593c110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:53.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 --2- 172.21.15.115:0/4275950718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 0x7f593c110e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f593400b0a0 tx=0x7f593400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 -- 172.21.15.115:0/4275950718 shutdown_connections 2024-05-09T05:37:53.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 --2- 172.21.15.115:0/4275950718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c1054f0 0x7f593c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 --2- 172.21.15.115:0/4275950718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 0x7f593c110e00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 --2- 172.21.15.115:0/4275950718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f593c105e40 0x7f593c10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 -- 172.21.15.115:0/4275950718 >> 172.21.15.115:0/4275950718 conn(0x7f593c078e00 msgr2=0x7f593c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:53.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 -- 172.21.15.115:0/4275950718 shutdown_connections 2024-05-09T05:37:53.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 -- 172.21.15.115:0/4275950718 wait complete. 2024-05-09T05:37:53.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 Processor -- start 2024-05-09T05:37:53.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.816+0000 7f5942352700 1 -- start start 2024-05-09T05:37:53.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f5942352700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f593c1054f0 0x7f593c1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f5942352700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c105e40 0x7f593c103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f5942352700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 0x7f593c103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f5942352700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1143b0 con 0x7f593c10e920 2024-05-09T05:37:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f5942352700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f593c114230 con 0x7f593c105e40 2024-05-09T05:37:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f5942352700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f593c114530 con 0x7f593c1054f0 2024-05-09T05:37:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f59408ef700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 0x7f593c103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f59408ef700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 0x7f593c103c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36612/0 (socket says 172.21.15.115:36612) 2024-05-09T05:37:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f59408ef700 1 -- 172.21.15.115:0/1926743370 learned_addr learned my addr 172.21.15.115:0/1926743370 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:53.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f593bfff700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f593c1054f0 0x7f593c1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f5933fff700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c105e40 0x7f593c103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f59408ef700 1 -- 172.21.15.115:0/1926743370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f593c1054f0 msgr2=0x7f593c1031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:53.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f59408ef700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f593c1054f0 0x7f593c1031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f59408ef700 1 -- 172.21.15.115:0/1926743370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c105e40 msgr2=0x7f593c103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:53.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f59408ef700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c105e40 0x7f593c103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f59408ef700 1 -- 172.21.15.115:0/1926743370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c1a9140 con 0x7f593c10e920 2024-05-09T05:37:53.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f5933fff700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c105e40 0x7f593c103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:53.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f59408ef700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 0x7f593c103c60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f59340062a0 tx=0x7f5934004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:53.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.817+0000 7f5939ffb700 1 -- 172.21.15.115:0/1926743370 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934021070 con 0x7f593c10e920 2024-05-09T05:37:53.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.818+0000 7f5939ffb700 1 -- 172.21.15.115:0/1926743370 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5934007c80 con 0x7f593c10e920 2024-05-09T05:37:53.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.818+0000 7f5942352700 1 -- 172.21.15.115:0/1926743370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c1a93d0 con 0x7f593c10e920 2024-05-09T05:37:53.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.818+0000 7f5939ffb700 1 -- 172.21.15.115:0/1926743370 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59340052a0 con 0x7f593c10e920 2024-05-09T05:37:53.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.818+0000 7f5942352700 1 -- 172.21.15.115:0/1926743370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c1a98b0 con 0x7f593c10e920 2024-05-09T05:37:53.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.820+0000 7f5939ffb700 1 -- 172.21.15.115:0/1926743370 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5934024070 con 0x7f593c10e920 2024-05-09T05:37:53.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.821+0000 7f5939ffb700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59280b2880 0x7f59280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.821+0000 7f593bfff700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59280b2880 0x7f59280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.821+0000 7f5939ffb700 1 -- 172.21.15.115:0/1926743370 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f593400f020 con 0x7f593c10e920 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.821+0000 7f5950d68b80 1 Processor -- start 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.821+0000 7f5950d68b80 1 -- start start 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.821+0000 7f593bfff700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59280b2880 0x7f59280b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f592c005950 tx=0x7f592c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 0x564ffb5e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5e1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5950d68b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564ffb503fa0 con 0x564ffb464450 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5950d68b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ffb4a27a0 con 0x564ffb5e1650 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5950d68b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ffb4fff60 con 0x564ffb54e110 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5e79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36626/0 (socket says 172.21.15.115:36626) 2024-05-09T05:37:53.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5933fff700 1 -- 172.21.15.115:0/3897033606 learned_addr learned my addr 172.21.15.115:0/3897033606 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f593bfff700 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 0x564ffb5e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f59408ef700 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5e1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5933fff700 1 -- 172.21.15.115:0/3897033606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 msgr2=0x564ffb5e74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5933fff700 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 0x564ffb5e74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5933fff700 1 -- 172.21.15.115:0/3897033606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 msgr2=0x564ffb5e1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5933fff700 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5e1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5933fff700 1 -- 172.21.15.115:0/3897033606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ffb49a320 con 0x564ffb464450 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f593bfff700 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 0x564ffb5e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f5933fff700 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5e79e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f592400aa70 tx=0x7f592400ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedb99aa2b1fcdd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f59327fc700 1 -- 172.21.15.115:0/3897033606 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924015070 con 0x564ffb464450 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f59327fc700 1 -- 172.21.15.115:0/3897033606 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59240113f0 con 0x564ffb464450 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.822+0000 7f59327fc700 1 -- 172.21.15.115:0/3897033606 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59240185c0 con 0x564ffb464450 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.823+0000 7f5950d68b80 1 -- 172.21.15.115:0/3897033606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 msgr2=0x564ffb5e79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:53.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.823+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5e79e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f592400aa70 tx=0x7f592400ad80 comp rx=0 tx=0).stop 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.823+0000 7f5950d68b80 1 -- 172.21.15.115:0/3897033606 shutdown_connections 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.823+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5e1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.823+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 0x564ffb5e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.823+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3897033606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5e79e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.823+0000 7f5950d68b80 1 -- 172.21.15.115:0/3897033606 >> 172.21.15.115:0/3897033606 conn(0x564ffb4c4650 msgr2=0x564ffb5dc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.823+0000 7f5950d68b80 1 -- 172.21.15.115:0/3897033606 shutdown_connections 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.823+0000 7f5950d68b80 1 -- 172.21.15.115:0/3897033606 wait complete. 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.824+0000 7f5950d68b80 1 -- start start 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.824+0000 7f5950d68b80 1 Processor -- start 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.824+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb464450 0x564ffb5eb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.824+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.824+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.824+0000 7f5950d68b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564ffb37fb50 con 0x564ffb54e110 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.824+0000 7f5950d68b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ffb4be100 con 0x564ffb5e1650 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.824+0000 7f5950d68b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ffb5ee180 con 0x564ffb464450 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.824+0000 7f5950d68b80 10 client.0 _ll_register_callbacks cb 0x564ffb375670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5eb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36632/0 (socket says 172.21.15.115:36632) 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5933fff700 1 -- 172.21.15.115:0/953656451 learned_addr learned my addr 172.21.15.115:0/953656451 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f593bfff700 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb464450 0x564ffb5eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f59408ef700 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5933fff700 1 -- 172.21.15.115:0/953656451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb464450 msgr2=0x564ffb5eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5933fff700 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb464450 0x564ffb5eb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5933fff700 1 -- 172.21.15.115:0/953656451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 msgr2=0x564ffb5ebbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:53.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5933fff700 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5ebbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5933fff700 1 -- 172.21.15.115:0/953656451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ffb37f440 con 0x564ffb54e110 2024-05-09T05:37:53.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f59408ef700 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:37:53.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f593bfff700 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb464450 0x564ffb5eb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:53.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5933fff700 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5eb670 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f59240048a0 tx=0x7f5924011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d4342eda0d37cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:53.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f591b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:53.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f591b7fe700 1 -- 172.21.15.115:0/953656451 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924015040 con 0x564ffb54e110 2024-05-09T05:37:53.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f591b7fe700 1 -- 172.21.15.115:0/953656451 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f592400cd60 con 0x564ffb54e110 2024-05-09T05:37:53.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5950d68b80 1 -- 172.21.15.115:0/953656451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59240113f0 con 0x564ffb54e110 2024-05-09T05:37:53.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f591b7fe700 1 -- 172.21.15.115:0/953656451 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f592401a450 con 0x564ffb54e110 2024-05-09T05:37:53.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5950d68b80 10 client.15243 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:53.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.825+0000 7f5950d68b80 1 -- 172.21.15.115:0/953656451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ffb5f8ba0 con 0x564ffb54e110 2024-05-09T05:37:53.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.826+0000 7f591b7fe700 1 -- 172.21.15.115:0/953656451 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59240113f0 con 0x564ffb54e110 2024-05-09T05:37:53.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.826+0000 7f591b7fe700 1 -- 172.21.15.115:0/953656451 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f59240255f0 con 0x564ffb54e110 2024-05-09T05:37:53.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.826+0000 7f5950d68b80 10 client.15243 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:53.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.826+0000 7f5950d68b80 10 client.15243 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T05:37:53.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.826+0000 7f5950d68b80 20 client.15243 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:53.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.826+0000 7f5950d68b80 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x564ffb5fadd0 0x564ffb5fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.826+0000 7f5950d68b80 4 client.15243 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:37:53.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.826+0000 7f5950d68b80 1 -- 172.21.15.115:0/953656451 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ffb511980 con 0x564ffb5fadd0 2024-05-09T05:37:53.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.826+0000 7f593bfff700 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x564ffb5fadd0 0x564ffb5fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.827+0000 7f591b7fe700 10 client.15243 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:37:53.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.827+0000 7f593bfff700 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x564ffb5fadd0 0x564ffb5fc240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba2e09296001aea9 server_cookie=6c796a6b7c8f7d16 in_seq=0 out_seq=0 2024-05-09T05:37:53.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f591b7fe700 1 -- 172.21.15.115:0/953656451 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ffb511980 con 0x564ffb5fadd0 2024-05-09T05:37:53.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f591b7fe700 10 client.15243 handle_command_reply: tid=0 2024-05-09T05:37:53.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 client.15243 shutdown 2024-05-09T05:37:53.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 -- 172.21.15.115:0/953656451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 msgr2=0x564ffb5eb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:53.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5eb670 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f59240048a0 tx=0x7f5924011990 comp rx=0 tx=0).stop 2024-05-09T05:37:53.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 -- 172.21.15.115:0/953656451 shutdown_connections 2024-05-09T05:37:53.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x564ffb5fadd0 0x564ffb5fc240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb464450 0x564ffb5eb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 --2- 172.21.15.115:0/953656451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5eb670 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 -- 172.21.15.115:0/953656451 >> 172.21.15.115:0/953656451 conn(0x564ffb4c4650 msgr2=0x564ffb5dc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:53.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f591b7fe700 0 client.15243 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:37:53.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 -- 172.21.15.115:0/953656451 shutdown_connections 2024-05-09T05:37:53.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 1 -- 172.21.15.115:0/953656451 wait complete. 2024-05-09T05:37:53.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.828+0000 7f5950d68b80 20 client.15243 trim_cache size 0 max 16384 2024-05-09T05:37:53.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.848+0000 7f5950d68b80 1 Processor -- start 2024-05-09T05:37:53.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.848+0000 7f5950d68b80 1 -- start start 2024-05-09T05:37:53.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 0x564ffb5eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f5950d68b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564ffb5fa580 con 0x564ffb464450 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f5950d68b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ffb5fac50 con 0x564ffb5e1650 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f5950d68b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ffb5fb320 con 0x564ffb54e110 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f593bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f593bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36638/0 (socket says 172.21.15.115:36638) 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f593bfff700 1 -- 172.21.15.115:0/3968668250 learned_addr learned my addr 172.21.15.115:0/3968668250 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f593bfff700 1 -- 172.21.15.115:0/3968668250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 msgr2=0x564ffb5eb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f593bfff700 1 --2- 172.21.15.115:0/3968668250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 0x564ffb5eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f593bfff700 1 -- 172.21.15.115:0/3968668250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 msgr2=0x564ffb5e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f593bfff700 1 --2- 172.21.15.115:0/3968668250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f593bfff700 1 -- 172.21.15.115:0/3968668250 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ffb5e7000 con 0x564ffb464450 2024-05-09T05:37:53.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f593bfff700 1 --2- 172.21.15.115:0/3968668250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5f73e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x564ffb5ea9b0 tx=0x7f592c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19cf4e9c8b59f15 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:53.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f5931ffb700 1 -- 172.21.15.115:0/3968668250 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c016070 con 0x564ffb464450 2024-05-09T05:37:53.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f5931ffb700 1 -- 172.21.15.115:0/3968668250 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f592c010cc0 con 0x564ffb464450 2024-05-09T05:37:53.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.849+0000 7f5931ffb700 1 -- 172.21.15.115:0/3968668250 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c0055e0 con 0x564ffb464450 2024-05-09T05:37:53.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.850+0000 7f5950d68b80 1 -- 172.21.15.115:0/3968668250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 msgr2=0x564ffb5f73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:53.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.850+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3968668250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5f73e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x564ffb5ea9b0 tx=0x7f592c012000 comp rx=0 tx=0).stop 2024-05-09T05:37:53.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.850+0000 7f5950d68b80 1 -- 172.21.15.115:0/3968668250 shutdown_connections 2024-05-09T05:37:53.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.850+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3968668250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb5e1650 0x564ffb5e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.850+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3968668250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb464450 0x564ffb5f73e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.850+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3968668250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb54e110 0x564ffb5eb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.850+0000 7f5950d68b80 1 -- 172.21.15.115:0/3968668250 >> 172.21.15.115:0/3968668250 conn(0x564ffb4c4650 msgr2=0x564ffb5e4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.850+0000 7f5950d68b80 1 -- 172.21.15.115:0/3968668250 shutdown_connections 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.850+0000 7f5950d68b80 1 -- 172.21.15.115:0/3968668250 wait complete. 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5950d68b80 1 -- start start 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5950d68b80 1 Processor -- start 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb464450 0x564ffb5f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5950d68b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb5e1650 0x564ffb5f7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5950d68b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564ffb5fc0c0 con 0x564ffb54e110 2024-05-09T05:37:53.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5950d68b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ffb5fb9f0 con 0x564ffb464450 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5950d68b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ffb5fc790 con 0x564ffb5e1650 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5950d68b80 10 client.0 _ll_register_callbacks cb 0x564ffb375670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.851+0000 7f5933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5f6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36650/0 (socket says 172.21.15.115:36650) 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f593bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb464450 0x564ffb5f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5933fff700 1 -- 172.21.15.115:0/3038238676 learned_addr learned my addr 172.21.15.115:0/3038238676 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f59408ef700 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb5e1650 0x564ffb5f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5933fff700 1 -- 172.21.15.115:0/3038238676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb5e1650 msgr2=0x564ffb5f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5933fff700 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb5e1650 0x564ffb5f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5933fff700 1 -- 172.21.15.115:0/3038238676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb464450 msgr2=0x564ffb5f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5933fff700 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb464450 0x564ffb5f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5933fff700 1 -- 172.21.15.115:0/3038238676 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ffb603da0 con 0x564ffb54e110 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5933fff700 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5f6d10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5924004790 tx=0x7f592400c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13512aed9e6e469e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f591b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f591b7fe700 1 -- 172.21.15.115:0/3038238676 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924015070 con 0x564ffb54e110 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5950d68b80 1 -- 172.21.15.115:0/3038238676 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592c010cc0 con 0x564ffb54e110 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f591b7fe700 1 -- 172.21.15.115:0/3038238676 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f5924011ab0 con 0x564ffb54e110 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f591b7fe700 1 -- 172.21.15.115:0/3038238676 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f592403b050 con 0x564ffb54e110 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5950d68b80 10 client.15249 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.852+0000 7f5950d68b80 1 -- 172.21.15.115:0/3038238676 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ffb666630 con 0x564ffb54e110 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.853+0000 7f591b7fe700 1 -- 172.21.15.115:0/3038238676 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f592c010cc0 con 0x564ffb54e110 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.853+0000 7f591b7fe700 1 -- 172.21.15.115:0/3038238676 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f592401a960 con 0x564ffb54e110 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.853+0000 7f5950d68b80 10 client.15249 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.853+0000 7f5950d68b80 10 client.15249 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.853+0000 7f5950d68b80 20 client.15249 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.853+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x564ffb6689b0 0x564ffb669e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.853+0000 7f5950d68b80 4 client.15249 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.853+0000 7f5950d68b80 1 -- 172.21.15.115:0/3038238676 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "status"}) -- 0x564ffb50ba20 con 0x564ffb6689b0 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.853+0000 7f593bfff700 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x564ffb6689b0 0x564ffb669e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.854+0000 7f591b7fe700 10 client.15249 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.854+0000 7f593bfff700 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x564ffb6689b0 0x564ffb669e20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e92e272a8565d31b server_cookie=2d05d72100c6c405 in_seq=0 out_seq=0 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.854+0000 7f591b7fe700 1 -- 172.21.15.115:0/3038238676 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564ffb50ba20 con 0x564ffb6689b0 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.854+0000 7f591b7fe700 10 client.15249 handle_command_reply: tid=0 2024-05-09T05:37:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.854+0000 7f5950d68b80 1 client.15249 shutdown 2024-05-09T05:37:53.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 -- 172.21.15.115:0/3038238676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 msgr2=0x564ffb5f6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:53.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5f6d10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5924004790 tx=0x7f592400c430 comp rx=0 tx=0).stop 2024-05-09T05:37:53.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 -- 172.21.15.115:0/3038238676 shutdown_connections 2024-05-09T05:37:53.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ffb464450 0x564ffb5f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x564ffb6689b0 0x564ffb669e20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ffb5e1650 0x564ffb5f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 --2- 172.21.15.115:0/3038238676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564ffb54e110 0x564ffb5f6d10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 -- 172.21.15.115:0/3038238676 >> 172.21.15.115:0/3038238676 conn(0x564ffb4c4650 msgr2=0x564ffb5e68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f591b7fe700 0 client.15249 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 -- 172.21.15.115:0/3038238676 shutdown_connections 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 1 -- 172.21.15.115:0/3038238676 wait complete. 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.855+0000 7f5950d68b80 20 client.15249 trim_cache size 0 max 16384 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 3, 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24440, 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:37:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:37:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:37:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 172.57185390500001, 2024-05-09T05:37:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:37:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:37:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:37:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 253.94240648900001 2024-05-09T05:37:53.855 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:37:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 -- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59280b2880 msgr2=0x7f59280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59280b2880 0x7f59280b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f592c005950 tx=0x7f592c00a300 comp rx=0 tx=0).stop 2024-05-09T05:37:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 -- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 msgr2=0x7f593c103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 0x7f593c103c60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f59340062a0 tx=0x7f5934004e20 comp rx=0 tx=0).stop 2024-05-09T05:37:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 -- 172.21.15.115:0/1926743370 shutdown_connections 2024-05-09T05:37:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59280b2880 0x7f59280b4d90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f593c10e920 0x7f593c103c60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f593c1054f0 0x7f593c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 --2- 172.21.15.115:0/1926743370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c105e40 0x7f593c103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:53.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 -- 172.21.15.115:0/1926743370 >> 172.21.15.115:0/1926743370 conn(0x7f593c078e00 msgr2=0x7f593c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:53.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 -- 172.21.15.115:0/1926743370 shutdown_connections 2024-05-09T05:37:53.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:53.856+0000 7f5942352700 1 -- 172.21.15.115:0/1926743370 wait complete. 2024-05-09T05:37:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:54 smithi195 ceph-mon[33844]: pgmap v417: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:54.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:54 smithi115 ceph-mon[30349]: pgmap v417: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:54 smithi161 ceph-mon[33805]: pgmap v417: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:54.719 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-09T05:37:54.966 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:56.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:56 smithi115 ceph-mon[30349]: pgmap v418: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:56.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:56 smithi195 ceph-mon[33844]: pgmap v418: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:56.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:56 smithi161 ceph-mon[33805]: pgmap v418: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-09T05:37:56.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.853+0000 7fa52e711700 1 -- 172.21.15.115:0/2040194016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5281034c0 msgr2=0x7fa5281038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:56.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.853+0000 7fa52e711700 1 --2- 172.21.15.115:0/2040194016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5281034c0 0x7fa5281038d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa5180099c0 tx=0x7fa518009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.853+0000 7fa52e711700 1 -- 172.21.15.115:0/2040194016 shutdown_connections 2024-05-09T05:37:56.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.853+0000 7fa52e711700 1 --2- 172.21.15.115:0/2040194016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa528108740 0x7fa5281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.853+0000 7fa52e711700 1 --2- 172.21.15.115:0/2040194016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5281034c0 0x7fa5281038d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.853+0000 7fa52e711700 1 --2- 172.21.15.115:0/2040194016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa528107df0 0x7fa528108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.853+0000 7fa52e711700 1 -- 172.21.15.115:0/2040194016 >> 172.21.15.115:0/2040194016 conn(0x7fa5280fedc0 msgr2=0x7fa5281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:56.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.854+0000 7fa52e711700 1 -- 172.21.15.115:0/2040194016 shutdown_connections 2024-05-09T05:37:56.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.854+0000 7fa52e711700 1 -- 172.21.15.115:0/2040194016 wait complete. 2024-05-09T05:37:56.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.854+0000 7fa52e711700 1 Processor -- start 2024-05-09T05:37:56.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.854+0000 7fa52e711700 1 -- start start 2024-05-09T05:37:56.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa52e711700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5281034c0 0x7fa52819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa52e711700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa528107df0 0x7fa52819c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa52e711700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa528108740 0x7fa52819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa52e711700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa528113220 con 0x7fa528107df0 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa52e711700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281130a0 con 0x7fa528108740 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa52e711700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281133a0 con 0x7fa5281034c0 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa528107df0 0x7fa52819c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa528107df0 0x7fa52819c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36672/0 (socket says 172.21.15.115:36672) 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa51ffff700 1 -- 172.21.15.115:0/1096980875 learned_addr learned my addr 172.21.15.115:0/1096980875 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa52ccae700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa528108740 0x7fa52819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa527fff700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5281034c0 0x7fa52819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa51ffff700 1 -- 172.21.15.115:0/1096980875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5281034c0 msgr2=0x7fa52819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa51ffff700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5281034c0 0x7fa52819bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa51ffff700 1 -- 172.21.15.115:0/1096980875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa528108740 msgr2=0x7fa52819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa51ffff700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa528108740 0x7fa52819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa51ffff700 1 -- 172.21.15.115:0/1096980875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5281a6e60 con 0x7fa528107df0 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa51ffff700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa528107df0 0x7fa52819c0e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c8f0 tx=0x7fa51000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.855+0000 7fa525ffb700 1 -- 172.21.15.115:0/1096980875 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510007940 con 0x7fa528107df0 2024-05-09T05:37:56.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.856+0000 7fa525ffb700 1 -- 172.21.15.115:0/1096980875 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa51000f450 con 0x7fa528107df0 2024-05-09T05:37:56.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.856+0000 7fa525ffb700 1 -- 172.21.15.115:0/1096980875 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510018660 con 0x7fa528107df0 2024-05-09T05:37:56.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.856+0000 7fa52e711700 1 -- 172.21.15.115:0/1096980875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa528199990 con 0x7fa528107df0 2024-05-09T05:37:56.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.856+0000 7fa52e711700 1 -- 172.21.15.115:0/1096980875 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa528199e80 con 0x7fa528107df0 2024-05-09T05:37:56.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.858+0000 7fa525ffb700 1 -- 172.21.15.115:0/1096980875 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa51000f900 con 0x7fa528107df0 2024-05-09T05:37:56.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa525ffb700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa5140b2900 0x7fa5140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa527fff700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa5140b2900 0x7fa5140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa53d127b80 1 Processor -- start 2024-05-09T05:37:56.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa525ffb700 1 -- 172.21.15.115:0/1096980875 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa5100e2400 con 0x7fa528107df0 2024-05-09T05:37:56.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa53d127b80 1 -- start start 2024-05-09T05:37:56.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa527fff700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa5140b2900 0x7fa5140b4e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5180058e0 tx=0x7fa518005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:56.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 0x55c35bd83730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd7d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa53d127b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c35bca0050 con 0x55c35bd7d3b0 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa53d127b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c35bc45950 con 0x55c35bc00450 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa53d127b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c35bc9c010 con 0x55c35bc7acb0 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa52ccae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd7d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa52ccae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd7d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36684/0 (socket says 172.21.15.115:36684) 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.860+0000 7fa52ccae700 1 -- 172.21.15.115:0/1473465855 learned_addr learned my addr 172.21.15.115:0/1473465855 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa527fff700 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa51ffff700 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 0x55c35bd83730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa52ccae700 1 -- 172.21.15.115:0/1473465855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 msgr2=0x55c35bd831f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa52ccae700 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd831f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa52ccae700 1 -- 172.21.15.115:0/1473465855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 msgr2=0x55c35bd83730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa52ccae700 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 0x55c35bd83730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa52ccae700 1 -- 172.21.15.115:0/1473465855 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c35bc364b0 con 0x55c35bd7d3b0 2024-05-09T05:37:56.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa527fff700 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd831f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:56.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa52ccae700 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd7d7c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa52000f380 tx=0x7fa52000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d018f6e16d9027a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:56.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa51e7fc700 1 -- 172.21.15.115:0/1473465855 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520018030 con 0x55c35bd7d3b0 2024-05-09T05:37:56.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa51e7fc700 1 -- 172.21.15.115:0/1473465855 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa520004bb0 con 0x55c35bd7d3b0 2024-05-09T05:37:56.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.861+0000 7fa51e7fc700 1 -- 172.21.15.115:0/1473465855 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520005020 con 0x55c35bd7d3b0 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 -- 172.21.15.115:0/1473465855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 msgr2=0x55c35bd7d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd7d7c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa52000f380 tx=0x7fa52000f690 comp rx=0 tx=0).stop 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 -- 172.21.15.115:0/1473465855 shutdown_connections 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd7d7c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd831f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1473465855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 0x55c35bd83730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 -- 172.21.15.115:0/1473465855 >> 172.21.15.115:0/1473465855 conn(0x55c35bc03f20 msgr2=0x55c35bd781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 -- 172.21.15.115:0/1473465855 shutdown_connections 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 -- 172.21.15.115:0/1473465855 wait complete. 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.862+0000 7fa53d127b80 1 -- start start 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa53d127b80 1 Processor -- start 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 0x55c35bd86f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa53d127b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c35bd7e110 con 0x55c35bd7d3b0 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa53d127b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c35bc3aad0 con 0x55c35bc00450 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa53d127b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c35bd89fd0 con 0x55c35bc7acb0 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa53d127b80 10 client.0 _ll_register_callbacks cb 0x55c35bb11670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa52ccae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa52ccae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36700/0 (socket says 172.21.15.115:36700) 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa52ccae700 1 -- 172.21.15.115:0/1667682713 learned_addr learned my addr 172.21.15.115:0/1667682713 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa527fff700 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 0x55c35bd86f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.863+0000 7fa51ffff700 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa52ccae700 1 -- 172.21.15.115:0/1667682713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 msgr2=0x55c35bd874c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa52ccae700 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd874c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa52ccae700 1 -- 172.21.15.115:0/1667682713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 msgr2=0x55c35bd86f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:56.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa52ccae700 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 0x55c35bd86f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa52ccae700 1 -- 172.21.15.115:0/1667682713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c35bb1b440 con 0x55c35bd7d3b0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa51ffff700 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa52ccae700 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87a00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa520007da0 tx=0x7fa520003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f09b9d5d2ea9820 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa5077fe700 1 -- 172.21.15.115:0/1667682713 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520014880 con 0x55c35bd7d3b0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa5077fe700 1 -- 172.21.15.115:0/1667682713 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa520014a20 con 0x55c35bd7d3b0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa53d127b80 1 -- 172.21.15.115:0/1667682713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa520004bb0 con 0x55c35bd7d3b0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa5077fe700 1 -- 172.21.15.115:0/1667682713 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa520004040 con 0x55c35bd7d3b0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa53d127b80 10 client.15261 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa53d127b80 1 -- 172.21.15.115:0/1667682713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c35bd94960 con 0x55c35bd7d3b0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa5077fe700 1 -- 172.21.15.115:0/1667682713 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa520004bb0 con 0x55c35bd7d3b0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.864+0000 7fa5077fe700 1 -- 172.21.15.115:0/1667682713 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa52001d7d0 con 0x55c35bd7d3b0 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.865+0000 7fa53d127b80 10 client.15261 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.865+0000 7fa53d127b80 10 client.15261 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T05:37:56.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.865+0000 7fa53d127b80 20 client.15261 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.865+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55c35bd96bb0 0x55c35bd98020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.865+0000 7fa53d127b80 4 client.15261 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.865+0000 7fa53d127b80 1 -- 172.21.15.115:0/1667682713 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c35bcada30 con 0x55c35bd96bb0 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.865+0000 7fa527fff700 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55c35bd96bb0 0x55c35bd98020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.865+0000 7fa527fff700 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55c35bd96bb0 0x55c35bd98020 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90fbac3fb1e6a91d server_cookie=d3a1ce33f94da09a in_seq=0 out_seq=0 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.865+0000 7fa5077fe700 10 client.15261 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa5077fe700 1 -- 172.21.15.115:0/1667682713 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c35bcada30 con 0x55c35bd96bb0 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa5077fe700 10 client.15261 handle_command_reply: tid=0 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa53d127b80 1 client.15261 shutdown 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa53d127b80 1 -- 172.21.15.115:0/1667682713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 msgr2=0x55c35bd87a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87a00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa520007da0 tx=0x7fa520003ce0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa53d127b80 1 -- 172.21.15.115:0/1667682713 shutdown_connections 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc00450 0x55c35bd86f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55c35bd96bb0 0x55c35bd98020 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87a00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa5077fe700 0 client.15261 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:37:56.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa53d127b80 1 --2- 172.21.15.115:0/1667682713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc7acb0 0x55c35bd874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.866+0000 7fa53d127b80 1 -- 172.21.15.115:0/1667682713 >> 172.21.15.115:0/1667682713 conn(0x55c35bc03f20 msgr2=0x55c35bd71fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:56.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.867+0000 7fa53d127b80 1 -- 172.21.15.115:0/1667682713 shutdown_connections 2024-05-09T05:37:56.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.867+0000 7fa53d127b80 1 -- 172.21.15.115:0/1667682713 wait complete. 2024-05-09T05:37:56.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.867+0000 7fa53d127b80 20 client.15261 trim_cache size 0 max 16384 2024-05-09T05:37:56.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.887+0000 7fa53d127b80 1 Processor -- start 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.887+0000 7fa53d127b80 1 -- start start 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 0x55c35bd862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc7acb0 0x55c35bd78340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa53d127b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c35bd96360 con 0x55c35bd7d3b0 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa53d127b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c35bd96a30 con 0x55c35bc7acb0 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa53d127b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c35bd97100 con 0x55c35bc00450 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa52ccae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa52ccae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36716/0 (socket says 172.21.15.115:36716) 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa52ccae700 1 -- 172.21.15.115:0/877539484 learned_addr learned my addr 172.21.15.115:0/877539484 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa51ffff700 1 --2- 172.21.15.115:0/877539484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc7acb0 0x55c35bd78340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa527fff700 1 --2- 172.21.15.115:0/877539484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 0x55c35bd862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa52ccae700 1 -- 172.21.15.115:0/877539484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 msgr2=0x55c35bd862c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:56.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa52ccae700 1 --2- 172.21.15.115:0/877539484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 0x55c35bd862c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa52ccae700 1 -- 172.21.15.115:0/877539484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc7acb0 msgr2=0x55c35bd78340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:56.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa52ccae700 1 --2- 172.21.15.115:0/877539484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc7acb0 0x55c35bd78340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa52ccae700 1 -- 172.21.15.115:0/877539484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c35bd74980 con 0x55c35bd7d3b0 2024-05-09T05:37:56.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.888+0000 7fa52ccae700 1 --2- 172.21.15.115:0/877539484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa52000f6d0 tx=0x7fa520007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d906b9552e9e39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:56.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.889+0000 7fa51dffb700 1 -- 172.21.15.115:0/877539484 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520018070 con 0x55c35bd7d3b0 2024-05-09T05:37:56.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.889+0000 7fa51dffb700 1 -- 172.21.15.115:0/877539484 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa520025400 con 0x55c35bd7d3b0 2024-05-09T05:37:56.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.889+0000 7fa51dffb700 1 -- 172.21.15.115:0/877539484 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520025960 con 0x55c35bd7d3b0 2024-05-09T05:37:56.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.889+0000 7fa53d127b80 1 -- 172.21.15.115:0/877539484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 msgr2=0x55c35bd87840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:56.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.889+0000 7fa53d127b80 1 --2- 172.21.15.115:0/877539484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa52000f6d0 tx=0x7fa520007cc0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.889+0000 7fa53d127b80 1 -- 172.21.15.115:0/877539484 shutdown_connections 2024-05-09T05:37:56.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.890+0000 7fa53d127b80 1 --2- 172.21.15.115:0/877539484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bd7d3b0 0x55c35bd87840 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.890+0000 7fa53d127b80 1 --2- 172.21.15.115:0/877539484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 0x55c35bd862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.890+0000 7fa53d127b80 1 --2- 172.21.15.115:0/877539484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bc7acb0 0x55c35bd78340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.890+0000 7fa53d127b80 1 -- 172.21.15.115:0/877539484 >> 172.21.15.115:0/877539484 conn(0x55c35bc03f20 msgr2=0x55c35bd826e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:56.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.890+0000 7fa53d127b80 1 -- 172.21.15.115:0/877539484 shutdown_connections 2024-05-09T05:37:56.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.890+0000 7fa53d127b80 1 -- 172.21.15.115:0/877539484 wait complete. 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.890+0000 7fa53d127b80 1 -- start start 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.890+0000 7fa53d127b80 1 Processor -- start 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 0x55c35bda1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bc7acb0 0x55c35bd9ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa53d127b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bd7d3b0 0x55c35bd9b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa53d127b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c35bd97ea0 con 0x55c35bc7acb0 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa53d127b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c35bd977d0 con 0x55c35bd7d3b0 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa53d127b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c35bd98570 con 0x55c35bc00450 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa53d127b80 10 client.0 _ll_register_callbacks cb 0x55c35bb11670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bc7acb0 0x55c35bd9ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bc7acb0 0x55c35bd9ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36732/0 (socket says 172.21.15.115:36732) 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa51ffff700 1 -- 172.21.15.115:0/2652195272 learned_addr learned my addr 172.21.15.115:0/2652195272 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa52ccae700 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bd7d3b0 0x55c35bd9b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa527fff700 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 0x55c35bda1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa51ffff700 1 -- 172.21.15.115:0/2652195272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 msgr2=0x55c35bda1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa51ffff700 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 0x55c35bda1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa51ffff700 1 -- 172.21.15.115:0/2652195272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bd7d3b0 msgr2=0x55c35bd9b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa51ffff700 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bd7d3b0 0x55c35bd9b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa51ffff700 1 -- 172.21.15.115:0/2652195272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c35bd83aa0 con 0x55c35bc7acb0 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa51ffff700 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bc7acb0 0x55c35bd9ae80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa5100a5740 tx=0x7fa5100a8ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe551db8930c00e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa5077fe700 1 -- 172.21.15.115:0/2652195272 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5100c2030 con 0x55c35bc7acb0 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa53d127b80 1 -- 172.21.15.115:0/2652195272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa520025400 con 0x55c35bc7acb0 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.891+0000 7fa5077fe700 1 -- 172.21.15.115:0/2652195272 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa5100a5cb0 con 0x55c35bc7acb0 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa5077fe700 1 -- 172.21.15.115:0/2652195272 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5100ada70 con 0x55c35bc7acb0 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa53d127b80 10 client.15267 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa53d127b80 1 -- 172.21.15.115:0/2652195272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c35be02020 con 0x55c35bc7acb0 2024-05-09T05:37:56.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa5077fe700 1 -- 172.21.15.115:0/2652195272 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa520025400 con 0x55c35bc7acb0 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa5077fe700 1 -- 172.21.15.115:0/2652195272 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa5100a9920 con 0x55c35bc7acb0 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa53d127b80 10 client.15267 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa53d127b80 10 client.15267 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa53d127b80 20 client.15267 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa53d127b80 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55c35bd96bb0 0x55c35be05880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa53d127b80 4 client.15267 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa53d127b80 1 -- 172.21.15.115:0/2652195272 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "status"}) -- 0x55c35bca7ad0 con 0x55c35bd96bb0 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.892+0000 7fa527fff700 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55c35bd96bb0 0x55c35be05880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa5077fe700 10 client.15267 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa527fff700 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55c35bd96bb0 0x55c35be05880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e83d594e867af15 server_cookie=f03ea0d2fe6a3e8f in_seq=0 out_seq=0 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa5077fe700 1 -- 172.21.15.115:0/2652195272 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c35bca7ad0 con 0x55c35bd96bb0 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa5077fe700 10 client.15267 handle_command_reply: tid=0 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 client.15267 shutdown 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 -- 172.21.15.115:0/2652195272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bc7acb0 msgr2=0x55c35bd9ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:56.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bc7acb0 0x55c35bd9ae80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa5100a5740 tx=0x7fa5100a8ef0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 -- 172.21.15.115:0/2652195272 shutdown_connections 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c35bd7d3b0 0x55c35bd9b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55c35bd96bb0 0x55c35be05880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c35bc00450 0x55c35bda1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 --2- 172.21.15.115:0/2652195272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c35bc7acb0 0x55c35bd9ae80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa5077fe700 0 client.15267 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 -- 172.21.15.115:0/2652195272 >> 172.21.15.115:0/2652195272 conn(0x55c35bc03f20 msgr2=0x55c35bd878a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 -- 172.21.15.115:0/2652195272 shutdown_connections 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 1 -- 172.21.15.115:0/2652195272 wait complete. 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.893+0000 7fa53d127b80 20 client.15267 trim_cache size 0 max 16384 2024-05-09T05:37:56.892 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 2, 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24451, 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 176.62925383699999, 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 276.34587952999999 2024-05-09T05:37:56.893 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:37:56.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 -- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa5140b2900 msgr2=0x7fa5140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:56.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa5140b2900 0x7fa5140b4e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5180058e0 tx=0x7fa518005940 comp rx=0 tx=0).stop 2024-05-09T05:37:56.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 -- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa528107df0 msgr2=0x7fa52819c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:56.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa528107df0 0x7fa52819c0e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c8f0 tx=0x7fa51000cc00 comp rx=0 tx=0).stop 2024-05-09T05:37:56.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 -- 172.21.15.115:0/1096980875 shutdown_connections 2024-05-09T05:37:56.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fa5140b2900 0x7fa5140b4e10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa528108740 0x7fa52819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5281034c0 0x7fa52819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 --2- 172.21.15.115:0/1096980875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa528107df0 0x7fa52819c0e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:56.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 -- 172.21.15.115:0/1096980875 >> 172.21.15.115:0/1096980875 conn(0x7fa5280fedc0 msgr2=0x7fa528101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:56.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 -- 172.21.15.115:0/1096980875 shutdown_connections 2024-05-09T05:37:56.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:56.894+0000 7fa52e711700 1 -- 172.21.15.115:0/1096980875 wait complete. 2024-05-09T05:37:57.762 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T05:37:58.009 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:37:58.385 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:37:58 smithi115 ceph-mon[30349]: pgmap v419: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:58.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:37:58 smithi195 ceph-mon[33844]: pgmap v419: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:58.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:37:58 smithi161 ceph-mon[33805]: pgmap v419: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:37:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.788+0000 7f802f90f700 1 -- 172.21.15.115:0/4021349813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80281034c0 msgr2=0x7f80281038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.788+0000 7f802f90f700 1 --2- 172.21.15.115:0/4021349813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80281034c0 0x7f80281038d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f80240099c0 tx=0x7f8024009cd0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.788+0000 7f802f90f700 1 -- 172.21.15.115:0/4021349813 shutdown_connections 2024-05-09T05:37:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.788+0000 7f802f90f700 1 --2- 172.21.15.115:0/4021349813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8028108740 0x7f80281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.788+0000 7f802f90f700 1 --2- 172.21.15.115:0/4021349813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80281034c0 0x7f80281038d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.788+0000 7f802f90f700 1 --2- 172.21.15.115:0/4021349813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8028107df0 0x7f8028108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.788+0000 7f802f90f700 1 -- 172.21.15.115:0/4021349813 >> 172.21.15.115:0/4021349813 conn(0x7f80280fedf0 msgr2=0x7f8028101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.788+0000 7f802f90f700 1 -- 172.21.15.115:0/4021349813 shutdown_connections 2024-05-09T05:37:59.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.788+0000 7f802f90f700 1 -- 172.21.15.115:0/4021349813 wait complete. 2024-05-09T05:37:59.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.789+0000 7f802f90f700 1 Processor -- start 2024-05-09T05:37:59.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.789+0000 7f802f90f700 1 -- start start 2024-05-09T05:37:59.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.789+0000 7f802f90f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80281034c0 0x7f80280ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.789+0000 7f802f90f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8028107df0 0x7f80280ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.789+0000 7f802f90f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028108740 0x7f80280ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.789+0000 7f802f90f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8028113220 con 0x7f8028108740 2024-05-09T05:37:59.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.789+0000 7f802f90f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80281130a0 con 0x7f80281034c0 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.789+0000 7f802f90f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80281133a0 con 0x7f8028107df0 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.789+0000 7f802deac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028108740 0x7f80280ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802deac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028108740 0x7f80280ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36744/0 (socket says 172.21.15.115:36744) 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802deac700 1 -- 172.21.15.115:0/45347263 learned_addr learned my addr 172.21.15.115:0/45347263 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802ceaa700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8028107df0 0x7f80280ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802d6ab700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80281034c0 0x7f80280ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802deac700 1 -- 172.21.15.115:0/45347263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8028107df0 msgr2=0x7f80280ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802deac700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8028107df0 0x7f80280ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802deac700 1 -- 172.21.15.115:0/45347263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80281034c0 msgr2=0x7f80280ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802deac700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80281034c0 0x7f80280ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802deac700 1 -- 172.21.15.115:0/45347263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80281a9080 con 0x7f8028108740 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802d6ab700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80281034c0 0x7f80280ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802deac700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028108740 0x7f80280ffd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f802000ef30 tx=0x7f802000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:59.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f801e7fc700 1 -- 172.21.15.115:0/45347263 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200092e0 con 0x7f8028108740 2024-05-09T05:37:59.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f801e7fc700 1 -- 172.21.15.115:0/45347263 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8020004bb0 con 0x7f8028108740 2024-05-09T05:37:59.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f801e7fc700 1 -- 172.21.15.115:0/45347263 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020004020 con 0x7f8028108740 2024-05-09T05:37:59.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802f90f700 1 -- 172.21.15.115:0/45347263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80281a9310 con 0x7f8028108740 2024-05-09T05:37:59.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.790+0000 7f802f90f700 1 -- 172.21.15.115:0/45347263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a9800 con 0x7f8028108740 2024-05-09T05:37:59.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.793+0000 7f801e7fc700 1 -- 172.21.15.115:0/45347263 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f802001b030 con 0x7f8028108740 2024-05-09T05:37:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.794+0000 7f801e7fc700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f80140b2880 0x7f80140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f802d6ab700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f80140b2880 0x7f80140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f803e325b80 1 Processor -- start 2024-05-09T05:37:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f801e7fc700 1 -- 172.21.15.115:0/45347263 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f80200e20f0 con 0x7f8028108740 2024-05-09T05:37:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f803e325b80 1 -- start start 2024-05-09T05:37:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f802d6ab700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f80140b2880 0x7f80140b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8024003760 tx=0x7f802401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f803e325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 0x55fe0fcc94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f803e325b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fcc99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f803e325b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 0x55fe0fcc3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f803e325b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fbe5fa0 con 0x55fe0fb46450 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f803e325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fb847a0 con 0x55fe0fc30110 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f803e325b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fbe1f60 con 0x55fe0fcc3650 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f802ceaa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fcc99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f802ceaa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fcc99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36760/0 (socket says 172.21.15.115:36760) 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f802ceaa700 1 -- 172.21.15.115:0/3806951116 learned_addr learned my addr 172.21.15.115:0/3806951116 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f802d6ab700 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 0x55fe0fcc94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.795+0000 7f802deac700 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 0x55fe0fcc3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f802ceaa700 1 -- 172.21.15.115:0/3806951116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 msgr2=0x55fe0fcc3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f802ceaa700 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 0x55fe0fcc3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f802ceaa700 1 -- 172.21.15.115:0/3806951116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 msgr2=0x55fe0fcc94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f802ceaa700 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 0x55fe0fcc94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f802ceaa700 1 -- 172.21.15.115:0/3806951116 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0fb7c320 con 0x55fe0fb46450 2024-05-09T05:37:59.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f802d6ab700 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 0x55fe0fcc94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:37:59.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f802ceaa700 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fcc99e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8018009cf0 tx=0x7f8018011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e59edc2eaaa3636 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:59.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f80137fe700 1 -- 172.21.15.115:0/3806951116 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018011e30 con 0x55fe0fb46450 2024-05-09T05:37:59.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f80137fe700 1 -- 172.21.15.115:0/3806951116 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80180044e0 con 0x55fe0fb46450 2024-05-09T05:37:59.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.796+0000 7f80137fe700 1 -- 172.21.15.115:0/3806951116 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018005280 con 0x55fe0fb46450 2024-05-09T05:37:59.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 -- 172.21.15.115:0/3806951116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 msgr2=0x55fe0fcc99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:59.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fcc99e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8018009cf0 tx=0x7f8018011440 comp rx=0 tx=0).stop 2024-05-09T05:37:59.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 -- 172.21.15.115:0/3806951116 shutdown_connections 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 0x55fe0fcc94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 0x55fe0fcc3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 --2- 172.21.15.115:0/3806951116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fcc99e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 -- 172.21.15.115:0/3806951116 >> 172.21.15.115:0/3806951116 conn(0x55fe0fba6650 msgr2=0x55fe0fcbe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 -- 172.21.15.115:0/3806951116 shutdown_connections 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 -- 172.21.15.115:0/3806951116 wait complete. 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 -- start start 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.797+0000 7f803e325b80 1 Processor -- start 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f803e325b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fccd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f803e325b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fc30110 0x55fe0fccd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f803e325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fcc3650 0x55fe0fccdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f803e325b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fa61b50 con 0x55fe0fb46450 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f803e325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fba0100 con 0x55fe0fcc3650 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f803e325b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fcd0180 con 0x55fe0fc30110 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f803e325b80 10 client.0 _ll_register_callbacks cb 0x55fe0fa57670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802d6ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fccd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802d6ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fccd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36770/0 (socket says 172.21.15.115:36770) 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802d6ab700 1 -- 172.21.15.115:0/637124178 learned_addr learned my addr 172.21.15.115:0/637124178 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802ceaa700 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fc30110 0x55fe0fccd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802deac700 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fcc3650 0x55fe0fccdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802d6ab700 1 -- 172.21.15.115:0/637124178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fc30110 msgr2=0x55fe0fccd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802d6ab700 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fc30110 0x55fe0fccd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802d6ab700 1 -- 172.21.15.115:0/637124178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fcc3650 msgr2=0x55fe0fccdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802d6ab700 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fcc3650 0x55fe0fccdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802d6ab700 1 -- 172.21.15.115:0/637124178 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0fa61440 con 0x55fe0fb46450 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.798+0000 7f802ceaa700 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fc30110 0x55fe0fccd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:37:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f802d6ab700 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fccd090 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8024015000 tx=0x7f8024009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d8eef666ff9e0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f8010ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f8010ff9700 1 -- 172.21.15.115:0/637124178 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802400bb20 con 0x55fe0fb46450 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f8010ff9700 1 -- 172.21.15.115:0/637124178 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f802400bcc0 con 0x55fe0fb46450 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f803e325b80 1 -- 172.21.15.115:0/637124178 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80180044e0 con 0x55fe0fb46450 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f8010ff9700 1 -- 172.21.15.115:0/637124178 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f802400f460 con 0x55fe0fb46450 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f803e325b80 10 client.15279 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f803e325b80 1 -- 172.21.15.115:0/637124178 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe0fcdab30 con 0x55fe0fb46450 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f8010ff9700 1 -- 172.21.15.115:0/637124178 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f802400fa90 con 0x55fe0fb46450 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f8010ff9700 1 -- 172.21.15.115:0/637124178 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8024022070 con 0x55fe0fb46450 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f803e325b80 10 client.15279 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f803e325b80 10 client.15279 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f803e325b80 20 client.15279 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f803e325b80 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55fe0fcdd5c0 0x55fe0fcdea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f803e325b80 4 client.15279 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:37:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.799+0000 7f803e325b80 1 -- 172.21.15.115:0/637124178 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f80180044e0 con 0x55fe0fcdd5c0 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.800+0000 7f802ceaa700 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55fe0fcdd5c0 0x55fe0fcdea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.800+0000 7f8010ff9700 10 client.15279 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.800+0000 7f802ceaa700 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55fe0fcdd5c0 0x55fe0fcdea30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31a693c697c6011b server_cookie=2cc3efb2344c10ab in_seq=0 out_seq=0 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f8010ff9700 1 -- 172.21.15.115:0/637124178 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f80180044e0 con 0x55fe0fcdd5c0 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f8010ff9700 10 client.15279 handle_command_reply: tid=0 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 client.15279 shutdown 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 -- 172.21.15.115:0/637124178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 msgr2=0x55fe0fccd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fccd090 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8024015000 tx=0x7f8024009310 comp rx=0 tx=0).stop 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 -- 172.21.15.115:0/637124178 shutdown_connections 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55fe0fcdd5c0 0x55fe0fcdea30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fcc3650 0x55fe0fccdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fb46450 0x55fe0fccd090 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 --2- 172.21.15.115:0/637124178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fc30110 0x55fe0fccd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f8010ff9700 0 client.15279 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 -- 172.21.15.115:0/637124178 >> 172.21.15.115:0/637124178 conn(0x55fe0fba6650 msgr2=0x55fe0fcbe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:59.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 -- 172.21.15.115:0/637124178 shutdown_connections 2024-05-09T05:37:59.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.801+0000 7f803e325b80 1 -- 172.21.15.115:0/637124178 wait complete. 2024-05-09T05:37:59.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.802+0000 7f803e325b80 20 client.15279 trim_cache size 0 max 16384 2024-05-09T05:37:59.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.822+0000 7f803e325b80 1 Processor -- start 2024-05-09T05:37:59.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.822+0000 7f803e325b80 1 -- start start 2024-05-09T05:37:59.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.822+0000 7f803e325b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fb46450 0x55fe0fccacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.822+0000 7f803e325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 0x55fe0fcd93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.822+0000 7f803e325b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fcc3650 0x55fe0fce7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.822+0000 7f803e325b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fcdcd70 con 0x55fe0fcc3650 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.822+0000 7f803e325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fcdd440 con 0x55fe0fc30110 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.822+0000 7f803e325b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fcddb10 con 0x55fe0fb46450 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.822+0000 7f802deac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fcc3650 0x55fe0fce7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802deac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fcc3650 0x55fe0fce7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36782/0 (socket says 172.21.15.115:36782) 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802deac700 1 -- 172.21.15.115:0/2614477999 learned_addr learned my addr 172.21.15.115:0/2614477999 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802d6ab700 1 --2- 172.21.15.115:0/2614477999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fb46450 0x55fe0fccacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802ceaa700 1 --2- 172.21.15.115:0/2614477999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 0x55fe0fcd93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802deac700 1 -- 172.21.15.115:0/2614477999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fb46450 msgr2=0x55fe0fccacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802deac700 1 --2- 172.21.15.115:0/2614477999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fb46450 0x55fe0fccacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802deac700 1 -- 172.21.15.115:0/2614477999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 msgr2=0x55fe0fcd93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802deac700 1 --2- 172.21.15.115:0/2614477999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 0x55fe0fcd93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802deac700 1 -- 172.21.15.115:0/2614477999 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0fcbfcb0 con 0x55fe0fcc3650 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f802deac700 1 --2- 172.21.15.115:0/2614477999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fcc3650 0x55fe0fce7e90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8020009510 tx=0x7f80200a8c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4d59c2a14e30e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:59.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f8012ffd700 1 -- 172.21.15.115:0/2614477999 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200a9ae0 con 0x55fe0fcc3650 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f8012ffd700 1 -- 172.21.15.115:0/2614477999 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80200a93e0 con 0x55fe0fcc3650 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.823+0000 7f8012ffd700 1 -- 172.21.15.115:0/2614477999 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200a3b00 con 0x55fe0fcc3650 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 -- 172.21.15.115:0/2614477999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fcc3650 msgr2=0x55fe0fce7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 --2- 172.21.15.115:0/2614477999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fcc3650 0x55fe0fce7e90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8020009510 tx=0x7f80200a8c40 comp rx=0 tx=0).stop 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 -- 172.21.15.115:0/2614477999 shutdown_connections 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 --2- 172.21.15.115:0/2614477999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fcc3650 0x55fe0fce7e90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 --2- 172.21.15.115:0/2614477999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fb46450 0x55fe0fccacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 --2- 172.21.15.115:0/2614477999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fc30110 0x55fe0fcd93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 -- 172.21.15.115:0/2614477999 >> 172.21.15.115:0/2614477999 conn(0x55fe0fba6650 msgr2=0x55fe0fccb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:59.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 -- 172.21.15.115:0/2614477999 shutdown_connections 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 -- 172.21.15.115:0/2614477999 wait complete. 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 -- start start 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.824+0000 7f803e325b80 1 Processor -- start 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f803e325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fb46450 0x55fe0fcd87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f803e325b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fc30110 0x55fe0fcd8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f803e325b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 0x55fe0fcd9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f803e325b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fcde8b0 con 0x55fe0fc30110 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f803e325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fcde1e0 con 0x55fe0fb46450 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f803e325b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fe0fcdef80 con 0x55fe0fcc3650 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f803e325b80 10 client.0 _ll_register_callbacks cb 0x55fe0fa57670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802ceaa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fc30110 0x55fe0fcd8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802ceaa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fc30110 0x55fe0fcd8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36786/0 (socket says 172.21.15.115:36786) 2024-05-09T05:37:59.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802ceaa700 1 -- 172.21.15.115:0/3305846623 learned_addr learned my addr 172.21.15.115:0/3305846623 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802d6ab700 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fb46450 0x55fe0fcd87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802deac700 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 0x55fe0fcd9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802ceaa700 1 -- 172.21.15.115:0/3305846623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 msgr2=0x55fe0fcd9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802ceaa700 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 0x55fe0fcd9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802ceaa700 1 -- 172.21.15.115:0/3305846623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fb46450 msgr2=0x55fe0fcd87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802ceaa700 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fb46450 0x55fe0fcd87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.825+0000 7f802ceaa700 1 -- 172.21.15.115:0/3305846623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0fce6440 con 0x55fe0fc30110 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f802ceaa700 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fc30110 0x55fe0fcd8d10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55fe0fcbfe80 tx=0x7f8018004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9855bad3e764e515 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f8010ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f8010ff9700 1 -- 172.21.15.115:0/3305846623 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018035060 con 0x55fe0fc30110 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f8010ff9700 1 -- 172.21.15.115:0/3305846623 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f80180042d0 con 0x55fe0fc30110 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f803e325b80 1 -- 172.21.15.115:0/3305846623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80200a93e0 con 0x55fe0fc30110 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f8010ff9700 1 -- 172.21.15.115:0/3305846623 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f801803c050 con 0x55fe0fc30110 2024-05-09T05:37:59.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f803e325b80 10 client.15285 fetch_fsmap learned FSMap version 29 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f803e325b80 1 -- 172.21.15.115:0/3305846623 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe0fd48d80 con 0x55fe0fc30110 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f8010ff9700 1 -- 172.21.15.115:0/3305846623 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80200a93e0 con 0x55fe0fc30110 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f8010ff9700 1 -- 172.21.15.115:0/3305846623 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f801802f030 con 0x55fe0fc30110 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f803e325b80 10 client.15285 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f803e325b80 10 client.15285 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f803e325b80 20 client.15285 populate_metadata read hostname 'smithi115' 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.826+0000 7f803e325b80 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55fe0fd4b0d0 0x55fe0fd4c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.827+0000 7f803e325b80 4 client.15285 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.827+0000 7f803e325b80 1 -- 172.21.15.115:0/3305846623 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "status"}) -- 0x55fe0fbf3980 con 0x55fe0fd4b0d0 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.827+0000 7f802d6ab700 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55fe0fd4b0d0 0x55fe0fd4c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.827+0000 7f8010ff9700 10 client.15285 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.827+0000 7f802d6ab700 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55fe0fd4b0d0 0x55fe0fd4c540 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc1aafdf27b9554f server_cookie=b1eaa30fff471650 in_seq=0 out_seq=0 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.827+0000 7f8010ff9700 1 -- 172.21.15.115:0/3305846623 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fe0fbf3980 con 0x55fe0fd4b0d0 2024-05-09T05:37:59.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.827+0000 7f8010ff9700 10 client.15285 handle_command_reply: tid=0 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.827+0000 7f803e325b80 1 client.15285 shutdown 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 -- 172.21.15.115:0/3305846623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fc30110 msgr2=0x55fe0fcd8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fc30110 0x55fe0fcd8d10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55fe0fcbfe80 tx=0x7f8018004050 comp rx=0 tx=0).stop 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 -- 172.21.15.115:0/3305846623 shutdown_connections 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55fe0fd4b0d0 0x55fe0fd4c540 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe0fb46450 0x55fe0fcd87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe0fcc3650 0x55fe0fcd9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 --2- 172.21.15.115:0/3305846623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe0fc30110 0x55fe0fcd8d10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f8010ff9700 0 client.15285 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 -- 172.21.15.115:0/3305846623 >> 172.21.15.115:0/3305846623 conn(0x55fe0fba6650 msgr2=0x55fe0fcc6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 -- 172.21.15.115:0/3305846623 shutdown_connections 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 1 -- 172.21.15.115:0/3305846623 wait complete. 2024-05-09T05:37:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f803e325b80 20 client.15285 trim_cache size 0 max 16384 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 1, 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24457, 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 180.569811719, 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 276.48254979500001 2024-05-09T05:37:59.830 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:37:59.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f802f90f700 1 -- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f80140b2880 msgr2=0x7f80140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:59.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f802f90f700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f80140b2880 0x7f80140b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8024003760 tx=0x7f802401a040 comp rx=0 tx=0).stop 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f802f90f700 1 -- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028108740 msgr2=0x7f80280ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.828+0000 7f802f90f700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028108740 0x7f80280ffd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f802000ef30 tx=0x7f802000c3a0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.829+0000 7f802f90f700 1 -- 172.21.15.115:0/45347263 shutdown_connections 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.829+0000 7f802f90f700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f80140b2880 0x7f80140b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.829+0000 7f802f90f700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80281034c0 0x7f80280ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.829+0000 7f802f90f700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8028108740 0x7f80280ffd50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.829+0000 7f802f90f700 1 --2- 172.21.15.115:0/45347263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8028107df0 0x7f80280ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.829+0000 7f802f90f700 1 -- 172.21.15.115:0/45347263 >> 172.21.15.115:0/45347263 conn(0x7f80280fedf0 msgr2=0x7f8028075d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.829+0000 7f802f90f700 1 -- 172.21.15.115:0/45347263 shutdown_connections 2024-05-09T05:37:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:37:59.829+0000 7f802f90f700 1 -- 172.21.15.115:0/45347263 wait complete. 2024-05-09T05:38:00.534 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:00 smithi115 ceph-mon[30349]: pgmap v420: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:38:00.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T05:38:00.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:00 smithi195 ceph-mon[33844]: pgmap v420: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:38:00.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:00 smithi161 ceph-mon[33805]: pgmap v420: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T05:38:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:02.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:02 smithi115 ceph-mon[30349]: pgmap v421: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T05:38:02.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:02 smithi195 ceph-mon[33844]: pgmap v421: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T05:38:02.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:02 smithi161 ceph-mon[33805]: pgmap v421: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-09T05:38:02.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.703+0000 7fac872d0700 1 -- 172.21.15.115:0/977244332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103290 msgr2=0x7fac801036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:02.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.703+0000 7fac872d0700 1 --2- 172.21.15.115:0/977244332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103290 0x7fac801036a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009990 tx=0x7fac7c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.703+0000 7fac872d0700 1 -- 172.21.15.115:0/977244332 shutdown_connections 2024-05-09T05:38:02.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.703+0000 7fac872d0700 1 --2- 172.21.15.115:0/977244332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac801086e0 0x7fac80108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.703+0000 7fac872d0700 1 --2- 172.21.15.115:0/977244332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103290 0x7fac801036a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.703+0000 7fac872d0700 1 --2- 172.21.15.115:0/977244332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac80103be0 0x7fac80108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.703+0000 7fac872d0700 1 -- 172.21.15.115:0/977244332 >> 172.21.15.115:0/977244332 conn(0x7fac80078e50 msgr2=0x7fac80079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:02.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.703+0000 7fac872d0700 1 -- 172.21.15.115:0/977244332 shutdown_connections 2024-05-09T05:38:02.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.703+0000 7fac872d0700 1 -- 172.21.15.115:0/977244332 wait complete. 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac872d0700 1 Processor -- start 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac872d0700 1 -- start start 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac872d0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac80103290 0x7fac801a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac872d0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103be0 0x7fac801a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac872d0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac801086e0 0x7fac801a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac872d0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fac8011a960 con 0x7fac80103be0 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac872d0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac8011a7e0 con 0x7fac80103290 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac872d0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fac8011aae0 con 0x7fac801086e0 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac8486b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103be0 0x7fac801a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac8486b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103be0 0x7fac801a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50446/0 (socket says 172.21.15.115:50446) 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac8486b700 1 -- 172.21.15.115:0/1521853695 learned_addr learned my addr 172.21.15.115:0/1521853695 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac8506c700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac80103290 0x7fac801a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac8586d700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac801086e0 0x7fac801a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac8486b700 1 -- 172.21.15.115:0/1521853695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac801086e0 msgr2=0x7fac801a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac8486b700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac801086e0 0x7fac801a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac8486b700 1 -- 172.21.15.115:0/1521853695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac80103290 msgr2=0x7fac801a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.704+0000 7fac8486b700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac80103290 0x7fac801a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.705+0000 7fac8486b700 1 -- 172.21.15.115:0/1521853695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac801a6240 con 0x7fac80103be0 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.705+0000 7fac8586d700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac801086e0 0x7fac801a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.705+0000 7fac8486b700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103be0 0x7fac801a1910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac7000d980 tx=0x7fac7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.705+0000 7fac767fc700 1 -- 172.21.15.115:0/1521853695 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70017070 con 0x7fac80103be0 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.705+0000 7fac872d0700 1 -- 172.21.15.115:0/1521853695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac801a6530 con 0x7fac80103be0 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.705+0000 7fac767fc700 1 -- 172.21.15.115:0/1521853695 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac70009dc0 con 0x7fac80103be0 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.705+0000 7fac767fc700 1 -- 172.21.15.115:0/1521853695 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7000f3c0 con 0x7fac80103be0 2024-05-09T05:38:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.705+0000 7fac872d0700 1 -- 172.21.15.115:0/1521853695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac801ade50 con 0x7fac80103be0 2024-05-09T05:38:02.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.707+0000 7fac767fc700 1 -- 172.21.15.115:0/1521853695 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fac70009930 con 0x7fac80103be0 2024-05-09T05:38:02.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.708+0000 7fac767fc700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac6c0b2880 0x7fac6c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.708+0000 7fac8506c700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac6c0b2880 0x7fac6c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.708+0000 7fac767fc700 1 -- 172.21.15.115:0/1521853695 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fac700e20d0 con 0x7fac80103be0 2024-05-09T05:38:02.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac8506c700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac6c0b2880 0x7fac6c0b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac7c006010 tx=0x7fac7c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:02.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac95ce6b80 1 Processor -- start 2024-05-09T05:38:02.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac95ce6b80 1 -- start start 2024-05-09T05:38:02.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 0x563a9bbb91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9ba36450 0x563a9bbb9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbb37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563a9bad6050 con 0x563a9bbb33b0 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a9ba7b950 con 0x563a9bab0cb0 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563a9bad2010 con 0x563a9ba36450 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac8586d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbb37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac8586d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbb37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50454/0 (socket says 172.21.15.115:50454) 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac8586d700 1 -- 172.21.15.115:0/310020365 learned_addr learned my addr 172.21.15.115:0/310020365 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac8486b700 1 --2- 172.21.15.115:0/310020365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9ba36450 0x563a9bbb9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.709+0000 7fac8506c700 1 --2- 172.21.15.115:0/310020365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 0x563a9bbb91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.710+0000 7fac8586d700 1 -- 172.21.15.115:0/310020365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9ba36450 msgr2=0x563a9bbb9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.710+0000 7fac8586d700 1 --2- 172.21.15.115:0/310020365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9ba36450 0x563a9bbb9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.710+0000 7fac8586d700 1 -- 172.21.15.115:0/310020365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 msgr2=0x563a9bbb91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.710+0000 7fac8586d700 1 --2- 172.21.15.115:0/310020365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 0x563a9bbb91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.710+0000 7fac8586d700 1 -- 172.21.15.115:0/310020365 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a9ba6c4b0 con 0x563a9bbb33b0 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.710+0000 7fac8586d700 1 --2- 172.21.15.115:0/310020365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbb37c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fac78009dd0 tx=0x7fac780077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6f789f00f499a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.710+0000 7fac6b7fe700 1 -- 172.21.15.115:0/310020365 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78014610 con 0x563a9bbb33b0 2024-05-09T05:38:02.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.710+0000 7fac6b7fe700 1 -- 172.21.15.115:0/310020365 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac78019070 con 0x563a9bbb33b0 2024-05-09T05:38:02.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.710+0000 7fac6b7fe700 1 -- 172.21.15.115:0/310020365 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac780138a0 con 0x563a9bbb33b0 2024-05-09T05:38:02.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 -- 172.21.15.115:0/310020365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 msgr2=0x563a9bbb37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:02.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/310020365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbb37c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fac78009dd0 tx=0x7fac780077e0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 -- 172.21.15.115:0/310020365 shutdown_connections 2024-05-09T05:38:02.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/310020365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 0x563a9bbb91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/310020365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbb37c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/310020365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9ba36450 0x563a9bbb9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 -- 172.21.15.115:0/310020365 >> 172.21.15.115:0/310020365 conn(0x563a9ba39f20 msgr2=0x563a9bbae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 -- 172.21.15.115:0/310020365 shutdown_connections 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 -- 172.21.15.115:0/310020365 wait complete. 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.711+0000 7fac95ce6b80 1 -- start start 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac95ce6b80 1 Processor -- start 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 0x563a9bbbcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bab0cb0 0x563a9bbbd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbbda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563a9bbb4110 con 0x563a9bbb33b0 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a9ba70ad0 con 0x563a9ba36450 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563a9bbbffd0 con 0x563a9bab0cb0 2024-05-09T05:38:02.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac8586d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbbda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac95ce6b80 10 client.0 _ll_register_callbacks cb 0x563a9b947670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac8586d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbbda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50456/0 (socket says 172.21.15.115:50456) 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac8586d700 1 -- 172.21.15.115:0/58500634 learned_addr learned my addr 172.21.15.115:0/58500634 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac8506c700 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 0x563a9bbbcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac8486b700 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bab0cb0 0x563a9bbbd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac8586d700 1 -- 172.21.15.115:0/58500634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bab0cb0 msgr2=0x563a9bbbd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.712+0000 7fac8586d700 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bab0cb0 0x563a9bbbd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac8586d700 1 -- 172.21.15.115:0/58500634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 msgr2=0x563a9bbbcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac8586d700 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 0x563a9bbbcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac8586d700 1 -- 172.21.15.115:0/58500634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a9b951440 con 0x563a9bbb33b0 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac8486b700 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bab0cb0 0x563a9bbbd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac8586d700 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbbda50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fac78000f80 tx=0x7fac78014770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7507b0b7c3a8fd7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac68ff9700 1 -- 172.21.15.115:0/58500634 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78013520 con 0x563a9bbb33b0 2024-05-09T05:38:02.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac68ff9700 1 -- 172.21.15.115:0/58500634 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fac780136c0 con 0x563a9bbb33b0 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac95ce6b80 1 -- 172.21.15.115:0/58500634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac78019070 con 0x563a9bbb33b0 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac68ff9700 1 -- 172.21.15.115:0/58500634 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac78030050 con 0x563a9bbb33b0 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac95ce6b80 10 client.15297 fetch_fsmap learned FSMap version 29 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac95ce6b80 1 -- 172.21.15.115:0/58500634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a9bbca9e0 con 0x563a9bbb33b0 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac68ff9700 1 -- 172.21.15.115:0/58500634 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac78019070 con 0x563a9bbb33b0 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac68ff9700 1 -- 172.21.15.115:0/58500634 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac78027730 con 0x563a9bbb33b0 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.713+0000 7fac95ce6b80 10 client.15297 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.714+0000 7fac95ce6b80 10 client.15297 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.714+0000 7fac95ce6b80 20 client.15297 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.714+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563a9bbccc60 0x563a9bbce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.714+0000 7fac95ce6b80 4 client.15297 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.714+0000 7fac95ce6b80 1 -- 172.21.15.115:0/58500634 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a9bae3a30 con 0x563a9bbccc60 2024-05-09T05:38:02.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.714+0000 7fac8506c700 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563a9bbccc60 0x563a9bbce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.714+0000 7fac68ff9700 10 client.15297 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.714+0000 7fac8506c700 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563a9bbccc60 0x563a9bbce0d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56580d97046991d0 server_cookie=9a75ff3007b3bddb in_seq=0 out_seq=0 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac68ff9700 1 -- 172.21.15.115:0/58500634 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563a9bae3a30 con 0x563a9bbccc60 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac68ff9700 10 client.15297 handle_command_reply: tid=0 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac95ce6b80 1 client.15297 shutdown 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac95ce6b80 1 -- 172.21.15.115:0/58500634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 msgr2=0x563a9bbbda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbbda50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fac78000f80 tx=0x7fac78014770 comp rx=0 tx=0).stop 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac95ce6b80 1 -- 172.21.15.115:0/58500634 shutdown_connections 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 0x563a9bbbcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563a9bbccc60 0x563a9bbce0d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bbb33b0 0x563a9bbbda50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/58500634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bab0cb0 0x563a9bbbd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac68ff9700 0 client.15297 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:02.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.715+0000 7fac95ce6b80 1 -- 172.21.15.115:0/58500634 >> 172.21.15.115:0/58500634 conn(0x563a9ba39f20 msgr2=0x563a9bba7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:02.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.716+0000 7fac95ce6b80 1 -- 172.21.15.115:0/58500634 shutdown_connections 2024-05-09T05:38:02.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.716+0000 7fac95ce6b80 1 -- 172.21.15.115:0/58500634 wait complete. 2024-05-09T05:38:02.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.716+0000 7fac95ce6b80 20 client.15297 trim_cache size 0 max 16384 2024-05-09T05:38:02.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac95ce6b80 1 Processor -- start 2024-05-09T05:38:02.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac95ce6b80 1 -- start start 2024-05-09T05:38:02.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 0x563a9bba89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bab0cb0 0x563a9bbb8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 0x563a9bbbd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563a9bbcc410 con 0x563a9bab0cb0 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a9bbccae0 con 0x563a9ba36450 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563a9bbcd1b0 con 0x563a9bbb33b0 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac8486b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bab0cb0 0x563a9bbb8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac8486b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bab0cb0 0x563a9bbb8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50468/0 (socket says 172.21.15.115:50468) 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac8486b700 1 -- 172.21.15.115:0/1353501357 learned_addr learned my addr 172.21.15.115:0/1353501357 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.737+0000 7fac8506c700 1 --2- 172.21.15.115:0/1353501357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 0x563a9bba89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac8586d700 1 --2- 172.21.15.115:0/1353501357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 0x563a9bbbd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac8486b700 1 -- 172.21.15.115:0/1353501357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 msgr2=0x563a9bbbd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:02.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac8486b700 1 --2- 172.21.15.115:0/1353501357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 0x563a9bbbd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac8486b700 1 -- 172.21.15.115:0/1353501357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 msgr2=0x563a9bba89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:02.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac8486b700 1 --2- 172.21.15.115:0/1353501357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 0x563a9bba89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac8486b700 1 -- 172.21.15.115:0/1353501357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a9bbaa980 con 0x563a9bab0cb0 2024-05-09T05:38:02.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac8486b700 1 --2- 172.21.15.115:0/1353501357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bab0cb0 0x563a9bbb8730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fac7000d950 tx=0x7fac70010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac2ddf6b3eb5491 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:02.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac6affd700 1 -- 172.21.15.115:0/1353501357 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70020040 con 0x563a9bab0cb0 2024-05-09T05:38:02.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac6affd700 1 -- 172.21.15.115:0/1353501357 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac700a9a90 con 0x563a9bab0cb0 2024-05-09T05:38:02.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.738+0000 7fac6affd700 1 -- 172.21.15.115:0/1353501357 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac700b1d60 con 0x563a9bab0cb0 2024-05-09T05:38:02.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1353501357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bab0cb0 msgr2=0x563a9bbb8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:02.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1353501357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bab0cb0 0x563a9bbb8730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fac7000d950 tx=0x7fac70010f90 comp rx=0 tx=0).stop 2024-05-09T05:38:02.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1353501357 shutdown_connections 2024-05-09T05:38:02.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1353501357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9ba36450 0x563a9bba89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1353501357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 0x563a9bbbd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1353501357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9bab0cb0 0x563a9bbb8730 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1353501357 >> 172.21.15.115:0/1353501357 conn(0x563a9ba39f20 msgr2=0x563a9bbb6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:02.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1353501357 shutdown_connections 2024-05-09T05:38:02.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1353501357 wait complete. 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.739+0000 7fac95ce6b80 1 -- start start 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac95ce6b80 1 Processor -- start 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9ba36450 0x563a9bbd1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 0x563a9bbd1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac95ce6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 0x563a9bbd5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563a9bbcdf50 con 0x563a9ba36450 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a9bbcd880 con 0x563a9bab0cb0 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac95ce6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563a9bbce620 con 0x563a9bbb33b0 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac95ce6b80 10 client.0 _ll_register_callbacks cb 0x563a9b947670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac8506c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9ba36450 0x563a9bbd1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac8506c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9ba36450 0x563a9bbd1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50484/0 (socket says 172.21.15.115:50484) 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.740+0000 7fac8506c700 1 -- 172.21.15.115:0/1413393072 learned_addr learned my addr 172.21.15.115:0/1413393072 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac8586d700 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 0x563a9bbd5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac8486b700 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 0x563a9bbd1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac8506c700 1 -- 172.21.15.115:0/1413393072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 msgr2=0x563a9bbd5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac8506c700 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 0x563a9bbd5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac8506c700 1 -- 172.21.15.115:0/1413393072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 msgr2=0x563a9bbd1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac8506c700 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 0x563a9bbd1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac8506c700 1 -- 172.21.15.115:0/1413393072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a9bbd6350 con 0x563a9ba36450 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac8506c700 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9ba36450 0x563a9bbd1850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009990 tx=0x7fac7c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b70e56a7408908e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac68ff9700 1 -- 172.21.15.115:0/1413393072 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c037030 con 0x563a9ba36450 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac68ff9700 1 -- 172.21.15.115:0/1413393072 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fac7c0039d0 con 0x563a9ba36450 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1413393072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac700a9a90 con 0x563a9ba36450 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac68ff9700 1 -- 172.21.15.115:0/1413393072 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac7c044050 con 0x563a9ba36450 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac95ce6b80 10 client.15303 fetch_fsmap learned FSMap version 29 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.741+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1413393072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a9bc38850 con 0x563a9ba36450 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.742+0000 7fac68ff9700 1 -- 172.21.15.115:0/1413393072 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac7c003bd0 con 0x563a9ba36450 2024-05-09T05:38:02.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.742+0000 7fac68ff9700 1 -- 172.21.15.115:0/1413393072 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac7c022070 con 0x563a9ba36450 2024-05-09T05:38:02.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.742+0000 7fac95ce6b80 10 client.15303 fetch_fsmap finished waiting for FSMap version 29 2024-05-09T05:38:02.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.742+0000 7fac95ce6b80 10 client.15303 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T05:38:02.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.742+0000 7fac95ce6b80 20 client.15303 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:02.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.742+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563a9bc3ad40 0x563a9bc3c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:02.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.742+0000 7fac95ce6b80 4 client.15303 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:38:02.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.742+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1413393072 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "status"}) -- 0x7fac700a9a90 con 0x563a9bc3ad40 2024-05-09T05:38:02.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.742+0000 7fac8486b700 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563a9bc3ad40 0x563a9bc3c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:02.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac68ff9700 10 client.15303 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:02.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac8486b700 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563a9bc3ad40 0x563a9bc3c1b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d66e99521470b425 server_cookie=28029154e11d2de9 in_seq=0 out_seq=0 2024-05-09T05:38:02.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac68ff9700 1 -- 172.21.15.115:0/1413393072 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7fac700a9a90 con 0x563a9bc3ad40 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac68ff9700 10 client.15303 handle_command_reply: tid=0 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac95ce6b80 1 client.15303 shutdown 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1413393072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9ba36450 msgr2=0x563a9bbd1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9ba36450 0x563a9bbd1850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009990 tx=0x7fac7c01a050 comp rx=0 tx=0).stop 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1413393072 shutdown_connections 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563a9bc3ad40 0x563a9bc3c1b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563a9bbb33b0 0x563a9bbd5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563a9ba36450 0x563a9bbd1850 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac95ce6b80 1 --2- 172.21.15.115:0/1413393072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a9bab0cb0 0x563a9bbd1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1413393072 >> 172.21.15.115:0/1413393072 conn(0x563a9ba39f20 msgr2=0x563a9bba8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.743+0000 7fac68ff9700 0 client.15303 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.744+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1413393072 shutdown_connections 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.744+0000 7fac95ce6b80 1 -- 172.21.15.115:0/1413393072 wait complete. 2024-05-09T05:38:02.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.744+0000 7fac95ce6b80 20 client.15303 trim_cache size 0 max 16384 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24469, 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 247.96949455800001, 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 29, 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 87, 2024-05-09T05:38:02.742 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:38:02.743 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 273.48290118599999 2024-05-09T05:38:02.743 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:38:02.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.744+0000 7fac872d0700 1 -- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac6c0b2880 msgr2=0x7fac6c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:02.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.744+0000 7fac872d0700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac6c0b2880 0x7fac6c0b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac7c006010 tx=0x7fac7c005960 comp rx=0 tx=0).stop 2024-05-09T05:38:02.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.744+0000 7fac872d0700 1 -- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103be0 msgr2=0x7fac801a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.744+0000 7fac872d0700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103be0 0x7fac801a1910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac7000d980 tx=0x7fac7000dc90 comp rx=0 tx=0).stop 2024-05-09T05:38:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.745+0000 7fac872d0700 1 -- 172.21.15.115:0/1521853695 shutdown_connections 2024-05-09T05:38:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.745+0000 7fac872d0700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7fac6c0b2880 0x7fac6c0b4d90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.745+0000 7fac872d0700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac80103290 0x7fac801a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.745+0000 7fac872d0700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac801086e0 0x7fac801a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.745+0000 7fac872d0700 1 --2- 172.21.15.115:0/1521853695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac80103be0 0x7fac801a1910 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.745+0000 7fac872d0700 1 -- 172.21.15.115:0/1521853695 >> 172.21.15.115:0/1521853695 conn(0x7fac80078e50 msgr2=0x7fac80106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.745+0000 7fac872d0700 1 -- 172.21.15.115:0/1521853695 shutdown_connections 2024-05-09T05:38:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:02.745+0000 7fac872d0700 1 -- 172.21.15.115:0/1521853695 wait complete. 2024-05-09T05:38:03.571 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T05:38:03.580 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T05:38:03.580 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T05:38:03.581 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T05:38:03.581 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-09T05:38:03.582 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T05:38:03.582 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T05:38:03.582 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-09T05:38:03.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.706+0000 7f36536e0640 1 -- 172.21.15.115:0/1165861987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c108aa0 msgr2=0x7f364c10aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.706+0000 7f36536e0640 1 --2- 172.21.15.115:0/1165861987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c108aa0 0x7f364c10aec0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f363c009900 tx=0x7f363c02a410 comp rx=0 tx=0).stop 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.706+0000 7f36536e0640 1 -- 172.21.15.115:0/1165861987 shutdown_connections 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.706+0000 7f36536e0640 1 --2- 172.21.15.115:0/1165861987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c108aa0 0x7f364c10aec0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.706+0000 7f36536e0640 1 --2- 172.21.15.115:0/1165861987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f364c103da0 0x7f364c1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.706+0000 7f36536e0640 1 --2- 172.21.15.115:0/1165861987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f364c102b60 0x7f364c103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.706+0000 7f36536e0640 1 -- 172.21.15.115:0/1165861987 >> 172.21.15.115:0/1165861987 conn(0x7f364c0fdd50 msgr2=0x7f364c100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 -- 172.21.15.115:0/1165861987 shutdown_connections 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 -- 172.21.15.115:0/1165861987 wait complete. 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 Processor -- start 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 -- start start 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f364c102b60 0x7f364c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:03.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c103da0 0x7f364c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f364c108aa0 0x7f364c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f364c10e540 con 0x7f364c103da0 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f364c10e3c0 con 0x7f364c102b60 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f36536e0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f364c10e6c0 con 0x7f364c108aa0 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.707+0000 7f3650c7b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c103da0 0x7f364c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f3650c7b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c103da0 0x7f364c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50498/0 (socket says 172.21.15.115:50498) 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f3650c7b640 1 -- 172.21.15.115:0/4269190636 learned_addr learned my addr 172.21.15.115:0/4269190636 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f3651c7d640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f364c108aa0 0x7f364c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f3650c7b640 1 -- 172.21.15.115:0/4269190636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f364c108aa0 msgr2=0x7f364c1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f3650c7b640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f364c108aa0 0x7f364c1a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f3650c7b640 1 -- 172.21.15.115:0/4269190636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f364c102b60 msgr2=0x7f364c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f3650c7b640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f364c102b60 0x7f364c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f3650c7b640 1 -- 172.21.15.115:0/4269190636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c1a1c50 con 0x7f364c103da0 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f3650c7b640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c103da0 0x7f364c19a700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f364000e9f0 tx=0x7f364000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f364a7fc640 1 -- 172.21.15.115:0/4269190636 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364000cd80 con 0x7f364c103da0 2024-05-09T05:38:03.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364c1a1f40 con 0x7f364c103da0 2024-05-09T05:38:03.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f364a7fc640 1 -- 172.21.15.115:0/4269190636 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36400054b0 con 0x7f364c103da0 2024-05-09T05:38:03.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f364a7fc640 1 -- 172.21.15.115:0/4269190636 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640010430 con 0x7f364c103da0 2024-05-09T05:38:03.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.708+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364c1a23a0 con 0x7f364c103da0 2024-05-09T05:38:03.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.709+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f364c1a2660 con 0x7f364c103da0 2024-05-09T05:38:03.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.711+0000 7f364a7fc640 1 -- 172.21.15.115:0/4269190636 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f36400046c0 con 0x7f364c103da0 2024-05-09T05:38:03.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.711+0000 7f364a7fc640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36340b27e0 0x7f36340b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:03.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.711+0000 7f365147c640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36340b27e0 0x7f36340b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:03.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.711+0000 7f364a7fc640 1 -- 172.21.15.115:0/4269190636 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f36400e2a40 con 0x7f364c103da0 2024-05-09T05:38:03.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.712+0000 7f365147c640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36340b27e0 0x7f36340b4cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f364c103760 tx=0x7f36440071e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:03.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.716+0000 7f364a7fc640 1 -- 172.21.15.115:0/4269190636 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36400aa400 con 0x7f364c103da0 2024-05-09T05:38:03.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:03.948+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f364c108200 con 0x7f364c103da0 2024-05-09T05:38:04.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.344+0000 7f364a7fc640 1 -- 172.21.15.115:0/4269190636 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f36400a9d00 con 0x7f364c103da0 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.346+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36340b27e0 msgr2=0x7f36340b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.346+0000 7f36536e0640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36340b27e0 0x7f36340b4cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f364c103760 tx=0x7f36440071e0 comp rx=0 tx=0).stop 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.346+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c103da0 msgr2=0x7f364c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.346+0000 7f36536e0640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c103da0 0x7f364c19a700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f364000e9f0 tx=0x7f364000edf0 comp rx=0 tx=0).stop 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.347+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 shutdown_connections 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.347+0000 7f36536e0640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f364c108aa0 0x7f364c1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.347+0000 7f36536e0640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f364c103da0 0x7f364c19a700 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.347+0000 7f36536e0640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f36340b27e0 0x7f36340b4cf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.347+0000 7f36536e0640 1 --2- 172.21.15.115:0/4269190636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f364c102b60 0x7f364c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:04.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.347+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 >> 172.21.15.115:0/4269190636 conn(0x7f364c0fdd50 msgr2=0x7f364c101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:04.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.347+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 shutdown_connections 2024-05-09T05:38:04.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:04.347+0000 7f36536e0640 1 -- 172.21.15.115:0/4269190636 wait complete. 2024-05-09T05:38:04.364 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-09T05:38:04.374 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-09T05:38:04.375 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-09T05:38:04.375 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-09T05:38:04.375 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T05:38:04.375 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:04.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:04.375 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2024-05-09T05:38:04.393 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-09T05:38:04.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:04.394 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T05:38:04.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:04.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link delete ceph-brx 2024-05-09T05:38:04.563 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-09T05:38:04.564 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T05:38:04.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:04 smithi195 ceph-mon[33844]: pgmap v422: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T05:38:04.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:04 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4269190636' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-09T05:38:04.616 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: ignoring --setuser ceph since I am not root 2024-05-09T05:38:04.616 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: ignoring --setgroup ceph since I am not root 2024-05-09T05:38:04.616 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:04 smithi195 ceph-mgr[35719]: -- 172.21.15.195:0/3085585651 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55afcff7e5a0 con 0x55afd4d08480 2024-05-09T05:38:04.616 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:04.559+0000 7f483aa1e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T05:38:04.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:04 smithi115 ceph-mon[30349]: pgmap v422: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T05:38:04.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:04 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4269190636' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-09T05:38:04.646 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: ignoring --setuser ceph since I am not root 2024-05-09T05:38:04.646 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: ignoring --setgroup ceph since I am not root 2024-05-09T05:38:04.647 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:04.574+0000 7fb569825200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T05:38:04.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:04 smithi161 ceph-mon[33805]: pgmap v422: 129 pgs: 129 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-09T05:38:04.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:04 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4269190636' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-09T05:38:04.660 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:04 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: ignoring --setuser ceph since I am not root 2024-05-09T05:38:04.660 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:04 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: ignoring --setgroup ceph since I am not root 2024-05-09T05:38:04.660 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:04 smithi161 ceph-mgr[35208]: -- 172.21.15.161:0/2862139145 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf173965a0 con 0x55cf1c120900 2024-05-09T05:38:04.660 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:04 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:04.553+0000 7fd089332200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T05:38:04.825 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:05.082 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:04.778+0000 7fb569825200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:38:05.082 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:04.904+0000 7fb569825200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T05:38:05.115 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:04.746+0000 7f483aa1e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:38:05.115 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:04.852+0000 7f483aa1e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T05:38:05.150 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:04 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:04.742+0000 7fd089332200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:38:05.150 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:04 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:04.848+0000 7fd089332200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T05:38:05.150 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:05 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:05.156+0000 7fd089332200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T05:38:05.351 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:05 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:05.225+0000 7fb569825200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T05:38:05.352 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:05 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:05.316+0000 7fb569825200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T05:38:05.410 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:05 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:05.242+0000 7fd089332200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T05:38:05.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:05 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4269190636' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-09T05:38:05.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:05 smithi161 ceph-mon[33805]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-09T05:38:05.615 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:05 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:05.162+0000 7f483aa1e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T05:38:05.615 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:05 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:05.249+0000 7f483aa1e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T05:38:05.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:05 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4269190636' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-09T05:38:05.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:05 smithi195 ceph-mon[33844]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-09T05:38:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:05 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4269190636' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-09T05:38:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:05 smithi115 ceph-mon[30349]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-09T05:38:06.634 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:06 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:06.549+0000 7fb569825200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T05:38:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2667292711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48107df0 msgr2=0x7f1d48108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2667292711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48107df0 0x7f1d48108200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1d440099c0 tx=0x7f1d44009cd0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2667292711 shutdown_connections 2024-05-09T05:38:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2667292711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d48108740 0x7f1d481100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2667292711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d481034c0 0x7f1d481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2667292711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48107df0 0x7f1d48108200 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2667292711 >> 172.21.15.115:0/2667292711 conn(0x7f1d480fedc0 msgr2=0x7f1d481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2667292711 shutdown_connections 2024-05-09T05:38:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2667292711 wait complete. 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.640+0000 7f1d4f5e7700 1 Processor -- start 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4f5e7700 1 -- start start 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4f5e7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d481034c0 0x7f1d4819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4f5e7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d48107df0 0x7f1d4819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4f5e7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48108740 0x7f1d481a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4f5e7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113200 con 0x7f1d48108740 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4f5e7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113080 con 0x7f1d48107df0 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4f5e7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113380 con 0x7f1d481034c0 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4db84700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48108740 0x7f1d481a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4db84700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48108740 0x7f1d481a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50536/0 (socket says 172.21.15.115:50536) 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4cb82700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d48107df0 0x7f1d4819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4db84700 1 -- 172.21.15.115:0/2183567143 learned_addr learned my addr 172.21.15.115:0/2183567143 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4d383700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d481034c0 0x7f1d4819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4db84700 1 -- 172.21.15.115:0/2183567143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d481034c0 msgr2=0x7f1d4819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4db84700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d481034c0 0x7f1d4819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4db84700 1 -- 172.21.15.115:0/2183567143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d48107df0 msgr2=0x7f1d4819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4db84700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d48107df0 0x7f1d4819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4db84700 1 -- 172.21.15.115:0/2183567143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d481a1dd0 con 0x7f1d48108740 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.641+0000 7f1d4cb82700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d48107df0 0x7f1d4819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.642+0000 7f1d4db84700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48108740 0x7f1d481a1730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000ef30 tx=0x7f1d4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.642+0000 7f1d3a7fc700 1 -- 172.21.15.115:0/2183567143 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400092e0 con 0x7f1d48108740 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.642+0000 7f1d3a7fc700 1 -- 172.21.15.115:0/2183567143 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d40004bb0 con 0x7f1d48108740 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.642+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d481a2060 con 0x7f1d48108740 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.642+0000 7f1d3a7fc700 1 -- 172.21.15.115:0/2183567143 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40004020 con 0x7f1d48108740 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.642+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d481a2570 con 0x7f1d48108740 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.643+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d480512c0 con 0x7f1d48108740 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.645+0000 7f1d3a7fc700 1 -- 172.21.15.115:0/2183567143 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) ==== 149670+0+0 (secure 0 0 0) 0x7f1d4001b030 con 0x7f1d48108740 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.645+0000 7f1d3a7fc700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1d340b2950 0x7f1d340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.645+0000 7f1d4d383700 1 -- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1d340b2950 msgr2=0x7f1d340b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2041699318 2024-05-09T05:38:06.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.645+0000 7f1d3a7fc700 1 -- 172.21.15.115:0/2183567143 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f1d400e1f80 con 0x7f1d48108740 2024-05-09T05:38:06.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.645+0000 7f1d4d383700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1d340b2950 0x7f1d340b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:38:06.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.650+0000 7f1d3a7fc700 1 -- 172.21.15.115:0/2183567143 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d40022020 con 0x7f1d48108740 2024-05-09T05:38:06.770 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:06 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:06.431+0000 7fd089332200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T05:38:06.771 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:06 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:06.603+0000 7fd089332200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T05:38:06.771 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:06 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:06.777+0000 7fd089332200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T05:38:06.782 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:06 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:06.439+0000 7f483aa1e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T05:38:06.782 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:06 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:06.612+0000 7f483aa1e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T05:38:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.846+0000 7f1d4d383700 1 -- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1d340b2950 msgr2=0x7f1d340b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2041699318 2024-05-09T05:38:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.846+0000 7f1d4d383700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1d340b2950 0x7f1d340b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T05:38:06.896 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:06 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:06.746+0000 7fb569825200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T05:38:06.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.920+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1d4802e5f0 con 0x7f1d48108740 2024-05-09T05:38:06.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.920+0000 7f1d3a7fc700 1 -- 172.21.15.115:0/2183567143 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f1d40010040 con 0x7f1d48108740 2024-05-09T05:38:06.914 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T05:38:06.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1d340b2950 msgr2=0x7f1d340b4e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:38:06.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1d340b2950 0x7f1d340b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48108740 msgr2=0x7f1d481a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:06.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48108740 0x7f1d481a1730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000ef30 tx=0x7f1d4000c3a0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 shutdown_connections 2024-05-09T05:38:06.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f1d340b2950 0x7f1d340b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d48108740 0x7f1d481a1730 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d481034c0 0x7f1d4819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 --2- 172.21.15.115:0/2183567143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d48107df0 0x7f1d4819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:06.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 >> 172.21.15.115:0/2183567143 conn(0x7f1d480fedc0 msgr2=0x7f1d481097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:06.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 shutdown_connections 2024-05-09T05:38:06.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:06.923+0000 7f1d4f5e7700 1 -- 172.21.15.115:0/2183567143 wait complete. 2024-05-09T05:38:07.061 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:06 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2183567143' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T05:38:07.077 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:06 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2183567143' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T05:38:07.077 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:06 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:06.789+0000 7f483aa1e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T05:38:07.231 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:06 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:06.925+0000 7fb569825200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T05:38:07.231 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:06 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2183567143' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T05:38:07.359 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:07 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:07.067+0000 7fd089332200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T05:38:07.359 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:07 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:07.160+0000 7fd089332200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T05:38:07.359 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:07 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:07.365+0000 7fd089332200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T05:38:07.365 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:07 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:07.084+0000 7f483aa1e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T05:38:07.365 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:07 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:07.178+0000 7f483aa1e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T05:38:07.538 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:07 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:07.236+0000 7fb569825200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T05:38:07.538 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:07 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:07.335+0000 7fb569825200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T05:38:07.538 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:07 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:07.544+0000 7fb569825200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T05:38:07.660 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:07 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:07.561+0000 7fd089332200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:38:07.695 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T05:38:07.696 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T05:38:07.696 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2024-05-09T05:38:07.696 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T05:38:07.696 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:07.696 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T05:38:07.696 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T05:38:07.696 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T05:38:07.696 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T05:38:07.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:07.696 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T05:38:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2024-05-09T05:38:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2024-05-09T05:38:07.714 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T05:38:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2024-05-09T05:38:07.714 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T05:38:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2024-05-09T05:38:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T05:38:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 42014sec preferred_lft 42014sec 2024-05-09T05:38:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2024-05-09T05:38:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2024-05-09T05:38:07.714 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T05:38:07.715 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:38:07.715 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:38:07.715 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add name ceph-brx type bridge 2024-05-09T05:38:07.715 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr flush dev ceph-brx 2024-05-09T05:38:07.715 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set ceph-brx up 2024-05-09T05:38:07.715 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T05:38:07.715 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:38:07.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:38:07.811 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:07 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:07.751+0000 7fb569825200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:38:07.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:38:07.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:07.857 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T05:38:07.865 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:07 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:07.386+0000 7f483aa1e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T05:38:07.865 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:07 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:07.582+0000 7f483aa1e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T05:38:07.881 INFO:teuthology.orchestra.run.smithi115.stdout:1 2024-05-09T05:38:07.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:07.883 DEBUG:teuthology.orchestra.run.smithi115:> route 2024-05-09T05:38:07.939 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2024-05-09T05:38:07.939 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T05:38:07.940 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T05:38:07.940 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T05:38:07.940 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T05:38:07.940 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:38:07.940 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:38:07.940 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T05:38:07.940 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T05:38:07.940 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T05:38:07.940 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:38:08.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:38:08.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:38:08.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:08.069 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2024-05-09T05:38:08.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:08.085 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list-id 2024-05-09T05:38:08.142 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:38:08.142 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:38:08.142 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T05:38:08.142 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T05:38:08.142 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:38:08.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:38:08.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:38:08.264 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T05:38:08.264 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:38:08.264 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:38:08.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T05:38:08.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T05:38:08.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T05:38:08.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T05:38:08.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T05:38:08.265 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:38:08.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:38:08.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:38:08.382 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T05:38:08.382 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2024-05-09T05:38:08.382 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2024-05-09T05:38:08.382 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T05:38:08.382 DEBUG:teuthology.orchestra.run.smithi115:> ') 2024-05-09T05:38:08.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T05:38:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:38:08.461 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-09T05:38:08.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T05:38:08.461 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:08.477 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T05:38:08.477 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T05:38:08.478 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:08.533 DEBUG:teuthology.orchestra.run.smithi115:> sudo modprobe fuse 2024-05-09T05:38:08.596 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:08.400+0000 7fb569825200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T05:38:08.601 DEBUG:teuthology.orchestra.run.smithi115:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T05:38:08.615 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:08 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:08.188+0000 7f483aa1e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T05:38:08.615 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:08 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:08.413+0000 7f483aa1e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T05:38:08.658 INFO:teuthology.orchestra.run.smithi115.stdout:/proc 2024-05-09T05:38:08.658 INFO:teuthology.orchestra.run.smithi115.stdout:/sys 2024-05-09T05:38:08.658 INFO:teuthology.orchestra.run.smithi115.stdout:/dev 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/security 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/shm 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/pts 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/run 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/cgroup 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/pstore 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/bpf 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/ 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/selinux 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/mqueue 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/hugepages 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/debug 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/tracing 2024-05-09T05:38:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/fuse/connections 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/config 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/1000 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/ceph 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/13a2e22bb6052153ab7c86470b37bef8d0bdfc7083091867e931e28879b67b86/merged 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/acd3ad335cfcee6ed0aa2a904af0b0960dadb67054786bc04b6dc799cc080a0a/merged 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/0 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/3fca301c4cc88f1303928810526de64ff814683cc20926b9033b93fd701806aa/merged 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/7e6b44b8f210ec9bbde40c7d64b8f6ce3dbb10bbc0fb21980fa9a03ab2b47c4b/merged 2024-05-09T05:38:08.660 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/34a01ea06d6341033cabb42ced8f8049b48f28ded742a71da8c92423c8ea2ce2/merged 2024-05-09T05:38:08.661 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6c0fe2d3f5e21d60d900b5e58abede3a279e865b4da38d63d7aef65a195c9f8d/merged 2024-05-09T05:38:08.661 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f803c522de6ea34098ff189db5afa25cfbc08f325cdcd31c1e77c8ed40c44ed7/merged 2024-05-09T05:38:08.661 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/cd6f5685d865a8044769d8ef3dbdb9da57e95198eae808f462bad7f5dd7ef90c/merged 2024-05-09T05:38:08.661 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/aed36e62638f1a670ec5fd9f39ce551bb3ab57cf62f78a9841e1a6c8084cee2b/merged 2024-05-09T05:38:08.661 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f1e281515338294c1c7e459159c0cad407353f82069f197b9c0c97d1772886ed/merged 2024-05-09T05:38:08.661 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns 2024-05-09T05:38:08.661 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T05:38:08.661 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T05:38:08.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:08.661 DEBUG:teuthology.orchestra.run.smithi115:> ls /sys/fs/fuse/connections 2024-05-09T05:38:08.662 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:08 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:08.166+0000 7fd089332200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T05:38:08.662 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:08 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:08.387+0000 7fd089332200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T05:38:08.715 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T05:38:08.716 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00 --client_fs=cephfs) 2024-05-09T05:38:08.758 DEBUG:teuthology.orchestra.run.smithi115:> sudo modprobe fuse 2024-05-09T05:38:08.784 DEBUG:teuthology.orchestra.run.smithi115:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T05:38:08.843 INFO:teuthology.orchestra.run.smithi115.stdout:/proc 2024-05-09T05:38:08.843 INFO:teuthology.orchestra.run.smithi115.stdout:/sys 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/dev 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/security 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/shm 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/pts 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/run 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/cgroup 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/pstore 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/bpf 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/ 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/selinux 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/mqueue 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/hugepages 2024-05-09T05:38:08.844 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/debug 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/tracing 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/fuse/connections 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/config 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/1000 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/ceph 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/13a2e22bb6052153ab7c86470b37bef8d0bdfc7083091867e931e28879b67b86/merged 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/acd3ad335cfcee6ed0aa2a904af0b0960dadb67054786bc04b6dc799cc080a0a/merged 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/0 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:38:08.845 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/3fca301c4cc88f1303928810526de64ff814683cc20926b9033b93fd701806aa/merged 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/7e6b44b8f210ec9bbde40c7d64b8f6ce3dbb10bbc0fb21980fa9a03ab2b47c4b/merged 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/34a01ea06d6341033cabb42ced8f8049b48f28ded742a71da8c92423c8ea2ce2/merged 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6c0fe2d3f5e21d60d900b5e58abede3a279e865b4da38d63d7aef65a195c9f8d/merged 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f803c522de6ea34098ff189db5afa25cfbc08f325cdcd31c1e77c8ed40c44ed7/merged 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/cd6f5685d865a8044769d8ef3dbdb9da57e95198eae808f462bad7f5dd7ef90c/merged 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/aed36e62638f1a670ec5fd9f39ce551bb3ab57cf62f78a9841e1a6c8084cee2b/merged 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f1e281515338294c1c7e459159c0cad407353f82069f197b9c0c97d1772886ed/merged 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T05:38:08.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:08.847 DEBUG:teuthology.orchestra.run.smithi115:> ls /sys/fs/fuse/connections 2024-05-09T05:38:08.853 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi115.stderr:2024-05-09T05:38:08.860+0000 7efca1309480 -1 init, newargv = 0x7efc840046a0 newargc=15 2024-05-09T05:38:08.853 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi115.stderr:ceph-fuse[87222]: starting ceph client 2024-05-09T05:38:08.874 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi115.stderr:ceph-fuse[87222]: starting fuse 2024-05-09T05:38:08.896 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:08.644+0000 7fb569825200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T05:38:09.115 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:08 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:08.843+0000 7f483aa1e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T05:38:09.115 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:08 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:08.944+0000 7f483aa1e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T05:38:09.160 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:08 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:08.812+0000 7fd089332200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T05:38:09.160 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:08 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:08.913+0000 7fd089332200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T05:38:09.396 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:09 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:09.110+0000 7fb569825200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T05:38:09.397 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:09 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:09.211+0000 7fb569825200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T05:38:09.615 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:09 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:09.325+0000 7f483aa1e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T05:38:09.640 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:09 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:09.288+0000 7fd089332200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T05:38:09.896 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:09 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:09.591+0000 7fb569825200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T05:38:09.910 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:09 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:09.645+0000 7fd089332200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T05:38:09.910 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:09 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:09.739+0000 7fd089332200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T05:38:09.971 DEBUG:teuthology.orchestra.run.smithi115:> sudo modprobe fuse 2024-05-09T05:38:09.998 DEBUG:teuthology.orchestra.run.smithi115:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T05:38:10.022 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:09 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:09.686+0000 7f483aa1e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T05:38:10.022 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:09 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:09.780+0000 7f483aa1e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T05:38:10.056 INFO:teuthology.orchestra.run.smithi115.stdout:/proc 2024-05-09T05:38:10.056 INFO:teuthology.orchestra.run.smithi115.stdout:/sys 2024-05-09T05:38:10.056 INFO:teuthology.orchestra.run.smithi115.stdout:/dev 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/security 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/shm 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/pts 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/run 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/cgroup 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/pstore 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/bpf 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/ 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/selinux 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:38:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/mqueue 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/dev/hugepages 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/debug 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/tracing 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/fs/fuse/connections 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/sys/kernel/config 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/1000 2024-05-09T05:38:10.058 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/ceph 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/13a2e22bb6052153ab7c86470b37bef8d0bdfc7083091867e931e28879b67b86/merged 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/acd3ad335cfcee6ed0aa2a904af0b0960dadb67054786bc04b6dc799cc080a0a/merged 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/run/user/0 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/3fca301c4cc88f1303928810526de64ff814683cc20926b9033b93fd701806aa/merged 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/7e6b44b8f210ec9bbde40c7d64b8f6ce3dbb10bbc0fb21980fa9a03ab2b47c4b/merged 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/34a01ea06d6341033cabb42ced8f8049b48f28ded742a71da8c92423c8ea2ce2/merged 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/6c0fe2d3f5e21d60d900b5e58abede3a279e865b4da38d63d7aef65a195c9f8d/merged 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f803c522de6ea34098ff189db5afa25cfbc08f325cdcd31c1e77c8ed40c44ed7/merged 2024-05-09T05:38:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/cd6f5685d865a8044769d8ef3dbdb9da57e95198eae808f462bad7f5dd7ef90c/merged 2024-05-09T05:38:10.060 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/aed36e62638f1a670ec5fd9f39ce551bb3ab57cf62f78a9841e1a6c8084cee2b/merged 2024-05-09T05:38:10.060 INFO:teuthology.orchestra.run.smithi115.stdout:/var/lib/containers/storage/overlay/f1e281515338294c1c7e459159c0cad407353f82069f197b9c0c97d1772886ed/merged 2024-05-09T05:38:10.060 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns 2024-05-09T05:38:10.060 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T05:38:10.060 INFO:teuthology.orchestra.run.smithi115.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T05:38:10.060 INFO:teuthology.orchestra.run.smithi115.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:10.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:10.061 DEBUG:teuthology.orchestra.run.smithi115:> ls /sys/fs/fuse/connections 2024-05-09T05:38:10.112 INFO:teuthology.orchestra.run.smithi115.stdout:114 2024-05-09T05:38:10.113 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> sudo stdin-killer -- python3 -c ' 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> import glob 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> import re 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> import os 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> import subprocess 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> def _find_admin_socket(client_name): 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> files = glob.glob(asok_path) 2024-05-09T05:38:10.113 DEBUG:teuthology.orchestra.run.smithi115:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> # Given a non-glob path, it better be there 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> if "*" not in asok_path: 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> assert(len(files) == 1) 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> return files[0] 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> for f in files: 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> contents = proc_f.read() 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> if mountpoint in contents: 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> return f 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> 2024-05-09T05:38:10.114 DEBUG:teuthology.orchestra.run.smithi115:> print(_find_admin_socket("client.0")) 2024-05-09T05:38:10.115 DEBUG:teuthology.orchestra.run.smithi115:> ' 2024-05-09T05:38:10.234 INFO:teuthology.orchestra.run.smithi115.stdout:/var/run/ceph/ceph-client.0.87222.asok 2024-05-09T05:38:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T05:38:10.244 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:09 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:09.956+0000 7fb569825200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T05:38:10.244 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:10 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:10.057+0000 7fb569825200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T05:38:10.245 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.87222.asok 2024-05-09T05:38:10.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:10.245 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.87222.asok status 2024-05-09T05:38:10.365 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:10 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:10.029+0000 7f483aa1e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T05:38:10.365 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:10 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:10.165+0000 7f483aa1e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T05:38:10.366 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:10 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:10.263+0000 7f483aa1e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: "metadata": { 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_sha1": "c9da4049f638c0f84bdd9b4e33c42e86046ac554", 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: "entity_id": "0", 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: "hostname": "smithi115", 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: "pid": "87222", 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: "root": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00" 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:38:10.393 INFO:teuthology.orchestra.run.smithi115.stdout: "dentry_count": 0, 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "dentry_pinned_count": 0, 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 15339, 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "inst": { 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "name": { 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "client", 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 15339 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": { 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "192.168.0.1:0", 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2923235599 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:38:10.394 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": { 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "192.168.0.1:0", 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2923235599 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "inst_str": "client.15339 192.168.0.1:0/2923235599", 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "addr_str": "192.168.0.1:0/2923235599", 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "inode_count": 5, 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_epoch": 29, 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_epoch": 87, 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_epoch_barrier": 0, 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "blocklisted": false, 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs" 2024-05-09T05:38:10.395 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:38:10.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:10.403 DEBUG:teuthology.orchestra.run.smithi115:> stat --file-system '--printf=%T 2024-05-09T05:38:10.403 DEBUG:teuthology.orchestra.run.smithi115:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:10.410 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:09 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:09.984+0000 7fd089332200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T05:38:10.410 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:10 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:10.119+0000 7fd089332200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T05:38:10.410 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:10 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:10.216+0000 7fd089332200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T05:38:10.420 INFO:teuthology.orchestra.run.smithi115.stdout:fuseblk 2024-05-09T05:38:10.420 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:10.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:38:10.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:10.492 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T05:38:10.504 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T05:38:10.505 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T05:38:10.505 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-09T05:38:10.558 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:10 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:10.327+0000 7fb569825200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T05:38:10.558 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:10 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:10.473+0000 7fb569825200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T05:38:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.659+0000 7f6c9119d640 1 -- 172.21.15.115:0/2931655566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 msgr2=0x7f6c8c105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.659+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2931655566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 0x7f6c8c105620 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009900 tx=0x7f6c7c02a410 comp rx=0 tx=0).stop 2024-05-09T05:38:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.659+0000 7f6c9119d640 1 -- 172.21.15.115:0/2931655566 shutdown_connections 2024-05-09T05:38:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.659+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2931655566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1023c0 0x7f6c8c10be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.659+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2931655566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c8c1018c0 0x7f6c8c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.660+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2931655566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 0x7f6c8c105620 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.660+0000 7f6c9119d640 1 -- 172.21.15.115:0/2931655566 >> 172.21.15.115:0/2931655566 conn(0x7f6c8c0fd4c0 msgr2=0x7f6c8c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:10.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.660+0000 7f6c9119d640 1 -- 172.21.15.115:0/2931655566 shutdown_connections 2024-05-09T05:38:10.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.660+0000 7f6c9119d640 1 -- 172.21.15.115:0/2931655566 wait complete. 2024-05-09T05:38:10.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.660+0000 7f6c9119d640 1 Processor -- start 2024-05-09T05:38:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c9119d640 1 -- start start 2024-05-09T05:38:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c9119d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1018c0 0x7f6c8c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c9119d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c8c1023c0 0x7f6c8c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c9119d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 0x7f6c8c1a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c9119d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10f610 con 0x7f6c8c105210 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c9119d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10f490 con 0x7f6c8c1023c0 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c9119d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10f790 con 0x7f6c8c1018c0 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8b59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 0x7f6c8c1a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c8c1023c0 0x7f6c8c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c8c1023c0 0x7f6c8c19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:38166/0 (socket says 172.21.15.115:38166) 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8b59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 0x7f6c8c1a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50556/0 (socket says 172.21.15.115:50556) 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8a59c640 1 -- 172.21.15.115:0/2323269 learned_addr learned my addr 172.21.15.115:0/2323269 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8ad9d640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1018c0 0x7f6c8c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8a59c640 1 -- 172.21.15.115:0/2323269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1018c0 msgr2=0x7f6c8c19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8a59c640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1018c0 0x7f6c8c19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8a59c640 1 -- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 msgr2=0x7f6c8c1a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8a59c640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 0x7f6c8c1a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8a59c640 1 -- 172.21.15.115:0/2323269 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c1a1de0 con 0x7f6c8c1023c0 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.661+0000 7f6c8b59e640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 0x7f6c8c1a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.662+0000 7f6c8a59c640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c8c1023c0 0x7f6c8c19a7e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000e9f0 tx=0x7f6c8000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.662+0000 7f6c73fff640 1 -- 172.21.15.115:0/2323269 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000ccc0 con 0x7f6c8c1023c0 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.662+0000 7f6c73fff640 1 -- 172.21.15.115:0/2323269 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c800108c0 con 0x7f6c8c1023c0 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.662+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c1a20d0 con 0x7f6c8c1023c0 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.662+0000 7f6c73fff640 1 -- 172.21.15.115:0/2323269 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80018750 con 0x7f6c8c1023c0 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.662+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c1a2620 con 0x7f6c8c1023c0 2024-05-09T05:38:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.663+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c8c108ba0 con 0x7f6c8c1023c0 2024-05-09T05:38:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.664+0000 7f6c73fff640 1 -- 172.21.15.115:0/2323269 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 19) ==== 149670+0+0 (secure 0 0 0) 0x7f6c80010430 con 0x7f6c8c1023c0 2024-05-09T05:38:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.665+0000 7f6c73fff640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6c740b2830 0x7f6c740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.665+0000 7f6c8ad9d640 1 -- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6c740b2830 msgr2=0x7f6c740b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2041699318 2024-05-09T05:38:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.665+0000 7f6c73fff640 1 -- 172.21.15.115:0/2323269 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6c800e2bb0 con 0x7f6c8c1023c0 2024-05-09T05:38:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.665+0000 7f6c8ad9d640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6c740b2830 0x7f6c740b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:38:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.669+0000 7f6c73fff640 1 -- 172.21.15.115:0/2323269 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c800aa570 con 0x7f6c8c1023c0 2024-05-09T05:38:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.866+0000 7f6c8ad9d640 1 -- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6c740b2830 msgr2=0x7f6c740b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2041699318 2024-05-09T05:38:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.866+0000 7f6c8ad9d640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6c740b2830 0x7f6c740b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T05:38:10.865 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:10 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:10.597+0000 7f483aa1e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T05:38:10.865 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:10 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:10.693+0000 7f483aa1e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T05:38:10.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.899+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f6c8c02dc30 con 0x7f6c8c1023c0 2024-05-09T05:38:10.896 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:10 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:10.580+0000 7fb569825200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T05:38:10.910 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:10 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:10.551+0000 7fd089332200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T05:38:10.910 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:10 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:10.647+0000 7fd089332200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T05:38:10.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.959+0000 7f6c73fff640 1 -- 172.21.15.115:0/2323269 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7f6c800a9e70 con 0x7f6c8c1023c0 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.961+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6c740b2830 msgr2=0x7f6c740b4d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.961+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6c740b2830 0x7f6c740b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c8c1023c0 msgr2=0x7f6c8c19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c8c1023c0 0x7f6c8c19a7e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000e9f0 tx=0x7f6c8000edf0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 shutdown_connections 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c105210 0x7f6c8c1a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f6c740b2830 0x7f6c740b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c8c1023c0 0x7f6c8c19a7e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 --2- 172.21.15.115:0/2323269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1018c0 0x7f6c8c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 >> 172.21.15.115:0/2323269 conn(0x7f6c8c0fd4c0 msgr2=0x7f6c8c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 shutdown_connections 2024-05-09T05:38:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:10.962+0000 7f6c9119d640 1 -- 172.21.15.115:0/2323269 wait complete. 2024-05-09T05:38:10.966 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T05:38:10.977 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-09T05:38:10.988 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T05:38:11.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.127+0000 7f52cc9ce640 1 -- 172.21.15.115:0/2183090923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8104580 msgr2=0x7f52c810acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.127+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/2183090923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8104580 0x7f52c810acc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009ca0 tx=0x7f52c002a8c0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.127+0000 7f52cc9ce640 1 -- 172.21.15.115:0/2183090923 shutdown_connections 2024-05-09T05:38:11.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.127+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/2183090923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8104580 0x7f52c810acc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.127+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/2183090923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52c8103a40 0x7f52c8103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.127+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/2183090923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52c8103060 0x7f52c8103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.127+0000 7f52cc9ce640 1 -- 172.21.15.115:0/2183090923 >> 172.21.15.115:0/2183090923 conn(0x7f52c80fd460 msgr2=0x7f52c80ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:11.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.127+0000 7f52cc9ce640 1 -- 172.21.15.115:0/2183090923 shutdown_connections 2024-05-09T05:38:11.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.127+0000 7f52cc9ce640 1 -- 172.21.15.115:0/2183090923 wait complete. 2024-05-09T05:38:11.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52cc9ce640 1 Processor -- start 2024-05-09T05:38:11.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52cc9ce640 1 -- start start 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52cc9ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8103060 0x7f52c8102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52cc9ce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52c8103a40 0x7f52c819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52cc9ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52c8104580 0x7f52c81a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52cc9ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f52c810e550 con 0x7f52c8103060 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52cc9ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f52c810e3d0 con 0x7f52c8104580 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52cc9ce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f52c810e6d0 con 0x7f52c8103a40 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52c659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8103060 0x7f52c8102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52c659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8103060 0x7f52c8102300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50582/0 (socket says 172.21.15.115:50582) 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52c659c640 1 -- 172.21.15.115:0/4172150421 learned_addr learned my addr 172.21.15.115:0/4172150421 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52c5d9b640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52c8103a40 0x7f52c819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.128+0000 7f52c6d9d640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52c8104580 0x7f52c81a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52c5d9b640 1 -- 172.21.15.115:0/4172150421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52c8104580 msgr2=0x7f52c81a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52c5d9b640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52c8104580 0x7f52c81a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52c5d9b640 1 -- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8103060 msgr2=0x7f52c8102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52c5d9b640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8103060 0x7f52c8102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52c5d9b640 1 -- 172.21.15.115:0/4172150421 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c81a1d70 con 0x7f52c8103a40 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52c659c640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8103060 0x7f52c8102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52c5d9b640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52c8103a40 0x7f52c819f000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f52b800e9f0 tx=0x7f52b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52bf7fe640 1 -- 172.21.15.115:0/4172150421 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b800ccc0 con 0x7f52c8103a40 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52bf7fe640 1 -- 172.21.15.115:0/4172150421 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b8005020 con 0x7f52c8103a40 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c81a2060 con 0x7f52c8103a40 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52bf7fe640 1 -- 172.21.15.115:0/4172150421 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b800ce60 con 0x7f52c8103a40 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.129+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c81a2560 con 0x7f52c8103a40 2024-05-09T05:38:11.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.130+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c8109a10 con 0x7f52c8103a40 2024-05-09T05:38:11.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.133+0000 7f52bf7fe640 1 -- 172.21.15.115:0/4172150421 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 19) ==== 149670+0+0 (secure 0 0 0) 0x7f52b8004a20 con 0x7f52c8103a40 2024-05-09T05:38:11.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.133+0000 7f52bf7fe640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f52b40b2840 0x7f52b40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.133+0000 7f52c659c640 1 -- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f52b40b2840 msgr2=0x7f52b40b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2041699318 2024-05-09T05:38:11.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.133+0000 7f52bf7fe640 1 -- 172.21.15.115:0/4172150421 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f52b80e25f0 con 0x7f52c8103a40 2024-05-09T05:38:11.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.133+0000 7f52c659c640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f52b40b2840 0x7f52b40b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:38:11.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.136+0000 7f52bf7fe640 1 -- 172.21.15.115:0/4172150421 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52b80aa040 con 0x7f52c8103a40 2024-05-09T05:38:11.253 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:10 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2323269' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T05:38:11.253 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:10 smithi161 ceph-mon[33805]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T05:38:11.295 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:10 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2323269' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T05:38:11.296 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:10 smithi195 ceph-mon[33844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T05:38:11.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.322+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52c80638d0 con 0x7f52c8103a40 2024-05-09T05:38:11.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.324+0000 7f52bf7fe640 1 -- 172.21.15.115:0/4172150421 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24724 (secure 0 0 0) 0x7f52b80a9940 con 0x7f52c8103a40 2024-05-09T05:38:11.317 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:11.318 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:34:33.901006+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:38:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f52b40b2840 msgr2=0x7f52b40b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:38:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f52b40b2840 0x7f52b40b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52c8103a40 msgr2=0x7f52c819f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52c8103a40 0x7f52c819f000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f52b800e9f0 tx=0x7f52b800eeb0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 shutdown_connections 2024-05-09T05:38:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f52b40b2840 0x7f52b40b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52c8104580 0x7f52c81a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52c8103a40 0x7f52c819f000 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 --2- 172.21.15.115:0/4172150421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52c8103060 0x7f52c8102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 >> 172.21.15.115:0/4172150421 conn(0x7f52c80fd460 msgr2=0x7f52c8106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:11.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 shutdown_connections 2024-05-09T05:38:11.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.327+0000 7f52cc9ce640 1 -- 172.21.15.115:0/4172150421 wait complete. 2024-05-09T05:38:11.330 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:10 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:10.925+0000 7fb569825200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T05:38:11.330 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:11.025+0000 7fb569825200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T05:38:11.330 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:10 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2323269' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T05:38:11.330 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:10 smithi115 ceph-mon[30349]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T05:38:11.331 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T05:28:58.644670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T05:33:47.793774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T05:33:51.611373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T05:34:02.810308+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T05:38:11.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T05:38:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 -- 172.21.15.115:0/4237691627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874102b10 msgr2=0x7f1874103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 --2- 172.21.15.115:0/4237691627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874102b10 0x7f1874103810 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1864009900 tx=0x7f186402a410 comp rx=0 tx=0).stop 2024-05-09T05:38:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 -- 172.21.15.115:0/4237691627 shutdown_connections 2024-05-09T05:38:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 --2- 172.21.15.115:0/4237691627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1874108a50 0x7f187410ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 --2- 172.21.15.115:0/4237691627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1874103d50 0x7f1874104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 --2- 172.21.15.115:0/4237691627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874102b10 0x7f1874103810 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 -- 172.21.15.115:0/4237691627 >> 172.21.15.115:0/4237691627 conn(0x7f18740fdd00 msgr2=0x7f18741001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 -- 172.21.15.115:0/4237691627 shutdown_connections 2024-05-09T05:38:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 -- 172.21.15.115:0/4237691627 wait complete. 2024-05-09T05:38:11.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.455+0000 7f187933f640 1 Processor -- start 2024-05-09T05:38:11.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f187933f640 1 -- start start 2024-05-09T05:38:11.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f187933f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1874102b10 0x7f187419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f187933f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1874103d50 0x7f187419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f187933f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874108a50 0x7f18741a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f187933f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f187410f5f0 con 0x7f1874108a50 2024-05-09T05:38:11.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f187933f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f187410f470 con 0x7f1874102b10 2024-05-09T05:38:11.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f187933f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f187410f770 con 0x7f1874103d50 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f18737fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874108a50 0x7f18741a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f18737fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874108a50 0x7f18741a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50600/0 (socket says 172.21.15.115:50600) 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f18737fe640 1 -- 172.21.15.115:0/4274355126 learned_addr learned my addr 172.21.15.115:0/4274355126 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f1872ffd640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1874102b10 0x7f187419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f18727fc640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1874103d50 0x7f187419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f18737fe640 1 -- 172.21.15.115:0/4274355126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1874103d50 msgr2=0x7f187419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f18737fe640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1874103d50 0x7f187419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f18737fe640 1 -- 172.21.15.115:0/4274355126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1874102b10 msgr2=0x7f187419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f18737fe640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1874102b10 0x7f187419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f18737fe640 1 -- 172.21.15.115:0/4274355126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18741a2120 con 0x7f1874108a50 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.456+0000 7f1872ffd640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1874102b10 0x7f187419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.457+0000 7f18737fe640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874108a50 0x7f18741a1ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f186c009fd0 tx=0x7f186c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.457+0000 7f186bfff640 1 -- 172.21.15.115:0/4274355126 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c017070 con 0x7f1874108a50 2024-05-09T05:38:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.457+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18741a2410 con 0x7f1874108a50 2024-05-09T05:38:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.457+0000 7f186bfff640 1 -- 172.21.15.115:0/4274355126 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f186c003e00 con 0x7f1874108a50 2024-05-09T05:38:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.457+0000 7f186bfff640 1 -- 172.21.15.115:0/4274355126 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c010460 con 0x7f1874108a50 2024-05-09T05:38:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.457+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741a2960 con 0x7f1874108a50 2024-05-09T05:38:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.458+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1874107860 con 0x7f1874108a50 2024-05-09T05:38:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.459+0000 7f186bfff640 1 -- 172.21.15.115:0/4274355126 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) ==== 149670+0+0 (secure 0 0 0) 0x7f186c003970 con 0x7f1874108a50 2024-05-09T05:38:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.460+0000 7f186bfff640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f18600b2840 0x7f18600b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.461+0000 7f1872ffd640 1 -- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f18600b2840 msgr2=0x7f18600b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2041699318 2024-05-09T05:38:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.461+0000 7f186bfff640 1 -- 172.21.15.115:0/4274355126 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f186c0e2760 con 0x7f1874108a50 2024-05-09T05:38:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.461+0000 7f1872ffd640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f18600b2840 0x7f18600b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:38:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.464+0000 7f186bfff640 1 -- 172.21.15.115:0/4274355126 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f186c0a9a70 con 0x7f1874108a50 2024-05-09T05:38:11.615 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:11 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:11.301+0000 7f483aa1e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T05:38:11.615 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 05:38:11 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z[35707]: 2024-05-09T05:38:11.521+0000 7f483aa1e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T05:38:11.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.654+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f187402de10 con 0x7f1874108a50 2024-05-09T05:38:11.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.655+0000 7f186bfff640 1 -- 172.21.15.115:0/4274355126 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f186c0add70 con 0x7f1874108a50 2024-05-09T05:38:11.648 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-05-09T05:38:11.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f18600b2840 msgr2=0x7f18600b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f18600b2840 0x7f18600b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874108a50 msgr2=0x7f18741a1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874108a50 0x7f18741a1ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f186c009fd0 tx=0x7f186c00b260 comp rx=0 tx=0).stop 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 shutdown_connections 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1874108a50 0x7f18741a1ab0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1874103d50 0x7f187419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f18600b2840 0x7f18600b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 --2- 172.21.15.115:0/4274355126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1874102b10 0x7f187419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 >> 172.21.15.115:0/4274355126 conn(0x7f18740fdd00 msgr2=0x7f1874104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 shutdown_connections 2024-05-09T05:38:11.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.657+0000 7f187933f640 1 -- 172.21.15.115:0/4274355126 wait complete. 2024-05-09T05:38:11.660 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:11 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:11.258+0000 7fd089332200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T05:38:11.660 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 05:38:11 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y[35204]: 2024-05-09T05:38:11.480+0000 7fd089332200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T05:38:11.662 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T05:38:11.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.785+0000 7f59eee61640 1 -- 172.21.15.115:0/3288643563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 msgr2=0x7f59e8104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.785+0000 7f59eee61640 1 --2- 172.21.15.115:0/3288643563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 0x7f59e8104130 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009900 tx=0x7f59d402a410 comp rx=0 tx=0).stop 2024-05-09T05:38:11.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 -- 172.21.15.115:0/3288643563 shutdown_connections 2024-05-09T05:38:11.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 --2- 172.21.15.115:0/3288643563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59e81047c0 0x7f59e810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 --2- 172.21.15.115:0/3288643563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 0x7f59e8104130 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 --2- 172.21.15.115:0/3288643563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59e8103100 0x7f59e8103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 -- 172.21.15.115:0/3288643563 >> 172.21.15.115:0/3288643563 conn(0x7f59e80fd480 msgr2=0x7f59e80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 -- 172.21.15.115:0/3288643563 shutdown_connections 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 -- 172.21.15.115:0/3288643563 wait complete. 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 Processor -- start 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 -- start start 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59e8103100 0x7f59e81023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 0x7f59e819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59e819f340 0x7f59e81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.786+0000 7f59eee61640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f59e810e770 con 0x7f59e8103c60 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59eee61640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f59e810e5f0 con 0x7f59e8103100 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59eee61640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f59e810e8f0 con 0x7f59e819f340 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 0x7f59e819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 0x7f59e819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50606/0 (socket says 172.21.15.115:50606) 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e7fff640 1 -- 172.21.15.115:0/1213651920 learned_addr learned my addr 172.21.15.115:0/1213651920 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:11.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59ecbfd640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59e8103100 0x7f59e81023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59ed3fe640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59e819f340 0x7f59e81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e7fff640 1 -- 172.21.15.115:0/1213651920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59e819f340 msgr2=0x7f59e81a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e7fff640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59e819f340 0x7f59e81a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e7fff640 1 -- 172.21.15.115:0/1213651920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59e8103100 msgr2=0x7f59e81023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e7fff640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59e8103100 0x7f59e81023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e7fff640 1 -- 172.21.15.115:0/1213651920 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e81a1f80 con 0x7f59e8103c60 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59ecbfd640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59e8103100 0x7f59e81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e7fff640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 0x7f59e819ee00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005760 tx=0x7f59d4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e5ffb640 1 -- 172.21.15.115:0/1213651920 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d403f070 con 0x7f59e8103c60 2024-05-09T05:38:11.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e5ffb640 1 -- 172.21.15.115:0/1213651920 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59d4031da0 con 0x7f59e8103c60 2024-05-09T05:38:11.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59eee61640 1 -- 172.21.15.115:0/1213651920 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e81a2210 con 0x7f59e8103c60 2024-05-09T05:38:11.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.787+0000 7f59e5ffb640 1 -- 172.21.15.115:0/1213651920 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4035030 con 0x7f59e8103c60 2024-05-09T05:38:11.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.788+0000 7f59eee61640 1 -- 172.21.15.115:0/1213651920 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e81a2750 con 0x7f59e8103c60 2024-05-09T05:38:11.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.789+0000 7f59eee61640 1 -- 172.21.15.115:0/1213651920 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e806b4a0 con 0x7f59e8103c60 2024-05-09T05:38:11.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.790+0000 7f59e5ffb640 1 -- 172.21.15.115:0/1213651920 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) ==== 149670+0+0 (secure 0 0 0) 0x7f59d4042070 con 0x7f59e8103c60 2024-05-09T05:38:11.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.791+0000 7f59e5ffb640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59d80b28c0 0x7f59d80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:11.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.791+0000 7f59ecbfd640 1 -- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59d80b28c0 msgr2=0x7f59d80b4dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6800/2041699318 2024-05-09T05:38:11.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.791+0000 7f59e5ffb640 1 -- 172.21.15.115:0/1213651920 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f59d4102f70 con 0x7f59e8103c60 2024-05-09T05:38:11.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.791+0000 7f59ecbfd640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59d80b28c0 0x7f59d80b4dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T05:38:11.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.795+0000 7f59e5ffb640 1 -- 172.21.15.115:0/1213651920 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59d40ca930 con 0x7f59e8103c60 2024-05-09T05:38:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.933+0000 7f59e5ffb640 1 -- 172.21.15.115:0/1213651920 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 20) ==== 149337+0+0 (secure 0 0 0) 0x7f59d40c64c0 con 0x7f59e8103c60 2024-05-09T05:38:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.934+0000 7f59e5ffb640 1 -- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59d80b28c0 msgr2=0x7f59d80b4dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:38:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:11.934+0000 7f59e5ffb640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59d80b28c0 0x7f59d80b4dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:11.951 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:11.680+0000 7fb569825200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T05:38:11.951 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:11 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:11.916+0000 7fb569825200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T05:38:12.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.023+0000 7f59eee61640 1 -- 172.21.15.115:0/1213651920 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f59e802df40 con 0x7f59e8103c60 2024-05-09T05:38:12.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.024+0000 7f59e5ffb640 1 -- 172.21.15.115:0/1213651920 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f59d40fb3b0 con 0x7f59e8103c60 2024-05-09T05:38:12.017 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-05-09T05:38:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 -- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 msgr2=0x7f59e819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 0x7f59e819ee00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005760 tx=0x7f59d4003d50 comp rx=0 tx=0).stop 2024-05-09T05:38:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 -- 172.21.15.115:0/1213651920 shutdown_connections 2024-05-09T05:38:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59e819f340 0x7f59e81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e8103c60 0x7f59e819ee00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.115:6800/2041699318,v1:172.21.15.115:6801/2041699318] conn(0x7f59d80b28c0 0x7f59d80b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 --2- 172.21.15.115:0/1213651920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59e8103100 0x7f59e81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 -- 172.21.15.115:0/1213651920 >> 172.21.15.115:0/1213651920 conn(0x7f59e80fd480 msgr2=0x7f59e80ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:12.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 -- 172.21.15.115:0/1213651920 shutdown_connections 2024-05-09T05:38:12.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.027+0000 7f59eee61640 1 -- 172.21.15.115:0/1213651920 wait complete. 2024-05-09T05:38:12.040 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T05:38:12.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.195+0000 7fc52a9ce640 1 -- 172.21.15.115:0/3220659564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524103c20 msgr2=0x7fc524104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:12.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.195+0000 7fc52a9ce640 1 --2- 172.21.15.115:0/3220659564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524103c20 0x7fc524104920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc514009900 tx=0x7fc51402a410 comp rx=0 tx=0).stop 2024-05-09T05:38:12.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.195+0000 7fc52a9ce640 1 -- 172.21.15.115:0/3220659564 shutdown_connections 2024-05-09T05:38:12.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.195+0000 7fc52a9ce640 1 --2- 172.21.15.115:0/3220659564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc524109b60 0x7fc52410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.195+0000 7fc52a9ce640 1 --2- 172.21.15.115:0/3220659564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc524104e60 0x7fc524105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.195+0000 7fc52a9ce640 1 --2- 172.21.15.115:0/3220659564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524103c20 0x7fc524104920 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.195+0000 7fc52a9ce640 1 -- 172.21.15.115:0/3220659564 >> 172.21.15.115:0/3220659564 conn(0x7fc5240fee10 msgr2=0x7fc5241012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:12.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.195+0000 7fc52a9ce640 1 -- 172.21.15.115:0/3220659564 shutdown_connections 2024-05-09T05:38:12.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.195+0000 7fc52a9ce640 1 -- 172.21.15.115:0/3220659564 wait complete. 2024-05-09T05:38:12.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.196+0000 7fc52a9ce640 1 Processor -- start 2024-05-09T05:38:12.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.196+0000 7fc52a9ce640 1 -- start start 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.196+0000 7fc52a9ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc524103c20 0x7fc52419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.196+0000 7fc52a9ce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524104e60 0x7fc52419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.196+0000 7fc52a9ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc524109b60 0x7fc5241a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.196+0000 7fc52a9ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc52410f650 con 0x7fc524109b60 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.196+0000 7fc52a9ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc52410f4d0 con 0x7fc524103c20 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.196+0000 7fc52a9ce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc52410f7d0 con 0x7fc524104e60 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc5237fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524104e60 0x7fc52419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc5237fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524104e60 0x7fc52419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:45816/0 (socket says 172.21.15.115:45816) 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc5237fe640 1 -- 172.21.15.115:0/2582682827 learned_addr learned my addr 172.21.15.115:0/2582682827 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc528f6b640 1 --2- 172.21.15.115:0/2582682827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc524109b60 0x7fc5241a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc523fff640 1 --2- 172.21.15.115:0/2582682827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc524103c20 0x7fc52419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc5237fe640 1 -- 172.21.15.115:0/2582682827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc524103c20 msgr2=0x7fc52419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc5237fe640 1 --2- 172.21.15.115:0/2582682827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc524103c20 0x7fc52419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc5237fe640 1 -- 172.21.15.115:0/2582682827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc524109b60 msgr2=0x7fc5241a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:12.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc5237fe640 1 --2- 172.21.15.115:0/2582682827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc524109b60 0x7fc5241a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc5237fe640 1 -- 172.21.15.115:0/2582682827 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc524076090 con 0x7fc524104e60 2024-05-09T05:38:12.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.197+0000 7fc5237fe640 1 --2- 172.21.15.115:0/2582682827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524104e60 0x7fc52419a890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc51800e9f0 tx=0x7fc51800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:12.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.198+0000 7fc5217fa640 1 -- 172.21.15.115:0/2582682827 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51800cd80 con 0x7fc524104e60 2024-05-09T05:38:12.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.198+0000 7fc52a9ce640 1 -- 172.21.15.115:0/2582682827 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc524076380 con 0x7fc524104e60 2024-05-09T05:38:12.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.198+0000 7fc5217fa640 1 -- 172.21.15.115:0/2582682827 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5180054b0 con 0x7fc524104e60 2024-05-09T05:38:12.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.198+0000 7fc5217fa640 1 -- 172.21.15.115:0/2582682827 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518010430 con 0x7fc524104e60 2024-05-09T05:38:12.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.198+0000 7fc52a9ce640 1 -- 172.21.15.115:0/2582682827 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5240768d0 con 0x7fc524104e60 2024-05-09T05:38:12.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.199+0000 7fc52a9ce640 1 -- 172.21.15.115:0/2582682827 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc524077d70 con 0x7fc524104e60 2024-05-09T05:38:12.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.201+0000 7fc5217fa640 1 -- 172.21.15.115:0/2582682827 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 20) ==== 149337+0+0 (secure 0 0 0) 0x7fc5180046c0 con 0x7fc524104e60 2024-05-09T05:38:12.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.201+0000 7fc5217fa640 1 -- 172.21.15.115:0/2582682827 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc5180e25f0 con 0x7fc524104e60 2024-05-09T05:38:12.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.205+0000 7fc5217fa640 1 -- 172.21.15.115:0/2582682827 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc518054df0 con 0x7fc524104e60 2024-05-09T05:38:12.206 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4172150421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: Standby manager daemon y restarted 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: Standby manager daemon y started 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: Standby manager daemon z restarted 2024-05-09T05:38:12.207 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: Standby manager daemon z started 2024-05-09T05:38:12.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T05:38:12.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:38:12.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T05:38:12.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:38:12.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4274355126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T05:38:12.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: Active manager daemon x restarted 2024-05-09T05:38:12.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: Activating manager daemon x 2024-05-09T05:38:12.208 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: mgrmap e20: x(active, starting, since 0.014526s), standbys: y, z 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T05:38:12.209 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:38:12.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T05:38:12.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T05:38:12.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T05:38:12.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T05:38:12.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T05:38:12.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T05:38:12.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T05:38:12.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T05:38:12.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T05:38:12.211 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:11 smithi195 ceph-mon[33844]: Manager daemon x is now available 2024-05-09T05:38:12.212 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T05:38:12.212 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:38:12.212 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4172150421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:38:12.212 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: Standby manager daemon y restarted 2024-05-09T05:38:12.212 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: Standby manager daemon y started 2024-05-09T05:38:12.212 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: Standby manager daemon z restarted 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: Standby manager daemon z started 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4274355126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: Active manager daemon x restarted 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: Activating manager daemon x 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: mgrmap e20: x(active, starting, since 0.014526s), standbys: y, z 2024-05-09T05:38:12.213 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T05:38:12.214 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:38:12.215 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:38:12.216 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:38:12.216 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:38:12.216 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:38:12.216 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:38:12.216 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T05:38:12.216 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T05:38:12.216 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T05:38:12.216 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:11 smithi115 ceph-mon[30349]: Manager daemon x is now available 2024-05-09T05:38:12.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4172150421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: Standby manager daemon y restarted 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: Standby manager daemon y started 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.161:0/577004596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: Standby manager daemon z restarted 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: Standby manager daemon z started 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T05:38:12.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.? 172.21.15.195:0/1261504838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4274355126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: Active manager daemon x restarted 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: Activating manager daemon x 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: mgrmap e20: x(active, starting, since 0.014526s), standbys: y, z 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T05:38:12.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T05:38:12.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T05:38:12.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:11 smithi161 ceph-mon[33805]: Manager daemon x is now available 2024-05-09T05:38:12.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.428+0000 7fc52a9ce640 1 -- 172.21.15.115:0/2582682827 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc524077f70 con 0x7fc524104e60 2024-05-09T05:38:12.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.429+0000 7fc5217fa640 1 -- 172.21.15.115:0/2582682827 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc5180aa2b0 con 0x7fc524104e60 2024-05-09T05:38:12.422 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2024-05-09T05:38:12.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.432+0000 7fc52a9ce640 1 -- 172.21.15.115:0/2582682827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524104e60 msgr2=0x7fc52419a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:12.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.432+0000 7fc52a9ce640 1 --2- 172.21.15.115:0/2582682827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524104e60 0x7fc52419a890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc51800e9f0 tx=0x7fc51800edf0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.432+0000 7fc52a9ce640 1 -- 172.21.15.115:0/2582682827 shutdown_connections 2024-05-09T05:38:12.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.432+0000 7fc52a9ce640 1 --2- 172.21.15.115:0/2582682827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc524109b60 0x7fc5241a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.432+0000 7fc52a9ce640 1 --2- 172.21.15.115:0/2582682827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc524104e60 0x7fc52419a890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.432+0000 7fc52a9ce640 1 --2- 172.21.15.115:0/2582682827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc524103c20 0x7fc52419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.432+0000 7fc52a9ce640 1 -- 172.21.15.115:0/2582682827 >> 172.21.15.115:0/2582682827 conn(0x7fc5240fee10 msgr2=0x7fc524105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:12.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.432+0000 7fc52a9ce640 1 -- 172.21.15.115:0/2582682827 shutdown_connections 2024-05-09T05:38:12.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.432+0000 7fc52a9ce640 1 -- 172.21.15.115:0/2582682827 wait complete. 2024-05-09T05:38:12.437 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T05:38:12.459 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4cae75700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.459 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4cae75700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.459 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4cae75700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.459 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4cae75700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.459 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4cae75700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.459 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4c7e6f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.460 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4c7e6f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.460 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4c7e6f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.460 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4c7e6f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.460 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:38:12 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:38:12.215+0000 7fb4c7e6f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:38:12.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.563+0000 7f7062dce640 1 -- 172.21.15.115:0/3763490103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705c108a50 msgr2=0x7f705c10ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:12.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.563+0000 7f7062dce640 1 --2- 172.21.15.115:0/3763490103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705c108a50 0x7f705c10ae70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7054009ca0 tx=0x7f705402a8c0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.563+0000 7f7062dce640 1 -- 172.21.15.115:0/3763490103 shutdown_connections 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.563+0000 7f7062dce640 1 --2- 172.21.15.115:0/3763490103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705c108a50 0x7f705c10ae70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.563+0000 7f7062dce640 1 --2- 172.21.15.115:0/3763490103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705c103d50 0x7f705c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.563+0000 7f7062dce640 1 --2- 172.21.15.115:0/3763490103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f705c102b10 0x7f705c103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.563+0000 7f7062dce640 1 -- 172.21.15.115:0/3763490103 >> 172.21.15.115:0/3763490103 conn(0x7f705c0fdd00 msgr2=0x7f705c1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.563+0000 7f7062dce640 1 -- 172.21.15.115:0/3763490103 shutdown_connections 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.563+0000 7f7062dce640 1 -- 172.21.15.115:0/3763490103 wait complete. 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7062dce640 1 Processor -- start 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7062dce640 1 -- start start 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7062dce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705c102b10 0x7f705c06aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7062dce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f705c103d50 0x7f705c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7062dce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705c108a50 0x7f705c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7062dce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f705c10e540 con 0x7f705c102b10 2024-05-09T05:38:12.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7062dce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f705c10e3c0 con 0x7f705c108a50 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7062dce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f705c10e6c0 con 0x7f705c103d50 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7060b6a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705c102b10 0x7f705c06aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7060b6a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705c102b10 0x7f705c06aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34742/0 (socket says 172.21.15.115:34742) 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f7060b6a640 1 -- 172.21.15.115:0/3839421432 learned_addr learned my addr 172.21.15.115:0/3839421432 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f705bfff640 1 --2- 172.21.15.115:0/3839421432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f705c103d50 0x7f705c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.564+0000 7f706136b640 1 --2- 172.21.15.115:0/3839421432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705c108a50 0x7f705c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f705bfff640 1 -- 172.21.15.115:0/3839421432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705c108a50 msgr2=0x7f705c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f705bfff640 1 --2- 172.21.15.115:0/3839421432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705c108a50 0x7f705c1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f705bfff640 1 -- 172.21.15.115:0/3839421432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705c102b10 msgr2=0x7f705c06aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f705bfff640 1 --2- 172.21.15.115:0/3839421432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705c102b10 0x7f705c06aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f705bfff640 1 -- 172.21.15.115:0/3839421432 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c1a2080 con 0x7f705c103d50 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f705bfff640 1 --2- 172.21.15.115:0/3839421432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f705c103d50 0x7f705c19f310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f704800e9f0 tx=0x7f704800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:12.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f7059ffb640 1 -- 172.21.15.115:0/3839421432 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704800ccc0 con 0x7f705c103d50 2024-05-09T05:38:12.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f7059ffb640 1 -- 172.21.15.115:0/3839421432 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70480108c0 con 0x7f705c103d50 2024-05-09T05:38:12.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f7062dce640 1 -- 172.21.15.115:0/3839421432 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c1a2370 con 0x7f705c103d50 2024-05-09T05:38:12.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.565+0000 7f7059ffb640 1 -- 172.21.15.115:0/3839421432 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70480187e0 con 0x7f705c103d50 2024-05-09T05:38:12.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.566+0000 7f7062dce640 1 -- 172.21.15.115:0/3839421432 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1a2870 con 0x7f705c103d50 2024-05-09T05:38:12.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.568+0000 7f7062dce640 1 -- 172.21.15.115:0/3839421432 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c107860 con 0x7f705c103d50 2024-05-09T05:38:12.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.568+0000 7f7059ffb640 1 -- 172.21.15.115:0/3839421432 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 20) ==== 149337+0+0 (secure 0 0 0) 0x7f7048018980 con 0x7f705c103d50 2024-05-09T05:38:12.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.569+0000 7f7059ffb640 1 -- 172.21.15.115:0/3839421432 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f70480e9350 con 0x7f705c103d50 2024-05-09T05:38:12.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.573+0000 7f7059ffb640 1 -- 172.21.15.115:0/3839421432 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70480a9f00 con 0x7f705c103d50 2024-05-09T05:38:12.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.765+0000 7f7062dce640 1 -- 172.21.15.115:0/3839421432 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f705c02ddc0 con 0x7f705c103d50 2024-05-09T05:38:12.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.766+0000 7f7059ffb640 1 -- 172.21.15.115:0/3839421432 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) ==== 103+0+11 (secure 0 0 0) 0x7f70480a9800 con 0x7f705c103d50 2024-05-09T05:38:12.759 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-05-09T05:38:12.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.768+0000 7f7062dce640 1 -- 172.21.15.115:0/3839421432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f705c103d50 msgr2=0x7f705c19f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:12.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.768+0000 7f7062dce640 1 --2- 172.21.15.115:0/3839421432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f705c103d50 0x7f705c19f310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f704800e9f0 tx=0x7f704800eeb0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.768+0000 7f7062dce640 1 -- 172.21.15.115:0/3839421432 shutdown_connections 2024-05-09T05:38:12.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.768+0000 7f7062dce640 1 --2- 172.21.15.115:0/3839421432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705c108a50 0x7f705c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.768+0000 7f7062dce640 1 --2- 172.21.15.115:0/3839421432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f705c103d50 0x7f705c19f310 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.768+0000 7f7062dce640 1 --2- 172.21.15.115:0/3839421432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705c102b10 0x7f705c06aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:12.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.768+0000 7f7062dce640 1 -- 172.21.15.115:0/3839421432 >> 172.21.15.115:0/3839421432 conn(0x7f705c0fdd00 msgr2=0x7f705c104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:12.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.768+0000 7f7062dce640 1 -- 172.21.15.115:0/3839421432 shutdown_connections 2024-05-09T05:38:12.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:12.768+0000 7f7062dce640 1 -- 172.21.15.115:0/3839421432 wait complete. 2024-05-09T05:38:12.772 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-09T05:38:12.773 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:38:13.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:12 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:13.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:12 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1213651920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T05:38:13.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:12 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T05:38:13.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:12 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T05:38:13.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:12 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2582682827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T05:38:13.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:12 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3839421432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T05:38:13.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:12 smithi115 ceph-mon[30349]: mgrmap e21: x(active, since 1.02578s), standbys: y, z 2024-05-09T05:38:13.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:12 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:13.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:12 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1213651920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T05:38:13.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:12 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T05:38:13.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:12 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T05:38:13.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:12 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2582682827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T05:38:13.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:12 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3839421432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T05:38:13.210 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:12 smithi195 ceph-mon[33844]: mgrmap e21: x(active, since 1.02578s), standbys: y, z 2024-05-09T05:38:13.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:12 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:13.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:12 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1213651920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T05:38:13.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:12 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T05:38:13.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:12 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T05:38:13.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:12 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2582682827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T05:38:13.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:12 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3839421432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T05:38:13.241 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:12 smithi161 ceph-mon[33805]: mgrmap e21: x(active, since 1.02578s), standbys: y, z 2024-05-09T05:38:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:14.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:13 smithi115 ceph-mon[30349]: [09/May/2024:05:38:12] ENGINE Bus STARTING 2024-05-09T05:38:14.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:13 smithi115 ceph-mon[30349]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:14.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:13 smithi115 ceph-mon[30349]: [09/May/2024:05:38:13] ENGINE Serving on https://172.21.15.115:7150 2024-05-09T05:38:14.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:13 smithi115 ceph-mon[30349]: [09/May/2024:05:38:13] ENGINE Serving on http://172.21.15.115:8765 2024-05-09T05:38:14.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:13 smithi115 ceph-mon[30349]: [09/May/2024:05:38:13] ENGINE Bus STARTED 2024-05-09T05:38:14.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:13 smithi195 ceph-mon[33844]: [09/May/2024:05:38:12] ENGINE Bus STARTING 2024-05-09T05:38:14.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:13 smithi195 ceph-mon[33844]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:14.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:13 smithi195 ceph-mon[33844]: [09/May/2024:05:38:13] ENGINE Serving on https://172.21.15.115:7150 2024-05-09T05:38:14.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:13 smithi195 ceph-mon[33844]: [09/May/2024:05:38:13] ENGINE Serving on http://172.21.15.115:8765 2024-05-09T05:38:14.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:13 smithi195 ceph-mon[33844]: [09/May/2024:05:38:13] ENGINE Bus STARTED 2024-05-09T05:38:14.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:13 smithi161 ceph-mon[33805]: [09/May/2024:05:38:12] ENGINE Bus STARTING 2024-05-09T05:38:14.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:13 smithi161 ceph-mon[33805]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:14.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:13 smithi161 ceph-mon[33805]: [09/May/2024:05:38:13] ENGINE Serving on https://172.21.15.115:7150 2024-05-09T05:38:14.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:13 smithi161 ceph-mon[33805]: [09/May/2024:05:38:13] ENGINE Serving on http://172.21.15.115:8765 2024-05-09T05:38:14.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:13 smithi161 ceph-mon[33805]: [09/May/2024:05:38:13] ENGINE Bus STARTED 2024-05-09T05:38:15.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:14 smithi115 ceph-mon[30349]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:15.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:14 smithi115 ceph-mon[30349]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-05-09T05:38:15.240 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:14 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:14 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:14 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:14 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:14 smithi195 ceph-mon[33844]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:15.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:14 smithi195 ceph-mon[33844]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-05-09T05:38:15.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:14 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:14 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:14 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.367 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:14 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:14 smithi161 ceph-mon[33805]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:15.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:14 smithi161 ceph-mon[33805]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-05-09T05:38:15.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:14 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:14 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:14 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:14 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.649+0000 7fb09a3e8700 1 -- 172.21.15.115:0/3752986987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094103310 msgr2=0x7fb09410f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.649+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/3752986987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094103310 0x7fb09410f890 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00b0a0 tx=0x7fb08c00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.649+0000 7fb09a3e8700 1 -- 172.21.15.115:0/3752986987 shutdown_connections 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.649+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/3752986987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0941089f0 0x7fb094108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.649+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/3752986987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094103310 0x7fb09410f890 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.649+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/3752986987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0941029c0 0x7fb094102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.649+0000 7fb09a3e8700 1 -- 172.21.15.115:0/3752986987 >> 172.21.15.115:0/3752986987 conn(0x7fb0940fe6f0 msgr2=0x7fb094100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.649+0000 7fb09a3e8700 1 -- 172.21.15.115:0/3752986987 shutdown_connections 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.649+0000 7fb09a3e8700 1 -- 172.21.15.115:0/3752986987 wait complete. 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.650+0000 7fb09a3e8700 1 Processor -- start 2024-05-09T05:38:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.650+0000 7fb09a3e8700 1 -- start start 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb09a3e8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0941029c0 0x7fb09419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb09a3e8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb094103310 0x7fb09419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb09a3e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0941089f0 0x7fb09419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb09a3e8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941121a0 con 0x7fb0941089f0 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb09a3e8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb094112020 con 0x7fb094103310 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb09a3e8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb094112320 con 0x7fb0941029c0 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb0937fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb094103310 0x7fb09419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb098985700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0941089f0 0x7fb09419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb0937fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb094103310 0x7fb09419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:56690/0 (socket says 172.21.15.115:56690) 2024-05-09T05:38:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb0937fe700 1 -- 172.21.15.115:0/941968186 learned_addr learned my addr 172.21.15.115:0/941968186 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.651+0000 7fb093fff700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0941029c0 0x7fb09419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb098985700 1 -- 172.21.15.115:0/941968186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0941029c0 msgr2=0x7fb09419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb098985700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0941029c0 0x7fb09419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb098985700 1 -- 172.21.15.115:0/941968186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb094103310 msgr2=0x7fb09419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb098985700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb094103310 0x7fb09419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb098985700 1 -- 172.21.15.115:0/941968186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09419fc90 con 0x7fb0941089f0 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb098985700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0941089f0 0x7fb09419f5f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00b0a0 tx=0x7fb08c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb0917fa700 1 -- 172.21.15.115:0/941968186 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c021070 con 0x7fb0941089f0 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb0917fa700 1 -- 172.21.15.115:0/941968186 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb08c007b80 con 0x7fb0941089f0 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09419ff20 con 0x7fb0941089f0 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb0917fa700 1 -- 172.21.15.115:0/941968186 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c005040 con 0x7fb0941089f0 2024-05-09T05:38:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.652+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0941a0400 con 0x7fb0941089f0 2024-05-09T05:38:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.654+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0940512c0 con 0x7fb0941089f0 2024-05-09T05:38:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.656+0000 7fb0917fa700 1 -- 172.21.15.115:0/941968186 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 22) ==== 149773+0+0 (secure 0 0 0) 0x7fb08c024070 con 0x7fb0941089f0 2024-05-09T05:38:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.656+0000 7fb0917fa700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb07c0b29f0 0x7fb07c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.657+0000 7fb093fff700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb07c0b29f0 0x7fb07c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.657+0000 7fb0917fa700 1 -- 172.21.15.115:0/941968186 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fb08c00f020 con 0x7fb0941089f0 2024-05-09T05:38:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.657+0000 7fb093fff700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb07c0b29f0 0x7fb07c0b4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb084005950 tx=0x7fb08400b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.661+0000 7fb0917fa700 1 -- 172.21.15.115:0/941968186 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb08c02b020 con 0x7fb0941089f0 2024-05-09T05:38:15.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.938+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb09402e5f0 con 0x7fb0941089f0 2024-05-09T05:38:15.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.938+0000 7fb0917fa700 1 -- 172.21.15.115:0/941968186 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7fb08c0ab9c0 con 0x7fb0941089f0 2024-05-09T05:38:15.932 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:15.932 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb07c0b29f0 msgr2=0x7fb07c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb07c0b29f0 0x7fb07c0b4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb084005950 tx=0x7fb08400b410 comp rx=0 tx=0).stop 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0941089f0 msgr2=0x7fb09419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0941089f0 0x7fb09419f5f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00b0a0 tx=0x7fb08c004ce0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 shutdown_connections 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb07c0b29f0 0x7fb07c0b4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0941089f0 0x7fb09419f5f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0941029c0 0x7fb09419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 --2- 172.21.15.115:0/941968186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb094103310 0x7fb09419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:15.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 >> 172.21.15.115:0/941968186 conn(0x7fb0940fe6f0 msgr2=0x7fb094107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:15.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 shutdown_connections 2024-05-09T05:38:15.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:15.941+0000 7fb09a3e8700 1 -- 172.21.15.115:0/941968186 wait complete. 2024-05-09T05:38:15.935 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:38:16.555 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: Unable to set osd_memory_target on smithi161 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.556 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.557 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-09T05:38:16.557 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: Unable to set osd_memory_target on smithi195 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T05:38:16.557 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.557 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.557 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:16 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/941968186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: Unable to set osd_memory_target on smithi161 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-09T05:38:16.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: Unable to set osd_memory_target on smithi195 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T05:38:16.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:16 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/941968186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:16.654 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-09T05:38:16.654 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-09T05:38:16.654 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T05:38:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-05-09T05:38:16.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: Unable to set osd_memory_target on smithi161 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: Unable to set osd_memory_target on smithi195 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:16.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:16 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/941968186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:16.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.781+0000 7f36e0649640 1 -- 172.21.15.115:0/908524108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8102af0 msgr2=0x7f36d81037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:16.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.781+0000 7f36e0649640 1 --2- 172.21.15.115:0/908524108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8102af0 0x7f36d81037f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009900 tx=0x7f36d402e820 comp rx=0 tx=0).stop 2024-05-09T05:38:16.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.781+0000 7f36e0649640 1 -- 172.21.15.115:0/908524108 shutdown_connections 2024-05-09T05:38:16.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.781+0000 7f36e0649640 1 --2- 172.21.15.115:0/908524108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d8108a30 0x7f36d810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:16.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.781+0000 7f36e0649640 1 --2- 172.21.15.115:0/908524108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8103d30 0x7f36d8104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:16.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.781+0000 7f36e0649640 1 --2- 172.21.15.115:0/908524108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8102af0 0x7f36d81037f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:16.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.781+0000 7f36e0649640 1 -- 172.21.15.115:0/908524108 >> 172.21.15.115:0/908524108 conn(0x7f36d80fdd00 msgr2=0x7f36d8100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:16.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.781+0000 7f36e0649640 1 -- 172.21.15.115:0/908524108 shutdown_connections 2024-05-09T05:38:16.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.781+0000 7f36e0649640 1 -- 172.21.15.115:0/908524108 wait complete. 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.782+0000 7f36e0649640 1 Processor -- start 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.782+0000 7f36e0649640 1 -- start start 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36e0649640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d8102af0 0x7f36d8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36e0649640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8103d30 0x7f36d81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36e0649640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8108a30 0x7f36d819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36e0649640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f36d810d420 con 0x7f36d8108a30 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36e0649640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36d810d2a0 con 0x7f36d8103d30 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36e0649640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36d810d5a0 con 0x7f36d8102af0 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36debe6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8108a30 0x7f36d819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36de3e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d8102af0 0x7f36d8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:16.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36de3e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d8102af0 0x7f36d8198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:40284/0 (socket says 172.21.15.115:40284) 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36debe6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8108a30 0x7f36d819f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34812/0 (socket says 172.21.15.115:34812) 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36de3e5640 1 -- 172.21.15.115:0/3798513441 learned_addr learned my addr 172.21.15.115:0/3798513441 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36ddbe4640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8103d30 0x7f36d81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36ddbe4640 1 -- 172.21.15.115:0/3798513441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d8102af0 msgr2=0x7f36d8198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36ddbe4640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d8102af0 0x7f36d8198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36ddbe4640 1 -- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8108a30 msgr2=0x7f36d819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36ddbe4640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8108a30 0x7f36d819f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36ddbe4640 1 -- 172.21.15.115:0/3798513441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d819fb70 con 0x7f36d8103d30 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36de3e5640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d8102af0 0x7f36d8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.783+0000 7f36debe6640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8108a30 0x7f36d819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.784+0000 7f36ddbe4640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8103d30 0x7f36d81985c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36c800e9f0 tx=0x7f36c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.784+0000 7f36cf7fe640 1 -- 172.21.15.115:0/3798513441 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c800ccc0 con 0x7f36d8103d30 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.784+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d819fe60 con 0x7f36d8103d30 2024-05-09T05:38:16.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.784+0000 7f36cf7fe640 1 -- 172.21.15.115:0/3798513441 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36c80108c0 con 0x7f36d8103d30 2024-05-09T05:38:16.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.784+0000 7f36cf7fe640 1 -- 172.21.15.115:0/3798513441 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c80187c0 con 0x7f36d8103d30 2024-05-09T05:38:16.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.784+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d81a0320 con 0x7f36d8103d30 2024-05-09T05:38:16.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.785+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d8107840 con 0x7f36d8103d30 2024-05-09T05:38:16.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.787+0000 7f36cf7fe640 1 -- 172.21.15.115:0/3798513441 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 22) ==== 149773+0+0 (secure 0 0 0) 0x7f36c8010430 con 0x7f36d8103d30 2024-05-09T05:38:16.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.788+0000 7f36cf7fe640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36c40b2990 0x7f36c40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:16.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.788+0000 7f36de3e5640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36c40b2990 0x7f36c40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:16.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.788+0000 7f36cf7fe640 1 -- 172.21.15.115:0/3798513441 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f36c80e2e70 con 0x7f36d8103d30 2024-05-09T05:38:16.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.788+0000 7f36de3e5640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36c40b2990 0x7f36c40b4ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f36d4005f70 tx=0x7f36d40056c0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:16.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.793+0000 7f36cf7fe640 1 -- 172.21.15.115:0/3798513441 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36c80a9ec0 con 0x7f36d8103d30 2024-05-09T05:38:16.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.933+0000 7f36cf7fe640 1 -- 172.21.15.115:0/3798513441 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f36c80e0500 con 0x7f36d8103d30 2024-05-09T05:38:16.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.988+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36d8063110 con 0x7f36c40b2990 2024-05-09T05:38:16.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:16.998+0000 7f36cf7fe640 1 -- 172.21.15.115:0/3798513441 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339703 (secure 0 0 0) 0x7f36d8063110 con 0x7f36c40b2990 2024-05-09T05:38:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:17.003 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36c40b2990 msgr2=0x7f36c40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36c40b2990 0x7f36c40b4ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f36d4005f70 tx=0x7f36d40056c0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8103d30 msgr2=0x7f36d81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8103d30 0x7f36d81985c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36c800e9f0 tx=0x7f36c800eeb0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 shutdown_connections 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36c40b2990 0x7f36c40b4ea0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36d8108a30 0x7f36d819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36d8103d30 0x7f36d81985c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 --2- 172.21.15.115:0/3798513441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36d8102af0 0x7f36d8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 >> 172.21.15.115:0/3798513441 conn(0x7f36d80fdd00 msgr2=0x7f36d8104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:17.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 shutdown_connections 2024-05-09T05:38:17.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.003+0000 7f36e0649640 1 -- 172.21.15.115:0/3798513441 wait complete. 2024-05-09T05:38:17.011 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-09T05:38:15.942399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195491,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2124,"num_read_kb":1211,"num_write":519,"num_write_kb":3632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":909,"ondisk_log_size":909,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344656,"kb_used_data":12092,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597168,"statfs":{"total":1151940427776,"available":1151587500032,"internally_reserved":0,"allocated":12382208,"data_stored":9553557,"data_compressed":531766,"data_compressed_allocated":716800,"data_compressed_original":4231168,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987801"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925762+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:38:11.925762+0000","last_peered":"2024-05-09T05:38:11.925762+0000","last_clean":"2024-05-09T05:38:11.925762+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:38:11.925762+0000","last_undegraded":"2024-05-09T05:38:11.925762+0000","last_fullsized":"2024-05-09T05:38:11.925762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979667+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:38:11.979667+0000","last_peered":"2024-05-09T05:38:11.979667+0000","last_clean":"2024-05-09T05:38:11.979667+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:38:11.979667+0000","last_undegraded":"2024-05-09T05:38:11.979667+0000","last_fullsized":"2024-05-09T05:38:11.979667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979267+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:38:11.979267+0000","last_peered":"2024-05-09T05:38:11.979267+0000","last_clean":"2024-05-09T05:38:11.979267+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:38:11.979267+0000","last_undegraded":"2024-05-09T05:38:11.979267+0000","last_fullsized":"2024-05-09T05:38:11.979267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133401+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:38:12.133401+0000","last_peered":"2024-05-09T05:38:12.133401+0000","last_clean":"2024-05-09T05:38:12.133401+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:38:12.133401+0000","last_undegraded":"2024-05-09T05:38:12.133401+0000","last_fullsized":"2024-05-09T05:38:12.133401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051332+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:38:12.051332+0000","last_peered":"2024-05-09T05:38:12.051332+0000","last_clean":"2024-05-09T05:38:12.051332+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:38:12.051332+0000","last_undegraded":"2024-05-09T05:38:12.051332+0000","last_fullsized":"2024-05-09T05:38:12.051332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925938+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:38:11.925938+0000","last_peered":"2024-05-09T05:38:11.925938+0000","last_clean":"2024-05-09T05:38:11.925938+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:38:11.925938+0000","last_undegraded":"2024-05-09T05:38:11.925938+0000","last_fullsized":"2024-05-09T05:38:11.925938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132223+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:38:12.132223+0000","last_peered":"2024-05-09T05:38:12.132223+0000","last_clean":"2024-05-09T05:38:12.132223+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:38:12.132223+0000","last_undegraded":"2024-05-09T05:38:12.132223+0000","last_fullsized":"2024-05-09T05:38:12.132223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051904+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:38:12.051904+0000","last_peered":"2024-05-09T05:38:12.051904+0000","last_clean":"2024-05-09T05:38:12.051904+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:38:12.051904+0000","last_undegraded":"2024-05-09T05:38:12.051904+0000","last_fullsized":"2024-05-09T05:38:12.051904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134129+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:38:12.134129+0000","last_peered":"2024-05-09T05:38:12.134129+0000","last_clean":"2024-05-09T05:38:12.134129+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:38:12.134129+0000","last_undegraded":"2024-05-09T05:38:12.134129+0000","last_fullsized":"2024-05-09T05:38:12.134129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.926126+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:38:11.926126+0000","last_peered":"2024-05-09T05:38:11.926126+0000","last_clean":"2024-05-09T05:38:11.926126+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:38:11.926126+0000","last_undegraded":"2024-05-09T05:38:11.926126+0000","last_fullsized":"2024-05-09T05:38:11.926126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133832+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:38:12.133832+0000","last_peered":"2024-05-09T05:38:12.133832+0000","last_clean":"2024-05-09T05:38:12.133832+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:38:12.133832+0000","last_undegraded":"2024-05-09T05:38:12.133832+0000","last_fullsized":"2024-05-09T05:38:12.133832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133357+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:38:12.133357+0000","last_peered":"2024-05-09T05:38:12.133357+0000","last_clean":"2024-05-09T05:38:12.133357+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:38:12.133357+0000","last_undegraded":"2024-05-09T05:38:12.133357+0000","last_fullsized":"2024-05-09T05:38:12.133357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225339+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:38:12.225339+0000","last_peered":"2024-05-09T05:38:12.225339+0000","last_clean":"2024-05-09T05:38:12.225339+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:38:12.225339+0000","last_undegraded":"2024-05-09T05:38:12.225339+0000","last_fullsized":"2024-05-09T05:38:12.225339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979195+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:38:11.979195+0000","last_peered":"2024-05-09T05:38:11.979195+0000","last_clean":"2024-05-09T05:38:11.979195+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:38:11.979195+0000","last_undegraded":"2024-05-09T05:38:11.979195+0000","last_fullsized":"2024-05-09T05:38:11.979195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979140+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:38:11.979140+0000","last_peered":"2024-05-09T05:38:11.979140+0000","last_clean":"2024-05-09T05:38:11.979140+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:38:11.979140+0000","last_undegraded":"2024-05-09T05:38:11.979140+0000","last_fullsized":"2024-05-09T05:38:11.979140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132049+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:38:12.132049+0000","last_peered":"2024-05-09T05:38:12.132049+0000","last_clean":"2024-05-09T05:38:12.132049+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:38:12.132049+0000","last_undegraded":"2024-05-09T05:38:12.132049+0000","last_fullsized":"2024-05-09T05:38:12.132049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925512+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:38:11.925512+0000","last_peered":"2024-05-09T05:38:11.925512+0000","last_clean":"2024-05-09T05:38:11.925512+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:38:11.925512+0000","last_undegraded":"2024-05-09T05:38:11.925512+0000","last_fullsized":"2024-05-09T05:38:11.925512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924748+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:38:11.924748+0000","last_peered":"2024-05-09T05:38:11.924748+0000","last_clean":"2024-05-09T05:38:11.924748+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:38:11.924748+0000","last_undegraded":"2024-05-09T05:38:11.924748+0000","last_fullsized":"2024-05-09T05:38:11.924748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132236+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:38:12.132236+0000","last_peered":"2024-05-09T05:38:12.132236+0000","last_clean":"2024-05-09T05:38:12.132236+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:38:12.132236+0000","last_undegraded":"2024-05-09T05:38:12.132236+0000","last_fullsized":"2024-05-09T05:38:12.132236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131837+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:38:12.131837+0000","last_peered":"2024-05-09T05:38:12.131837+0000","last_clean":"2024-05-09T05:38:12.131837+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:38:12.131837+0000","last_undegraded":"2024-05-09T05:38:12.131837+0000","last_fullsized":"2024-05-09T05:38:12.131837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":249,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924836+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:38:11.924836+0000","last_peered":"2024-05-09T05:38:11.924836+0000","last_clean":"2024-05-09T05:38:11.924836+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:38:11.924836+0000","last_undegraded":"2024-05-09T05:38:11.924836+0000","last_fullsized":"2024-05-09T05:38:11.924836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053507+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:38:12.053507+0000","last_peered":"2024-05-09T05:38:12.053507+0000","last_clean":"2024-05-09T05:38:12.053507+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:38:12.053507+0000","last_undegraded":"2024-05-09T05:38:12.053507+0000","last_fullsized":"2024-05-09T05:38:12.053507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132572+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:38:12.132572+0000","last_peered":"2024-05-09T05:38:12.132572+0000","last_clean":"2024-05-09T05:38:12.132572+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:38:12.132572+0000","last_undegraded":"2024-05-09T05:38:12.132572+0000","last_fullsized":"2024-05-09T05:38:12.132572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053764+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:38:12.053764+0000","last_peered":"2024-05-09T05:38:12.053764+0000","last_clean":"2024-05-09T05:38:12.053764+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:38:12.053764+0000","last_undegraded":"2024-05-09T05:38:12.053764+0000","last_fullsized":"2024-05-09T05:38:12.053764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979670+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:38:11.979670+0000","last_peered":"2024-05-09T05:38:11.979670+0000","last_clean":"2024-05-09T05:38:11.979670+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:38:11.979670+0000","last_undegraded":"2024-05-09T05:38:11.979670+0000","last_fullsized":"2024-05-09T05:38:11.979670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132963+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:38:12.132963+0000","last_peered":"2024-05-09T05:38:12.132963+0000","last_clean":"2024-05-09T05:38:12.132963+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:38:12.132963+0000","last_undegraded":"2024-05-09T05:38:12.132963+0000","last_fullsized":"2024-05-09T05:38:12.132963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051620+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:38:12.051620+0000","last_peered":"2024-05-09T05:38:12.051620+0000","last_clean":"2024-05-09T05:38:12.051620+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:38:12.051620+0000","last_undegraded":"2024-05-09T05:38:12.051620+0000","last_fullsized":"2024-05-09T05:38:12.051620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133077+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:38:12.133077+0000","last_peered":"2024-05-09T05:38:12.133077+0000","last_clean":"2024-05-09T05:38:12.133077+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:38:12.133077+0000","last_undegraded":"2024-05-09T05:38:12.133077+0000","last_fullsized":"2024-05-09T05:38:12.133077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132705+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:38:12.132705+0000","last_peered":"2024-05-09T05:38:12.132705+0000","last_clean":"2024-05-09T05:38:12.132705+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:38:12.132705+0000","last_undegraded":"2024-05-09T05:38:12.132705+0000","last_fullsized":"2024-05-09T05:38:12.132705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051663+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:38:12.051663+0000","last_peered":"2024-05-09T05:38:12.051663+0000","last_clean":"2024-05-09T05:38:12.051663+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:38:12.051663+0000","last_undegraded":"2024-05-09T05:38:12.051663+0000","last_fullsized":"2024-05-09T05:38:12.051663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059550+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:38:12.059550+0000","last_peered":"2024-05-09T05:38:12.059550+0000","last_clean":"2024-05-09T05:38:12.059550+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:38:12.059550+0000","last_undegraded":"2024-05-09T05:38:12.059550+0000","last_fullsized":"2024-05-09T05:38:12.059550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131984+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:38:12.131984+0000","last_peered":"2024-05-09T05:38:12.131984+0000","last_clean":"2024-05-09T05:38:12.131984+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:38:12.131984+0000","last_undegraded":"2024-05-09T05:38:12.131984+0000","last_fullsized":"2024-05-09T05:38:12.131984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":245,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059421+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:38:12.059421+0000","last_peered":"2024-05-09T05:38:12.059421+0000","last_clean":"2024-05-09T05:38:12.059421+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:38:12.059421+0000","last_undegraded":"2024-05-09T05:38:12.059421+0000","last_fullsized":"2024-05-09T05:38:12.059421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051377+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:38:12.051377+0000","last_peered":"2024-05-09T05:38:12.051377+0000","last_clean":"2024-05-09T05:38:12.051377+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:38:12.051377+0000","last_undegraded":"2024-05-09T05:38:12.051377+0000","last_fullsized":"2024-05-09T05:38:12.051377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132066+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:38:12.132066+0000","last_peered":"2024-05-09T05:38:12.132066+0000","last_clean":"2024-05-09T05:38:12.132066+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:38:12.132066+0000","last_undegraded":"2024-05-09T05:38:12.132066+0000","last_fullsized":"2024-05-09T05:38:12.132066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925853+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:38:11.925853+0000","last_peered":"2024-05-09T05:38:11.925853+0000","last_clean":"2024-05-09T05:38:11.925853+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:38:11.925853+0000","last_undegraded":"2024-05-09T05:38:11.925853+0000","last_fullsized":"2024-05-09T05:38:11.925853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059515+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:38:12.059515+0000","last_peered":"2024-05-09T05:38:12.059515+0000","last_clean":"2024-05-09T05:38:12.059515+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:38:12.059515+0000","last_undegraded":"2024-05-09T05:38:12.059515+0000","last_fullsized":"2024-05-09T05:38:12.059515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925545+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:38:11.925545+0000","last_peered":"2024-05-09T05:38:11.925545+0000","last_clean":"2024-05-09T05:38:11.925545+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:38:11.925545+0000","last_undegraded":"2024-05-09T05:38:11.925545+0000","last_fullsized":"2024-05-09T05:38:11.925545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224943+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:38:12.224943+0000","last_peered":"2024-05-09T05:38:12.224943+0000","last_clean":"2024-05-09T05:38:12.224943+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:38:12.224943+0000","last_undegraded":"2024-05-09T05:38:12.224943+0000","last_fullsized":"2024-05-09T05:38:12.224943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052036+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:38:12.052036+0000","last_peered":"2024-05-09T05:38:12.052036+0000","last_clean":"2024-05-09T05:38:12.052036+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:38:12.052036+0000","last_undegraded":"2024-05-09T05:38:12.052036+0000","last_fullsized":"2024-05-09T05:38:12.052036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979576+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:38:11.979576+0000","last_peered":"2024-05-09T05:38:11.979576+0000","last_clean":"2024-05-09T05:38:11.979576+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:38:11.979576+0000","last_undegraded":"2024-05-09T05:38:11.979576+0000","last_fullsized":"2024-05-09T05:38:11.979576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059004+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:38:12.059004+0000","last_peered":"2024-05-09T05:38:12.059004+0000","last_clean":"2024-05-09T05:38:12.059004+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:38:12.059004+0000","last_undegraded":"2024-05-09T05:38:12.059004+0000","last_fullsized":"2024-05-09T05:38:12.059004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924546+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:38:11.924546+0000","last_peered":"2024-05-09T05:38:11.924546+0000","last_clean":"2024-05-09T05:38:11.924546+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:38:11.924546+0000","last_undegraded":"2024-05-09T05:38:11.924546+0000","last_fullsized":"2024-05-09T05:38:11.924546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053026+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:38:12.053026+0000","last_peered":"2024-05-09T05:38:12.053026+0000","last_clean":"2024-05-09T05:38:12.053026+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:38:12.053026+0000","last_undegraded":"2024-05-09T05:38:12.053026+0000","last_fullsized":"2024-05-09T05:38:12.053026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'7","reported_seq":429,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.592382+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:38:12.592382+0000","last_peered":"2024-05-09T05:38:12.592382+0000","last_clean":"2024-05-09T05:38:12.592382+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:38:12.592382+0000","last_undegraded":"2024-05-09T05:38:12.592382+0000","last_fullsized":"2024-05-09T05:38:12.592382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":380,"num_read_kb":380,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133858+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:38:12.133858+0000","last_peered":"2024-05-09T05:38:12.133858+0000","last_clean":"2024-05-09T05:38:12.133858+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:38:12.133858+0000","last_undegraded":"2024-05-09T05:38:12.133858+0000","last_fullsized":"2024-05-09T05:38:12.133858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924463+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:38:11.924463+0000","last_peered":"2024-05-09T05:38:11.924463+0000","last_clean":"2024-05-09T05:38:11.924463+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:38:11.924463+0000","last_undegraded":"2024-05-09T05:38:11.924463+0000","last_fullsized":"2024-05-09T05:38:11.924463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133086+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:38:12.133086+0000","last_peered":"2024-05-09T05:38:12.133086+0000","last_clean":"2024-05-09T05:38:12.133086+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:38:12.133086+0000","last_undegraded":"2024-05-09T05:38:12.133086+0000","last_fullsized":"2024-05-09T05:38:12.133086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979599+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:38:11.979599+0000","last_peered":"2024-05-09T05:38:11.979599+0000","last_clean":"2024-05-09T05:38:11.979599+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:38:11.979599+0000","last_undegraded":"2024-05-09T05:38:11.979599+0000","last_fullsized":"2024-05-09T05:38:11.979599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133847+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:38:12.133847+0000","last_peered":"2024-05-09T05:38:12.133847+0000","last_clean":"2024-05-09T05:38:12.133847+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:38:12.133847+0000","last_undegraded":"2024-05-09T05:38:12.133847+0000","last_fullsized":"2024-05-09T05:38:12.133847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059520+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:38:12.059520+0000","last_peered":"2024-05-09T05:38:12.059520+0000","last_clean":"2024-05-09T05:38:12.059520+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:38:12.059520+0000","last_undegraded":"2024-05-09T05:38:12.059520+0000","last_fullsized":"2024-05-09T05:38:12.059520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133129+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:38:12.133129+0000","last_peered":"2024-05-09T05:38:12.133129+0000","last_clean":"2024-05-09T05:38:12.133129+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:38:12.133129+0000","last_undegraded":"2024-05-09T05:38:12.133129+0000","last_fullsized":"2024-05-09T05:38:12.133129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924514+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:38:11.924514+0000","last_peered":"2024-05-09T05:38:11.924514+0000","last_clean":"2024-05-09T05:38:11.924514+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:38:11.924514+0000","last_undegraded":"2024-05-09T05:38:11.924514+0000","last_fullsized":"2024-05-09T05:38:11.924514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'43","reported_seq":477,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.596639+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:38:12.596639+0000","last_peered":"2024-05-09T05:38:12.596639+0000","last_clean":"2024-05-09T05:38:12.596639+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:38:12.596639+0000","last_undegraded":"2024-05-09T05:38:12.596639+0000","last_fullsized":"2024-05-09T05:38:12.596639+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69974,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":392,"num_read_kb":264,"num_write":45,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051722+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:38:12.051722+0000","last_peered":"2024-05-09T05:38:12.051722+0000","last_clean":"2024-05-09T05:38:12.051722+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:38:12.051722+0000","last_undegraded":"2024-05-09T05:38:12.051722+0000","last_fullsized":"2024-05-09T05:38:12.051722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059857+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:38:12.059857+0000","last_peered":"2024-05-09T05:38:12.059857+0000","last_clean":"2024-05-09T05:38:12.059857+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:38:12.059857+0000","last_undegraded":"2024-05-09T05:38:12.059857+0000","last_fullsized":"2024-05-09T05:38:12.059857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132158+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:38:12.132158+0000","last_peered":"2024-05-09T05:38:12.132158+0000","last_clean":"2024-05-09T05:38:12.132158+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:38:12.132158+0000","last_undegraded":"2024-05-09T05:38:12.132158+0000","last_fullsized":"2024-05-09T05:38:12.132158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225173+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:38:12.225173+0000","last_peered":"2024-05-09T05:38:12.225173+0000","last_clean":"2024-05-09T05:38:12.225173+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:38:12.225173+0000","last_undegraded":"2024-05-09T05:38:12.225173+0000","last_fullsized":"2024-05-09T05:38:12.225173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059308+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:38:12.059308+0000","last_peered":"2024-05-09T05:38:12.059308+0000","last_clean":"2024-05-09T05:38:12.059308+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:38:12.059308+0000","last_undegraded":"2024-05-09T05:38:12.059308+0000","last_fullsized":"2024-05-09T05:38:12.059308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132718+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:38:12.132718+0000","last_peered":"2024-05-09T05:38:12.132718+0000","last_clean":"2024-05-09T05:38:12.132718+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:38:12.132718+0000","last_undegraded":"2024-05-09T05:38:12.132718+0000","last_fullsized":"2024-05-09T05:38:12.132718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051717+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:38:12.051717+0000","last_peered":"2024-05-09T05:38:12.051717+0000","last_clean":"2024-05-09T05:38:12.051717+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:38:12.051717+0000","last_undegraded":"2024-05-09T05:38:12.051717+0000","last_fullsized":"2024-05-09T05:38:12.051717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.566344+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:38:12.566344+0000","last_peered":"2024-05-09T05:38:12.566344+0000","last_clean":"2024-05-09T05:38:12.566344+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:38:12.566344+0000","last_undegraded":"2024-05-09T05:38:12.566344+0000","last_fullsized":"2024-05-09T05:38:12.566344+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132335+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:38:12.132335+0000","last_peered":"2024-05-09T05:38:12.132335+0000","last_clean":"2024-05-09T05:38:12.132335+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:38:12.132335+0000","last_undegraded":"2024-05-09T05:38:12.132335+0000","last_fullsized":"2024-05-09T05:38:12.132335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225537+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:38:12.225537+0000","last_peered":"2024-05-09T05:38:12.225537+0000","last_clean":"2024-05-09T05:38:12.225537+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:38:12.225537+0000","last_undegraded":"2024-05-09T05:38:12.225537+0000","last_fullsized":"2024-05-09T05:38:12.225537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051224+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:38:12.051224+0000","last_peered":"2024-05-09T05:38:12.051224+0000","last_clean":"2024-05-09T05:38:12.051224+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:38:12.051224+0000","last_undegraded":"2024-05-09T05:38:12.051224+0000","last_fullsized":"2024-05-09T05:38:12.051224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924664+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:38:11.924664+0000","last_peered":"2024-05-09T05:38:11.924664+0000","last_clean":"2024-05-09T05:38:11.924664+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:38:11.924664+0000","last_undegraded":"2024-05-09T05:38:11.924664+0000","last_fullsized":"2024-05-09T05:38:11.924664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059325+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:38:12.059325+0000","last_peered":"2024-05-09T05:38:12.059325+0000","last_clean":"2024-05-09T05:38:12.059325+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:38:12.059325+0000","last_undegraded":"2024-05-09T05:38:12.059325+0000","last_fullsized":"2024-05-09T05:38:12.059325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051882+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:38:12.051882+0000","last_peered":"2024-05-09T05:38:12.051882+0000","last_clean":"2024-05-09T05:38:12.051882+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:38:12.051882+0000","last_undegraded":"2024-05-09T05:38:12.051882+0000","last_fullsized":"2024-05-09T05:38:12.051882+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925361+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:38:11.925361+0000","last_peered":"2024-05-09T05:38:11.925361+0000","last_clean":"2024-05-09T05:38:11.925361+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:38:11.925361+0000","last_undegraded":"2024-05-09T05:38:11.925361+0000","last_fullsized":"2024-05-09T05:38:11.925361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051281+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:38:12.051281+0000","last_peered":"2024-05-09T05:38:12.051281+0000","last_clean":"2024-05-09T05:38:12.051281+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:38:12.051281+0000","last_undegraded":"2024-05-09T05:38:12.051281+0000","last_fullsized":"2024-05-09T05:38:12.051281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925850+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:38:11.925850+0000","last_peered":"2024-05-09T05:38:11.925850+0000","last_clean":"2024-05-09T05:38:11.925850+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:38:11.925850+0000","last_undegraded":"2024-05-09T05:38:11.925850+0000","last_fullsized":"2024-05-09T05:38:11.925850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225102+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:38:12.225102+0000","last_peered":"2024-05-09T05:38:12.225102+0000","last_clean":"2024-05-09T05:38:12.225102+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:38:12.225102+0000","last_undegraded":"2024-05-09T05:38:12.225102+0000","last_fullsized":"2024-05-09T05:38:12.225102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'7","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.565968+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:38:12.565968+0000","last_peered":"2024-05-09T05:38:12.565968+0000","last_clean":"2024-05-09T05:38:12.565968+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:38:12.565968+0000","last_undegraded":"2024-05-09T05:38:12.565968+0000","last_fullsized":"2024-05-09T05:38:12.565968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925837+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:38:11.925837+0000","last_peered":"2024-05-09T05:38:11.925837+0000","last_clean":"2024-05-09T05:38:11.925837+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:38:11.925837+0000","last_undegraded":"2024-05-09T05:38:11.925837+0000","last_fullsized":"2024-05-09T05:38:11.925837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225665+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:38:12.225665+0000","last_peered":"2024-05-09T05:38:12.225665+0000","last_clean":"2024-05-09T05:38:12.225665+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:38:12.225665+0000","last_undegraded":"2024-05-09T05:38:12.225665+0000","last_fullsized":"2024-05-09T05:38:12.225665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059336+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:38:12.059336+0000","last_peered":"2024-05-09T05:38:12.059336+0000","last_clean":"2024-05-09T05:38:12.059336+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:38:12.059336+0000","last_undegraded":"2024-05-09T05:38:12.059336+0000","last_fullsized":"2024-05-09T05:38:12.059336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":240,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224856+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:38:12.224856+0000","last_peered":"2024-05-09T05:38:12.224856+0000","last_clean":"2024-05-09T05:38:12.224856+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:38:12.224856+0000","last_undegraded":"2024-05-09T05:38:12.224856+0000","last_fullsized":"2024-05-09T05:38:12.224856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133630+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:38:12.133630+0000","last_peered":"2024-05-09T05:38:12.133630+0000","last_clean":"2024-05-09T05:38:12.133630+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:38:12.133630+0000","last_undegraded":"2024-05-09T05:38:12.133630+0000","last_fullsized":"2024-05-09T05:38:12.133630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052441+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:38:12.052441+0000","last_peered":"2024-05-09T05:38:12.052441+0000","last_clean":"2024-05-09T05:38:12.052441+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:38:12.052441+0000","last_undegraded":"2024-05-09T05:38:12.052441+0000","last_fullsized":"2024-05-09T05:38:12.052441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133733+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:38:12.133733+0000","last_peered":"2024-05-09T05:38:12.133733+0000","last_clean":"2024-05-09T05:38:12.133733+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:38:12.133733+0000","last_undegraded":"2024-05-09T05:38:12.133733+0000","last_fullsized":"2024-05-09T05:38:12.133733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132080+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:38:12.132080+0000","last_peered":"2024-05-09T05:38:12.132080+0000","last_clean":"2024-05-09T05:38:12.132080+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:38:12.132080+0000","last_undegraded":"2024-05-09T05:38:12.132080+0000","last_fullsized":"2024-05-09T05:38:12.132080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059542+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:38:12.059542+0000","last_peered":"2024-05-09T05:38:12.059542+0000","last_clean":"2024-05-09T05:38:12.059542+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:38:12.059542+0000","last_undegraded":"2024-05-09T05:38:12.059542+0000","last_fullsized":"2024-05-09T05:38:12.059542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224704+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:38:12.224704+0000","last_peered":"2024-05-09T05:38:12.224704+0000","last_clean":"2024-05-09T05:38:12.224704+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:38:12.224704+0000","last_undegraded":"2024-05-09T05:38:12.224704+0000","last_fullsized":"2024-05-09T05:38:12.224704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132204+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:38:12.132204+0000","last_peered":"2024-05-09T05:38:12.132204+0000","last_clean":"2024-05-09T05:38:12.132204+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:38:12.132204+0000","last_undegraded":"2024-05-09T05:38:12.132204+0000","last_fullsized":"2024-05-09T05:38:12.132204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925084+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:38:11.925084+0000","last_peered":"2024-05-09T05:38:11.925084+0000","last_clean":"2024-05-09T05:38:11.925084+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:38:11.925084+0000","last_undegraded":"2024-05-09T05:38:11.925084+0000","last_fullsized":"2024-05-09T05:38:11.925084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925336+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:38:11.925336+0000","last_peered":"2024-05-09T05:38:11.925336+0000","last_clean":"2024-05-09T05:38:11.925336+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:38:11.925336+0000","last_undegraded":"2024-05-09T05:38:11.925336+0000","last_fullsized":"2024-05-09T05:38:11.925336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225403+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:38:12.225403+0000","last_peered":"2024-05-09T05:38:12.225403+0000","last_clean":"2024-05-09T05:38:12.225403+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:38:12.225403+0000","last_undegraded":"2024-05-09T05:38:12.225403+0000","last_fullsized":"2024-05-09T05:38:12.225403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051969+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:38:12.051969+0000","last_peered":"2024-05-09T05:38:12.051969+0000","last_clean":"2024-05-09T05:38:12.051969+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:38:12.051969+0000","last_undegraded":"2024-05-09T05:38:12.051969+0000","last_fullsized":"2024-05-09T05:38:12.051969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'54","reported_seq":121,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.817878+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:38:12.817878+0000","last_peered":"2024-05-09T05:38:12.817878+0000","last_clean":"2024-05-09T05:38:12.817878+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:38:12.817878+0000","last_undegraded":"2024-05-09T05:38:12.817878+0000","last_fullsized":"2024-05-09T05:38:12.817878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":96,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053724+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:38:12.053724+0000","last_peered":"2024-05-09T05:38:12.053724+0000","last_clean":"2024-05-09T05:38:12.053724+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:38:12.053724+0000","last_undegraded":"2024-05-09T05:38:12.053724+0000","last_fullsized":"2024-05-09T05:38:12.053724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131738+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:38:12.131738+0000","last_peered":"2024-05-09T05:38:12.131738+0000","last_clean":"2024-05-09T05:38:12.131738+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:38:12.131738+0000","last_undegraded":"2024-05-09T05:38:12.131738+0000","last_fullsized":"2024-05-09T05:38:12.131738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134405+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:38:12.134405+0000","last_peered":"2024-05-09T05:38:12.134405+0000","last_clean":"2024-05-09T05:38:12.134405+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:38:12.134405+0000","last_undegraded":"2024-05-09T05:38:12.134405+0000","last_fullsized":"2024-05-09T05:38:12.134405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979676+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:38:11.979676+0000","last_peered":"2024-05-09T05:38:11.979676+0000","last_clean":"2024-05-09T05:38:11.979676+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:38:11.979676+0000","last_undegraded":"2024-05-09T05:38:11.979676+0000","last_fullsized":"2024-05-09T05:38:11.979676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133578+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:38:12.133578+0000","last_peered":"2024-05-09T05:38:12.133578+0000","last_clean":"2024-05-09T05:38:12.133578+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:38:12.133578+0000","last_undegraded":"2024-05-09T05:38:12.133578+0000","last_fullsized":"2024-05-09T05:38:12.133578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132717+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:38:12.132717+0000","last_peered":"2024-05-09T05:38:12.132717+0000","last_clean":"2024-05-09T05:38:12.132717+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:38:12.132717+0000","last_undegraded":"2024-05-09T05:38:12.132717+0000","last_fullsized":"2024-05-09T05:38:12.132717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051234+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:38:12.051234+0000","last_peered":"2024-05-09T05:38:12.051234+0000","last_clean":"2024-05-09T05:38:12.051234+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:38:12.051234+0000","last_undegraded":"2024-05-09T05:38:12.051234+0000","last_fullsized":"2024-05-09T05:38:12.051234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924777+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:38:11.924777+0000","last_peered":"2024-05-09T05:38:11.924777+0000","last_clean":"2024-05-09T05:38:11.924777+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:38:11.924777+0000","last_undegraded":"2024-05-09T05:38:11.924777+0000","last_fullsized":"2024-05-09T05:38:11.924777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053182+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:38:12.053182+0000","last_peered":"2024-05-09T05:38:12.053182+0000","last_clean":"2024-05-09T05:38:12.053182+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:38:12.053182+0000","last_undegraded":"2024-05-09T05:38:12.053182+0000","last_fullsized":"2024-05-09T05:38:12.053182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924921+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:38:11.924921+0000","last_peered":"2024-05-09T05:38:11.924921+0000","last_clean":"2024-05-09T05:38:11.924921+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:38:11.924921+0000","last_undegraded":"2024-05-09T05:38:11.924921+0000","last_fullsized":"2024-05-09T05:38:11.924921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133063+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:38:12.133063+0000","last_peered":"2024-05-09T05:38:12.133063+0000","last_clean":"2024-05-09T05:38:12.133063+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:38:12.133063+0000","last_undegraded":"2024-05-09T05:38:12.133063+0000","last_fullsized":"2024-05-09T05:38:12.133063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.315433+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:38:12.315433+0000","last_peered":"2024-05-09T05:38:12.315433+0000","last_clean":"2024-05-09T05:38:12.315433+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:38:12.315433+0000","last_undegraded":"2024-05-09T05:38:12.315433+0000","last_fullsized":"2024-05-09T05:38:12.315433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053037+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:38:12.053037+0000","last_peered":"2024-05-09T05:38:12.053037+0000","last_clean":"2024-05-09T05:38:12.053037+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:38:12.053037+0000","last_undegraded":"2024-05-09T05:38:12.053037+0000","last_fullsized":"2024-05-09T05:38:12.053037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133542+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:38:12.133542+0000","last_peered":"2024-05-09T05:38:12.133542+0000","last_clean":"2024-05-09T05:38:12.133542+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:38:12.133542+0000","last_undegraded":"2024-05-09T05:38:12.133542+0000","last_fullsized":"2024-05-09T05:38:12.133542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051361+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:38:12.051361+0000","last_peered":"2024-05-09T05:38:12.051361+0000","last_clean":"2024-05-09T05:38:12.051361+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:38:12.051361+0000","last_undegraded":"2024-05-09T05:38:12.051361+0000","last_fullsized":"2024-05-09T05:38:12.051361+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132494+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:38:12.132494+0000","last_peered":"2024-05-09T05:38:12.132494+0000","last_clean":"2024-05-09T05:38:12.132494+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:38:12.132494+0000","last_undegraded":"2024-05-09T05:38:12.132494+0000","last_fullsized":"2024-05-09T05:38:12.132494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133025+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:38:12.133025+0000","last_peered":"2024-05-09T05:38:12.133025+0000","last_clean":"2024-05-09T05:38:12.133025+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:38:12.133025+0000","last_undegraded":"2024-05-09T05:38:12.133025+0000","last_fullsized":"2024-05-09T05:38:12.133025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":243,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.627036+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:38:12.627036+0000","last_peered":"2024-05-09T05:38:12.627036+0000","last_clean":"2024-05-09T05:38:12.627036+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:38:12.627036+0000","last_undegraded":"2024-05-09T05:38:12.627036+0000","last_fullsized":"2024-05-09T05:38:12.627036+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133932+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:38:12.133932+0000","last_peered":"2024-05-09T05:38:12.133932+0000","last_clean":"2024-05-09T05:38:12.133932+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:38:12.133932+0000","last_undegraded":"2024-05-09T05:38:12.133932+0000","last_fullsized":"2024-05-09T05:38:12.133932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979193+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:38:11.979193+0000","last_peered":"2024-05-09T05:38:11.979193+0000","last_clean":"2024-05-09T05:38:11.979193+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:38:11.979193+0000","last_undegraded":"2024-05-09T05:38:11.979193+0000","last_fullsized":"2024-05-09T05:38:11.979193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.175546+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:38:12.175546+0000","last_peered":"2024-05-09T05:38:12.175546+0000","last_clean":"2024-05-09T05:38:12.175546+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:38:12.175546+0000","last_undegraded":"2024-05-09T05:38:12.175546+0000","last_fullsized":"2024-05-09T05:38:12.175546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132613+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:38:12.132613+0000","last_peered":"2024-05-09T05:38:12.132613+0000","last_clean":"2024-05-09T05:38:12.132613+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:38:12.132613+0000","last_undegraded":"2024-05-09T05:38:12.132613+0000","last_fullsized":"2024-05-09T05:38:12.132613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925491+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:38:11.925491+0000","last_peered":"2024-05-09T05:38:11.925491+0000","last_clean":"2024-05-09T05:38:11.925491+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:38:11.925491+0000","last_undegraded":"2024-05-09T05:38:11.925491+0000","last_fullsized":"2024-05-09T05:38:11.925491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225540+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:38:12.225540+0000","last_peered":"2024-05-09T05:38:12.225540+0000","last_clean":"2024-05-09T05:38:12.225540+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:38:12.225540+0000","last_undegraded":"2024-05-09T05:38:12.225540+0000","last_fullsized":"2024-05-09T05:38:12.225540+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133446+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:38:12.133446+0000","last_peered":"2024-05-09T05:38:12.133446+0000","last_clean":"2024-05-09T05:38:12.133446+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:38:12.133446+0000","last_undegraded":"2024-05-09T05:38:12.133446+0000","last_fullsized":"2024-05-09T05:38:12.133446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925076+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:38:11.925076+0000","last_peered":"2024-05-09T05:38:11.925076+0000","last_clean":"2024-05-09T05:38:11.925076+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:38:11.925076+0000","last_undegraded":"2024-05-09T05:38:11.925076+0000","last_fullsized":"2024-05-09T05:38:11.925076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132222+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:38:12.132222+0000","last_peered":"2024-05-09T05:38:12.132222+0000","last_clean":"2024-05-09T05:38:12.132222+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:38:12.132222+0000","last_undegraded":"2024-05-09T05:38:12.132222+0000","last_fullsized":"2024-05-09T05:38:12.132222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131810+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:38:12.131810+0000","last_peered":"2024-05-09T05:38:12.131810+0000","last_clean":"2024-05-09T05:38:12.131810+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:38:12.131810+0000","last_undegraded":"2024-05-09T05:38:12.131810+0000","last_fullsized":"2024-05-09T05:38:12.131810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979502+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:38:11.979502+0000","last_peered":"2024-05-09T05:38:11.979502+0000","last_clean":"2024-05-09T05:38:11.979502+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:38:11.979502+0000","last_undegraded":"2024-05-09T05:38:11.979502+0000","last_fullsized":"2024-05-09T05:38:11.979502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225302+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:38:12.225302+0000","last_peered":"2024-05-09T05:38:12.225302+0000","last_clean":"2024-05-09T05:38:12.225302+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:38:12.225302+0000","last_undegraded":"2024-05-09T05:38:12.225302+0000","last_fullsized":"2024-05-09T05:38:12.225302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925541+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:38:11.925541+0000","last_peered":"2024-05-09T05:38:11.925541+0000","last_clean":"2024-05-09T05:38:11.925541+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:38:11.925541+0000","last_undegraded":"2024-05-09T05:38:11.925541+0000","last_fullsized":"2024-05-09T05:38:11.925541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132345+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:38:12.132345+0000","last_peered":"2024-05-09T05:38:12.132345+0000","last_clean":"2024-05-09T05:38:12.132345+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:38:12.132345+0000","last_undegraded":"2024-05-09T05:38:12.132345+0000","last_fullsized":"2024-05-09T05:38:12.132345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053867+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:38:12.053867+0000","last_peered":"2024-05-09T05:38:12.053867+0000","last_clean":"2024-05-09T05:38:12.053867+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:38:12.053867+0000","last_undegraded":"2024-05-09T05:38:12.053867+0000","last_fullsized":"2024-05-09T05:38:12.053867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059087+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:38:12.059087+0000","last_peered":"2024-05-09T05:38:12.059087+0000","last_clean":"2024-05-09T05:38:12.059087+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:38:12.059087+0000","last_undegraded":"2024-05-09T05:38:12.059087+0000","last_fullsized":"2024-05-09T05:38:12.059087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053259+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:38:12.053259+0000","last_peered":"2024-05-09T05:38:12.053259+0000","last_clean":"2024-05-09T05:38:12.053259+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:38:12.053259+0000","last_undegraded":"2024-05-09T05:38:12.053259+0000","last_fullsized":"2024-05-09T05:38:12.053259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133378+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:38:12.133378+0000","last_peered":"2024-05-09T05:38:12.133378+0000","last_clean":"2024-05-09T05:38:12.133378+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:38:12.133378+0000","last_undegraded":"2024-05-09T05:38:12.133378+0000","last_fullsized":"2024-05-09T05:38:12.133378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224935+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:38:12.224935+0000","last_peered":"2024-05-09T05:38:12.224935+0000","last_clean":"2024-05-09T05:38:12.224935+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:38:12.224935+0000","last_undegraded":"2024-05-09T05:38:12.224935+0000","last_fullsized":"2024-05-09T05:38:12.224935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925262+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:38:11.925262+0000","last_peered":"2024-05-09T05:38:11.925262+0000","last_clean":"2024-05-09T05:38:11.925262+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:38:11.925262+0000","last_undegraded":"2024-05-09T05:38:11.925262+0000","last_fullsized":"2024-05-09T05:38:11.925262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107766,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604889,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2043,"num_read_kb":1142,"num_write":394,"num_write_kb":1480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1814667,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710786,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507638,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907016,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1138688,"data_stored":1224885,"data_compressed":68521,"data_compressed_allocated":90112,"data_compressed_original":733184,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070542,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":903884,"data_compressed":48658,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266771,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":900054,"data_compressed":48658,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463000,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":1105849,"data_compressed":65434,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626525,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":492761,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":573964,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":504416,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":575209,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1026600,"data_compressed":54814,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986278,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507034,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822755,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":1231263,"data_compressed":71590,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135445,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":204194,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:38:17.018 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T05:38:17.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.172+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/4063267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54109920 msgr2=0x7f3b5410be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.172+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/4063267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54109920 0x7f3b5410be30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009ca0 tx=0x7f3b4c02a8c0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.172+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/4063267 shutdown_connections 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.172+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/4063267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54109920 0x7f3b5410be30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.172+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/4063267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b54106db0 0x7f3b54109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.172+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/4063267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54101fd0 0x7f3b541023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.172+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/4063267 >> 172.21.15.115:0/4063267 conn(0x7f3b540fd4e0 msgr2=0x7f3b540ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.172+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/4063267 shutdown_connections 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.172+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/4063267 wait complete. 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5b6ee640 1 Processor -- start 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5b6ee640 1 -- start start 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5b6ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54101fd0 0x7f3b5419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5b6ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b54106db0 0x7f3b5419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5b6ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54109920 0x7f3b541a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:17.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5b6ee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410f660 con 0x7f3b54101fd0 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5b6ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410f4e0 con 0x7f3b54109920 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5b6ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410f7e0 con 0x7f3b54106db0 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5948a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54101fd0 0x7f3b5419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5948a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54101fd0 0x7f3b5419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34840/0 (socket says 172.21.15.115:34840) 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5948a640 1 -- 172.21.15.115:0/795109154 learned_addr learned my addr 172.21.15.115:0/795109154 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b58c89640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b54106db0 0x7f3b5419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b59c8b640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54109920 0x7f3b541a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5948a640 1 -- 172.21.15.115:0/795109154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b54106db0 msgr2=0x7f3b5419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5948a640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b54106db0 0x7f3b5419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5948a640 1 -- 172.21.15.115:0/795109154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54109920 msgr2=0x7f3b541a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5948a640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54109920 0x7f3b541a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b5948a640 1 -- 172.21.15.115:0/795109154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b541a1d40 con 0x7f3b54101fd0 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.173+0000 7f3b59c8b640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54109920 0x7f3b541a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.174+0000 7f3b5948a640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54101fd0 0x7f3b5419a200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000d740 tx=0x7f3b5000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:17.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.174+0000 7f3b4a7fc640 1 -- 172.21.15.115:0/795109154 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50017070 con 0x7f3b54101fd0 2024-05-09T05:38:17.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.174+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b541a1fd0 con 0x7f3b54101fd0 2024-05-09T05:38:17.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.174+0000 7f3b4a7fc640 1 -- 172.21.15.115:0/795109154 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b5000de50 con 0x7f3b54101fd0 2024-05-09T05:38:17.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.174+0000 7f3b4a7fc640 1 -- 172.21.15.115:0/795109154 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50004c60 con 0x7f3b54101fd0 2024-05-09T05:38:17.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.174+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b541a2490 con 0x7f3b54101fd0 2024-05-09T05:38:17.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.175+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b540ff000 con 0x7f3b54101fd0 2024-05-09T05:38:17.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.177+0000 7f3b4a7fc640 1 -- 172.21.15.115:0/795109154 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f3b50004e00 con 0x7f3b54101fd0 2024-05-09T05:38:17.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.177+0000 7f3b4a7fc640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b400b29f0 0x7f3b400b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:17.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.177+0000 7f3b4a7fc640 1 -- 172.21.15.115:0/795109154 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f3b500e2890 con 0x7f3b54101fd0 2024-05-09T05:38:17.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.178+0000 7f3b58c89640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b400b29f0 0x7f3b400b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:17.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.178+0000 7f3b58c89640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b400b29f0 0x7f3b400b4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b5419b2a0 tx=0x7f3b44005f90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:17.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.186+0000 7f3b4a7fc640 1 -- 172.21.15.115:0/795109154 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b500a9dc0 con 0x7f3b54101fd0 2024-05-09T05:38:17.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.377+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b54063110 con 0x7f3b400b29f0 2024-05-09T05:38:17.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.387+0000 7f3b4a7fc640 1 -- 172.21.15.115:0/795109154 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339703 (secure 0 0 0) 0x7f3b54063110 con 0x7f3b400b29f0 2024-05-09T05:38:17.384 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:17.385 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:38:17.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.390+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b400b29f0 msgr2=0x7f3b400b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.390+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b400b29f0 0x7f3b400b4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b5419b2a0 tx=0x7f3b44005f90 comp rx=0 tx=0).stop 2024-05-09T05:38:17.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.390+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54101fd0 msgr2=0x7f3b5419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.390+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54101fd0 0x7f3b5419a200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000d740 tx=0x7f3b5000dc00 comp rx=0 tx=0).stop 2024-05-09T05:38:17.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.391+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 shutdown_connections 2024-05-09T05:38:17.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.391+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b400b29f0 0x7f3b400b4f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.391+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b54109920 0x7f3b541a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.391+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b54106db0 0x7f3b5419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.391+0000 7f3b5b6ee640 1 --2- 172.21.15.115:0/795109154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b54101fd0 0x7f3b5419a200 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.391+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 >> 172.21.15.115:0/795109154 conn(0x7f3b540fd4e0 msgr2=0x7f3b54107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:17.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.391+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 shutdown_connections 2024-05-09T05:38:17.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.391+0000 7f3b5b6ee640 1 -- 172.21.15.115:0/795109154 wait complete. 2024-05-09T05:38:17.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:17 smithi115 ceph-mon[30349]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:17.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:17 smithi115 ceph-mon[30349]: mgrmap e23: x(active, since 5s), standbys: y, z 2024-05-09T05:38:17.396 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-09T05:38:15.942399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195491,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2124,"num_read_kb":1211,"num_write":519,"num_write_kb":3632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":909,"ondisk_log_size":909,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344656,"kb_used_data":12092,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597168,"statfs":{"total":1151940427776,"available":1151587500032,"internally_reserved":0,"allocated":12382208,"data_stored":9553557,"data_compressed":531766,"data_compressed_allocated":716800,"data_compressed_original":4231168,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987801"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925762+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:38:11.925762+0000","last_peered":"2024-05-09T05:38:11.925762+0000","last_clean":"2024-05-09T05:38:11.925762+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:38:11.925762+0000","last_undegraded":"2024-05-09T05:38:11.925762+0000","last_fullsized":"2024-05-09T05:38:11.925762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979667+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:38:11.979667+0000","last_peered":"2024-05-09T05:38:11.979667+0000","last_clean":"2024-05-09T05:38:11.979667+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:38:11.979667+0000","last_undegraded":"2024-05-09T05:38:11.979667+0000","last_fullsized":"2024-05-09T05:38:11.979667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979267+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:38:11.979267+0000","last_peered":"2024-05-09T05:38:11.979267+0000","last_clean":"2024-05-09T05:38:11.979267+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:38:11.979267+0000","last_undegraded":"2024-05-09T05:38:11.979267+0000","last_fullsized":"2024-05-09T05:38:11.979267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133401+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:38:12.133401+0000","last_peered":"2024-05-09T05:38:12.133401+0000","last_clean":"2024-05-09T05:38:12.133401+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:38:12.133401+0000","last_undegraded":"2024-05-09T05:38:12.133401+0000","last_fullsized":"2024-05-09T05:38:12.133401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051332+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:38:12.051332+0000","last_peered":"2024-05-09T05:38:12.051332+0000","last_clean":"2024-05-09T05:38:12.051332+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:38:12.051332+0000","last_undegraded":"2024-05-09T05:38:12.051332+0000","last_fullsized":"2024-05-09T05:38:12.051332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925938+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:38:11.925938+0000","last_peered":"2024-05-09T05:38:11.925938+0000","last_clean":"2024-05-09T05:38:11.925938+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:38:11.925938+0000","last_undegraded":"2024-05-09T05:38:11.925938+0000","last_fullsized":"2024-05-09T05:38:11.925938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132223+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:38:12.132223+0000","last_peered":"2024-05-09T05:38:12.132223+0000","last_clean":"2024-05-09T05:38:12.132223+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:38:12.132223+0000","last_undegraded":"2024-05-09T05:38:12.132223+0000","last_fullsized":"2024-05-09T05:38:12.132223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051904+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:38:12.051904+0000","last_peered":"2024-05-09T05:38:12.051904+0000","last_clean":"2024-05-09T05:38:12.051904+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:38:12.051904+0000","last_undegraded":"2024-05-09T05:38:12.051904+0000","last_fullsized":"2024-05-09T05:38:12.051904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134129+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:38:12.134129+0000","last_peered":"2024-05-09T05:38:12.134129+0000","last_clean":"2024-05-09T05:38:12.134129+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:38:12.134129+0000","last_undegraded":"2024-05-09T05:38:12.134129+0000","last_fullsized":"2024-05-09T05:38:12.134129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.926126+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:38:11.926126+0000","last_peered":"2024-05-09T05:38:11.926126+0000","last_clean":"2024-05-09T05:38:11.926126+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:38:11.926126+0000","last_undegraded":"2024-05-09T05:38:11.926126+0000","last_fullsized":"2024-05-09T05:38:11.926126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133832+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:38:12.133832+0000","last_peered":"2024-05-09T05:38:12.133832+0000","last_clean":"2024-05-09T05:38:12.133832+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:38:12.133832+0000","last_undegraded":"2024-05-09T05:38:12.133832+0000","last_fullsized":"2024-05-09T05:38:12.133832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133357+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:38:12.133357+0000","last_peered":"2024-05-09T05:38:12.133357+0000","last_clean":"2024-05-09T05:38:12.133357+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:38:12.133357+0000","last_undegraded":"2024-05-09T05:38:12.133357+0000","last_fullsized":"2024-05-09T05:38:12.133357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225339+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:38:12.225339+0000","last_peered":"2024-05-09T05:38:12.225339+0000","last_clean":"2024-05-09T05:38:12.225339+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:38:12.225339+0000","last_undegraded":"2024-05-09T05:38:12.225339+0000","last_fullsized":"2024-05-09T05:38:12.225339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979195+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:38:11.979195+0000","last_peered":"2024-05-09T05:38:11.979195+0000","last_clean":"2024-05-09T05:38:11.979195+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:38:11.979195+0000","last_undegraded":"2024-05-09T05:38:11.979195+0000","last_fullsized":"2024-05-09T05:38:11.979195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979140+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:38:11.979140+0000","last_peered":"2024-05-09T05:38:11.979140+0000","last_clean":"2024-05-09T05:38:11.979140+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:38:11.979140+0000","last_undegraded":"2024-05-09T05:38:11.979140+0000","last_fullsized":"2024-05-09T05:38:11.979140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132049+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:38:12.132049+0000","last_peered":"2024-05-09T05:38:12.132049+0000","last_clean":"2024-05-09T05:38:12.132049+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:38:12.132049+0000","last_undegraded":"2024-05-09T05:38:12.132049+0000","last_fullsized":"2024-05-09T05:38:12.132049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925512+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:38:11.925512+0000","last_peered":"2024-05-09T05:38:11.925512+0000","last_clean":"2024-05-09T05:38:11.925512+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:38:11.925512+0000","last_undegraded":"2024-05-09T05:38:11.925512+0000","last_fullsized":"2024-05-09T05:38:11.925512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924748+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:38:11.924748+0000","last_peered":"2024-05-09T05:38:11.924748+0000","last_clean":"2024-05-09T05:38:11.924748+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:38:11.924748+0000","last_undegraded":"2024-05-09T05:38:11.924748+0000","last_fullsized":"2024-05-09T05:38:11.924748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132236+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:38:12.132236+0000","last_peered":"2024-05-09T05:38:12.132236+0000","last_clean":"2024-05-09T05:38:12.132236+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:38:12.132236+0000","last_undegraded":"2024-05-09T05:38:12.132236+0000","last_fullsized":"2024-05-09T05:38:12.132236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131837+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:38:12.131837+0000","last_peered":"2024-05-09T05:38:12.131837+0000","last_clean":"2024-05-09T05:38:12.131837+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:38:12.131837+0000","last_undegraded":"2024-05-09T05:38:12.131837+0000","last_fullsized":"2024-05-09T05:38:12.131837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":249,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924836+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:38:11.924836+0000","last_peered":"2024-05-09T05:38:11.924836+0000","last_clean":"2024-05-09T05:38:11.924836+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:38:11.924836+0000","last_undegraded":"2024-05-09T05:38:11.924836+0000","last_fullsized":"2024-05-09T05:38:11.924836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053507+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:38:12.053507+0000","last_peered":"2024-05-09T05:38:12.053507+0000","last_clean":"2024-05-09T05:38:12.053507+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:38:12.053507+0000","last_undegraded":"2024-05-09T05:38:12.053507+0000","last_fullsized":"2024-05-09T05:38:12.053507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132572+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:38:12.132572+0000","last_peered":"2024-05-09T05:38:12.132572+0000","last_clean":"2024-05-09T05:38:12.132572+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:38:12.132572+0000","last_undegraded":"2024-05-09T05:38:12.132572+0000","last_fullsized":"2024-05-09T05:38:12.132572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053764+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:38:12.053764+0000","last_peered":"2024-05-09T05:38:12.053764+0000","last_clean":"2024-05-09T05:38:12.053764+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:38:12.053764+0000","last_undegraded":"2024-05-09T05:38:12.053764+0000","last_fullsized":"2024-05-09T05:38:12.053764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979670+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:38:11.979670+0000","last_peered":"2024-05-09T05:38:11.979670+0000","last_clean":"2024-05-09T05:38:11.979670+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:38:11.979670+0000","last_undegraded":"2024-05-09T05:38:11.979670+0000","last_fullsized":"2024-05-09T05:38:11.979670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132963+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:38:12.132963+0000","last_peered":"2024-05-09T05:38:12.132963+0000","last_clean":"2024-05-09T05:38:12.132963+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:38:12.132963+0000","last_undegraded":"2024-05-09T05:38:12.132963+0000","last_fullsized":"2024-05-09T05:38:12.132963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051620+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:38:12.051620+0000","last_peered":"2024-05-09T05:38:12.051620+0000","last_clean":"2024-05-09T05:38:12.051620+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:38:12.051620+0000","last_undegraded":"2024-05-09T05:38:12.051620+0000","last_fullsized":"2024-05-09T05:38:12.051620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133077+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:38:12.133077+0000","last_peered":"2024-05-09T05:38:12.133077+0000","last_clean":"2024-05-09T05:38:12.133077+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:38:12.133077+0000","last_undegraded":"2024-05-09T05:38:12.133077+0000","last_fullsized":"2024-05-09T05:38:12.133077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132705+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:38:12.132705+0000","last_peered":"2024-05-09T05:38:12.132705+0000","last_clean":"2024-05-09T05:38:12.132705+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:38:12.132705+0000","last_undegraded":"2024-05-09T05:38:12.132705+0000","last_fullsized":"2024-05-09T05:38:12.132705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051663+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:38:12.051663+0000","last_peered":"2024-05-09T05:38:12.051663+0000","last_clean":"2024-05-09T05:38:12.051663+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:38:12.051663+0000","last_undegraded":"2024-05-09T05:38:12.051663+0000","last_fullsized":"2024-05-09T05:38:12.051663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059550+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:38:12.059550+0000","last_peered":"2024-05-09T05:38:12.059550+0000","last_clean":"2024-05-09T05:38:12.059550+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:38:12.059550+0000","last_undegraded":"2024-05-09T05:38:12.059550+0000","last_fullsized":"2024-05-09T05:38:12.059550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131984+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:38:12.131984+0000","last_peered":"2024-05-09T05:38:12.131984+0000","last_clean":"2024-05-09T05:38:12.131984+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:38:12.131984+0000","last_undegraded":"2024-05-09T05:38:12.131984+0000","last_fullsized":"2024-05-09T05:38:12.131984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":245,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059421+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:38:12.059421+0000","last_peered":"2024-05-09T05:38:12.059421+0000","last_clean":"2024-05-09T05:38:12.059421+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:38:12.059421+0000","last_undegraded":"2024-05-09T05:38:12.059421+0000","last_fullsized":"2024-05-09T05:38:12.059421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051377+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:38:12.051377+0000","last_peered":"2024-05-09T05:38:12.051377+0000","last_clean":"2024-05-09T05:38:12.051377+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:38:12.051377+0000","last_undegraded":"2024-05-09T05:38:12.051377+0000","last_fullsized":"2024-05-09T05:38:12.051377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132066+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:38:12.132066+0000","last_peered":"2024-05-09T05:38:12.132066+0000","last_clean":"2024-05-09T05:38:12.132066+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:38:12.132066+0000","last_undegraded":"2024-05-09T05:38:12.132066+0000","last_fullsized":"2024-05-09T05:38:12.132066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925853+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:38:11.925853+0000","last_peered":"2024-05-09T05:38:11.925853+0000","last_clean":"2024-05-09T05:38:11.925853+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:38:11.925853+0000","last_undegraded":"2024-05-09T05:38:11.925853+0000","last_fullsized":"2024-05-09T05:38:11.925853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059515+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:38:12.059515+0000","last_peered":"2024-05-09T05:38:12.059515+0000","last_clean":"2024-05-09T05:38:12.059515+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:38:12.059515+0000","last_undegraded":"2024-05-09T05:38:12.059515+0000","last_fullsized":"2024-05-09T05:38:12.059515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925545+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:38:11.925545+0000","last_peered":"2024-05-09T05:38:11.925545+0000","last_clean":"2024-05-09T05:38:11.925545+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:38:11.925545+0000","last_undegraded":"2024-05-09T05:38:11.925545+0000","last_fullsized":"2024-05-09T05:38:11.925545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224943+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:38:12.224943+0000","last_peered":"2024-05-09T05:38:12.224943+0000","last_clean":"2024-05-09T05:38:12.224943+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:38:12.224943+0000","last_undegraded":"2024-05-09T05:38:12.224943+0000","last_fullsized":"2024-05-09T05:38:12.224943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052036+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:38:12.052036+0000","last_peered":"2024-05-09T05:38:12.052036+0000","last_clean":"2024-05-09T05:38:12.052036+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:38:12.052036+0000","last_undegraded":"2024-05-09T05:38:12.052036+0000","last_fullsized":"2024-05-09T05:38:12.052036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979576+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:38:11.979576+0000","last_peered":"2024-05-09T05:38:11.979576+0000","last_clean":"2024-05-09T05:38:11.979576+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:38:11.979576+0000","last_undegraded":"2024-05-09T05:38:11.979576+0000","last_fullsized":"2024-05-09T05:38:11.979576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059004+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:38:12.059004+0000","last_peered":"2024-05-09T05:38:12.059004+0000","last_clean":"2024-05-09T05:38:12.059004+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:38:12.059004+0000","last_undegraded":"2024-05-09T05:38:12.059004+0000","last_fullsized":"2024-05-09T05:38:12.059004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924546+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:38:11.924546+0000","last_peered":"2024-05-09T05:38:11.924546+0000","last_clean":"2024-05-09T05:38:11.924546+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:38:11.924546+0000","last_undegraded":"2024-05-09T05:38:11.924546+0000","last_fullsized":"2024-05-09T05:38:11.924546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053026+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:38:12.053026+0000","last_peered":"2024-05-09T05:38:12.053026+0000","last_clean":"2024-05-09T05:38:12.053026+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:38:12.053026+0000","last_undegraded":"2024-05-09T05:38:12.053026+0000","last_fullsized":"2024-05-09T05:38:12.053026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'7","reported_seq":429,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.592382+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:38:12.592382+0000","last_peered":"2024-05-09T05:38:12.592382+0000","last_clean":"2024-05-09T05:38:12.592382+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:38:12.592382+0000","last_undegraded":"2024-05-09T05:38:12.592382+0000","last_fullsized":"2024-05-09T05:38:12.592382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":380,"num_read_kb":380,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133858+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:38:12.133858+0000","last_peered":"2024-05-09T05:38:12.133858+0000","last_clean":"2024-05-09T05:38:12.133858+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:38:12.133858+0000","last_undegraded":"2024-05-09T05:38:12.133858+0000","last_fullsized":"2024-05-09T05:38:12.133858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924463+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:38:11.924463+0000","last_peered":"2024-05-09T05:38:11.924463+0000","last_clean":"2024-05-09T05:38:11.924463+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:38:11.924463+0000","last_undegraded":"2024-05-09T05:38:11.924463+0000","last_fullsized":"2024-05-09T05:38:11.924463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133086+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:38:12.133086+0000","last_peered":"2024-05-09T05:38:12.133086+0000","last_clean":"2024-05-09T05:38:12.133086+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:38:12.133086+0000","last_undegraded":"2024-05-09T05:38:12.133086+0000","last_fullsized":"2024-05-09T05:38:12.133086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979599+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:38:11.979599+0000","last_peered":"2024-05-09T05:38:11.979599+0000","last_clean":"2024-05-09T05:38:11.979599+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:38:11.979599+0000","last_undegraded":"2024-05-09T05:38:11.979599+0000","last_fullsized":"2024-05-09T05:38:11.979599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133847+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:38:12.133847+0000","last_peered":"2024-05-09T05:38:12.133847+0000","last_clean":"2024-05-09T05:38:12.133847+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:38:12.133847+0000","last_undegraded":"2024-05-09T05:38:12.133847+0000","last_fullsized":"2024-05-09T05:38:12.133847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059520+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:38:12.059520+0000","last_peered":"2024-05-09T05:38:12.059520+0000","last_clean":"2024-05-09T05:38:12.059520+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:38:12.059520+0000","last_undegraded":"2024-05-09T05:38:12.059520+0000","last_fullsized":"2024-05-09T05:38:12.059520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133129+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:38:12.133129+0000","last_peered":"2024-05-09T05:38:12.133129+0000","last_clean":"2024-05-09T05:38:12.133129+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:38:12.133129+0000","last_undegraded":"2024-05-09T05:38:12.133129+0000","last_fullsized":"2024-05-09T05:38:12.133129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924514+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:38:11.924514+0000","last_peered":"2024-05-09T05:38:11.924514+0000","last_clean":"2024-05-09T05:38:11.924514+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:38:11.924514+0000","last_undegraded":"2024-05-09T05:38:11.924514+0000","last_fullsized":"2024-05-09T05:38:11.924514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'43","reported_seq":477,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.596639+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:38:12.596639+0000","last_peered":"2024-05-09T05:38:12.596639+0000","last_clean":"2024-05-09T05:38:12.596639+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:38:12.596639+0000","last_undegraded":"2024-05-09T05:38:12.596639+0000","last_fullsized":"2024-05-09T05:38:12.596639+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69974,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":392,"num_read_kb":264,"num_write":45,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051722+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:38:12.051722+0000","last_peered":"2024-05-09T05:38:12.051722+0000","last_clean":"2024-05-09T05:38:12.051722+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:38:12.051722+0000","last_undegraded":"2024-05-09T05:38:12.051722+0000","last_fullsized":"2024-05-09T05:38:12.051722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059857+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:38:12.059857+0000","last_peered":"2024-05-09T05:38:12.059857+0000","last_clean":"2024-05-09T05:38:12.059857+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:38:12.059857+0000","last_undegraded":"2024-05-09T05:38:12.059857+0000","last_fullsized":"2024-05-09T05:38:12.059857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132158+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:38:12.132158+0000","last_peered":"2024-05-09T05:38:12.132158+0000","last_clean":"2024-05-09T05:38:12.132158+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:38:12.132158+0000","last_undegraded":"2024-05-09T05:38:12.132158+0000","last_fullsized":"2024-05-09T05:38:12.132158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225173+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:38:12.225173+0000","last_peered":"2024-05-09T05:38:12.225173+0000","last_clean":"2024-05-09T05:38:12.225173+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:38:12.225173+0000","last_undegraded":"2024-05-09T05:38:12.225173+0000","last_fullsized":"2024-05-09T05:38:12.225173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059308+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:38:12.059308+0000","last_peered":"2024-05-09T05:38:12.059308+0000","last_clean":"2024-05-09T05:38:12.059308+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:38:12.059308+0000","last_undegraded":"2024-05-09T05:38:12.059308+0000","last_fullsized":"2024-05-09T05:38:12.059308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132718+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:38:12.132718+0000","last_peered":"2024-05-09T05:38:12.132718+0000","last_clean":"2024-05-09T05:38:12.132718+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:38:12.132718+0000","last_undegraded":"2024-05-09T05:38:12.132718+0000","last_fullsized":"2024-05-09T05:38:12.132718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051717+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:38:12.051717+0000","last_peered":"2024-05-09T05:38:12.051717+0000","last_clean":"2024-05-09T05:38:12.051717+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:38:12.051717+0000","last_undegraded":"2024-05-09T05:38:12.051717+0000","last_fullsized":"2024-05-09T05:38:12.051717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.566344+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:38:12.566344+0000","last_peered":"2024-05-09T05:38:12.566344+0000","last_clean":"2024-05-09T05:38:12.566344+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:38:12.566344+0000","last_undegraded":"2024-05-09T05:38:12.566344+0000","last_fullsized":"2024-05-09T05:38:12.566344+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132335+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:38:12.132335+0000","last_peered":"2024-05-09T05:38:12.132335+0000","last_clean":"2024-05-09T05:38:12.132335+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:38:12.132335+0000","last_undegraded":"2024-05-09T05:38:12.132335+0000","last_fullsized":"2024-05-09T05:38:12.132335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225537+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:38:12.225537+0000","last_peered":"2024-05-09T05:38:12.225537+0000","last_clean":"2024-05-09T05:38:12.225537+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:38:12.225537+0000","last_undegraded":"2024-05-09T05:38:12.225537+0000","last_fullsized":"2024-05-09T05:38:12.225537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051224+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:38:12.051224+0000","last_peered":"2024-05-09T05:38:12.051224+0000","last_clean":"2024-05-09T05:38:12.051224+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:38:12.051224+0000","last_undegraded":"2024-05-09T05:38:12.051224+0000","last_fullsized":"2024-05-09T05:38:12.051224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924664+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:38:11.924664+0000","last_peered":"2024-05-09T05:38:11.924664+0000","last_clean":"2024-05-09T05:38:11.924664+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:38:11.924664+0000","last_undegraded":"2024-05-09T05:38:11.924664+0000","last_fullsized":"2024-05-09T05:38:11.924664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059325+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:38:12.059325+0000","last_peered":"2024-05-09T05:38:12.059325+0000","last_clean":"2024-05-09T05:38:12.059325+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:38:12.059325+0000","last_undegraded":"2024-05-09T05:38:12.059325+0000","last_fullsized":"2024-05-09T05:38:12.059325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051882+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:38:12.051882+0000","last_peered":"2024-05-09T05:38:12.051882+0000","last_clean":"2024-05-09T05:38:12.051882+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:38:12.051882+0000","last_undegraded":"2024-05-09T05:38:12.051882+0000","last_fullsized":"2024-05-09T05:38:12.051882+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925361+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:38:11.925361+0000","last_peered":"2024-05-09T05:38:11.925361+0000","last_clean":"2024-05-09T05:38:11.925361+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:38:11.925361+0000","last_undegraded":"2024-05-09T05:38:11.925361+0000","last_fullsized":"2024-05-09T05:38:11.925361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051281+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:38:12.051281+0000","last_peered":"2024-05-09T05:38:12.051281+0000","last_clean":"2024-05-09T05:38:12.051281+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:38:12.051281+0000","last_undegraded":"2024-05-09T05:38:12.051281+0000","last_fullsized":"2024-05-09T05:38:12.051281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925850+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:38:11.925850+0000","last_peered":"2024-05-09T05:38:11.925850+0000","last_clean":"2024-05-09T05:38:11.925850+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:38:11.925850+0000","last_undegraded":"2024-05-09T05:38:11.925850+0000","last_fullsized":"2024-05-09T05:38:11.925850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225102+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:38:12.225102+0000","last_peered":"2024-05-09T05:38:12.225102+0000","last_clean":"2024-05-09T05:38:12.225102+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:38:12.225102+0000","last_undegraded":"2024-05-09T05:38:12.225102+0000","last_fullsized":"2024-05-09T05:38:12.225102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'7","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.565968+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:38:12.565968+0000","last_peered":"2024-05-09T05:38:12.565968+0000","last_clean":"2024-05-09T05:38:12.565968+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:38:12.565968+0000","last_undegraded":"2024-05-09T05:38:12.565968+0000","last_fullsized":"2024-05-09T05:38:12.565968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925837+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:38:11.925837+0000","last_peered":"2024-05-09T05:38:11.925837+0000","last_clean":"2024-05-09T05:38:11.925837+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:38:11.925837+0000","last_undegraded":"2024-05-09T05:38:11.925837+0000","last_fullsized":"2024-05-09T05:38:11.925837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225665+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:38:12.225665+0000","last_peered":"2024-05-09T05:38:12.225665+0000","last_clean":"2024-05-09T05:38:12.225665+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:38:12.225665+0000","last_undegraded":"2024-05-09T05:38:12.225665+0000","last_fullsized":"2024-05-09T05:38:12.225665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059336+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:38:12.059336+0000","last_peered":"2024-05-09T05:38:12.059336+0000","last_clean":"2024-05-09T05:38:12.059336+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:38:12.059336+0000","last_undegraded":"2024-05-09T05:38:12.059336+0000","last_fullsized":"2024-05-09T05:38:12.059336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":240,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224856+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:38:12.224856+0000","last_peered":"2024-05-09T05:38:12.224856+0000","last_clean":"2024-05-09T05:38:12.224856+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:38:12.224856+0000","last_undegraded":"2024-05-09T05:38:12.224856+0000","last_fullsized":"2024-05-09T05:38:12.224856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133630+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:38:12.133630+0000","last_peered":"2024-05-09T05:38:12.133630+0000","last_clean":"2024-05-09T05:38:12.133630+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:38:12.133630+0000","last_undegraded":"2024-05-09T05:38:12.133630+0000","last_fullsized":"2024-05-09T05:38:12.133630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052441+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:38:12.052441+0000","last_peered":"2024-05-09T05:38:12.052441+0000","last_clean":"2024-05-09T05:38:12.052441+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:38:12.052441+0000","last_undegraded":"2024-05-09T05:38:12.052441+0000","last_fullsized":"2024-05-09T05:38:12.052441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133733+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:38:12.133733+0000","last_peered":"2024-05-09T05:38:12.133733+0000","last_clean":"2024-05-09T05:38:12.133733+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:38:12.133733+0000","last_undegraded":"2024-05-09T05:38:12.133733+0000","last_fullsized":"2024-05-09T05:38:12.133733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132080+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:38:12.132080+0000","last_peered":"2024-05-09T05:38:12.132080+0000","last_clean":"2024-05-09T05:38:12.132080+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:38:12.132080+0000","last_undegraded":"2024-05-09T05:38:12.132080+0000","last_fullsized":"2024-05-09T05:38:12.132080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059542+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:38:12.059542+0000","last_peered":"2024-05-09T05:38:12.059542+0000","last_clean":"2024-05-09T05:38:12.059542+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:38:12.059542+0000","last_undegraded":"2024-05-09T05:38:12.059542+0000","last_fullsized":"2024-05-09T05:38:12.059542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224704+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:38:12.224704+0000","last_peered":"2024-05-09T05:38:12.224704+0000","last_clean":"2024-05-09T05:38:12.224704+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:38:12.224704+0000","last_undegraded":"2024-05-09T05:38:12.224704+0000","last_fullsized":"2024-05-09T05:38:12.224704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132204+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:38:12.132204+0000","last_peered":"2024-05-09T05:38:12.132204+0000","last_clean":"2024-05-09T05:38:12.132204+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:38:12.132204+0000","last_undegraded":"2024-05-09T05:38:12.132204+0000","last_fullsized":"2024-05-09T05:38:12.132204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925084+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:38:11.925084+0000","last_peered":"2024-05-09T05:38:11.925084+0000","last_clean":"2024-05-09T05:38:11.925084+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:38:11.925084+0000","last_undegraded":"2024-05-09T05:38:11.925084+0000","last_fullsized":"2024-05-09T05:38:11.925084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925336+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:38:11.925336+0000","last_peered":"2024-05-09T05:38:11.925336+0000","last_clean":"2024-05-09T05:38:11.925336+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:38:11.925336+0000","last_undegraded":"2024-05-09T05:38:11.925336+0000","last_fullsized":"2024-05-09T05:38:11.925336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225403+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:38:12.225403+0000","last_peered":"2024-05-09T05:38:12.225403+0000","last_clean":"2024-05-09T05:38:12.225403+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:38:12.225403+0000","last_undegraded":"2024-05-09T05:38:12.225403+0000","last_fullsized":"2024-05-09T05:38:12.225403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051969+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:38:12.051969+0000","last_peered":"2024-05-09T05:38:12.051969+0000","last_clean":"2024-05-09T05:38:12.051969+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:38:12.051969+0000","last_undegraded":"2024-05-09T05:38:12.051969+0000","last_fullsized":"2024-05-09T05:38:12.051969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'54","reported_seq":121,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.817878+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:38:12.817878+0000","last_peered":"2024-05-09T05:38:12.817878+0000","last_clean":"2024-05-09T05:38:12.817878+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:38:12.817878+0000","last_undegraded":"2024-05-09T05:38:12.817878+0000","last_fullsized":"2024-05-09T05:38:12.817878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":96,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053724+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:38:12.053724+0000","last_peered":"2024-05-09T05:38:12.053724+0000","last_clean":"2024-05-09T05:38:12.053724+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:38:12.053724+0000","last_undegraded":"2024-05-09T05:38:12.053724+0000","last_fullsized":"2024-05-09T05:38:12.053724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131738+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:38:12.131738+0000","last_peered":"2024-05-09T05:38:12.131738+0000","last_clean":"2024-05-09T05:38:12.131738+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:38:12.131738+0000","last_undegraded":"2024-05-09T05:38:12.131738+0000","last_fullsized":"2024-05-09T05:38:12.131738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134405+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:38:12.134405+0000","last_peered":"2024-05-09T05:38:12.134405+0000","last_clean":"2024-05-09T05:38:12.134405+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:38:12.134405+0000","last_undegraded":"2024-05-09T05:38:12.134405+0000","last_fullsized":"2024-05-09T05:38:12.134405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979676+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:38:11.979676+0000","last_peered":"2024-05-09T05:38:11.979676+0000","last_clean":"2024-05-09T05:38:11.979676+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:38:11.979676+0000","last_undegraded":"2024-05-09T05:38:11.979676+0000","last_fullsized":"2024-05-09T05:38:11.979676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133578+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:38:12.133578+0000","last_peered":"2024-05-09T05:38:12.133578+0000","last_clean":"2024-05-09T05:38:12.133578+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:38:12.133578+0000","last_undegraded":"2024-05-09T05:38:12.133578+0000","last_fullsized":"2024-05-09T05:38:12.133578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132717+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:38:12.132717+0000","last_peered":"2024-05-09T05:38:12.132717+0000","last_clean":"2024-05-09T05:38:12.132717+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:38:12.132717+0000","last_undegraded":"2024-05-09T05:38:12.132717+0000","last_fullsized":"2024-05-09T05:38:12.132717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051234+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:38:12.051234+0000","last_peered":"2024-05-09T05:38:12.051234+0000","last_clean":"2024-05-09T05:38:12.051234+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:38:12.051234+0000","last_undegraded":"2024-05-09T05:38:12.051234+0000","last_fullsized":"2024-05-09T05:38:12.051234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924777+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:38:11.924777+0000","last_peered":"2024-05-09T05:38:11.924777+0000","last_clean":"2024-05-09T05:38:11.924777+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:38:11.924777+0000","last_undegraded":"2024-05-09T05:38:11.924777+0000","last_fullsized":"2024-05-09T05:38:11.924777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053182+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:38:12.053182+0000","last_peered":"2024-05-09T05:38:12.053182+0000","last_clean":"2024-05-09T05:38:12.053182+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:38:12.053182+0000","last_undegraded":"2024-05-09T05:38:12.053182+0000","last_fullsized":"2024-05-09T05:38:12.053182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924921+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:38:11.924921+0000","last_peered":"2024-05-09T05:38:11.924921+0000","last_clean":"2024-05-09T05:38:11.924921+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:38:11.924921+0000","last_undegraded":"2024-05-09T05:38:11.924921+0000","last_fullsized":"2024-05-09T05:38:11.924921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133063+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:38:12.133063+0000","last_peered":"2024-05-09T05:38:12.133063+0000","last_clean":"2024-05-09T05:38:12.133063+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:38:12.133063+0000","last_undegraded":"2024-05-09T05:38:12.133063+0000","last_fullsized":"2024-05-09T05:38:12.133063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.315433+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:38:12.315433+0000","last_peered":"2024-05-09T05:38:12.315433+0000","last_clean":"2024-05-09T05:38:12.315433+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:38:12.315433+0000","last_undegraded":"2024-05-09T05:38:12.315433+0000","last_fullsized":"2024-05-09T05:38:12.315433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053037+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:38:12.053037+0000","last_peered":"2024-05-09T05:38:12.053037+0000","last_clean":"2024-05-09T05:38:12.053037+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:38:12.053037+0000","last_undegraded":"2024-05-09T05:38:12.053037+0000","last_fullsized":"2024-05-09T05:38:12.053037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133542+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:38:12.133542+0000","last_peered":"2024-05-09T05:38:12.133542+0000","last_clean":"2024-05-09T05:38:12.133542+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:38:12.133542+0000","last_undegraded":"2024-05-09T05:38:12.133542+0000","last_fullsized":"2024-05-09T05:38:12.133542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051361+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:38:12.051361+0000","last_peered":"2024-05-09T05:38:12.051361+0000","last_clean":"2024-05-09T05:38:12.051361+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:38:12.051361+0000","last_undegraded":"2024-05-09T05:38:12.051361+0000","last_fullsized":"2024-05-09T05:38:12.051361+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132494+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:38:12.132494+0000","last_peered":"2024-05-09T05:38:12.132494+0000","last_clean":"2024-05-09T05:38:12.132494+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:38:12.132494+0000","last_undegraded":"2024-05-09T05:38:12.132494+0000","last_fullsized":"2024-05-09T05:38:12.132494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133025+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:38:12.133025+0000","last_peered":"2024-05-09T05:38:12.133025+0000","last_clean":"2024-05-09T05:38:12.133025+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:38:12.133025+0000","last_undegraded":"2024-05-09T05:38:12.133025+0000","last_fullsized":"2024-05-09T05:38:12.133025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":243,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.627036+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:38:12.627036+0000","last_peered":"2024-05-09T05:38:12.627036+0000","last_clean":"2024-05-09T05:38:12.627036+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:38:12.627036+0000","last_undegraded":"2024-05-09T05:38:12.627036+0000","last_fullsized":"2024-05-09T05:38:12.627036+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133932+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:38:12.133932+0000","last_peered":"2024-05-09T05:38:12.133932+0000","last_clean":"2024-05-09T05:38:12.133932+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:38:12.133932+0000","last_undegraded":"2024-05-09T05:38:12.133932+0000","last_fullsized":"2024-05-09T05:38:12.133932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979193+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:38:11.979193+0000","last_peered":"2024-05-09T05:38:11.979193+0000","last_clean":"2024-05-09T05:38:11.979193+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:38:11.979193+0000","last_undegraded":"2024-05-09T05:38:11.979193+0000","last_fullsized":"2024-05-09T05:38:11.979193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.175546+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:38:12.175546+0000","last_peered":"2024-05-09T05:38:12.175546+0000","last_clean":"2024-05-09T05:38:12.175546+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:38:12.175546+0000","last_undegraded":"2024-05-09T05:38:12.175546+0000","last_fullsized":"2024-05-09T05:38:12.175546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132613+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:38:12.132613+0000","last_peered":"2024-05-09T05:38:12.132613+0000","last_clean":"2024-05-09T05:38:12.132613+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:38:12.132613+0000","last_undegraded":"2024-05-09T05:38:12.132613+0000","last_fullsized":"2024-05-09T05:38:12.132613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925491+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:38:11.925491+0000","last_peered":"2024-05-09T05:38:11.925491+0000","last_clean":"2024-05-09T05:38:11.925491+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:38:11.925491+0000","last_undegraded":"2024-05-09T05:38:11.925491+0000","last_fullsized":"2024-05-09T05:38:11.925491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225540+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:38:12.225540+0000","last_peered":"2024-05-09T05:38:12.225540+0000","last_clean":"2024-05-09T05:38:12.225540+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:38:12.225540+0000","last_undegraded":"2024-05-09T05:38:12.225540+0000","last_fullsized":"2024-05-09T05:38:12.225540+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133446+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:38:12.133446+0000","last_peered":"2024-05-09T05:38:12.133446+0000","last_clean":"2024-05-09T05:38:12.133446+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:38:12.133446+0000","last_undegraded":"2024-05-09T05:38:12.133446+0000","last_fullsized":"2024-05-09T05:38:12.133446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925076+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:38:11.925076+0000","last_peered":"2024-05-09T05:38:11.925076+0000","last_clean":"2024-05-09T05:38:11.925076+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:38:11.925076+0000","last_undegraded":"2024-05-09T05:38:11.925076+0000","last_fullsized":"2024-05-09T05:38:11.925076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132222+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:38:12.132222+0000","last_peered":"2024-05-09T05:38:12.132222+0000","last_clean":"2024-05-09T05:38:12.132222+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:38:12.132222+0000","last_undegraded":"2024-05-09T05:38:12.132222+0000","last_fullsized":"2024-05-09T05:38:12.132222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131810+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:38:12.131810+0000","last_peered":"2024-05-09T05:38:12.131810+0000","last_clean":"2024-05-09T05:38:12.131810+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:38:12.131810+0000","last_undegraded":"2024-05-09T05:38:12.131810+0000","last_fullsized":"2024-05-09T05:38:12.131810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979502+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:38:11.979502+0000","last_peered":"2024-05-09T05:38:11.979502+0000","last_clean":"2024-05-09T05:38:11.979502+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:38:11.979502+0000","last_undegraded":"2024-05-09T05:38:11.979502+0000","last_fullsized":"2024-05-09T05:38:11.979502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225302+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:38:12.225302+0000","last_peered":"2024-05-09T05:38:12.225302+0000","last_clean":"2024-05-09T05:38:12.225302+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:38:12.225302+0000","last_undegraded":"2024-05-09T05:38:12.225302+0000","last_fullsized":"2024-05-09T05:38:12.225302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925541+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:38:11.925541+0000","last_peered":"2024-05-09T05:38:11.925541+0000","last_clean":"2024-05-09T05:38:11.925541+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:38:11.925541+0000","last_undegraded":"2024-05-09T05:38:11.925541+0000","last_fullsized":"2024-05-09T05:38:11.925541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132345+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:38:12.132345+0000","last_peered":"2024-05-09T05:38:12.132345+0000","last_clean":"2024-05-09T05:38:12.132345+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:38:12.132345+0000","last_undegraded":"2024-05-09T05:38:12.132345+0000","last_fullsized":"2024-05-09T05:38:12.132345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053867+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:38:12.053867+0000","last_peered":"2024-05-09T05:38:12.053867+0000","last_clean":"2024-05-09T05:38:12.053867+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:38:12.053867+0000","last_undegraded":"2024-05-09T05:38:12.053867+0000","last_fullsized":"2024-05-09T05:38:12.053867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059087+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:38:12.059087+0000","last_peered":"2024-05-09T05:38:12.059087+0000","last_clean":"2024-05-09T05:38:12.059087+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:38:12.059087+0000","last_undegraded":"2024-05-09T05:38:12.059087+0000","last_fullsized":"2024-05-09T05:38:12.059087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053259+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:38:12.053259+0000","last_peered":"2024-05-09T05:38:12.053259+0000","last_clean":"2024-05-09T05:38:12.053259+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:38:12.053259+0000","last_undegraded":"2024-05-09T05:38:12.053259+0000","last_fullsized":"2024-05-09T05:38:12.053259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133378+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:38:12.133378+0000","last_peered":"2024-05-09T05:38:12.133378+0000","last_clean":"2024-05-09T05:38:12.133378+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:38:12.133378+0000","last_undegraded":"2024-05-09T05:38:12.133378+0000","last_fullsized":"2024-05-09T05:38:12.133378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224935+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:38:12.224935+0000","last_peered":"2024-05-09T05:38:12.224935+0000","last_clean":"2024-05-09T05:38:12.224935+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:38:12.224935+0000","last_undegraded":"2024-05-09T05:38:12.224935+0000","last_fullsized":"2024-05-09T05:38:12.224935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925262+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:38:11.925262+0000","last_peered":"2024-05-09T05:38:11.925262+0000","last_clean":"2024-05-09T05:38:11.925262+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:38:11.925262+0000","last_undegraded":"2024-05-09T05:38:11.925262+0000","last_fullsized":"2024-05-09T05:38:11.925262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107766,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604889,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2043,"num_read_kb":1142,"num_write":394,"num_write_kb":1480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1814667,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710786,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507638,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907016,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1138688,"data_stored":1224885,"data_compressed":68521,"data_compressed_allocated":90112,"data_compressed_original":733184,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070542,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":903884,"data_compressed":48658,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266771,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":900054,"data_compressed":48658,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463000,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":1105849,"data_compressed":65434,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626525,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":492761,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":573964,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":504416,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":575209,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1026600,"data_compressed":54814,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986278,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507034,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822755,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":1231263,"data_compressed":71590,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135445,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":204194,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:38:17.400 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-09T05:38:17.401 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T05:38:17.497 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:17 smithi195 ceph-mon[33844]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:17.497 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:17 smithi195 ceph-mon[33844]: mgrmap e23: x(active, since 5s), standbys: y, z 2024-05-09T05:38:17.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.559+0000 7f6272d47640 1 -- 172.21.15.115:0/3791462803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c104380 msgr2=0x7f626c1067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.559+0000 7f6272d47640 1 --2- 172.21.15.115:0/3791462803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c104380 0x7f626c1067a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6260009900 tx=0x7f626002a410 comp rx=0 tx=0).stop 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.559+0000 7f6272d47640 1 -- 172.21.15.115:0/3791462803 shutdown_connections 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.559+0000 7f6272d47640 1 --2- 172.21.15.115:0/3791462803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f626c109880 0x7f626c10bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.559+0000 7f6272d47640 1 --2- 172.21.15.115:0/3791462803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f626c106d70 0x7f626c1091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.559+0000 7f6272d47640 1 --2- 172.21.15.115:0/3791462803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c104380 0x7f626c1067a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.559+0000 7f6272d47640 1 -- 172.21.15.115:0/3791462803 >> 172.21.15.115:0/3791462803 conn(0x7f626c0fdd00 msgr2=0x7f626c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.559+0000 7f6272d47640 1 -- 172.21.15.115:0/3791462803 shutdown_connections 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.559+0000 7f6272d47640 1 -- 172.21.15.115:0/3791462803 wait complete. 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6272d47640 1 Processor -- start 2024-05-09T05:38:17.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6272d47640 1 -- start start 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6272d47640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f626c104380 0x7f626c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6272d47640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c106d70 0x7f626c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6272d47640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f626c109880 0x7f626c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6272d47640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f626c10f580 con 0x7f626c106d70 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6272d47640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f626c10f400 con 0x7f626c109880 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6272d47640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f626c10f700 con 0x7f626c104380 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f626bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c106d70 0x7f626c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6270ae3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f626c104380 0x7f626c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f626bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c106d70 0x7f626c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34864/0 (socket says 172.21.15.115:34864) 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f626bfff640 1 -- 172.21.15.115:0/3879335187 learned_addr learned my addr 172.21.15.115:0/3879335187 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:17.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f62712e4640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f626c109880 0x7f626c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f626bfff640 1 -- 172.21.15.115:0/3879335187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f626c104380 msgr2=0x7f626c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f626bfff640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f626c104380 0x7f626c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f626bfff640 1 -- 172.21.15.115:0/3879335187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f626c109880 msgr2=0x7f626c1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f626bfff640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f626c109880 0x7f626c1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f626bfff640 1 -- 172.21.15.115:0/3879335187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c1a1d80 con 0x7f626c106d70 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f62712e4640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f626c109880 0x7f626c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.560+0000 7f6270ae3640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f626c104380 0x7f626c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.561+0000 7f626bfff640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c106d70 0x7f626c19a780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f625800e9f0 tx=0x7f625800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.561+0000 7f6269ffb640 1 -- 172.21.15.115:0/3879335187 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625800cd80 con 0x7f626c106d70 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.561+0000 7f6269ffb640 1 -- 172.21.15.115:0/3879335187 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62580108c0 con 0x7f626c106d70 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.561+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c1a2070 con 0x7f626c106d70 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.561+0000 7f6269ffb640 1 -- 172.21.15.115:0/3879335187 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62580187c0 con 0x7f626c106d70 2024-05-09T05:38:17.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.561+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c1a25c0 con 0x7f626c106d70 2024-05-09T05:38:17.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.562+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626c0ff820 con 0x7f626c106d70 2024-05-09T05:38:17.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.565+0000 7f6269ffb640 1 -- 172.21.15.115:0/3879335187 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f6258010430 con 0x7f626c106d70 2024-05-09T05:38:17.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.565+0000 7f6269ffb640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f625c0b29f0 0x7f625c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:17.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.565+0000 7f6269ffb640 1 -- 172.21.15.115:0/3879335187 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f62580e3390 con 0x7f626c106d70 2024-05-09T05:38:17.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.565+0000 7f6270ae3640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f625c0b29f0 0x7f625c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:17.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.566+0000 7f6270ae3640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f625c0b29f0 0x7f625c0b4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6260005ab0 tx=0x7f6260000bc0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:17.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.568+0000 7f6269ffb640 1 -- 172.21.15.115:0/3879335187 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62580aa8c0 con 0x7f626c106d70 2024-05-09T05:38:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:17 smithi161 ceph-mon[33805]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:17 smithi161 ceph-mon[33805]: mgrmap e23: x(active, since 5s), standbys: y, z 2024-05-09T05:38:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.761+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f626c063110 con 0x7f625c0b29f0 2024-05-09T05:38:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.771+0000 7f6269ffb640 1 -- 172.21.15.115:0/3879335187 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339703 (secure 0 0 0) 0x7f626c063110 con 0x7f625c0b29f0 2024-05-09T05:38:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:38:17.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f625c0b29f0 msgr2=0x7f625c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f625c0b29f0 0x7f625c0b4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6260005ab0 tx=0x7f6260000bc0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c106d70 msgr2=0x7f626c19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:17.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c106d70 0x7f626c19a780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f625800e9f0 tx=0x7f625800eeb0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 shutdown_connections 2024-05-09T05:38:17.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f625c0b29f0 0x7f625c0b4f00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f626c109880 0x7f626c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f626c106d70 0x7f626c19a780 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 --2- 172.21.15.115:0/3879335187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f626c104380 0x7f626c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:17.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 >> 172.21.15.115:0/3879335187 conn(0x7f626c0fdd00 msgr2=0x7f626c106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:17.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 shutdown_connections 2024-05-09T05:38:17.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:17.774+0000 7f6272d47640 1 -- 172.21.15.115:0/3879335187 wait complete. 2024-05-09T05:38:17.779 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-09T05:38:15.942399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195491,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2124,"num_read_kb":1211,"num_write":519,"num_write_kb":3632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":909,"ondisk_log_size":909,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344656,"kb_used_data":12092,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597168,"statfs":{"total":1151940427776,"available":1151587500032,"internally_reserved":0,"allocated":12382208,"data_stored":9553557,"data_compressed":531766,"data_compressed_allocated":716800,"data_compressed_original":4231168,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987801"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925762+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:38:11.925762+0000","last_peered":"2024-05-09T05:38:11.925762+0000","last_clean":"2024-05-09T05:38:11.925762+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:38:11.925762+0000","last_undegraded":"2024-05-09T05:38:11.925762+0000","last_fullsized":"2024-05-09T05:38:11.925762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979667+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:38:11.979667+0000","last_peered":"2024-05-09T05:38:11.979667+0000","last_clean":"2024-05-09T05:38:11.979667+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:38:11.979667+0000","last_undegraded":"2024-05-09T05:38:11.979667+0000","last_fullsized":"2024-05-09T05:38:11.979667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979267+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:38:11.979267+0000","last_peered":"2024-05-09T05:38:11.979267+0000","last_clean":"2024-05-09T05:38:11.979267+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:38:11.979267+0000","last_undegraded":"2024-05-09T05:38:11.979267+0000","last_fullsized":"2024-05-09T05:38:11.979267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133401+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:38:12.133401+0000","last_peered":"2024-05-09T05:38:12.133401+0000","last_clean":"2024-05-09T05:38:12.133401+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:38:12.133401+0000","last_undegraded":"2024-05-09T05:38:12.133401+0000","last_fullsized":"2024-05-09T05:38:12.133401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051332+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:38:12.051332+0000","last_peered":"2024-05-09T05:38:12.051332+0000","last_clean":"2024-05-09T05:38:12.051332+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:38:12.051332+0000","last_undegraded":"2024-05-09T05:38:12.051332+0000","last_fullsized":"2024-05-09T05:38:12.051332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925938+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:38:11.925938+0000","last_peered":"2024-05-09T05:38:11.925938+0000","last_clean":"2024-05-09T05:38:11.925938+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:38:11.925938+0000","last_undegraded":"2024-05-09T05:38:11.925938+0000","last_fullsized":"2024-05-09T05:38:11.925938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132223+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:38:12.132223+0000","last_peered":"2024-05-09T05:38:12.132223+0000","last_clean":"2024-05-09T05:38:12.132223+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:38:12.132223+0000","last_undegraded":"2024-05-09T05:38:12.132223+0000","last_fullsized":"2024-05-09T05:38:12.132223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051904+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:38:12.051904+0000","last_peered":"2024-05-09T05:38:12.051904+0000","last_clean":"2024-05-09T05:38:12.051904+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:38:12.051904+0000","last_undegraded":"2024-05-09T05:38:12.051904+0000","last_fullsized":"2024-05-09T05:38:12.051904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134129+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:38:12.134129+0000","last_peered":"2024-05-09T05:38:12.134129+0000","last_clean":"2024-05-09T05:38:12.134129+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:38:12.134129+0000","last_undegraded":"2024-05-09T05:38:12.134129+0000","last_fullsized":"2024-05-09T05:38:12.134129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.926126+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:38:11.926126+0000","last_peered":"2024-05-09T05:38:11.926126+0000","last_clean":"2024-05-09T05:38:11.926126+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:38:11.926126+0000","last_undegraded":"2024-05-09T05:38:11.926126+0000","last_fullsized":"2024-05-09T05:38:11.926126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133832+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:38:12.133832+0000","last_peered":"2024-05-09T05:38:12.133832+0000","last_clean":"2024-05-09T05:38:12.133832+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:38:12.133832+0000","last_undegraded":"2024-05-09T05:38:12.133832+0000","last_fullsized":"2024-05-09T05:38:12.133832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133357+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:38:12.133357+0000","last_peered":"2024-05-09T05:38:12.133357+0000","last_clean":"2024-05-09T05:38:12.133357+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:38:12.133357+0000","last_undegraded":"2024-05-09T05:38:12.133357+0000","last_fullsized":"2024-05-09T05:38:12.133357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225339+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:38:12.225339+0000","last_peered":"2024-05-09T05:38:12.225339+0000","last_clean":"2024-05-09T05:38:12.225339+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:38:12.225339+0000","last_undegraded":"2024-05-09T05:38:12.225339+0000","last_fullsized":"2024-05-09T05:38:12.225339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979195+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:38:11.979195+0000","last_peered":"2024-05-09T05:38:11.979195+0000","last_clean":"2024-05-09T05:38:11.979195+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:38:11.979195+0000","last_undegraded":"2024-05-09T05:38:11.979195+0000","last_fullsized":"2024-05-09T05:38:11.979195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979140+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:38:11.979140+0000","last_peered":"2024-05-09T05:38:11.979140+0000","last_clean":"2024-05-09T05:38:11.979140+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:38:11.979140+0000","last_undegraded":"2024-05-09T05:38:11.979140+0000","last_fullsized":"2024-05-09T05:38:11.979140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132049+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:38:12.132049+0000","last_peered":"2024-05-09T05:38:12.132049+0000","last_clean":"2024-05-09T05:38:12.132049+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:38:12.132049+0000","last_undegraded":"2024-05-09T05:38:12.132049+0000","last_fullsized":"2024-05-09T05:38:12.132049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925512+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:38:11.925512+0000","last_peered":"2024-05-09T05:38:11.925512+0000","last_clean":"2024-05-09T05:38:11.925512+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:38:11.925512+0000","last_undegraded":"2024-05-09T05:38:11.925512+0000","last_fullsized":"2024-05-09T05:38:11.925512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924748+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:38:11.924748+0000","last_peered":"2024-05-09T05:38:11.924748+0000","last_clean":"2024-05-09T05:38:11.924748+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:38:11.924748+0000","last_undegraded":"2024-05-09T05:38:11.924748+0000","last_fullsized":"2024-05-09T05:38:11.924748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132236+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:38:12.132236+0000","last_peered":"2024-05-09T05:38:12.132236+0000","last_clean":"2024-05-09T05:38:12.132236+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:38:12.132236+0000","last_undegraded":"2024-05-09T05:38:12.132236+0000","last_fullsized":"2024-05-09T05:38:12.132236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131837+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:38:12.131837+0000","last_peered":"2024-05-09T05:38:12.131837+0000","last_clean":"2024-05-09T05:38:12.131837+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:38:12.131837+0000","last_undegraded":"2024-05-09T05:38:12.131837+0000","last_fullsized":"2024-05-09T05:38:12.131837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":249,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924836+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:38:11.924836+0000","last_peered":"2024-05-09T05:38:11.924836+0000","last_clean":"2024-05-09T05:38:11.924836+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:38:11.924836+0000","last_undegraded":"2024-05-09T05:38:11.924836+0000","last_fullsized":"2024-05-09T05:38:11.924836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053507+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:38:12.053507+0000","last_peered":"2024-05-09T05:38:12.053507+0000","last_clean":"2024-05-09T05:38:12.053507+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:38:12.053507+0000","last_undegraded":"2024-05-09T05:38:12.053507+0000","last_fullsized":"2024-05-09T05:38:12.053507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132572+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:38:12.132572+0000","last_peered":"2024-05-09T05:38:12.132572+0000","last_clean":"2024-05-09T05:38:12.132572+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:38:12.132572+0000","last_undegraded":"2024-05-09T05:38:12.132572+0000","last_fullsized":"2024-05-09T05:38:12.132572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053764+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:38:12.053764+0000","last_peered":"2024-05-09T05:38:12.053764+0000","last_clean":"2024-05-09T05:38:12.053764+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:38:12.053764+0000","last_undegraded":"2024-05-09T05:38:12.053764+0000","last_fullsized":"2024-05-09T05:38:12.053764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979670+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:38:11.979670+0000","last_peered":"2024-05-09T05:38:11.979670+0000","last_clean":"2024-05-09T05:38:11.979670+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:38:11.979670+0000","last_undegraded":"2024-05-09T05:38:11.979670+0000","last_fullsized":"2024-05-09T05:38:11.979670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132963+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:38:12.132963+0000","last_peered":"2024-05-09T05:38:12.132963+0000","last_clean":"2024-05-09T05:38:12.132963+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:38:12.132963+0000","last_undegraded":"2024-05-09T05:38:12.132963+0000","last_fullsized":"2024-05-09T05:38:12.132963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051620+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:38:12.051620+0000","last_peered":"2024-05-09T05:38:12.051620+0000","last_clean":"2024-05-09T05:38:12.051620+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:38:12.051620+0000","last_undegraded":"2024-05-09T05:38:12.051620+0000","last_fullsized":"2024-05-09T05:38:12.051620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133077+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:38:12.133077+0000","last_peered":"2024-05-09T05:38:12.133077+0000","last_clean":"2024-05-09T05:38:12.133077+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:38:12.133077+0000","last_undegraded":"2024-05-09T05:38:12.133077+0000","last_fullsized":"2024-05-09T05:38:12.133077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132705+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:38:12.132705+0000","last_peered":"2024-05-09T05:38:12.132705+0000","last_clean":"2024-05-09T05:38:12.132705+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:38:12.132705+0000","last_undegraded":"2024-05-09T05:38:12.132705+0000","last_fullsized":"2024-05-09T05:38:12.132705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051663+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:38:12.051663+0000","last_peered":"2024-05-09T05:38:12.051663+0000","last_clean":"2024-05-09T05:38:12.051663+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:38:12.051663+0000","last_undegraded":"2024-05-09T05:38:12.051663+0000","last_fullsized":"2024-05-09T05:38:12.051663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059550+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:38:12.059550+0000","last_peered":"2024-05-09T05:38:12.059550+0000","last_clean":"2024-05-09T05:38:12.059550+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:38:12.059550+0000","last_undegraded":"2024-05-09T05:38:12.059550+0000","last_fullsized":"2024-05-09T05:38:12.059550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131984+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:38:12.131984+0000","last_peered":"2024-05-09T05:38:12.131984+0000","last_clean":"2024-05-09T05:38:12.131984+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:38:12.131984+0000","last_undegraded":"2024-05-09T05:38:12.131984+0000","last_fullsized":"2024-05-09T05:38:12.131984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":245,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059421+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:38:12.059421+0000","last_peered":"2024-05-09T05:38:12.059421+0000","last_clean":"2024-05-09T05:38:12.059421+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:38:12.059421+0000","last_undegraded":"2024-05-09T05:38:12.059421+0000","last_fullsized":"2024-05-09T05:38:12.059421+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051377+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:38:12.051377+0000","last_peered":"2024-05-09T05:38:12.051377+0000","last_clean":"2024-05-09T05:38:12.051377+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:38:12.051377+0000","last_undegraded":"2024-05-09T05:38:12.051377+0000","last_fullsized":"2024-05-09T05:38:12.051377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132066+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:38:12.132066+0000","last_peered":"2024-05-09T05:38:12.132066+0000","last_clean":"2024-05-09T05:38:12.132066+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:38:12.132066+0000","last_undegraded":"2024-05-09T05:38:12.132066+0000","last_fullsized":"2024-05-09T05:38:12.132066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925853+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:38:11.925853+0000","last_peered":"2024-05-09T05:38:11.925853+0000","last_clean":"2024-05-09T05:38:11.925853+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:38:11.925853+0000","last_undegraded":"2024-05-09T05:38:11.925853+0000","last_fullsized":"2024-05-09T05:38:11.925853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059515+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:38:12.059515+0000","last_peered":"2024-05-09T05:38:12.059515+0000","last_clean":"2024-05-09T05:38:12.059515+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:38:12.059515+0000","last_undegraded":"2024-05-09T05:38:12.059515+0000","last_fullsized":"2024-05-09T05:38:12.059515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925545+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:38:11.925545+0000","last_peered":"2024-05-09T05:38:11.925545+0000","last_clean":"2024-05-09T05:38:11.925545+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:38:11.925545+0000","last_undegraded":"2024-05-09T05:38:11.925545+0000","last_fullsized":"2024-05-09T05:38:11.925545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224943+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:38:12.224943+0000","last_peered":"2024-05-09T05:38:12.224943+0000","last_clean":"2024-05-09T05:38:12.224943+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:38:12.224943+0000","last_undegraded":"2024-05-09T05:38:12.224943+0000","last_fullsized":"2024-05-09T05:38:12.224943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052036+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:38:12.052036+0000","last_peered":"2024-05-09T05:38:12.052036+0000","last_clean":"2024-05-09T05:38:12.052036+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:38:12.052036+0000","last_undegraded":"2024-05-09T05:38:12.052036+0000","last_fullsized":"2024-05-09T05:38:12.052036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979576+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:38:11.979576+0000","last_peered":"2024-05-09T05:38:11.979576+0000","last_clean":"2024-05-09T05:38:11.979576+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:38:11.979576+0000","last_undegraded":"2024-05-09T05:38:11.979576+0000","last_fullsized":"2024-05-09T05:38:11.979576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059004+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:38:12.059004+0000","last_peered":"2024-05-09T05:38:12.059004+0000","last_clean":"2024-05-09T05:38:12.059004+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:38:12.059004+0000","last_undegraded":"2024-05-09T05:38:12.059004+0000","last_fullsized":"2024-05-09T05:38:12.059004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924546+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:38:11.924546+0000","last_peered":"2024-05-09T05:38:11.924546+0000","last_clean":"2024-05-09T05:38:11.924546+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:38:11.924546+0000","last_undegraded":"2024-05-09T05:38:11.924546+0000","last_fullsized":"2024-05-09T05:38:11.924546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053026+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:38:12.053026+0000","last_peered":"2024-05-09T05:38:12.053026+0000","last_clean":"2024-05-09T05:38:12.053026+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:38:12.053026+0000","last_undegraded":"2024-05-09T05:38:12.053026+0000","last_fullsized":"2024-05-09T05:38:12.053026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'7","reported_seq":429,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.592382+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:38:12.592382+0000","last_peered":"2024-05-09T05:38:12.592382+0000","last_clean":"2024-05-09T05:38:12.592382+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:38:12.592382+0000","last_undegraded":"2024-05-09T05:38:12.592382+0000","last_fullsized":"2024-05-09T05:38:12.592382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":380,"num_read_kb":380,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133858+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:38:12.133858+0000","last_peered":"2024-05-09T05:38:12.133858+0000","last_clean":"2024-05-09T05:38:12.133858+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:38:12.133858+0000","last_undegraded":"2024-05-09T05:38:12.133858+0000","last_fullsized":"2024-05-09T05:38:12.133858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924463+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:38:11.924463+0000","last_peered":"2024-05-09T05:38:11.924463+0000","last_clean":"2024-05-09T05:38:11.924463+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:38:11.924463+0000","last_undegraded":"2024-05-09T05:38:11.924463+0000","last_fullsized":"2024-05-09T05:38:11.924463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133086+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:38:12.133086+0000","last_peered":"2024-05-09T05:38:12.133086+0000","last_clean":"2024-05-09T05:38:12.133086+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:38:12.133086+0000","last_undegraded":"2024-05-09T05:38:12.133086+0000","last_fullsized":"2024-05-09T05:38:12.133086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979599+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:38:11.979599+0000","last_peered":"2024-05-09T05:38:11.979599+0000","last_clean":"2024-05-09T05:38:11.979599+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:38:11.979599+0000","last_undegraded":"2024-05-09T05:38:11.979599+0000","last_fullsized":"2024-05-09T05:38:11.979599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133847+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:38:12.133847+0000","last_peered":"2024-05-09T05:38:12.133847+0000","last_clean":"2024-05-09T05:38:12.133847+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:38:12.133847+0000","last_undegraded":"2024-05-09T05:38:12.133847+0000","last_fullsized":"2024-05-09T05:38:12.133847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059520+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:38:12.059520+0000","last_peered":"2024-05-09T05:38:12.059520+0000","last_clean":"2024-05-09T05:38:12.059520+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:38:12.059520+0000","last_undegraded":"2024-05-09T05:38:12.059520+0000","last_fullsized":"2024-05-09T05:38:12.059520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133129+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:38:12.133129+0000","last_peered":"2024-05-09T05:38:12.133129+0000","last_clean":"2024-05-09T05:38:12.133129+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:38:12.133129+0000","last_undegraded":"2024-05-09T05:38:12.133129+0000","last_fullsized":"2024-05-09T05:38:12.133129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924514+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:38:11.924514+0000","last_peered":"2024-05-09T05:38:11.924514+0000","last_clean":"2024-05-09T05:38:11.924514+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:38:11.924514+0000","last_undegraded":"2024-05-09T05:38:11.924514+0000","last_fullsized":"2024-05-09T05:38:11.924514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'43","reported_seq":477,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.596639+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:38:12.596639+0000","last_peered":"2024-05-09T05:38:12.596639+0000","last_clean":"2024-05-09T05:38:12.596639+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:38:12.596639+0000","last_undegraded":"2024-05-09T05:38:12.596639+0000","last_fullsized":"2024-05-09T05:38:12.596639+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69974,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":392,"num_read_kb":264,"num_write":45,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051722+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:38:12.051722+0000","last_peered":"2024-05-09T05:38:12.051722+0000","last_clean":"2024-05-09T05:38:12.051722+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:38:12.051722+0000","last_undegraded":"2024-05-09T05:38:12.051722+0000","last_fullsized":"2024-05-09T05:38:12.051722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059857+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:38:12.059857+0000","last_peered":"2024-05-09T05:38:12.059857+0000","last_clean":"2024-05-09T05:38:12.059857+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:38:12.059857+0000","last_undegraded":"2024-05-09T05:38:12.059857+0000","last_fullsized":"2024-05-09T05:38:12.059857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132158+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:38:12.132158+0000","last_peered":"2024-05-09T05:38:12.132158+0000","last_clean":"2024-05-09T05:38:12.132158+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:38:12.132158+0000","last_undegraded":"2024-05-09T05:38:12.132158+0000","last_fullsized":"2024-05-09T05:38:12.132158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225173+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:38:12.225173+0000","last_peered":"2024-05-09T05:38:12.225173+0000","last_clean":"2024-05-09T05:38:12.225173+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:38:12.225173+0000","last_undegraded":"2024-05-09T05:38:12.225173+0000","last_fullsized":"2024-05-09T05:38:12.225173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059308+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:38:12.059308+0000","last_peered":"2024-05-09T05:38:12.059308+0000","last_clean":"2024-05-09T05:38:12.059308+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:38:12.059308+0000","last_undegraded":"2024-05-09T05:38:12.059308+0000","last_fullsized":"2024-05-09T05:38:12.059308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132718+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:38:12.132718+0000","last_peered":"2024-05-09T05:38:12.132718+0000","last_clean":"2024-05-09T05:38:12.132718+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:38:12.132718+0000","last_undegraded":"2024-05-09T05:38:12.132718+0000","last_fullsized":"2024-05-09T05:38:12.132718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051717+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:38:12.051717+0000","last_peered":"2024-05-09T05:38:12.051717+0000","last_clean":"2024-05-09T05:38:12.051717+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:38:12.051717+0000","last_undegraded":"2024-05-09T05:38:12.051717+0000","last_fullsized":"2024-05-09T05:38:12.051717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.566344+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:38:12.566344+0000","last_peered":"2024-05-09T05:38:12.566344+0000","last_clean":"2024-05-09T05:38:12.566344+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:38:12.566344+0000","last_undegraded":"2024-05-09T05:38:12.566344+0000","last_fullsized":"2024-05-09T05:38:12.566344+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132335+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:38:12.132335+0000","last_peered":"2024-05-09T05:38:12.132335+0000","last_clean":"2024-05-09T05:38:12.132335+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:38:12.132335+0000","last_undegraded":"2024-05-09T05:38:12.132335+0000","last_fullsized":"2024-05-09T05:38:12.132335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225537+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:38:12.225537+0000","last_peered":"2024-05-09T05:38:12.225537+0000","last_clean":"2024-05-09T05:38:12.225537+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:38:12.225537+0000","last_undegraded":"2024-05-09T05:38:12.225537+0000","last_fullsized":"2024-05-09T05:38:12.225537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051224+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:38:12.051224+0000","last_peered":"2024-05-09T05:38:12.051224+0000","last_clean":"2024-05-09T05:38:12.051224+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:38:12.051224+0000","last_undegraded":"2024-05-09T05:38:12.051224+0000","last_fullsized":"2024-05-09T05:38:12.051224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924664+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:38:11.924664+0000","last_peered":"2024-05-09T05:38:11.924664+0000","last_clean":"2024-05-09T05:38:11.924664+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:38:11.924664+0000","last_undegraded":"2024-05-09T05:38:11.924664+0000","last_fullsized":"2024-05-09T05:38:11.924664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059325+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:38:12.059325+0000","last_peered":"2024-05-09T05:38:12.059325+0000","last_clean":"2024-05-09T05:38:12.059325+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:38:12.059325+0000","last_undegraded":"2024-05-09T05:38:12.059325+0000","last_fullsized":"2024-05-09T05:38:12.059325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051882+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:38:12.051882+0000","last_peered":"2024-05-09T05:38:12.051882+0000","last_clean":"2024-05-09T05:38:12.051882+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:38:12.051882+0000","last_undegraded":"2024-05-09T05:38:12.051882+0000","last_fullsized":"2024-05-09T05:38:12.051882+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925361+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:38:11.925361+0000","last_peered":"2024-05-09T05:38:11.925361+0000","last_clean":"2024-05-09T05:38:11.925361+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:38:11.925361+0000","last_undegraded":"2024-05-09T05:38:11.925361+0000","last_fullsized":"2024-05-09T05:38:11.925361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051281+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:38:12.051281+0000","last_peered":"2024-05-09T05:38:12.051281+0000","last_clean":"2024-05-09T05:38:12.051281+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:38:12.051281+0000","last_undegraded":"2024-05-09T05:38:12.051281+0000","last_fullsized":"2024-05-09T05:38:12.051281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925850+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:38:11.925850+0000","last_peered":"2024-05-09T05:38:11.925850+0000","last_clean":"2024-05-09T05:38:11.925850+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:38:11.925850+0000","last_undegraded":"2024-05-09T05:38:11.925850+0000","last_fullsized":"2024-05-09T05:38:11.925850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225102+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:38:12.225102+0000","last_peered":"2024-05-09T05:38:12.225102+0000","last_clean":"2024-05-09T05:38:12.225102+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:38:12.225102+0000","last_undegraded":"2024-05-09T05:38:12.225102+0000","last_fullsized":"2024-05-09T05:38:12.225102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'7","reported_seq":244,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.565968+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:38:12.565968+0000","last_peered":"2024-05-09T05:38:12.565968+0000","last_clean":"2024-05-09T05:38:12.565968+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:38:12.565968+0000","last_undegraded":"2024-05-09T05:38:12.565968+0000","last_fullsized":"2024-05-09T05:38:12.565968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925837+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:38:11.925837+0000","last_peered":"2024-05-09T05:38:11.925837+0000","last_clean":"2024-05-09T05:38:11.925837+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:38:11.925837+0000","last_undegraded":"2024-05-09T05:38:11.925837+0000","last_fullsized":"2024-05-09T05:38:11.925837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225665+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:38:12.225665+0000","last_peered":"2024-05-09T05:38:12.225665+0000","last_clean":"2024-05-09T05:38:12.225665+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:38:12.225665+0000","last_undegraded":"2024-05-09T05:38:12.225665+0000","last_fullsized":"2024-05-09T05:38:12.225665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059336+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:38:12.059336+0000","last_peered":"2024-05-09T05:38:12.059336+0000","last_clean":"2024-05-09T05:38:12.059336+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:38:12.059336+0000","last_undegraded":"2024-05-09T05:38:12.059336+0000","last_fullsized":"2024-05-09T05:38:12.059336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":240,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224856+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:38:12.224856+0000","last_peered":"2024-05-09T05:38:12.224856+0000","last_clean":"2024-05-09T05:38:12.224856+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:38:12.224856+0000","last_undegraded":"2024-05-09T05:38:12.224856+0000","last_fullsized":"2024-05-09T05:38:12.224856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133630+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:38:12.133630+0000","last_peered":"2024-05-09T05:38:12.133630+0000","last_clean":"2024-05-09T05:38:12.133630+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:38:12.133630+0000","last_undegraded":"2024-05-09T05:38:12.133630+0000","last_fullsized":"2024-05-09T05:38:12.133630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052441+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:38:12.052441+0000","last_peered":"2024-05-09T05:38:12.052441+0000","last_clean":"2024-05-09T05:38:12.052441+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:38:12.052441+0000","last_undegraded":"2024-05-09T05:38:12.052441+0000","last_fullsized":"2024-05-09T05:38:12.052441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133733+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:38:12.133733+0000","last_peered":"2024-05-09T05:38:12.133733+0000","last_clean":"2024-05-09T05:38:12.133733+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:38:12.133733+0000","last_undegraded":"2024-05-09T05:38:12.133733+0000","last_fullsized":"2024-05-09T05:38:12.133733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132080+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:38:12.132080+0000","last_peered":"2024-05-09T05:38:12.132080+0000","last_clean":"2024-05-09T05:38:12.132080+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:38:12.132080+0000","last_undegraded":"2024-05-09T05:38:12.132080+0000","last_fullsized":"2024-05-09T05:38:12.132080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059542+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:38:12.059542+0000","last_peered":"2024-05-09T05:38:12.059542+0000","last_clean":"2024-05-09T05:38:12.059542+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:38:12.059542+0000","last_undegraded":"2024-05-09T05:38:12.059542+0000","last_fullsized":"2024-05-09T05:38:12.059542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224704+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:38:12.224704+0000","last_peered":"2024-05-09T05:38:12.224704+0000","last_clean":"2024-05-09T05:38:12.224704+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:38:12.224704+0000","last_undegraded":"2024-05-09T05:38:12.224704+0000","last_fullsized":"2024-05-09T05:38:12.224704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132204+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:38:12.132204+0000","last_peered":"2024-05-09T05:38:12.132204+0000","last_clean":"2024-05-09T05:38:12.132204+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:38:12.132204+0000","last_undegraded":"2024-05-09T05:38:12.132204+0000","last_fullsized":"2024-05-09T05:38:12.132204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925084+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:38:11.925084+0000","last_peered":"2024-05-09T05:38:11.925084+0000","last_clean":"2024-05-09T05:38:11.925084+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:38:11.925084+0000","last_undegraded":"2024-05-09T05:38:11.925084+0000","last_fullsized":"2024-05-09T05:38:11.925084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925336+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:38:11.925336+0000","last_peered":"2024-05-09T05:38:11.925336+0000","last_clean":"2024-05-09T05:38:11.925336+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:38:11.925336+0000","last_undegraded":"2024-05-09T05:38:11.925336+0000","last_fullsized":"2024-05-09T05:38:11.925336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225403+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:38:12.225403+0000","last_peered":"2024-05-09T05:38:12.225403+0000","last_clean":"2024-05-09T05:38:12.225403+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:38:12.225403+0000","last_undegraded":"2024-05-09T05:38:12.225403+0000","last_fullsized":"2024-05-09T05:38:12.225403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051969+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:38:12.051969+0000","last_peered":"2024-05-09T05:38:12.051969+0000","last_clean":"2024-05-09T05:38:12.051969+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:38:12.051969+0000","last_undegraded":"2024-05-09T05:38:12.051969+0000","last_fullsized":"2024-05-09T05:38:12.051969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'54","reported_seq":121,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.817878+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:38:12.817878+0000","last_peered":"2024-05-09T05:38:12.817878+0000","last_clean":"2024-05-09T05:38:12.817878+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:38:12.817878+0000","last_undegraded":"2024-05-09T05:38:12.817878+0000","last_fullsized":"2024-05-09T05:38:12.817878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":96,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053724+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:38:12.053724+0000","last_peered":"2024-05-09T05:38:12.053724+0000","last_clean":"2024-05-09T05:38:12.053724+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:38:12.053724+0000","last_undegraded":"2024-05-09T05:38:12.053724+0000","last_fullsized":"2024-05-09T05:38:12.053724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131738+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:38:12.131738+0000","last_peered":"2024-05-09T05:38:12.131738+0000","last_clean":"2024-05-09T05:38:12.131738+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:38:12.131738+0000","last_undegraded":"2024-05-09T05:38:12.131738+0000","last_fullsized":"2024-05-09T05:38:12.131738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134405+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:38:12.134405+0000","last_peered":"2024-05-09T05:38:12.134405+0000","last_clean":"2024-05-09T05:38:12.134405+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:38:12.134405+0000","last_undegraded":"2024-05-09T05:38:12.134405+0000","last_fullsized":"2024-05-09T05:38:12.134405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979676+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:38:11.979676+0000","last_peered":"2024-05-09T05:38:11.979676+0000","last_clean":"2024-05-09T05:38:11.979676+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:38:11.979676+0000","last_undegraded":"2024-05-09T05:38:11.979676+0000","last_fullsized":"2024-05-09T05:38:11.979676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133578+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:38:12.133578+0000","last_peered":"2024-05-09T05:38:12.133578+0000","last_clean":"2024-05-09T05:38:12.133578+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:38:12.133578+0000","last_undegraded":"2024-05-09T05:38:12.133578+0000","last_fullsized":"2024-05-09T05:38:12.133578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132717+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:38:12.132717+0000","last_peered":"2024-05-09T05:38:12.132717+0000","last_clean":"2024-05-09T05:38:12.132717+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:38:12.132717+0000","last_undegraded":"2024-05-09T05:38:12.132717+0000","last_fullsized":"2024-05-09T05:38:12.132717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051234+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:38:12.051234+0000","last_peered":"2024-05-09T05:38:12.051234+0000","last_clean":"2024-05-09T05:38:12.051234+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:38:12.051234+0000","last_undegraded":"2024-05-09T05:38:12.051234+0000","last_fullsized":"2024-05-09T05:38:12.051234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924777+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:38:11.924777+0000","last_peered":"2024-05-09T05:38:11.924777+0000","last_clean":"2024-05-09T05:38:11.924777+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:38:11.924777+0000","last_undegraded":"2024-05-09T05:38:11.924777+0000","last_fullsized":"2024-05-09T05:38:11.924777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053182+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:38:12.053182+0000","last_peered":"2024-05-09T05:38:12.053182+0000","last_clean":"2024-05-09T05:38:12.053182+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:38:12.053182+0000","last_undegraded":"2024-05-09T05:38:12.053182+0000","last_fullsized":"2024-05-09T05:38:12.053182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924921+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:38:11.924921+0000","last_peered":"2024-05-09T05:38:11.924921+0000","last_clean":"2024-05-09T05:38:11.924921+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:38:11.924921+0000","last_undegraded":"2024-05-09T05:38:11.924921+0000","last_fullsized":"2024-05-09T05:38:11.924921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133063+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:38:12.133063+0000","last_peered":"2024-05-09T05:38:12.133063+0000","last_clean":"2024-05-09T05:38:12.133063+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:38:12.133063+0000","last_undegraded":"2024-05-09T05:38:12.133063+0000","last_fullsized":"2024-05-09T05:38:12.133063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.315433+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:38:12.315433+0000","last_peered":"2024-05-09T05:38:12.315433+0000","last_clean":"2024-05-09T05:38:12.315433+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:38:12.315433+0000","last_undegraded":"2024-05-09T05:38:12.315433+0000","last_fullsized":"2024-05-09T05:38:12.315433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053037+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:38:12.053037+0000","last_peered":"2024-05-09T05:38:12.053037+0000","last_clean":"2024-05-09T05:38:12.053037+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:38:12.053037+0000","last_undegraded":"2024-05-09T05:38:12.053037+0000","last_fullsized":"2024-05-09T05:38:12.053037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133542+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:38:12.133542+0000","last_peered":"2024-05-09T05:38:12.133542+0000","last_clean":"2024-05-09T05:38:12.133542+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:38:12.133542+0000","last_undegraded":"2024-05-09T05:38:12.133542+0000","last_fullsized":"2024-05-09T05:38:12.133542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051361+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:38:12.051361+0000","last_peered":"2024-05-09T05:38:12.051361+0000","last_clean":"2024-05-09T05:38:12.051361+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:38:12.051361+0000","last_undegraded":"2024-05-09T05:38:12.051361+0000","last_fullsized":"2024-05-09T05:38:12.051361+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132494+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:38:12.132494+0000","last_peered":"2024-05-09T05:38:12.132494+0000","last_clean":"2024-05-09T05:38:12.132494+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:38:12.132494+0000","last_undegraded":"2024-05-09T05:38:12.132494+0000","last_fullsized":"2024-05-09T05:38:12.132494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133025+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:38:12.133025+0000","last_peered":"2024-05-09T05:38:12.133025+0000","last_clean":"2024-05-09T05:38:12.133025+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:38:12.133025+0000","last_undegraded":"2024-05-09T05:38:12.133025+0000","last_fullsized":"2024-05-09T05:38:12.133025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":243,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.627036+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:38:12.627036+0000","last_peered":"2024-05-09T05:38:12.627036+0000","last_clean":"2024-05-09T05:38:12.627036+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:38:12.627036+0000","last_undegraded":"2024-05-09T05:38:12.627036+0000","last_fullsized":"2024-05-09T05:38:12.627036+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133932+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:38:12.133932+0000","last_peered":"2024-05-09T05:38:12.133932+0000","last_clean":"2024-05-09T05:38:12.133932+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:38:12.133932+0000","last_undegraded":"2024-05-09T05:38:12.133932+0000","last_fullsized":"2024-05-09T05:38:12.133932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979193+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:38:11.979193+0000","last_peered":"2024-05-09T05:38:11.979193+0000","last_clean":"2024-05-09T05:38:11.979193+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:38:11.979193+0000","last_undegraded":"2024-05-09T05:38:11.979193+0000","last_fullsized":"2024-05-09T05:38:11.979193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.175546+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:38:12.175546+0000","last_peered":"2024-05-09T05:38:12.175546+0000","last_clean":"2024-05-09T05:38:12.175546+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:38:12.175546+0000","last_undegraded":"2024-05-09T05:38:12.175546+0000","last_fullsized":"2024-05-09T05:38:12.175546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132613+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:38:12.132613+0000","last_peered":"2024-05-09T05:38:12.132613+0000","last_clean":"2024-05-09T05:38:12.132613+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:38:12.132613+0000","last_undegraded":"2024-05-09T05:38:12.132613+0000","last_fullsized":"2024-05-09T05:38:12.132613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925491+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:38:11.925491+0000","last_peered":"2024-05-09T05:38:11.925491+0000","last_clean":"2024-05-09T05:38:11.925491+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:38:11.925491+0000","last_undegraded":"2024-05-09T05:38:11.925491+0000","last_fullsized":"2024-05-09T05:38:11.925491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225540+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:38:12.225540+0000","last_peered":"2024-05-09T05:38:12.225540+0000","last_clean":"2024-05-09T05:38:12.225540+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:38:12.225540+0000","last_undegraded":"2024-05-09T05:38:12.225540+0000","last_fullsized":"2024-05-09T05:38:12.225540+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133446+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:38:12.133446+0000","last_peered":"2024-05-09T05:38:12.133446+0000","last_clean":"2024-05-09T05:38:12.133446+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:38:12.133446+0000","last_undegraded":"2024-05-09T05:38:12.133446+0000","last_fullsized":"2024-05-09T05:38:12.133446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925076+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:38:11.925076+0000","last_peered":"2024-05-09T05:38:11.925076+0000","last_clean":"2024-05-09T05:38:11.925076+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:38:11.925076+0000","last_undegraded":"2024-05-09T05:38:11.925076+0000","last_fullsized":"2024-05-09T05:38:11.925076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132222+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:38:12.132222+0000","last_peered":"2024-05-09T05:38:12.132222+0000","last_clean":"2024-05-09T05:38:12.132222+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:38:12.132222+0000","last_undegraded":"2024-05-09T05:38:12.132222+0000","last_fullsized":"2024-05-09T05:38:12.132222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131810+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:38:12.131810+0000","last_peered":"2024-05-09T05:38:12.131810+0000","last_clean":"2024-05-09T05:38:12.131810+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:38:12.131810+0000","last_undegraded":"2024-05-09T05:38:12.131810+0000","last_fullsized":"2024-05-09T05:38:12.131810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979502+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:38:11.979502+0000","last_peered":"2024-05-09T05:38:11.979502+0000","last_clean":"2024-05-09T05:38:11.979502+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:38:11.979502+0000","last_undegraded":"2024-05-09T05:38:11.979502+0000","last_fullsized":"2024-05-09T05:38:11.979502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225302+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:38:12.225302+0000","last_peered":"2024-05-09T05:38:12.225302+0000","last_clean":"2024-05-09T05:38:12.225302+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:38:12.225302+0000","last_undegraded":"2024-05-09T05:38:12.225302+0000","last_fullsized":"2024-05-09T05:38:12.225302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925541+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:38:11.925541+0000","last_peered":"2024-05-09T05:38:11.925541+0000","last_clean":"2024-05-09T05:38:11.925541+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:38:11.925541+0000","last_undegraded":"2024-05-09T05:38:11.925541+0000","last_fullsized":"2024-05-09T05:38:11.925541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132345+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:38:12.132345+0000","last_peered":"2024-05-09T05:38:12.132345+0000","last_clean":"2024-05-09T05:38:12.132345+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:38:12.132345+0000","last_undegraded":"2024-05-09T05:38:12.132345+0000","last_fullsized":"2024-05-09T05:38:12.132345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053867+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:38:12.053867+0000","last_peered":"2024-05-09T05:38:12.053867+0000","last_clean":"2024-05-09T05:38:12.053867+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:38:12.053867+0000","last_undegraded":"2024-05-09T05:38:12.053867+0000","last_fullsized":"2024-05-09T05:38:12.053867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059087+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:38:12.059087+0000","last_peered":"2024-05-09T05:38:12.059087+0000","last_clean":"2024-05-09T05:38:12.059087+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:38:12.059087+0000","last_undegraded":"2024-05-09T05:38:12.059087+0000","last_fullsized":"2024-05-09T05:38:12.059087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053259+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:38:12.053259+0000","last_peered":"2024-05-09T05:38:12.053259+0000","last_clean":"2024-05-09T05:38:12.053259+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:38:12.053259+0000","last_undegraded":"2024-05-09T05:38:12.053259+0000","last_fullsized":"2024-05-09T05:38:12.053259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133378+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:38:12.133378+0000","last_peered":"2024-05-09T05:38:12.133378+0000","last_clean":"2024-05-09T05:38:12.133378+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:38:12.133378+0000","last_undegraded":"2024-05-09T05:38:12.133378+0000","last_fullsized":"2024-05-09T05:38:12.133378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224935+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:38:12.224935+0000","last_peered":"2024-05-09T05:38:12.224935+0000","last_clean":"2024-05-09T05:38:12.224935+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:38:12.224935+0000","last_undegraded":"2024-05-09T05:38:12.224935+0000","last_fullsized":"2024-05-09T05:38:12.224935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925262+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:38:11.925262+0000","last_peered":"2024-05-09T05:38:11.925262+0000","last_clean":"2024-05-09T05:38:11.925262+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:38:11.925262+0000","last_undegraded":"2024-05-09T05:38:11.925262+0000","last_fullsized":"2024-05-09T05:38:11.925262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107766,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604889,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2043,"num_read_kb":1142,"num_write":394,"num_write_kb":1480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1814667,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710786,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507638,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907016,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1138688,"data_stored":1224885,"data_compressed":68521,"data_compressed_allocated":90112,"data_compressed_original":733184,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070542,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":903884,"data_compressed":48658,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266771,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":900054,"data_compressed":48658,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463000,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":1105849,"data_compressed":65434,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626525,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":492761,"data_compressed":26531,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":573964,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":504416,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":575209,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1026600,"data_compressed":54814,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986278,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507034,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822755,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":1231263,"data_compressed":71590,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135445,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":204194,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:38:17.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:38:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:18.468 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:18.468 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi115 to 176.1M 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: Unable to set osd_memory_target on smithi115 to 184750899: error parsing value: Value '184750899' is below minimum 939524096 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: Updating smithi115:/etc/ceph/ceph.conf 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-09T05:38:18.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:18 smithi115 ceph-mon[30349]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-09T05:38:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi115 to 176.1M 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: Unable to set osd_memory_target on smithi115 to 184750899: error parsing value: Value '184750899' is below minimum 939524096 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: Updating smithi115:/etc/ceph/ceph.conf 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-09T05:38:18.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:18 smithi161 ceph-mon[33805]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-09T05:38:18.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi115 to 176.1M 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: Unable to set osd_memory_target on smithi115 to 184750899: error parsing value: Value '184750899' is below minimum 939524096 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: Updating smithi115:/etc/ceph/ceph.conf 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-09T05:38:18.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:18 smithi195 ceph-mon[33844]: Updating smithi195:/etc/ceph/ceph.conf 2024-05-09T05:38:19.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:38:19.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:38:19.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:38:19.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:19.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:19.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:38:19.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:38:19.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:38:19.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:38:19.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:38:19.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:38:19.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.518 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:19.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:19.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:19.519 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:19 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:19.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:19.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:19.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:19 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:38:19.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:38:19.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T05:38:19.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: Updating smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: Updating smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: Updating smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:19.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:19.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:19.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:19 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:19.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.922+0000 7f46e7eec700 1 -- 172.21.15.115:0/3293000982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e0108740 msgr2=0x7f46e01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:19.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.922+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3293000982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e0108740 0x7f46e01100a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f46d800b0d0 tx=0x7f46d800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:38:19.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.922+0000 7f46e7eec700 1 -- 172.21.15.115:0/3293000982 shutdown_connections 2024-05-09T05:38:19.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.922+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3293000982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e0108740 0x7f46e01100a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:19.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.922+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3293000982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e01034c0 0x7f46e01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:19.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.922+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3293000982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46e0107df0 0x7f46e0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:19.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.922+0000 7f46e7eec700 1 -- 172.21.15.115:0/3293000982 >> 172.21.15.115:0/3293000982 conn(0x7f46e00fedc0 msgr2=0x7f46e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:19.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.922+0000 7f46e7eec700 1 -- 172.21.15.115:0/3293000982 shutdown_connections 2024-05-09T05:38:19.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.922+0000 7f46e7eec700 1 -- 172.21.15.115:0/3293000982 wait complete. 2024-05-09T05:38:19.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e7eec700 1 Processor -- start 2024-05-09T05:38:19.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e7eec700 1 -- start start 2024-05-09T05:38:19.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e7eec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e01034c0 0x7f46e019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:19.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e7eec700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e0107df0 0x7f46e019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:19.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e7eec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46e0108740 0x7f46e019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:19.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e7eec700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f46e0113220 con 0x7f46e01034c0 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e7eec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46e01130a0 con 0x7f46e0108740 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e7eec700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46e01133a0 con 0x7f46e0107df0 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e5c88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e01034c0 0x7f46e019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e5c88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e01034c0 0x7f46e019bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34880/0 (socket says 172.21.15.115:34880) 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.923+0000 7f46e5c88700 1 -- 172.21.15.115:0/3706245067 learned_addr learned my addr 172.21.15.115:0/3706245067 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e5487700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e0107df0 0x7f46e019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e6489700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46e0108740 0x7f46e019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e5c88700 1 -- 172.21.15.115:0/3706245067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e0107df0 msgr2=0x7f46e019c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e5c88700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e0107df0 0x7f46e019c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e5c88700 1 -- 172.21.15.115:0/3706245067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46e0108740 msgr2=0x7f46e019c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e5c88700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46e0108740 0x7f46e019c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e5c88700 1 -- 172.21.15.115:0/3706245067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e01a6e90 con 0x7f46e01034c0 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e6489700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46e0108740 0x7f46e019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:19.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e5c88700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e01034c0 0x7f46e019bb40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00d950 tx=0x7f46dc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:19.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46d6ffd700 1 -- 172.21.15.115:0/3706245067 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc016070 con 0x7f46e01034c0 2024-05-09T05:38:19.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e01a7180 con 0x7f46e01034c0 2024-05-09T05:38:19.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46d6ffd700 1 -- 172.21.15.115:0/3706245067 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46dc010460 con 0x7f46e01034c0 2024-05-09T05:38:19.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46d6ffd700 1 -- 172.21.15.115:0/3706245067 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc00f620 con 0x7f46e01034c0 2024-05-09T05:38:19.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.924+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e01a76c0 con 0x7f46e01034c0 2024-05-09T05:38:19.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.925+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e00512c0 con 0x7f46e01034c0 2024-05-09T05:38:19.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.927+0000 7f46d6ffd700 1 -- 172.21.15.115:0/3706245067 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f46dc019070 con 0x7f46e01034c0 2024-05-09T05:38:19.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.928+0000 7f46d6ffd700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f46cc0b2a80 0x7f46cc0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:19.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.928+0000 7f46e5487700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f46cc0b2a80 0x7f46cc0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:19.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.928+0000 7f46d6ffd700 1 -- 172.21.15.115:0/3706245067 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f46dc0e2790 con 0x7f46e01034c0 2024-05-09T05:38:19.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.929+0000 7f46e5487700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f46cc0b2a80 0x7f46cc0b4f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46e0102440 tx=0x7f46d0009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:19.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:19.932+0000 7f46d6ffd700 1 -- 172.21.15.115:0/3706245067 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46dc0a9c60 con 0x7f46e01034c0 2024-05-09T05:38:20.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.180+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46e006bc00 con 0x7f46e01034c0 2024-05-09T05:38:20.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.181+0000 7f46d6ffd700 1 -- 172.21.15.115:0/3706245067 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7f46dc0a9560 con 0x7f46e01034c0 2024-05-09T05:38:20.174 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:20.175 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f46cc0b2a80 msgr2=0x7f46cc0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f46cc0b2a80 0x7f46cc0b4f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46e0102440 tx=0x7f46d0009260 comp rx=0 tx=0).stop 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e01034c0 msgr2=0x7f46e019bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e01034c0 0x7f46e019bb40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00d950 tx=0x7f46dc00dd10 comp rx=0 tx=0).stop 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 shutdown_connections 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f46cc0b2a80 0x7f46cc0b4f90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46e0108740 0x7f46e019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46e01034c0 0x7f46e019bb40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 --2- 172.21.15.115:0/3706245067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46e0107df0 0x7f46e019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 >> 172.21.15.115:0/3706245067 conn(0x7f46e00fedc0 msgr2=0x7f46e01075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:20.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 shutdown_connections 2024-05-09T05:38:20.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:20.184+0000 7f46e7eec700 1 -- 172.21.15.115:0/3706245067 wait complete. 2024-05-09T05:38:20.178 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:38:20.550 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:20 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3706245067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:20.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:20 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3706245067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:20.897 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:38:20.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:20 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3706245067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:21.142 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:21.538 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:21 smithi115 ceph-mon[30349]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 0 B/s wr, 17 op/s 2024-05-09T05:38:21.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:21 smithi195 ceph-mon[33844]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 0 B/s wr, 17 op/s 2024-05-09T05:38:21.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:21 smithi161 ceph-mon[33805]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 0 B/s wr, 17 op/s 2024-05-09T05:38:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.942+0000 7fced383d700 1 -- 172.21.15.115:0/4236571702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc108780 msgr2=0x7fcecc1100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.942+0000 7fced383d700 1 --2- 172.21.15.115:0/4236571702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc108780 0x7fcecc1100e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcec400b0d0 tx=0x7fcec400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:38:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.942+0000 7fced383d700 1 -- 172.21.15.115:0/4236571702 shutdown_connections 2024-05-09T05:38:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.942+0000 7fced383d700 1 --2- 172.21.15.115:0/4236571702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc108780 0x7fcecc1100e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.942+0000 7fced383d700 1 --2- 172.21.15.115:0/4236571702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcecc103550 0x7fcecc103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.942+0000 7fced383d700 1 --2- 172.21.15.115:0/4236571702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcecc107e30 0x7fcecc108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:22.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.942+0000 7fced383d700 1 -- 172.21.15.115:0/4236571702 >> 172.21.15.115:0/4236571702 conn(0x7fcecc078e00 msgr2=0x7fcecc079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:22.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.942+0000 7fced383d700 1 -- 172.21.15.115:0/4236571702 shutdown_connections 2024-05-09T05:38:22.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.942+0000 7fced383d700 1 -- 172.21.15.115:0/4236571702 wait complete. 2024-05-09T05:38:22.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced383d700 1 Processor -- start 2024-05-09T05:38:22.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced383d700 1 -- start start 2024-05-09T05:38:22.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced383d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc103550 0x7fcecc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced383d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcecc107e30 0x7fcecc19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced383d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcecc108780 0x7fcecc199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced383d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc113260 con 0x7fcecc103550 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced383d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1130e0 con 0x7fcecc108780 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced383d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1133e0 con 0x7fcecc107e30 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced15d9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc103550 0x7fcecc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced15d9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc103550 0x7fcecc19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39056/0 (socket says 172.21.15.115:39056) 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced15d9700 1 -- 172.21.15.115:0/1867557035 learned_addr learned my addr 172.21.15.115:0/1867557035 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced0dd8700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcecc107e30 0x7fcecc19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.943+0000 7fced1dda700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcecc108780 0x7fcecc199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fced15d9700 1 -- 172.21.15.115:0/1867557035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcecc107e30 msgr2=0x7fcecc19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fced15d9700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcecc107e30 0x7fcecc19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fced15d9700 1 -- 172.21.15.115:0/1867557035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcecc108780 msgr2=0x7fcecc199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fced15d9700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcecc108780 0x7fcecc199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fced15d9700 1 -- 172.21.15.115:0/1867557035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc19a0d0 con 0x7fcecc103550 2024-05-09T05:38:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fced15d9700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc103550 0x7fcecc19f800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcec8007910 tx=0x7fcec8007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fcec27fc700 1 -- 172.21.15.115:0/1867557035 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8004020 con 0x7fcecc103550 2024-05-09T05:38:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fcec27fc700 1 -- 172.21.15.115:0/1867557035 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcec8004830 con 0x7fcecc103550 2024-05-09T05:38:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc19a3c0 con 0x7fcecc103550 2024-05-09T05:38:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fcec27fc700 1 -- 172.21.15.115:0/1867557035 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8005480 con 0x7fcecc103550 2024-05-09T05:38:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.944+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc0759f0 con 0x7fcecc103550 2024-05-09T05:38:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.945+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcecc0512c0 con 0x7fcecc103550 2024-05-09T05:38:22.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.947+0000 7fcec27fc700 1 -- 172.21.15.115:0/1867557035 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fcec801f020 con 0x7fcecc103550 2024-05-09T05:38:22.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.948+0000 7fcec27fc700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb80b2b00 0x7fceb80b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:22.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.948+0000 7fced0dd8700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb80b2b00 0x7fceb80b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:22.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.948+0000 7fcec27fc700 1 -- 172.21.15.115:0/1867557035 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fcec80e4510 con 0x7fcecc103550 2024-05-09T05:38:22.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.948+0000 7fced0dd8700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb80b2b00 0x7fceb80b5010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcecc1086f0 tx=0x7fcebc00b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:22.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:22.952+0000 7fcec27fc700 1 -- 172.21.15.115:0/1867557035 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcec8026030 con 0x7fcecc103550 2024-05-09T05:38:23.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.158+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcecc076420 con 0x7fcecc103550 2024-05-09T05:38:23.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.159+0000 7fcec27fc700 1 -- 172.21.15.115:0/1867557035 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25107 (secure 0 0 0) 0x7fcec80aba40 con 0x7fcecc103550 2024-05-09T05:38:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:23.154 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":88,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:38:11.918691+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1352570099":"2024-05-10T05:38:11.918655+0000","172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:6800/2041699318":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/617068092":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/2041699318":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/4237548909":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/1853670702":"2024-05-10T05:38:11.918655+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:38:23.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb80b2b00 msgr2=0x7fceb80b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:23.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb80b2b00 0x7fceb80b5010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcecc1086f0 tx=0x7fcebc00b410 comp rx=0 tx=0).stop 2024-05-09T05:38:23.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc103550 msgr2=0x7fcecc19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:23.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc103550 0x7fcecc19f800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcec8007910 tx=0x7fcec8007c20 comp rx=0 tx=0).stop 2024-05-09T05:38:23.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 shutdown_connections 2024-05-09T05:38:23.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb80b2b00 0x7fceb80b5010 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:23.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcecc108780 0x7fcecc199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:23.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc103550 0x7fcecc19f800 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:23.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 --2- 172.21.15.115:0/1867557035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcecc107e30 0x7fcecc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:23.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 >> 172.21.15.115:0/1867557035 conn(0x7fcecc078e00 msgr2=0x7fcecc100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:23.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 shutdown_connections 2024-05-09T05:38:23.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:23.162+0000 7fced383d700 1 -- 172.21.15.115:0/1867557035 wait complete. 2024-05-09T05:38:23.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:23 smithi195 ceph-mon[33844]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-09T05:38:23.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:23 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1867557035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:38:23.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:23 smithi115 ceph-mon[30349]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-09T05:38:23.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:23 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1867557035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:38:23.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:23 smithi161 ceph-mon[33805]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-09T05:38:23.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:23 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1867557035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:38:23.963 DEBUG:tasks.fwd_scrub:Yielding 2024-05-09T05:38:23.963 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T05:38:23.973 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T05:38:23.973 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T05:38:23.973 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-09T05:38:23.973 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-09T05:38:23.974 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:38:23.974 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:38:24.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.104+0000 7fb5203f3640 1 -- 172.21.15.115:0/1535711934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108ba0 msgr2=0x7fb518102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.104+0000 7fb5203f3640 1 --2- 172.21.15.115:0/1535711934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108ba0 0x7fb518102740 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb508009860 tx=0x7fb50802a340 comp rx=0 tx=0).stop 2024-05-09T05:38:24.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.105+0000 7fb5203f3640 1 -- 172.21.15.115:0/1535711934 shutdown_connections 2024-05-09T05:38:24.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.105+0000 7fb5203f3640 1 --2- 172.21.15.115:0/1535711934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5181032f0 0x7fb51810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.105+0000 7fb5203f3640 1 --2- 172.21.15.115:0/1535711934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518109040 0x7fb518102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.105+0000 7fb5203f3640 1 --2- 172.21.15.115:0/1535711934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb518108ba0 0x7fb518102740 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.105+0000 7fb5203f3640 1 -- 172.21.15.115:0/1535711934 >> 172.21.15.115:0/1535711934 conn(0x7fb5180782e0 msgr2=0x7fb518078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:24.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.105+0000 7fb5203f3640 1 -- 172.21.15.115:0/1535711934 shutdown_connections 2024-05-09T05:38:24.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.105+0000 7fb5203f3640 1 -- 172.21.15.115:0/1535711934 wait complete. 2024-05-09T05:38:24.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.105+0000 7fb5203f3640 1 Processor -- start 2024-05-09T05:38:24.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.105+0000 7fb5203f3640 1 -- start start 2024-05-09T05:38:24.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb5203f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5181032f0 0x7fb518199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb5203f3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb518108ba0 0x7fb518199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb5203f3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518109040 0x7fb51819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb5203f3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb518112c40 con 0x7fb5181032f0 2024-05-09T05:38:24.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb5203f3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb518112ac0 con 0x7fb518109040 2024-05-09T05:38:24.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb5203f3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb518112dc0 con 0x7fb518108ba0 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e18f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5181032f0 0x7fb518199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e18f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5181032f0 0x7fb518199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39076/0 (socket says 172.21.15.115:39076) 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e18f640 1 -- 172.21.15.115:0/880258699 learned_addr learned my addr 172.21.15.115:0/880258699 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51d98e640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb518108ba0 0x7fb518199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e990640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518109040 0x7fb51819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e18f640 1 -- 172.21.15.115:0/880258699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb518108ba0 msgr2=0x7fb518199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e18f640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb518108ba0 0x7fb518199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e18f640 1 -- 172.21.15.115:0/880258699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518109040 msgr2=0x7fb51819df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e18f640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518109040 0x7fb51819df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e18f640 1 -- 172.21.15.115:0/880258699 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51819e560 con 0x7fb5181032f0 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e990640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518109040 0x7fb51819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb51e18f640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5181032f0 0x7fb518199700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb508035040 tx=0x7fb508032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb50f7fe640 1 -- 172.21.15.115:0/880258699 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508032700 con 0x7fb5181032f0 2024-05-09T05:38:24.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb50f7fe640 1 -- 172.21.15.115:0/880258699 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb508030340 con 0x7fb5181032f0 2024-05-09T05:38:24.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb50f7fe640 1 -- 172.21.15.115:0/880258699 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508003850 con 0x7fb5181032f0 2024-05-09T05:38:24.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.106+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51819e7f0 con 0x7fb5181032f0 2024-05-09T05:38:24.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.107+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51819ed30 con 0x7fb5181032f0 2024-05-09T05:38:24.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.108+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51806a240 con 0x7fb5181032f0 2024-05-09T05:38:24.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.109+0000 7fb50f7fe640 1 -- 172.21.15.115:0/880258699 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fb508004050 con 0x7fb5181032f0 2024-05-09T05:38:24.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.110+0000 7fb50f7fe640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5040b2a60 0x7fb5040b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.110+0000 7fb50f7fe640 1 -- 172.21.15.115:0/880258699 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fb508104270 con 0x7fb5181032f0 2024-05-09T05:38:24.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.110+0000 7fb51d98e640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5040b2a60 0x7fb5040b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.110+0000 7fb51d98e640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5040b2a60 0x7fb5040b4f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb518100870 tx=0x7fb514009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:24.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.115+0000 7fb50f7fe640 1 -- 172.21.15.115:0/880258699 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb50803f070 con 0x7fb5181032f0 2024-05-09T05:38:24.231 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:24.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.312+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fb51819aa50 con 0x7fb5181032f0 2024-05-09T05:38:24.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.316+0000 7fb50f7fe640 1 -- 172.21.15.115:0/880258699 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v25)=0 v25) ==== 129+0+0 (secure 0 0 0) 0x7fb5080cb8b0 con 0x7fb5181032f0 2024-05-09T05:38:24.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.318+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5040b2a60 msgr2=0x7fb5040b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.318+0000 7fb5203f3640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5040b2a60 0x7fb5040b4f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb518100870 tx=0x7fb514009260 comp rx=0 tx=0).stop 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.318+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5181032f0 msgr2=0x7fb518199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.318+0000 7fb5203f3640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5181032f0 0x7fb518199700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb508035040 tx=0x7fb508032cb0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.318+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 shutdown_connections 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.318+0000 7fb5203f3640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5040b2a60 0x7fb5040b4f70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.318+0000 7fb5203f3640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb518109040 0x7fb51819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.319+0000 7fb5203f3640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb518108ba0 0x7fb518199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.319+0000 7fb5203f3640 1 --2- 172.21.15.115:0/880258699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5181032f0 0x7fb518199700 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.319+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 >> 172.21.15.115:0/880258699 conn(0x7fb5180782e0 msgr2=0x7fb518100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.319+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 shutdown_connections 2024-05-09T05:38:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.319+0000 7fb5203f3640 1 -- 172.21.15.115:0/880258699 wait complete. 2024-05-09T05:38:24.324 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-09T05:38:24.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.446+0000 7f9db43f3640 1 -- 172.21.15.115:0/952867445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac102580 msgr2=0x7f9dac102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.446+0000 7f9db43f3640 1 --2- 172.21.15.115:0/952867445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac102580 0x7f9dac102990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009900 tx=0x7f9da002a410 comp rx=0 tx=0).stop 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.446+0000 7f9db43f3640 1 -- 172.21.15.115:0/952867445 shutdown_connections 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.446+0000 7f9db43f3640 1 --2- 172.21.15.115:0/952867445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dac1075d0 0x7f9dac10ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.446+0000 7f9db43f3640 1 --2- 172.21.15.115:0/952867445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dac106b10 0x7f9dac106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.446+0000 7f9db43f3640 1 --2- 172.21.15.115:0/952867445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac102580 0x7f9dac102990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.446+0000 7f9db43f3640 1 -- 172.21.15.115:0/952867445 >> 172.21.15.115:0/952867445 conn(0x7f9dac0fd9e0 msgr2=0x7f9dac0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.446+0000 7f9db43f3640 1 -- 172.21.15.115:0/952867445 shutdown_connections 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.446+0000 7f9db43f3640 1 -- 172.21.15.115:0/952867445 wait complete. 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.447+0000 7f9db43f3640 1 Processor -- start 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.447+0000 7f9db43f3640 1 -- start start 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.447+0000 7f9db43f3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dac102580 0x7f9dac19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.447+0000 7f9db43f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac106b10 0x7f9dac19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.447+0000 7f9db43f3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dac1075d0 0x7f9dac19c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.447+0000 7f9db43f3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac112c40 con 0x7f9dac106b10 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.447+0000 7f9db43f3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac112ac0 con 0x7f9dac1075d0 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.447+0000 7f9db43f3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac112dc0 con 0x7f9dac102580 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db198e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac106b10 0x7f9dac19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db198e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac106b10 0x7f9dac19bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39092/0 (socket says 172.21.15.115:39092) 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db198e640 1 -- 172.21.15.115:0/646679346 learned_addr learned my addr 172.21.15.115:0/646679346 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db218f640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dac102580 0x7f9dac19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db2990640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dac1075d0 0x7f9dac19c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db198e640 1 -- 172.21.15.115:0/646679346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dac102580 msgr2=0x7f9dac19b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db198e640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dac102580 0x7f9dac19b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db198e640 1 -- 172.21.15.115:0/646679346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dac1075d0 msgr2=0x7f9dac19c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db198e640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dac1075d0 0x7f9dac19c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db198e640 1 -- 172.21.15.115:0/646679346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac1a6940 con 0x7f9dac106b10 2024-05-09T05:38:24.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db198e640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac106b10 0x7f9dac19bb40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e9c0 tx=0x7f9da800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:24.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9d9f7fe640 1 -- 172.21.15.115:0/646679346 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da800cd80 con 0x7f9dac106b10 2024-05-09T05:38:24.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac1a6c30 con 0x7f9dac106b10 2024-05-09T05:38:24.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9d9f7fe640 1 -- 172.21.15.115:0/646679346 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9da80054b0 con 0x7f9dac106b10 2024-05-09T05:38:24.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9d9f7fe640 1 -- 172.21.15.115:0/646679346 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8010430 con 0x7f9dac106b10 2024-05-09T05:38:24.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.448+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac1a71d0 con 0x7f9dac106b10 2024-05-09T05:38:24.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.449+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac050870 con 0x7f9dac106b10 2024-05-09T05:38:24.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.451+0000 7f9d9f7fe640 1 -- 172.21.15.115:0/646679346 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f9da8004f20 con 0x7f9dac106b10 2024-05-09T05:38:24.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.452+0000 7f9d9f7fe640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d980b29e0 0x7f9d980b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.452+0000 7f9db218f640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d980b29e0 0x7f9d980b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.452+0000 7f9d9f7fe640 1 -- 172.21.15.115:0/646679346 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9da80e3680 con 0x7f9dac106b10 2024-05-09T05:38:24.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.453+0000 7f9db218f640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d980b29e0 0x7f9d980b4ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9da00057c0 tx=0x7f9da00058d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:24.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.455+0000 7f9d9f7fe640 1 -- 172.21.15.115:0/646679346 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9da80aabb0 con 0x7f9dac106b10 2024-05-09T05:38:24.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.646+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f9dac111560 con 0x7f9dac106b10 2024-05-09T05:38:24.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.651+0000 7f9d9f7fe640 1 -- 172.21.15.115:0/646679346 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v26)=0 v26) ==== 138+0+0 (secure 0 0 0) 0x7f9da80aa4b0 con 0x7f9dac106b10 2024-05-09T05:38:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d980b29e0 msgr2=0x7f9d980b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d980b29e0 0x7f9d980b4ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9da00057c0 tx=0x7f9da00058d0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac106b10 msgr2=0x7f9dac19bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac106b10 0x7f9dac19bb40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e9c0 tx=0x7f9da800ee80 comp rx=0 tx=0).stop 2024-05-09T05:38:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 shutdown_connections 2024-05-09T05:38:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d980b29e0 0x7f9d980b4ef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dac1075d0 0x7f9dac19c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dac106b10 0x7f9dac19bb40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 --2- 172.21.15.115:0/646679346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dac102580 0x7f9dac19b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 >> 172.21.15.115:0/646679346 conn(0x7f9dac0fd9e0 msgr2=0x7f9dac10dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:24.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 shutdown_connections 2024-05-09T05:38:24.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.654+0000 7f9db43f3640 1 -- 172.21.15.115:0/646679346 wait complete. 2024-05-09T05:38:24.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-09T05:38:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.782+0000 7f79a6a42640 1 -- 172.21.15.115:0/1712744439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a01065f0 msgr2=0x7f79a01129c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.782+0000 7f79a6a42640 1 --2- 172.21.15.115:0/1712744439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a01065f0 0x7f79a01129c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7998009ca0 tx=0x7f799802a8c0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.782+0000 7f79a6a42640 1 -- 172.21.15.115:0/1712744439 shutdown_connections 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.782+0000 7f79a6a42640 1 --2- 172.21.15.115:0/1712744439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a01065f0 0x7f79a01129c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.782+0000 7f79a6a42640 1 --2- 172.21.15.115:0/1712744439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79a010c340 0x7f79a0105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.782+0000 7f79a6a42640 1 --2- 172.21.15.115:0/1712744439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79a010bea0 0x7f79a0105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.782+0000 7f79a6a42640 1 -- 172.21.15.115:0/1712744439 >> 172.21.15.115:0/1712744439 conn(0x7f79a0077130 msgr2=0x7f79a0077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.782+0000 7f79a6a42640 1 -- 172.21.15.115:0/1712744439 shutdown_connections 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.782+0000 7f79a6a42640 1 -- 172.21.15.115:0/1712744439 wait complete. 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a6a42640 1 Processor -- start 2024-05-09T05:38:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a6a42640 1 -- start start 2024-05-09T05:38:24.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a6a42640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79a01065f0 0x7f79a01a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a6a42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79a010bea0 0x7f79a01a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a6a42640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a010c340 0x7f79a019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a6a42640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0115fb0 con 0x7f79a010c340 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a6a42640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0115e30 con 0x7f79a01065f0 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a6a42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0116130 con 0x7f79a010bea0 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a4fdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a010c340 0x7f79a019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a4fdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a010c340 0x7f79a019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39112/0 (socket says 172.21.15.115:39112) 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f79a4fdf640 1 -- 172.21.15.115:0/862280112 learned_addr learned my addr 172.21.15.115:0/862280112 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f799f7fe640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79a010bea0 0x7f79a01a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.783+0000 7f799ffff640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79a01065f0 0x7f79a01a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f79a4fdf640 1 -- 172.21.15.115:0/862280112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79a010bea0 msgr2=0x7f79a01a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f79a4fdf640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79a010bea0 0x7f79a01a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f79a4fdf640 1 -- 172.21.15.115:0/862280112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79a01065f0 msgr2=0x7f79a01a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f79a4fdf640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79a01065f0 0x7f79a01a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f79a4fdf640 1 -- 172.21.15.115:0/862280112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a019ced0 con 0x7f79a010c340 2024-05-09T05:38:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f79a4fdf640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a010c340 0x7f79a019c7f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7998005cc0 tx=0x7f7998003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f799d7fa640 1 -- 172.21.15.115:0/862280112 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998032c10 con 0x7f79a010c340 2024-05-09T05:38:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a019d160 con 0x7f79a010c340 2024-05-09T05:38:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f799d7fa640 1 -- 172.21.15.115:0/862280112 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7998032db0 con 0x7f79a010c340 2024-05-09T05:38:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f799d7fa640 1 -- 172.21.15.115:0/862280112 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998030a60 con 0x7f79a010c340 2024-05-09T05:38:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.784+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a01a9410 con 0x7f79a010c340 2024-05-09T05:38:24.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.785+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a0107180 con 0x7f79a010c340 2024-05-09T05:38:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.787+0000 7f799d7fa640 1 -- 172.21.15.115:0/862280112 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f7998030c00 con 0x7f79a010c340 2024-05-09T05:38:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.787+0000 7f799d7fa640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f79880b29e0 0x7f79880b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.787+0000 7f799ffff640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f79880b29e0 0x7f79880b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.787+0000 7f799d7fa640 1 -- 172.21.15.115:0/862280112 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7998103ee0 con 0x7f79a010c340 2024-05-09T05:38:24.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.788+0000 7f799ffff640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f79880b29e0 0x7f79880b4ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f79a019db20 tx=0x7f7990005f90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:24.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.791+0000 7f799d7fa640 1 -- 172.21.15.115:0/862280112 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79980cb520 con 0x7f79a010c340 2024-05-09T05:38:24.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.979+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f79a010dfb0 con 0x7f79a010c340 2024-05-09T05:38:24.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.984+0000 7f799d7fa640 1 -- 172.21.15.115:0/862280112 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v27)=0 v27) ==== 133+0+0 (secure 0 0 0) 0x7f79980098b0 con 0x7f79a010c340 2024-05-09T05:38:24.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f79880b29e0 msgr2=0x7f79880b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f79880b29e0 0x7f79880b4ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f79a019db20 tx=0x7f7990005f90 comp rx=0 tx=0).stop 2024-05-09T05:38:24.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a010c340 msgr2=0x7f79a019c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:24.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a010c340 0x7f79a019c7f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7998005cc0 tx=0x7f7998003f90 comp rx=0 tx=0).stop 2024-05-09T05:38:24.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 shutdown_connections 2024-05-09T05:38:24.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f79880b29e0 0x7f79880b4ef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a010c340 0x7f79a019c7f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79a010bea0 0x7f79a01a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 --2- 172.21.15.115:0/862280112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79a01065f0 0x7f79a01a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:24.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 >> 172.21.15.115:0/862280112 conn(0x7f79a0077130 msgr2=0x7f79a01032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:24.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 shutdown_connections 2024-05-09T05:38:24.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:24.986+0000 7f79a6a42640 1 -- 172.21.15.115:0/862280112 wait complete. 2024-05-09T05:38:24.991 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-09T05:38:25.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.115+0000 7efe68e2b640 1 -- 172.21.15.115:0/861528660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64108dc0 msgr2=0x7efe64102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.115+0000 7efe68e2b640 1 --2- 172.21.15.115:0/861528660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64108dc0 0x7efe64102e80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efe54009900 tx=0x7efe5402a410 comp rx=0 tx=0).stop 2024-05-09T05:38:25.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.115+0000 7efe68e2b640 1 -- 172.21.15.115:0/861528660 shutdown_connections 2024-05-09T05:38:25.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.115+0000 7efe68e2b640 1 --2- 172.21.15.115:0/861528660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe64103a30 0x7efe6410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.115+0000 7efe68e2b640 1 --2- 172.21.15.115:0/861528660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64109260 0x7efe641033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.115+0000 7efe68e2b640 1 --2- 172.21.15.115:0/861528660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64108dc0 0x7efe64102e80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.115+0000 7efe68e2b640 1 -- 172.21.15.115:0/861528660 >> 172.21.15.115:0/861528660 conn(0x7efe640779e0 msgr2=0x7efe64077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:25.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.116+0000 7efe68e2b640 1 -- 172.21.15.115:0/861528660 shutdown_connections 2024-05-09T05:38:25.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.116+0000 7efe68e2b640 1 -- 172.21.15.115:0/861528660 wait complete. 2024-05-09T05:38:25.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.116+0000 7efe68e2b640 1 Processor -- start 2024-05-09T05:38:25.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.116+0000 7efe68e2b640 1 -- start start 2024-05-09T05:38:25.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe68e2b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64103a30 0x7efe6419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe68e2b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe64108dc0 0x7efe6419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe68e2b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64109260 0x7efe6419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe68e2b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe64112e50 con 0x7efe64109260 2024-05-09T05:38:25.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe68e2b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe64112cd0 con 0x7efe64103a30 2024-05-09T05:38:25.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe68e2b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe64112fd0 con 0x7efe64108dc0 2024-05-09T05:38:25.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe62d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64109260 0x7efe6419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe62d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64109260 0x7efe6419c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39122/0 (socket says 172.21.15.115:39122) 2024-05-09T05:38:25.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe62d9d640 1 -- 172.21.15.115:0/3891102171 learned_addr learned my addr 172.21.15.115:0/3891102171 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:25.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe6259c640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64103a30 0x7efe6419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe61d9b640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe64108dc0 0x7efe6419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe62d9d640 1 -- 172.21.15.115:0/3891102171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe64108dc0 msgr2=0x7efe6419bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe62d9d640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe64108dc0 0x7efe6419bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe62d9d640 1 -- 172.21.15.115:0/3891102171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64103a30 msgr2=0x7efe6419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe62d9d640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64103a30 0x7efe6419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe62d9d640 1 -- 172.21.15.115:0/3891102171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe641a6ae0 con 0x7efe64109260 2024-05-09T05:38:25.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe62d9d640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64109260 0x7efe6419c330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00cce0 tx=0x7efe5c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:25.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe537fe640 1 -- 172.21.15.115:0/3891102171 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00f450 con 0x7efe64109260 2024-05-09T05:38:25.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe641a6dd0 con 0x7efe64109260 2024-05-09T05:38:25.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.117+0000 7efe537fe640 1 -- 172.21.15.115:0/3891102171 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe5c00fa80 con 0x7efe64109260 2024-05-09T05:38:25.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.118+0000 7efe537fe640 1 -- 172.21.15.115:0/3891102171 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c018ad0 con 0x7efe64109260 2024-05-09T05:38:25.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.118+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe641a7210 con 0x7efe64109260 2024-05-09T05:38:25.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.118+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe64050870 con 0x7efe64109260 2024-05-09T05:38:25.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.120+0000 7efe537fe640 1 -- 172.21.15.115:0/3891102171 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7efe5c00f5f0 con 0x7efe64109260 2024-05-09T05:38:25.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.121+0000 7efe537fe640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe4c0b29e0 0x7efe4c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.121+0000 7efe6259c640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe4c0b29e0 0x7efe4c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.121+0000 7efe537fe640 1 -- 172.21.15.115:0/3891102171 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efe5c0e3380 con 0x7efe64109260 2024-05-09T05:38:25.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.121+0000 7efe6259c640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe4c0b29e0 0x7efe4c0b4ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efe54004770 tx=0x7efe540058d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:25.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.124+0000 7efe537fe640 1 -- 172.21.15.115:0/3891102171 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe5c014030 con 0x7efe64109260 2024-05-09T05:38:25.309 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-09T05:38:25.309 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/880258699' entity='client.admin' 2024-05-09T05:38:25.309 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:25.309 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:25.309 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:25.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.338+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7efe64063110 con 0x7efe4c0b29e0 2024-05-09T05:38:25.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.401+0000 7efe537fe640 1 -- 172.21.15.115:0/3891102171 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efe64063110 con 0x7efe4c0b29e0 2024-05-09T05:38:25.395 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path / 2024-05-09T05:38:25.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.403+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe4c0b29e0 msgr2=0x7efe4c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe4c0b29e0 0x7efe4c0b4ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efe54004770 tx=0x7efe540058d0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64109260 msgr2=0x7efe6419c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64109260 0x7efe6419c330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00cce0 tx=0x7efe5c00a620 comp rx=0 tx=0).stop 2024-05-09T05:38:25.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 shutdown_connections 2024-05-09T05:38:25.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe4c0b29e0 0x7efe4c0b4ef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe64109260 0x7efe6419c330 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe64108dc0 0x7efe6419bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 --2- 172.21.15.115:0/3891102171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64103a30 0x7efe6419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 >> 172.21.15.115:0/3891102171 conn(0x7efe640779e0 msgr2=0x7efe64104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:25.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 shutdown_connections 2024-05-09T05:38:25.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.404+0000 7efe68e2b640 1 -- 172.21.15.115:0/3891102171 wait complete. 2024-05-09T05:38:25.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-09T05:38:25.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 -- 172.21.15.115:0/1887047204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 msgr2=0x7fa984102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 --2- 172.21.15.115:0/1887047204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 0x7fa984102c80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa974009900 tx=0x7fa97402a410 comp rx=0 tx=0).stop 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 -- 172.21.15.115:0/1887047204 shutdown_connections 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 --2- 172.21.15.115:0/1887047204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9841032f0 0x7fa98410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 --2- 172.21.15.115:0/1887047204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 0x7fa984102c80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 --2- 172.21.15.115:0/1887047204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa984108ba0 0x7fa984102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 -- 172.21.15.115:0/1887047204 >> 172.21.15.115:0/1887047204 conn(0x7fa9840782e0 msgr2=0x7fa984078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 -- 172.21.15.115:0/1887047204 shutdown_connections 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 -- 172.21.15.115:0/1887047204 wait complete. 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 Processor -- start 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.539+0000 7fa98be8b640 1 -- start start 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98be8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9841032f0 0x7fa98419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98be8b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa984108ba0 0x7fa98419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98be8b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 0x7fa98419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98be8b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa984112c40 con 0x7fa984109040 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98be8b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa984112ac0 con 0x7fa984108ba0 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98be8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa984112dc0 con 0x7fa9841032f0 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98a428640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 0x7fa98419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98a428640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 0x7fa98419c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39144/0 (socket says 172.21.15.115:39144) 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98a428640 1 -- 172.21.15.115:0/2810713859 learned_addr learned my addr 172.21.15.115:0/2810713859 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa989426640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa984108ba0 0x7fa98419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa989c27640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9841032f0 0x7fa98419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98a428640 1 -- 172.21.15.115:0/2810713859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9841032f0 msgr2=0x7fa98419b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98a428640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9841032f0 0x7fa98419b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98a428640 1 -- 172.21.15.115:0/2810713859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa984108ba0 msgr2=0x7fa98419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98a428640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa984108ba0 0x7fa98419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98a428640 1 -- 172.21.15.115:0/2810713859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9841a6950 con 0x7fa984109040 2024-05-09T05:38:25.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa989426640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa984108ba0 0x7fa98419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:25.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa989c27640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9841032f0 0x7fa98419b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:25.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa98a428640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 0x7fa98419c0c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00eb10 tx=0x7fa97c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:25.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.540+0000 7fa97affd640 1 -- 172.21.15.115:0/2810713859 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c004b20 con 0x7fa984109040 2024-05-09T05:38:25.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.541+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9841a6c40 con 0x7fa984109040 2024-05-09T05:38:25.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.541+0000 7fa97affd640 1 -- 172.21.15.115:0/2810713859 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa97c004cc0 con 0x7fa984109040 2024-05-09T05:38:25.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.541+0000 7fa97affd640 1 -- 172.21.15.115:0/2810713859 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c010690 con 0x7fa984109040 2024-05-09T05:38:25.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.541+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9841a71b0 con 0x7fa984109040 2024-05-09T05:38:25.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.542+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98406a240 con 0x7fa984109040 2024-05-09T05:38:25.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.544+0000 7fa97affd640 1 -- 172.21.15.115:0/2810713859 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fa97c005000 con 0x7fa984109040 2024-05-09T05:38:25.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.544+0000 7fa97affd640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9700b29e0 0x7fa9700b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.545+0000 7fa989c27640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9700b29e0 0x7fa9700b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.545+0000 7fa97affd640 1 -- 172.21.15.115:0/2810713859 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa97c0e33a0 con 0x7fa984109040 2024-05-09T05:38:25.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.545+0000 7fa989c27640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9700b29e0 0x7fa9700b4ef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa980005e00 tx=0x7fa980005c70 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:25.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.548+0000 7fa97affd640 1 -- 172.21.15.115:0/2810713859 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa97c054df0 con 0x7fa984109040 2024-05-09T05:38:25.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/880258699' entity='client.admin' 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/646679346' entity='client.admin' 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/862280112' entity='client.admin' 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:25.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:25.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:25.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:25.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/646679346' entity='client.admin' 2024-05-09T05:38:25.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:25.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:25.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:25.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:25.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/862280112' entity='client.admin' 2024-05-09T05:38:25.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:25.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:25.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:25.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-09T05:38:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/880258699' entity='client.admin' 2024-05-09T05:38:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/646679346' entity='client.admin' 2024-05-09T05:38:25.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/862280112' entity='client.admin' 2024-05-09T05:38:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:38:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:38:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:38:25.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:38:25.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.763+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fa9841019d0 con 0x7fa9700b29e0 2024-05-09T05:38:25.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.826+0000 7fa97affd640 1 -- 172.21.15.115:0/2810713859 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fa9841019d0 con 0x7fa9700b29e0 2024-05-09T05:38:25.819 INFO:teuthology.orchestra.run.smithi115.stdout:Retention added to path / 2024-05-09T05:38:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9700b29e0 msgr2=0x7fa9700b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9700b29e0 0x7fa9700b4ef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa980005e00 tx=0x7fa980005c70 comp rx=0 tx=0).stop 2024-05-09T05:38:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 msgr2=0x7fa98419c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 0x7fa98419c0c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00eb10 tx=0x7fa97c00c2d0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 shutdown_connections 2024-05-09T05:38:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9700b29e0 0x7fa9700b4ef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa984109040 0x7fa98419c0c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa984108ba0 0x7fa98419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 --2- 172.21.15.115:0/2810713859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9841032f0 0x7fa98419b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 >> 172.21.15.115:0/2810713859 conn(0x7fa9840782e0 msgr2=0x7fa9841012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 shutdown_connections 2024-05-09T05:38:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.828+0000 7fa98be8b640 1 -- 172.21.15.115:0/2810713859 wait complete. 2024-05-09T05:38:25.834 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-09T05:38:25.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.955+0000 7f7e75784640 1 -- 172.21.15.115:0/580099438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 msgr2=0x7f7e70102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.955+0000 7f7e75784640 1 --2- 172.21.15.115:0/580099438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 0x7f7e70102760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009900 tx=0x7f7e6002a410 comp rx=0 tx=0).stop 2024-05-09T05:38:25.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.955+0000 7f7e75784640 1 -- 172.21.15.115:0/580099438 shutdown_connections 2024-05-09T05:38:25.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.955+0000 7f7e75784640 1 --2- 172.21.15.115:0/580099438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e70103310 0x7f7e7010f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.955+0000 7f7e75784640 1 --2- 172.21.15.115:0/580099438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e70109060 0x7f7e70102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.955+0000 7f7e75784640 1 --2- 172.21.15.115:0/580099438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 0x7f7e70102760 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.955+0000 7f7e75784640 1 -- 172.21.15.115:0/580099438 >> 172.21.15.115:0/580099438 conn(0x7f7e700fe390 msgr2=0x7f7e701007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:25.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.956+0000 7f7e75784640 1 -- 172.21.15.115:0/580099438 shutdown_connections 2024-05-09T05:38:25.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.956+0000 7f7e75784640 1 -- 172.21.15.115:0/580099438 wait complete. 2024-05-09T05:38:25.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.956+0000 7f7e75784640 1 Processor -- start 2024-05-09T05:38:25.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.956+0000 7f7e75784640 1 -- start start 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.956+0000 7f7e75784640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e70103310 0x7f7e7019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.956+0000 7f7e75784640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 0x7f7e7019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e75784640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e70109060 0x7f7e7019c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e75784640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70112c60 con 0x7f7e70108bc0 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e75784640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70112ae0 con 0x7f7e70103310 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e75784640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70112de0 con 0x7f7e70109060 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e67fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 0x7f7e7019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e67fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 0x7f7e7019bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39166/0 (socket says 172.21.15.115:39166) 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e67fff640 1 -- 172.21.15.115:0/1460159104 learned_addr learned my addr 172.21.15.115:0/1460159104 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:25.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e6f7fe640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e70109060 0x7f7e7019c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e6effd640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e70103310 0x7f7e7019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e67fff640 1 -- 172.21.15.115:0/1460159104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e70109060 msgr2=0x7f7e7019c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e67fff640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e70109060 0x7f7e7019c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e67fff640 1 -- 172.21.15.115:0/1460159104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e70103310 msgr2=0x7f7e7019b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e67fff640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e70103310 0x7f7e7019b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e67fff640 1 -- 172.21.15.115:0/1460159104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e701a6980 con 0x7f7e70108bc0 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e6f7fe640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e70109060 0x7f7e7019c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e67fff640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 0x7f7e7019bc00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800e9f0 tx=0x7f7e5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e6cff9640 1 -- 172.21.15.115:0/1460159104 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800ccc0 con 0x7f7e70108bc0 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e6cff9640 1 -- 172.21.15.115:0/1460159104 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e580044b0 con 0x7f7e70108bc0 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e701a6c70 con 0x7f7e70108bc0 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e6cff9640 1 -- 172.21.15.115:0/1460159104 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58010430 con 0x7f7e70108bc0 2024-05-09T05:38:25.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.957+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e701a71c0 con 0x7f7e70108bc0 2024-05-09T05:38:25.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.958+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e70050870 con 0x7f7e70108bc0 2024-05-09T05:38:25.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.960+0000 7f7e6cff9640 1 -- 172.21.15.115:0/1460159104 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f7e58003f90 con 0x7f7e70108bc0 2024-05-09T05:38:25.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.960+0000 7f7e6cff9640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e5c0b29e0 0x7f7e5c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:25.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.961+0000 7f7e6effd640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e5c0b29e0 0x7f7e5c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:25.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.961+0000 7f7e6cff9640 1 -- 172.21.15.115:0/1460159104 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7e580e3010 con 0x7f7e70108bc0 2024-05-09T05:38:25.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.961+0000 7f7e6effd640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e5c0b29e0 0x7f7e5c0b4ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006010 tx=0x7f7e60032ba0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:25.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:25.964+0000 7f7e6cff9640 1 -- 172.21.15.115:0/1460159104 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e580aa5d0 con 0x7f7e70108bc0 2024-05-09T05:38:26.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.095+0000 7fbdec889700 1 -- 172.21.15.115:0/2805065767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde8107df0 msgr2=0x7fbde8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.095+0000 7fbdec889700 1 --2- 172.21.15.115:0/2805065767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde8107df0 0x7fbde8108200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0099c0 tx=0x7fbddc009cd0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.095+0000 7fbdec889700 1 -- 172.21.15.115:0/2805065767 shutdown_connections 2024-05-09T05:38:26.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.095+0000 7fbdec889700 1 --2- 172.21.15.115:0/2805065767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde81034c0 0x7fbde81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.095+0000 7fbdec889700 1 --2- 172.21.15.115:0/2805065767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbde8108740 0x7fbde81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.095+0000 7fbdec889700 1 --2- 172.21.15.115:0/2805065767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde8107df0 0x7fbde8108200 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.095+0000 7fbdec889700 1 -- 172.21.15.115:0/2805065767 >> 172.21.15.115:0/2805065767 conn(0x7fbde80fedc0 msgr2=0x7fbde81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:26.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.095+0000 7fbdec889700 1 -- 172.21.15.115:0/2805065767 shutdown_connections 2024-05-09T05:38:26.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.095+0000 7fbdec889700 1 -- 172.21.15.115:0/2805065767 wait complete. 2024-05-09T05:38:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.096+0000 7fbdec889700 1 Processor -- start 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.096+0000 7fbdec889700 1 -- start start 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.096+0000 7fbdec889700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde81034c0 0x7fbde81155c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.096+0000 7fbdec889700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde8107df0 0x7fbde8113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.096+0000 7fbdec889700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbde8108740 0x7fbde8113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.096+0000 7fbdec889700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8113200 con 0x7fbde81034c0 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.096+0000 7fbdec889700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8113080 con 0x7fbde8107df0 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.096+0000 7fbdec889700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8113380 con 0x7fbde8108740 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde81034c0 0x7fbde81155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde81034c0 0x7fbde81155c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39186/0 (socket says 172.21.15.115:39186) 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde8107df0 0x7fbde8113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde659c700 1 -- 172.21.15.115:0/3403824163 learned_addr learned my addr 172.21.15.115:0/3403824163 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbde8108740 0x7fbde8113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde659c700 1 -- 172.21.15.115:0/3403824163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbde8108740 msgr2=0x7fbde8113fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde659c700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbde8108740 0x7fbde8113fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde659c700 1 -- 172.21.15.115:0/3403824163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde8107df0 msgr2=0x7fbde8113a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde659c700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde8107df0 0x7fbde8113a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde659c700 1 -- 172.21.15.115:0/3403824163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde81147d0 con 0x7fbde81034c0 2024-05-09T05:38:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbde8108740 0x7fbde8113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbde659c700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde81034c0 0x7fbde81155c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800d950 tx=0x7fbdd800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbdd77fe700 1 -- 172.21.15.115:0/3403824163 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8016070 con 0x7fbde81034c0 2024-05-09T05:38:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbdec889700 1 -- 172.21.15.115:0/3403824163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde81ad7c0 con 0x7fbde81034c0 2024-05-09T05:38:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbdd77fe700 1 -- 172.21.15.115:0/3403824163 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdd8009dd0 con 0x7fbde81034c0 2024-05-09T05:38:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbdd77fe700 1 -- 172.21.15.115:0/3403824163 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800f3c0 con 0x7fbde81034c0 2024-05-09T05:38:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.097+0000 7fbdec889700 1 -- 172.21.15.115:0/3403824163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde81add20 con 0x7fbde81034c0 2024-05-09T05:38:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.100+0000 7fbdd77fe700 1 -- 172.21.15.115:0/3403824163 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fbdd8019070 con 0x7fbde81034c0 2024-05-09T05:38:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.101+0000 7fbdfb29fb80 1 Processor -- start 2024-05-09T05:38:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.101+0000 7fbdd77fe700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbdd00b2a80 0x7fbdd00b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbdd00b2a80 0x7fbdd00b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbdfb29fb80 1 -- start start 2024-05-09T05:38:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbdd77fe700 1 -- 172.21.15.115:0/3403824163 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbdd80e27a0 con 0x7fbde81034c0 2024-05-09T05:38:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e6152520 0x55c2e62d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e62cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c2e61f2070 con 0x55c2e6152520 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2e6190870 con 0x55c2e62cf720 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2e61ee030 con 0x55c2e623c1e0 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbde5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e6152520 0x55c2e62d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbde5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e6152520 0x55c2e62d5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39198/0 (socket says 172.21.15.115:39198) 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbde5d9b700 1 -- 172.21.15.115:0/860415254 learned_addr learned my addr 172.21.15.115:0/860415254 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e62cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbde659c700 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.102+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbdd00b2a80 0x7fbdd00b4f90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009990 tx=0x7fbddc005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbde5d9b700 1 -- 172.21.15.115:0/860415254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 msgr2=0x55c2e62d5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbde5d9b700 1 -- 172.21.15.115:0/860415254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 msgr2=0x55c2e62cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e62cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbde5d9b700 1 -- 172.21.15.115:0/860415254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e61883f0 con 0x55c2e6152520 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e62cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e6152520 0x55c2e62d5ab0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0099c0 tx=0x7fbddc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b7cf73e44cb385 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbdd4ff9700 1 -- 172.21.15.115:0/860415254 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00bc30 con 0x55c2e6152520 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbdd4ff9700 1 -- 172.21.15.115:0/860415254 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbddc00bdd0 con 0x55c2e6152520 2024-05-09T05:38:26.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.103+0000 7fbdd4ff9700 1 -- 172.21.15.115:0/860415254 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc02f900 con 0x55c2e6152520 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.104+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/860415254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e6152520 msgr2=0x55c2e62d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.104+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e6152520 0x55c2e62d5ab0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0099c0 tx=0x7fbddc01e000 comp rx=0 tx=0).stop 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.104+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/860415254 shutdown_connections 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.104+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e62cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.104+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.104+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/860415254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e6152520 0x55c2e62d5ab0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.104+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/860415254 >> 172.21.15.115:0/860415254 conn(0x55c2e61b2720 msgr2=0x55c2e62ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.104+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/860415254 shutdown_connections 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.104+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/860415254 wait complete. 2024-05-09T05:38:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbdfb29fb80 1 -- start start 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbdfb29fb80 1 Processor -- start 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c2e606dc20 con 0x55c2e62cf720 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2e61ac1d0 con 0x55c2e6152520 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2e62dc250 con 0x55c2e623c1e0 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbdfb29fb80 10 client.0 _ll_register_callbacks cb 0x55c2e6063740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbde6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbde6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62d9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39202/0 (socket says 172.21.15.115:39202) 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.105+0000 7fbde6d9d700 1 -- 172.21.15.115:0/2047358051 learned_addr learned my addr 172.21.15.115:0/2047358051 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde659c700 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde6d9d700 1 -- 172.21.15.115:0/2047358051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 msgr2=0x55c2e62d9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde6d9d700 1 -- 172.21.15.115:0/2047358051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 msgr2=0x55c2e62d91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde6d9d700 1 -- 172.21.15.115:0/2047358051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e606d510 con 0x55c2e62cf720 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde659c700 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62d9c40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbde0007bf0 tx=0x7fbde0007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4fb2858b648d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbdcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2047358051 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde001a070 con 0x55c2e62cf720 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2047358051 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbde0012cc0 con 0x55c2e62cf720 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2047358051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbddc00bdd0 con 0x55c2e62cf720 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2047358051 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbde0026050 con 0x55c2e62cf720 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbdfb29fb80 10 client.15495 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:26.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.106+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2047358051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2e62e6cc0 con 0x55c2e62cf720 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.107+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2047358051 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbddc00bdd0 con 0x55c2e62cf720 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.107+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2047358051 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbde00153c0 con 0x55c2e62cf720 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.107+0000 7fbdfb29fb80 10 client.15495 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.107+0000 7fbdfb29fb80 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.107+0000 7fbdfb29fb80 20 client.15495 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.107+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2e62e9250 0x55c2e62eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.107+0000 7fbdfb29fb80 4 client.15495 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.107+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2047358051 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2e61ffa50 con 0x55c2e62e9250 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.107+0000 7fbde659c700 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2e62e9250 0x55c2e62eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.108+0000 7fbde659c700 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2e62e9250 0x55c2e62eb670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5fdc5df57a800a server_cookie=27bb919c77bbd8cb in_seq=0 out_seq=0 2024-05-09T05:38:26.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.108+0000 7fbdcdffb700 10 client.15495 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:26.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.108+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2047358051 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2e61ffa50 con 0x55c2e62e9250 2024-05-09T05:38:26.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.108+0000 7fbdcdffb700 10 client.15495 handle_command_reply: tid=0 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 client.15495 shutdown 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2047358051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 msgr2=0x55c2e62d9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62d9c40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbde0007bf0 tx=0x7fbde0007f00 comp rx=0 tx=0).stop 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2047358051 shutdown_connections 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2e62e9250 0x55c2e62eb670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62d9c40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2047358051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2047358051 >> 172.21.15.115:0/2047358051 conn(0x55c2e61b2720 msgr2=0x55c2e62ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdcdffb700 0 client.15495 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2047358051 shutdown_connections 2024-05-09T05:38:26.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2047358051 wait complete. 2024-05-09T05:38:26.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.109+0000 7fbdfb29fb80 20 client.15495 trim_cache size 0 max 16384 2024-05-09T05:38:26.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.129+0000 7fbdfb29fb80 1 Processor -- start 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.129+0000 7fbdfb29fb80 1 -- start start 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.129+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.129+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.129+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.129+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b68d0 con 0x55c2e62cf720 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.129+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b8410 con 0x55c2e6152520 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.129+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b6fa0 con 0x55c2e623c1e0 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39216/0 (socket says 172.21.15.115:39216) 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde6d9d700 1 -- 172.21.15.115:0/1630581307 learned_addr learned my addr 172.21.15.115:0/1630581307 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde659c700 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde6d9d700 1 -- 172.21.15.115:0/1630581307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 msgr2=0x55c2e62d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde6d9d700 1 -- 172.21.15.115:0/1630581307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 msgr2=0x55c2e62d6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde6d9d700 1 -- 172.21.15.115:0/1630581307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e62dba60 con 0x55c2e62cf720 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde659c700 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62cee20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbde0006050 tx=0x7fbde0015dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f432502cd59600a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbdcffff700 1 -- 172.21.15.115:0/1630581307 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde002f070 con 0x55c2e62cf720 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbdcffff700 1 -- 172.21.15.115:0/1630581307 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbde0015a00 con 0x55c2e62cf720 2024-05-09T05:38:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.130+0000 7fbdcffff700 1 -- 172.21.15.115:0/1630581307 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde001a070 con 0x55c2e62cf720 2024-05-09T05:38:26.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.131+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/1630581307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 msgr2=0x55c2e62cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.131+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62cee20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbde0006050 tx=0x7fbde0015dd0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.131+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/1630581307 shutdown_connections 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.131+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e6152520 0x55c2e62d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.131+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e62cf720 0x55c2e62cee20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.131+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/1630581307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e623c1e0 0x55c2e62d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.131+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/1630581307 >> 172.21.15.115:0/1630581307 conn(0x55c2e61b2720 msgr2=0x55c2e62ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.131+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/1630581307 shutdown_connections 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.131+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/1630581307 wait complete. 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbdfb29fb80 1 -- start start 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbdfb29fb80 1 Processor -- start 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e6152520 0x55c2e62f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e623c1e0 0x55c2e62f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbdfb29fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e63529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b7d40 con 0x55c2e623c1e0 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b7670 con 0x55c2e62cf720 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbdfb29fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b4870 con 0x55c2e6152520 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbdfb29fb80 10 client.0 _ll_register_callbacks cb 0x55c2e6063740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e623c1e0 0x55c2e62f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e623c1e0 0x55c2e62f0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39226/0 (socket says 172.21.15.115:39226) 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde5d9b700 1 -- 172.21.15.115:0/2711109798 learned_addr learned my addr 172.21.15.115:0/2711109798 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde659c700 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e6152520 0x55c2e62f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e63529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde5d9b700 1 -- 172.21.15.115:0/2711109798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e6152520 msgr2=0x55c2e62f09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e6152520 0x55c2e62f09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde5d9b700 1 -- 172.21.15.115:0/2711109798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 msgr2=0x55c2e63529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e63529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde5d9b700 1 -- 172.21.15.115:0/2711109798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e62f1740 con 0x55c2e623c1e0 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde659c700 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e6152520 0x55c2e62f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.132+0000 7fbde6d9d700 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e63529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbde5d9b700 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e623c1e0 0x55c2e62f0f20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbddc005290 tx=0x7fbddc02f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ef2e87f1f67638 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2711109798 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc003890 con 0x55c2e623c1e0 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2711109798 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbddc003a30 con 0x55c2e623c1e0 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2711109798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde0015a00 con 0x55c2e623c1e0 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2711109798 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbddc003c30 con 0x55c2e623c1e0 2024-05-09T05:38:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdfb29fb80 10 client.15501 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2711109798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2e6353de0 con 0x55c2e623c1e0 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2711109798 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbde0015a00 con 0x55c2e623c1e0 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2711109798 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbddc022070 con 0x55c2e623c1e0 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdfb29fb80 10 client.15501 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdfb29fb80 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdfb29fb80 20 client.15501 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2e6354060 0x55c2e6356480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdfb29fb80 4 client.15501 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.133+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2711109798 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c2e61f9af0 con 0x55c2e6354060 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.134+0000 7fbde659c700 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2e6354060 0x55c2e6356480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.134+0000 7fbde659c700 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2e6354060 0x55c2e6356480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424511787fe900e8 server_cookie=ca81c9aaf3ba2cb in_seq=0 out_seq=0 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.134+0000 7fbdcdffb700 10 client.15501 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdcdffb700 1 -- 172.21.15.115:0/2711109798 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c2e61f9af0 con 0x55c2e6354060 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdcdffb700 10 client.15501 handle_command_reply: tid=0 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdfb29fb80 1 client.15501 shutdown 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2711109798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e623c1e0 msgr2=0x55c2e62f0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e623c1e0 0x55c2e62f0f20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbddc005290 tx=0x7fbddc02f7b0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2711109798 shutdown_connections 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2e62cf720 0x55c2e63529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2e6354060 0x55c2e6356480 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2e6152520 0x55c2e62f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.135+0000 7fbdfb29fb80 1 --2- 172.21.15.115:0/2711109798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2e623c1e0 0x55c2e62f0f20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.136+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2711109798 >> 172.21.15.115:0/2711109798 conn(0x55c2e61b2720 msgr2=0x55c2e62ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.136+0000 7fbdcdffb700 0 client.15501 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.136+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2711109798 shutdown_connections 2024-05-09T05:38:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.136+0000 7fbdfb29fb80 1 -- 172.21.15.115:0/2711109798 wait complete. 2024-05-09T05:38:26.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.136+0000 7fbdfb29fb80 20 client.15501 trim_cache size 0 max 16384 2024-05-09T05:38:26.136 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:38:26.136 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:38:26.136 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "9c09d748-8d3a-46c8-8a64-dcad968c6552", 2024-05-09T05:38:26.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:38:26.136 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:38:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 -- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbdd00b2a80 msgr2=0x7fbdd00b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbdd00b2a80 0x7fbdd00b4f90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009990 tx=0x7fbddc005fb0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 -- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde81034c0 msgr2=0x7fbde81155c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde81034c0 0x7fbde81155c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800d950 tx=0x7fbdd800dc60 comp rx=0 tx=0).stop 2024-05-09T05:38:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 -- 172.21.15.115:0/3403824163 shutdown_connections 2024-05-09T05:38:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbdd00b2a80 0x7fbdd00b4f90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbde8108740 0x7fbde8113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbde81034c0 0x7fbde81155c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 --2- 172.21.15.115:0/3403824163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbde8107df0 0x7fbde8113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 -- 172.21.15.115:0/3403824163 >> 172.21.15.115:0/3403824163 conn(0x7fbde80fedc0 msgr2=0x7fbde81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:26.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 -- 172.21.15.115:0/3403824163 shutdown_connections 2024-05-09T05:38:26.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.137+0000 7fbdec889700 1 -- 172.21.15.115:0/3403824163 wait complete. 2024-05-09T05:38:26.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.190+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7e700feeb0 con 0x7f7e5c0b29e0 2024-05-09T05:38:26.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.199+0000 7f7e6cff9640 1 -- 172.21.15.115:0/1460159104 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f7e700feeb0 con 0x7f7e5c0b29e0 2024-05-09T05:38:26.192 INFO:teuthology.orchestra.run.smithi115.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-09T00:00:00", "created": "2024-05-09T05:38:25", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-09T05:38:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e5c0b29e0 msgr2=0x7f7e5c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e5c0b29e0 0x7f7e5c0b4ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006010 tx=0x7f7e60032ba0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 msgr2=0x7f7e7019bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 0x7f7e7019bc00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800e9f0 tx=0x7f7e5800eeb0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 shutdown_connections 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e5c0b29e0 0x7f7e5c0b4ef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e70109060 0x7f7e7019c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e70108bc0 0x7f7e7019bc00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 --2- 172.21.15.115:0/1460159104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e70103310 0x7f7e7019b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 >> 172.21.15.115:0/1460159104 conn(0x7f7e700fe390 msgr2=0x7f7e700fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 shutdown_connections 2024-05-09T05:38:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.201+0000 7f7e75784640 1 -- 172.21.15.115:0/1460159104 wait complete. 2024-05-09T05:38:26.206 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-09T05:38:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.330+0000 7f2f6d069640 1 -- 172.21.15.115:0/1964076876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68102410 msgr2=0x7f2f68102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.330+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1964076876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68102410 0x7f2f68102820 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009900 tx=0x7f2f5802a410 comp rx=0 tx=0).stop 2024-05-09T05:38:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.330+0000 7f2f6d069640 1 -- 172.21.15.115:0/1964076876 shutdown_connections 2024-05-09T05:38:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.330+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1964076876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f68108f50 0x7f2f681106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.330+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1964076876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f68108490 0x7f2f681088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.330+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1964076876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68102410 0x7f2f68102820 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.330+0000 7f2f6d069640 1 -- 172.21.15.115:0/1964076876 >> 172.21.15.115:0/1964076876 conn(0x7f2f680779e0 msgr2=0x7f2f68077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.330+0000 7f2f6d069640 1 -- 172.21.15.115:0/1964076876 shutdown_connections 2024-05-09T05:38:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.330+0000 7f2f6d069640 1 -- 172.21.15.115:0/1964076876 wait complete. 2024-05-09T05:38:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.331+0000 7f2f6d069640 1 Processor -- start 2024-05-09T05:38:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.331+0000 7f2f6d069640 1 -- start start 2024-05-09T05:38:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6d069640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f68102410 0x7f2f68102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6d069640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f68108490 0x7f2f68103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6d069640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68108f50 0x7f2f681039b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6d069640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68113d10 con 0x7f2f68108f50 2024-05-09T05:38:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6d069640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68113b90 con 0x7f2f68108490 2024-05-09T05:38:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6d069640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68113e90 con 0x7f2f68102410 2024-05-09T05:38:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68108f50 0x7f2f681039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68108f50 0x7f2f681039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39236/0 (socket says 172.21.15.115:39236) 2024-05-09T05:38:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6759e640 1 -- 172.21.15.115:0/1896067299 learned_addr learned my addr 172.21.15.115:0/1896067299 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f66d9d640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f68102410 0x7f2f68102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f5ffff640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f68108490 0x7f2f68103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6759e640 1 -- 172.21.15.115:0/1896067299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f68102410 msgr2=0x7f2f68102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6759e640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f68102410 0x7f2f68102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6759e640 1 -- 172.21.15.115:0/1896067299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f68108490 msgr2=0x7f2f68103470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6759e640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f68108490 0x7f2f68103470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6759e640 1 -- 172.21.15.115:0/1896067299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f681a8af0 con 0x7f2f68108f50 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6759e640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68108f50 0x7f2f681039b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000eb10 tx=0x7f2f6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f64d99640 1 -- 172.21.15.115:0/1896067299 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60004b20 con 0x7f2f68108f50 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f64d99640 1 -- 172.21.15.115:0/1896067299 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f60004cc0 con 0x7f2f68108f50 2024-05-09T05:38:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f64d99640 1 -- 172.21.15.115:0/1896067299 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60010690 con 0x7f2f68108f50 2024-05-09T05:38:26.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f681a8de0 con 0x7f2f68108f50 2024-05-09T05:38:26.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.332+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f681a9270 con 0x7f2f68108f50 2024-05-09T05:38:26.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.333+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f68050870 con 0x7f2f68108f50 2024-05-09T05:38:26.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.335+0000 7f2f64d99640 1 -- 172.21.15.115:0/1896067299 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f2f60005000 con 0x7f2f68108f50 2024-05-09T05:38:26.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.336+0000 7f2f64d99640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2f540b2a60 0x7f2f540b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:26.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.336+0000 7f2f66d9d640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2f540b2a60 0x7f2f540b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:26.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.336+0000 7f2f64d99640 1 -- 172.21.15.115:0/1896067299 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2f600e32d0 con 0x7f2f68108f50 2024-05-09T05:38:26.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.337+0000 7f2f66d9d640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2f540b2a60 0x7f2f540b4f70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f58000c00 tx=0x7f2f58009680 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.340+0000 7f2f64d99640 1 -- 172.21.15.115:0/1896067299 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f60054df0 con 0x7f2f68108f50 2024-05-09T05:38:26.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.555+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2f68101150 con 0x7f2f540b2a60 2024-05-09T05:38:26.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.563+0000 7f2f64d99640 1 -- 172.21.15.115:0/1896067299 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2f68101150 con 0x7f2f540b2a60 2024-05-09T05:38:26.557 INFO:teuthology.orchestra.run.smithi115.stdout:/ 1m 6m3h 2024-05-09T05:38:26.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2f540b2a60 msgr2=0x7f2f540b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2f540b2a60 0x7f2f540b4f70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f58000c00 tx=0x7f2f58009680 comp rx=0 tx=0).stop 2024-05-09T05:38:26.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68108f50 msgr2=0x7f2f681039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:26.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68108f50 0x7f2f681039b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000eb10 tx=0x7f2f6000c2d0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 shutdown_connections 2024-05-09T05:38:26.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2f540b2a60 0x7f2f540b4f70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f68108f50 0x7f2f681039b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f68108490 0x7f2f68103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 --2- 172.21.15.115:0/1896067299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f68102410 0x7f2f68102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:26.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 >> 172.21.15.115:0/1896067299 conn(0x7f2f680779e0 msgr2=0x7f2f68100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:26.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 shutdown_connections 2024-05-09T05:38:26.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:26.566+0000 7f2f6d069640 1 -- 172.21.15.115:0/1896067299 wait complete. 2024-05-09T05:38:26.569 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:26 smithi115 ceph-mon[30349]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:26.570 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:26 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:38:26.570 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:26 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:38:26.570 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:26 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:38:26.570 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:26 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:38:26.571 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-09T05:38:26.599 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-09T05:38:26.611 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-09T05:38:26.635 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T05:38:26.635 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-09T05:38:26.636 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:26 smithi195 ceph-mon[33844]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:26.636 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:26 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:38:26.636 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:26 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:38:26.636 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:26 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:38:26.636 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:26 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:38:26.641 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-09T05:38:26.655 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T05:38:26.655 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T05:38:26.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:26 smithi161 ceph-mon[33805]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:26.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:26 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:38:26.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:26 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:38:26.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:26 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:38:26.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:26 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:38:26.971 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:38:27.045 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:27.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:27 smithi115 ceph-mon[30349]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:27.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:27 smithi115 ceph-mon[30349]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-09T05:38:27.464 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:27 smithi115 ceph-mon[30349]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:27 smithi195 ceph-mon[33844]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:27 smithi195 ceph-mon[33844]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-09T05:38:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:27 smithi195 ceph-mon[33844]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:27.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:27 smithi161 ceph-mon[33805]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:27.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:27 smithi161 ceph-mon[33805]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-09T05:38:27.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:27 smithi161 ceph-mon[33805]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:28 smithi195 ceph-mon[33844]: from='client.15507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:28 smithi115 ceph-mon[30349]: from='client.15507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:28.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:28 smithi161 ceph-mon[33805]: from='client.15507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:38:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.977+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3073707681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d78103130 msgr2=0x7f2d78111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.977+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3073707681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d78103130 0x7f2d78111120 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b0a0 tx=0x7f2d7000b3b0 comp rx=0 tx=0).stop 2024-05-09T05:38:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.978+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3073707681 shutdown_connections 2024-05-09T05:38:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.978+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3073707681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d7810a3e0 0x7f2d7810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.978+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3073707681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d78103130 0x7f2d78111120 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.978+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3073707681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d781027e0 0x7f2d78102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.978+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3073707681 >> 172.21.15.115:0/3073707681 conn(0x7f2d780fe6f0 msgr2=0x7f2d78100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.978+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3073707681 shutdown_connections 2024-05-09T05:38:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.978+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3073707681 wait complete. 2024-05-09T05:38:28.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7ecd5700 1 Processor -- start 2024-05-09T05:38:28.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7ecd5700 1 -- start start 2024-05-09T05:38:28.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7ecd5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d781027e0 0x7f2d7819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7ecd5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d78103130 0x7f2d7819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7ecd5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7810a3e0 0x7f2d7819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7ecd5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78114380 con 0x7f2d7810a3e0 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7ecd5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78114200 con 0x7f2d78103130 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7ecd5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78114500 con 0x7f2d781027e0 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7d272700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7810a3e0 0x7f2d7819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7d272700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7810a3e0 0x7f2d7819f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39264/0 (socket says 172.21.15.115:39264) 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7d272700 1 -- 172.21.15.115:0/3924368604 learned_addr learned my addr 172.21.15.115:0/3924368604 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d77fff700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d78103130 0x7f2d7819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7ca71700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d781027e0 0x7f2d7819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:28.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.979+0000 7f2d7d272700 1 -- 172.21.15.115:0/3924368604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d781027e0 msgr2=0x7f2d7819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:28.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d7d272700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d781027e0 0x7f2d7819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:28.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d7d272700 1 -- 172.21.15.115:0/3924368604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d78103130 msgr2=0x7f2d7819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:28.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d7d272700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d78103130 0x7f2d7819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:28.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d7d272700 1 -- 172.21.15.115:0/3924368604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7819fc20 con 0x7f2d7810a3e0 2024-05-09T05:38:28.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d7d272700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7810a3e0 0x7f2d7819f580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b070 tx=0x7f2d70004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:28.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d75ffb700 1 -- 172.21.15.115:0/3924368604 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70021070 con 0x7f2d7810a3e0 2024-05-09T05:38:28.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d75ffb700 1 -- 172.21.15.115:0/3924368604 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d70007b80 con 0x7f2d7810a3e0 2024-05-09T05:38:28.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7819feb0 con 0x7f2d7810a3e0 2024-05-09T05:38:28.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d75ffb700 1 -- 172.21.15.115:0/3924368604 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d700046e0 con 0x7f2d7810a3e0 2024-05-09T05:38:28.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.980+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d781a0390 con 0x7f2d7810a3e0 2024-05-09T05:38:28.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.981+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d780512c0 con 0x7f2d7810a3e0 2024-05-09T05:38:28.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.983+0000 7f2d75ffb700 1 -- 172.21.15.115:0/3924368604 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f2d70003a00 con 0x7f2d7810a3e0 2024-05-09T05:38:28.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.984+0000 7f2d75ffb700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d680b2a80 0x7f2d680b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:28.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.984+0000 7f2d7ca71700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d680b2a80 0x7f2d680b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:28.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.984+0000 7f2d75ffb700 1 -- 172.21.15.115:0/3924368604 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2d7000f020 con 0x7f2d7810a3e0 2024-05-09T05:38:28.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.984+0000 7f2d7ca71700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d680b2a80 0x7f2d680b4f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c005950 tx=0x7f2d6c0058e0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:28.978 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:28.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:28.988+0000 7f2d75ffb700 1 -- 172.21.15.115:0/3924368604 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d7002a020 con 0x7f2d7810a3e0 2024-05-09T05:38:29.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.195+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d78063c00 con 0x7f2d680b2a80 2024-05-09T05:38:29.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.204+0000 7f2d75ffb700 1 -- 172.21.15.115:0/3924368604 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339714 (secure 0 0 0) 0x7f2d78063c00 con 0x7f2d680b2a80 2024-05-09T05:38:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:29.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d680b2a80 msgr2=0x7f2d680b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:29.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d680b2a80 0x7f2d680b4f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c005950 tx=0x7f2d6c0058e0 comp rx=0 tx=0).stop 2024-05-09T05:38:29.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7810a3e0 msgr2=0x7f2d7819f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:29.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7810a3e0 0x7f2d7819f580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b070 tx=0x7f2d70004d50 comp rx=0 tx=0).stop 2024-05-09T05:38:29.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 shutdown_connections 2024-05-09T05:38:29.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d680b2a80 0x7f2d680b4f90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:29.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d7810a3e0 0x7f2d7819f580 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:29.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d781027e0 0x7f2d7819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 --2- 172.21.15.115:0/3924368604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d78103130 0x7f2d7819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 >> 172.21.15.115:0/3924368604 conn(0x7f2d780fe6f0 msgr2=0x7f2d78100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 shutdown_connections 2024-05-09T05:38:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:29.208+0000 7f2d7ecd5700 1 -- 172.21.15.115:0/3924368604 wait complete. 2024-05-09T05:38:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:38:29.574 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:29 smithi115 ceph-mon[30349]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-09T05:38:29.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:29 smithi195 ceph-mon[33844]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-09T05:38:29.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:29 smithi161 ceph-mon[33805]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-09T05:38:30.024 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-05-09T05:38:27.945645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195491,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2287,"num_read_kb":1440,"num_write":531,"num_write_kb":3632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":916,"ondisk_log_size":916,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344664,"kb_used_data":12116,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597160,"statfs":{"total":1151940427776,"available":1151587491840,"internally_reserved":0,"allocated":12406784,"data_stored":9634787,"data_compressed":537816,"data_compressed_allocated":724992,"data_compressed_original":4300800,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":163,"num_read_kb":229,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003246"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925762+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:38:11.925762+0000","last_peered":"2024-05-09T05:38:11.925762+0000","last_clean":"2024-05-09T05:38:11.925762+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:38:11.925762+0000","last_undegraded":"2024-05-09T05:38:11.925762+0000","last_fullsized":"2024-05-09T05:38:11.925762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979667+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:38:11.979667+0000","last_peered":"2024-05-09T05:38:11.979667+0000","last_clean":"2024-05-09T05:38:11.979667+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:38:11.979667+0000","last_undegraded":"2024-05-09T05:38:11.979667+0000","last_fullsized":"2024-05-09T05:38:11.979667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979267+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:38:11.979267+0000","last_peered":"2024-05-09T05:38:11.979267+0000","last_clean":"2024-05-09T05:38:11.979267+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:38:11.979267+0000","last_undegraded":"2024-05-09T05:38:11.979267+0000","last_fullsized":"2024-05-09T05:38:11.979267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133401+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:38:12.133401+0000","last_peered":"2024-05-09T05:38:12.133401+0000","last_clean":"2024-05-09T05:38:12.133401+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:38:12.133401+0000","last_undegraded":"2024-05-09T05:38:12.133401+0000","last_fullsized":"2024-05-09T05:38:12.133401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051332+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:38:12.051332+0000","last_peered":"2024-05-09T05:38:12.051332+0000","last_clean":"2024-05-09T05:38:12.051332+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:38:12.051332+0000","last_undegraded":"2024-05-09T05:38:12.051332+0000","last_fullsized":"2024-05-09T05:38:12.051332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925938+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:38:11.925938+0000","last_peered":"2024-05-09T05:38:11.925938+0000","last_clean":"2024-05-09T05:38:11.925938+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:38:11.925938+0000","last_undegraded":"2024-05-09T05:38:11.925938+0000","last_fullsized":"2024-05-09T05:38:11.925938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132223+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:38:12.132223+0000","last_peered":"2024-05-09T05:38:12.132223+0000","last_clean":"2024-05-09T05:38:12.132223+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:38:12.132223+0000","last_undegraded":"2024-05-09T05:38:12.132223+0000","last_fullsized":"2024-05-09T05:38:12.132223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051904+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:38:12.051904+0000","last_peered":"2024-05-09T05:38:12.051904+0000","last_clean":"2024-05-09T05:38:12.051904+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:38:12.051904+0000","last_undegraded":"2024-05-09T05:38:12.051904+0000","last_fullsized":"2024-05-09T05:38:12.051904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134129+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:38:12.134129+0000","last_peered":"2024-05-09T05:38:12.134129+0000","last_clean":"2024-05-09T05:38:12.134129+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:38:12.134129+0000","last_undegraded":"2024-05-09T05:38:12.134129+0000","last_fullsized":"2024-05-09T05:38:12.134129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.926126+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:38:11.926126+0000","last_peered":"2024-05-09T05:38:11.926126+0000","last_clean":"2024-05-09T05:38:11.926126+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:38:11.926126+0000","last_undegraded":"2024-05-09T05:38:11.926126+0000","last_fullsized":"2024-05-09T05:38:11.926126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133832+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:38:12.133832+0000","last_peered":"2024-05-09T05:38:12.133832+0000","last_clean":"2024-05-09T05:38:12.133832+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:38:12.133832+0000","last_undegraded":"2024-05-09T05:38:12.133832+0000","last_fullsized":"2024-05-09T05:38:12.133832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133357+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:38:12.133357+0000","last_peered":"2024-05-09T05:38:12.133357+0000","last_clean":"2024-05-09T05:38:12.133357+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:38:12.133357+0000","last_undegraded":"2024-05-09T05:38:12.133357+0000","last_fullsized":"2024-05-09T05:38:12.133357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225339+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:38:12.225339+0000","last_peered":"2024-05-09T05:38:12.225339+0000","last_clean":"2024-05-09T05:38:12.225339+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:38:12.225339+0000","last_undegraded":"2024-05-09T05:38:12.225339+0000","last_fullsized":"2024-05-09T05:38:12.225339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979195+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:38:11.979195+0000","last_peered":"2024-05-09T05:38:11.979195+0000","last_clean":"2024-05-09T05:38:11.979195+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:38:11.979195+0000","last_undegraded":"2024-05-09T05:38:11.979195+0000","last_fullsized":"2024-05-09T05:38:11.979195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979140+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:38:11.979140+0000","last_peered":"2024-05-09T05:38:11.979140+0000","last_clean":"2024-05-09T05:38:11.979140+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:38:11.979140+0000","last_undegraded":"2024-05-09T05:38:11.979140+0000","last_fullsized":"2024-05-09T05:38:11.979140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132049+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:38:12.132049+0000","last_peered":"2024-05-09T05:38:12.132049+0000","last_clean":"2024-05-09T05:38:12.132049+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:38:12.132049+0000","last_undegraded":"2024-05-09T05:38:12.132049+0000","last_fullsized":"2024-05-09T05:38:12.132049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925512+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:38:11.925512+0000","last_peered":"2024-05-09T05:38:11.925512+0000","last_clean":"2024-05-09T05:38:11.925512+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:38:11.925512+0000","last_undegraded":"2024-05-09T05:38:11.925512+0000","last_fullsized":"2024-05-09T05:38:11.925512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924748+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:38:11.924748+0000","last_peered":"2024-05-09T05:38:11.924748+0000","last_clean":"2024-05-09T05:38:11.924748+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:38:11.924748+0000","last_undegraded":"2024-05-09T05:38:11.924748+0000","last_fullsized":"2024-05-09T05:38:11.924748+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132289+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:38:12.132289+0000","last_peered":"2024-05-09T05:38:12.132289+0000","last_clean":"2024-05-09T05:38:12.132289+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:38:12.132289+0000","last_undegraded":"2024-05-09T05:38:12.132289+0000","last_fullsized":"2024-05-09T05:38:12.132289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131837+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:38:12.131837+0000","last_peered":"2024-05-09T05:38:12.131837+0000","last_clean":"2024-05-09T05:38:12.131837+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:38:12.131837+0000","last_undegraded":"2024-05-09T05:38:12.131837+0000","last_fullsized":"2024-05-09T05:38:12.131837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":259,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:21.577983+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:38:21.577983+0000","last_peered":"2024-05-09T05:38:21.577983+0000","last_clean":"2024-05-09T05:38:21.577983+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:38:21.577983+0000","last_undegraded":"2024-05-09T05:38:21.577983+0000","last_fullsized":"2024-05-09T05:38:21.577983+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":201,"num_read_kb":201,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053507+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:38:12.053507+0000","last_peered":"2024-05-09T05:38:12.053507+0000","last_clean":"2024-05-09T05:38:12.053507+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:38:12.053507+0000","last_undegraded":"2024-05-09T05:38:12.053507+0000","last_fullsized":"2024-05-09T05:38:12.053507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132572+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:38:12.132572+0000","last_peered":"2024-05-09T05:38:12.132572+0000","last_clean":"2024-05-09T05:38:12.132572+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:38:12.132572+0000","last_undegraded":"2024-05-09T05:38:12.132572+0000","last_fullsized":"2024-05-09T05:38:12.132572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053764+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:38:12.053764+0000","last_peered":"2024-05-09T05:38:12.053764+0000","last_clean":"2024-05-09T05:38:12.053764+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:38:12.053764+0000","last_undegraded":"2024-05-09T05:38:12.053764+0000","last_fullsized":"2024-05-09T05:38:12.053764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979670+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:38:11.979670+0000","last_peered":"2024-05-09T05:38:11.979670+0000","last_clean":"2024-05-09T05:38:11.979670+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:38:11.979670+0000","last_undegraded":"2024-05-09T05:38:11.979670+0000","last_fullsized":"2024-05-09T05:38:11.979670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132963+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:38:12.132963+0000","last_peered":"2024-05-09T05:38:12.132963+0000","last_clean":"2024-05-09T05:38:12.132963+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:38:12.132963+0000","last_undegraded":"2024-05-09T05:38:12.132963+0000","last_fullsized":"2024-05-09T05:38:12.132963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051620+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:38:12.051620+0000","last_peered":"2024-05-09T05:38:12.051620+0000","last_clean":"2024-05-09T05:38:12.051620+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:38:12.051620+0000","last_undegraded":"2024-05-09T05:38:12.051620+0000","last_fullsized":"2024-05-09T05:38:12.051620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133077+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:38:12.133077+0000","last_peered":"2024-05-09T05:38:12.133077+0000","last_clean":"2024-05-09T05:38:12.133077+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:38:12.133077+0000","last_undegraded":"2024-05-09T05:38:12.133077+0000","last_fullsized":"2024-05-09T05:38:12.133077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132705+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:38:12.132705+0000","last_peered":"2024-05-09T05:38:12.132705+0000","last_clean":"2024-05-09T05:38:12.132705+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:38:12.132705+0000","last_undegraded":"2024-05-09T05:38:12.132705+0000","last_fullsized":"2024-05-09T05:38:12.132705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051663+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:38:12.051663+0000","last_peered":"2024-05-09T05:38:12.051663+0000","last_clean":"2024-05-09T05:38:12.051663+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:38:12.051663+0000","last_undegraded":"2024-05-09T05:38:12.051663+0000","last_fullsized":"2024-05-09T05:38:12.051663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059550+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:38:12.059550+0000","last_peered":"2024-05-09T05:38:12.059550+0000","last_clean":"2024-05-09T05:38:12.059550+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:38:12.059550+0000","last_undegraded":"2024-05-09T05:38:12.059550+0000","last_fullsized":"2024-05-09T05:38:12.059550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131984+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:38:12.131984+0000","last_peered":"2024-05-09T05:38:12.131984+0000","last_clean":"2024-05-09T05:38:12.131984+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:38:12.131984+0000","last_undegraded":"2024-05-09T05:38:12.131984+0000","last_fullsized":"2024-05-09T05:38:12.131984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":255,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:21.604946+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:38:21.604946+0000","last_peered":"2024-05-09T05:38:21.604946+0000","last_clean":"2024-05-09T05:38:21.604946+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:38:21.604946+0000","last_undegraded":"2024-05-09T05:38:21.604946+0000","last_fullsized":"2024-05-09T05:38:21.604946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":205,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051377+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:38:12.051377+0000","last_peered":"2024-05-09T05:38:12.051377+0000","last_clean":"2024-05-09T05:38:12.051377+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:38:12.051377+0000","last_undegraded":"2024-05-09T05:38:12.051377+0000","last_fullsized":"2024-05-09T05:38:12.051377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132066+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:38:12.132066+0000","last_peered":"2024-05-09T05:38:12.132066+0000","last_clean":"2024-05-09T05:38:12.132066+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:38:12.132066+0000","last_undegraded":"2024-05-09T05:38:12.132066+0000","last_fullsized":"2024-05-09T05:38:12.132066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925853+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:38:11.925853+0000","last_peered":"2024-05-09T05:38:11.925853+0000","last_clean":"2024-05-09T05:38:11.925853+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:38:11.925853+0000","last_undegraded":"2024-05-09T05:38:11.925853+0000","last_fullsized":"2024-05-09T05:38:11.925853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059515+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:38:12.059515+0000","last_peered":"2024-05-09T05:38:12.059515+0000","last_clean":"2024-05-09T05:38:12.059515+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:38:12.059515+0000","last_undegraded":"2024-05-09T05:38:12.059515+0000","last_fullsized":"2024-05-09T05:38:12.059515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925545+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:38:11.925545+0000","last_peered":"2024-05-09T05:38:11.925545+0000","last_clean":"2024-05-09T05:38:11.925545+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:38:11.925545+0000","last_undegraded":"2024-05-09T05:38:11.925545+0000","last_fullsized":"2024-05-09T05:38:11.925545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224943+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:38:12.224943+0000","last_peered":"2024-05-09T05:38:12.224943+0000","last_clean":"2024-05-09T05:38:12.224943+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:38:12.224943+0000","last_undegraded":"2024-05-09T05:38:12.224943+0000","last_fullsized":"2024-05-09T05:38:12.224943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052036+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:38:12.052036+0000","last_peered":"2024-05-09T05:38:12.052036+0000","last_clean":"2024-05-09T05:38:12.052036+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:38:12.052036+0000","last_undegraded":"2024-05-09T05:38:12.052036+0000","last_fullsized":"2024-05-09T05:38:12.052036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979576+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:38:11.979576+0000","last_peered":"2024-05-09T05:38:11.979576+0000","last_clean":"2024-05-09T05:38:11.979576+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:38:11.979576+0000","last_undegraded":"2024-05-09T05:38:11.979576+0000","last_fullsized":"2024-05-09T05:38:11.979576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059004+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:38:12.059004+0000","last_peered":"2024-05-09T05:38:12.059004+0000","last_clean":"2024-05-09T05:38:12.059004+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:38:12.059004+0000","last_undegraded":"2024-05-09T05:38:12.059004+0000","last_fullsized":"2024-05-09T05:38:12.059004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924546+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:38:11.924546+0000","last_peered":"2024-05-09T05:38:11.924546+0000","last_clean":"2024-05-09T05:38:11.924546+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:38:11.924546+0000","last_undegraded":"2024-05-09T05:38:11.924546+0000","last_fullsized":"2024-05-09T05:38:11.924546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053026+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:38:12.053026+0000","last_peered":"2024-05-09T05:38:12.053026+0000","last_clean":"2024-05-09T05:38:12.053026+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:38:12.053026+0000","last_undegraded":"2024-05-09T05:38:12.053026+0000","last_fullsized":"2024-05-09T05:38:12.053026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'7","reported_seq":449,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:22.605960+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:38:22.605960+0000","last_peered":"2024-05-09T05:38:22.605960+0000","last_clean":"2024-05-09T05:38:22.605960+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:38:22.605960+0000","last_undegraded":"2024-05-09T05:38:22.605960+0000","last_fullsized":"2024-05-09T05:38:22.605960+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":400,"num_read_kb":400,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133858+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:38:12.133858+0000","last_peered":"2024-05-09T05:38:12.133858+0000","last_clean":"2024-05-09T05:38:12.133858+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:38:12.133858+0000","last_undegraded":"2024-05-09T05:38:12.133858+0000","last_fullsized":"2024-05-09T05:38:12.133858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924463+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:38:11.924463+0000","last_peered":"2024-05-09T05:38:11.924463+0000","last_clean":"2024-05-09T05:38:11.924463+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:38:11.924463+0000","last_undegraded":"2024-05-09T05:38:11.924463+0000","last_fullsized":"2024-05-09T05:38:11.924463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133086+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:38:12.133086+0000","last_peered":"2024-05-09T05:38:12.133086+0000","last_clean":"2024-05-09T05:38:12.133086+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:38:12.133086+0000","last_undegraded":"2024-05-09T05:38:12.133086+0000","last_fullsized":"2024-05-09T05:38:12.133086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979599+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:38:11.979599+0000","last_peered":"2024-05-09T05:38:11.979599+0000","last_clean":"2024-05-09T05:38:11.979599+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:38:11.979599+0000","last_undegraded":"2024-05-09T05:38:11.979599+0000","last_fullsized":"2024-05-09T05:38:11.979599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133847+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:38:12.133847+0000","last_peered":"2024-05-09T05:38:12.133847+0000","last_clean":"2024-05-09T05:38:12.133847+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:38:12.133847+0000","last_undegraded":"2024-05-09T05:38:12.133847+0000","last_fullsized":"2024-05-09T05:38:12.133847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059520+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:38:12.059520+0000","last_peered":"2024-05-09T05:38:12.059520+0000","last_clean":"2024-05-09T05:38:12.059520+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:38:12.059520+0000","last_undegraded":"2024-05-09T05:38:12.059520+0000","last_fullsized":"2024-05-09T05:38:12.059520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133129+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:38:12.133129+0000","last_peered":"2024-05-09T05:38:12.133129+0000","last_clean":"2024-05-09T05:38:12.133129+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:38:12.133129+0000","last_undegraded":"2024-05-09T05:38:12.133129+0000","last_fullsized":"2024-05-09T05:38:12.133129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924514+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:38:11.924514+0000","last_peered":"2024-05-09T05:38:11.924514+0000","last_clean":"2024-05-09T05:38:11.924514+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:38:11.924514+0000","last_undegraded":"2024-05-09T05:38:11.924514+0000","last_fullsized":"2024-05-09T05:38:11.924514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'43","reported_seq":497,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:22.609791+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:38:22.609791+0000","last_peered":"2024-05-09T05:38:22.609791+0000","last_clean":"2024-05-09T05:38:22.609791+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:38:22.609791+0000","last_undegraded":"2024-05-09T05:38:22.609791+0000","last_fullsized":"2024-05-09T05:38:22.609791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69974,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":412,"num_read_kb":274,"num_write":45,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051722+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:38:12.051722+0000","last_peered":"2024-05-09T05:38:12.051722+0000","last_clean":"2024-05-09T05:38:12.051722+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:38:12.051722+0000","last_undegraded":"2024-05-09T05:38:12.051722+0000","last_fullsized":"2024-05-09T05:38:12.051722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059857+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:38:12.059857+0000","last_peered":"2024-05-09T05:38:12.059857+0000","last_clean":"2024-05-09T05:38:12.059857+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:38:12.059857+0000","last_undegraded":"2024-05-09T05:38:12.059857+0000","last_fullsized":"2024-05-09T05:38:12.059857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132158+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:38:12.132158+0000","last_peered":"2024-05-09T05:38:12.132158+0000","last_clean":"2024-05-09T05:38:12.132158+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:38:12.132158+0000","last_undegraded":"2024-05-09T05:38:12.132158+0000","last_fullsized":"2024-05-09T05:38:12.132158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225173+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:38:12.225173+0000","last_peered":"2024-05-09T05:38:12.225173+0000","last_clean":"2024-05-09T05:38:12.225173+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:38:12.225173+0000","last_undegraded":"2024-05-09T05:38:12.225173+0000","last_fullsized":"2024-05-09T05:38:12.225173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059308+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:38:12.059308+0000","last_peered":"2024-05-09T05:38:12.059308+0000","last_clean":"2024-05-09T05:38:12.059308+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:38:12.059308+0000","last_undegraded":"2024-05-09T05:38:12.059308+0000","last_fullsized":"2024-05-09T05:38:12.059308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132718+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:38:12.132718+0000","last_peered":"2024-05-09T05:38:12.132718+0000","last_clean":"2024-05-09T05:38:12.132718+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:38:12.132718+0000","last_undegraded":"2024-05-09T05:38:12.132718+0000","last_fullsized":"2024-05-09T05:38:12.132718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051717+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:38:12.051717+0000","last_peered":"2024-05-09T05:38:12.051717+0000","last_clean":"2024-05-09T05:38:12.051717+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:38:12.051717+0000","last_undegraded":"2024-05-09T05:38:12.051717+0000","last_fullsized":"2024-05-09T05:38:12.051717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":254,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:21.578966+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:38:21.578966+0000","last_peered":"2024-05-09T05:38:21.578966+0000","last_clean":"2024-05-09T05:38:21.578966+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:38:21.578966+0000","last_undegraded":"2024-05-09T05:38:21.578966+0000","last_fullsized":"2024-05-09T05:38:21.578966+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":205,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132335+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:38:12.132335+0000","last_peered":"2024-05-09T05:38:12.132335+0000","last_clean":"2024-05-09T05:38:12.132335+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:38:12.132335+0000","last_undegraded":"2024-05-09T05:38:12.132335+0000","last_fullsized":"2024-05-09T05:38:12.132335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225537+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:38:12.225537+0000","last_peered":"2024-05-09T05:38:12.225537+0000","last_clean":"2024-05-09T05:38:12.225537+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:38:12.225537+0000","last_undegraded":"2024-05-09T05:38:12.225537+0000","last_fullsized":"2024-05-09T05:38:12.225537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051224+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:38:12.051224+0000","last_peered":"2024-05-09T05:38:12.051224+0000","last_clean":"2024-05-09T05:38:12.051224+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:38:12.051224+0000","last_undegraded":"2024-05-09T05:38:12.051224+0000","last_fullsized":"2024-05-09T05:38:12.051224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924664+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:38:11.924664+0000","last_peered":"2024-05-09T05:38:11.924664+0000","last_clean":"2024-05-09T05:38:11.924664+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:38:11.924664+0000","last_undegraded":"2024-05-09T05:38:11.924664+0000","last_fullsized":"2024-05-09T05:38:11.924664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059325+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:38:12.059325+0000","last_peered":"2024-05-09T05:38:12.059325+0000","last_clean":"2024-05-09T05:38:12.059325+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:38:12.059325+0000","last_undegraded":"2024-05-09T05:38:12.059325+0000","last_fullsized":"2024-05-09T05:38:12.059325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051937+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:38:12.051937+0000","last_peered":"2024-05-09T05:38:12.051937+0000","last_clean":"2024-05-09T05:38:12.051937+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:38:12.051937+0000","last_undegraded":"2024-05-09T05:38:12.051937+0000","last_fullsized":"2024-05-09T05:38:12.051937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925361+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:38:11.925361+0000","last_peered":"2024-05-09T05:38:11.925361+0000","last_clean":"2024-05-09T05:38:11.925361+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:38:11.925361+0000","last_undegraded":"2024-05-09T05:38:11.925361+0000","last_fullsized":"2024-05-09T05:38:11.925361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051341+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:38:12.051341+0000","last_peered":"2024-05-09T05:38:12.051341+0000","last_clean":"2024-05-09T05:38:12.051341+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:38:12.051341+0000","last_undegraded":"2024-05-09T05:38:12.051341+0000","last_fullsized":"2024-05-09T05:38:12.051341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925850+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:38:11.925850+0000","last_peered":"2024-05-09T05:38:11.925850+0000","last_clean":"2024-05-09T05:38:11.925850+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:38:11.925850+0000","last_undegraded":"2024-05-09T05:38:11.925850+0000","last_fullsized":"2024-05-09T05:38:11.925850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225147+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:38:12.225147+0000","last_peered":"2024-05-09T05:38:12.225147+0000","last_clean":"2024-05-09T05:38:12.225147+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:38:12.225147+0000","last_undegraded":"2024-05-09T05:38:12.225147+0000","last_fullsized":"2024-05-09T05:38:12.225147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'7","reported_seq":254,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:21.578793+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:38:21.578793+0000","last_peered":"2024-05-09T05:38:21.578793+0000","last_clean":"2024-05-09T05:38:21.578793+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:38:21.578793+0000","last_undegraded":"2024-05-09T05:38:21.578793+0000","last_fullsized":"2024-05-09T05:38:21.578793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":203,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925837+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:38:11.925837+0000","last_peered":"2024-05-09T05:38:11.925837+0000","last_clean":"2024-05-09T05:38:11.925837+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:38:11.925837+0000","last_undegraded":"2024-05-09T05:38:11.925837+0000","last_fullsized":"2024-05-09T05:38:11.925837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225711+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:38:12.225711+0000","last_peered":"2024-05-09T05:38:12.225711+0000","last_clean":"2024-05-09T05:38:12.225711+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:38:12.225711+0000","last_undegraded":"2024-05-09T05:38:12.225711+0000","last_fullsized":"2024-05-09T05:38:12.225711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059336+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:38:12.059336+0000","last_peered":"2024-05-09T05:38:12.059336+0000","last_clean":"2024-05-09T05:38:12.059336+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:38:12.059336+0000","last_undegraded":"2024-05-09T05:38:12.059336+0000","last_fullsized":"2024-05-09T05:38:12.059336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":250,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:21.578128+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:38:21.578128+0000","last_peered":"2024-05-09T05:38:21.578128+0000","last_clean":"2024-05-09T05:38:21.578128+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:38:21.578128+0000","last_undegraded":"2024-05-09T05:38:21.578128+0000","last_fullsized":"2024-05-09T05:38:21.578128+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":201,"num_read_kb":201,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133686+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:38:12.133686+0000","last_peered":"2024-05-09T05:38:12.133686+0000","last_clean":"2024-05-09T05:38:12.133686+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:38:12.133686+0000","last_undegraded":"2024-05-09T05:38:12.133686+0000","last_fullsized":"2024-05-09T05:38:12.133686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052441+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:38:12.052441+0000","last_peered":"2024-05-09T05:38:12.052441+0000","last_clean":"2024-05-09T05:38:12.052441+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:38:12.052441+0000","last_undegraded":"2024-05-09T05:38:12.052441+0000","last_fullsized":"2024-05-09T05:38:12.052441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133733+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:38:12.133733+0000","last_peered":"2024-05-09T05:38:12.133733+0000","last_clean":"2024-05-09T05:38:12.133733+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:38:12.133733+0000","last_undegraded":"2024-05-09T05:38:12.133733+0000","last_fullsized":"2024-05-09T05:38:12.133733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"88'99","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:14.055036+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:38:14.055036+0000","last_peered":"2024-05-09T05:38:14.055036+0000","last_clean":"2024-05-09T05:38:14.055036+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:38:14.055036+0000","last_undegraded":"2024-05-09T05:38:14.055036+0000","last_fullsized":"2024-05-09T05:38:14.055036+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059582+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:38:12.059582+0000","last_peered":"2024-05-09T05:38:12.059582+0000","last_clean":"2024-05-09T05:38:12.059582+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:38:12.059582+0000","last_undegraded":"2024-05-09T05:38:12.059582+0000","last_fullsized":"2024-05-09T05:38:12.059582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224704+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:38:12.224704+0000","last_peered":"2024-05-09T05:38:12.224704+0000","last_clean":"2024-05-09T05:38:12.224704+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:38:12.224704+0000","last_undegraded":"2024-05-09T05:38:12.224704+0000","last_fullsized":"2024-05-09T05:38:12.224704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132204+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:38:12.132204+0000","last_peered":"2024-05-09T05:38:12.132204+0000","last_clean":"2024-05-09T05:38:12.132204+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:38:12.132204+0000","last_undegraded":"2024-05-09T05:38:12.132204+0000","last_fullsized":"2024-05-09T05:38:12.132204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925156+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:38:11.925156+0000","last_peered":"2024-05-09T05:38:11.925156+0000","last_clean":"2024-05-09T05:38:11.925156+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:38:11.925156+0000","last_undegraded":"2024-05-09T05:38:11.925156+0000","last_fullsized":"2024-05-09T05:38:11.925156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925336+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:38:11.925336+0000","last_peered":"2024-05-09T05:38:11.925336+0000","last_clean":"2024-05-09T05:38:11.925336+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:38:11.925336+0000","last_undegraded":"2024-05-09T05:38:11.925336+0000","last_fullsized":"2024-05-09T05:38:11.925336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225403+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:38:12.225403+0000","last_peered":"2024-05-09T05:38:12.225403+0000","last_clean":"2024-05-09T05:38:12.225403+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:38:12.225403+0000","last_undegraded":"2024-05-09T05:38:12.225403+0000","last_fullsized":"2024-05-09T05:38:12.225403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051969+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:38:12.051969+0000","last_peered":"2024-05-09T05:38:12.051969+0000","last_clean":"2024-05-09T05:38:12.051969+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:38:12.051969+0000","last_undegraded":"2024-05-09T05:38:12.051969+0000","last_fullsized":"2024-05-09T05:38:12.051969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'54","reported_seq":121,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.817878+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:38:12.817878+0000","last_peered":"2024-05-09T05:38:12.817878+0000","last_clean":"2024-05-09T05:38:12.817878+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:38:12.817878+0000","last_undegraded":"2024-05-09T05:38:12.817878+0000","last_fullsized":"2024-05-09T05:38:12.817878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":96,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053724+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:38:12.053724+0000","last_peered":"2024-05-09T05:38:12.053724+0000","last_clean":"2024-05-09T05:38:12.053724+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:38:12.053724+0000","last_undegraded":"2024-05-09T05:38:12.053724+0000","last_fullsized":"2024-05-09T05:38:12.053724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131803+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:38:12.131803+0000","last_peered":"2024-05-09T05:38:12.131803+0000","last_clean":"2024-05-09T05:38:12.131803+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:38:12.131803+0000","last_undegraded":"2024-05-09T05:38:12.131803+0000","last_fullsized":"2024-05-09T05:38:12.131803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134405+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:38:12.134405+0000","last_peered":"2024-05-09T05:38:12.134405+0000","last_clean":"2024-05-09T05:38:12.134405+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:38:12.134405+0000","last_undegraded":"2024-05-09T05:38:12.134405+0000","last_fullsized":"2024-05-09T05:38:12.134405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979676+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:38:11.979676+0000","last_peered":"2024-05-09T05:38:11.979676+0000","last_clean":"2024-05-09T05:38:11.979676+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:38:11.979676+0000","last_undegraded":"2024-05-09T05:38:11.979676+0000","last_fullsized":"2024-05-09T05:38:11.979676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133578+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:38:12.133578+0000","last_peered":"2024-05-09T05:38:12.133578+0000","last_clean":"2024-05-09T05:38:12.133578+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:38:12.133578+0000","last_undegraded":"2024-05-09T05:38:12.133578+0000","last_fullsized":"2024-05-09T05:38:12.133578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132717+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:38:12.132717+0000","last_peered":"2024-05-09T05:38:12.132717+0000","last_clean":"2024-05-09T05:38:12.132717+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:38:12.132717+0000","last_undegraded":"2024-05-09T05:38:12.132717+0000","last_fullsized":"2024-05-09T05:38:12.132717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051314+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:38:12.051314+0000","last_peered":"2024-05-09T05:38:12.051314+0000","last_clean":"2024-05-09T05:38:12.051314+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:38:12.051314+0000","last_undegraded":"2024-05-09T05:38:12.051314+0000","last_fullsized":"2024-05-09T05:38:12.051314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924777+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:38:11.924777+0000","last_peered":"2024-05-09T05:38:11.924777+0000","last_clean":"2024-05-09T05:38:11.924777+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:38:11.924777+0000","last_undegraded":"2024-05-09T05:38:11.924777+0000","last_fullsized":"2024-05-09T05:38:11.924777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053182+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:38:12.053182+0000","last_peered":"2024-05-09T05:38:12.053182+0000","last_clean":"2024-05-09T05:38:12.053182+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:38:12.053182+0000","last_undegraded":"2024-05-09T05:38:12.053182+0000","last_fullsized":"2024-05-09T05:38:12.053182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924983+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:38:11.924983+0000","last_peered":"2024-05-09T05:38:11.924983+0000","last_clean":"2024-05-09T05:38:11.924983+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:38:11.924983+0000","last_undegraded":"2024-05-09T05:38:11.924983+0000","last_fullsized":"2024-05-09T05:38:11.924983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133063+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:38:12.133063+0000","last_peered":"2024-05-09T05:38:12.133063+0000","last_clean":"2024-05-09T05:38:12.133063+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:38:12.133063+0000","last_undegraded":"2024-05-09T05:38:12.133063+0000","last_fullsized":"2024-05-09T05:38:12.133063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.315433+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:38:12.315433+0000","last_peered":"2024-05-09T05:38:12.315433+0000","last_clean":"2024-05-09T05:38:12.315433+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:38:12.315433+0000","last_undegraded":"2024-05-09T05:38:12.315433+0000","last_fullsized":"2024-05-09T05:38:12.315433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053097+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:38:12.053097+0000","last_peered":"2024-05-09T05:38:12.053097+0000","last_clean":"2024-05-09T05:38:12.053097+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:38:12.053097+0000","last_undegraded":"2024-05-09T05:38:12.053097+0000","last_fullsized":"2024-05-09T05:38:12.053097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133542+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:38:12.133542+0000","last_peered":"2024-05-09T05:38:12.133542+0000","last_clean":"2024-05-09T05:38:12.133542+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:38:12.133542+0000","last_undegraded":"2024-05-09T05:38:12.133542+0000","last_fullsized":"2024-05-09T05:38:12.133542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051419+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:38:12.051419+0000","last_peered":"2024-05-09T05:38:12.051419+0000","last_clean":"2024-05-09T05:38:12.051419+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:38:12.051419+0000","last_undegraded":"2024-05-09T05:38:12.051419+0000","last_fullsized":"2024-05-09T05:38:12.051419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132543+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:38:12.132543+0000","last_peered":"2024-05-09T05:38:12.132543+0000","last_clean":"2024-05-09T05:38:12.132543+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:38:12.132543+0000","last_undegraded":"2024-05-09T05:38:12.132543+0000","last_fullsized":"2024-05-09T05:38:12.132543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133025+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:38:12.133025+0000","last_peered":"2024-05-09T05:38:12.133025+0000","last_clean":"2024-05-09T05:38:12.133025+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:38:12.133025+0000","last_undegraded":"2024-05-09T05:38:12.133025+0000","last_fullsized":"2024-05-09T05:38:12.133025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":254,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:22.642614+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:38:22.642614+0000","last_peered":"2024-05-09T05:38:22.642614+0000","last_clean":"2024-05-09T05:38:22.642614+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:38:22.642614+0000","last_undegraded":"2024-05-09T05:38:22.642614+0000","last_fullsized":"2024-05-09T05:38:22.642614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133983+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:38:12.133983+0000","last_peered":"2024-05-09T05:38:12.133983+0000","last_clean":"2024-05-09T05:38:12.133983+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:38:12.133983+0000","last_undegraded":"2024-05-09T05:38:12.133983+0000","last_fullsized":"2024-05-09T05:38:12.133983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979193+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:38:11.979193+0000","last_peered":"2024-05-09T05:38:11.979193+0000","last_clean":"2024-05-09T05:38:11.979193+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:38:11.979193+0000","last_undegraded":"2024-05-09T05:38:11.979193+0000","last_fullsized":"2024-05-09T05:38:11.979193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.179270+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:38:12.179270+0000","last_peered":"2024-05-09T05:38:12.179270+0000","last_clean":"2024-05-09T05:38:12.179270+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:38:12.179270+0000","last_undegraded":"2024-05-09T05:38:12.179270+0000","last_fullsized":"2024-05-09T05:38:12.179270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132662+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:38:12.132662+0000","last_peered":"2024-05-09T05:38:12.132662+0000","last_clean":"2024-05-09T05:38:12.132662+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:38:12.132662+0000","last_undegraded":"2024-05-09T05:38:12.132662+0000","last_fullsized":"2024-05-09T05:38:12.132662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925491+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:38:11.925491+0000","last_peered":"2024-05-09T05:38:11.925491+0000","last_clean":"2024-05-09T05:38:11.925491+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:38:11.925491+0000","last_undegraded":"2024-05-09T05:38:11.925491+0000","last_fullsized":"2024-05-09T05:38:11.925491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225592+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:38:12.225592+0000","last_peered":"2024-05-09T05:38:12.225592+0000","last_clean":"2024-05-09T05:38:12.225592+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:38:12.225592+0000","last_undegraded":"2024-05-09T05:38:12.225592+0000","last_fullsized":"2024-05-09T05:38:12.225592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133446+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:38:12.133446+0000","last_peered":"2024-05-09T05:38:12.133446+0000","last_clean":"2024-05-09T05:38:12.133446+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:38:12.133446+0000","last_undegraded":"2024-05-09T05:38:12.133446+0000","last_fullsized":"2024-05-09T05:38:12.133446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925076+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:38:11.925076+0000","last_peered":"2024-05-09T05:38:11.925076+0000","last_clean":"2024-05-09T05:38:11.925076+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:38:11.925076+0000","last_undegraded":"2024-05-09T05:38:11.925076+0000","last_fullsized":"2024-05-09T05:38:11.925076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132222+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:38:12.132222+0000","last_peered":"2024-05-09T05:38:12.132222+0000","last_clean":"2024-05-09T05:38:12.132222+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:38:12.132222+0000","last_undegraded":"2024-05-09T05:38:12.132222+0000","last_fullsized":"2024-05-09T05:38:12.132222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131810+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:38:12.131810+0000","last_peered":"2024-05-09T05:38:12.131810+0000","last_clean":"2024-05-09T05:38:12.131810+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:38:12.131810+0000","last_undegraded":"2024-05-09T05:38:12.131810+0000","last_fullsized":"2024-05-09T05:38:12.131810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979502+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:38:11.979502+0000","last_peered":"2024-05-09T05:38:11.979502+0000","last_clean":"2024-05-09T05:38:11.979502+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:38:11.979502+0000","last_undegraded":"2024-05-09T05:38:11.979502+0000","last_fullsized":"2024-05-09T05:38:11.979502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225302+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:38:12.225302+0000","last_peered":"2024-05-09T05:38:12.225302+0000","last_clean":"2024-05-09T05:38:12.225302+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:38:12.225302+0000","last_undegraded":"2024-05-09T05:38:12.225302+0000","last_fullsized":"2024-05-09T05:38:12.225302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925541+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:38:11.925541+0000","last_peered":"2024-05-09T05:38:11.925541+0000","last_clean":"2024-05-09T05:38:11.925541+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:38:11.925541+0000","last_undegraded":"2024-05-09T05:38:11.925541+0000","last_fullsized":"2024-05-09T05:38:11.925541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132345+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:38:12.132345+0000","last_peered":"2024-05-09T05:38:12.132345+0000","last_clean":"2024-05-09T05:38:12.132345+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:38:12.132345+0000","last_undegraded":"2024-05-09T05:38:12.132345+0000","last_fullsized":"2024-05-09T05:38:12.132345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053867+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:38:12.053867+0000","last_peered":"2024-05-09T05:38:12.053867+0000","last_clean":"2024-05-09T05:38:12.053867+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:38:12.053867+0000","last_undegraded":"2024-05-09T05:38:12.053867+0000","last_fullsized":"2024-05-09T05:38:12.053867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059147+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:38:12.059147+0000","last_peered":"2024-05-09T05:38:12.059147+0000","last_clean":"2024-05-09T05:38:12.059147+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:38:12.059147+0000","last_undegraded":"2024-05-09T05:38:12.059147+0000","last_fullsized":"2024-05-09T05:38:12.059147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053305+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:38:12.053305+0000","last_peered":"2024-05-09T05:38:12.053305+0000","last_clean":"2024-05-09T05:38:12.053305+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:38:12.053305+0000","last_undegraded":"2024-05-09T05:38:12.053305+0000","last_fullsized":"2024-05-09T05:38:12.053305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133431+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:38:12.133431+0000","last_peered":"2024-05-09T05:38:12.133431+0000","last_clean":"2024-05-09T05:38:12.133431+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:38:12.133431+0000","last_undegraded":"2024-05-09T05:38:12.133431+0000","last_fullsized":"2024-05-09T05:38:12.133431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224991+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:38:12.224991+0000","last_peered":"2024-05-09T05:38:12.224991+0000","last_clean":"2024-05-09T05:38:12.224991+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:38:12.224991+0000","last_undegraded":"2024-05-09T05:38:12.224991+0000","last_fullsized":"2024-05-09T05:38:12.224991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925262+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:38:11.925262+0000","last_peered":"2024-05-09T05:38:11.925262+0000","last_clean":"2024-05-09T05:38:11.925262+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:38:11.925262+0000","last_undegraded":"2024-05-09T05:38:11.925262+0000","last_fullsized":"2024-05-09T05:38:11.925262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107766,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604889,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2143,"num_read_kb":1192,"num_write":394,"num_write_kb":1480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1814667,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710788,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507638,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907018,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1295916,"data_compressed":71590,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070544,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":903884,"data_compressed":48658,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266773,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":900054,"data_compressed":48658,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463002,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":1105849,"data_compressed":65434,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626527,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":502960,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":573964,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":504416,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":575209,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1026600,"data_compressed":54814,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986280,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507034,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822757,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1231263,"data_compressed":71590,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135445,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":204194,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:38:30.033 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T05:38:30.474 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:30.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:30 smithi195 ceph-mon[33844]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:30.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:30 smithi115 ceph-mon[30349]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:30.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:30 smithi161 ceph-mon[33805]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:31.417 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:31 smithi115 ceph-mon[30349]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-05-09T05:38:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:31 smithi195 ceph-mon[33844]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-05-09T05:38:31.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:31 smithi161 ceph-mon[33805]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-05-09T05:38:31.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.736+0000 7f0151441700 1 -- 172.21.15.115:0/1202978838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 msgr2=0x7f014c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.736+0000 7f0151441700 1 --2- 172.21.15.115:0/1202978838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 0x7f014c109310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f01400099c0 tx=0x7f0140009cd0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.736+0000 7f0151441700 1 -- 172.21.15.115:0/1202978838 shutdown_connections 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.736+0000 7f0151441700 1 --2- 172.21.15.115:0/1202978838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c109850 0x7f014c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.736+0000 7f0151441700 1 --2- 172.21.15.115:0/1202978838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c102b50 0x7f014c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.736+0000 7f0151441700 1 --2- 172.21.15.115:0/1202978838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 0x7f014c109310 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.736+0000 7f0151441700 1 -- 172.21.15.115:0/1202978838 >> 172.21.15.115:0/1202978838 conn(0x7f014c0fe6f0 msgr2=0x7f014c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.736+0000 7f0151441700 1 -- 172.21.15.115:0/1202978838 shutdown_connections 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.736+0000 7f0151441700 1 -- 172.21.15.115:0/1202978838 wait complete. 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.737+0000 7f0151441700 1 Processor -- start 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.737+0000 7f0151441700 1 -- start start 2024-05-09T05:38:31.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.737+0000 7f0151441700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c102b50 0x7f014c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.737+0000 7f0151441700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 0x7f014c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.737+0000 7f0151441700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c109850 0x7f014c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.737+0000 7f0151441700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f014c114310 con 0x7f014c108f00 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.737+0000 7f0151441700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f014c114190 con 0x7f014c109850 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.737+0000 7f0151441700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f014c114490 con 0x7f014c102b50 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 0x7f014c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 0x7f014c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39288/0 (socket says 172.21.15.115:39288) 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014a7fc700 1 -- 172.21.15.115:0/2082170164 learned_addr learned my addr 172.21.15.115:0/2082170164 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014affd700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c102b50 0x7f014c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014b7fe700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c109850 0x7f014c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014a7fc700 1 -- 172.21.15.115:0/2082170164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c102b50 msgr2=0x7f014c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014a7fc700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c102b50 0x7f014c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014a7fc700 1 -- 172.21.15.115:0/2082170164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c109850 msgr2=0x7f014c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014a7fc700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c109850 0x7f014c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014a7fc700 1 -- 172.21.15.115:0/2082170164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c19fc70 con 0x7f014c108f00 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014affd700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c102b50 0x7f014c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014b7fe700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c109850 0x7f014c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f014a7fc700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 0x7f014c19b340 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f014000b580 tx=0x7f014000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f0133fff700 1 -- 172.21.15.115:0/2082170164 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140027070 con 0x7f014c108f00 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f0133fff700 1 -- 172.21.15.115:0/2082170164 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f014000fc20 con 0x7f014c108f00 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f0151441700 1 -- 172.21.15.115:0/2082170164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c19ff00 con 0x7f014c108f00 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f0133fff700 1 -- 172.21.15.115:0/2082170164 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140017410 con 0x7f014c108f00 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.738+0000 7f0151441700 1 -- 172.21.15.115:0/2082170164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c1a03e0 con 0x7f014c108f00 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.741+0000 7f0133fff700 1 -- 172.21.15.115:0/2082170164 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f014002a070 con 0x7f014c108f00 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.741+0000 7f0133fff700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01340b2a80 0x7f01340b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f014affd700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01340b2a80 0x7f01340b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f0133fff700 1 -- 172.21.15.115:0/2082170164 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f01400e39c0 con 0x7f014c108f00 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f015fe57b80 1 Processor -- start 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f015fe57b80 1 -- start start 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f014affd700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01340b2a80 0x7f01340b4f90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f013c005950 tx=0x7f013c005f50 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 0x55a09a6dc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f015fe57b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a09a5ff0a0 con 0x55a09a55f4a0 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f015fe57b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a09a5a49a0 con 0x55a09a6dc400 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f015fe57b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a09a5fb060 con 0x55a09a5d9d00 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f014a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f014a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39294/0 (socket says 172.21.15.115:39294) 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f014a7fc700 1 -- 172.21.15.115:0/1703545297 learned_addr learned my addr 172.21.15.115:0/1703545297 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.742+0000 7f014affd700 1 --2- 172.21.15.115:0/1703545297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f014a7fc700 1 -- 172.21.15.115:0/1703545297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 msgr2=0x55a09a6e2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f014b7fe700 1 --2- 172.21.15.115:0/1703545297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 0x55a09a6dc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f014a7fc700 1 --2- 172.21.15.115:0/1703545297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6e2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f014a7fc700 1 -- 172.21.15.115:0/1703545297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 msgr2=0x55a09a6dc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f014a7fc700 1 --2- 172.21.15.115:0/1703545297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 0x55a09a6dc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f014a7fc700 1 -- 172.21.15.115:0/1703545297 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a09a595500 con 0x55a09a55f4a0 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f014a7fc700 1 --2- 172.21.15.115:0/1703545297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e2780 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f014000ffd0 tx=0x7f01400aa1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0be52c0db6025cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f01317fa700 1 -- 172.21.15.115:0/1703545297 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01400aa390 con 0x55a09a55f4a0 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f01317fa700 1 -- 172.21.15.115:0/1703545297 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f014000fdd0 con 0x55a09a55f4a0 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.743+0000 7f01317fa700 1 -- 172.21.15.115:0/1703545297 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01400aad00 con 0x55a09a55f4a0 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 -- 172.21.15.115:0/1703545297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 msgr2=0x55a09a6e2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 --2- 172.21.15.115:0/1703545297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e2780 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f014000ffd0 tx=0x7f01400aa1d0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 -- 172.21.15.115:0/1703545297 shutdown_connections 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 --2- 172.21.15.115:0/1703545297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 0x55a09a6dc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 --2- 172.21.15.115:0/1703545297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 --2- 172.21.15.115:0/1703545297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e2780 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 -- 172.21.15.115:0/1703545297 >> 172.21.15.115:0/1703545297 conn(0x55a09a562f70 msgr2=0x55a09a6d7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 -- 172.21.15.115:0/1703545297 shutdown_connections 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 -- 172.21.15.115:0/1703545297 wait complete. 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 -- start start 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.744+0000 7f015fe57b80 1 Processor -- start 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a55f4a0 0x55a09a6eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a6dc400 0x55a09a6e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f015fe57b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a09a5ff0a0 con 0x55a09a6dc400 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f015fe57b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a09a599b20 con 0x55a09a55f4a0 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f015fe57b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a09a6dcfb0 con 0x55a09a5d9d00 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f015fe57b80 10 client.0 _ll_register_callbacks cb 0x55a09a4706c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:31.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a6dc400 0x55a09a6e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a6dc400 0x55a09a6e5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39296/0 (socket says 172.21.15.115:39296) 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014b7fe700 1 -- 172.21.15.115:0/934250015 learned_addr learned my addr 172.21.15.115:0/934250015 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014affd700 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a55f4a0 0x55a09a6eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014a7fc700 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014b7fe700 1 -- 172.21.15.115:0/934250015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 msgr2=0x55a09a6ebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014b7fe700 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6ebf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014b7fe700 1 -- 172.21.15.115:0/934250015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a55f4a0 msgr2=0x55a09a6eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014b7fe700 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a55f4a0 0x55a09a6eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014b7fe700 1 -- 172.21.15.115:0/934250015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a09a47a490 con 0x55a09a6dc400 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f014b7fe700 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a6dc400 0x55a09a6e5de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0144008bf0 tx=0x7f014400f0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b34df2a6b7dac88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f012eff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f012eff5700 1 -- 172.21.15.115:0/934250015 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144004bb0 con 0x55a09a6dc400 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f015fe57b80 1 -- 172.21.15.115:0/934250015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f014000fdd0 con 0x55a09a6dc400 2024-05-09T05:38:31.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.745+0000 7f012eff5700 1 -- 172.21.15.115:0/934250015 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0144004d50 con 0x55a09a6dc400 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.746+0000 7f012eff5700 1 -- 172.21.15.115:0/934250015 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0144021050 con 0x55a09a6dc400 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.746+0000 7f015fe57b80 10 client.15525 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.746+0000 7f015fe57b80 1 -- 172.21.15.115:0/934250015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a09a6f3660 con 0x55a09a6dc400 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.747+0000 7f012eff5700 1 -- 172.21.15.115:0/934250015 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f014000fdd0 con 0x55a09a6dc400 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.747+0000 7f012eff5700 1 -- 172.21.15.115:0/934250015 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0144005690 con 0x55a09a6dc400 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.747+0000 7f015fe57b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.747+0000 7f015fe57b80 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.747+0000 7f015fe57b80 20 client.15525 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.747+0000 7f015fe57b80 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a09a6f5cf0 0x55a09a6f8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.747+0000 7f015fe57b80 4 client.15525 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.747+0000 7f015fe57b80 1 -- 172.21.15.115:0/934250015 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a09a60ca80 con 0x55a09a6f5cf0 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.747+0000 7f014affd700 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a09a6f5cf0 0x55a09a6f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.748+0000 7f012eff5700 10 client.15525 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:31.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.748+0000 7f014affd700 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a09a6f5cf0 0x55a09a6f8110 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea83ce453cad83d server_cookie=8a15a242fc4b898d in_seq=0 out_seq=0 2024-05-09T05:38:31.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.748+0000 7f012eff5700 1 -- 172.21.15.115:0/934250015 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a09a60ca80 con 0x55a09a6f5cf0 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.748+0000 7f012eff5700 10 client.15525 handle_command_reply: tid=0 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.748+0000 7f015fe57b80 1 client.15525 shutdown 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.748+0000 7f015fe57b80 1 -- 172.21.15.115:0/934250015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a6dc400 msgr2=0x55a09a6e5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.748+0000 7f015fe57b80 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a6dc400 0x55a09a6e5de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0144008bf0 tx=0x7f014400f0e0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f015fe57b80 1 -- 172.21.15.115:0/934250015 shutdown_connections 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f015fe57b80 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a55f4a0 0x55a09a6eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f015fe57b80 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a09a6f5cf0 0x55a09a6f8110 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f015fe57b80 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a6dc400 0x55a09a6e5de0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f012eff5700 0 client.15525 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f015fe57b80 1 --2- 172.21.15.115:0/934250015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f015fe57b80 1 -- 172.21.15.115:0/934250015 >> 172.21.15.115:0/934250015 conn(0x55a09a562f70 msgr2=0x55a09a6d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f015fe57b80 1 -- 172.21.15.115:0/934250015 shutdown_connections 2024-05-09T05:38:31.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f015fe57b80 1 -- 172.21.15.115:0/934250015 wait complete. 2024-05-09T05:38:31.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.749+0000 7f015fe57b80 20 client.15525 trim_cache size 0 max 16384 2024-05-09T05:38:31.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f015fe57b80 1 Processor -- start 2024-05-09T05:38:31.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f015fe57b80 1 -- start start 2024-05-09T05:38:31.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 0x55a09a6fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f015fe57b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1b0320 con 0x55a09a55f4a0 2024-05-09T05:38:31.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f015fe57b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1b1e60 con 0x55a09a6dc400 2024-05-09T05:38:31.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f015fe57b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1b09f0 con 0x55a09a5d9d00 2024-05-09T05:38:31.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39308/0 (socket says 172.21.15.115:39308) 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014affd700 1 -- 172.21.15.115:0/83185802 learned_addr learned my addr 172.21.15.115:0/83185802 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014b7fe700 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 0x55a09a6fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014a7fc700 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014affd700 1 -- 172.21.15.115:0/83185802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 msgr2=0x55a09a6e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014affd700 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014affd700 1 -- 172.21.15.115:0/83185802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 msgr2=0x55a09a6fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014affd700 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 0x55a09a6fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014affd700 1 -- 172.21.15.115:0/83185802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a09a6d39d0 con 0x55a09a55f4a0 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014a7fc700 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.769+0000 7f014affd700 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e4a40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f013c00a540 tx=0x7f013c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaadd4bb8a54ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:31.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.770+0000 7f0130ff9700 1 -- 172.21.15.115:0/83185802 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c004830 con 0x55a09a55f4a0 2024-05-09T05:38:31.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.770+0000 7f0130ff9700 1 -- 172.21.15.115:0/83185802 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f013c005020 con 0x55a09a55f4a0 2024-05-09T05:38:31.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.770+0000 7f0130ff9700 1 -- 172.21.15.115:0/83185802 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c027520 con 0x55a09a55f4a0 2024-05-09T05:38:31.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 -- 172.21.15.115:0/83185802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 msgr2=0x55a09a6e4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:31.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e4a40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f013c00a540 tx=0x7f013c014000 comp rx=0 tx=0).stop 2024-05-09T05:38:31.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 -- 172.21.15.115:0/83185802 shutdown_connections 2024-05-09T05:38:31.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a6dc400 0x55a09a6fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6e4a40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 --2- 172.21.15.115:0/83185802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a5d9d00 0x55a09a6e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 -- 172.21.15.115:0/83185802 >> 172.21.15.115:0/83185802 conn(0x55a09a562f70 msgr2=0x55a09a6dfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:31.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 -- 172.21.15.115:0/83185802 shutdown_connections 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 -- 172.21.15.115:0/83185802 wait complete. 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 -- start start 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.771+0000 7f015fe57b80 1 Processor -- start 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6ef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a5d9d00 0x55a09a6ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f015fe57b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a6dc400 0x55a09a75f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f015fe57b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1b1790 con 0x55a09a55f4a0 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f015fe57b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1b10c0 con 0x55a09a5d9d00 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f015fe57b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1ae2c0 con 0x55a09a6dc400 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f015fe57b80 10 client.0 _ll_register_callbacks cb 0x55a09a4706c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:31.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6ef460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39318/0 (socket says 172.21.15.115:39318) 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014affd700 1 -- 172.21.15.115:0/4127064878 learned_addr learned my addr 172.21.15.115:0/4127064878 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014a7fc700 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a5d9d00 0x55a09a6ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014b7fe700 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a6dc400 0x55a09a75f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014affd700 1 -- 172.21.15.115:0/4127064878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a6dc400 msgr2=0x55a09a75f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014affd700 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a6dc400 0x55a09a75f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014affd700 1 -- 172.21.15.115:0/4127064878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a5d9d00 msgr2=0x55a09a6ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014affd700 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a5d9d00 0x55a09a6ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014affd700 1 -- 172.21.15.115:0/4127064878 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a09a6f0280 con 0x55a09a55f4a0 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f014affd700 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6ef460 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f013c005400 tx=0x7f013c005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222b12df6b1c37ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f012eff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f012eff5700 1 -- 172.21.15.115:0/4127064878 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c004b60 con 0x55a09a55f4a0 2024-05-09T05:38:31.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f012eff5700 1 -- 172.21.15.115:0/4127064878 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f013c004d00 con 0x55a09a55f4a0 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.772+0000 7f015fe57b80 1 -- 172.21.15.115:0/4127064878 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f013c005020 con 0x55a09a55f4a0 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f012eff5700 1 -- 172.21.15.115:0/4127064878 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f013c0272c0 con 0x55a09a55f4a0 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f015fe57b80 10 client.15531 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f015fe57b80 1 -- 172.21.15.115:0/4127064878 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a09a7609e0 con 0x55a09a55f4a0 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f012eff5700 1 -- 172.21.15.115:0/4127064878 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f013c005020 con 0x55a09a55f4a0 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f012eff5700 1 -- 172.21.15.115:0/4127064878 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f013c037070 con 0x55a09a55f4a0 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f015fe57b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f015fe57b80 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f015fe57b80 20 client.15531 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f015fe57b80 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a09a6f5cf0 0x55a09a762ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f015fe57b80 4 client.15531 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f015fe57b80 1 -- 172.21.15.115:0/4127064878 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a09a606b20 con 0x55a09a6f5cf0 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.773+0000 7f014a7fc700 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a09a6f5cf0 0x55a09a762ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f014a7fc700 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a09a6f5cf0 0x55a09a762ff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7106f1d7b9c4eaf9 server_cookie=2e19cadc0643ea2e in_seq=0 out_seq=0 2024-05-09T05:38:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f012eff5700 10 client.15531 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f012eff5700 1 -- 172.21.15.115:0/4127064878 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a09a606b20 con 0x55a09a6f5cf0 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f012eff5700 10 client.15531 handle_command_reply: tid=0 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 client.15531 shutdown 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 -- 172.21.15.115:0/4127064878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 msgr2=0x55a09a6ef460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6ef460 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f013c005400 tx=0x7f013c005430 comp rx=0 tx=0).stop 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 -- 172.21.15.115:0/4127064878 shutdown_connections 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a09a6f5cf0 0x55a09a762ff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a09a6dc400 0x55a09a75f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a09a55f4a0 0x55a09a6ef460 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f012eff5700 0 client.15531 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 --2- 172.21.15.115:0/4127064878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a09a5d9d00 0x55a09a6ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 -- 172.21.15.115:0/4127064878 >> 172.21.15.115:0/4127064878 conn(0x55a09a562f70 msgr2=0x55a09a6d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 -- 172.21.15.115:0/4127064878 shutdown_connections 2024-05-09T05:38:31.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 1 -- 172.21.15.115:0/4127064878 wait complete. 2024-05-09T05:38:31.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.774+0000 7f015fe57b80 20 client.15531 trim_cache size 0 max 16384 2024-05-09T05:38:31.774 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:38:31.774 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T05:38:31.774 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": { 2024-05-09T05:38:31.774 INFO:teuthology.orchestra.run.smithi115.stdout: "9c09d748-8d3a-46c8-8a64-dcad968c6552": { 2024-05-09T05:38:31.774 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/", 2024-05-09T05:38:31.774 INFO:teuthology.orchestra.run.smithi115.stdout: "tag": "9c09d748-8d3a-46c8-8a64-dcad968c6552", 2024-05-09T05:38:31.774 INFO:teuthology.orchestra.run.smithi115.stdout: "options": "recursive,force" 2024-05-09T05:38:31.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:38:31.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:38:31.775 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:38:31.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 -- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01340b2a80 msgr2=0x7f01340b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:31.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01340b2a80 0x7f01340b4f90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f013c005950 tx=0x7f013c005f50 comp rx=0 tx=0).stop 2024-05-09T05:38:31.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 -- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 msgr2=0x7f014c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:31.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 0x7f014c19b340 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f014000b580 tx=0x7f014000f740 comp rx=0 tx=0).stop 2024-05-09T05:38:31.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 -- 172.21.15.115:0/2082170164 shutdown_connections 2024-05-09T05:38:31.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01340b2a80 0x7f01340b4f90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c109850 0x7f014c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c102b50 0x7f014c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 --2- 172.21.15.115:0/2082170164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f014c108f00 0x7f014c19b340 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:31.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 -- 172.21.15.115:0/2082170164 >> 172.21.15.115:0/2082170164 conn(0x7f014c0fe6f0 msgr2=0x7f014c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:31.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 -- 172.21.15.115:0/2082170164 shutdown_connections 2024-05-09T05:38:31.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:31.775+0000 7f0151441700 1 -- 172.21.15.115:0/2082170164 wait complete. 2024-05-09T05:38:33.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:33 smithi115 ceph-mon[30349]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-09T05:38:33.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:33 smithi195 ceph-mon[33844]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-09T05:38:33.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:33 smithi161 ceph-mon[33805]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-09T05:38:33.663 INFO:tasks.cephfs.filesystem:scrub status for tag:9c09d748-8d3a-46c8-8a64-dcad968c6552 - {'path': '/', 'tag': '9c09d748-8d3a-46c8-8a64-dcad968c6552', 'options': 'recursive,force'} 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.193+0000 7fbc5873d700 1 -- 172.21.15.115:0/1708383083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50109870 msgr2=0x7fbc501111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.193+0000 7fbc5873d700 1 --2- 172.21.15.115:0/1708383083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50109870 0x7fbc501111d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800b0d0 tx=0x7fbc4800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.194+0000 7fbc5873d700 1 -- 172.21.15.115:0/1708383083 shutdown_connections 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.194+0000 7fbc5873d700 1 --2- 172.21.15.115:0/1708383083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc501045f0 0x7fbc50104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.194+0000 7fbc5873d700 1 --2- 172.21.15.115:0/1708383083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50109870 0x7fbc501111d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.194+0000 7fbc5873d700 1 --2- 172.21.15.115:0/1708383083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc50108f20 0x7fbc50109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.194+0000 7fbc5873d700 1 -- 172.21.15.115:0/1708383083 >> 172.21.15.115:0/1708383083 conn(0x7fbc50100040 msgr2=0x7fbc50102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.194+0000 7fbc5873d700 1 -- 172.21.15.115:0/1708383083 shutdown_connections 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.194+0000 7fbc5873d700 1 -- 172.21.15.115:0/1708383083 wait complete. 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.194+0000 7fbc5873d700 1 Processor -- start 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.194+0000 7fbc5873d700 1 -- start start 2024-05-09T05:38:34.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc5873d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc501045f0 0x7fbc5019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc5873d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50108f20 0x7fbc5019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc5873d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc50109870 0x7fbc5019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc5873d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50114350 con 0x7fbc50108f20 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc5873d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501141d0 con 0x7fbc50109870 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc5873d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501144d0 con 0x7fbc501045f0 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc55cd8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50108f20 0x7fbc5019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc55cd8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50108f20 0x7fbc5019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52420/0 (socket says 172.21.15.115:52420) 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc55cd8700 1 -- 172.21.15.115:0/3379112634 learned_addr learned my addr 172.21.15.115:0/3379112634 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc564d9700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc501045f0 0x7fbc5019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc56cda700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc50109870 0x7fbc5019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:34.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc55cd8700 1 -- 172.21.15.115:0/3379112634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc501045f0 msgr2=0x7fbc5019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc55cd8700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc501045f0 0x7fbc5019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc55cd8700 1 -- 172.21.15.115:0/3379112634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc50109870 msgr2=0x7fbc5019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc55cd8700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc50109870 0x7fbc5019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc55cd8700 1 -- 172.21.15.115:0/3379112634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5019fd30 con 0x7fbc50108f20 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc55cd8700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50108f20 0x7fbc5019b400 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400d980 tx=0x7fbc4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.195+0000 7fbc437fe700 1 -- 172.21.15.115:0/3379112634 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44017070 con 0x7fbc50108f20 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.196+0000 7fbc437fe700 1 -- 172.21.15.115:0/3379112634 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc4400fd20 con 0x7fbc50108f20 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.196+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc501a0020 con 0x7fbc50108f20 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.196+0000 7fbc437fe700 1 -- 172.21.15.115:0/3379112634 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400e3b0 con 0x7fbc50108f20 2024-05-09T05:38:34.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.196+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc501a7940 con 0x7fbc50108f20 2024-05-09T05:38:34.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.197+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc500512c0 con 0x7fbc50108f20 2024-05-09T05:38:34.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.198+0000 7fbc437fe700 1 -- 172.21.15.115:0/3379112634 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fbc4400f890 con 0x7fbc50108f20 2024-05-09T05:38:34.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.199+0000 7fbc437fe700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbc3c0b2b00 0x7fbc3c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:34.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.199+0000 7fbc564d9700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbc3c0b2b00 0x7fbc3c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:34.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.199+0000 7fbc437fe700 1 -- 172.21.15.115:0/3379112634 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbc440e27e0 con 0x7fbc50108f20 2024-05-09T05:38:34.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.199+0000 7fbc564d9700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbc3c0b2b00 0x7fbc3c0b5010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007be0 tx=0x7fbc4c0073d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:34.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.204+0000 7fbc437fe700 1 -- 172.21.15.115:0/3379112634 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc440a9d10 con 0x7fbc50108f20 2024-05-09T05:38:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.404+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc5019c1c0 con 0x7fbc3c0b2b00 2024-05-09T05:38:34.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.418+0000 7fbc437fe700 1 -- 172.21.15.115:0/3379112634 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339739 (secure 0 0 0) 0x7fbc5019c1c0 con 0x7fbc3c0b2b00 2024-05-09T05:38:34.415 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:34.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbc3c0b2b00 msgr2=0x7fbc3c0b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:34.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbc3c0b2b00 0x7fbc3c0b5010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007be0 tx=0x7fbc4c0073d0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50108f20 msgr2=0x7fbc5019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50108f20 0x7fbc5019b400 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400d980 tx=0x7fbc4400dc90 comp rx=0 tx=0).stop 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 shutdown_connections 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbc3c0b2b00 0x7fbc3c0b5010 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc50109870 0x7fbc5019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc501045f0 0x7fbc5019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 --2- 172.21.15.115:0/3379112634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc50108f20 0x7fbc5019b400 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 >> 172.21.15.115:0/3379112634 conn(0x7fbc50100040 msgr2=0x7fbc50108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 shutdown_connections 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:34.421+0000 7fbc5873d700 1 -- 172.21.15.115:0/3379112634 wait complete. 2024-05-09T05:38:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:38:35.195 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-05-09T05:38:33.947160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392123,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2642,"num_read_kb":1722,"num_write":657,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":995,"ondisk_log_size":995,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344752,"kb_used_data":12188,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597072,"statfs":{"total":1151940427776,"available":1151587401728,"internally_reserved":0,"allocated":12480512,"data_stored":10224683,"data_compressed":565383,"data_compressed_allocated":761856,"data_compressed_original":4853760,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":405,"num_read_kb":307,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003191"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925762+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:38:11.925762+0000","last_peered":"2024-05-09T05:38:11.925762+0000","last_clean":"2024-05-09T05:38:11.925762+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:38:11.925762+0000","last_undegraded":"2024-05-09T05:38:11.925762+0000","last_fullsized":"2024-05-09T05:38:11.925762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979667+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:38:11.979667+0000","last_peered":"2024-05-09T05:38:11.979667+0000","last_clean":"2024-05-09T05:38:11.979667+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:38:11.979667+0000","last_undegraded":"2024-05-09T05:38:11.979667+0000","last_fullsized":"2024-05-09T05:38:11.979667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979267+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:38:11.979267+0000","last_peered":"2024-05-09T05:38:11.979267+0000","last_clean":"2024-05-09T05:38:11.979267+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:38:11.979267+0000","last_undegraded":"2024-05-09T05:38:11.979267+0000","last_fullsized":"2024-05-09T05:38:11.979267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133401+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:38:12.133401+0000","last_peered":"2024-05-09T05:38:12.133401+0000","last_clean":"2024-05-09T05:38:12.133401+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:38:12.133401+0000","last_undegraded":"2024-05-09T05:38:12.133401+0000","last_fullsized":"2024-05-09T05:38:12.133401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051332+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:38:12.051332+0000","last_peered":"2024-05-09T05:38:12.051332+0000","last_clean":"2024-05-09T05:38:12.051332+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:38:12.051332+0000","last_undegraded":"2024-05-09T05:38:12.051332+0000","last_fullsized":"2024-05-09T05:38:12.051332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925938+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:38:11.925938+0000","last_peered":"2024-05-09T05:38:11.925938+0000","last_clean":"2024-05-09T05:38:11.925938+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:38:11.925938+0000","last_undegraded":"2024-05-09T05:38:11.925938+0000","last_fullsized":"2024-05-09T05:38:11.925938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132223+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:38:12.132223+0000","last_peered":"2024-05-09T05:38:12.132223+0000","last_clean":"2024-05-09T05:38:12.132223+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:38:12.132223+0000","last_undegraded":"2024-05-09T05:38:12.132223+0000","last_fullsized":"2024-05-09T05:38:12.132223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051904+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:38:12.051904+0000","last_peered":"2024-05-09T05:38:12.051904+0000","last_clean":"2024-05-09T05:38:12.051904+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:38:12.051904+0000","last_undegraded":"2024-05-09T05:38:12.051904+0000","last_fullsized":"2024-05-09T05:38:12.051904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134129+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:38:12.134129+0000","last_peered":"2024-05-09T05:38:12.134129+0000","last_clean":"2024-05-09T05:38:12.134129+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:38:12.134129+0000","last_undegraded":"2024-05-09T05:38:12.134129+0000","last_fullsized":"2024-05-09T05:38:12.134129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.926126+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:38:11.926126+0000","last_peered":"2024-05-09T05:38:11.926126+0000","last_clean":"2024-05-09T05:38:11.926126+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:38:11.926126+0000","last_undegraded":"2024-05-09T05:38:11.926126+0000","last_fullsized":"2024-05-09T05:38:11.926126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133832+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:38:12.133832+0000","last_peered":"2024-05-09T05:38:12.133832+0000","last_clean":"2024-05-09T05:38:12.133832+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:38:12.133832+0000","last_undegraded":"2024-05-09T05:38:12.133832+0000","last_fullsized":"2024-05-09T05:38:12.133832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133357+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:38:12.133357+0000","last_peered":"2024-05-09T05:38:12.133357+0000","last_clean":"2024-05-09T05:38:12.133357+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:38:12.133357+0000","last_undegraded":"2024-05-09T05:38:12.133357+0000","last_fullsized":"2024-05-09T05:38:12.133357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225339+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:38:12.225339+0000","last_peered":"2024-05-09T05:38:12.225339+0000","last_clean":"2024-05-09T05:38:12.225339+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:38:12.225339+0000","last_undegraded":"2024-05-09T05:38:12.225339+0000","last_fullsized":"2024-05-09T05:38:12.225339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979195+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:38:11.979195+0000","last_peered":"2024-05-09T05:38:11.979195+0000","last_clean":"2024-05-09T05:38:11.979195+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:38:11.979195+0000","last_undegraded":"2024-05-09T05:38:11.979195+0000","last_fullsized":"2024-05-09T05:38:11.979195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979140+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:38:11.979140+0000","last_peered":"2024-05-09T05:38:11.979140+0000","last_clean":"2024-05-09T05:38:11.979140+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:38:11.979140+0000","last_undegraded":"2024-05-09T05:38:11.979140+0000","last_fullsized":"2024-05-09T05:38:11.979140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132115+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:38:12.132115+0000","last_peered":"2024-05-09T05:38:12.132115+0000","last_clean":"2024-05-09T05:38:12.132115+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:38:12.132115+0000","last_undegraded":"2024-05-09T05:38:12.132115+0000","last_fullsized":"2024-05-09T05:38:12.132115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925542+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:38:11.925542+0000","last_peered":"2024-05-09T05:38:11.925542+0000","last_clean":"2024-05-09T05:38:11.925542+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:38:11.925542+0000","last_undegraded":"2024-05-09T05:38:11.925542+0000","last_fullsized":"2024-05-09T05:38:11.925542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924841+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:38:11.924841+0000","last_peered":"2024-05-09T05:38:11.924841+0000","last_clean":"2024-05-09T05:38:11.924841+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:38:11.924841+0000","last_undegraded":"2024-05-09T05:38:11.924841+0000","last_fullsized":"2024-05-09T05:38:11.924841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132289+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:38:12.132289+0000","last_peered":"2024-05-09T05:38:12.132289+0000","last_clean":"2024-05-09T05:38:12.132289+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:38:12.132289+0000","last_undegraded":"2024-05-09T05:38:12.132289+0000","last_fullsized":"2024-05-09T05:38:12.132289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131837+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:38:12.131837+0000","last_peered":"2024-05-09T05:38:12.131837+0000","last_clean":"2024-05-09T05:38:12.131837+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:38:12.131837+0000","last_undegraded":"2024-05-09T05:38:12.131837+0000","last_fullsized":"2024-05-09T05:38:12.131837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":270,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.592616+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:38:31.592616+0000","last_peered":"2024-05-09T05:38:31.592616+0000","last_clean":"2024-05-09T05:38:31.592616+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:38:31.592616+0000","last_undegraded":"2024-05-09T05:38:31.592616+0000","last_fullsized":"2024-05-09T05:38:31.592616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053507+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:38:12.053507+0000","last_peered":"2024-05-09T05:38:12.053507+0000","last_clean":"2024-05-09T05:38:12.053507+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:38:12.053507+0000","last_undegraded":"2024-05-09T05:38:12.053507+0000","last_fullsized":"2024-05-09T05:38:12.053507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132572+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:38:12.132572+0000","last_peered":"2024-05-09T05:38:12.132572+0000","last_clean":"2024-05-09T05:38:12.132572+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:38:12.132572+0000","last_undegraded":"2024-05-09T05:38:12.132572+0000","last_fullsized":"2024-05-09T05:38:12.132572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053764+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:38:12.053764+0000","last_peered":"2024-05-09T05:38:12.053764+0000","last_clean":"2024-05-09T05:38:12.053764+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:38:12.053764+0000","last_undegraded":"2024-05-09T05:38:12.053764+0000","last_fullsized":"2024-05-09T05:38:12.053764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979670+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:38:11.979670+0000","last_peered":"2024-05-09T05:38:11.979670+0000","last_clean":"2024-05-09T05:38:11.979670+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:38:11.979670+0000","last_undegraded":"2024-05-09T05:38:11.979670+0000","last_fullsized":"2024-05-09T05:38:11.979670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132963+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:38:12.132963+0000","last_peered":"2024-05-09T05:38:12.132963+0000","last_clean":"2024-05-09T05:38:12.132963+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:38:12.132963+0000","last_undegraded":"2024-05-09T05:38:12.132963+0000","last_fullsized":"2024-05-09T05:38:12.132963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051620+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:38:12.051620+0000","last_peered":"2024-05-09T05:38:12.051620+0000","last_clean":"2024-05-09T05:38:12.051620+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:38:12.051620+0000","last_undegraded":"2024-05-09T05:38:12.051620+0000","last_fullsized":"2024-05-09T05:38:12.051620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133077+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:38:12.133077+0000","last_peered":"2024-05-09T05:38:12.133077+0000","last_clean":"2024-05-09T05:38:12.133077+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:38:12.133077+0000","last_undegraded":"2024-05-09T05:38:12.133077+0000","last_fullsized":"2024-05-09T05:38:12.133077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132705+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:38:12.132705+0000","last_peered":"2024-05-09T05:38:12.132705+0000","last_clean":"2024-05-09T05:38:12.132705+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:38:12.132705+0000","last_undegraded":"2024-05-09T05:38:12.132705+0000","last_fullsized":"2024-05-09T05:38:12.132705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051663+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:38:12.051663+0000","last_peered":"2024-05-09T05:38:12.051663+0000","last_clean":"2024-05-09T05:38:12.051663+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:38:12.051663+0000","last_undegraded":"2024-05-09T05:38:12.051663+0000","last_fullsized":"2024-05-09T05:38:12.051663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059550+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:38:12.059550+0000","last_peered":"2024-05-09T05:38:12.059550+0000","last_clean":"2024-05-09T05:38:12.059550+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:38:12.059550+0000","last_undegraded":"2024-05-09T05:38:12.059550+0000","last_fullsized":"2024-05-09T05:38:12.059550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131984+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:38:12.131984+0000","last_peered":"2024-05-09T05:38:12.131984+0000","last_clean":"2024-05-09T05:38:12.131984+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:38:12.131984+0000","last_undegraded":"2024-05-09T05:38:12.131984+0000","last_fullsized":"2024-05-09T05:38:12.131984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":265,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.619785+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:38:31.619785+0000","last_peered":"2024-05-09T05:38:31.619785+0000","last_clean":"2024-05-09T05:38:31.619785+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:38:31.619785+0000","last_undegraded":"2024-05-09T05:38:31.619785+0000","last_fullsized":"2024-05-09T05:38:31.619785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051377+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:38:12.051377+0000","last_peered":"2024-05-09T05:38:12.051377+0000","last_clean":"2024-05-09T05:38:12.051377+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:38:12.051377+0000","last_undegraded":"2024-05-09T05:38:12.051377+0000","last_fullsized":"2024-05-09T05:38:12.051377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132066+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:38:12.132066+0000","last_peered":"2024-05-09T05:38:12.132066+0000","last_clean":"2024-05-09T05:38:12.132066+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:38:12.132066+0000","last_undegraded":"2024-05-09T05:38:12.132066+0000","last_fullsized":"2024-05-09T05:38:12.132066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925853+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:38:11.925853+0000","last_peered":"2024-05-09T05:38:11.925853+0000","last_clean":"2024-05-09T05:38:11.925853+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:38:11.925853+0000","last_undegraded":"2024-05-09T05:38:11.925853+0000","last_fullsized":"2024-05-09T05:38:11.925853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059515+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:38:12.059515+0000","last_peered":"2024-05-09T05:38:12.059515+0000","last_clean":"2024-05-09T05:38:12.059515+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:38:12.059515+0000","last_undegraded":"2024-05-09T05:38:12.059515+0000","last_fullsized":"2024-05-09T05:38:12.059515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925545+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:38:11.925545+0000","last_peered":"2024-05-09T05:38:11.925545+0000","last_clean":"2024-05-09T05:38:11.925545+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:38:11.925545+0000","last_undegraded":"2024-05-09T05:38:11.925545+0000","last_fullsized":"2024-05-09T05:38:11.925545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224943+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:38:12.224943+0000","last_peered":"2024-05-09T05:38:12.224943+0000","last_clean":"2024-05-09T05:38:12.224943+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:38:12.224943+0000","last_undegraded":"2024-05-09T05:38:12.224943+0000","last_fullsized":"2024-05-09T05:38:12.224943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052036+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:38:12.052036+0000","last_peered":"2024-05-09T05:38:12.052036+0000","last_clean":"2024-05-09T05:38:12.052036+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:38:12.052036+0000","last_undegraded":"2024-05-09T05:38:12.052036+0000","last_fullsized":"2024-05-09T05:38:12.052036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979576+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:38:11.979576+0000","last_peered":"2024-05-09T05:38:11.979576+0000","last_clean":"2024-05-09T05:38:11.979576+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:38:11.979576+0000","last_undegraded":"2024-05-09T05:38:11.979576+0000","last_fullsized":"2024-05-09T05:38:11.979576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059004+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:38:12.059004+0000","last_peered":"2024-05-09T05:38:12.059004+0000","last_clean":"2024-05-09T05:38:12.059004+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:38:12.059004+0000","last_undegraded":"2024-05-09T05:38:12.059004+0000","last_fullsized":"2024-05-09T05:38:12.059004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924546+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:38:11.924546+0000","last_peered":"2024-05-09T05:38:11.924546+0000","last_clean":"2024-05-09T05:38:11.924546+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:38:11.924546+0000","last_undegraded":"2024-05-09T05:38:11.924546+0000","last_fullsized":"2024-05-09T05:38:11.924546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053026+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:38:12.053026+0000","last_peered":"2024-05-09T05:38:12.053026+0000","last_clean":"2024-05-09T05:38:12.053026+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:38:12.053026+0000","last_undegraded":"2024-05-09T05:38:12.053026+0000","last_fullsized":"2024-05-09T05:38:12.053026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'7","reported_seq":469,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:32.620577+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:38:32.620577+0000","last_peered":"2024-05-09T05:38:32.620577+0000","last_clean":"2024-05-09T05:38:32.620577+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:38:32.620577+0000","last_undegraded":"2024-05-09T05:38:32.620577+0000","last_fullsized":"2024-05-09T05:38:32.620577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":420,"num_read_kb":420,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133858+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:38:12.133858+0000","last_peered":"2024-05-09T05:38:12.133858+0000","last_clean":"2024-05-09T05:38:12.133858+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:38:12.133858+0000","last_undegraded":"2024-05-09T05:38:12.133858+0000","last_fullsized":"2024-05-09T05:38:12.133858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924463+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:38:11.924463+0000","last_peered":"2024-05-09T05:38:11.924463+0000","last_clean":"2024-05-09T05:38:11.924463+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:38:11.924463+0000","last_undegraded":"2024-05-09T05:38:11.924463+0000","last_fullsized":"2024-05-09T05:38:11.924463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133086+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:38:12.133086+0000","last_peered":"2024-05-09T05:38:12.133086+0000","last_clean":"2024-05-09T05:38:12.133086+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:38:12.133086+0000","last_undegraded":"2024-05-09T05:38:12.133086+0000","last_fullsized":"2024-05-09T05:38:12.133086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979599+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:38:11.979599+0000","last_peered":"2024-05-09T05:38:11.979599+0000","last_clean":"2024-05-09T05:38:11.979599+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:38:11.979599+0000","last_undegraded":"2024-05-09T05:38:11.979599+0000","last_fullsized":"2024-05-09T05:38:11.979599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133847+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:38:12.133847+0000","last_peered":"2024-05-09T05:38:12.133847+0000","last_clean":"2024-05-09T05:38:12.133847+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:38:12.133847+0000","last_undegraded":"2024-05-09T05:38:12.133847+0000","last_fullsized":"2024-05-09T05:38:12.133847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059520+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:38:12.059520+0000","last_peered":"2024-05-09T05:38:12.059520+0000","last_clean":"2024-05-09T05:38:12.059520+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:38:12.059520+0000","last_undegraded":"2024-05-09T05:38:12.059520+0000","last_fullsized":"2024-05-09T05:38:12.059520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133129+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:38:12.133129+0000","last_peered":"2024-05-09T05:38:12.133129+0000","last_clean":"2024-05-09T05:38:12.133129+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:38:12.133129+0000","last_undegraded":"2024-05-09T05:38:12.133129+0000","last_fullsized":"2024-05-09T05:38:12.133129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924514+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:38:11.924514+0000","last_peered":"2024-05-09T05:38:11.924514+0000","last_clean":"2024-05-09T05:38:11.924514+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:38:11.924514+0000","last_undegraded":"2024-05-09T05:38:11.924514+0000","last_fullsized":"2024-05-09T05:38:11.924514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'43","reported_seq":519,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:32.625423+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:38:32.625423+0000","last_peered":"2024-05-09T05:38:32.625423+0000","last_clean":"2024-05-09T05:38:32.625423+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:38:32.625423+0000","last_undegraded":"2024-05-09T05:38:32.625423+0000","last_fullsized":"2024-05-09T05:38:32.625423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69974,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":433,"num_read_kb":285,"num_write":45,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051722+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:38:12.051722+0000","last_peered":"2024-05-09T05:38:12.051722+0000","last_clean":"2024-05-09T05:38:12.051722+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:38:12.051722+0000","last_undegraded":"2024-05-09T05:38:12.051722+0000","last_fullsized":"2024-05-09T05:38:12.051722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059857+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:38:12.059857+0000","last_peered":"2024-05-09T05:38:12.059857+0000","last_clean":"2024-05-09T05:38:12.059857+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:38:12.059857+0000","last_undegraded":"2024-05-09T05:38:12.059857+0000","last_fullsized":"2024-05-09T05:38:12.059857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132213+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:38:12.132213+0000","last_peered":"2024-05-09T05:38:12.132213+0000","last_clean":"2024-05-09T05:38:12.132213+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:38:12.132213+0000","last_undegraded":"2024-05-09T05:38:12.132213+0000","last_fullsized":"2024-05-09T05:38:12.132213+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225225+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:38:12.225225+0000","last_peered":"2024-05-09T05:38:12.225225+0000","last_clean":"2024-05-09T05:38:12.225225+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:38:12.225225+0000","last_undegraded":"2024-05-09T05:38:12.225225+0000","last_fullsized":"2024-05-09T05:38:12.225225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059364+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:38:12.059364+0000","last_peered":"2024-05-09T05:38:12.059364+0000","last_clean":"2024-05-09T05:38:12.059364+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:38:12.059364+0000","last_undegraded":"2024-05-09T05:38:12.059364+0000","last_fullsized":"2024-05-09T05:38:12.059364+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132766+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:38:12.132766+0000","last_peered":"2024-05-09T05:38:12.132766+0000","last_clean":"2024-05-09T05:38:12.132766+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:38:12.132766+0000","last_undegraded":"2024-05-09T05:38:12.132766+0000","last_fullsized":"2024-05-09T05:38:12.132766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051717+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:38:12.051717+0000","last_peered":"2024-05-09T05:38:12.051717+0000","last_clean":"2024-05-09T05:38:12.051717+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:38:12.051717+0000","last_undegraded":"2024-05-09T05:38:12.051717+0000","last_fullsized":"2024-05-09T05:38:12.051717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":265,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:32.594602+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:38:32.594602+0000","last_peered":"2024-05-09T05:38:32.594602+0000","last_clean":"2024-05-09T05:38:32.594602+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:38:32.594602+0000","last_undegraded":"2024-05-09T05:38:32.594602+0000","last_fullsized":"2024-05-09T05:38:32.594602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132385+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:38:12.132385+0000","last_peered":"2024-05-09T05:38:12.132385+0000","last_clean":"2024-05-09T05:38:12.132385+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:38:12.132385+0000","last_undegraded":"2024-05-09T05:38:12.132385+0000","last_fullsized":"2024-05-09T05:38:12.132385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225537+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:38:12.225537+0000","last_peered":"2024-05-09T05:38:12.225537+0000","last_clean":"2024-05-09T05:38:12.225537+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:38:12.225537+0000","last_undegraded":"2024-05-09T05:38:12.225537+0000","last_fullsized":"2024-05-09T05:38:12.225537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051295+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:38:12.051295+0000","last_peered":"2024-05-09T05:38:12.051295+0000","last_clean":"2024-05-09T05:38:12.051295+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:38:12.051295+0000","last_undegraded":"2024-05-09T05:38:12.051295+0000","last_fullsized":"2024-05-09T05:38:12.051295+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924664+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:38:11.924664+0000","last_peered":"2024-05-09T05:38:11.924664+0000","last_clean":"2024-05-09T05:38:11.924664+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:38:11.924664+0000","last_undegraded":"2024-05-09T05:38:11.924664+0000","last_fullsized":"2024-05-09T05:38:11.924664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059379+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:38:12.059379+0000","last_peered":"2024-05-09T05:38:12.059379+0000","last_clean":"2024-05-09T05:38:12.059379+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:38:12.059379+0000","last_undegraded":"2024-05-09T05:38:12.059379+0000","last_fullsized":"2024-05-09T05:38:12.059379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051937+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:38:12.051937+0000","last_peered":"2024-05-09T05:38:12.051937+0000","last_clean":"2024-05-09T05:38:12.051937+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:38:12.051937+0000","last_undegraded":"2024-05-09T05:38:12.051937+0000","last_fullsized":"2024-05-09T05:38:12.051937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925408+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:38:11.925408+0000","last_peered":"2024-05-09T05:38:11.925408+0000","last_clean":"2024-05-09T05:38:11.925408+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:38:11.925408+0000","last_undegraded":"2024-05-09T05:38:11.925408+0000","last_fullsized":"2024-05-09T05:38:11.925408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051341+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:38:12.051341+0000","last_peered":"2024-05-09T05:38:12.051341+0000","last_clean":"2024-05-09T05:38:12.051341+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:38:12.051341+0000","last_undegraded":"2024-05-09T05:38:12.051341+0000","last_fullsized":"2024-05-09T05:38:12.051341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925850+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:38:11.925850+0000","last_peered":"2024-05-09T05:38:11.925850+0000","last_clean":"2024-05-09T05:38:11.925850+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:38:11.925850+0000","last_undegraded":"2024-05-09T05:38:11.925850+0000","last_fullsized":"2024-05-09T05:38:11.925850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225147+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:38:12.225147+0000","last_peered":"2024-05-09T05:38:12.225147+0000","last_clean":"2024-05-09T05:38:12.225147+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:38:12.225147+0000","last_undegraded":"2024-05-09T05:38:12.225147+0000","last_fullsized":"2024-05-09T05:38:12.225147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'7","reported_seq":264,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.593165+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:38:31.593165+0000","last_peered":"2024-05-09T05:38:31.593165+0000","last_clean":"2024-05-09T05:38:31.593165+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:38:31.593165+0000","last_undegraded":"2024-05-09T05:38:31.593165+0000","last_fullsized":"2024-05-09T05:38:31.593165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":213,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925837+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:38:11.925837+0000","last_peered":"2024-05-09T05:38:11.925837+0000","last_clean":"2024-05-09T05:38:11.925837+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:38:11.925837+0000","last_undegraded":"2024-05-09T05:38:11.925837+0000","last_fullsized":"2024-05-09T05:38:11.925837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225711+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:38:12.225711+0000","last_peered":"2024-05-09T05:38:12.225711+0000","last_clean":"2024-05-09T05:38:12.225711+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:38:12.225711+0000","last_undegraded":"2024-05-09T05:38:12.225711+0000","last_fullsized":"2024-05-09T05:38:12.225711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059336+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:38:12.059336+0000","last_peered":"2024-05-09T05:38:12.059336+0000","last_clean":"2024-05-09T05:38:12.059336+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:38:12.059336+0000","last_undegraded":"2024-05-09T05:38:12.059336+0000","last_fullsized":"2024-05-09T05:38:12.059336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":260,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.592617+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:38:31.592617+0000","last_peered":"2024-05-09T05:38:31.592617+0000","last_clean":"2024-05-09T05:38:31.592617+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:38:31.592617+0000","last_undegraded":"2024-05-09T05:38:31.592617+0000","last_fullsized":"2024-05-09T05:38:31.592617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":211,"num_read_kb":211,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133686+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:38:12.133686+0000","last_peered":"2024-05-09T05:38:12.133686+0000","last_clean":"2024-05-09T05:38:12.133686+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:38:12.133686+0000","last_undegraded":"2024-05-09T05:38:12.133686+0000","last_fullsized":"2024-05-09T05:38:12.133686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052441+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:38:12.052441+0000","last_peered":"2024-05-09T05:38:12.052441+0000","last_clean":"2024-05-09T05:38:12.052441+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:38:12.052441+0000","last_undegraded":"2024-05-09T05:38:12.052441+0000","last_fullsized":"2024-05-09T05:38:12.052441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133733+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:38:12.133733+0000","last_peered":"2024-05-09T05:38:12.133733+0000","last_clean":"2024-05-09T05:38:12.133733+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:38:12.133733+0000","last_undegraded":"2024-05-09T05:38:12.133733+0000","last_fullsized":"2024-05-09T05:38:12.133733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"88'99","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:14.055036+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:38:14.055036+0000","last_peered":"2024-05-09T05:38:14.055036+0000","last_clean":"2024-05-09T05:38:14.055036+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:38:14.055036+0000","last_undegraded":"2024-05-09T05:38:14.055036+0000","last_fullsized":"2024-05-09T05:38:14.055036+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059582+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:38:12.059582+0000","last_peered":"2024-05-09T05:38:12.059582+0000","last_clean":"2024-05-09T05:38:12.059582+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:38:12.059582+0000","last_undegraded":"2024-05-09T05:38:12.059582+0000","last_fullsized":"2024-05-09T05:38:12.059582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224704+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:38:12.224704+0000","last_peered":"2024-05-09T05:38:12.224704+0000","last_clean":"2024-05-09T05:38:12.224704+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:38:12.224704+0000","last_undegraded":"2024-05-09T05:38:12.224704+0000","last_fullsized":"2024-05-09T05:38:12.224704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132204+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:38:12.132204+0000","last_peered":"2024-05-09T05:38:12.132204+0000","last_clean":"2024-05-09T05:38:12.132204+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:38:12.132204+0000","last_undegraded":"2024-05-09T05:38:12.132204+0000","last_fullsized":"2024-05-09T05:38:12.132204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925156+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:38:11.925156+0000","last_peered":"2024-05-09T05:38:11.925156+0000","last_clean":"2024-05-09T05:38:11.925156+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:38:11.925156+0000","last_undegraded":"2024-05-09T05:38:11.925156+0000","last_fullsized":"2024-05-09T05:38:11.925156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925336+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:38:11.925336+0000","last_peered":"2024-05-09T05:38:11.925336+0000","last_clean":"2024-05-09T05:38:11.925336+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:38:11.925336+0000","last_undegraded":"2024-05-09T05:38:11.925336+0000","last_fullsized":"2024-05-09T05:38:11.925336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225403+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:38:12.225403+0000","last_peered":"2024-05-09T05:38:12.225403+0000","last_clean":"2024-05-09T05:38:12.225403+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:38:12.225403+0000","last_undegraded":"2024-05-09T05:38:12.225403+0000","last_fullsized":"2024-05-09T05:38:12.225403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051969+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:38:12.051969+0000","last_peered":"2024-05-09T05:38:12.051969+0000","last_clean":"2024-05-09T05:38:12.051969+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:38:12.051969+0000","last_undegraded":"2024-05-09T05:38:12.051969+0000","last_fullsized":"2024-05-09T05:38:12.051969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'107","reported_seq":243,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:26.562070+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:38:26.562070+0000","last_peered":"2024-05-09T05:38:26.562070+0000","last_clean":"2024-05-09T05:38:26.562070+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:38:26.562070+0000","last_undegraded":"2024-05-09T05:38:26.562070+0000","last_fullsized":"2024-05-09T05:38:26.562070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":195,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053724+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:38:12.053724+0000","last_peered":"2024-05-09T05:38:12.053724+0000","last_clean":"2024-05-09T05:38:12.053724+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:38:12.053724+0000","last_undegraded":"2024-05-09T05:38:12.053724+0000","last_fullsized":"2024-05-09T05:38:12.053724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131803+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:38:12.131803+0000","last_peered":"2024-05-09T05:38:12.131803+0000","last_clean":"2024-05-09T05:38:12.131803+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:38:12.131803+0000","last_undegraded":"2024-05-09T05:38:12.131803+0000","last_fullsized":"2024-05-09T05:38:12.131803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134405+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:38:12.134405+0000","last_peered":"2024-05-09T05:38:12.134405+0000","last_clean":"2024-05-09T05:38:12.134405+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:38:12.134405+0000","last_undegraded":"2024-05-09T05:38:12.134405+0000","last_fullsized":"2024-05-09T05:38:12.134405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979676+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:38:11.979676+0000","last_peered":"2024-05-09T05:38:11.979676+0000","last_clean":"2024-05-09T05:38:11.979676+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:38:11.979676+0000","last_undegraded":"2024-05-09T05:38:11.979676+0000","last_fullsized":"2024-05-09T05:38:11.979676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133578+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:38:12.133578+0000","last_peered":"2024-05-09T05:38:12.133578+0000","last_clean":"2024-05-09T05:38:12.133578+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:38:12.133578+0000","last_undegraded":"2024-05-09T05:38:12.133578+0000","last_fullsized":"2024-05-09T05:38:12.133578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132717+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:38:12.132717+0000","last_peered":"2024-05-09T05:38:12.132717+0000","last_clean":"2024-05-09T05:38:12.132717+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:38:12.132717+0000","last_undegraded":"2024-05-09T05:38:12.132717+0000","last_fullsized":"2024-05-09T05:38:12.132717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051314+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:38:12.051314+0000","last_peered":"2024-05-09T05:38:12.051314+0000","last_clean":"2024-05-09T05:38:12.051314+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:38:12.051314+0000","last_undegraded":"2024-05-09T05:38:12.051314+0000","last_fullsized":"2024-05-09T05:38:12.051314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924777+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:38:11.924777+0000","last_peered":"2024-05-09T05:38:11.924777+0000","last_clean":"2024-05-09T05:38:11.924777+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:38:11.924777+0000","last_undegraded":"2024-05-09T05:38:11.924777+0000","last_fullsized":"2024-05-09T05:38:11.924777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053182+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:38:12.053182+0000","last_peered":"2024-05-09T05:38:12.053182+0000","last_clean":"2024-05-09T05:38:12.053182+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:38:12.053182+0000","last_undegraded":"2024-05-09T05:38:12.053182+0000","last_fullsized":"2024-05-09T05:38:12.053182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924983+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:38:11.924983+0000","last_peered":"2024-05-09T05:38:11.924983+0000","last_clean":"2024-05-09T05:38:11.924983+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:38:11.924983+0000","last_undegraded":"2024-05-09T05:38:11.924983+0000","last_fullsized":"2024-05-09T05:38:11.924983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133063+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:38:12.133063+0000","last_peered":"2024-05-09T05:38:12.133063+0000","last_clean":"2024-05-09T05:38:12.133063+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:38:12.133063+0000","last_undegraded":"2024-05-09T05:38:12.133063+0000","last_fullsized":"2024-05-09T05:38:12.133063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.315433+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:38:12.315433+0000","last_peered":"2024-05-09T05:38:12.315433+0000","last_clean":"2024-05-09T05:38:12.315433+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:38:12.315433+0000","last_undegraded":"2024-05-09T05:38:12.315433+0000","last_fullsized":"2024-05-09T05:38:12.315433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053097+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:38:12.053097+0000","last_peered":"2024-05-09T05:38:12.053097+0000","last_clean":"2024-05-09T05:38:12.053097+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:38:12.053097+0000","last_undegraded":"2024-05-09T05:38:12.053097+0000","last_fullsized":"2024-05-09T05:38:12.053097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133542+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:38:12.133542+0000","last_peered":"2024-05-09T05:38:12.133542+0000","last_clean":"2024-05-09T05:38:12.133542+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:38:12.133542+0000","last_undegraded":"2024-05-09T05:38:12.133542+0000","last_fullsized":"2024-05-09T05:38:12.133542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051419+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:38:12.051419+0000","last_peered":"2024-05-09T05:38:12.051419+0000","last_clean":"2024-05-09T05:38:12.051419+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:38:12.051419+0000","last_undegraded":"2024-05-09T05:38:12.051419+0000","last_fullsized":"2024-05-09T05:38:12.051419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132543+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:38:12.132543+0000","last_peered":"2024-05-09T05:38:12.132543+0000","last_clean":"2024-05-09T05:38:12.132543+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:38:12.132543+0000","last_undegraded":"2024-05-09T05:38:12.132543+0000","last_fullsized":"2024-05-09T05:38:12.132543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133080+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:38:12.133080+0000","last_peered":"2024-05-09T05:38:12.133080+0000","last_clean":"2024-05-09T05:38:12.133080+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:38:12.133080+0000","last_undegraded":"2024-05-09T05:38:12.133080+0000","last_fullsized":"2024-05-09T05:38:12.133080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":264,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.656965+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:38:31.656965+0000","last_peered":"2024-05-09T05:38:31.656965+0000","last_clean":"2024-05-09T05:38:31.656965+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:38:31.656965+0000","last_undegraded":"2024-05-09T05:38:31.656965+0000","last_fullsized":"2024-05-09T05:38:31.656965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133983+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:38:12.133983+0000","last_peered":"2024-05-09T05:38:12.133983+0000","last_clean":"2024-05-09T05:38:12.133983+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:38:12.133983+0000","last_undegraded":"2024-05-09T05:38:12.133983+0000","last_fullsized":"2024-05-09T05:38:12.133983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979240+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:38:11.979240+0000","last_peered":"2024-05-09T05:38:11.979240+0000","last_clean":"2024-05-09T05:38:11.979240+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:38:11.979240+0000","last_undegraded":"2024-05-09T05:38:11.979240+0000","last_fullsized":"2024-05-09T05:38:11.979240+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'67","reported_seq":141,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:26.196028+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:38:26.196028+0000","last_peered":"2024-05-09T05:38:26.196028+0000","last_clean":"2024-05-09T05:38:26.196028+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:38:26.196028+0000","last_undegraded":"2024-05-09T05:38:26.196028+0000","last_fullsized":"2024-05-09T05:38:26.196028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132662+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:38:12.132662+0000","last_peered":"2024-05-09T05:38:12.132662+0000","last_clean":"2024-05-09T05:38:12.132662+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:38:12.132662+0000","last_undegraded":"2024-05-09T05:38:12.132662+0000","last_fullsized":"2024-05-09T05:38:12.132662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925491+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:38:11.925491+0000","last_peered":"2024-05-09T05:38:11.925491+0000","last_clean":"2024-05-09T05:38:11.925491+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:38:11.925491+0000","last_undegraded":"2024-05-09T05:38:11.925491+0000","last_fullsized":"2024-05-09T05:38:11.925491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225592+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:38:12.225592+0000","last_peered":"2024-05-09T05:38:12.225592+0000","last_clean":"2024-05-09T05:38:12.225592+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:38:12.225592+0000","last_undegraded":"2024-05-09T05:38:12.225592+0000","last_fullsized":"2024-05-09T05:38:12.225592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133498+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:38:12.133498+0000","last_peered":"2024-05-09T05:38:12.133498+0000","last_clean":"2024-05-09T05:38:12.133498+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:38:12.133498+0000","last_undegraded":"2024-05-09T05:38:12.133498+0000","last_fullsized":"2024-05-09T05:38:12.133498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925076+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:38:11.925076+0000","last_peered":"2024-05-09T05:38:11.925076+0000","last_clean":"2024-05-09T05:38:11.925076+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:38:11.925076+0000","last_undegraded":"2024-05-09T05:38:11.925076+0000","last_fullsized":"2024-05-09T05:38:11.925076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132222+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:38:12.132222+0000","last_peered":"2024-05-09T05:38:12.132222+0000","last_clean":"2024-05-09T05:38:12.132222+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:38:12.132222+0000","last_undegraded":"2024-05-09T05:38:12.132222+0000","last_fullsized":"2024-05-09T05:38:12.132222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131810+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:38:12.131810+0000","last_peered":"2024-05-09T05:38:12.131810+0000","last_clean":"2024-05-09T05:38:12.131810+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:38:12.131810+0000","last_undegraded":"2024-05-09T05:38:12.131810+0000","last_fullsized":"2024-05-09T05:38:12.131810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979502+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:38:11.979502+0000","last_peered":"2024-05-09T05:38:11.979502+0000","last_clean":"2024-05-09T05:38:11.979502+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:38:11.979502+0000","last_undegraded":"2024-05-09T05:38:11.979502+0000","last_fullsized":"2024-05-09T05:38:11.979502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225302+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:38:12.225302+0000","last_peered":"2024-05-09T05:38:12.225302+0000","last_clean":"2024-05-09T05:38:12.225302+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:38:12.225302+0000","last_undegraded":"2024-05-09T05:38:12.225302+0000","last_fullsized":"2024-05-09T05:38:12.225302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925541+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:38:11.925541+0000","last_peered":"2024-05-09T05:38:11.925541+0000","last_clean":"2024-05-09T05:38:11.925541+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:38:11.925541+0000","last_undegraded":"2024-05-09T05:38:11.925541+0000","last_fullsized":"2024-05-09T05:38:11.925541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132345+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:38:12.132345+0000","last_peered":"2024-05-09T05:38:12.132345+0000","last_clean":"2024-05-09T05:38:12.132345+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:38:12.132345+0000","last_undegraded":"2024-05-09T05:38:12.132345+0000","last_fullsized":"2024-05-09T05:38:12.132345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053867+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:38:12.053867+0000","last_peered":"2024-05-09T05:38:12.053867+0000","last_clean":"2024-05-09T05:38:12.053867+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:38:12.053867+0000","last_undegraded":"2024-05-09T05:38:12.053867+0000","last_fullsized":"2024-05-09T05:38:12.053867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059147+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:38:12.059147+0000","last_peered":"2024-05-09T05:38:12.059147+0000","last_clean":"2024-05-09T05:38:12.059147+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:38:12.059147+0000","last_undegraded":"2024-05-09T05:38:12.059147+0000","last_fullsized":"2024-05-09T05:38:12.059147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053305+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:38:12.053305+0000","last_peered":"2024-05-09T05:38:12.053305+0000","last_clean":"2024-05-09T05:38:12.053305+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:38:12.053305+0000","last_undegraded":"2024-05-09T05:38:12.053305+0000","last_fullsized":"2024-05-09T05:38:12.053305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133431+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:38:12.133431+0000","last_peered":"2024-05-09T05:38:12.133431+0000","last_clean":"2024-05-09T05:38:12.133431+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:38:12.133431+0000","last_undegraded":"2024-05-09T05:38:12.133431+0000","last_fullsized":"2024-05-09T05:38:12.133431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224991+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:38:12.224991+0000","last_peered":"2024-05-09T05:38:12.224991+0000","last_clean":"2024-05-09T05:38:12.224991+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:38:12.224991+0000","last_undegraded":"2024-05-09T05:38:12.224991+0000","last_fullsized":"2024-05-09T05:38:12.224991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925314+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:38:11.925314+0000","last_peered":"2024-05-09T05:38:11.925314+0000","last_clean":"2024-05-09T05:38:11.925314+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:38:11.925314+0000","last_undegraded":"2024-05-09T05:38:11.925314+0000","last_fullsized":"2024-05-09T05:38:11.925314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107766,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801521,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2498,"num_read_kb":1474,"num_write":520,"num_write_kb":2394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2404563,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710790,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507638,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907020,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1492548,"data_compressed":80636,"data_compressed_allocated":106496,"data_compressed_original":978944,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070546,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":903884,"data_compressed":48801,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266775,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":900054,"data_compressed":48801,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463004,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":1105849,"data_compressed":65434,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626529,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":502960,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":573964,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":504416,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":575209,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182512,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1223232,"data_compressed":64003,"data_compressed_allocated":86016,"data_compressed_original":782336,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986282,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507034,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822759,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1427895,"data_compressed":80636,"data_compressed_allocated":106496,"data_compressed_original":978944,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332077,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400826,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:38:35.200 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T05:38:35.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T05:38:35.449 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:35.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:35 smithi115 ceph-mon[30349]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-09T05:38:35.572 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:35 smithi115 ceph-mon[30349]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:35.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:35 smithi195 ceph-mon[33844]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-09T05:38:35.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:35 smithi195 ceph-mon[33844]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:35.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:35 smithi161 ceph-mon[33805]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-09T05:38:35.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:35 smithi161 ceph-mon[33805]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.324+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/663991905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e810e910 msgr2=0x7fd6e8110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.324+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/663991905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e810e910 0x7fd6e8110de0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000b0a0 tx=0x7fd6e000b3b0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.324+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/663991905 shutdown_connections 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.324+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/663991905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6e81054e0 0x7fd6e81058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.324+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/663991905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e810e910 0x7fd6e8110de0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.324+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/663991905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e8105e30 0x7fd6e810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.324+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/663991905 >> 172.21.15.115:0/663991905 conn(0x7fd6e80fedf0 msgr2=0x7fd6e8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.324+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/663991905 shutdown_connections 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.324+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/663991905 wait complete. 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6eeaa0700 1 Processor -- start 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6eeaa0700 1 -- start start 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6eeaa0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e81054e0 0x7fd6e81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6eeaa0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e8105e30 0x7fd6e81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6eeaa0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6e810e910 0x7fd6e819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6eeaa0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8114310 con 0x7fd6e81054e0 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6eeaa0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8114190 con 0x7fd6e810e910 2024-05-09T05:38:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6eeaa0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8114490 con 0x7fd6e8105e30 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6ec83c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e81054e0 0x7fd6e81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6ec83c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e81054e0 0x7fd6e81a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52444/0 (socket says 172.21.15.115:52444) 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6ec83c700 1 -- 172.21.15.115:0/1576133574 learned_addr learned my addr 172.21.15.115:0/1576133574 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6e7fff700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e8105e30 0x7fd6e81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6ed03d700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6e810e910 0x7fd6e819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6ec83c700 1 -- 172.21.15.115:0/1576133574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e8105e30 msgr2=0x7fd6e81a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6ec83c700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e8105e30 0x7fd6e81a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6ec83c700 1 -- 172.21.15.115:0/1576133574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6e810e910 msgr2=0x7fd6e819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6ec83c700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6e810e910 0x7fd6e819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.325+0000 7fd6ec83c700 1 -- 172.21.15.115:0/1576133574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e819b160 con 0x7fd6e81054e0 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.326+0000 7fd6ec83c700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e81054e0 0x7fd6e81a0890 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00e970 tx=0x7fd6dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.326+0000 7fd6e5ffb700 1 -- 172.21.15.115:0/1576133574 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00cbd0 con 0x7fd6e81054e0 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.326+0000 7fd6e5ffb700 1 -- 172.21.15.115:0/1576133574 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6dc0044b0 con 0x7fd6e81054e0 2024-05-09T05:38:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.326+0000 7fd6e5ffb700 1 -- 172.21.15.115:0/1576133574 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc003c90 con 0x7fd6e81054e0 2024-05-09T05:38:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.326+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e819b450 con 0x7fd6e81054e0 2024-05-09T05:38:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.326+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e81a7720 con 0x7fd6e81054e0 2024-05-09T05:38:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.327+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e80512c0 con 0x7fd6e81054e0 2024-05-09T05:38:37.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.328+0000 7fd6e5ffb700 1 -- 172.21.15.115:0/1576133574 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fd6dc003e50 con 0x7fd6e81054e0 2024-05-09T05:38:37.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.329+0000 7fd6e5ffb700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6d80b2a80 0x7fd6d80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:37.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.329+0000 7fd6e7fff700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6d80b2a80 0x7fd6d80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:37.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.329+0000 7fd6e5ffb700 1 -- 172.21.15.115:0/1576133574 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd6dc0e2f50 con 0x7fd6e81054e0 2024-05-09T05:38:37.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.330+0000 7fd6e7fff700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6d80b2a80 0x7fd6d80b4f90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8102380 tx=0x7fd6d400b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.333+0000 7fd6e5ffb700 1 -- 172.21.15.115:0/1576133574 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd6dc0aa510 con 0x7fd6e81054e0 2024-05-09T05:38:37.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.535+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6e8063c00 con 0x7fd6d80b2a80 2024-05-09T05:38:37.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:37 smithi115 ceph-mon[30349]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-09T05:38:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.544+0000 7fd6e5ffb700 1 -- 172.21.15.115:0/1576133574 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339739 (secure 0 0 0) 0x7fd6e8063c00 con 0x7fd6d80b2a80 2024-05-09T05:38:37.542 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6d80b2a80 msgr2=0x7fd6d80b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6d80b2a80 0x7fd6d80b4f90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8102380 tx=0x7fd6d400b410 comp rx=0 tx=0).stop 2024-05-09T05:38:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e81054e0 msgr2=0x7fd6e81a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e81054e0 0x7fd6e81a0890 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00e970 tx=0x7fd6dc00ec80 comp rx=0 tx=0).stop 2024-05-09T05:38:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 shutdown_connections 2024-05-09T05:38:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6d80b2a80 0x7fd6d80b4f90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6e810e910 0x7fd6e819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6e81054e0 0x7fd6e81a0890 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 --2- 172.21.15.115:0/1576133574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6e8105e30 0x7fd6e81a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 >> 172.21.15.115:0/1576133574 conn(0x7fd6e80fedf0 msgr2=0x7fd6e8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 shutdown_connections 2024-05-09T05:38:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:37.548+0000 7fd6eeaa0700 1 -- 172.21.15.115:0/1576133574 wait complete. 2024-05-09T05:38:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-05-09T05:38:37.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:37 smithi195 ceph-mon[33844]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-09T05:38:37.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:37 smithi161 ceph-mon[33805]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-09T05:38:38.299 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-09T05:38:35.948044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392123,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2642,"num_read_kb":1722,"num_write":657,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":995,"ondisk_log_size":995,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344752,"kb_used_data":12188,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597072,"statfs":{"total":1151940427776,"available":1151587401728,"internally_reserved":0,"allocated":12480512,"data_stored":10224683,"data_compressed":565383,"data_compressed_allocated":761856,"data_compressed_original":4853760,"omap_allocated":19141,"internal_metadata":340374843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":282,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003697"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925762+0000","last_change":"2024-05-09T05:33:52.785777+0000","last_active":"2024-05-09T05:38:11.925762+0000","last_peered":"2024-05-09T05:38:11.925762+0000","last_clean":"2024-05-09T05:38:11.925762+0000","last_became_active":"2024-05-09T05:33:52.785439+0000","last_became_peered":"2024-05-09T05:33:52.785439+0000","last_unstale":"2024-05-09T05:38:11.925762+0000","last_undegraded":"2024-05-09T05:38:11.925762+0000","last_fullsized":"2024-05-09T05:38:11.925762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:30.403564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979667+0000","last_change":"2024-05-09T05:33:52.779439+0000","last_active":"2024-05-09T05:38:11.979667+0000","last_peered":"2024-05-09T05:38:11.979667+0000","last_clean":"2024-05-09T05:38:11.979667+0000","last_became_active":"2024-05-09T05:33:52.778753+0000","last_became_peered":"2024-05-09T05:33:52.778753+0000","last_unstale":"2024-05-09T05:38:11.979667+0000","last_undegraded":"2024-05-09T05:38:11.979667+0000","last_fullsized":"2024-05-09T05:38:11.979667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:24.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979267+0000","last_change":"2024-05-09T05:33:52.785653+0000","last_active":"2024-05-09T05:38:11.979267+0000","last_peered":"2024-05-09T05:38:11.979267+0000","last_clean":"2024-05-09T05:38:11.979267+0000","last_became_active":"2024-05-09T05:33:52.785010+0000","last_became_peered":"2024-05-09T05:33:52.785010+0000","last_unstale":"2024-05-09T05:38:11.979267+0000","last_undegraded":"2024-05-09T05:38:11.979267+0000","last_fullsized":"2024-05-09T05:38:11.979267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:35.241430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133401+0000","last_change":"2024-05-09T05:33:52.788196+0000","last_active":"2024-05-09T05:38:12.133401+0000","last_peered":"2024-05-09T05:38:12.133401+0000","last_clean":"2024-05-09T05:38:12.133401+0000","last_became_active":"2024-05-09T05:33:52.787610+0000","last_became_peered":"2024-05-09T05:33:52.787610+0000","last_unstale":"2024-05-09T05:38:12.133401+0000","last_undegraded":"2024-05-09T05:38:12.133401+0000","last_fullsized":"2024-05-09T05:38:12.133401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051332+0000","last_change":"2024-05-09T05:33:52.779367+0000","last_active":"2024-05-09T05:38:12.051332+0000","last_peered":"2024-05-09T05:38:12.051332+0000","last_clean":"2024-05-09T05:38:12.051332+0000","last_became_active":"2024-05-09T05:33:52.778982+0000","last_became_peered":"2024-05-09T05:33:52.778982+0000","last_unstale":"2024-05-09T05:38:12.051332+0000","last_undegraded":"2024-05-09T05:38:12.051332+0000","last_fullsized":"2024-05-09T05:38:12.051332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:15.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925938+0000","last_change":"2024-05-09T05:33:52.785512+0000","last_active":"2024-05-09T05:38:11.925938+0000","last_peered":"2024-05-09T05:38:11.925938+0000","last_clean":"2024-05-09T05:38:11.925938+0000","last_became_active":"2024-05-09T05:33:52.785132+0000","last_became_peered":"2024-05-09T05:33:52.785132+0000","last_unstale":"2024-05-09T05:38:11.925938+0000","last_undegraded":"2024-05-09T05:38:11.925938+0000","last_fullsized":"2024-05-09T05:38:11.925938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:49.780739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132223+0000","last_change":"2024-05-09T05:33:52.781241+0000","last_active":"2024-05-09T05:38:12.132223+0000","last_peered":"2024-05-09T05:38:12.132223+0000","last_clean":"2024-05-09T05:38:12.132223+0000","last_became_active":"2024-05-09T05:33:52.780863+0000","last_became_peered":"2024-05-09T05:33:52.780863+0000","last_unstale":"2024-05-09T05:38:12.132223+0000","last_undegraded":"2024-05-09T05:38:12.132223+0000","last_fullsized":"2024-05-09T05:38:12.132223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:40:51.695317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051904+0000","last_change":"2024-05-09T05:33:52.780251+0000","last_active":"2024-05-09T05:38:12.051904+0000","last_peered":"2024-05-09T05:38:12.051904+0000","last_clean":"2024-05-09T05:38:12.051904+0000","last_became_active":"2024-05-09T05:33:52.778955+0000","last_became_peered":"2024-05-09T05:33:52.778955+0000","last_unstale":"2024-05-09T05:38:12.051904+0000","last_undegraded":"2024-05-09T05:38:12.051904+0000","last_fullsized":"2024-05-09T05:38:12.051904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:10:16.245495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134129+0000","last_change":"2024-05-09T05:33:52.785778+0000","last_active":"2024-05-09T05:38:12.134129+0000","last_peered":"2024-05-09T05:38:12.134129+0000","last_clean":"2024-05-09T05:38:12.134129+0000","last_became_active":"2024-05-09T05:33:52.785029+0000","last_became_peered":"2024-05-09T05:33:52.785029+0000","last_unstale":"2024-05-09T05:38:12.134129+0000","last_undegraded":"2024-05-09T05:38:12.134129+0000","last_fullsized":"2024-05-09T05:38:12.134129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:15.733618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.926126+0000","last_change":"2024-05-09T05:34:33.941852+0000","last_active":"2024-05-09T05:38:11.926126+0000","last_peered":"2024-05-09T05:38:11.926126+0000","last_clean":"2024-05-09T05:38:11.926126+0000","last_became_active":"2024-05-09T05:34:33.941448+0000","last_became_peered":"2024-05-09T05:34:33.941448+0000","last_unstale":"2024-05-09T05:38:11.926126+0000","last_undegraded":"2024-05-09T05:38:11.926126+0000","last_fullsized":"2024-05-09T05:38:11.926126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:25.808225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133832+0000","last_change":"2024-05-09T05:33:52.785670+0000","last_active":"2024-05-09T05:38:12.133832+0000","last_peered":"2024-05-09T05:38:12.133832+0000","last_clean":"2024-05-09T05:38:12.133832+0000","last_became_active":"2024-05-09T05:33:52.784651+0000","last_became_peered":"2024-05-09T05:33:52.784651+0000","last_unstale":"2024-05-09T05:38:12.133832+0000","last_undegraded":"2024-05-09T05:38:12.133832+0000","last_fullsized":"2024-05-09T05:38:12.133832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:20:19.087529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133357+0000","last_change":"2024-05-09T05:33:52.779403+0000","last_active":"2024-05-09T05:38:12.133357+0000","last_peered":"2024-05-09T05:38:12.133357+0000","last_clean":"2024-05-09T05:38:12.133357+0000","last_became_active":"2024-05-09T05:33:52.779054+0000","last_became_peered":"2024-05-09T05:33:52.779054+0000","last_unstale":"2024-05-09T05:38:12.133357+0000","last_undegraded":"2024-05-09T05:38:12.133357+0000","last_fullsized":"2024-05-09T05:38:12.133357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:13.014947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225339+0000","last_change":"2024-05-09T05:33:52.781902+0000","last_active":"2024-05-09T05:38:12.225339+0000","last_peered":"2024-05-09T05:38:12.225339+0000","last_clean":"2024-05-09T05:38:12.225339+0000","last_became_active":"2024-05-09T05:33:52.781175+0000","last_became_peered":"2024-05-09T05:33:52.781175+0000","last_unstale":"2024-05-09T05:38:12.225339+0000","last_undegraded":"2024-05-09T05:38:12.225339+0000","last_fullsized":"2024-05-09T05:38:12.225339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:09.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979195+0000","last_change":"2024-05-09T05:33:52.778772+0000","last_active":"2024-05-09T05:38:11.979195+0000","last_peered":"2024-05-09T05:38:11.979195+0000","last_clean":"2024-05-09T05:38:11.979195+0000","last_became_active":"2024-05-09T05:33:52.778394+0000","last_became_peered":"2024-05-09T05:33:52.778394+0000","last_unstale":"2024-05-09T05:38:11.979195+0000","last_undegraded":"2024-05-09T05:38:11.979195+0000","last_fullsized":"2024-05-09T05:38:11.979195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:49.523850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979140+0000","last_change":"2024-05-09T05:33:52.784748+0000","last_active":"2024-05-09T05:38:11.979140+0000","last_peered":"2024-05-09T05:38:11.979140+0000","last_clean":"2024-05-09T05:38:11.979140+0000","last_became_active":"2024-05-09T05:33:52.784396+0000","last_became_peered":"2024-05-09T05:33:52.784396+0000","last_unstale":"2024-05-09T05:38:11.979140+0000","last_undegraded":"2024-05-09T05:38:11.979140+0000","last_fullsized":"2024-05-09T05:38:11.979140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:49.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132115+0000","last_change":"2024-05-09T05:33:52.792498+0000","last_active":"2024-05-09T05:38:12.132115+0000","last_peered":"2024-05-09T05:38:12.132115+0000","last_clean":"2024-05-09T05:38:12.132115+0000","last_became_active":"2024-05-09T05:33:52.791859+0000","last_became_peered":"2024-05-09T05:33:52.791859+0000","last_unstale":"2024-05-09T05:38:12.132115+0000","last_undegraded":"2024-05-09T05:38:12.132115+0000","last_fullsized":"2024-05-09T05:38:12.132115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:10.775329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925542+0000","last_change":"2024-05-09T05:33:52.784885+0000","last_active":"2024-05-09T05:38:11.925542+0000","last_peered":"2024-05-09T05:38:11.925542+0000","last_clean":"2024-05-09T05:38:11.925542+0000","last_became_active":"2024-05-09T05:33:52.784545+0000","last_became_peered":"2024-05-09T05:33:52.784545+0000","last_unstale":"2024-05-09T05:38:11.925542+0000","last_undegraded":"2024-05-09T05:38:11.925542+0000","last_fullsized":"2024-05-09T05:38:11.925542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:23:09.502044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924841+0000","last_change":"2024-05-09T05:33:52.781111+0000","last_active":"2024-05-09T05:38:11.924841+0000","last_peered":"2024-05-09T05:38:11.924841+0000","last_clean":"2024-05-09T05:38:11.924841+0000","last_became_active":"2024-05-09T05:33:52.780769+0000","last_became_peered":"2024-05-09T05:33:52.780769+0000","last_unstale":"2024-05-09T05:38:11.924841+0000","last_undegraded":"2024-05-09T05:38:11.924841+0000","last_fullsized":"2024-05-09T05:38:11.924841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:23.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132289+0000","last_change":"2024-05-09T05:33:52.775233+0000","last_active":"2024-05-09T05:38:12.132289+0000","last_peered":"2024-05-09T05:38:12.132289+0000","last_clean":"2024-05-09T05:38:12.132289+0000","last_became_active":"2024-05-09T05:33:52.774877+0000","last_became_peered":"2024-05-09T05:33:52.774877+0000","last_unstale":"2024-05-09T05:38:12.132289+0000","last_undegraded":"2024-05-09T05:38:12.132289+0000","last_fullsized":"2024-05-09T05:38:12.132289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:43.927970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131837+0000","last_change":"2024-05-09T05:34:04.831884+0000","last_active":"2024-05-09T05:38:12.131837+0000","last_peered":"2024-05-09T05:38:12.131837+0000","last_clean":"2024-05-09T05:38:12.131837+0000","last_became_active":"2024-05-09T05:34:04.831429+0000","last_became_peered":"2024-05-09T05:34:04.831429+0000","last_unstale":"2024-05-09T05:38:12.131837+0000","last_undegraded":"2024-05-09T05:38:12.131837+0000","last_fullsized":"2024-05-09T05:38:12.131837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:45.331641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":270,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.592616+0000","last_change":"2024-05-09T05:33:49.740587+0000","last_active":"2024-05-09T05:38:31.592616+0000","last_peered":"2024-05-09T05:38:31.592616+0000","last_clean":"2024-05-09T05:38:31.592616+0000","last_became_active":"2024-05-09T05:33:49.740241+0000","last_became_peered":"2024-05-09T05:33:49.740241+0000","last_unstale":"2024-05-09T05:38:31.592616+0000","last_undegraded":"2024-05-09T05:38:31.592616+0000","last_fullsized":"2024-05-09T05:38:31.592616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:02.406798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053507+0000","last_change":"2024-05-09T05:33:52.778758+0000","last_active":"2024-05-09T05:38:12.053507+0000","last_peered":"2024-05-09T05:38:12.053507+0000","last_clean":"2024-05-09T05:38:12.053507+0000","last_became_active":"2024-05-09T05:33:52.777767+0000","last_became_peered":"2024-05-09T05:33:52.777767+0000","last_unstale":"2024-05-09T05:38:12.053507+0000","last_undegraded":"2024-05-09T05:38:12.053507+0000","last_fullsized":"2024-05-09T05:38:12.053507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:26.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132572+0000","last_change":"2024-05-09T05:34:04.828444+0000","last_active":"2024-05-09T05:38:12.132572+0000","last_peered":"2024-05-09T05:38:12.132572+0000","last_clean":"2024-05-09T05:38:12.132572+0000","last_became_active":"2024-05-09T05:34:04.827998+0000","last_became_peered":"2024-05-09T05:34:04.827998+0000","last_unstale":"2024-05-09T05:38:12.132572+0000","last_undegraded":"2024-05-09T05:38:12.132572+0000","last_fullsized":"2024-05-09T05:38:12.132572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:52.478947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053764+0000","last_change":"2024-05-09T05:33:49.832692+0000","last_active":"2024-05-09T05:38:12.053764+0000","last_peered":"2024-05-09T05:38:12.053764+0000","last_clean":"2024-05-09T05:38:12.053764+0000","last_became_active":"2024-05-09T05:33:49.832356+0000","last_became_peered":"2024-05-09T05:33:49.832356+0000","last_unstale":"2024-05-09T05:38:12.053764+0000","last_undegraded":"2024-05-09T05:38:12.053764+0000","last_fullsized":"2024-05-09T05:38:12.053764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:21.930683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979670+0000","last_change":"2024-05-09T05:33:52.786190+0000","last_active":"2024-05-09T05:38:11.979670+0000","last_peered":"2024-05-09T05:38:11.979670+0000","last_clean":"2024-05-09T05:38:11.979670+0000","last_became_active":"2024-05-09T05:33:52.785837+0000","last_became_peered":"2024-05-09T05:33:52.785837+0000","last_unstale":"2024-05-09T05:38:11.979670+0000","last_undegraded":"2024-05-09T05:38:11.979670+0000","last_fullsized":"2024-05-09T05:38:11.979670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:19.342992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132963+0000","last_change":"2024-05-09T05:34:04.830937+0000","last_active":"2024-05-09T05:38:12.132963+0000","last_peered":"2024-05-09T05:38:12.132963+0000","last_clean":"2024-05-09T05:38:12.132963+0000","last_became_active":"2024-05-09T05:34:04.830366+0000","last_became_peered":"2024-05-09T05:34:04.830366+0000","last_unstale":"2024-05-09T05:38:12.132963+0000","last_undegraded":"2024-05-09T05:38:12.132963+0000","last_fullsized":"2024-05-09T05:38:12.132963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:53.419511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051620+0000","last_change":"2024-05-09T05:33:49.746797+0000","last_active":"2024-05-09T05:38:12.051620+0000","last_peered":"2024-05-09T05:38:12.051620+0000","last_clean":"2024-05-09T05:38:12.051620+0000","last_became_active":"2024-05-09T05:33:49.746028+0000","last_became_peered":"2024-05-09T05:33:49.746028+0000","last_unstale":"2024-05-09T05:38:12.051620+0000","last_undegraded":"2024-05-09T05:38:12.051620+0000","last_fullsized":"2024-05-09T05:38:12.051620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:01.734852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133077+0000","last_change":"2024-05-09T05:33:52.791779+0000","last_active":"2024-05-09T05:38:12.133077+0000","last_peered":"2024-05-09T05:38:12.133077+0000","last_clean":"2024-05-09T05:38:12.133077+0000","last_became_active":"2024-05-09T05:33:52.788879+0000","last_became_peered":"2024-05-09T05:33:52.788879+0000","last_unstale":"2024-05-09T05:38:12.133077+0000","last_undegraded":"2024-05-09T05:38:12.133077+0000","last_fullsized":"2024-05-09T05:38:12.133077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:33.516745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132705+0000","last_change":"2024-05-09T05:34:04.821854+0000","last_active":"2024-05-09T05:38:12.132705+0000","last_peered":"2024-05-09T05:38:12.132705+0000","last_clean":"2024-05-09T05:38:12.132705+0000","last_became_active":"2024-05-09T05:34:04.821319+0000","last_became_peered":"2024-05-09T05:34:04.821319+0000","last_unstale":"2024-05-09T05:38:12.132705+0000","last_undegraded":"2024-05-09T05:38:12.132705+0000","last_fullsized":"2024-05-09T05:38:12.132705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:15:32.147126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051663+0000","last_change":"2024-05-09T05:33:49.744111+0000","last_active":"2024-05-09T05:38:12.051663+0000","last_peered":"2024-05-09T05:38:12.051663+0000","last_clean":"2024-05-09T05:38:12.051663+0000","last_became_active":"2024-05-09T05:33:49.743697+0000","last_became_peered":"2024-05-09T05:33:49.743697+0000","last_unstale":"2024-05-09T05:38:12.051663+0000","last_undegraded":"2024-05-09T05:38:12.051663+0000","last_fullsized":"2024-05-09T05:38:12.051663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:29:57.222454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059550+0000","last_change":"2024-05-09T05:34:33.941216+0000","last_active":"2024-05-09T05:38:12.059550+0000","last_peered":"2024-05-09T05:38:12.059550+0000","last_clean":"2024-05-09T05:38:12.059550+0000","last_became_active":"2024-05-09T05:34:33.940710+0000","last_became_peered":"2024-05-09T05:34:33.940710+0000","last_unstale":"2024-05-09T05:38:12.059550+0000","last_undegraded":"2024-05-09T05:38:12.059550+0000","last_fullsized":"2024-05-09T05:38:12.059550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:56.018609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131984+0000","last_change":"2024-05-09T05:34:04.825350+0000","last_active":"2024-05-09T05:38:12.131984+0000","last_peered":"2024-05-09T05:38:12.131984+0000","last_clean":"2024-05-09T05:38:12.131984+0000","last_became_active":"2024-05-09T05:34:04.824863+0000","last_became_peered":"2024-05-09T05:34:04.824863+0000","last_unstale":"2024-05-09T05:38:12.131984+0000","last_undegraded":"2024-05-09T05:38:12.131984+0000","last_fullsized":"2024-05-09T05:38:12.131984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:37.131870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":265,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.619785+0000","last_change":"2024-05-09T05:33:49.746157+0000","last_active":"2024-05-09T05:38:31.619785+0000","last_peered":"2024-05-09T05:38:31.619785+0000","last_clean":"2024-05-09T05:38:31.619785+0000","last_became_active":"2024-05-09T05:33:49.745768+0000","last_became_peered":"2024-05-09T05:33:49.745768+0000","last_unstale":"2024-05-09T05:38:31.619785+0000","last_undegraded":"2024-05-09T05:38:31.619785+0000","last_fullsized":"2024-05-09T05:38:31.619785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:03.519755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051377+0000","last_change":"2024-05-09T05:33:52.777590+0000","last_active":"2024-05-09T05:38:12.051377+0000","last_peered":"2024-05-09T05:38:12.051377+0000","last_clean":"2024-05-09T05:38:12.051377+0000","last_became_active":"2024-05-09T05:33:52.777180+0000","last_became_peered":"2024-05-09T05:33:52.777180+0000","last_unstale":"2024-05-09T05:38:12.051377+0000","last_undegraded":"2024-05-09T05:38:12.051377+0000","last_fullsized":"2024-05-09T05:38:12.051377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:17.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132066+0000","last_change":"2024-05-09T05:34:04.830766+0000","last_active":"2024-05-09T05:38:12.132066+0000","last_peered":"2024-05-09T05:38:12.132066+0000","last_clean":"2024-05-09T05:38:12.132066+0000","last_became_active":"2024-05-09T05:34:04.830289+0000","last_became_peered":"2024-05-09T05:34:04.830289+0000","last_unstale":"2024-05-09T05:38:12.132066+0000","last_undegraded":"2024-05-09T05:38:12.132066+0000","last_fullsized":"2024-05-09T05:38:12.132066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:37.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925853+0000","last_change":"2024-05-09T05:33:49.748515+0000","last_active":"2024-05-09T05:38:11.925853+0000","last_peered":"2024-05-09T05:38:11.925853+0000","last_clean":"2024-05-09T05:38:11.925853+0000","last_became_active":"2024-05-09T05:33:49.748095+0000","last_became_peered":"2024-05-09T05:33:49.748095+0000","last_unstale":"2024-05-09T05:38:11.925853+0000","last_undegraded":"2024-05-09T05:38:11.925853+0000","last_fullsized":"2024-05-09T05:38:11.925853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:41:41.555842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059515+0000","last_change":"2024-05-09T05:33:52.783333+0000","last_active":"2024-05-09T05:38:12.059515+0000","last_peered":"2024-05-09T05:38:12.059515+0000","last_clean":"2024-05-09T05:38:12.059515+0000","last_became_active":"2024-05-09T05:33:52.781042+0000","last_became_peered":"2024-05-09T05:33:52.781042+0000","last_unstale":"2024-05-09T05:38:12.059515+0000","last_undegraded":"2024-05-09T05:38:12.059515+0000","last_fullsized":"2024-05-09T05:38:12.059515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:59.051744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925545+0000","last_change":"2024-05-09T05:34:34.051017+0000","last_active":"2024-05-09T05:38:11.925545+0000","last_peered":"2024-05-09T05:38:11.925545+0000","last_clean":"2024-05-09T05:38:11.925545+0000","last_became_active":"2024-05-09T05:34:34.050309+0000","last_became_peered":"2024-05-09T05:34:34.050309+0000","last_unstale":"2024-05-09T05:38:11.925545+0000","last_undegraded":"2024-05-09T05:38:11.925545+0000","last_fullsized":"2024-05-09T05:38:11.925545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:17.895227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224943+0000","last_change":"2024-05-09T05:33:49.741610+0000","last_active":"2024-05-09T05:38:12.224943+0000","last_peered":"2024-05-09T05:38:12.224943+0000","last_clean":"2024-05-09T05:38:12.224943+0000","last_became_active":"2024-05-09T05:33:49.741089+0000","last_became_peered":"2024-05-09T05:33:49.741089+0000","last_unstale":"2024-05-09T05:38:12.224943+0000","last_undegraded":"2024-05-09T05:38:12.224943+0000","last_fullsized":"2024-05-09T05:38:12.224943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:39.889377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052036+0000","last_change":"2024-05-09T05:33:52.772124+0000","last_active":"2024-05-09T05:38:12.052036+0000","last_peered":"2024-05-09T05:38:12.052036+0000","last_clean":"2024-05-09T05:38:12.052036+0000","last_became_active":"2024-05-09T05:33:52.771280+0000","last_became_peered":"2024-05-09T05:33:52.771280+0000","last_unstale":"2024-05-09T05:38:12.052036+0000","last_undegraded":"2024-05-09T05:38:12.052036+0000","last_fullsized":"2024-05-09T05:38:12.052036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:54.249111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979576+0000","last_change":"2024-05-09T05:34:04.826199+0000","last_active":"2024-05-09T05:38:11.979576+0000","last_peered":"2024-05-09T05:38:11.979576+0000","last_clean":"2024-05-09T05:38:11.979576+0000","last_became_active":"2024-05-09T05:34:04.825820+0000","last_became_peered":"2024-05-09T05:34:04.825820+0000","last_unstale":"2024-05-09T05:38:11.979576+0000","last_undegraded":"2024-05-09T05:38:11.979576+0000","last_fullsized":"2024-05-09T05:38:11.979576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:15.728526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059004+0000","last_change":"2024-05-09T05:33:49.742544+0000","last_active":"2024-05-09T05:38:12.059004+0000","last_peered":"2024-05-09T05:38:12.059004+0000","last_clean":"2024-05-09T05:38:12.059004+0000","last_became_active":"2024-05-09T05:33:49.742148+0000","last_became_peered":"2024-05-09T05:33:49.742148+0000","last_unstale":"2024-05-09T05:38:12.059004+0000","last_undegraded":"2024-05-09T05:38:12.059004+0000","last_fullsized":"2024-05-09T05:38:12.059004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:42.917462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924546+0000","last_change":"2024-05-09T05:33:52.776397+0000","last_active":"2024-05-09T05:38:11.924546+0000","last_peered":"2024-05-09T05:38:11.924546+0000","last_clean":"2024-05-09T05:38:11.924546+0000","last_became_active":"2024-05-09T05:33:52.776067+0000","last_became_peered":"2024-05-09T05:33:52.776067+0000","last_unstale":"2024-05-09T05:38:11.924546+0000","last_undegraded":"2024-05-09T05:38:11.924546+0000","last_fullsized":"2024-05-09T05:38:11.924546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:02.894803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053026+0000","last_change":"2024-05-09T05:34:04.828885+0000","last_active":"2024-05-09T05:38:12.053026+0000","last_peered":"2024-05-09T05:38:12.053026+0000","last_clean":"2024-05-09T05:38:12.053026+0000","last_became_active":"2024-05-09T05:34:04.826338+0000","last_became_peered":"2024-05-09T05:34:04.826338+0000","last_unstale":"2024-05-09T05:38:12.053026+0000","last_undegraded":"2024-05-09T05:38:12.053026+0000","last_fullsized":"2024-05-09T05:38:12.053026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:37.595770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'7","reported_seq":469,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:32.620577+0000","last_change":"2024-05-09T05:33:49.743561+0000","last_active":"2024-05-09T05:38:32.620577+0000","last_peered":"2024-05-09T05:38:32.620577+0000","last_clean":"2024-05-09T05:38:32.620577+0000","last_became_active":"2024-05-09T05:33:49.743184+0000","last_became_peered":"2024-05-09T05:33:49.743184+0000","last_unstale":"2024-05-09T05:38:32.620577+0000","last_undegraded":"2024-05-09T05:38:32.620577+0000","last_fullsized":"2024-05-09T05:38:32.620577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:23.990465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":420,"num_read_kb":420,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133858+0000","last_change":"2024-05-09T05:33:52.791584+0000","last_active":"2024-05-09T05:38:12.133858+0000","last_peered":"2024-05-09T05:38:12.133858+0000","last_clean":"2024-05-09T05:38:12.133858+0000","last_became_active":"2024-05-09T05:33:52.788736+0000","last_became_peered":"2024-05-09T05:33:52.788736+0000","last_unstale":"2024-05-09T05:38:12.133858+0000","last_undegraded":"2024-05-09T05:38:12.133858+0000","last_fullsized":"2024-05-09T05:38:12.133858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:06.252392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924463+0000","last_change":"2024-05-09T05:34:04.835724+0000","last_active":"2024-05-09T05:38:11.924463+0000","last_peered":"2024-05-09T05:38:11.924463+0000","last_clean":"2024-05-09T05:38:11.924463+0000","last_became_active":"2024-05-09T05:34:04.835333+0000","last_became_peered":"2024-05-09T05:34:04.835333+0000","last_unstale":"2024-05-09T05:38:11.924463+0000","last_undegraded":"2024-05-09T05:38:11.924463+0000","last_fullsized":"2024-05-09T05:38:11.924463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:42:04.120432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133086+0000","last_change":"2024-05-09T05:33:49.749173+0000","last_active":"2024-05-09T05:38:12.133086+0000","last_peered":"2024-05-09T05:38:12.133086+0000","last_clean":"2024-05-09T05:38:12.133086+0000","last_became_active":"2024-05-09T05:33:49.748504+0000","last_became_peered":"2024-05-09T05:33:49.748504+0000","last_unstale":"2024-05-09T05:38:12.133086+0000","last_undegraded":"2024-05-09T05:38:12.133086+0000","last_fullsized":"2024-05-09T05:38:12.133086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:24.095706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979599+0000","last_change":"2024-05-09T05:33:52.791898+0000","last_active":"2024-05-09T05:38:11.979599+0000","last_peered":"2024-05-09T05:38:11.979599+0000","last_clean":"2024-05-09T05:38:11.979599+0000","last_became_active":"2024-05-09T05:33:52.791615+0000","last_became_peered":"2024-05-09T05:33:52.791615+0000","last_unstale":"2024-05-09T05:38:11.979599+0000","last_undegraded":"2024-05-09T05:38:11.979599+0000","last_fullsized":"2024-05-09T05:38:11.979599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:52:23.139046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133847+0000","last_change":"2024-05-09T05:34:04.832027+0000","last_active":"2024-05-09T05:38:12.133847+0000","last_peered":"2024-05-09T05:38:12.133847+0000","last_clean":"2024-05-09T05:38:12.133847+0000","last_became_active":"2024-05-09T05:34:04.831410+0000","last_became_peered":"2024-05-09T05:34:04.831410+0000","last_unstale":"2024-05-09T05:38:12.133847+0000","last_undegraded":"2024-05-09T05:38:12.133847+0000","last_fullsized":"2024-05-09T05:38:12.133847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:24.802342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059520+0000","last_change":"2024-05-09T05:33:49.832263+0000","last_active":"2024-05-09T05:38:12.059520+0000","last_peered":"2024-05-09T05:38:12.059520+0000","last_clean":"2024-05-09T05:38:12.059520+0000","last_became_active":"2024-05-09T05:33:49.831729+0000","last_became_peered":"2024-05-09T05:33:49.831729+0000","last_unstale":"2024-05-09T05:38:12.059520+0000","last_undegraded":"2024-05-09T05:38:12.059520+0000","last_fullsized":"2024-05-09T05:38:12.059520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:33.831055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133129+0000","last_change":"2024-05-09T05:33:52.788451+0000","last_active":"2024-05-09T05:38:12.133129+0000","last_peered":"2024-05-09T05:38:12.133129+0000","last_clean":"2024-05-09T05:38:12.133129+0000","last_became_active":"2024-05-09T05:33:52.788062+0000","last_became_peered":"2024-05-09T05:33:52.788062+0000","last_unstale":"2024-05-09T05:38:12.133129+0000","last_undegraded":"2024-05-09T05:38:12.133129+0000","last_fullsized":"2024-05-09T05:38:12.133129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:22:16.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924514+0000","last_change":"2024-05-09T05:34:04.833356+0000","last_active":"2024-05-09T05:38:11.924514+0000","last_peered":"2024-05-09T05:38:11.924514+0000","last_clean":"2024-05-09T05:38:11.924514+0000","last_became_active":"2024-05-09T05:34:04.832905+0000","last_became_peered":"2024-05-09T05:34:04.832905+0000","last_unstale":"2024-05-09T05:38:11.924514+0000","last_undegraded":"2024-05-09T05:38:11.924514+0000","last_fullsized":"2024-05-09T05:38:11.924514+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:30.872536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'43","reported_seq":519,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:32.625423+0000","last_change":"2024-05-09T05:33:49.750664+0000","last_active":"2024-05-09T05:38:32.625423+0000","last_peered":"2024-05-09T05:38:32.625423+0000","last_clean":"2024-05-09T05:38:32.625423+0000","last_became_active":"2024-05-09T05:33:49.750219+0000","last_became_peered":"2024-05-09T05:33:49.750219+0000","last_unstale":"2024-05-09T05:38:32.625423+0000","last_undegraded":"2024-05-09T05:38:32.625423+0000","last_fullsized":"2024-05-09T05:38:32.625423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:04.363265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69974,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":433,"num_read_kb":285,"num_write":45,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051722+0000","last_change":"2024-05-09T05:33:52.782974+0000","last_active":"2024-05-09T05:38:12.051722+0000","last_peered":"2024-05-09T05:38:12.051722+0000","last_clean":"2024-05-09T05:38:12.051722+0000","last_became_active":"2024-05-09T05:33:52.782634+0000","last_became_peered":"2024-05-09T05:33:52.782634+0000","last_unstale":"2024-05-09T05:38:12.051722+0000","last_undegraded":"2024-05-09T05:38:12.051722+0000","last_fullsized":"2024-05-09T05:38:12.051722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:18:01.202844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059857+0000","last_change":"2024-05-09T05:34:04.822779+0000","last_active":"2024-05-09T05:38:12.059857+0000","last_peered":"2024-05-09T05:38:12.059857+0000","last_clean":"2024-05-09T05:38:12.059857+0000","last_became_active":"2024-05-09T05:34:04.822293+0000","last_became_peered":"2024-05-09T05:34:04.822293+0000","last_unstale":"2024-05-09T05:38:12.059857+0000","last_undegraded":"2024-05-09T05:38:12.059857+0000","last_fullsized":"2024-05-09T05:38:12.059857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:56.552254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132213+0000","last_change":"2024-05-09T05:33:49.748457+0000","last_active":"2024-05-09T05:38:12.132213+0000","last_peered":"2024-05-09T05:38:12.132213+0000","last_clean":"2024-05-09T05:38:12.132213+0000","last_became_active":"2024-05-09T05:33:49.748065+0000","last_became_peered":"2024-05-09T05:33:49.748065+0000","last_unstale":"2024-05-09T05:38:12.132213+0000","last_undegraded":"2024-05-09T05:38:12.132213+0000","last_fullsized":"2024-05-09T05:38:12.132213+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:49:39.379387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225225+0000","last_change":"2024-05-09T05:33:52.779237+0000","last_active":"2024-05-09T05:38:12.225225+0000","last_peered":"2024-05-09T05:38:12.225225+0000","last_clean":"2024-05-09T05:38:12.225225+0000","last_became_active":"2024-05-09T05:33:52.778880+0000","last_became_peered":"2024-05-09T05:33:52.778880+0000","last_unstale":"2024-05-09T05:38:12.225225+0000","last_undegraded":"2024-05-09T05:38:12.225225+0000","last_fullsized":"2024-05-09T05:38:12.225225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:35.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:04.828735+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:04.828299+0000","last_became_peered":"2024-05-09T05:34:04.828299+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:16.122715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059364+0000","last_change":"2024-05-09T05:33:49.736390+0000","last_active":"2024-05-09T05:38:12.059364+0000","last_peered":"2024-05-09T05:38:12.059364+0000","last_clean":"2024-05-09T05:38:12.059364+0000","last_became_active":"2024-05-09T05:33:49.735961+0000","last_became_peered":"2024-05-09T05:33:49.735961+0000","last_unstale":"2024-05-09T05:38:12.059364+0000","last_undegraded":"2024-05-09T05:38:12.059364+0000","last_fullsized":"2024-05-09T05:38:12.059364+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:07.452550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132766+0000","last_change":"2024-05-09T05:33:52.786560+0000","last_active":"2024-05-09T05:38:12.132766+0000","last_peered":"2024-05-09T05:38:12.132766+0000","last_clean":"2024-05-09T05:38:12.132766+0000","last_became_active":"2024-05-09T05:33:52.786199+0000","last_became_peered":"2024-05-09T05:33:52.786199+0000","last_unstale":"2024-05-09T05:38:12.132766+0000","last_undegraded":"2024-05-09T05:38:12.132766+0000","last_fullsized":"2024-05-09T05:38:12.132766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:10.460651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051717+0000","last_change":"2024-05-09T05:34:34.051026+0000","last_active":"2024-05-09T05:38:12.051717+0000","last_peered":"2024-05-09T05:38:12.051717+0000","last_clean":"2024-05-09T05:38:12.051717+0000","last_became_active":"2024-05-09T05:34:34.050359+0000","last_became_peered":"2024-05-09T05:34:34.050359+0000","last_unstale":"2024-05-09T05:38:12.051717+0000","last_undegraded":"2024-05-09T05:38:12.051717+0000","last_fullsized":"2024-05-09T05:38:12.051717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:14.673711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":265,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:32.594602+0000","last_change":"2024-05-09T05:33:49.745366+0000","last_active":"2024-05-09T05:38:32.594602+0000","last_peered":"2024-05-09T05:38:32.594602+0000","last_clean":"2024-05-09T05:38:32.594602+0000","last_became_active":"2024-05-09T05:33:49.744883+0000","last_became_peered":"2024-05-09T05:33:49.744883+0000","last_unstale":"2024-05-09T05:38:32.594602+0000","last_undegraded":"2024-05-09T05:38:32.594602+0000","last_fullsized":"2024-05-09T05:38:32.594602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:12.016784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132385+0000","last_change":"2024-05-09T05:33:52.782009+0000","last_active":"2024-05-09T05:38:12.132385+0000","last_peered":"2024-05-09T05:38:12.132385+0000","last_clean":"2024-05-09T05:38:12.132385+0000","last_became_active":"2024-05-09T05:33:52.780641+0000","last_became_peered":"2024-05-09T05:33:52.780641+0000","last_unstale":"2024-05-09T05:38:12.132385+0000","last_undegraded":"2024-05-09T05:38:12.132385+0000","last_fullsized":"2024-05-09T05:38:12.132385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:08.709653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225537+0000","last_change":"2024-05-09T05:34:04.825100+0000","last_active":"2024-05-09T05:38:12.225537+0000","last_peered":"2024-05-09T05:38:12.225537+0000","last_clean":"2024-05-09T05:38:12.225537+0000","last_became_active":"2024-05-09T05:34:04.824468+0000","last_became_peered":"2024-05-09T05:34:04.824468+0000","last_unstale":"2024-05-09T05:38:12.225537+0000","last_undegraded":"2024-05-09T05:38:12.225537+0000","last_fullsized":"2024-05-09T05:38:12.225537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:40:52.657573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051295+0000","last_change":"2024-05-09T05:33:49.831874+0000","last_active":"2024-05-09T05:38:12.051295+0000","last_peered":"2024-05-09T05:38:12.051295+0000","last_clean":"2024-05-09T05:38:12.051295+0000","last_became_active":"2024-05-09T05:33:49.831435+0000","last_became_peered":"2024-05-09T05:33:49.831435+0000","last_unstale":"2024-05-09T05:38:12.051295+0000","last_undegraded":"2024-05-09T05:38:12.051295+0000","last_fullsized":"2024-05-09T05:38:12.051295+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:59.328290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924664+0000","last_change":"2024-05-09T05:34:34.049898+0000","last_active":"2024-05-09T05:38:11.924664+0000","last_peered":"2024-05-09T05:38:11.924664+0000","last_clean":"2024-05-09T05:38:11.924664+0000","last_became_active":"2024-05-09T05:34:34.049491+0000","last_became_peered":"2024-05-09T05:34:34.049491+0000","last_unstale":"2024-05-09T05:38:11.924664+0000","last_undegraded":"2024-05-09T05:38:11.924664+0000","last_fullsized":"2024-05-09T05:38:11.924664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:05:04.303901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059379+0000","last_change":"2024-05-09T05:34:04.822791+0000","last_active":"2024-05-09T05:38:12.059379+0000","last_peered":"2024-05-09T05:38:12.059379+0000","last_clean":"2024-05-09T05:38:12.059379+0000","last_became_active":"2024-05-09T05:34:04.822341+0000","last_became_peered":"2024-05-09T05:34:04.822341+0000","last_unstale":"2024-05-09T05:38:12.059379+0000","last_undegraded":"2024-05-09T05:38:12.059379+0000","last_fullsized":"2024-05-09T05:38:12.059379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:34.089547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051937+0000","last_change":"2024-05-09T05:33:49.747373+0000","last_active":"2024-05-09T05:38:12.051937+0000","last_peered":"2024-05-09T05:38:12.051937+0000","last_clean":"2024-05-09T05:38:12.051937+0000","last_became_active":"2024-05-09T05:33:49.746958+0000","last_became_peered":"2024-05-09T05:33:49.746958+0000","last_unstale":"2024-05-09T05:38:12.051937+0000","last_undegraded":"2024-05-09T05:38:12.051937+0000","last_fullsized":"2024-05-09T05:38:12.051937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:03.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925408+0000","last_change":"2024-05-09T05:33:52.772945+0000","last_active":"2024-05-09T05:38:11.925408+0000","last_peered":"2024-05-09T05:38:11.925408+0000","last_clean":"2024-05-09T05:38:11.925408+0000","last_became_active":"2024-05-09T05:33:52.772560+0000","last_became_peered":"2024-05-09T05:33:52.772560+0000","last_unstale":"2024-05-09T05:38:11.925408+0000","last_undegraded":"2024-05-09T05:38:11.925408+0000","last_fullsized":"2024-05-09T05:38:11.925408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:34.723803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051341+0000","last_change":"2024-05-09T05:34:33.942337+0000","last_active":"2024-05-09T05:38:12.051341+0000","last_peered":"2024-05-09T05:38:12.051341+0000","last_clean":"2024-05-09T05:38:12.051341+0000","last_became_active":"2024-05-09T05:34:33.941797+0000","last_became_peered":"2024-05-09T05:34:33.941797+0000","last_unstale":"2024-05-09T05:38:12.051341+0000","last_undegraded":"2024-05-09T05:38:12.051341+0000","last_fullsized":"2024-05-09T05:38:12.051341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:49:23.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925071+0000","last_change":"2024-05-09T05:34:33.909800+0000","last_active":"2024-05-09T05:38:11.925071+0000","last_peered":"2024-05-09T05:38:11.925071+0000","last_clean":"2024-05-09T05:38:11.925071+0000","last_became_active":"2024-05-09T05:34:33.909429+0000","last_became_peered":"2024-05-09T05:34:33.909429+0000","last_unstale":"2024-05-09T05:38:11.925071+0000","last_undegraded":"2024-05-09T05:38:11.925071+0000","last_fullsized":"2024-05-09T05:38:11.925071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:24.077688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925850+0000","last_change":"2024-05-09T05:33:52.780154+0000","last_active":"2024-05-09T05:38:11.925850+0000","last_peered":"2024-05-09T05:38:11.925850+0000","last_clean":"2024-05-09T05:38:11.925850+0000","last_became_active":"2024-05-09T05:33:52.779757+0000","last_became_peered":"2024-05-09T05:33:52.779757+0000","last_unstale":"2024-05-09T05:38:11.925850+0000","last_undegraded":"2024-05-09T05:38:11.925850+0000","last_fullsized":"2024-05-09T05:38:11.925850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:36.902318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225147+0000","last_change":"2024-05-09T05:34:04.823146+0000","last_active":"2024-05-09T05:38:12.225147+0000","last_peered":"2024-05-09T05:38:12.225147+0000","last_clean":"2024-05-09T05:38:12.225147+0000","last_became_active":"2024-05-09T05:34:04.822668+0000","last_became_peered":"2024-05-09T05:34:04.822668+0000","last_unstale":"2024-05-09T05:38:12.225147+0000","last_undegraded":"2024-05-09T05:38:12.225147+0000","last_fullsized":"2024-05-09T05:38:12.225147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:02.833465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'7","reported_seq":264,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.593165+0000","last_change":"2024-05-09T05:33:49.741127+0000","last_active":"2024-05-09T05:38:31.593165+0000","last_peered":"2024-05-09T05:38:31.593165+0000","last_clean":"2024-05-09T05:38:31.593165+0000","last_became_active":"2024-05-09T05:33:49.740695+0000","last_became_peered":"2024-05-09T05:33:49.740695+0000","last_unstale":"2024-05-09T05:38:31.593165+0000","last_undegraded":"2024-05-09T05:38:31.593165+0000","last_fullsized":"2024-05-09T05:38:31.593165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.245717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":213,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925837+0000","last_change":"2024-05-09T05:34:33.913062+0000","last_active":"2024-05-09T05:38:11.925837+0000","last_peered":"2024-05-09T05:38:11.925837+0000","last_clean":"2024-05-09T05:38:11.925837+0000","last_became_active":"2024-05-09T05:34:33.912544+0000","last_became_peered":"2024-05-09T05:34:33.912544+0000","last_unstale":"2024-05-09T05:38:11.925837+0000","last_undegraded":"2024-05-09T05:38:11.925837+0000","last_fullsized":"2024-05-09T05:38:11.925837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:39:35.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225711+0000","last_change":"2024-05-09T05:34:05.090543+0000","last_active":"2024-05-09T05:38:12.225711+0000","last_peered":"2024-05-09T05:38:12.225711+0000","last_clean":"2024-05-09T05:38:12.225711+0000","last_became_active":"2024-05-09T05:34:05.090021+0000","last_became_peered":"2024-05-09T05:34:05.090021+0000","last_unstale":"2024-05-09T05:38:12.225711+0000","last_undegraded":"2024-05-09T05:38:12.225711+0000","last_fullsized":"2024-05-09T05:38:12.225711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:36.241462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059336+0000","last_change":"2024-05-09T05:33:52.780398+0000","last_active":"2024-05-09T05:38:12.059336+0000","last_peered":"2024-05-09T05:38:12.059336+0000","last_clean":"2024-05-09T05:38:12.059336+0000","last_became_active":"2024-05-09T05:33:52.780072+0000","last_became_peered":"2024-05-09T05:33:52.780072+0000","last_unstale":"2024-05-09T05:38:12.059336+0000","last_undegraded":"2024-05-09T05:38:12.059336+0000","last_fullsized":"2024-05-09T05:38:12.059336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:14:50.038641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":260,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.592617+0000","last_change":"2024-05-09T05:33:49.831672+0000","last_active":"2024-05-09T05:38:31.592617+0000","last_peered":"2024-05-09T05:38:31.592617+0000","last_clean":"2024-05-09T05:38:31.592617+0000","last_became_active":"2024-05-09T05:33:49.831158+0000","last_became_peered":"2024-05-09T05:33:49.831158+0000","last_unstale":"2024-05-09T05:38:31.592617+0000","last_undegraded":"2024-05-09T05:38:31.592617+0000","last_fullsized":"2024-05-09T05:38:31.592617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:08:09.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":211,"num_read_kb":211,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133686+0000","last_change":"2024-05-09T05:34:04.829387+0000","last_active":"2024-05-09T05:38:12.133686+0000","last_peered":"2024-05-09T05:38:12.133686+0000","last_clean":"2024-05-09T05:38:12.133686+0000","last_became_active":"2024-05-09T05:34:04.828961+0000","last_became_peered":"2024-05-09T05:34:04.828961+0000","last_unstale":"2024-05-09T05:38:12.133686+0000","last_undegraded":"2024-05-09T05:38:12.133686+0000","last_fullsized":"2024-05-09T05:38:12.133686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:20:53.740064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.052441+0000","last_change":"2024-05-09T05:33:52.778028+0000","last_active":"2024-05-09T05:38:12.052441+0000","last_peered":"2024-05-09T05:38:12.052441+0000","last_clean":"2024-05-09T05:38:12.052441+0000","last_became_active":"2024-05-09T05:33:52.777196+0000","last_became_peered":"2024-05-09T05:33:52.777196+0000","last_unstale":"2024-05-09T05:38:12.052441+0000","last_undegraded":"2024-05-09T05:38:12.052441+0000","last_fullsized":"2024-05-09T05:38:12.052441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:51.391295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133733+0000","last_change":"2024-05-09T05:33:49.746247+0000","last_active":"2024-05-09T05:38:12.133733+0000","last_peered":"2024-05-09T05:38:12.133733+0000","last_clean":"2024-05-09T05:38:12.133733+0000","last_became_active":"2024-05-09T05:33:49.745420+0000","last_became_peered":"2024-05-09T05:33:49.745420+0000","last_unstale":"2024-05-09T05:38:12.133733+0000","last_undegraded":"2024-05-09T05:38:12.133733+0000","last_fullsized":"2024-05-09T05:38:12.133733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:11:36.224625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"88'99","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:14.055036+0000","last_change":"2024-05-09T05:32:25.081600+0000","last_active":"2024-05-09T05:38:14.055036+0000","last_peered":"2024-05-09T05:38:14.055036+0000","last_clean":"2024-05-09T05:38:14.055036+0000","last_became_active":"2024-05-09T05:32:25.076255+0000","last_became_peered":"2024-05-09T05:32:25.076255+0000","last_unstale":"2024-05-09T05:38:14.055036+0000","last_undegraded":"2024-05-09T05:38:14.055036+0000","last_fullsized":"2024-05-09T05:38:14.055036+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:28:58.704789+0000","last_clean_scrub_stamp":"2024-05-09T05:28:58.704789+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:27.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059582+0000","last_change":"2024-05-09T05:34:05.091189+0000","last_active":"2024-05-09T05:38:12.059582+0000","last_peered":"2024-05-09T05:38:12.059582+0000","last_clean":"2024-05-09T05:38:12.059582+0000","last_became_active":"2024-05-09T05:34:05.090686+0000","last_became_peered":"2024-05-09T05:34:05.090686+0000","last_unstale":"2024-05-09T05:38:12.059582+0000","last_undegraded":"2024-05-09T05:38:12.059582+0000","last_fullsized":"2024-05-09T05:38:12.059582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:53.913948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224704+0000","last_change":"2024-05-09T05:33:52.784229+0000","last_active":"2024-05-09T05:38:12.224704+0000","last_peered":"2024-05-09T05:38:12.224704+0000","last_clean":"2024-05-09T05:38:12.224704+0000","last_became_active":"2024-05-09T05:33:52.783861+0000","last_became_peered":"2024-05-09T05:33:52.783861+0000","last_unstale":"2024-05-09T05:38:12.224704+0000","last_undegraded":"2024-05-09T05:38:12.224704+0000","last_fullsized":"2024-05-09T05:38:12.224704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:39:31.196924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132204+0000","last_change":"2024-05-09T05:33:49.743713+0000","last_active":"2024-05-09T05:38:12.132204+0000","last_peered":"2024-05-09T05:38:12.132204+0000","last_clean":"2024-05-09T05:38:12.132204+0000","last_became_active":"2024-05-09T05:33:49.743365+0000","last_became_peered":"2024-05-09T05:33:49.743365+0000","last_unstale":"2024-05-09T05:38:12.132204+0000","last_undegraded":"2024-05-09T05:38:12.132204+0000","last_fullsized":"2024-05-09T05:38:12.132204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:20.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925156+0000","last_change":"2024-05-09T05:34:04.823514+0000","last_active":"2024-05-09T05:38:11.925156+0000","last_peered":"2024-05-09T05:38:11.925156+0000","last_clean":"2024-05-09T05:38:11.925156+0000","last_became_active":"2024-05-09T05:34:04.823041+0000","last_became_peered":"2024-05-09T05:34:04.823041+0000","last_unstale":"2024-05-09T05:38:11.925156+0000","last_undegraded":"2024-05-09T05:38:11.925156+0000","last_fullsized":"2024-05-09T05:38:11.925156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:17.119541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925336+0000","last_change":"2024-05-09T05:34:33.912938+0000","last_active":"2024-05-09T05:38:11.925336+0000","last_peered":"2024-05-09T05:38:11.925336+0000","last_clean":"2024-05-09T05:38:11.925336+0000","last_became_active":"2024-05-09T05:34:33.912517+0000","last_became_peered":"2024-05-09T05:34:33.912517+0000","last_unstale":"2024-05-09T05:38:11.925336+0000","last_undegraded":"2024-05-09T05:38:11.925336+0000","last_fullsized":"2024-05-09T05:38:11.925336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:56.254097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225403+0000","last_change":"2024-05-09T05:33:52.778002+0000","last_active":"2024-05-09T05:38:12.225403+0000","last_peered":"2024-05-09T05:38:12.225403+0000","last_clean":"2024-05-09T05:38:12.225403+0000","last_became_active":"2024-05-09T05:33:52.777369+0000","last_became_peered":"2024-05-09T05:33:52.777369+0000","last_unstale":"2024-05-09T05:38:12.225403+0000","last_undegraded":"2024-05-09T05:38:12.225403+0000","last_fullsized":"2024-05-09T05:38:12.225403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:55.437896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051969+0000","last_change":"2024-05-09T05:34:05.090802+0000","last_active":"2024-05-09T05:38:12.051969+0000","last_peered":"2024-05-09T05:38:12.051969+0000","last_clean":"2024-05-09T05:38:12.051969+0000","last_became_active":"2024-05-09T05:34:05.089900+0000","last_became_peered":"2024-05-09T05:34:05.089900+0000","last_unstale":"2024-05-09T05:38:12.051969+0000","last_undegraded":"2024-05-09T05:38:12.051969+0000","last_fullsized":"2024-05-09T05:38:12.051969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:59.033674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'107","reported_seq":243,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:26.562070+0000","last_change":"2024-05-09T05:33:49.740183+0000","last_active":"2024-05-09T05:38:26.562070+0000","last_peered":"2024-05-09T05:38:26.562070+0000","last_clean":"2024-05-09T05:38:26.562070+0000","last_became_active":"2024-05-09T05:33:49.739810+0000","last_became_peered":"2024-05-09T05:33:49.739810+0000","last_unstale":"2024-05-09T05:38:26.562070+0000","last_undegraded":"2024-05-09T05:38:26.562070+0000","last_fullsized":"2024-05-09T05:38:26.562070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.276597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":195,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053724+0000","last_change":"2024-05-09T05:33:52.774584+0000","last_active":"2024-05-09T05:38:12.053724+0000","last_peered":"2024-05-09T05:38:12.053724+0000","last_clean":"2024-05-09T05:38:12.053724+0000","last_became_active":"2024-05-09T05:33:52.774231+0000","last_became_peered":"2024-05-09T05:33:52.774231+0000","last_unstale":"2024-05-09T05:38:12.053724+0000","last_undegraded":"2024-05-09T05:38:12.053724+0000","last_fullsized":"2024-05-09T05:38:12.053724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:57:37.733710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131803+0000","last_change":"2024-05-09T05:34:04.827303+0000","last_active":"2024-05-09T05:38:12.131803+0000","last_peered":"2024-05-09T05:38:12.131803+0000","last_clean":"2024-05-09T05:38:12.131803+0000","last_became_active":"2024-05-09T05:34:04.826859+0000","last_became_peered":"2024-05-09T05:34:04.826859+0000","last_unstale":"2024-05-09T05:38:12.131803+0000","last_undegraded":"2024-05-09T05:38:12.131803+0000","last_fullsized":"2024-05-09T05:38:12.131803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:09.847554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.134405+0000","last_change":"2024-05-09T05:33:49.873083+0000","last_active":"2024-05-09T05:38:12.134405+0000","last_peered":"2024-05-09T05:38:12.134405+0000","last_clean":"2024-05-09T05:38:12.134405+0000","last_became_active":"2024-05-09T05:33:49.872592+0000","last_became_peered":"2024-05-09T05:33:49.872592+0000","last_unstale":"2024-05-09T05:38:12.134405+0000","last_undegraded":"2024-05-09T05:38:12.134405+0000","last_fullsized":"2024-05-09T05:38:12.134405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:40:03.918547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979676+0000","last_change":"2024-05-09T05:33:52.784416+0000","last_active":"2024-05-09T05:38:11.979676+0000","last_peered":"2024-05-09T05:38:11.979676+0000","last_clean":"2024-05-09T05:38:11.979676+0000","last_became_active":"2024-05-09T05:33:52.784076+0000","last_became_peered":"2024-05-09T05:33:52.784076+0000","last_unstale":"2024-05-09T05:38:11.979676+0000","last_undegraded":"2024-05-09T05:38:11.979676+0000","last_fullsized":"2024-05-09T05:38:11.979676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:08.246357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133578+0000","last_change":"2024-05-09T05:34:05.091928+0000","last_active":"2024-05-09T05:38:12.133578+0000","last_peered":"2024-05-09T05:38:12.133578+0000","last_clean":"2024-05-09T05:38:12.133578+0000","last_became_active":"2024-05-09T05:34:05.091346+0000","last_became_peered":"2024-05-09T05:34:05.091346+0000","last_unstale":"2024-05-09T05:38:12.133578+0000","last_undegraded":"2024-05-09T05:38:12.133578+0000","last_fullsized":"2024-05-09T05:38:12.133578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:51.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132717+0000","last_change":"2024-05-09T05:33:49.832072+0000","last_active":"2024-05-09T05:38:12.132717+0000","last_peered":"2024-05-09T05:38:12.132717+0000","last_clean":"2024-05-09T05:38:12.132717+0000","last_became_active":"2024-05-09T05:33:49.831657+0000","last_became_peered":"2024-05-09T05:33:49.831657+0000","last_unstale":"2024-05-09T05:38:12.132717+0000","last_undegraded":"2024-05-09T05:38:12.132717+0000","last_fullsized":"2024-05-09T05:38:12.132717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:38.124193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051314+0000","last_change":"2024-05-09T05:33:52.781841+0000","last_active":"2024-05-09T05:38:12.051314+0000","last_peered":"2024-05-09T05:38:12.051314+0000","last_clean":"2024-05-09T05:38:12.051314+0000","last_became_active":"2024-05-09T05:33:52.781465+0000","last_became_peered":"2024-05-09T05:33:52.781465+0000","last_unstale":"2024-05-09T05:38:12.051314+0000","last_undegraded":"2024-05-09T05:38:12.051314+0000","last_fullsized":"2024-05-09T05:38:12.051314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:19:52.917190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924777+0000","last_change":"2024-05-09T05:34:04.829889+0000","last_active":"2024-05-09T05:38:11.924777+0000","last_peered":"2024-05-09T05:38:11.924777+0000","last_clean":"2024-05-09T05:38:11.924777+0000","last_became_active":"2024-05-09T05:34:04.829493+0000","last_became_peered":"2024-05-09T05:34:04.829493+0000","last_unstale":"2024-05-09T05:38:11.924777+0000","last_undegraded":"2024-05-09T05:38:11.924777+0000","last_fullsized":"2024-05-09T05:38:11.924777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:27.765388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053182+0000","last_change":"2024-05-09T05:33:49.740055+0000","last_active":"2024-05-09T05:38:12.053182+0000","last_peered":"2024-05-09T05:38:12.053182+0000","last_clean":"2024-05-09T05:38:12.053182+0000","last_became_active":"2024-05-09T05:33:49.739718+0000","last_became_peered":"2024-05-09T05:33:49.739718+0000","last_unstale":"2024-05-09T05:38:12.053182+0000","last_undegraded":"2024-05-09T05:38:12.053182+0000","last_fullsized":"2024-05-09T05:38:12.053182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:43:17.908322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.924983+0000","last_change":"2024-05-09T05:33:52.777213+0000","last_active":"2024-05-09T05:38:11.924983+0000","last_peered":"2024-05-09T05:38:11.924983+0000","last_clean":"2024-05-09T05:38:11.924983+0000","last_became_active":"2024-05-09T05:33:52.776797+0000","last_became_peered":"2024-05-09T05:33:52.776797+0000","last_unstale":"2024-05-09T05:38:11.924983+0000","last_undegraded":"2024-05-09T05:38:11.924983+0000","last_fullsized":"2024-05-09T05:38:11.924983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:07.317851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133063+0000","last_change":"2024-05-09T05:34:04.828680+0000","last_active":"2024-05-09T05:38:12.133063+0000","last_peered":"2024-05-09T05:38:12.133063+0000","last_clean":"2024-05-09T05:38:12.133063+0000","last_became_active":"2024-05-09T05:34:04.828254+0000","last_became_peered":"2024-05-09T05:34:04.828254+0000","last_unstale":"2024-05-09T05:38:12.133063+0000","last_undegraded":"2024-05-09T05:38:12.133063+0000","last_fullsized":"2024-05-09T05:38:12.133063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:53.759921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.315433+0000","last_change":"2024-05-09T05:33:49.746139+0000","last_active":"2024-05-09T05:38:12.315433+0000","last_peered":"2024-05-09T05:38:12.315433+0000","last_clean":"2024-05-09T05:38:12.315433+0000","last_became_active":"2024-05-09T05:33:49.743465+0000","last_became_peered":"2024-05-09T05:33:49.743465+0000","last_unstale":"2024-05-09T05:38:12.315433+0000","last_undegraded":"2024-05-09T05:38:12.315433+0000","last_fullsized":"2024-05-09T05:38:12.315433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:52.578229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053097+0000","last_change":"2024-05-09T05:33:52.775802+0000","last_active":"2024-05-09T05:38:12.053097+0000","last_peered":"2024-05-09T05:38:12.053097+0000","last_clean":"2024-05-09T05:38:12.053097+0000","last_became_active":"2024-05-09T05:33:52.775061+0000","last_became_peered":"2024-05-09T05:33:52.775061+0000","last_unstale":"2024-05-09T05:38:12.053097+0000","last_undegraded":"2024-05-09T05:38:12.053097+0000","last_fullsized":"2024-05-09T05:38:12.053097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:51.977250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133542+0000","last_change":"2024-05-09T05:34:05.091104+0000","last_active":"2024-05-09T05:38:12.133542+0000","last_peered":"2024-05-09T05:38:12.133542+0000","last_clean":"2024-05-09T05:38:12.133542+0000","last_became_active":"2024-05-09T05:34:05.090590+0000","last_became_peered":"2024-05-09T05:34:05.090590+0000","last_unstale":"2024-05-09T05:38:12.133542+0000","last_undegraded":"2024-05-09T05:38:12.133542+0000","last_fullsized":"2024-05-09T05:38:12.133542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:57:07.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.051419+0000","last_change":"2024-05-09T05:33:49.748698+0000","last_active":"2024-05-09T05:38:12.051419+0000","last_peered":"2024-05-09T05:38:12.051419+0000","last_clean":"2024-05-09T05:38:12.051419+0000","last_became_active":"2024-05-09T05:33:49.748348+0000","last_became_peered":"2024-05-09T05:33:49.748348+0000","last_unstale":"2024-05-09T05:38:12.051419+0000","last_undegraded":"2024-05-09T05:38:12.051419+0000","last_fullsized":"2024-05-09T05:38:12.051419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:19.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132543+0000","last_change":"2024-05-09T05:33:52.785246+0000","last_active":"2024-05-09T05:38:12.132543+0000","last_peered":"2024-05-09T05:38:12.132543+0000","last_clean":"2024-05-09T05:38:12.132543+0000","last_became_active":"2024-05-09T05:33:52.784871+0000","last_became_peered":"2024-05-09T05:33:52.784871+0000","last_unstale":"2024-05-09T05:38:12.132543+0000","last_undegraded":"2024-05-09T05:38:12.132543+0000","last_fullsized":"2024-05-09T05:38:12.132543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:22.507051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133080+0000","last_change":"2024-05-09T05:34:04.824251+0000","last_active":"2024-05-09T05:38:12.133080+0000","last_peered":"2024-05-09T05:38:12.133080+0000","last_clean":"2024-05-09T05:38:12.133080+0000","last_became_active":"2024-05-09T05:34:04.823618+0000","last_became_peered":"2024-05-09T05:34:04.823618+0000","last_unstale":"2024-05-09T05:38:12.133080+0000","last_undegraded":"2024-05-09T05:38:12.133080+0000","last_fullsized":"2024-05-09T05:38:12.133080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:44.455175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":264,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:31.656965+0000","last_change":"2024-05-09T05:33:49.872540+0000","last_active":"2024-05-09T05:38:31.656965+0000","last_peered":"2024-05-09T05:38:31.656965+0000","last_clean":"2024-05-09T05:38:31.656965+0000","last_became_active":"2024-05-09T05:33:49.872115+0000","last_became_peered":"2024-05-09T05:33:49.872115+0000","last_unstale":"2024-05-09T05:38:31.656965+0000","last_undegraded":"2024-05-09T05:38:31.656965+0000","last_fullsized":"2024-05-09T05:38:31.656965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:38:24.826836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133983+0000","last_change":"2024-05-09T05:33:52.785382+0000","last_active":"2024-05-09T05:38:12.133983+0000","last_peered":"2024-05-09T05:38:12.133983+0000","last_clean":"2024-05-09T05:38:12.133983+0000","last_became_active":"2024-05-09T05:33:52.785030+0000","last_became_peered":"2024-05-09T05:33:52.785030+0000","last_unstale":"2024-05-09T05:38:12.133983+0000","last_undegraded":"2024-05-09T05:38:12.133983+0000","last_fullsized":"2024-05-09T05:38:12.133983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:36.133635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979240+0000","last_change":"2024-05-09T05:34:04.830613+0000","last_active":"2024-05-09T05:38:11.979240+0000","last_peered":"2024-05-09T05:38:11.979240+0000","last_clean":"2024-05-09T05:38:11.979240+0000","last_became_active":"2024-05-09T05:34:04.830190+0000","last_became_peered":"2024-05-09T05:34:04.830190+0000","last_unstale":"2024-05-09T05:38:11.979240+0000","last_undegraded":"2024-05-09T05:38:11.979240+0000","last_fullsized":"2024-05-09T05:38:11.979240+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:37.695090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'67","reported_seq":141,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:26.196028+0000","last_change":"2024-05-09T05:33:49.740791+0000","last_active":"2024-05-09T05:38:26.196028+0000","last_peered":"2024-05-09T05:38:26.196028+0000","last_clean":"2024-05-09T05:38:26.196028+0000","last_became_active":"2024-05-09T05:33:49.739630+0000","last_became_peered":"2024-05-09T05:33:49.739630+0000","last_unstale":"2024-05-09T05:38:26.196028+0000","last_undegraded":"2024-05-09T05:38:26.196028+0000","last_fullsized":"2024-05-09T05:38:26.196028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:27.810701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132662+0000","last_change":"2024-05-09T05:33:52.781607+0000","last_active":"2024-05-09T05:38:12.132662+0000","last_peered":"2024-05-09T05:38:12.132662+0000","last_clean":"2024-05-09T05:38:12.132662+0000","last_became_active":"2024-05-09T05:33:52.781256+0000","last_became_peered":"2024-05-09T05:33:52.781256+0000","last_unstale":"2024-05-09T05:38:12.132662+0000","last_undegraded":"2024-05-09T05:38:12.132662+0000","last_fullsized":"2024-05-09T05:38:12.132662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:21.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925491+0000","last_change":"2024-05-09T05:34:33.941437+0000","last_active":"2024-05-09T05:38:11.925491+0000","last_peered":"2024-05-09T05:38:11.925491+0000","last_clean":"2024-05-09T05:38:11.925491+0000","last_became_active":"2024-05-09T05:34:33.940814+0000","last_became_peered":"2024-05-09T05:34:33.940814+0000","last_unstale":"2024-05-09T05:38:11.925491+0000","last_undegraded":"2024-05-09T05:38:11.925491+0000","last_fullsized":"2024-05-09T05:38:11.925491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:34:03.801975+0000","last_clean_scrub_stamp":"2024-05-09T05:34:03.801975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:34.166167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,7,1],"acting":[10,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225592+0000","last_change":"2024-05-09T05:33:49.745180+0000","last_active":"2024-05-09T05:38:12.225592+0000","last_peered":"2024-05-09T05:38:12.225592+0000","last_clean":"2024-05-09T05:38:12.225592+0000","last_became_active":"2024-05-09T05:33:49.744797+0000","last_became_peered":"2024-05-09T05:33:49.744797+0000","last_unstale":"2024-05-09T05:38:12.225592+0000","last_undegraded":"2024-05-09T05:38:12.225592+0000","last_fullsized":"2024-05-09T05:38:12.225592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:48.725500+0000","last_clean_scrub_stamp":"2024-05-09T05:33:48.725500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:37.958584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133498+0000","last_change":"2024-05-09T05:33:52.783205+0000","last_active":"2024-05-09T05:38:12.133498+0000","last_peered":"2024-05-09T05:38:12.133498+0000","last_clean":"2024-05-09T05:38:12.133498+0000","last_became_active":"2024-05-09T05:33:52.782829+0000","last_became_peered":"2024-05-09T05:33:52.782829+0000","last_unstale":"2024-05-09T05:38:12.133498+0000","last_undegraded":"2024-05-09T05:38:12.133498+0000","last_fullsized":"2024-05-09T05:38:12.133498+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:57.091445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925076+0000","last_change":"2024-05-09T05:33:52.778792+0000","last_active":"2024-05-09T05:38:11.925076+0000","last_peered":"2024-05-09T05:38:11.925076+0000","last_clean":"2024-05-09T05:38:11.925076+0000","last_became_active":"2024-05-09T05:33:52.777736+0000","last_became_peered":"2024-05-09T05:33:52.777736+0000","last_unstale":"2024-05-09T05:38:11.925076+0000","last_undegraded":"2024-05-09T05:38:11.925076+0000","last_fullsized":"2024-05-09T05:38:11.925076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:55:59.298344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132222+0000","last_change":"2024-05-09T05:33:52.785105+0000","last_active":"2024-05-09T05:38:12.132222+0000","last_peered":"2024-05-09T05:38:12.132222+0000","last_clean":"2024-05-09T05:38:12.132222+0000","last_became_active":"2024-05-09T05:33:52.784716+0000","last_became_peered":"2024-05-09T05:33:52.784716+0000","last_unstale":"2024-05-09T05:38:12.132222+0000","last_undegraded":"2024-05-09T05:38:12.132222+0000","last_fullsized":"2024-05-09T05:38:12.132222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:27.721601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.131810+0000","last_change":"2024-05-09T05:33:52.776237+0000","last_active":"2024-05-09T05:38:12.131810+0000","last_peered":"2024-05-09T05:38:12.131810+0000","last_clean":"2024-05-09T05:38:12.131810+0000","last_became_active":"2024-05-09T05:33:52.775865+0000","last_became_peered":"2024-05-09T05:33:52.775865+0000","last_unstale":"2024-05-09T05:38:12.131810+0000","last_undegraded":"2024-05-09T05:38:12.131810+0000","last_fullsized":"2024-05-09T05:38:12.131810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:15.818598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.979502+0000","last_change":"2024-05-09T05:33:52.784440+0000","last_active":"2024-05-09T05:38:11.979502+0000","last_peered":"2024-05-09T05:38:11.979502+0000","last_clean":"2024-05-09T05:38:11.979502+0000","last_became_active":"2024-05-09T05:33:52.784013+0000","last_became_peered":"2024-05-09T05:33:52.784013+0000","last_unstale":"2024-05-09T05:38:11.979502+0000","last_undegraded":"2024-05-09T05:38:11.979502+0000","last_fullsized":"2024-05-09T05:38:11.979502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:02:47.294013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.225302+0000","last_change":"2024-05-09T05:33:52.784330+0000","last_active":"2024-05-09T05:38:12.225302+0000","last_peered":"2024-05-09T05:38:12.225302+0000","last_clean":"2024-05-09T05:38:12.225302+0000","last_became_active":"2024-05-09T05:33:52.783883+0000","last_became_peered":"2024-05-09T05:33:52.783883+0000","last_unstale":"2024-05-09T05:38:12.225302+0000","last_undegraded":"2024-05-09T05:38:12.225302+0000","last_fullsized":"2024-05-09T05:38:12.225302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:05.049431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925541+0000","last_change":"2024-05-09T05:33:52.782276+0000","last_active":"2024-05-09T05:38:11.925541+0000","last_peered":"2024-05-09T05:38:11.925541+0000","last_clean":"2024-05-09T05:38:11.925541+0000","last_became_active":"2024-05-09T05:33:52.781828+0000","last_became_peered":"2024-05-09T05:33:52.781828+0000","last_unstale":"2024-05-09T05:38:11.925541+0000","last_undegraded":"2024-05-09T05:38:11.925541+0000","last_fullsized":"2024-05-09T05:38:11.925541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:06:34.091436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.132345+0000","last_change":"2024-05-09T05:33:52.780713+0000","last_active":"2024-05-09T05:38:12.132345+0000","last_peered":"2024-05-09T05:38:12.132345+0000","last_clean":"2024-05-09T05:38:12.132345+0000","last_became_active":"2024-05-09T05:33:52.780274+0000","last_became_peered":"2024-05-09T05:33:52.780274+0000","last_unstale":"2024-05-09T05:38:12.132345+0000","last_undegraded":"2024-05-09T05:38:12.132345+0000","last_fullsized":"2024-05-09T05:38:12.132345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:22.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053867+0000","last_change":"2024-05-09T05:33:52.777698+0000","last_active":"2024-05-09T05:38:12.053867+0000","last_peered":"2024-05-09T05:38:12.053867+0000","last_clean":"2024-05-09T05:38:12.053867+0000","last_became_active":"2024-05-09T05:33:52.776752+0000","last_became_peered":"2024-05-09T05:33:52.776752+0000","last_unstale":"2024-05-09T05:38:12.053867+0000","last_undegraded":"2024-05-09T05:38:12.053867+0000","last_fullsized":"2024-05-09T05:38:12.053867+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:23.063464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.059147+0000","last_change":"2024-05-09T05:33:52.775154+0000","last_active":"2024-05-09T05:38:12.059147+0000","last_peered":"2024-05-09T05:38:12.059147+0000","last_clean":"2024-05-09T05:38:12.059147+0000","last_became_active":"2024-05-09T05:33:52.774715+0000","last_became_peered":"2024-05-09T05:33:52.774715+0000","last_unstale":"2024-05-09T05:38:12.059147+0000","last_undegraded":"2024-05-09T05:38:12.059147+0000","last_fullsized":"2024-05-09T05:38:12.059147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:42:39.077594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.053305+0000","last_change":"2024-05-09T05:33:52.777566+0000","last_active":"2024-05-09T05:38:12.053305+0000","last_peered":"2024-05-09T05:38:12.053305+0000","last_clean":"2024-05-09T05:38:12.053305+0000","last_became_active":"2024-05-09T05:33:52.777117+0000","last_became_peered":"2024-05-09T05:33:52.777117+0000","last_unstale":"2024-05-09T05:38:12.053305+0000","last_undegraded":"2024-05-09T05:38:12.053305+0000","last_fullsized":"2024-05-09T05:38:12.053305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:25.117493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.133431+0000","last_change":"2024-05-09T05:33:52.792690+0000","last_active":"2024-05-09T05:38:12.133431+0000","last_peered":"2024-05-09T05:38:12.133431+0000","last_clean":"2024-05-09T05:38:12.133431+0000","last_became_active":"2024-05-09T05:33:52.792015+0000","last_became_peered":"2024-05-09T05:33:52.792015+0000","last_unstale":"2024-05-09T05:38:12.133431+0000","last_undegraded":"2024-05-09T05:38:12.133431+0000","last_fullsized":"2024-05-09T05:38:12.133431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:58:05.579977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:12.224991+0000","last_change":"2024-05-09T05:33:52.780304+0000","last_active":"2024-05-09T05:38:12.224991+0000","last_peered":"2024-05-09T05:38:12.224991+0000","last_clean":"2024-05-09T05:38:12.224991+0000","last_became_active":"2024-05-09T05:33:52.779745+0000","last_became_peered":"2024-05-09T05:33:52.779745+0000","last_unstale":"2024-05-09T05:38:12.224991+0000","last_undegraded":"2024-05-09T05:38:12.224991+0000","last_fullsized":"2024-05-09T05:38:12.224991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:31.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T05:38:11.925314+0000","last_change":"2024-05-09T05:33:52.778972+0000","last_active":"2024-05-09T05:38:11.925314+0000","last_peered":"2024-05-09T05:38:11.925314+0000","last_clean":"2024-05-09T05:38:11.925314+0000","last_became_active":"2024-05-09T05:33:52.777773+0000","last_became_peered":"2024-05-09T05:33:52.777773+0000","last_unstale":"2024-05-09T05:38:11.925314+0000","last_undegraded":"2024-05-09T05:38:11.925314+0000","last_fullsized":"2024-05-09T05:38:11.925314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T05:33:51.742563+0000","last_clean_scrub_stamp":"2024-05-09T05:33:51.742563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:11.091809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107766,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801521,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2498,"num_read_kb":1474,"num_write":520,"num_write_kb":2394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2404563,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710790,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507638,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907020,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1492548,"data_compressed":80636,"data_compressed_allocated":106496,"data_compressed_original":978944,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070546,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":903884,"data_compressed":48801,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266775,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":900054,"data_compressed":48801,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463004,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":1105849,"data_compressed":65434,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626529,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":502960,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":573964,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":504416,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":575209,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182512,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1223232,"data_compressed":64003,"data_compressed_allocated":86016,"data_compressed_original":782336,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986282,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507034,"data_compressed":29512,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822759,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1427895,"data_compressed":80636,"data_compressed_allocated":106496,"data_compressed_original":978944,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35922,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332077,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400826,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T05:38:38.305 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:38:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:39.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:39 smithi195 ceph-mon[33844]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:39.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:39 smithi195 ceph-mon[33844]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-09T05:38:39.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:39 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:38:39.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:39 smithi115 ceph-mon[30349]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:39.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:39 smithi115 ceph-mon[30349]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-09T05:38:39.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:39 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:38:39.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:39 smithi161 ceph-mon[33805]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T05:38:39.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:39 smithi161 ceph-mon[33805]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-09T05:38:39.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:39 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:38:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 -- 172.21.15.115:0/1399032779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4108f80 msgr2=0x7f48c4110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/1399032779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4108f80 0x7f48c4110720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009ca0 tx=0x7f48bc02a8c0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 -- 172.21.15.115:0/1399032779 shutdown_connections 2024-05-09T05:38:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/1399032779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4108f80 0x7f48c4110720 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/1399032779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c41084e0 0x7f48c41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/1399032779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c4102460 0x7f48c4102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 -- 172.21.15.115:0/1399032779 >> 172.21.15.115:0/1399032779 conn(0x7f48c40fd9d0 msgr2=0x7f48c40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 -- 172.21.15.115:0/1399032779 shutdown_connections 2024-05-09T05:38:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 -- 172.21.15.115:0/1399032779 wait complete. 2024-05-09T05:38:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.792+0000 7f48c9b6b640 1 Processor -- start 2024-05-09T05:38:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c9b6b640 1 -- start start 2024-05-09T05:38:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c9b6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4102460 0x7f48c419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c9b6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c41084e0 0x7f48c419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c9b6b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c4108f80 0x7f48c419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c9b6b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4113db0 con 0x7f48c4102460 2024-05-09T05:38:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c9b6b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4113c30 con 0x7f48c41084e0 2024-05-09T05:38:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c9b6b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4113f30 con 0x7f48c4108f80 2024-05-09T05:38:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4102460 0x7f48c419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4102460 0x7f48c419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52468/0 (socket says 172.21.15.115:52468) 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c37fe640 1 -- 172.21.15.115:0/2701874925 learned_addr learned my addr 172.21.15.115:0/2701874925 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c2ffd640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c41084e0 0x7f48c419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c3fff640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c4108f80 0x7f48c419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c37fe640 1 -- 172.21.15.115:0/2701874925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c4108f80 msgr2=0x7f48c419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c37fe640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c4108f80 0x7f48c419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c37fe640 1 -- 172.21.15.115:0/2701874925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c41084e0 msgr2=0x7f48c419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c37fe640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c41084e0 0x7f48c419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.793+0000 7f48c37fe640 1 -- 172.21.15.115:0/2701874925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c419f5f0 con 0x7f48c4102460 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.794+0000 7f48c37fe640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4102460 0x7f48c419a790 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f48b400d950 tx=0x7f48b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.794+0000 7f48c0ff9640 1 -- 172.21.15.115:0/2701874925 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4017020 con 0x7f48c4102460 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.794+0000 7f48c0ff9640 1 -- 172.21.15.115:0/2701874925 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48b4009c30 con 0x7f48c4102460 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.794+0000 7f48c0ff9640 1 -- 172.21.15.115:0/2701874925 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b400f620 con 0x7f48c4102460 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.794+0000 7f48c9b6b640 1 -- 172.21.15.115:0/2701874925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c419f8e0 con 0x7f48c4102460 2024-05-09T05:38:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.794+0000 7f48c9b6b640 1 -- 172.21.15.115:0/2701874925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c41a7260 con 0x7f48c4102460 2024-05-09T05:38:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.796+0000 7f48d0a1d740 1 Processor -- start 2024-05-09T05:38:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.796+0000 7f48d0a1d740 1 -- start start 2024-05-09T05:38:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.796+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559427a0f1b0 0x559427db2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.796+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279e62f0 0x559427db2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279ee4d0 0x559427db3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559427d177c0 con 0x5594279ee4d0 2024-05-09T05:38:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559427cb30e0 con 0x5594279e62f0 2024-05-09T05:38:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559427d13780 con 0x559427a0f1b0 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279ee4d0 0x559427db3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c0ff9640 1 -- 172.21.15.115:0/2701874925 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f48b401a020 con 0x7f48c4102460 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279ee4d0 0x559427db3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52470/0 (socket says 172.21.15.115:52470) 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c2ffd640 1 -- 172.21.15.115:0/4212542278 learned_addr learned my addr 172.21.15.115:0/4212542278 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c3fff640 1 --2- 172.21.15.115:0/4212542278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559427a0f1b0 0x559427db2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c2ffd640 1 -- 172.21.15.115:0/4212542278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559427a0f1b0 msgr2=0x559427db2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c2ffd640 1 --2- 172.21.15.115:0/4212542278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559427a0f1b0 0x559427db2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c2ffd640 1 -- 172.21.15.115:0/4212542278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279e62f0 msgr2=0x559427db2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c2ffd640 1 --2- 172.21.15.115:0/4212542278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279e62f0 0x559427db2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c2ffd640 1 -- 172.21.15.115:0/4212542278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559427cdc7c0 con 0x5594279ee4d0 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48c2ffd640 1 --2- 172.21.15.115:0/4212542278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279ee4d0 0x559427db3100 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f48b800bb90 tx=0x7f48b800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d16b6e2285b439 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48a9ffb640 1 -- 172.21.15.115:0/4212542278 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8015070 con 0x5594279ee4d0 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48a9ffb640 1 -- 172.21.15.115:0/4212542278 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48b80054b0 con 0x5594279ee4d0 2024-05-09T05:38:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.797+0000 7f48a9ffb640 1 -- 172.21.15.115:0/4212542278 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8011440 con 0x5594279ee4d0 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48c0ff9640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48ac0b29e0 0x7f48ac0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48c3fff640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48ac0b29e0 0x7f48ac0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48c0ff9640 1 -- 172.21.15.115:0/2701874925 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f48b40e2430 con 0x7f48c4102460 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48d0a1d740 1 -- 172.21.15.115:0/4212542278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279ee4d0 msgr2=0x559427db3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48c3fff640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48ac0b29e0 0x7f48ac0b4ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x559427daf7f0 tx=0x7f48bc009430 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/4212542278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279ee4d0 0x559427db3100 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f48b800bb90 tx=0x7f48b800f730 comp rx=0 tx=0).stop 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48d0a1d740 1 -- 172.21.15.115:0/4212542278 shutdown_connections 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/4212542278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279ee4d0 0x559427db3100 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/4212542278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279e62f0 0x559427db2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/4212542278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559427a0f1b0 0x559427db2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48d0a1d740 1 -- 172.21.15.115:0/4212542278 >> 172.21.15.115:0/4212542278 conn(0x559427a03300 msgr2=0x559427daee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48d0a1d740 1 -- 172.21.15.115:0/4212542278 shutdown_connections 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.798+0000 7f48d0a1d740 1 -- 172.21.15.115:0/4212542278 wait complete. 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48d0a1d740 1 -- start start 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48d0a1d740 1 Processor -- start 2024-05-09T05:38:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279e62f0 0x559427dbe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279ee4d0 0x559427dbecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dbcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559427dbf7b0 con 0x559427a0f1b0 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559427c9e9c0 con 0x5594279e62f0 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559427dbf930 con 0x5594279ee4d0 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48d0a1d740 10 client.0 _ll_register_callbacks cb 0x559427ccb020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dbcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dbcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52472/0 (socket says 172.21.15.115:52472) 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.799+0000 7f48c3fff640 1 -- 172.21.15.115:0/2659264104 learned_addr learned my addr 172.21.15.115:0/2659264104 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48c2ffd640 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279ee4d0 0x559427dbecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48c37fe640 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279e62f0 0x559427dbe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48c3fff640 1 -- 172.21.15.115:0/2659264104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279ee4d0 msgr2=0x559427dbecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48c3fff640 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279ee4d0 0x559427dbecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48c3fff640 1 -- 172.21.15.115:0/2659264104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279e62f0 msgr2=0x559427dbe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48c3fff640 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279e62f0 0x559427dbe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48c3fff640 1 -- 172.21.15.115:0/2659264104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559427cc51f0 con 0x559427a0f1b0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48c3fff640 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dbcbf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f48bc035000 tx=0x7f48bc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe527b11b8ec72f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48a2ffd640 1 -- 172.21.15.115:0/2659264104 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc04f070 con 0x559427a0f1b0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48a2ffd640 1 -- 172.21.15.115:0/2659264104 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f48bc004450 con 0x559427a0f1b0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48d0a1d740 1 -- 172.21.15.115:0/2659264104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48b80054b0 con 0x559427a0f1b0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48a2ffd640 1 -- 172.21.15.115:0/2659264104 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f48bc004690 con 0x559427a0f1b0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48d0a1d740 10 client.15555 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48d0a1d740 1 -- 172.21.15.115:0/2659264104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559427a7a790 con 0x559427a0f1b0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.800+0000 7f48a2ffd640 1 -- 172.21.15.115:0/2659264104 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48b80054b0 con 0x559427a0f1b0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48a2ffd640 1 -- 172.21.15.115:0/2659264104 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f48bc044040 con 0x559427a0f1b0 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48d0a1d740 10 client.15555 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48d0a1d740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:38:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48d0a1d740 20 client.15555 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594279cdd80 0x559427dcf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48d0a1d740 4 client.15555 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48d0a1d740 1 -- 172.21.15.115:0/2659264104 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559427d253d0 con 0x5594279cdd80 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48c37fe640 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594279cdd80 0x559427dcf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48a2ffd640 10 client.15555 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.801+0000 7f48c37fe640 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594279cdd80 0x559427dcf070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692ef98c7148bc40 server_cookie=eef9f0947a5f4fb1 in_seq=0 out_seq=0 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48a2ffd640 1 -- 172.21.15.115:0/2659264104 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559427d253d0 con 0x5594279cdd80 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48a2ffd640 10 client.15555 handle_command_reply: tid=0 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48d0a1d740 1 client.15555 shutdown 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48d0a1d740 1 -- 172.21.15.115:0/2659264104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 msgr2=0x559427dbcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dbcbf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f48bc035000 tx=0x7f48bc0095c0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48d0a1d740 1 -- 172.21.15.115:0/2659264104 shutdown_connections 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dbcbf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594279cdd80 0x559427dcf070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279ee4d0 0x559427dbecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/2659264104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279e62f0 0x559427dbe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48a2ffd640 0 client.15555 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.802+0000 7f48d0a1d740 1 -- 172.21.15.115:0/2659264104 >> 172.21.15.115:0/2659264104 conn(0x559427a03300 msgr2=0x559427db43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.803+0000 7f48d0a1d740 1 -- 172.21.15.115:0/2659264104 shutdown_connections 2024-05-09T05:38:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.803+0000 7f48d0a1d740 1 -- 172.21.15.115:0/2659264104 wait complete. 2024-05-09T05:38:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.803+0000 7f48d0a1d740 20 client.15555 trim_cache size 0 max 16384 2024-05-09T05:38:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.817+0000 7f48d0a1d740 1 Processor -- start 2024-05-09T05:38:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.817+0000 7f48d0a1d740 1 -- start start 2024-05-09T05:38:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279e62f0 0x559427dbea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279ee4d0 0x559427dd4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559427a0f1b0 0x559427dd4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b1050 con 0x5594279e62f0 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b2b90 con 0x5594279ee4d0 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b1720 con 0x559427a0f1b0 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48c37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279e62f0 0x559427dbea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48c37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279e62f0 0x559427dbea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52478/0 (socket says 172.21.15.115:52478) 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48c37fe640 1 -- 172.21.15.115:0/3986431896 learned_addr learned my addr 172.21.15.115:0/3986431896 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48c37fe640 1 -- 172.21.15.115:0/3986431896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559427a0f1b0 msgr2=0x559427dd4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48c37fe640 1 --2- 172.21.15.115:0/3986431896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559427a0f1b0 0x559427dd4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48c37fe640 1 -- 172.21.15.115:0/3986431896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279ee4d0 msgr2=0x559427dd4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48c37fe640 1 --2- 172.21.15.115:0/3986431896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279ee4d0 0x559427dd4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48c37fe640 1 -- 172.21.15.115:0/3986431896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559427af15f0 con 0x5594279e62f0 2024-05-09T05:38:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48c37fe640 1 --2- 172.21.15.115:0/3986431896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279e62f0 0x559427dbea20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f48b4005fd0 tx=0x7f48b40a9a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b468cc6fc84b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48a97fa640 1 -- 172.21.15.115:0/3986431896 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b40c3070 con 0x5594279e62f0 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48a97fa640 1 -- 172.21.15.115:0/3986431896 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48b40a9300 con 0x5594279e62f0 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.818+0000 7f48a97fa640 1 -- 172.21.15.115:0/3986431896 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4010980 con 0x5594279e62f0 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 -- 172.21.15.115:0/3986431896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279e62f0 msgr2=0x559427dbea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/3986431896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279e62f0 0x559427dbea20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f48b4005fd0 tx=0x7f48b40a9a00 comp rx=0 tx=0).stop 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 -- 172.21.15.115:0/3986431896 shutdown_connections 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/3986431896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559427a0f1b0 0x559427dd4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/3986431896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279ee4d0 0x559427dd4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/3986431896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594279e62f0 0x559427dbea20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 -- 172.21.15.115:0/3986431896 >> 172.21.15.115:0/3986431896 conn(0x559427a03300 msgr2=0x559427dbe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 -- 172.21.15.115:0/3986431896 shutdown_connections 2024-05-09T05:38:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 -- 172.21.15.115:0/3986431896 wait complete. 2024-05-09T05:38:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.819+0000 7f48d0a1d740 1 -- start start 2024-05-09T05:38:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48d0a1d740 1 Processor -- start 2024-05-09T05:38:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279e62f0 0x559427dc4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279ee4d0 0x559427dc02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48d0a1d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dc0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b24c0 con 0x559427a0f1b0 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b1df0 con 0x5594279ee4d0 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48d0a1d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41af030 con 0x5594279e62f0 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48d0a1d740 10 client.0 _ll_register_callbacks cb 0x7f48bc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dc0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dc0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52492/0 (socket says 172.21.15.115:52492) 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c3fff640 1 -- 172.21.15.115:0/1925004215 learned_addr learned my addr 172.21.15.115:0/1925004215 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c37fe640 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279e62f0 0x559427dc4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c2ffd640 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279ee4d0 0x559427dc02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c3fff640 1 -- 172.21.15.115:0/1925004215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279e62f0 msgr2=0x559427dc4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c3fff640 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279e62f0 0x559427dc4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c3fff640 1 -- 172.21.15.115:0/1925004215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279ee4d0 msgr2=0x559427dc02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c3fff640 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279ee4d0 0x559427dc02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.820+0000 7f48c3fff640 1 -- 172.21.15.115:0/1925004215 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559427cdbed0 con 0x559427a0f1b0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48c3fff640 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dc0810 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009bf0 tx=0x7f48bc0467b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304a62fa06273f19 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48a2ffd640 1 -- 172.21.15.115:0/1925004215 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc04f040 con 0x559427a0f1b0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48d0a1d740 1 -- 172.21.15.115:0/1925004215 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48b40a9300 con 0x559427a0f1b0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48a2ffd640 1 -- 172.21.15.115:0/1925004215 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f48bc0468a0 con 0x559427a0f1b0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48a2ffd640 1 -- 172.21.15.115:0/1925004215 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f48bc046aa0 con 0x559427a0f1b0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48d0a1d740 10 client.15561 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48d0a1d740 1 -- 172.21.15.115:0/1925004215 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559427e36f50 con 0x559427a0f1b0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48a2ffd640 1 -- 172.21.15.115:0/1925004215 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48b40a9300 con 0x559427a0f1b0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48a2ffd640 1 -- 172.21.15.115:0/1925004215 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f48bc044040 con 0x559427a0f1b0 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48d0a1d740 10 client.15561 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48d0a1d740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48d0a1d740 20 client.15561 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594279a31c0 0x559427e396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48d0a1d740 4 client.15561 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:38:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.821+0000 7f48d0a1d740 1 -- 172.21.15.115:0/1925004215 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559427d1f240 con 0x5594279a31c0 2024-05-09T05:38:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.822+0000 7f48c37fe640 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594279a31c0 0x559427e396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.822+0000 7f48a2ffd640 10 client.15561 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.822+0000 7f48c37fe640 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594279a31c0 0x559427e396a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b1a1e40332d8e5 server_cookie=bab5de4b8955b108 in_seq=0 out_seq=0 2024-05-09T05:38:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48a2ffd640 1 -- 172.21.15.115:0/1925004215 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559427d1f240 con 0x5594279a31c0 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48a2ffd640 10 client.15561 handle_command_reply: tid=0 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 client.15561 shutdown 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 -- 172.21.15.115:0/1925004215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 msgr2=0x559427dc0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dc0810 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009bf0 tx=0x7f48bc0467b0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 -- 172.21.15.115:0/1925004215 shutdown_connections 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559427a0f1b0 0x559427dc0810 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594279ee4d0 0x559427dc02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594279a31c0 0x559427e396a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 --2- 172.21.15.115:0/1925004215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594279e62f0 0x559427dc4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48a2ffd640 0 client.15561 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 -- 172.21.15.115:0/1925004215 >> 172.21.15.115:0/1925004215 conn(0x559427a03300 msgr2=0x559427dbe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 -- 172.21.15.115:0/1925004215 shutdown_connections 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 1 -- 172.21.15.115:0/1925004215 wait complete. 2024-05-09T05:38:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.835+0000 7f48d0a1d740 20 client.15561 trim_cache size 0 max 16384 2024-05-09T05:38:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:38:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:38:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 -- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48ac0b29e0 msgr2=0x7f48ac0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48ac0b29e0 0x7f48ac0b4ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x559427daf7f0 tx=0x7f48bc009430 comp rx=0 tx=0).stop 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 -- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4102460 msgr2=0x7f48c419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4102460 0x7f48c419a790 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f48b400d950 tx=0x7f48b400de10 comp rx=0 tx=0).stop 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 -- 172.21.15.115:0/2701874925 shutdown_connections 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48ac0b29e0 0x7f48ac0b4ef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c4108f80 0x7f48c419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c41084e0 0x7f48c419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 --2- 172.21.15.115:0/2701874925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c4102460 0x7f48c419a790 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 -- 172.21.15.115:0/2701874925 >> 172.21.15.115:0/2701874925 conn(0x7f48c40fd9d0 msgr2=0x7f48c41066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 -- 172.21.15.115:0/2701874925 shutdown_connections 2024-05-09T05:38:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:39.836+0000 7f48c9b6b640 1 -- 172.21.15.115:0/2701874925 wait complete. 2024-05-09T05:38:40.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.392+0000 7fa959abb700 1 -- 172.21.15.115:0/1717355449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 msgr2=0x7fa954110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.392+0000 7fa959abb700 1 --2- 172.21.15.115:0/1717355449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 0x7fa954110e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00b0a0 tx=0x7fa94c00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.392+0000 7fa959abb700 1 -- 172.21.15.115:0/1717355449 shutdown_connections 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.392+0000 7fa959abb700 1 --2- 172.21.15.115:0/1717355449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 0x7fa954110e10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.392+0000 7fa959abb700 1 --2- 172.21.15.115:0/1717355449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954105550 0x7fa954105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.392+0000 7fa959abb700 1 --2- 172.21.15.115:0/1717355449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa954105ea0 0x7fa95410e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.392+0000 7fa959abb700 1 -- 172.21.15.115:0/1717355449 >> 172.21.15.115:0/1717355449 conn(0x7fa954078e00 msgr2=0x7fa954079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.392+0000 7fa959abb700 1 -- 172.21.15.115:0/1717355449 shutdown_connections 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.392+0000 7fa959abb700 1 -- 172.21.15.115:0/1717355449 wait complete. 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.393+0000 7fa959abb700 1 Processor -- start 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.393+0000 7fa959abb700 1 -- start start 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.393+0000 7fa959abb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954105550 0x7fa9541a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.393+0000 7fa959abb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa954105ea0 0x7fa9541a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.393+0000 7fa959abb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 0x7fa95419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.393+0000 7fa959abb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa954114330 con 0x7fa95410e930 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.393+0000 7fa959abb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541141b0 con 0x7fa954105ea0 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa959abb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541144b0 con 0x7fa954105550 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa953fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 0x7fa95419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa953fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 0x7fa95419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52514/0 (socket says 172.21.15.115:52514) 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa953fff700 1 -- 172.21.15.115:0/1689138575 learned_addr learned my addr 172.21.15.115:0/1689138575 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa952ffd700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa954105ea0 0x7fa9541a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa9537fe700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954105550 0x7fa9541a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa953fff700 1 -- 172.21.15.115:0/1689138575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954105550 msgr2=0x7fa9541a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa953fff700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954105550 0x7fa9541a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa953fff700 1 -- 172.21.15.115:0/1689138575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa954105ea0 msgr2=0x7fa9541a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa953fff700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa954105ea0 0x7fa9541a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa953fff700 1 -- 172.21.15.115:0/1689138575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95419b160 con 0x7fa95410e930 2024-05-09T05:38:40.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa953fff700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 0x7fa95419aaf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00b830 tx=0x7fa94c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa950ff9700 1 -- 172.21.15.115:0/1689138575 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c021070 con 0x7fa95410e930 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa950ff9700 1 -- 172.21.15.115:0/1689138575 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa94c007b80 con 0x7fa95410e930 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa950ff9700 1 -- 172.21.15.115:0/1689138575 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0052a0 con 0x7fa95410e930 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95419b3f0 con 0x7fa95410e930 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.394+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95419b920 con 0x7fa95410e930 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.396+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9540512c0 con 0x7fa95410e930 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.397+0000 7fa950ff9700 1 -- 172.21.15.115:0/1689138575 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fa94c024070 con 0x7fa95410e930 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.398+0000 7fa950ff9700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9400b2b00 0x7fa9400b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.398+0000 7fa9537fe700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9400b2b00 0x7fa9400b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.398+0000 7fa950ff9700 1 -- 172.21.15.115:0/1689138575 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa94c00f020 con 0x7fa95410e930 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.398+0000 7fa9537fe700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9400b2b00 0x7fa9400b5010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa944005ea0 tx=0x7fa944005e30 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:40.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.402+0000 7fa950ff9700 1 -- 172.21.15.115:0/1689138575 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa95419b920 con 0x7fa95410e930 2024-05-09T05:38:40.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.651+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa95419bfe0 con 0x7fa95410e930 2024-05-09T05:38:40.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.652+0000 7fa950ff9700 1 -- 172.21.15.115:0/1689138575 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7fa94c02b020 con 0x7fa95410e930 2024-05-09T05:38:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:40.646 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.654+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9400b2b00 msgr2=0x7fa9400b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.654+0000 7fa959abb700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9400b2b00 0x7fa9400b5010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa944005ea0 tx=0x7fa944005e30 comp rx=0 tx=0).stop 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.654+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 msgr2=0x7fa95419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.654+0000 7fa959abb700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 0x7fa95419aaf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00b830 tx=0x7fa94c004d00 comp rx=0 tx=0).stop 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.655+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 shutdown_connections 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.655+0000 7fa959abb700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9400b2b00 0x7fa9400b5010 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.655+0000 7fa959abb700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa95410e930 0x7fa95419aaf0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.655+0000 7fa959abb700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa954105550 0x7fa9541a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.655+0000 7fa959abb700 1 --2- 172.21.15.115:0/1689138575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa954105ea0 0x7fa9541a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.655+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 >> 172.21.15.115:0/1689138575 conn(0x7fa954078e00 msgr2=0x7fa9541000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.655+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 shutdown_connections 2024-05-09T05:38:40.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:40.655+0000 7fa959abb700 1 -- 172.21.15.115:0/1689138575 wait complete. 2024-05-09T05:38:40.650 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:38:41.467 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:38:41.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:41 smithi195 ceph-mon[33844]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-09T05:38:41.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:41 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1689138575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:41.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:41 smithi115 ceph-mon[30349]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-09T05:38:41.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:41 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1689138575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:41.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:41 smithi161 ceph-mon[33805]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-09T05:38:41.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:41 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1689138575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:41.713 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:43.562 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:43 smithi115 ceph-mon[30349]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:38:43.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.581+0000 7fd0a15f2700 1 -- 172.21.15.115:0/2779603823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c109300 msgr2=0x7fd09c109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:43.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.581+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/2779603823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c109300 0x7fd09c109710 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009990 tx=0x7fd08c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.581+0000 7fd0a15f2700 1 -- 172.21.15.115:0/2779603823 shutdown_connections 2024-05-09T05:38:43.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.581+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/2779603823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd09c1031d0 0x7fd09c111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.581+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/2779603823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c109300 0x7fd09c109710 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.581+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/2779603823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd09c102880 0x7fd09c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.581+0000 7fd0a15f2700 1 -- 172.21.15.115:0/2779603823 >> 172.21.15.115:0/2779603823 conn(0x7fd09c0fe6f0 msgr2=0x7fd09c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:43.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.581+0000 7fd0a15f2700 1 -- 172.21.15.115:0/2779603823 shutdown_connections 2024-05-09T05:38:43.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.581+0000 7fd0a15f2700 1 -- 172.21.15.115:0/2779603823 wait complete. 2024-05-09T05:38:43.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.582+0000 7fd0a15f2700 1 Processor -- start 2024-05-09T05:38:43.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.582+0000 7fd0a15f2700 1 -- start start 2024-05-09T05:38:43.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.582+0000 7fd0a15f2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c102880 0x7fd09c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:43.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd0a15f2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd09c1031d0 0x7fd09c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:43.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd0a15f2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd09c109300 0x7fd09c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:43.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd0a15f2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c114440 con 0x7fd09c102880 2024-05-09T05:38:43.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd0a15f2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1142c0 con 0x7fd09c1031d0 2024-05-09T05:38:43.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd0a15f2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1145c0 con 0x7fd09c109300 2024-05-09T05:38:43.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c102880 0x7fd09c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c102880 0x7fd09c19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53964/0 (socket says 172.21.15.115:53964) 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09affd700 1 -- 172.21.15.115:0/4215991138 learned_addr learned my addr 172.21.15.115:0/4215991138 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09b7fe700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd09c109300 0x7fd09c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd0927fc700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd09c1031d0 0x7fd09c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09affd700 1 -- 172.21.15.115:0/4215991138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd09c109300 msgr2=0x7fd09c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09affd700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd09c109300 0x7fd09c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09affd700 1 -- 172.21.15.115:0/4215991138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd09c1031d0 msgr2=0x7fd09c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09affd700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd09c1031d0 0x7fd09c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09affd700 1 -- 172.21.15.115:0/4215991138 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c19fc40 con 0x7fd09c102880 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd0927fc700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd09c1031d0 0x7fd09c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd09affd700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c102880 0x7fd09c19ae70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd08c006010 tx=0x7fd08c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd098ff9700 1 -- 172.21.15.115:0/4215991138 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c027070 con 0x7fd09c102880 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd098ff9700 1 -- 172.21.15.115:0/4215991138 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd08c00fbc0 con 0x7fd09c102880 2024-05-09T05:38:43.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09c19fed0 con 0x7fd09c102880 2024-05-09T05:38:43.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.583+0000 7fd098ff9700 1 -- 172.21.15.115:0/4215991138 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c022b30 con 0x7fd09c102880 2024-05-09T05:38:43.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.584+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd09c1a03b0 con 0x7fd09c102880 2024-05-09T05:38:43.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.584+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd09c10c8a0 con 0x7fd09c102880 2024-05-09T05:38:43.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.586+0000 7fd098ff9700 1 -- 172.21.15.115:0/4215991138 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fd08c02a070 con 0x7fd09c102880 2024-05-09T05:38:43.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.587+0000 7fd098ff9700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd0880b2a80 0x7fd0880b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:43.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.587+0000 7fd0927fc700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd0880b2a80 0x7fd0880b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:43.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.587+0000 7fd098ff9700 1 -- 172.21.15.115:0/4215991138 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd08c0e42b0 con 0x7fd09c102880 2024-05-09T05:38:43.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.587+0000 7fd0927fc700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd0880b2a80 0x7fd0880b4f90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd084005fd0 tx=0x7fd084005e20 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:43.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.590+0000 7fd098ff9700 1 -- 172.21.15.115:0/4215991138 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd08c031020 con 0x7fd09c102880 2024-05-09T05:38:43.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:43 smithi195 ceph-mon[33844]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:38:43.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:43 smithi161 ceph-mon[33805]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:38:43.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.841+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd09c0512c0 con 0x7fd09c102880 2024-05-09T05:38:43.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.842+0000 7fd098ff9700 1 -- 172.21.15.115:0/4215991138 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7fd08c013070 con 0x7fd09c102880 2024-05-09T05:38:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:43.837 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:38:43.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd0880b2a80 msgr2=0x7fd0880b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:43.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd0880b2a80 0x7fd0880b4f90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd084005fd0 tx=0x7fd084005e20 comp rx=0 tx=0).stop 2024-05-09T05:38:43.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c102880 msgr2=0x7fd09c19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:43.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c102880 0x7fd09c19ae70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd08c006010 tx=0x7fd08c0047b0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 shutdown_connections 2024-05-09T05:38:43.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd0880b2a80 0x7fd0880b4f90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd09c109300 0x7fd09c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd09c102880 0x7fd09c19ae70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 --2- 172.21.15.115:0/4215991138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd09c1031d0 0x7fd09c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 >> 172.21.15.115:0/4215991138 conn(0x7fd09c0fe6f0 msgr2=0x7fd09c10a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 shutdown_connections 2024-05-09T05:38:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:43.845+0000 7fd0a15f2700 1 -- 172.21.15.115:0/4215991138 wait complete. 2024-05-09T05:38:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:38:44.474 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:44 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/4215991138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:44.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:44 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/4215991138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:44.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:44 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/4215991138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:44.678 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:38:44.921 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:45 smithi195 ceph-mon[33844]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T05:38:45.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:45 smithi115 ceph-mon[30349]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T05:38:45.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:45 smithi161 ceph-mon[33805]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.637+0000 7f91901e1700 1 -- 172.21.15.115:0/841318472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 msgr2=0x7f9188108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.637+0000 7f91901e1700 1 --2- 172.21.15.115:0/841318472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 0x7f9188108430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f91780099c0 tx=0x7f9178009cd0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.637+0000 7f91901e1700 1 -- 172.21.15.115:0/841318472 shutdown_connections 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.637+0000 7f91901e1700 1 --2- 172.21.15.115:0/841318472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9188108970 0x7f91881102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.637+0000 7f91901e1700 1 --2- 172.21.15.115:0/841318472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9188103c10 0x7f9188104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.637+0000 7f91901e1700 1 --2- 172.21.15.115:0/841318472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 0x7f9188108430 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.637+0000 7f91901e1700 1 -- 172.21.15.115:0/841318472 >> 172.21.15.115:0/841318472 conn(0x7f91880fe520 msgr2=0x7f9188100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.637+0000 7f91901e1700 1 -- 172.21.15.115:0/841318472 shutdown_connections 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.637+0000 7f91901e1700 1 -- 172.21.15.115:0/841318472 wait complete. 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f91901e1700 1 Processor -- start 2024-05-09T05:38:46.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f91901e1700 1 -- start start 2024-05-09T05:38:46.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f91901e1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9188103c10 0x7f91880fea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:46.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f91901e1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 0x7f91880fef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:46.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f91901e1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9188108970 0x7f91880ff480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:46.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f91901e1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9188113430 con 0x7f9188108020 2024-05-09T05:38:46.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f91901e1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f91881132b0 con 0x7f9188108970 2024-05-09T05:38:46.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f91901e1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91881135b0 con 0x7f9188103c10 2024-05-09T05:38:46.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f918d77c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 0x7f91880fef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f918d77c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 0x7f91880fef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53984/0 (socket says 172.21.15.115:53984) 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f918d77c700 1 -- 172.21.15.115:0/3276455596 learned_addr learned my addr 172.21.15.115:0/3276455596 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f918df7d700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9188103c10 0x7f91880fea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.638+0000 7f918e77e700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9188108970 0x7f91880ff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f918d77c700 1 -- 172.21.15.115:0/3276455596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9188103c10 msgr2=0x7f91880fea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f918d77c700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9188103c10 0x7f91880fea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f918d77c700 1 -- 172.21.15.115:0/3276455596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9188108970 msgr2=0x7f91880ff480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f918d77c700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9188108970 0x7f91880ff480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f918d77c700 1 -- 172.21.15.115:0/3276455596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91881a93a0 con 0x7f9188108020 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f918d77c700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 0x7f91880fef40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f917800b580 tx=0x7f9178004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f917effd700 1 -- 172.21.15.115:0/3276455596 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917800f8a0 con 0x7f9188108020 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f917effd700 1 -- 172.21.15.115:0/3276455596 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9178004830 con 0x7f9188108020 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91881a9630 con 0x7f9188108020 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f917effd700 1 -- 172.21.15.115:0/3276455596 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178005350 con 0x7f9188108020 2024-05-09T05:38:46.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.639+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91881a9a80 con 0x7f9188108020 2024-05-09T05:38:46.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.640+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91880512c0 con 0x7f9188108020 2024-05-09T05:38:46.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.642+0000 7f917effd700 1 -- 172.21.15.115:0/3276455596 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f9178029060 con 0x7f9188108020 2024-05-09T05:38:46.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.642+0000 7f917effd700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f91740b2b00 0x7f91740b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:46.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.642+0000 7f918df7d700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f91740b2b00 0x7f91740b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:46.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.642+0000 7f917effd700 1 -- 172.21.15.115:0/3276455596 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f91780e38a0 con 0x7f9188108020 2024-05-09T05:38:46.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.643+0000 7f918df7d700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f91740b2b00 0x7f91740b5010 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9184005950 tx=0x7f918400a300 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:46.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.646+0000 7f917effd700 1 -- 172.21.15.115:0/3276455596 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9178013070 con 0x7f9188108020 2024-05-09T05:38:46.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.851+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f918806c0c0 con 0x7f9188108020 2024-05-09T05:38:46.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.853+0000 7f917effd700 1 -- 172.21.15.115:0/3276455596 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25107 (secure 0 0 0) 0x7f91780aae60 con 0x7f9188108020 2024-05-09T05:38:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:46.848 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":88,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T05:38:11.918691+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1352570099":"2024-05-10T05:38:11.918655+0000","172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:6800/2041699318":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/617068092":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/2041699318":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/4237548909":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/1853670702":"2024-05-10T05:38:11.918655+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:38:46.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f91740b2b00 msgr2=0x7f91740b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f91740b2b00 0x7f91740b5010 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9184005950 tx=0x7f918400a300 comp rx=0 tx=0).stop 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 msgr2=0x7f91880fef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 0x7f91880fef40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f917800b580 tx=0x7f9178004790 comp rx=0 tx=0).stop 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 shutdown_connections 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f91740b2b00 0x7f91740b5010 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9188108970 0x7f91880ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9188103c10 0x7f91880fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 --2- 172.21.15.115:0/3276455596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9188108020 0x7f91880fef40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 >> 172.21.15.115:0/3276455596 conn(0x7f91880fe520 msgr2=0x7f9188102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 shutdown_connections 2024-05-09T05:38:46.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:46.855+0000 7f91901e1700 1 -- 172.21.15.115:0/3276455596 wait complete. 2024-05-09T05:38:47.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:47 smithi195 ceph-mon[33844]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T05:38:47.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:47 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3276455596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:38:47.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:47 smithi115 ceph-mon[30349]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T05:38:47.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:47 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3276455596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:38:47.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:47 smithi161 ceph-mon[33805]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T05:38:47.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:47 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3276455596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:38:47.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-09T05:38:47.942 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:49.501 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:49 smithi115 ceph-mon[30349]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T05:38:49.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:49 smithi195 ceph-mon[33844]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T05:38:49.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:49 smithi161 ceph-mon[33805]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T05:38:50.592 INFO:teuthology.orchestra.run.smithi115.stdout:{"run_dir":"/var/run/ceph"} 2024-05-09T05:38:50.592 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-09T05:38:50.592 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1722 2024-05-09T05:38:50.593 DEBUG:tasks.quiescer:Yielding 2024-05-09T05:38:50.593 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T05:38:50.609 INFO:tasks.workunit:Pulling workunits from ref c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:38:50.610 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T05:38:50.610 INFO:tasks.workunit:timeout=3h 2024-05-09T05:38:50.610 INFO:tasks.workunit:cleanup=True 2024-05-09T05:38:50.610 DEBUG:teuthology.orchestra.run.smithi115:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:50.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:38:50.627 INFO:teuthology.orchestra.run.smithi115.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:50.627 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-09T05:38:50.627 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-05-09T05:38:50.627 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T05:38:50.627 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-09T05:38:50.627 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-05-09 05:35:11.386670092 +0000 2024-05-09T05:38:50.627 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-05-09 05:35:11.386670092 +0000 2024-05-09T05:38:50.628 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-05-09 05:38:10.494849597 +0000 2024-05-09T05:38:50.628 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-05-09T05:38:50.628 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T05:38:50.628 DEBUG:teuthology.orchestra.run.smithi115:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T05:38:50.700 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T05:38:50.756 INFO:tasks.workunit.client.0.smithi115.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T05:38:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:51.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:51 smithi115 ceph-mon[30349]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T05:38:51.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:51 smithi161 ceph-mon[33805]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T05:38:51.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:51 smithi195 ceph-mon[33844]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-05-09T05:38:52.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.715+0000 7f3690fbc700 1 -- 172.21.15.115:0/848863694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c108f20 msgr2=0x7f368c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.715+0000 7f3690fbc700 1 --2- 172.21.15.115:0/848863694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c108f20 0x7f368c109330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f36740099c0 tx=0x7f3674009cd0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.715+0000 7f3690fbc700 1 -- 172.21.15.115:0/848863694 shutdown_connections 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.715+0000 7f3690fbc700 1 --2- 172.21.15.115:0/848863694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f368c109870 0x7f368c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.715+0000 7f3690fbc700 1 --2- 172.21.15.115:0/848863694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f368c1045f0 0x7f368c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.715+0000 7f3690fbc700 1 --2- 172.21.15.115:0/848863694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c108f20 0x7f368c109330 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.715+0000 7f3690fbc700 1 -- 172.21.15.115:0/848863694 >> 172.21.15.115:0/848863694 conn(0x7f368c100040 msgr2=0x7f368c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.715+0000 7f3690fbc700 1 -- 172.21.15.115:0/848863694 shutdown_connections 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.715+0000 7f3690fbc700 1 -- 172.21.15.115:0/848863694 wait complete. 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f3690fbc700 1 Processor -- start 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f3690fbc700 1 -- start start 2024-05-09T05:38:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f3690fbc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c1045f0 0x7f368c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f3690fbc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f368c108f20 0x7f368c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f3690fbc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f368c109870 0x7f368c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f3690fbc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f368c114330 con 0x7f368c1045f0 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f3690fbc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1141b0 con 0x7f368c109870 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f3690fbc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1144b0 con 0x7f368c108f20 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f368a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c1045f0 0x7f368c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f368a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c1045f0 0x7f368c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42340/0 (socket says 172.21.15.115:42340) 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f368a59c700 1 -- 172.21.15.115:0/438276061 learned_addr learned my addr 172.21.15.115:0/438276061 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f368ad9d700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f368c109870 0x7f368c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f3689d9b700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f368c108f20 0x7f368c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f368a59c700 1 -- 172.21.15.115:0/438276061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f368c108f20 msgr2=0x7f368c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f368a59c700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f368c108f20 0x7f368c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f368a59c700 1 -- 172.21.15.115:0/438276061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f368c109870 msgr2=0x7f368c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f368a59c700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f368c109870 0x7f368c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.716+0000 7f368a59c700 1 -- 172.21.15.115:0/438276061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c19fc70 con 0x7f368c1045f0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.717+0000 7f368a59c700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c1045f0 0x7f368c19ada0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f367c00e9a0 tx=0x7f367c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.717+0000 7f36837fe700 1 -- 172.21.15.115:0/438276061 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c00cbd0 con 0x7f368c1045f0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.717+0000 7f36837fe700 1 -- 172.21.15.115:0/438276061 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f367c00cd70 con 0x7f368c1045f0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.717+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c19ff60 con 0x7f368c1045f0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.717+0000 7f36837fe700 1 -- 172.21.15.115:0/438276061 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c018680 con 0x7f368c1045f0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.717+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c1a7880 con 0x7f368c1045f0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.718+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c105aa0 con 0x7f368c1045f0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.720+0000 7f36837fe700 1 -- 172.21.15.115:0/438276061 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f367c010430 con 0x7f368c1045f0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.721+0000 7f36837fe700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36780b2a80 0x7f36780b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.721+0000 7f3689d9b700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36780b2a80 0x7f36780b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.721+0000 7f36837fe700 1 -- 172.21.15.115:0/438276061 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f367c0e2ff0 con 0x7f368c1045f0 2024-05-09T05:38:52.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.721+0000 7f3689d9b700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36780b2a80 0x7f36780b4f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f367401a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:52.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.725+0000 7f36837fe700 1 -- 172.21.15.115:0/438276061 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f367c0aa630 con 0x7f368c1045f0 2024-05-09T05:38:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.937+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2636046681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38109060 msgr2=0x7f6e38102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.937+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2636046681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38109060 0x7f6e38102ca0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009900 tx=0x7f6e2002a410 comp rx=0 tx=0).stop 2024-05-09T05:38:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.937+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2636046681 shutdown_connections 2024-05-09T05:38:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.937+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2636046681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e38103310 0x7f6e3810f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.937+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2636046681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38109060 0x7f6e38102ca0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.937+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2636046681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e38108bc0 0x7f6e38102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.937+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2636046681 >> 172.21.15.115:0/2636046681 conn(0x7f6e380781b0 msgr2=0x7f6e38078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.937+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2636046681 shutdown_connections 2024-05-09T05:38:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.937+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2636046681 wait complete. 2024-05-09T05:38:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e3e2e0640 1 Processor -- start 2024-05-09T05:38:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e3e2e0640 1 -- start start 2024-05-09T05:38:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e3e2e0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e38103310 0x7f6e3819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e3e2e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38108bc0 0x7f6e3819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e3e2e0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e38109060 0x7f6e3819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e3e2e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6e38112c60 con 0x7f6e38108bc0 2024-05-09T05:38:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e3e2e0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e38112ae0 con 0x7f6e38109060 2024-05-09T05:38:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e3e2e0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6e38112de0 con 0x7f6e38103310 2024-05-09T05:38:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e2ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38108bc0 0x7f6e3819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e2ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38108bc0 0x7f6e3819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42352/0 (socket says 172.21.15.115:42352) 2024-05-09T05:38:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e2ffff640 1 -- 172.21.15.115:0/2591888925 learned_addr learned my addr 172.21.15.115:0/2591888925 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e3c87d640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e38109060 0x7f6e3819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e37fff640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e38103310 0x7f6e3819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e2ffff640 1 -- 172.21.15.115:0/2591888925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e38103310 msgr2=0x7f6e3819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e38103310 0x7f6e3819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e2ffff640 1 -- 172.21.15.115:0/2591888925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e38109060 msgr2=0x7f6e3819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e38109060 0x7f6e3819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.938+0000 7f6e2ffff640 1 -- 172.21.15.115:0/2591888925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3819f5d0 con 0x7f6e38108bc0 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.939+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38108bc0 0x7f6e3819ad50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009230 tx=0x7f6e200040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.939+0000 7f6e35ffb640 1 -- 172.21.15.115:0/2591888925 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2003f030 con 0x7f6e38108bc0 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.939+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2591888925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3819f860 con 0x7f6e38108bc0 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.939+0000 7f6e35ffb640 1 -- 172.21.15.115:0/2591888925 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e20003a80 con 0x7f6e38108bc0 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.939+0000 7f6e35ffb640 1 -- 172.21.15.115:0/2591888925 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20035070 con 0x7f6e38108bc0 2024-05-09T05:38:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.939+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2591888925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3819fda0 con 0x7f6e38108bc0 2024-05-09T05:38:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.941+0000 7f6e4518b740 1 Processor -- start 2024-05-09T05:38:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e4518b740 1 -- start start 2024-05-09T05:38:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ef81b0 0x5645f929b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ecf2f0 0x5645f929bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 0x5645f929c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e4518b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5645f92007c0 con 0x5645f8ed74d0 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e4518b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5645f919c0e0 con 0x5645f8ef81b0 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e4518b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5645f91fc780 con 0x5645f8ecf2f0 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e3c87d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ef81b0 0x5645f929b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e3c87d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ef81b0 0x5645f929b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:38594/0 (socket says 172.21.15.115:38594) 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e3c87d640 1 -- 172.21.15.115:0/451819733 learned_addr learned my addr 172.21.15.115:0/451819733 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e37fff640 1 --2- 172.21.15.115:0/451819733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ecf2f0 0x5645f929bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e3c87d640 1 -- 172.21.15.115:0/451819733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ecf2f0 msgr2=0x5645f929bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e3c87d640 1 --2- 172.21.15.115:0/451819733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ecf2f0 0x5645f929bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e3c87d640 1 -- 172.21.15.115:0/451819733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 msgr2=0x5645f929c100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e3c87d640 1 --2- 172.21.15.115:0/451819733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 0x5645f929c100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.942+0000 7f6e3c87d640 1 -- 172.21.15.115:0/451819733 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f91c57c0 con 0x5645f8ef81b0 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.943+0000 7f6e3c87d640 1 --2- 172.21.15.115:0/451819733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ef81b0 0x5645f929b680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6e30007f40 tx=0x7f6e3000f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e815c1ce388ce82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.943+0000 7f6e35ffb640 1 -- 172.21.15.115:0/2591888925 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f6e20042030 con 0x7f6e38108bc0 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.943+0000 7f6e2e7fc640 1 -- 172.21.15.115:0/451819733 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30010900 con 0x5645f8ef81b0 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.943+0000 7f6e2e7fc640 1 -- 172.21.15.115:0/451819733 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e300183f0 con 0x5645f8ef81b0 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.943+0000 7f6e2e7fc640 1 -- 172.21.15.115:0/451819733 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30017600 con 0x5645f8ef81b0 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e35ffb640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6e240b29e0 0x7f6e240b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e37fff640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6e240b29e0 0x7f6e240b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e35ffb640 1 -- 172.21.15.115:0/2591888925 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6e20103010 con 0x7f6e38108bc0 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e4518b740 1 -- 172.21.15.115:0/451819733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ef81b0 msgr2=0x5645f929b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e4518b740 1 --2- 172.21.15.115:0/451819733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ef81b0 0x5645f929b680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6e30007f40 tx=0x7f6e3000f850 comp rx=0 tx=0).stop 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e4518b740 1 -- 172.21.15.115:0/451819733 shutdown_connections 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e4518b740 1 --2- 172.21.15.115:0/451819733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 0x5645f929c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e4518b740 1 --2- 172.21.15.115:0/451819733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ecf2f0 0x5645f929bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e4518b740 1 --2- 172.21.15.115:0/451819733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ef81b0 0x5645f929b680 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e4518b740 1 -- 172.21.15.115:0/451819733 >> 172.21.15.115:0/451819733 conn(0x5645f8eec300 msgr2=0x5645f9297e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e4518b740 1 -- 172.21.15.115:0/451819733 shutdown_connections 2024-05-09T05:38:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e4518b740 1 -- 172.21.15.115:0/451819733 wait complete. 2024-05-09T05:38:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.944+0000 7f6e37fff640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6e240b29e0 0x7f6e240b4ef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5645f929ca30 tx=0x7f6e2800b4d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e4518b740 1 -- start start 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e4518b740 1 Processor -- start 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92a7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ed74d0 0x5645f92a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ef81b0 0x5645f92a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e4518b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5645f92a87b0 con 0x5645f8ecf2f0 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e4518b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5645f91879c0 con 0x5645f8ed74d0 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e4518b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5645f92a8930 con 0x5645f8ef81b0 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e4518b740 10 client.0 _ll_register_callbacks cb 0x5645f91b4020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.945+0000 7f6e37fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e37fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92a7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42354/0 (socket says 172.21.15.115:42354) 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e37fff640 1 -- 172.21.15.115:0/2945243052 learned_addr learned my addr 172.21.15.115:0/2945243052 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ed74d0 0x5645f92a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e3c87d640 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ef81b0 0x5645f92a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e37fff640 1 -- 172.21.15.115:0/2945243052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ef81b0 msgr2=0x5645f92a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e37fff640 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ef81b0 0x5645f92a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e37fff640 1 -- 172.21.15.115:0/2945243052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ed74d0 msgr2=0x5645f92a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e37fff640 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ed74d0 0x5645f92a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e37fff640 1 -- 172.21.15.115:0/2945243052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f91ae1f0 con 0x5645f8ecf2f0 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e37fff640 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92a7810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6e2801d620 tx=0x7f6e280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930038a91b113e2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e177fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e177fe640 1 -- 172.21.15.115:0/2945243052 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28029680 con 0x5645f8ecf2f0 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e177fe640 1 -- 172.21.15.115:0/2945243052 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6e28029820 con 0x5645f8ecf2f0 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e4518b740 1 -- 172.21.15.115:0/2945243052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e300183f0 con 0x5645f8ecf2f0 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e177fe640 1 -- 172.21.15.115:0/2945243052 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6e28029a20 con 0x5645f8ecf2f0 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e4518b740 10 client.15594 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.946+0000 7f6e4518b740 1 -- 172.21.15.115:0/2945243052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645f8f63790 con 0x5645f8ecf2f0 2024-05-09T05:38:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.947+0000 7f6e177fe640 1 -- 172.21.15.115:0/2945243052 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e300183f0 con 0x5645f8ecf2f0 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.947+0000 7f6e177fe640 1 -- 172.21.15.115:0/2945243052 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6e2801ec10 con 0x5645f8ecf2f0 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.947+0000 7f6e4518b740 10 client.15594 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.947+0000 7f6e4518b740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.947+0000 7f6e4518b740 20 client.15594 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.947+0000 7f6e4518b740 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645f8eb6d80 0x5645f92b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.947+0000 7f6e4518b740 4 client.15594 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.947+0000 7f6e4518b740 1 -- 172.21.15.115:0/2945243052 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645f920e3d0 con 0x5645f8eb6d80 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.947+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645f8eb6d80 0x5645f92b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.948+0000 7f6e177fe640 10 client.15594 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.948+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645f8eb6d80 0x5645f92b8110 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466c1ed5af09790c server_cookie=bd6fae3f6875ac5c in_seq=0 out_seq=0 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.948+0000 7f6e177fe640 1 -- 172.21.15.115:0/2945243052 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645f920e3d0 con 0x5645f8eb6d80 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.948+0000 7f6e177fe640 10 client.15594 handle_command_reply: tid=0 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 client.15594 shutdown 2024-05-09T05:38:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 -- 172.21.15.115:0/2945243052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 msgr2=0x5645f92a7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92a7810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6e2801d620 tx=0x7f6e280046c0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 -- 172.21.15.115:0/2945243052 shutdown_connections 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645f8eb6d80 0x5645f92b8110 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ef81b0 0x5645f92a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e177fe640 0 client.15594 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ed74d0 0x5645f92a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 --2- 172.21.15.115:0/2945243052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92a7810 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 -- 172.21.15.115:0/2945243052 >> 172.21.15.115:0/2945243052 conn(0x5645f8eec300 msgr2=0x5645f9296700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 -- 172.21.15.115:0/2945243052 shutdown_connections 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 1 -- 172.21.15.115:0/2945243052 wait complete. 2024-05-09T05:38:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.949+0000 7f6e4518b740 20 client.15594 trim_cache size 0 max 16384 2024-05-09T05:38:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e4518b740 1 Processor -- start 2024-05-09T05:38:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e4518b740 1 -- start start 2024-05-09T05:38:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8eb6d80 0x5645f9296e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ed74d0 0x5645f92bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e4518b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381b1050 con 0x5645f8ecf2f0 2024-05-09T05:38:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e4518b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381b2b90 con 0x5645f8eb6d80 2024-05-09T05:38:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e4518b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381b1720 con 0x5645f8ed74d0 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e2ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e2ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92bd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42358/0 (socket says 172.21.15.115:42358) 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e2ffff640 1 -- 172.21.15.115:0/1965083196 learned_addr learned my addr 172.21.15.115:0/1965083196 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e37fff640 1 --2- 172.21.15.115:0/1965083196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8eb6d80 0x5645f9296e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e2ffff640 1 -- 172.21.15.115:0/1965083196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ed74d0 msgr2=0x5645f92bde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/1965083196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ed74d0 0x5645f92bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e2ffff640 1 -- 172.21.15.115:0/1965083196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8eb6d80 msgr2=0x5645f9296e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/1965083196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8eb6d80 0x5645f9296e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.965+0000 7f6e2ffff640 1 -- 172.21.15.115:0/1965083196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f8fda5f0 con 0x5645f8ecf2f0 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.966+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/1965083196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92bd910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6e200c9b40 tx=0x7f6e2003b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7134078758e64008 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.966+0000 7f6e2dffb640 1 -- 172.21.15.115:0/1965083196 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200cc040 con 0x5645f8ecf2f0 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.966+0000 7f6e2dffb640 1 -- 172.21.15.115:0/1965083196 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e200ca540 con 0x5645f8ecf2f0 2024-05-09T05:38:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.966+0000 7f6e2dffb640 1 -- 172.21.15.115:0/1965083196 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20049040 con 0x5645f8ecf2f0 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 -- 172.21.15.115:0/1965083196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 msgr2=0x5645f92bd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 --2- 172.21.15.115:0/1965083196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92bd910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6e200c9b40 tx=0x7f6e2003b460 comp rx=0 tx=0).stop 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 -- 172.21.15.115:0/1965083196 shutdown_connections 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 --2- 172.21.15.115:0/1965083196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8ed74d0 0x5645f92bde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 --2- 172.21.15.115:0/1965083196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ecf2f0 0x5645f92bd910 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 --2- 172.21.15.115:0/1965083196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8eb6d80 0x5645f9296e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 -- 172.21.15.115:0/1965083196 >> 172.21.15.115:0/1965083196 conn(0x5645f8eec300 msgr2=0x5645f9296700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 -- 172.21.15.115:0/1965083196 shutdown_connections 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 -- 172.21.15.115:0/1965083196 wait complete. 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 -- start start 2024-05-09T05:38:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.967+0000 7f6e4518b740 1 Processor -- start 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8eb6d80 0x5645f92ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ecf2f0 0x5645f92ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e4518b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 0x5645f92ab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e4518b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381b24c0 con 0x5645f8ed74d0 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e4518b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381b1df0 con 0x5645f8ecf2f0 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e4518b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6e381af030 con 0x5645f8eb6d80 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e4518b740 10 client.0 _ll_register_callbacks cb 0x7f6e2800b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e3c87d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 0x5645f92ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e3c87d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 0x5645f92ab980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42372/0 (socket says 172.21.15.115:42372) 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e3c87d640 1 -- 172.21.15.115:0/4123155073 learned_addr learned my addr 172.21.15.115:0/4123155073 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e37fff640 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8eb6d80 0x5645f92ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ecf2f0 0x5645f92ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e2ffff640 1 -- 172.21.15.115:0/4123155073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8eb6d80 msgr2=0x5645f92ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8eb6d80 0x5645f92ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e2ffff640 1 -- 172.21.15.115:0/4123155073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 msgr2=0x5645f92ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 0x5645f92ab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e2ffff640 1 -- 172.21.15.115:0/4123155073 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f91c4ed0 con 0x5645f8ecf2f0 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.968+0000 7f6e2ffff640 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ecf2f0 0x5645f92ab440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5645f920e4a0 tx=0x7f6e200d2e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d16d96319d920c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e177fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e177fe640 1 -- 172.21.15.115:0/4123155073 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20049050 con 0x5645f8ecf2f0 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e177fe640 1 -- 172.21.15.115:0/4123155073 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6e200c6430 con 0x5645f8ecf2f0 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e4518b740 1 -- 172.21.15.115:0/4123155073 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e200ca540 con 0x5645f8ecf2f0 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e177fe640 1 -- 172.21.15.115:0/4123155073 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6e200cc040 con 0x5645f8ecf2f0 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e4518b740 10 client.25181 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e4518b740 1 -- 172.21.15.115:0/4123155073 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645f9320140 con 0x5645f8ecf2f0 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e177fe640 1 -- 172.21.15.115:0/4123155073 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e200ca540 con 0x5645f8ecf2f0 2024-05-09T05:38:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e177fe640 1 -- 172.21.15.115:0/4123155073 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6e200af020 con 0x5645f8ecf2f0 2024-05-09T05:38:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e4518b740 10 client.25181 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e4518b740 10 client.25181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:38:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e4518b740 20 client.25181 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e4518b740 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645f8ef81b0 0x5645f93227e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e4518b740 4 client.25181 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:38:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.969+0000 7f6e4518b740 1 -- 172.21.15.115:0/4123155073 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645f9208240 con 0x5645f8ef81b0 2024-05-09T05:38:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.970+0000 7f6e37fff640 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645f8ef81b0 0x5645f93227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.970+0000 7f6e177fe640 10 client.25181 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.970+0000 7f6e37fff640 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645f8ef81b0 0x5645f93227e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aaa176dbf286ccd server_cookie=8e24296117a2f662 in_seq=0 out_seq=0 2024-05-09T05:38:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e177fe640 1 -- 172.21.15.115:0/4123155073 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5645f9208240 con 0x5645f8ef81b0 2024-05-09T05:38:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e177fe640 10 client.25181 handle_command_reply: tid=0 2024-05-09T05:38:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 client.25181 shutdown 2024-05-09T05:38:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 -- 172.21.15.115:0/4123155073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ecf2f0 msgr2=0x5645f92ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ecf2f0 0x5645f92ab440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5645f920e4a0 tx=0x7f6e200d2e20 comp rx=0 tx=0).stop 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 -- 172.21.15.115:0/4123155073 shutdown_connections 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645f8ed74d0 0x5645f92ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645f8ecf2f0 0x5645f92ab440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645f8ef81b0 0x5645f93227e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 --2- 172.21.15.115:0/4123155073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645f8eb6d80 0x5645f92ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e177fe640 0 client.25181 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 -- 172.21.15.115:0/4123155073 >> 172.21.15.115:0/4123155073 conn(0x5645f8eec300 msgr2=0x5645f9297760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 -- 172.21.15.115:0/4123155073 shutdown_connections 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 1 -- 172.21.15.115:0/4123155073 wait complete. 2024-05-09T05:38:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.979+0000 7f6e4518b740 20 client.25181 trim_cache size 0 max 16384 2024-05-09T05:38:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:38:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:38:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:38:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6e240b29e0 msgr2=0x7f6e240b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6e240b29e0 0x7f6e240b4ef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5645f929ca30 tx=0x7f6e2800b4d0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38108bc0 msgr2=0x7f6e3819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38108bc0 0x7f6e3819ad50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009230 tx=0x7f6e200040e0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2591888925 shutdown_connections 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6e240b29e0 0x7f6e240b4ef0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e38109060 0x7f6e3819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e38108bc0 0x7f6e3819ad50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 --2- 172.21.15.115:0/2591888925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e38103310 0x7f6e3819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2591888925 >> 172.21.15.115:0/2591888925 conn(0x7f6e380781b0 msgr2=0x7f6e381014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2591888925 shutdown_connections 2024-05-09T05:38:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:38:52.980+0000 7f6e3e2e0640 1 -- 172.21.15.115:0/2591888925 wait complete. 2024-05-09T05:38:52.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.987+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f368c19c100 con 0x7f368c1045f0 2024-05-09T05:38:52.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.988+0000 7f36837fe700 1 -- 172.21.15.115:0/438276061 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7f367c0a9f30 con 0x7f368c1045f0 2024-05-09T05:38:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:38:52.982 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:38:52.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36780b2a80 msgr2=0x7f36780b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36780b2a80 0x7f36780b4f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f367401a040 comp rx=0 tx=0).stop 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c1045f0 msgr2=0x7f368c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c1045f0 0x7f368c19ada0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f367c00e9a0 tx=0x7f367c00ecb0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 shutdown_connections 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36780b2a80 0x7f36780b4f90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f368c109870 0x7f368c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c1045f0 0x7f368c19ada0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 --2- 172.21.15.115:0/438276061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f368c108f20 0x7f368c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 >> 172.21.15.115:0/438276061 conn(0x7f368c100040 msgr2=0x7f368c10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 shutdown_connections 2024-05-09T05:38:52.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:52.990+0000 7f3690fbc700 1 -- 172.21.15.115:0/438276061 wait complete. 2024-05-09T05:38:52.986 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:38:53.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:53 smithi115 ceph-mon[30349]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T05:38:53.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:53 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/438276061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:53.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:53 smithi161 ceph-mon[33805]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T05:38:53.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:53 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/438276061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:53.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T05:33:54.732251+0000', 'modified': '2024-05-09T05:38:10.901453+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24469, 'mds_1': 24457, 'mds_2': 24451, 'mds_3': 24440, 'mds_4': 24425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6839/3801478957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 3801478957}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 3801478957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6841/1081505697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6840', 'nonce': 1081505697}, {'type': 'v1', 'addr': '172.21.15.115:6841', 'nonce': 1081505697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24386': {'gid': 24386, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6833/634985009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 634985009}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 634985009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/1779202002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 1779202002}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 1779202002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6833/2784864553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6832', 'nonce': 2784864553}, {'type': 'v1', 'addr': '172.21.15.195:6833', 'nonce': 2784864553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.161:6837/3591780613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3591780613}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3591780613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'b', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.161:6839/234220396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 234220396}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 234220396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.195:6835/969435997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 969435997}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 969435997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.195:6837/4026198088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 4026198088}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 4026198088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.195:6839/1691623113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 1691623113}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 1691623113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24469, 'qdb_cluster': [24469, 24457, 24451, 24425, 24440]} 2024-05-09T05:38:53.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T05:38:53.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T05:38:53.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:53 smithi195 ceph-mon[33844]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-09T05:38:53.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:53 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/438276061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:38:54.117 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:55.510 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:55 smithi115 ceph-mon[30349]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-05-09T05:38:55.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:55 smithi161 ceph-mon[33805]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-05-09T05:38:55.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:55 smithi195 ceph-mon[33844]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-05-09T05:38:55.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.952+0000 7fa08b760700 1 -- 172.21.15.115:0/1446799298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa084103bb0 msgr2=0x7fa084103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:55.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.952+0000 7fa08b760700 1 --2- 172.21.15.115:0/1446799298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa084103bb0 0x7fa084103fc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa074009990 tx=0x7fa074009ca0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.952+0000 7fa08b760700 1 -- 172.21.15.115:0/1446799298 shutdown_connections 2024-05-09T05:38:55.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.952+0000 7fa08b760700 1 --2- 172.21.15.115:0/1446799298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0841087b0 0x7fa0841100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.952+0000 7fa08b760700 1 --2- 172.21.15.115:0/1446799298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa084103bb0 0x7fa084103fc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.952+0000 7fa08b760700 1 --2- 172.21.15.115:0/1446799298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa084107e60 0x7fa084108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.953+0000 7fa08b760700 1 -- 172.21.15.115:0/1446799298 >> 172.21.15.115:0/1446799298 conn(0x7fa0840fe520 msgr2=0x7fa084100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:55.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.953+0000 7fa08b760700 1 -- 172.21.15.115:0/1446799298 shutdown_connections 2024-05-09T05:38:55.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.953+0000 7fa08b760700 1 -- 172.21.15.115:0/1446799298 wait complete. 2024-05-09T05:38:55.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.953+0000 7fa08b760700 1 Processor -- start 2024-05-09T05:38:55.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.953+0000 7fa08b760700 1 -- start start 2024-05-09T05:38:55.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa08b760700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa084103bb0 0x7fa08419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa08b760700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa084107e60 0x7fa08419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa08b760700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0841087b0 0x7fa08419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa08b760700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa084113250 con 0x7fa0841087b0 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa08b760700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa0841130d0 con 0x7fa084103bb0 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa08b760700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa0841133d0 con 0x7fa084107e60 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa089cfd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0841087b0 0x7fa08419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa089cfd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0841087b0 0x7fa08419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42384/0 (socket says 172.21.15.115:42384) 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa089cfd700 1 -- 172.21.15.115:0/2540915462 learned_addr learned my addr 172.21.15.115:0/2540915462 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa0894fc700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa084103bb0 0x7fa08419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa088cfb700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa084107e60 0x7fa08419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa089cfd700 1 -- 172.21.15.115:0/2540915462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa084107e60 msgr2=0x7fa08419c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa089cfd700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa084107e60 0x7fa08419c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa089cfd700 1 -- 172.21.15.115:0/2540915462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa084103bb0 msgr2=0x7fa08419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa089cfd700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa084103bb0 0x7fa08419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa089cfd700 1 -- 172.21.15.115:0/2540915462 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0841a6f20 con 0x7fa0841087b0 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa0894fc700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa084103bb0 0x7fa08419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:55.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa089cfd700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0841087b0 0x7fa08419c670 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa07c007ee0 tx=0x7fa07c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:55.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa07a7fc700 1 -- 172.21.15.115:0/2540915462 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c00dbd0 con 0x7fa0841087b0 2024-05-09T05:38:55.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.954+0000 7fa08b760700 1 -- 172.21.15.115:0/2540915462 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0841a7210 con 0x7fa0841087b0 2024-05-09T05:38:55.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.955+0000 7fa07a7fc700 1 -- 172.21.15.115:0/2540915462 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa07c00dd70 con 0x7fa0841087b0 2024-05-09T05:38:55.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.955+0000 7fa07a7fc700 1 -- 172.21.15.115:0/2540915462 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0058d0 con 0x7fa0841087b0 2024-05-09T05:38:55.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.955+0000 7fa08b760700 1 -- 172.21.15.115:0/2540915462 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0841a7750 con 0x7fa0841087b0 2024-05-09T05:38:55.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.957+0000 7fa07a7fc700 1 -- 172.21.15.115:0/2540915462 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fa07c01a020 con 0x7fa0841087b0 2024-05-09T05:38:55.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.958+0000 7fa07a7fc700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa0700b2a80 0x7fa0700b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.958+0000 7fa0894fc700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa0700b2a80 0x7fa0700b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.958+0000 7fa07a7fc700 1 -- 172.21.15.115:0/2540915462 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa07c0e2380 con 0x7fa0841087b0 2024-05-09T05:38:55.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa0894fc700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa0700b2a80 0x7fa0700b4f90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0740098e0 tx=0x7fa07400b580 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:55.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa09a176b80 1 Processor -- start 2024-05-09T05:38:55.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa09a176b80 1 -- start start 2024-05-09T05:38:55.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b47784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b45f5450 0x5627b47789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b4772650 0x5627b4772a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa09a176b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5627b4694fa0 con 0x5627b46df110 2024-05-09T05:38:55.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa09a176b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5627b46337a0 con 0x5627b4772650 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa09a176b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627b4690f60 con 0x5627b45f5450 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa0894fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b47784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa0894fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b47784a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42392/0 (socket says 172.21.15.115:42392) 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa0894fc700 1 -- 172.21.15.115:0/4084591503 learned_addr learned my addr 172.21.15.115:0/4084591503 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa088cfb700 1 --2- 172.21.15.115:0/4084591503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b45f5450 0x5627b47789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa089cfd700 1 --2- 172.21.15.115:0/4084591503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b4772650 0x5627b4772a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa0894fc700 1 -- 172.21.15.115:0/4084591503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b45f5450 msgr2=0x5627b47789e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa0894fc700 1 --2- 172.21.15.115:0/4084591503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b45f5450 0x5627b47789e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa0894fc700 1 -- 172.21.15.115:0/4084591503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b4772650 msgr2=0x5627b4772a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.959+0000 7fa0894fc700 1 --2- 172.21.15.115:0/4084591503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b4772650 0x5627b4772a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.960+0000 7fa0894fc700 1 -- 172.21.15.115:0/4084591503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b462b320 con 0x5627b46df110 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.960+0000 7fa0894fc700 1 --2- 172.21.15.115:0/4084591503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b47784a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa074015000 tx=0x7fa07401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d512d88e3d99b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.960+0000 7fa06f7fe700 1 -- 172.21.15.115:0/4084591503 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074003c50 con 0x5627b46df110 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.960+0000 7fa06f7fe700 1 -- 172.21.15.115:0/4084591503 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa07402a070 con 0x5627b46df110 2024-05-09T05:38:55.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.960+0000 7fa06f7fe700 1 -- 172.21.15.115:0/4084591503 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07400f780 con 0x5627b46df110 2024-05-09T05:38:55.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 -- 172.21.15.115:0/4084591503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 msgr2=0x5627b47784a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:55.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4084591503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b47784a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa074015000 tx=0x7fa07401d000 comp rx=0 tx=0).stop 2024-05-09T05:38:55.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 -- 172.21.15.115:0/4084591503 shutdown_connections 2024-05-09T05:38:55.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4084591503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b4772650 0x5627b4772a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4084591503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b47784a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4084591503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b45f5450 0x5627b47789e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 -- 172.21.15.115:0/4084591503 >> 172.21.15.115:0/4084591503 conn(0x5627b4655650 msgr2=0x5627b476d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 -- 172.21.15.115:0/4084591503 shutdown_connections 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 -- 172.21.15.115:0/4084591503 wait complete. 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 -- start start 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.961+0000 7fa09a176b80 1 Processor -- start 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b45f5450 0x5627b477c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b46df110 0x5627b477c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b477cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa09a176b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5627b4510b50 con 0x5627b45f5450 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa09a176b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5627b464f100 con 0x5627b46df110 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa09a176b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627b477f180 con 0x5627b4772650 2024-05-09T05:38:55.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa09a176b80 10 client.0 _ll_register_callbacks cb 0x5627b4506670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa0894fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b45f5450 0x5627b477c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa0894fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b45f5450 0x5627b477c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42394/0 (socket says 172.21.15.115:42394) 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa0894fc700 1 -- 172.21.15.115:0/843089342 learned_addr learned my addr 172.21.15.115:0/843089342 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa089cfd700 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b477cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa088cfb700 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b46df110 0x5627b477c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa0894fc700 1 -- 172.21.15.115:0/843089342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 msgr2=0x5627b477cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa0894fc700 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b477cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa0894fc700 1 -- 172.21.15.115:0/843089342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b46df110 msgr2=0x5627b477c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa0894fc700 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b46df110 0x5627b477c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa0894fc700 1 -- 172.21.15.115:0/843089342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b4510440 con 0x5627b45f5450 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa0894fc700 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b45f5450 0x5627b477c130 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa07400c010 tx=0x7fa07400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271dd7c8140c633c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.962+0000 7fa06cff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa06cff9700 1 -- 172.21.15.115:0/843089342 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074036070 con 0x5627b45f5450 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa09a176b80 1 -- 172.21.15.115:0/843089342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa07402a070 con 0x5627b45f5450 2024-05-09T05:38:55.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa06cff9700 1 -- 172.21.15.115:0/843089342 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa07400b7d0 con 0x5627b45f5450 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa06cff9700 1 -- 172.21.15.115:0/843089342 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa074042050 con 0x5627b45f5450 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa09a176b80 10 client.15612 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa09a176b80 1 -- 172.21.15.115:0/843089342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627b4789ad0 con 0x5627b45f5450 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa06cff9700 1 -- 172.21.15.115:0/843089342 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa07402a070 con 0x5627b45f5450 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa06cff9700 1 -- 172.21.15.115:0/843089342 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa07402f890 con 0x5627b45f5450 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa09a176b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa09a176b80 10 client.15612 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa09a176b80 20 client.15612 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa09a176b80 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5627b478bbe0 0x5627b478d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa09a176b80 4 client.15612 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.963+0000 7fa09a176b80 1 -- 172.21.15.115:0/843089342 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627b46a2980 con 0x5627b478bbe0 2024-05-09T05:38:55.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.964+0000 7fa088cfb700 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5627b478bbe0 0x5627b478d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.964+0000 7fa06cff9700 10 client.15612 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.964+0000 7fa088cfb700 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5627b478bbe0 0x5627b478d050 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d6a2d150dc306111 server_cookie=3e2973692e0dee4 in_seq=0 out_seq=0 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa06cff9700 1 -- 172.21.15.115:0/843089342 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5627b46a2980 con 0x5627b478bbe0 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa06cff9700 10 client.15612 handle_command_reply: tid=0 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 client.15612 shutdown 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 -- 172.21.15.115:0/843089342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b45f5450 msgr2=0x5627b477c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b45f5450 0x5627b477c130 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa07400c010 tx=0x7fa07400f460 comp rx=0 tx=0).stop 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 -- 172.21.15.115:0/843089342 shutdown_connections 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5627b478bbe0 0x5627b478d050 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b477cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b45f5450 0x5627b477c130 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 --2- 172.21.15.115:0/843089342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b46df110 0x5627b477c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 -- 172.21.15.115:0/843089342 >> 172.21.15.115:0/843089342 conn(0x5627b4655650 msgr2=0x5627b476d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:55.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa06cff9700 0 client.15612 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:38:55.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 -- 172.21.15.115:0/843089342 shutdown_connections 2024-05-09T05:38:55.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 1 -- 172.21.15.115:0/843089342 wait complete. 2024-05-09T05:38:55.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.965+0000 7fa09a176b80 20 client.15612 trim_cache size 0 max 16384 2024-05-09T05:38:55.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa09a176b80 1 Processor -- start 2024-05-09T05:38:55.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa09a176b80 1 -- start start 2024-05-09T05:38:55.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 0x5627b47764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b477c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa09a176b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5627b478b390 con 0x5627b46df110 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa09a176b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5627b478ba60 con 0x5627b45f5450 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa09a176b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627b478c130 con 0x5627b4772650 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa088cfb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa088cfb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42404/0 (socket says 172.21.15.115:42404) 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa088cfb700 1 -- 172.21.15.115:0/692851350 learned_addr learned my addr 172.21.15.115:0/692851350 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa0894fc700 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 0x5627b47764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa089cfd700 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b477c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa088cfb700 1 -- 172.21.15.115:0/692851350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 msgr2=0x5627b477c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa088cfb700 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b477c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa088cfb700 1 -- 172.21.15.115:0/692851350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 msgr2=0x5627b47764c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa088cfb700 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 0x5627b47764c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa088cfb700 1 -- 172.21.15.115:0/692851350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b476ecb0 con 0x5627b46df110 2024-05-09T05:38:55.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.986+0000 7fa0894fc700 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 0x5627b47764c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:55.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa088cfb700 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787f80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa0841065c0 tx=0x7fa080004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668b2e03dcc8c8f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:55.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa06effd700 1 -- 172.21.15.115:0/692851350 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080022070 con 0x5627b46df110 2024-05-09T05:38:55.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa06effd700 1 -- 172.21.15.115:0/692851350 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0800044c0 con 0x5627b46df110 2024-05-09T05:38:55.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa06effd700 1 -- 172.21.15.115:0/692851350 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0800038e0 con 0x5627b46df110 2024-05-09T05:38:55.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa09a176b80 1 -- 172.21.15.115:0/692851350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 msgr2=0x5627b4787f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:55.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa09a176b80 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787f80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa0841065c0 tx=0x7fa080004750 comp rx=0 tx=0).stop 2024-05-09T05:38:55.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa09a176b80 1 -- 172.21.15.115:0/692851350 shutdown_connections 2024-05-09T05:38:55.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa09a176b80 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 0x5627b47764c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa09a176b80 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b477c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.987+0000 7fa09a176b80 1 --2- 172.21.15.115:0/692851350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787f80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 -- 172.21.15.115:0/692851350 >> 172.21.15.115:0/692851350 conn(0x5627b4655650 msgr2=0x5627b4775db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 -- 172.21.15.115:0/692851350 shutdown_connections 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 -- 172.21.15.115:0/692851350 wait complete. 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 -- start start 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 Processor -- start 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 0x5627b4787550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b4787fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5627b478ced0 con 0x5627b46df110 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5627b478c800 con 0x5627b45f5450 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.988+0000 7fa09a176b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5627b478d5a0 con 0x5627b4772650 2024-05-09T05:38:55.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa09a176b80 10 client.0 _ll_register_callbacks cb 0x5627b4506670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa088cfb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa088cfb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42420/0 (socket says 172.21.15.115:42420) 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa088cfb700 1 -- 172.21.15.115:0/4182055384 learned_addr learned my addr 172.21.15.115:0/4182055384 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa089cfd700 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b4787fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa0894fc700 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 0x5627b4787550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa088cfb700 1 -- 172.21.15.115:0/4182055384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 msgr2=0x5627b4787fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa088cfb700 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b4787fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa088cfb700 1 -- 172.21.15.115:0/4182055384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 msgr2=0x5627b4787550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa088cfb700 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 0x5627b4787550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa088cfb700 1 -- 172.21.15.115:0/4182055384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b47946e0 con 0x5627b46df110 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa088cfb700 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787a90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa08001d9a0 tx=0x7fa08001da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b0dfe15bb175a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:55.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa06cff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:55.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa06cff9700 1 -- 172.21.15.115:0/4182055384 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08002b070 con 0x5627b46df110 2024-05-09T05:38:55.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa06cff9700 1 -- 172.21.15.115:0/4182055384 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa080005020 con 0x5627b46df110 2024-05-09T05:38:55.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa09a176b80 1 -- 172.21.15.115:0/4182055384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0800044c0 con 0x5627b46df110 2024-05-09T05:38:55.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa06cff9700 1 -- 172.21.15.115:0/4182055384 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa080036050 con 0x5627b46df110 2024-05-09T05:38:55.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa09a176b80 10 client.15618 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:55.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.989+0000 7fa09a176b80 1 -- 172.21.15.115:0/4182055384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627b47f7180 con 0x5627b46df110 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.990+0000 7fa06cff9700 1 -- 172.21.15.115:0/4182055384 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0800044c0 con 0x5627b46df110 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.990+0000 7fa06cff9700 1 -- 172.21.15.115:0/4182055384 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa080027410 con 0x5627b46df110 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.990+0000 7fa09a176b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.990+0000 7fa09a176b80 10 client.15618 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.990+0000 7fa09a176b80 20 client.15618 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.990+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5627b47f9190 0x5627b47fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.990+0000 7fa09a176b80 4 client.15618 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.990+0000 7fa09a176b80 1 -- 172.21.15.115:0/4182055384 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "status"}) -- 0x5627b469ca20 con 0x5627b47f9190 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.990+0000 7fa0894fc700 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5627b47f9190 0x5627b47fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:55.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa0894fc700 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5627b47f9190 0x5627b47fa600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=738478b1ae0f768b server_cookie=43eba639382e83be in_seq=0 out_seq=0 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa06cff9700 10 client.15618 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa06cff9700 1 -- 172.21.15.115:0/4182055384 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5627b469ca20 con 0x5627b47f9190 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa06cff9700 10 client.15618 handle_command_reply: tid=0 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 client.15618 shutdown 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 -- 172.21.15.115:0/4182055384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 msgr2=0x5627b4787a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787a90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa08001d9a0 tx=0x7fa08001da80 comp rx=0 tx=0).stop 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 -- 172.21.15.115:0/4182055384 shutdown_connections 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627b45f5450 0x5627b4787550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5627b47f9190 0x5627b47fa600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627b4772650 0x5627b4787fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 --2- 172.21.15.115:0/4182055384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627b46df110 0x5627b4787a90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 -- 172.21.15.115:0/4182055384 >> 172.21.15.115:0/4182055384 conn(0x5627b4655650 msgr2=0x5627b477c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa06cff9700 0 client.15618 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:38:55.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 -- 172.21.15.115:0/4182055384 shutdown_connections 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 1 -- 172.21.15.115:0/4182055384 wait complete. 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.991+0000 7fa09a176b80 20 client.15618 trim_cache size 0 max 16384 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 4, 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24425, 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:38:55.994 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 233.705132077, 2024-05-09T05:38:55.995 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 30, 2024-05-09T05:38:55.995 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 88, 2024-05-09T05:38:55.995 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:38:55.995 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 329.67586450800002 2024-05-09T05:38:55.995 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 -- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa0700b2a80 msgr2=0x7fa0700b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa0700b2a80 0x7fa0700b4f90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0740098e0 tx=0x7fa07400b580 comp rx=0 tx=0).stop 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 -- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0841087b0 msgr2=0x7fa08419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0841087b0 0x7fa08419c670 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa07c007ee0 tx=0x7fa07c00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 -- 172.21.15.115:0/2540915462 shutdown_connections 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa0700b2a80 0x7fa0700b4f90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa084103bb0 0x7fa08419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0841087b0 0x7fa08419c670 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 --2- 172.21.15.115:0/2540915462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa084107e60 0x7fa08419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 -- 172.21.15.115:0/2540915462 >> 172.21.15.115:0/2540915462 conn(0x7fa0840fe520 msgr2=0x7fa0841014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 -- 172.21.15.115:0/2540915462 shutdown_connections 2024-05-09T05:38:55.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:55.992+0000 7fa08b760700 1 -- 172.21.15.115:0/2540915462 wait complete. 2024-05-09T05:38:56.800 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T05:38:57.062 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:38:57.492 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:57 smithi115 ceph-mon[30349]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:38:57.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:57 smithi161 ceph-mon[33805]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:38:57.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:57 smithi195 ceph-mon[33844]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.931+0000 7fe02c016700 1 -- 172.21.15.115:0/1362820301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe024103130 msgr2=0x7fe024111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.931+0000 7fe02c016700 1 --2- 172.21.15.115:0/1362820301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe024103130 0x7fe024111120 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009990 tx=0x7fe01c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.931+0000 7fe02c016700 1 -- 172.21.15.115:0/1362820301 shutdown_connections 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.931+0000 7fe02c016700 1 --2- 172.21.15.115:0/1362820301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe02410a3e0 0x7fe02410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.931+0000 7fe02c016700 1 --2- 172.21.15.115:0/1362820301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe024103130 0x7fe024111120 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.931+0000 7fe02c016700 1 --2- 172.21.15.115:0/1362820301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0241027e0 0x7fe024102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.931+0000 7fe02c016700 1 -- 172.21.15.115:0/1362820301 >> 172.21.15.115:0/1362820301 conn(0x7fe0240fe6f0 msgr2=0x7fe024100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.931+0000 7fe02c016700 1 -- 172.21.15.115:0/1362820301 shutdown_connections 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.931+0000 7fe02c016700 1 -- 172.21.15.115:0/1362820301 wait complete. 2024-05-09T05:38:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.932+0000 7fe02c016700 1 Processor -- start 2024-05-09T05:38:58.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.932+0000 7fe02c016700 1 -- start start 2024-05-09T05:38:58.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.932+0000 7fe02c016700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0241027e0 0x7fe0241a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.932+0000 7fe02c016700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe024103130 0x7fe0241a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.932+0000 7fe02c016700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02410a3e0 0x7fe02419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.932+0000 7fe02c016700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe024114330 con 0x7fe02410a3e0 2024-05-09T05:38:58.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.932+0000 7fe02c016700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0241141b0 con 0x7fe024103130 2024-05-09T05:38:58.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.932+0000 7fe02c016700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0241144b0 con 0x7fe0241027e0 2024-05-09T05:38:58.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02a5b3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02410a3e0 0x7fe02419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02a5b3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02410a3e0 0x7fe02419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42428/0 (socket says 172.21.15.115:42428) 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02a5b3700 1 -- 172.21.15.115:0/209176148 learned_addr learned my addr 172.21.15.115:0/209176148 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe0295b1700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe024103130 0x7fe0241a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe029db2700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0241027e0 0x7fe0241a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02a5b3700 1 -- 172.21.15.115:0/209176148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0241027e0 msgr2=0x7fe0241a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02a5b3700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0241027e0 0x7fe0241a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02a5b3700 1 -- 172.21.15.115:0/209176148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe024103130 msgr2=0x7fe0241a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02a5b3700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe024103130 0x7fe0241a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02a5b3700 1 -- 172.21.15.115:0/209176148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02419b1d0 con 0x7fe02410a3e0 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02a5b3700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02410a3e0 0x7fe02419aaf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0047b0 tx=0x7fe01c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe01affd700 1 -- 172.21.15.115:0/209176148 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c027070 con 0x7fe02410a3e0 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe01affd700 1 -- 172.21.15.115:0/209176148 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe01c00fb00 con 0x7fe02410a3e0 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02c016700 1 -- 172.21.15.115:0/209176148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02419b460 con 0x7fe02410a3e0 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe01affd700 1 -- 172.21.15.115:0/209176148 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c022b20 con 0x7fe02410a3e0 2024-05-09T05:38:58.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.933+0000 7fe02c016700 1 -- 172.21.15.115:0/209176148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0241a7760 con 0x7fe02410a3e0 2024-05-09T05:38:58.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.936+0000 7fe01affd700 1 -- 172.21.15.115:0/209176148 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7fe01c02a070 con 0x7fe02410a3e0 2024-05-09T05:38:58.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.937+0000 7fe01affd700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe0100b2b00 0x7fe0100b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.937+0000 7fe029db2700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe0100b2b00 0x7fe0100b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.937+0000 7fe01affd700 1 -- 172.21.15.115:0/209176148 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fe01c0e3ac0 con 0x7fe02410a3e0 2024-05-09T05:38:58.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe03aa2cb80 1 Processor -- start 2024-05-09T05:38:58.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe029db2700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe0100b2b00 0x7fe0100b5010 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe020007fd0 tx=0x7fe02000c040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:58.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe03aa2cb80 1 -- start start 2024-05-09T05:38:58.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 0x558d0dd249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd1ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558d0dc40fa0 con 0x558d0dc8b110 2024-05-09T05:38:58.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558d0dbdf7a0 con 0x558d0dba1450 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558d0dc3cf60 con 0x558d0dd1e650 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe029db2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe029db2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd244a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42436/0 (socket says 172.21.15.115:42436) 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe029db2700 1 -- 172.21.15.115:0/411909574 learned_addr learned my addr 172.21.15.115:0/411909574 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe02a5b3700 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd1ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.938+0000 7fe0295b1700 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 0x558d0dd249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe029db2700 1 -- 172.21.15.115:0/411909574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 msgr2=0x558d0dd1ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe029db2700 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd1ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe029db2700 1 -- 172.21.15.115:0/411909574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 msgr2=0x558d0dd249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe029db2700 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 0x558d0dd249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe029db2700 1 -- 172.21.15.115:0/411909574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d0dbd7320 con 0x558d0dc8b110 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe0295b1700 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 0x558d0dd249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe029db2700 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd244a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe020016780 tx=0x7fe020012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41cdc219500303f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe00ffff700 1 -- 172.21.15.115:0/411909574 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02001c040 con 0x558d0dc8b110 2024-05-09T05:38:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe00ffff700 1 -- 172.21.15.115:0/411909574 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe020025070 con 0x558d0dc8b110 2024-05-09T05:38:58.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.939+0000 7fe00ffff700 1 -- 172.21.15.115:0/411909574 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0200147e0 con 0x558d0dc8b110 2024-05-09T05:38:58.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/411909574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 msgr2=0x558d0dd244a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:58.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd244a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe020016780 tx=0x7fe020012000 comp rx=0 tx=0).stop 2024-05-09T05:38:58.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/411909574 shutdown_connections 2024-05-09T05:38:58.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd1ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd244a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/411909574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 0x558d0dd249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/411909574 >> 172.21.15.115:0/411909574 conn(0x558d0dc01650 msgr2=0x558d0dd193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:58.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/411909574 shutdown_connections 2024-05-09T05:38:58.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/411909574 wait complete. 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.940+0000 7fe03aa2cb80 1 -- start start 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe03aa2cb80 1 Processor -- start 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 0x558d0dd281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd28720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd28c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558d0dabcb50 con 0x558d0dc8b110 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558d0dbfb100 con 0x558d0dba1450 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558d0dd2b180 con 0x558d0dd1e650 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe03aa2cb80 10 client.0 _ll_register_callbacks cb 0x558d0dab2670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe0295b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd28720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe0295b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd28720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42452/0 (socket says 172.21.15.115:42452) 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe0295b1700 1 -- 172.21.15.115:0/4007802010 learned_addr learned my addr 172.21.15.115:0/4007802010 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe02a5b3700 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd28c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.941+0000 7fe029db2700 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 0x558d0dd281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe0295b1700 1 -- 172.21.15.115:0/4007802010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 msgr2=0x558d0dd28c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe0295b1700 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd28c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe0295b1700 1 -- 172.21.15.115:0/4007802010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 msgr2=0x558d0dd281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe0295b1700 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 0x558d0dd281e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe0295b1700 1 -- 172.21.15.115:0/4007802010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d0dabc440 con 0x558d0dc8b110 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe02a5b3700 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd28c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe0295b1700 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd28720 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe01400bf10 tx=0x7fe014012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc51438c0511ff6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe00d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe00d7fa700 1 -- 172.21.15.115:0/4007802010 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014004440 con 0x558d0dc8b110 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe00d7fa700 1 -- 172.21.15.115:0/4007802010 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fe014004830 con 0x558d0dc8b110 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/4007802010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe020025070 con 0x558d0dc8b110 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe00d7fa700 1 -- 172.21.15.115:0/4007802010 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe014023050 con 0x558d0dc8b110 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe03aa2cb80 10 client.15630 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/4007802010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d0dd35c20 con 0x558d0dc8b110 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.942+0000 7fe00d7fa700 1 -- 172.21.15.115:0/4007802010 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe020025070 con 0x558d0dc8b110 2024-05-09T05:38:58.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe00d7fa700 1 -- 172.21.15.115:0/4007802010 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe014019030 con 0x558d0dc8b110 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe03aa2cb80 10 client.15630 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe03aa2cb80 10 client.15630 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe03aa2cb80 20 client.15630 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x558d0dd38270 0x558d0dd396e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe03aa2cb80 4 client.15630 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/4007802010 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d0dc4e980 con 0x558d0dd38270 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe029db2700 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x558d0dd38270 0x558d0dd396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe00d7fa700 10 client.15630 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.943+0000 7fe029db2700 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x558d0dd38270 0x558d0dd396e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d75d731d12ee8ac8 server_cookie=9ac1951ecb46623f in_seq=0 out_seq=0 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.944+0000 7fe00d7fa700 1 -- 172.21.15.115:0/4007802010 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d0dc4e980 con 0x558d0dd38270 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.944+0000 7fe00d7fa700 10 client.15630 handle_command_reply: tid=0 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.944+0000 7fe03aa2cb80 1 client.15630 shutdown 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.944+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/4007802010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 msgr2=0x558d0dd28720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:58.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.944+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd28720 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe01400bf10 tx=0x7fe014012730 comp rx=0 tx=0).stop 2024-05-09T05:38:58.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/4007802010 shutdown_connections 2024-05-09T05:38:58.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dba1450 0x558d0dd281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x558d0dd38270 0x558d0dd396e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd28c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/4007802010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd28720 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe00d7fa700 0 client.15630 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:38:58.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/4007802010 >> 172.21.15.115:0/4007802010 conn(0x558d0dc01650 msgr2=0x558d0dd19170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:58.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/4007802010 shutdown_connections 2024-05-09T05:38:58.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/4007802010 wait complete. 2024-05-09T05:38:58.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.945+0000 7fe03aa2cb80 20 client.15630 trim_cache size 0 max 16384 2024-05-09T05:38:58.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.965+0000 7fe03aa2cb80 1 Processor -- start 2024-05-09T05:38:58.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.965+0000 7fe03aa2cb80 1 -- start start 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dba1450 0x558d0dd25f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dc8b110 0x558d0dd26ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd27010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558d0dd37a20 con 0x558d0dba1450 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558d0dd380f0 con 0x558d0dc8b110 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558d0dd387c0 con 0x558d0dd1e650 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe029db2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dba1450 0x558d0dd25f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe029db2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dba1450 0x558d0dd25f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42464/0 (socket says 172.21.15.115:42464) 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe029db2700 1 -- 172.21.15.115:0/1558798147 learned_addr learned my addr 172.21.15.115:0/1558798147 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe0295b1700 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dc8b110 0x558d0dd26ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe02a5b3700 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd27010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe029db2700 1 -- 172.21.15.115:0/1558798147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 msgr2=0x558d0dd27010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:58.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe029db2700 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd27010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe029db2700 1 -- 172.21.15.115:0/1558798147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dc8b110 msgr2=0x558d0dd26ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe029db2700 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dc8b110 0x558d0dd26ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe029db2700 1 -- 172.21.15.115:0/1558798147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d0dd1b130 con 0x558d0dba1450 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe02a5b3700 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd27010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe0295b1700 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dc8b110 0x558d0dd26ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe029db2700 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dba1450 0x558d0dd25f20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe020016780 tx=0x7fe020016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef00e03588fff742 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe00f7fe700 1 -- 172.21.15.115:0/1558798147 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020014530 con 0x558d0dba1450 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe00f7fe700 1 -- 172.21.15.115:0/1558798147 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe02003a070 con 0x558d0dba1450 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.966+0000 7fe00f7fe700 1 -- 172.21.15.115:0/1558798147 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020035920 con 0x558d0dba1450 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.967+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/1558798147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dba1450 msgr2=0x558d0dd25f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.967+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dba1450 0x558d0dd25f20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe020016780 tx=0x7fe020016000 comp rx=0 tx=0).stop 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.967+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/1558798147 shutdown_connections 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.967+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dd1e650 0x558d0dd27010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.967+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dba1450 0x558d0dd25f20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.967+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/1558798147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dc8b110 0x558d0dd26ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.967+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/1558798147 >> 172.21.15.115:0/1558798147 conn(0x558d0dc01650 msgr2=0x558d0dd23320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.967+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/1558798147 shutdown_connections 2024-05-09T05:38:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.967+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/1558798147 wait complete. 2024-05-09T05:38:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe03aa2cb80 1 -- start start 2024-05-09T05:38:58.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe03aa2cb80 1 Processor -- start 2024-05-09T05:38:58.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dba1450 0x558d0dd24b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd25050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe03aa2cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dd1e650 0x558d0dd25590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558d0dd39560 con 0x558d0dc8b110 2024-05-09T05:38:58.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558d0dd38e90 con 0x558d0dd1e650 2024-05-09T05:38:58.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe03aa2cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558d0dd39c30 con 0x558d0dba1450 2024-05-09T05:38:58.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe03aa2cb80 10 client.0 _ll_register_callbacks cb 0x558d0dab2670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe0295b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd25050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe0295b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd25050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42480/0 (socket says 172.21.15.115:42480) 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe0295b1700 1 -- 172.21.15.115:0/3842486530 learned_addr learned my addr 172.21.15.115:0/3842486530 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe02a5b3700 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dd1e650 0x558d0dd25590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.968+0000 7fe029db2700 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dba1450 0x558d0dd24b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe0295b1700 1 -- 172.21.15.115:0/3842486530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dba1450 msgr2=0x558d0dd24b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe0295b1700 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dba1450 0x558d0dd24b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe0295b1700 1 -- 172.21.15.115:0/3842486530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dd1e650 msgr2=0x558d0dd25590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe0295b1700 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dd1e650 0x558d0dd25590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe0295b1700 1 -- 172.21.15.115:0/3842486530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d0dd41bd0 con 0x558d0dc8b110 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe0295b1700 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd25050 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe01400bf10 tx=0x7fe014012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49aefa85d6c80d44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe00d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe00d7fa700 1 -- 172.21.15.115:0/3842486530 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014010a20 con 0x558d0dc8b110 2024-05-09T05:38:58.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe00d7fa700 1 -- 172.21.15.115:0/3842486530 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fe014010bc0 con 0x558d0dc8b110 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/3842486530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02003a070 con 0x558d0dc8b110 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe00d7fa700 1 -- 172.21.15.115:0/3842486530 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0140052c0 con 0x558d0dc8b110 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe03aa2cb80 10 client.15636 fetch_fsmap learned FSMap version 30 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/3842486530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d0dd42dc0 con 0x558d0dc8b110 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.969+0000 7fe00d7fa700 1 -- 172.21.15.115:0/3842486530 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe02003a070 con 0x558d0dc8b110 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe00d7fa700 1 -- 172.21.15.115:0/3842486530 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe014019040 con 0x558d0dc8b110 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe03aa2cb80 10 client.15636 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe03aa2cb80 10 client.15636 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe03aa2cb80 20 client.15636 populate_metadata read hostname 'smithi115' 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x558d0dda5140 0x558d0dda65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe03aa2cb80 4 client.15636 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/3842486530 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "status"}) -- 0x558d0dc48a20 con 0x558d0dda5140 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe029db2700 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x558d0dda5140 0x558d0dda65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe029db2700 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x558d0dda5140 0x558d0dda65b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18f56f5dceefa66d server_cookie=3ff6d8374dffab74 in_seq=0 out_seq=0 2024-05-09T05:38:58.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.970+0000 7fe00d7fa700 10 client.15636 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe00d7fa700 1 -- 172.21.15.115:0/3842486530 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558d0dc48a20 con 0x558d0dda5140 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe00d7fa700 10 client.15636 handle_command_reply: tid=0 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 client.15636 shutdown 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/3842486530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 msgr2=0x558d0dd25050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd25050 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe01400bf10 tx=0x7fe014012b00 comp rx=0 tx=0).stop 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/3842486530 shutdown_connections 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d0dd1e650 0x558d0dd25590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x558d0dda5140 0x558d0dda65b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558d0dba1450 0x558d0dd24b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 --2- 172.21.15.115:0/3842486530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558d0dc8b110 0x558d0dd25050 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe00d7fa700 0 client.15636 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/3842486530 >> 172.21.15.115:0/3842486530 conn(0x558d0dc01650 msgr2=0x558d0dd23320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/3842486530 shutdown_connections 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 1 -- 172.21.15.115:0/3842486530 wait complete. 2024-05-09T05:38:58.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.971+0000 7fe03aa2cb80 20 client.15636 trim_cache size 0 max 16384 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 3, 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24440, 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 237.68819687000001, 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 30, 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 88, 2024-05-09T05:38:58.971 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:38:58.972 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 319.05874865200002 2024-05-09T05:38:58.972 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:38:58.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 -- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe0100b2b00 msgr2=0x7fe0100b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:58.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe0100b2b00 0x7fe0100b5010 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe020007fd0 tx=0x7fe02000c040 comp rx=0 tx=0).stop 2024-05-09T05:38:58.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 -- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02410a3e0 msgr2=0x7fe02419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:38:58.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02410a3e0 0x7fe02419aaf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0047b0 tx=0x7fe01c005e80 comp rx=0 tx=0).stop 2024-05-09T05:38:58.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 -- 172.21.15.115:0/209176148 shutdown_connections 2024-05-09T05:38:58.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe0100b2b00 0x7fe0100b5010 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe02410a3e0 0x7fe02419aaf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0241027e0 0x7fe0241a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 --2- 172.21.15.115:0/209176148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe024103130 0x7fe0241a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:38:58.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 -- 172.21.15.115:0/209176148 >> 172.21.15.115:0/209176148 conn(0x7fe0240fe6f0 msgr2=0x7fe024100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:38:58.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 -- 172.21.15.115:0/209176148 shutdown_connections 2024-05-09T05:38:58.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:38:58.972+0000 7fe02c016700 1 -- 172.21.15.115:0/209176148 wait complete. 2024-05-09T05:38:59.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:38:59 smithi115 ceph-mon[30349]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-05-09T05:38:59.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:38:59 smithi161 ceph-mon[33805]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-05-09T05:38:59.688 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:38:59 smithi195 ceph-mon[33844]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-05-09T05:38:59.697 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-09T05:38:59.943 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:01.146 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:39:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:39:00.835+0000 7fb4a48e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:39:01.146 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:39:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:39:00.835+0000 7fb4a48e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:39:01.147 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:39:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:39:00.835+0000 7fb4a48e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:39:01.147 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:39:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:39:00.835+0000 7fb4a48e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:39:01.147 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 05:39:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x[30580]: 2024-05-09T05:39:00.835+0000 7fb4a48e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T05:39:01.535 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:01 smithi115 ceph-mon[30349]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-09T05:39:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:01 smithi161 ceph-mon[33805]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-09T05:39:01.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.801+0000 7f1eeaee2700 1 -- 172.21.15.115:0/513047815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4103c10 msgr2=0x7f1ee4104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:01.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.801+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/513047815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4103c10 0x7f1ee4104020 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00099c0 tx=0x7f1ee0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.801+0000 7f1eeaee2700 1 -- 172.21.15.115:0/513047815 shutdown_connections 2024-05-09T05:39:01.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.801+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/513047815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee4108970 0x7f1ee41102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.801+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/513047815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4103c10 0x7f1ee4104020 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.801+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/513047815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee4108020 0x7f1ee4108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.801+0000 7f1eeaee2700 1 -- 172.21.15.115:0/513047815 >> 172.21.15.115:0/513047815 conn(0x7f1ee40fe520 msgr2=0x7f1ee4100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:01.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.801+0000 7f1eeaee2700 1 -- 172.21.15.115:0/513047815 shutdown_connections 2024-05-09T05:39:01.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.801+0000 7f1eeaee2700 1 -- 172.21.15.115:0/513047815 wait complete. 2024-05-09T05:39:01.873 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:01 smithi195 ceph-mon[33844]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-09T05:39:01.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1eeaee2700 1 Processor -- start 2024-05-09T05:39:01.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1eeaee2700 1 -- start start 2024-05-09T05:39:01.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1eeaee2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee4103c10 0x7f1ee419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1eeaee2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee4108020 0x7f1ee419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1eeaee2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4108970 0x7f1ee419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1eeaee2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4113450 con 0x7f1ee4108970 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1eeaee2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee41132d0 con 0x7f1ee4108020 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1eeaee2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee41135d0 con 0x7f1ee4103c10 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1ee947f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4108970 0x7f1ee419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1ee947f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4108970 0x7f1ee419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42520/0 (socket says 172.21.15.115:42520) 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.802+0000 7f1ee947f700 1 -- 172.21.15.115:0/3863167347 learned_addr learned my addr 172.21.15.115:0/3863167347 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1edbfff700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee4108020 0x7f1ee419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee4103c10 0x7f1ee419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ee947f700 1 -- 172.21.15.115:0/3863167347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee4103c10 msgr2=0x7f1ee419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee4103c10 0x7f1ee419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ee947f700 1 -- 172.21.15.115:0/3863167347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee4108020 msgr2=0x7f1ee419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee4108020 0x7f1ee419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ee947f700 1 -- 172.21.15.115:0/3863167347 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee41a7070 con 0x7f1ee4108970 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee4103c10 0x7f1ee419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4108970 0x7f1ee419c850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007d70 tx=0x7f1edc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ed9ffb700 1 -- 172.21.15.115:0/3863167347 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc016020 con 0x7f1ee4108970 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ed9ffb700 1 -- 172.21.15.115:0/3863167347 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1edc00ad40 con 0x7f1ee4108970 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1eeaee2700 1 -- 172.21.15.115:0/3863167347 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee41a7360 con 0x7f1ee4108970 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1ed9ffb700 1 -- 172.21.15.115:0/3863167347 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0115e0 con 0x7f1ee4108970 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.803+0000 7f1eeaee2700 1 -- 172.21.15.115:0/3863167347 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee41a78a0 con 0x7f1ee4108970 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.806+0000 7f1ed9ffb700 1 -- 172.21.15.115:0/3863167347 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) ==== 149829+0+0 (secure 0 0 0) 0x7f1edc019020 con 0x7f1ee4108970 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.807+0000 7f1ed9ffb700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1ecc0b2b00 0x7f1ecc0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.807+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1ecc0b2b00 0x7f1ecc0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.807+0000 7f1ed9ffb700 1 -- 172.21.15.115:0/3863167347 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1edc0e28c0 con 0x7f1ee4108970 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1ecc0b2b00 0x7f1ecc0b5010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00058e0 tx=0x7f1ee0005ab0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:01.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ef98f8b80 1 Processor -- start 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ef98f8b80 1 -- start start 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 0x557dcc4101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc28d450 0x557dcc410730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc40a3b0 0x557dcc40a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557dcc32d050 con 0x557dcc28d450 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557dcc2d2950 con 0x557dcc307cb0 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557dcc329010 con 0x557dcc40a3b0 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1edbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc28d450 0x557dcc410730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1edbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc28d450 0x557dcc410730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42536/0 (socket says 172.21.15.115:42536) 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1edbfff700 1 -- 172.21.15.115:0/2821330141 learned_addr learned my addr 172.21.15.115:0/2821330141 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/2821330141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 0x557dcc4101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1ee947f700 1 --2- 172.21.15.115:0/2821330141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc40a3b0 0x557dcc40a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1edbfff700 1 -- 172.21.15.115:0/2821330141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc40a3b0 msgr2=0x557dcc40a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1edbfff700 1 --2- 172.21.15.115:0/2821330141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc40a3b0 0x557dcc40a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1edbfff700 1 -- 172.21.15.115:0/2821330141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 msgr2=0x557dcc4101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:01.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1edbfff700 1 --2- 172.21.15.115:0/2821330141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 0x557dcc4101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.808+0000 7f1edbfff700 1 -- 172.21.15.115:0/2821330141 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcc2c34b0 con 0x557dcc28d450 2024-05-09T05:39:01.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.809+0000 7f1edbfff700 1 --2- 172.21.15.115:0/2821330141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc28d450 0x557dcc410730 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400bcf0 tx=0x7f1ed40103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea623dc8061ecfb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:01.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.809+0000 7f1ecaffd700 1 -- 172.21.15.115:0/2821330141 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4010d10 con 0x557dcc28d450 2024-05-09T05:39:01.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.809+0000 7f1ecaffd700 1 -- 172.21.15.115:0/2821330141 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ed40044c0 con 0x557dcc28d450 2024-05-09T05:39:01.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.809+0000 7f1ecaffd700 1 -- 172.21.15.115:0/2821330141 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4005280 con 0x557dcc28d450 2024-05-09T05:39:01.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/2821330141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc28d450 msgr2=0x557dcc410730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:01.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/2821330141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc28d450 0x557dcc410730 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400bcf0 tx=0x7f1ed40103d0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/2821330141 shutdown_connections 2024-05-09T05:39:01.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/2821330141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 0x557dcc4101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/2821330141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc40a3b0 0x557dcc40a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/2821330141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc28d450 0x557dcc410730 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/2821330141 >> 172.21.15.115:0/2821330141 conn(0x557dcc290f20 msgr2=0x557dcc4051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:01.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/2821330141 shutdown_connections 2024-05-09T05:39:01.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/2821330141 wait complete. 2024-05-09T05:39:01.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 -- start start 2024-05-09T05:39:01.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.810+0000 7f1ef98f8b80 1 Processor -- start 2024-05-09T05:39:01.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc28d450 0x557dcc414040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc307cb0 0x557dcc414580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc414ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557dcc40b110 con 0x557dcc40a3b0 2024-05-09T05:39:01.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557dcc2c7ad0 con 0x557dcc28d450 2024-05-09T05:39:01.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557dcc416fd0 con 0x557dcc307cb0 2024-05-09T05:39:01.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ef98f8b80 10 client.0 _ll_register_callbacks cb 0x557dcc19e670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:01.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee947f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc414ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee947f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc414ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42538/0 (socket says 172.21.15.115:42538) 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee947f700 1 -- 172.21.15.115:0/3549383104 learned_addr learned my addr 172.21.15.115:0/3549383104 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc28d450 0x557dcc414040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1edbfff700 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc307cb0 0x557dcc414580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee947f700 1 -- 172.21.15.115:0/3549383104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc307cb0 msgr2=0x557dcc414580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc307cb0 0x557dcc414580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee947f700 1 -- 172.21.15.115:0/3549383104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc28d450 msgr2=0x557dcc414040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc28d450 0x557dcc414040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee947f700 1 -- 172.21.15.115:0/3549383104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dcc1a8440 con 0x557dcc40a3b0 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc414ac0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0a6a50 tx=0x7f1edc0e0180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf90e36f245f8ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.811+0000 7f1ec3fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3549383104 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0a92f0 con 0x557dcc40a3b0 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3549383104 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1edc0b23c0 con 0x557dcc40a3b0 2024-05-09T05:39:01.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3549383104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed40044c0 con 0x557dcc40a3b0 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3549383104 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1edc0a9ee0 con 0x557dcc40a3b0 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ef98f8b80 10 client.15654 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3549383104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dcc421b00 con 0x557dcc40a3b0 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3549383104 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ed40044c0 con 0x557dcc40a3b0 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3549383104 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1edc0aab10 con 0x557dcc40a3b0 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ef98f8b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ef98f8b80 10 client.15654 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ef98f8b80 20 client.15654 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x557dcc424070 0x557dcc4254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ef98f8b80 4 client.15654 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.812+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3549383104 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dcc33aa30 con 0x557dcc424070 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.813+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x557dcc424070 0x557dcc4254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.813+0000 7f1ec3fff700 10 client.15654 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.813+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x557dcc424070 0x557dcc4254e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3cf145d9947d80d4 server_cookie=687a2778b583c5be in_seq=0 out_seq=0 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3549383104 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557dcc33aa30 con 0x557dcc424070 2024-05-09T05:39:01.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ec3fff700 10 client.15654 handle_command_reply: tid=0 2024-05-09T05:39:01.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 client.15654 shutdown 2024-05-09T05:39:01.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3549383104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 msgr2=0x557dcc414ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:01.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc414ac0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0a6a50 tx=0x7f1edc0e0180 comp rx=0 tx=0).stop 2024-05-09T05:39:01.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3549383104 shutdown_connections 2024-05-09T05:39:01.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc28d450 0x557dcc414040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x557dcc424070 0x557dcc4254e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc414ac0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3549383104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc307cb0 0x557dcc414580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3549383104 >> 172.21.15.115:0/3549383104 conn(0x557dcc290f20 msgr2=0x557dcc3fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ec3fff700 0 client.15654 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3549383104 shutdown_connections 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3549383104 wait complete. 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.814+0000 7f1ef98f8b80 20 client.15654 trim_cache size 0 max 16384 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.835+0000 7f1ef98f8b80 1 Processor -- start 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.835+0000 7f1ef98f8b80 1 -- start start 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 0x557dcc40f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc307cb0 0x557dcc420230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc40a3b0 0x557dcc414900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557dcc423820 con 0x557dcc307cb0 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557dcc423ef0 con 0x557dcc40a3b0 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557dcc4245c0 con 0x557dcc28d450 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1edbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc307cb0 0x557dcc420230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1edbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc307cb0 0x557dcc420230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42550/0 (socket says 172.21.15.115:42550) 2024-05-09T05:39:01.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1edbfff700 1 -- 172.21.15.115:0/3965724888 learned_addr learned my addr 172.21.15.115:0/3965724888 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 0x557dcc40f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc40a3b0 0x557dcc414900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1edbfff700 1 -- 172.21.15.115:0/3965724888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 msgr2=0x557dcc40f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1edbfff700 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 0x557dcc40f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1edbfff700 1 -- 172.21.15.115:0/3965724888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc40a3b0 msgr2=0x557dcc414900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1edbfff700 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc40a3b0 0x557dcc414900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1edbfff700 1 -- 172.21.15.115:0/3965724888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcc401980 con 0x557dcc307cb0 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 0x557dcc40f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.836+0000 7f1edbfff700 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc307cb0 0x557dcc420230 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4019000 tx=0x7f1ed400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e82cb2f800552b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.837+0000 7f1eca7fc700 1 -- 172.21.15.115:0/3965724888 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4005070 con 0x557dcc307cb0 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.837+0000 7f1eca7fc700 1 -- 172.21.15.115:0/3965724888 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ed40056a0 con 0x557dcc307cb0 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.837+0000 7f1eca7fc700 1 -- 172.21.15.115:0/3965724888 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4011440 con 0x557dcc307cb0 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3965724888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc307cb0 msgr2=0x557dcc420230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc307cb0 0x557dcc420230 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4019000 tx=0x7f1ed400b630 comp rx=0 tx=0).stop 2024-05-09T05:39:01.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3965724888 shutdown_connections 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc40a3b0 0x557dcc414900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 0x557dcc40f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3965724888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc307cb0 0x557dcc420230 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3965724888 >> 172.21.15.115:0/3965724888 conn(0x557dcc290f20 msgr2=0x557dcc40f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3965724888 shutdown_connections 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3965724888 wait complete. 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 -- start start 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.838+0000 7f1ef98f8b80 1 Processor -- start 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 0x557dcc42ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 0x557dcc410860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ef98f8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc410da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557dcc425360 con 0x557dcc40a3b0 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557dcc424c90 con 0x557dcc307cb0 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ef98f8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557dcc425a30 con 0x557dcc28d450 2024-05-09T05:39:01.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ef98f8b80 10 client.0 _ll_register_callbacks cb 0x557dcc19e670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee947f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc410da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee947f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc410da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42562/0 (socket says 172.21.15.115:42562) 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee947f700 1 -- 172.21.15.115:0/3579507772 learned_addr learned my addr 172.21.15.115:0/3579507772 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 0x557dcc42ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1edbfff700 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 0x557dcc410860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee947f700 1 -- 172.21.15.115:0/3579507772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 msgr2=0x557dcc42ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 0x557dcc42ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee947f700 1 -- 172.21.15.115:0/3579507772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 msgr2=0x557dcc410860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 0x557dcc410860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee947f700 1 -- 172.21.15.115:0/3579507772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dcc4112e0 con 0x557dcc40a3b0 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ee947f700 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc410da0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007d40 tx=0x7f1edc0dfcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88e2a4698be1cf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:01.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ec3fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.839+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3579507772 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0a92f0 con 0x557dcc40a3b0 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3579507772 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1edc0910b0 con 0x557dcc40a3b0 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3579507772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed40056a0 con 0x557dcc40a3b0 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3579507772 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1edc0a97e0 con 0x557dcc40a3b0 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ef98f8b80 10 client.15660 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3579507772 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dcc494140 con 0x557dcc40a3b0 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3579507772 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ed40056a0 con 0x557dcc40a3b0 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3579507772 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1edc004ad0 con 0x557dcc40a3b0 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ef98f8b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ef98f8b80 10 client.15660 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ef98f8b80 20 client.15660 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x557dcc4964a0 0x557dcc497910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ef98f8b80 4 client.15660 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3579507772 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "status"}) -- 0x557dcc334ad0 con 0x557dcc4964a0 2024-05-09T05:39:01.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.840+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x557dcc4964a0 0x557dcc497910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ec3fff700 10 client.15660 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ee8c7e700 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x557dcc4964a0 0x557dcc497910 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c93f60c51ada26dd server_cookie=9f0a49903ca2b13 in_seq=0 out_seq=0 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ec3fff700 1 -- 172.21.15.115:0/3579507772 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557dcc334ad0 con 0x557dcc4964a0 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ec3fff700 10 client.15660 handle_command_reply: tid=0 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 client.15660 shutdown 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3579507772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 msgr2=0x557dcc410da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc410da0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007d40 tx=0x7f1edc0dfcf0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3579507772 shutdown_connections 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x557dcc4964a0 0x557dcc497910 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557dcc40a3b0 0x557dcc410da0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557dcc28d450 0x557dcc42ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 --2- 172.21.15.115:0/3579507772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcc307cb0 0x557dcc410860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3579507772 >> 172.21.15.115:0/3579507772 conn(0x557dcc290f20 msgr2=0x557dcc40dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ec3fff700 0 client.15660 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:01.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.841+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3579507772 shutdown_connections 2024-05-09T05:39:01.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1ef98f8b80 1 -- 172.21.15.115:0/3579507772 wait complete. 2024-05-09T05:39:01.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1ef98f8b80 20 client.15660 trim_cache size 0 max 16384 2024-05-09T05:39:01.897 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:39:01.897 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:39:01.897 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 2, 2024-05-09T05:39:01.897 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24451, 2024-05-09T05:39:01.897 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:39:01.897 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:39:01.897 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:39:01.898 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 241.57748959200001, 2024-05-09T05:39:01.898 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 30, 2024-05-09T05:39:01.898 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 88, 2024-05-09T05:39:01.898 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:39:01.898 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 341.29411547799998 2024-05-09T05:39:01.898 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 -- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1ecc0b2b00 msgr2=0x7f1ecc0b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1ecc0b2b00 0x7f1ecc0b5010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00058e0 tx=0x7f1ee0005ab0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 -- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4108970 msgr2=0x7f1ee419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4108970 0x7f1ee419c850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007d70 tx=0x7f1edc00a340 comp rx=0 tx=0).stop 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 -- 172.21.15.115:0/3863167347 shutdown_connections 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1ecc0b2b00 0x7f1ecc0b5010 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee4108970 0x7f1ee419c850 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee4103c10 0x7f1ee419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 --2- 172.21.15.115:0/3863167347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ee4108020 0x7f1ee419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 -- 172.21.15.115:0/3863167347 >> 172.21.15.115:0/3863167347 conn(0x7f1ee40fe520 msgr2=0x7f1ee4100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.842+0000 7f1eeaee2700 1 -- 172.21.15.115:0/3863167347 shutdown_connections 2024-05-09T05:39:01.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:01.843+0000 7f1eeaee2700 1 -- 172.21.15.115:0/3863167347 wait complete. 2024-05-09T05:39:02.649 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T05:39:02.891 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:03.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:03 smithi115 ceph-mon[30349]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-09T05:39:03.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:03 smithi115 ceph-mon[30349]: mgrmap e24: x(active, since 50s), standbys: y, z 2024-05-09T05:39:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:03 smithi161 ceph-mon[33805]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-09T05:39:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:03 smithi161 ceph-mon[33805]: mgrmap e24: x(active, since 50s), standbys: y, z 2024-05-09T05:39:03.663 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:39:03.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:03 smithi195 ceph-mon[33844]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-09T05:39:03.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:03 smithi195 ceph-mon[33844]: mgrmap e24: x(active, since 50s), standbys: y, z 2024-05-09T05:39:04.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.834+0000 7f9687ee6700 1 -- 172.21.15.115:0/2937568973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 msgr2=0x7f9680109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:04.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.834+0000 7f9687ee6700 1 --2- 172.21.15.115:0/2937568973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 0x7f9680109330 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f96700099c0 tx=0x7f9670009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.834+0000 7f9687ee6700 1 -- 172.21.15.115:0/2937568973 shutdown_connections 2024-05-09T05:39:04.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.834+0000 7f9687ee6700 1 --2- 172.21.15.115:0/2937568973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9680109870 0x7f96801111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.834+0000 7f9687ee6700 1 --2- 172.21.15.115:0/2937568973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96801045f0 0x7f9680104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.834+0000 7f9687ee6700 1 --2- 172.21.15.115:0/2937568973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 0x7f9680109330 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.834+0000 7f9687ee6700 1 -- 172.21.15.115:0/2937568973 >> 172.21.15.115:0/2937568973 conn(0x7f96800fffe0 msgr2=0x7f9680102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:04.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.834+0000 7f9687ee6700 1 -- 172.21.15.115:0/2937568973 shutdown_connections 2024-05-09T05:39:04.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.834+0000 7f9687ee6700 1 -- 172.21.15.115:0/2937568973 wait complete. 2024-05-09T05:39:04.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.835+0000 7f9687ee6700 1 Processor -- start 2024-05-09T05:39:04.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.835+0000 7f9687ee6700 1 -- start start 2024-05-09T05:39:04.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.835+0000 7f9687ee6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96801045f0 0x7f968019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.835+0000 7f9687ee6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 0x7f968019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.835+0000 7f9687ee6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9680109870 0x7f968019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9687ee6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9680114330 con 0x7f9680108f20 2024-05-09T05:39:04.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9687ee6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f96801141b0 con 0x7f96801045f0 2024-05-09T05:39:04.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9687ee6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96801144b0 con 0x7f9680109870 2024-05-09T05:39:04.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685481700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 0x7f968019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685481700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 0x7f968019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33234/0 (socket says 172.21.15.115:33234) 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685481700 1 -- 172.21.15.115:0/4251412098 learned_addr learned my addr 172.21.15.115:0/4251412098 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9686483700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9680109870 0x7f968019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685c82700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96801045f0 0x7f968019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685481700 1 -- 172.21.15.115:0/4251412098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9680109870 msgr2=0x7f968019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685481700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9680109870 0x7f968019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685481700 1 -- 172.21.15.115:0/4251412098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96801045f0 msgr2=0x7f968019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685481700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96801045f0 0x7f968019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685481700 1 -- 172.21.15.115:0/4251412098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968019fc30 con 0x7f9680108f20 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9686483700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9680109870 0x7f968019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685c82700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96801045f0 0x7f968019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.836+0000 7f9685481700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 0x7f968019b300 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f967000b580 tx=0x7f967000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.837+0000 7f9676ffd700 1 -- 172.21.15.115:0/4251412098 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670027070 con 0x7f9680108f20 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.837+0000 7f9687ee6700 1 -- 172.21.15.115:0/4251412098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968019fec0 con 0x7f9680108f20 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.837+0000 7f9676ffd700 1 -- 172.21.15.115:0/4251412098 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f967000fda0 con 0x7f9680108f20 2024-05-09T05:39:04.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.837+0000 7f9676ffd700 1 -- 172.21.15.115:0/4251412098 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96700173f0 con 0x7f9680108f20 2024-05-09T05:39:04.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.837+0000 7f9687ee6700 1 -- 172.21.15.115:0/4251412098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96801a7840 con 0x7f9680108f20 2024-05-09T05:39:04.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.841+0000 7f9676ffd700 1 -- 172.21.15.115:0/4251412098 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f967002a070 con 0x7f9680108f20 2024-05-09T05:39:04.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f96968fcb80 1 Processor -- start 2024-05-09T05:39:04.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f96968fcb80 1 -- start start 2024-05-09T05:39:04.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f9676ffd700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f966c0b2b90 0x7f966c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 0x5644f27b7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f27b13b0 0x5644f27b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f96968fcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644f26d4050 con 0x5644f27b13b0 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f96968fcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644f2679950 con 0x5644f26aecb0 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f96968fcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644f26d0010 con 0x5644f2634450 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f9686483700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f966c0b2b90 0x7f966c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f9676ffd700 1 -- 172.21.15.115:0/4251412098 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f96700e3a00 con 0x7f9680108f20 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f9685c82700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f9686483700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f27b13b0 0x5644f27b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f9685c82700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:41244/0 (socket says 172.21.15.115:41244) 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.842+0000 7f9685c82700 1 -- 172.21.15.115:0/2795539601 learned_addr learned my addr 172.21.15.115:0/2795539601 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f9685481700 1 --2- 172.21.15.115:0/2795539601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 0x5644f27b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f9686483700 1 -- 172.21.15.115:0/2795539601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 msgr2=0x5644f27b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f9686483700 1 --2- 172.21.15.115:0/2795539601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 0x5644f27b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f9686483700 1 -- 172.21.15.115:0/2795539601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 msgr2=0x5644f27b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f9686483700 1 --2- 172.21.15.115:0/2795539601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f9686483700 1 -- 172.21.15.115:0/2795539601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f266a4b0 con 0x5644f27b13b0 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f9686483700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f966c0b2b90 0x7f966c0b50a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9678007b90 tx=0x7f9678009f90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f9686483700 1 --2- 172.21.15.115:0/2795539601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f27b13b0 0x5644f27b17c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9678018a10 tx=0x7f967801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9496f9a03862f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f966bfff700 1 -- 172.21.15.115:0/2795539601 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678021070 con 0x5644f27b13b0 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f966bfff700 1 -- 172.21.15.115:0/2795539601 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9678004db0 con 0x5644f27b13b0 2024-05-09T05:39:04.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.843+0000 7f966bfff700 1 -- 172.21.15.115:0/2795539601 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678012040 con 0x5644f27b13b0 2024-05-09T05:39:04.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.844+0000 7f96968fcb80 1 -- 172.21.15.115:0/2795539601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f27b13b0 msgr2=0x5644f27b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:04.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.844+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2795539601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f27b13b0 0x5644f27b17c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9678018a10 tx=0x7f967801b000 comp rx=0 tx=0).stop 2024-05-09T05:39:04.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.845+0000 7f96968fcb80 1 -- 172.21.15.115:0/2795539601 shutdown_connections 2024-05-09T05:39:04.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.845+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2795539601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.845+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2795539601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f27b13b0 0x5644f27b17c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.845+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2795539601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 0x5644f27b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.845+0000 7f96968fcb80 1 -- 172.21.15.115:0/2795539601 >> 172.21.15.115:0/2795539601 conn(0x5644f2637f20 msgr2=0x5644f27ac1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.845+0000 7f96968fcb80 1 -- 172.21.15.115:0/2795539601 shutdown_connections 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.845+0000 7f96968fcb80 1 -- 172.21.15.115:0/2795539601 wait complete. 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.845+0000 7f96968fcb80 1 -- start start 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.845+0000 7f96968fcb80 1 Processor -- start 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27bb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 0x5644f27bbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f96968fcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644f27b20b0 con 0x5644f2634450 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f96968fcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644f266ead0 con 0x5644f26aecb0 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f9685c82700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f96968fcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644f27cb590 con 0x5644f27b13b0 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f9685c82700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33250/0 (socket says 172.21.15.115:33250) 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f9685c82700 1 -- 172.21.15.115:0/883408189 learned_addr learned my addr 172.21.15.115:0/883408189 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f9686483700 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 0x5644f27bbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.846+0000 7f96968fcb80 10 client.0 _ll_register_callbacks cb 0x5644f2545670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f9685c82700 1 -- 172.21.15.115:0/883408189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 msgr2=0x5644f27bbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f9685c82700 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 0x5644f27bbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f9685c82700 1 -- 172.21.15.115:0/883408189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 msgr2=0x5644f27bb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f9685c82700 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27bb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f9685c82700 1 -- 172.21.15.115:0/883408189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f254f440 con 0x5644f2634450 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f9685c82700 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb030 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f967c012500 tx=0x7f967c012810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebee9a14cbd4cb91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f96697fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f96697fa700 1 -- 172.21.15.115:0/883408189 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c004230 con 0x5644f2634450 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f96968fcb80 1 -- 172.21.15.115:0/883408189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9678004db0 con 0x5644f2634450 2024-05-09T05:39:04.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f96697fa700 1 -- 172.21.15.115:0/883408189 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f967c0043d0 con 0x5644f2634450 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f96697fa700 1 -- 172.21.15.115:0/883408189 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f967c003680 con 0x5644f2634450 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f96968fcb80 10 client.15672 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.847+0000 7f96968fcb80 1 -- 172.21.15.115:0/883408189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644f27ccfa0 con 0x5644f2634450 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.848+0000 7f96697fa700 1 -- 172.21.15.115:0/883408189 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9678004db0 con 0x5644f2634450 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.848+0000 7f96697fa700 1 -- 172.21.15.115:0/883408189 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f967c010950 con 0x5644f2634450 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.848+0000 7f96968fcb80 10 client.15672 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.848+0000 7f96968fcb80 10 client.15672 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.848+0000 7f96968fcb80 20 client.15672 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.848+0000 7f96968fcb80 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5644f27cf650 0x5644f27d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.848+0000 7f96968fcb80 4 client.15672 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.848+0000 7f96968fcb80 1 -- 172.21.15.115:0/883408189 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644f26e1a30 con 0x5644f27cf650 2024-05-09T05:39:04.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.848+0000 7f9685481700 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5644f27cf650 0x5644f27d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.849+0000 7f9685481700 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5644f27cf650 0x5644f27d0ac0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c73a7308ffc63873 server_cookie=993b7b383b41f08e in_seq=0 out_seq=0 2024-05-09T05:39:04.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.849+0000 7f96697fa700 10 client.15672 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:04.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96697fa700 1 -- 172.21.15.115:0/883408189 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644f26e1a30 con 0x5644f27cf650 2024-05-09T05:39:04.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96697fa700 10 client.15672 handle_command_reply: tid=0 2024-05-09T05:39:04.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 client.15672 shutdown 2024-05-09T05:39:04.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 -- 172.21.15.115:0/883408189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 msgr2=0x5644f27bb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:04.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb030 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f967c012500 tx=0x7f967c012810 comp rx=0 tx=0).stop 2024-05-09T05:39:04.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 -- 172.21.15.115:0/883408189 shutdown_connections 2024-05-09T05:39:04.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5644f27cf650 0x5644f27d0ac0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 0x5644f27bbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb030 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 --2- 172.21.15.115:0/883408189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27bb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 -- 172.21.15.115:0/883408189 >> 172.21.15.115:0/883408189 conn(0x5644f2637f20 msgr2=0x5644f27a5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:04.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96697fa700 0 client.15672 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:04.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 -- 172.21.15.115:0/883408189 shutdown_connections 2024-05-09T05:39:04.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 1 -- 172.21.15.115:0/883408189 wait complete. 2024-05-09T05:39:04.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.850+0000 7f96968fcb80 20 client.15672 trim_cache size 0 max 16384 2024-05-09T05:39:04.852 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:04.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.872+0000 7f96968fcb80 1 Processor -- start 2024-05-09T05:39:04.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.872+0000 7f96968fcb80 1 -- start start 2024-05-09T05:39:04.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.872+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.872+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.872+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 0x5644f27b9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.872+0000 7f96968fcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644f27cee00 con 0x5644f2634450 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.872+0000 7f96968fcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644f27cf4d0 con 0x5644f26aecb0 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.872+0000 7f96968fcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644f27cfba0 con 0x5644f27b13b0 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685c82700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685c82700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33266/0 (socket says 172.21.15.115:33266) 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685c82700 1 -- 172.21.15.115:0/2029363406 learned_addr learned my addr 172.21.15.115:0/2029363406 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685481700 1 --2- 172.21.15.115:0/2029363406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9686483700 1 --2- 172.21.15.115:0/2029363406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 0x5644f27b9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685c82700 1 -- 172.21.15.115:0/2029363406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 msgr2=0x5644f27b9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685c82700 1 --2- 172.21.15.115:0/2029363406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 0x5644f27b9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685c82700 1 -- 172.21.15.115:0/2029363406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 msgr2=0x5644f27cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685c82700 1 --2- 172.21.15.115:0/2029363406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685c82700 1 -- 172.21.15.115:0/2029363406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f27a8980 con 0x5644f2634450 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f9685c82700 1 --2- 172.21.15.115:0/2029363406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f967c010f60 tx=0x7f967c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f944a9e230e265e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.873+0000 7f966b7fe700 1 -- 172.21.15.115:0/2029363406 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c0107e0 con 0x5644f2634450 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.874+0000 7f966b7fe700 1 -- 172.21.15.115:0/2029363406 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f967c010980 con 0x5644f2634450 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.874+0000 7f966b7fe700 1 -- 172.21.15.115:0/2029363406 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c0049e0 con 0x5644f2634450 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 -- 172.21.15.115:0/2029363406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 msgr2=0x5644f27bb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2029363406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f967c010f60 tx=0x7f967c005a00 comp rx=0 tx=0).stop 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 -- 172.21.15.115:0/2029363406 shutdown_connections 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2029363406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f27b13b0 0x5644f27b9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2029363406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f2634450 0x5644f27bb8f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2029363406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f26aecb0 0x5644f27cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 -- 172.21.15.115:0/2029363406 >> 172.21.15.115:0/2029363406 conn(0x5644f2637f20 msgr2=0x5644f27b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:04.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 -- 172.21.15.115:0/2029363406 shutdown_connections 2024-05-09T05:39:04.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 -- 172.21.15.115:0/2029363406 wait complete. 2024-05-09T05:39:04.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 -- start start 2024-05-09T05:39:04.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.875+0000 7f96968fcb80 1 Processor -- start 2024-05-09T05:39:04.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 0x5644f27d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f26aecb0 0x5644f27d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f96968fcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f27b13b0 0x5644f27d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f96968fcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644f27d0940 con 0x5644f26aecb0 2024-05-09T05:39:04.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f96968fcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644f27d0270 con 0x5644f27b13b0 2024-05-09T05:39:04.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f96968fcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644f27d1010 con 0x5644f2634450 2024-05-09T05:39:04.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f96968fcb80 10 client.0 _ll_register_callbacks cb 0x5644f2545670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:04.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9685481700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f26aecb0 0x5644f27d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9685481700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f26aecb0 0x5644f27d4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33272/0 (socket says 172.21.15.115:33272) 2024-05-09T05:39:04.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9685481700 1 -- 172.21.15.115:0/2092989245 learned_addr learned my addr 172.21.15.115:0/2092989245 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:04.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9686483700 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f27b13b0 0x5644f27d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9685c82700 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 0x5644f27d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9685481700 1 -- 172.21.15.115:0/2092989245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 msgr2=0x5644f27d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9685481700 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 0x5644f27d3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9685481700 1 -- 172.21.15.115:0/2092989245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f27b13b0 msgr2=0x5644f27d47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9685481700 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f27b13b0 0x5644f27d47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.876+0000 7f9685481700 1 -- 172.21.15.115:0/2092989245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f27ade10 con 0x5644f26aecb0 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.877+0000 7f9685481700 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f26aecb0 0x5644f27d4240 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9670000c00 tx=0x7f967000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca02be8bcc54b5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.877+0000 7f96697fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.877+0000 7f96697fa700 1 -- 172.21.15.115:0/2092989245 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96700aa360 con 0x5644f26aecb0 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.877+0000 7f96697fa700 1 -- 172.21.15.115:0/2092989245 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f96700b10b0 con 0x5644f26aecb0 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.877+0000 7f96968fcb80 1 -- 172.21.15.115:0/2092989245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f967c010980 con 0x5644f26aecb0 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.877+0000 7f96697fa700 1 -- 172.21.15.115:0/2092989245 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96700aaf50 con 0x5644f26aecb0 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.877+0000 7f96968fcb80 10 client.15678 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:04.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.877+0000 7f96968fcb80 1 -- 172.21.15.115:0/2092989245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644f283adb0 con 0x5644f26aecb0 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.877+0000 7f96697fa700 1 -- 172.21.15.115:0/2092989245 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96700aa850 con 0x5644f26aecb0 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.878+0000 7f96697fa700 1 -- 172.21.15.115:0/2092989245 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f96700aabb0 con 0x5644f26aecb0 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.878+0000 7f96968fcb80 10 client.15678 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.878+0000 7f96968fcb80 10 client.15678 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.878+0000 7f96968fcb80 20 client.15678 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.878+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5644f283d280 0x5644f283e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.878+0000 7f96968fcb80 4 client.15678 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.878+0000 7f96968fcb80 1 -- 172.21.15.115:0/2092989245 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "status"}) -- 0x7f967c010980 con 0x5644f283d280 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.878+0000 7f9685c82700 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5644f283d280 0x5644f283e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96697fa700 10 client.15678 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f9685c82700 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5644f283d280 0x5644f283e6f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93e1ecafdb6ede1c server_cookie=65e4443cc2e85fbf in_seq=0 out_seq=0 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96697fa700 1 -- 172.21.15.115:0/2092989245 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x7f967c010980 con 0x5644f283d280 2024-05-09T05:39:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96697fa700 10 client.15678 handle_command_reply: tid=0 2024-05-09T05:39:04.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 client.15678 shutdown 2024-05-09T05:39:04.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 -- 172.21.15.115:0/2092989245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f26aecb0 msgr2=0x5644f27d4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:04.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f26aecb0 0x5644f27d4240 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9670000c00 tx=0x7f967000b5b0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 -- 172.21.15.115:0/2092989245 shutdown_connections 2024-05-09T05:39:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5644f283d280 0x5644f283e6f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644f27b13b0 0x5644f27d47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644f2634450 0x5644f27d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 --2- 172.21.15.115:0/2092989245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644f26aecb0 0x5644f27d4240 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96697fa700 0 client.15678 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 -- 172.21.15.115:0/2092989245 >> 172.21.15.115:0/2092989245 conn(0x5644f2637f20 msgr2=0x5644f27a5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 -- 172.21.15.115:0/2092989245 shutdown_connections 2024-05-09T05:39:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 1 -- 172.21.15.115:0/2092989245 wait complete. 2024-05-09T05:39:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.879+0000 7f96968fcb80 20 client.15678 trim_cache size 0 max 16384 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 1, 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24457, 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 245.620221851, 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 30, 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 88, 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 341.53295997999999 2024-05-09T05:39:04.879 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:39:04.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.880+0000 7f9687ee6700 1 -- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f966c0b2b90 msgr2=0x7f966c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:04.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.880+0000 7f9687ee6700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f966c0b2b90 0x7f966c0b50a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9678007b90 tx=0x7f9678009f90 comp rx=0 tx=0).stop 2024-05-09T05:39:04.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.880+0000 7f9687ee6700 1 -- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 msgr2=0x7f968019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:04.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.880+0000 7f9687ee6700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 0x7f968019b300 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f967000b580 tx=0x7f967000f460 comp rx=0 tx=0).stop 2024-05-09T05:39:04.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.881+0000 7f9687ee6700 1 -- 172.21.15.115:0/4251412098 shutdown_connections 2024-05-09T05:39:04.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.881+0000 7f9687ee6700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f966c0b2b90 0x7f966c0b50a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.881+0000 7f9687ee6700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96801045f0 0x7f968019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.881+0000 7f9687ee6700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9680109870 0x7f968019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.881+0000 7f9687ee6700 1 --2- 172.21.15.115:0/4251412098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9680108f20 0x7f968019b300 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:04.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.881+0000 7f9687ee6700 1 -- 172.21.15.115:0/4251412098 >> 172.21.15.115:0/4251412098 conn(0x7f96800fffe0 msgr2=0x7f96801070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:04.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.881+0000 7f9687ee6700 1 -- 172.21.15.115:0/4251412098 shutdown_connections 2024-05-09T05:39:04.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:04.881+0000 7f9687ee6700 1 -- 172.21.15.115:0/4251412098 wait complete. 2024-05-09T05:39:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:05 smithi115 ceph-mon[30349]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-05-09T05:39:05.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:05 smithi161 ceph-mon[33805]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-05-09T05:39:05.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:05 smithi195 ceph-mon[33844]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-05-09T05:39:05.982 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T05:39:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.084+0000 7f7e1fb38640 1 -- 172.21.15.115:0/532089607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e181032f0 msgr2=0x7f7e1810f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.084+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/532089607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e181032f0 0x7f7e1810f6c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009ca0 tx=0x7f7e1002a8c0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.084+0000 7f7e1fb38640 1 -- 172.21.15.115:0/532089607 shutdown_connections 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.084+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/532089607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e181032f0 0x7f7e1810f6c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.084+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/532089607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18109040 0x7f7e18102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.084+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/532089607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e18108ba0 0x7f7e18102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.084+0000 7f7e1fb38640 1 -- 172.21.15.115:0/532089607 >> 172.21.15.115:0/532089607 conn(0x7f7e180782e0 msgr2=0x7f7e18078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.084+0000 7f7e1fb38640 1 -- 172.21.15.115:0/532089607 shutdown_connections 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.084+0000 7f7e1fb38640 1 -- 172.21.15.115:0/532089607 wait complete. 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1fb38640 1 Processor -- start 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1fb38640 1 -- start start 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1fb38640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e181032f0 0x7f7e1819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1fb38640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18108ba0 0x7f7e1819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1fb38640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e18109040 0x7f7e1819c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1fb38640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18112cb0 con 0x7f7e18109040 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1fb38640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18112b30 con 0x7f7e181032f0 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1fb38640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18112e30 con 0x7f7e18108ba0 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1e0d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e18109040 0x7f7e1819c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1e0d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e18109040 0x7f7e1819c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33284/0 (socket says 172.21.15.115:33284) 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1e0d5640 1 -- 172.21.15.115:0/907640328 learned_addr learned my addr 172.21.15.115:0/907640328 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.085+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18108ba0 0x7f7e1819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e181032f0 0x7f7e1819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e1e0d5640 1 -- 172.21.15.115:0/907640328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18108ba0 msgr2=0x7f7e1819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e1e0d5640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18108ba0 0x7f7e1819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e1e0d5640 1 -- 172.21.15.115:0/907640328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e181032f0 msgr2=0x7f7e1819b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e1e0d5640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e181032f0 0x7f7e1819b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e1e0d5640 1 -- 172.21.15.115:0/907640328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e181a6890 con 0x7f7e18109040 2024-05-09T05:39:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e1e0d5640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e18109040 0x7f7e1819c090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7e10005cc0 tx=0x7f7e10032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e0affd640 1 -- 172.21.15.115:0/907640328 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10004040 con 0x7f7e18109040 2024-05-09T05:39:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e1fb38640 1 -- 172.21.15.115:0/907640328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e181a6b20 con 0x7f7e18109040 2024-05-09T05:39:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e0affd640 1 -- 172.21.15.115:0/907640328 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e10004670 con 0x7f7e18109040 2024-05-09T05:39:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e0affd640 1 -- 172.21.15.115:0/907640328 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10030cc0 con 0x7f7e18109040 2024-05-09T05:39:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.086+0000 7f7e1fb38640 1 -- 172.21.15.115:0/907640328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181a70c0 con 0x7f7e18109040 2024-05-09T05:39:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.089+0000 7f7e269ec740 1 Processor -- start 2024-05-09T05:39:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.089+0000 7f7e0affd640 1 -- 172.21.15.115:0/907640328 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f7e100041e0 con 0x7f7e18109040 2024-05-09T05:39:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.089+0000 7f7e269ec740 1 -- start start 2024-05-09T05:39:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.089+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 0x56546e8d8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.089+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8d8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8d9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e269ec740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56546e83d7c0 con 0x56546e5144d0 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e269ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56546e7d90e0 con 0x56546e5351b0 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e269ec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56546e839780 con 0x56546e50c2f0 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d0d3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d0d3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8d9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33288/0 (socket says 172.21.15.115:33288) 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d0d3640 1 -- 172.21.15.115:0/3824860544 learned_addr learned my addr 172.21.15.115:0/3824860544 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8d8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 0x56546e8d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e0affd640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e040b2a70 0x7f7e040b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/3824860544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 msgr2=0x56546e8d8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 0x56546e8d8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/3824860544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 msgr2=0x56546e8d9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8d9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/3824860544 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546e8027c0 con 0x56546e50c2f0 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8d9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1e0d5640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e040b2a70 0x7f7e040b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e0affd640 1 -- 172.21.15.115:0/907640328 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7e10104340 con 0x7f7e18109040 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.090+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8d8bc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400cb70 tx=0x7f7e1400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f5dc90d8f00b710 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.091+0000 7f7e03fff640 1 -- 172.21.15.115:0/3824860544 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14004220 con 0x56546e50c2f0 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.091+0000 7f7e1e0d5640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e040b2a70 0x7f7e040b4f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7e10101740 tx=0x7f7e10098d70 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.091+0000 7f7e03fff640 1 -- 172.21.15.115:0/3824860544 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e140043c0 con 0x56546e50c2f0 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.091+0000 7f7e03fff640 1 -- 172.21.15.115:0/3824860544 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14004b50 con 0x56546e50c2f0 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.091+0000 7f7e269ec740 1 -- 172.21.15.115:0/3824860544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 msgr2=0x56546e8d8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.091+0000 7f7e269ec740 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8d8bc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400cb70 tx=0x7f7e1400e720 comp rx=0 tx=0).stop 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.092+0000 7f7e269ec740 1 -- 172.21.15.115:0/3824860544 shutdown_connections 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.092+0000 7f7e269ec740 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8d9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.092+0000 7f7e269ec740 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8d8bc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.092+0000 7f7e269ec740 1 --2- 172.21.15.115:0/3824860544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 0x56546e8d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.092+0000 7f7e269ec740 1 -- 172.21.15.115:0/3824860544 >> 172.21.15.115:0/3824860544 conn(0x56546e529300 msgr2=0x56546e8d4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.092+0000 7f7e269ec740 1 -- 172.21.15.115:0/3824860544 shutdown_connections 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.092+0000 7f7e269ec740 1 -- 172.21.15.115:0/3824860544 wait complete. 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.092+0000 7f7e269ec740 1 -- start start 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.092+0000 7f7e269ec740 1 Processor -- start 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e50c2f0 0x56546e8e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e5144d0 0x56546e8e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e269ec740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56546e8e57b0 con 0x56546e5351b0 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e269ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56546e7c49c0 con 0x56546e50c2f0 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e269ec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56546e8e5930 con 0x56546e5144d0 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e269ec740 10 client.0 _ll_register_callbacks cb 0x56546e7f1020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8e2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33304/0 (socket says 172.21.15.115:33304) 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 -- 172.21.15.115:0/2457015168 learned_addr learned my addr 172.21.15.115:0/2457015168 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e50c2f0 0x56546e8e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e5144d0 0x56546e8e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 -- 172.21.15.115:0/2457015168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e5144d0 msgr2=0x56546e8e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e5144d0 0x56546e8e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 -- 172.21.15.115:0/2457015168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e50c2f0 msgr2=0x56546e8e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e50c2f0 0x56546e8e46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 -- 172.21.15.115:0/2457015168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56546e7eb1f0 con 0x56546e5351b0 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8e2b40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0058c0 tx=0x7f7e0c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8041c808bc96cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e017fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.093+0000 7f7e017fa640 1 -- 172.21.15.115:0/2457015168 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c011040 con 0x56546e5351b0 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e017fa640 1 -- 172.21.15.115:0/2457015168 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7e0c0111e0 con 0x56546e5351b0 2024-05-09T05:39:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e269ec740 1 -- 172.21.15.115:0/2457015168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e140043c0 con 0x56546e5351b0 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e017fa640 1 -- 172.21.15.115:0/2457015168 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e0c029050 con 0x56546e5351b0 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e269ec740 10 client.15690 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e269ec740 1 -- 172.21.15.115:0/2457015168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56546e5a0790 con 0x56546e5351b0 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e017fa640 1 -- 172.21.15.115:0/2457015168 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e140043c0 con 0x56546e5351b0 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e017fa640 1 -- 172.21.15.115:0/2457015168 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7e0c00dac0 con 0x56546e5351b0 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e269ec740 10 client.15690 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e269ec740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e269ec740 20 client.15690 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e269ec740 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56546e4f3d80 0x56546e8f5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e269ec740 4 client.15690 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.094+0000 7f7e269ec740 1 -- 172.21.15.115:0/2457015168 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56546e84b3d0 con 0x56546e4f3d80 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.095+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56546e4f3d80 0x56546e8f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.095+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56546e4f3d80 0x56546e8f5030 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1cc096b2dd033ad server_cookie=95a91d68ccf36c0 in_seq=0 out_seq=0 2024-05-09T05:39:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.095+0000 7f7e017fa640 10 client.15690 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e017fa640 1 -- 172.21.15.115:0/2457015168 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56546e84b3d0 con 0x56546e4f3d80 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e017fa640 10 client.15690 handle_command_reply: tid=0 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 client.15690 shutdown 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 -- 172.21.15.115:0/2457015168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 msgr2=0x56546e8e2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8e2b40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0058c0 tx=0x7f7e0c013040 comp rx=0 tx=0).stop 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 -- 172.21.15.115:0/2457015168 shutdown_connections 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8e2b40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56546e4f3d80 0x56546e8f5030 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e5144d0 0x56546e8e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 --2- 172.21.15.115:0/2457015168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e50c2f0 0x56546e8e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e017fa640 0 client.15690 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 -- 172.21.15.115:0/2457015168 >> 172.21.15.115:0/2457015168 conn(0x56546e529300 msgr2=0x56546e8d84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 -- 172.21.15.115:0/2457015168 shutdown_connections 2024-05-09T05:39:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 1 -- 172.21.15.115:0/2457015168 wait complete. 2024-05-09T05:39:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.096+0000 7f7e269ec740 20 client.15690 trim_cache size 0 max 16384 2024-05-09T05:39:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.111+0000 7f7e269ec740 1 Processor -- start 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.111+0000 7f7e269ec740 1 -- start start 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.111+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.111+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5144d0 0x56546e8fad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.111+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8eac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.111+0000 7f7e269ec740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b0fb0 con 0x56546e5351b0 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.111+0000 7f7e269ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b2af0 con 0x56546e5144d0 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.111+0000 7f7e269ec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b1680 con 0x56546e50c2f0 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d0d3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8eac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d0d3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8eac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33308/0 (socket says 172.21.15.115:33308) 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d8d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5144d0 0x56546e8fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d0d3640 1 -- 172.21.15.115:0/736726804 learned_addr learned my addr 172.21.15.115:0/736726804 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/736726804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/736726804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 msgr2=0x56546e8fa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/736726804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8fa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/736726804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 msgr2=0x56546e8eac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/736726804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8eac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/736726804 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546e6175f0 con 0x56546e5144d0 2024-05-09T05:39:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/736726804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/736726804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5144d0 0x56546e8fad70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x56546e8d9a30 tx=0x7f7e1401e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df84e66e9e58deda server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.112+0000 7f7e037fe640 1 -- 172.21.15.115:0/736726804 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1402f070 con 0x56546e5144d0 2024-05-09T05:39:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.113+0000 7f7e037fe640 1 -- 172.21.15.115:0/736726804 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e1400ce70 con 0x56546e5144d0 2024-05-09T05:39:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.113+0000 7f7e037fe640 1 -- 172.21.15.115:0/736726804 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1402a390 con 0x56546e5144d0 2024-05-09T05:39:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.113+0000 7f7e269ec740 1 -- 172.21.15.115:0/736726804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5144d0 msgr2=0x56546e8fad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.113+0000 7f7e269ec740 1 --2- 172.21.15.115:0/736726804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5144d0 0x56546e8fad70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x56546e8d9a30 tx=0x7f7e1401e040 comp rx=0 tx=0).stop 2024-05-09T05:39:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.113+0000 7f7e269ec740 1 -- 172.21.15.115:0/736726804 shutdown_connections 2024-05-09T05:39:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.113+0000 7f7e269ec740 1 --2- 172.21.15.115:0/736726804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5351b0 0x56546e8eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.113+0000 7f7e269ec740 1 --2- 172.21.15.115:0/736726804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5144d0 0x56546e8fad70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.113+0000 7f7e269ec740 1 --2- 172.21.15.115:0/736726804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.113+0000 7f7e269ec740 1 -- 172.21.15.115:0/736726804 >> 172.21.15.115:0/736726804 conn(0x56546e529300 msgr2=0x56546e8e1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 -- 172.21.15.115:0/736726804 shutdown_connections 2024-05-09T05:39:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 -- 172.21.15.115:0/736726804 wait complete. 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 -- start start 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 Processor -- start 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 0x56546e8e9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b2420 con 0x56546e5144d0 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181b1d50 con 0x56546e5351b0 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.114+0000 7f7e269ec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181aef90 con 0x56546e50c2f0 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e269ec740 10 client.0 _ll_register_callbacks cb 0x7f7e0c011540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d8d4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d8d4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8e8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33316/0 (socket says 172.21.15.115:33316) 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/1229445474 learned_addr learned my addr 172.21.15.115:0/1229445474 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/1229445474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 msgr2=0x56546e8e8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 0x56546e8e9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8e8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/1229445474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 msgr2=0x56546e8e9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 0x56546e8e9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d8d4640 1 -- 172.21.15.115:0/1229445474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56546e801ed0 con 0x56546e5144d0 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e1d8d4640 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8e8fe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400cb40 tx=0x7f7e1400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bbdd4b568dfd68 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e017fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e017fa640 1 -- 172.21.15.115:0/1229445474 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14038060 con 0x56546e5144d0 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e017fa640 1 -- 172.21.15.115:0/1229445474 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7e14004030 con 0x56546e5144d0 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.115+0000 7f7e269ec740 1 -- 172.21.15.115:0/1229445474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1400ce70 con 0x56546e5144d0 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e017fa640 1 -- 172.21.15.115:0/1229445474 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e1402f040 con 0x56546e5144d0 2024-05-09T05:39:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e269ec740 10 client.15696 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e269ec740 1 -- 172.21.15.115:0/1229445474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56546e95d6b0 con 0x56546e5144d0 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e017fa640 1 -- 172.21.15.115:0/1229445474 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e1400ce70 con 0x56546e5144d0 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e017fa640 1 -- 172.21.15.115:0/1229445474 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7e14032070 con 0x56546e5144d0 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e269ec740 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e269ec740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e269ec740 20 client.15696 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e269ec740 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56546e4c91c0 0x56546e95fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e269ec740 4 client.15696 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e269ec740 1 -- 172.21.15.115:0/1229445474 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56546e845240 con 0x56546e4c91c0 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.116+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56546e4c91c0 0x56546e95fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.117+0000 7f7e017fa640 10 client.15696 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.117+0000 7f7e1d0d3640 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56546e4c91c0 0x56546e95fb90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bea8ad516d7c081 server_cookie=ef846cb6ffecd9b8 in_seq=0 out_seq=0 2024-05-09T05:39:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.125+0000 7f7e017fa640 1 -- 172.21.15.115:0/1229445474 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56546e845240 con 0x56546e4c91c0 2024-05-09T05:39:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.125+0000 7f7e017fa640 10 client.15696 handle_command_reply: tid=0 2024-05-09T05:39:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.125+0000 7f7e269ec740 1 client.15696 shutdown 2024-05-09T05:39:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 -- 172.21.15.115:0/1229445474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 msgr2=0x56546e8e8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8e8fe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400cb40 tx=0x7f7e1400ecd0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 -- 172.21.15.115:0/1229445474 shutdown_connections 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56546e5351b0 0x56546e8e9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56546e5144d0 0x56546e8e8fe0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56546e4c91c0 0x56546e95fb90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 --2- 172.21.15.115:0/1229445474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56546e50c2f0 0x56546e8e8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e017fa640 0 client.15696 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 -- 172.21.15.115:0/1229445474 >> 172.21.15.115:0/1229445474 conn(0x56546e529300 msgr2=0x56546e8d8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 -- 172.21.15.115:0/1229445474 shutdown_connections 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 1 -- 172.21.15.115:0/1229445474 wait complete. 2024-05-09T05:39:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.126+0000 7f7e269ec740 20 client.15696 trim_cache size 0 max 16384 2024-05-09T05:39:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:39:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:39:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:39:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 -- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e040b2a70 msgr2=0x7f7e040b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e040b2a70 0x7f7e040b4f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7e10101740 tx=0x7f7e10098d70 comp rx=0 tx=0).stop 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 -- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e18109040 msgr2=0x7f7e1819c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e18109040 0x7f7e1819c090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7e10005cc0 tx=0x7f7e10032c40 comp rx=0 tx=0).stop 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 -- 172.21.15.115:0/907640328 shutdown_connections 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7e040b2a70 0x7f7e040b4f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e18109040 0x7f7e1819c090 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18108ba0 0x7f7e1819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 --2- 172.21.15.115:0/907640328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e181032f0 0x7f7e1819b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 -- 172.21.15.115:0/907640328 >> 172.21.15.115:0/907640328 conn(0x7f7e180782e0 msgr2=0x7f7e18100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 -- 172.21.15.115:0/907640328 shutdown_connections 2024-05-09T05:39:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:06.127+0000 7f7e1fb38640 1 -- 172.21.15.115:0/907640328 wait complete. 2024-05-09T05:39:07.078 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:07.582 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:07 smithi115 ceph-mon[30349]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T05:39:07.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.597+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3560963456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c41034e0 msgr2=0x7f00c41038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:07.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.597+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3560963456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c41034e0 0x7f00c41038f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f00c00099c0 tx=0x7f00c0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.597+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3560963456 shutdown_connections 2024-05-09T05:39:07.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.597+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3560963456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00c4108760 0x7f00c41100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.597+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3560963456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c41034e0 0x7f00c41038f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.597+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3560963456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c4107e10 0x7f00c4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.597+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3560963456 >> 172.21.15.115:0/3560963456 conn(0x7f00c40fef50 msgr2=0x7f00c4101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:07.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.597+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3560963456 shutdown_connections 2024-05-09T05:39:07.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.597+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3560963456 wait complete. 2024-05-09T05:39:07.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00cbbc8700 1 Processor -- start 2024-05-09T05:39:07.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00cbbc8700 1 -- start start 2024-05-09T05:39:07.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00cbbc8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00c41034e0 0x7f00c419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00cbbc8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c4107e10 0x7f00c419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00cbbc8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c4108760 0x7f00c4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00cbbc8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f00c4113240 con 0x7f00c4108760 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00cbbc8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41130c0 con 0x7f00c4107e10 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00cbbc8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41133c0 con 0x7f00c41034e0 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00ca165700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c4108760 0x7f00c4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00ca165700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c4108760 0x7f00c4199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33340/0 (socket says 172.21.15.115:33340) 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00ca165700 1 -- 172.21.15.115:0/3019220009 learned_addr learned my addr 172.21.15.115:0/3019220009 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00c9964700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00c41034e0 0x7f00c419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.598+0000 7f00c9163700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c4107e10 0x7f00c419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00ca165700 1 -- 172.21.15.115:0/3019220009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00c41034e0 msgr2=0x7f00c419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00ca165700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00c41034e0 0x7f00c419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00ca165700 1 -- 172.21.15.115:0/3019220009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c4107e10 msgr2=0x7f00c419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00ca165700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c4107e10 0x7f00c419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00ca165700 1 -- 172.21.15.115:0/3019220009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c419a110 con 0x7f00c4108760 2024-05-09T05:39:07.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00ca165700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c4108760 0x7f00c4199a60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f00bc007ee0 tx=0x7f00bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:07.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00baffd700 1 -- 172.21.15.115:0/3019220009 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00dd20 con 0x7f00c4108760 2024-05-09T05:39:07.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00baffd700 1 -- 172.21.15.115:0/3019220009 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00bc004020 con 0x7f00c4108760 2024-05-09T05:39:07.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00baffd700 1 -- 172.21.15.115:0/3019220009 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc015660 con 0x7f00c4108760 2024-05-09T05:39:07.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3019220009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c419a400 con 0x7f00c4108760 2024-05-09T05:39:07.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.599+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3019220009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c41a66d0 con 0x7f00c4108760 2024-05-09T05:39:07.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.602+0000 7f00baffd700 1 -- 172.21.15.115:0/3019220009 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f00bc015800 con 0x7f00c4108760 2024-05-09T05:39:07.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.603+0000 7f00baffd700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00b00b2b90 0x7f00b00b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.603+0000 7f00da5deb80 1 Processor -- start 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.603+0000 7f00c9964700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00b00b2b90 0x7f00b00b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.603+0000 7f00da5deb80 1 -- start start 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.603+0000 7f00baffd700 1 -- 172.21.15.115:0/3019220009 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f00bc0e2490 con 0x7f00c4108760 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00b00b2b90 0x7f00b00b50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009990 tx=0x7f00c0005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d219160 0x56021d2b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 0x56021d2b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d2ac6a0 0x56021d2acad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00da5deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56021d1ceff0 con 0x56021d219160 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00da5deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56021d16d7f0 con 0x56021d2ac6a0 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00da5deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56021d1cafb0 con 0x56021d12f4a0 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d219160 0x56021d2b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d219160 0x56021d2b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33352/0 (socket says 172.21.15.115:33352) 2024-05-09T05:39:07.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 -- 172.21.15.115:0/602250035 learned_addr learned my addr 172.21.15.115:0/602250035 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00ca165700 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d2ac6a0 0x56021d2acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9163700 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 0x56021d2b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 -- 172.21.15.115:0/602250035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 msgr2=0x56021d2b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 0x56021d2b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 -- 172.21.15.115:0/602250035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d2ac6a0 msgr2=0x56021d2acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d2ac6a0 0x56021d2acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 -- 172.21.15.115:0/602250035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021d165370 con 0x56021d219160 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9163700 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 0x56021d2b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.604+0000 7f00c9964700 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d219160 0x56021d2b24f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009750 tx=0x7f00c001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495d1382d5e4bd12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.605+0000 7f00affff700 1 -- 172.21.15.115:0/602250035 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c000be50 con 0x56021d219160 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.605+0000 7f00affff700 1 -- 172.21.15.115:0/602250035 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c00055d0 con 0x56021d219160 2024-05-09T05:39:07.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.605+0000 7f00affff700 1 -- 172.21.15.115:0/602250035 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c000f830 con 0x56021d219160 2024-05-09T05:39:07.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.605+0000 7f00da5deb80 1 -- 172.21.15.115:0/602250035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d219160 msgr2=0x56021d2b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.605+0000 7f00da5deb80 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d219160 0x56021d2b24f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009750 tx=0x7f00c001e000 comp rx=0 tx=0).stop 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.606+0000 7f00da5deb80 1 -- 172.21.15.115:0/602250035 shutdown_connections 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.606+0000 7f00da5deb80 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d2ac6a0 0x56021d2acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.606+0000 7f00da5deb80 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d219160 0x56021d2b24f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.606+0000 7f00da5deb80 1 --2- 172.21.15.115:0/602250035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 0x56021d2b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.606+0000 7f00da5deb80 1 -- 172.21.15.115:0/602250035 >> 172.21.15.115:0/602250035 conn(0x56021d18f6a0 msgr2=0x56021d2a7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.606+0000 7f00da5deb80 1 -- 172.21.15.115:0/602250035 shutdown_connections 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.606+0000 7f00da5deb80 1 -- 172.21.15.115:0/602250035 wait complete. 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.606+0000 7f00da5deb80 1 -- start start 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.606+0000 7f00da5deb80 1 Processor -- start 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 0x56021d2b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d2b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00da5deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56021d04aba0 con 0x56021d2ac6a0 2024-05-09T05:39:07.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00da5deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56021d189150 con 0x56021d219160 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00da5deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56021d2ad4e0 con 0x56021d12f4a0 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00da5deb80 10 client.0 _ll_register_callbacks cb 0x56021d0406c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ca165700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d2b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ca165700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d2b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33362/0 (socket says 172.21.15.115:33362) 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ca165700 1 -- 172.21.15.115:0/487337901 learned_addr learned my addr 172.21.15.115:0/487337901 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00c9163700 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00c9964700 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 0x56021d2b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ca165700 1 -- 172.21.15.115:0/487337901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 msgr2=0x56021d2b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ca165700 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 0x56021d2b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ca165700 1 -- 172.21.15.115:0/487337901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 msgr2=0x56021d2b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ca165700 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ca165700 1 -- 172.21.15.115:0/487337901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56021d04a490 con 0x56021d2ac6a0 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00c9163700 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ca165700 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d2b6bd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0a6490 tx=0x7f00bc004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4577cbe061f8f07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:07.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.607+0000 7f00ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00ad7fa700 1 -- 172.21.15.115:0/487337901 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0c3060 con 0x56021d2ac6a0 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00ad7fa700 1 -- 172.21.15.115:0/487337901 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f00bc0e20d0 con 0x56021d2ac6a0 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00da5deb80 1 -- 172.21.15.115:0/487337901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c00055d0 con 0x56021d2ac6a0 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00ad7fa700 1 -- 172.21.15.115:0/487337901 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00bc0ca050 con 0x56021d2ac6a0 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00da5deb80 10 client.15708 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00da5deb80 1 -- 172.21.15.115:0/487337901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56021d2c3ab0 con 0x56021d2ac6a0 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00ad7fa700 1 -- 172.21.15.115:0/487337901 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c00055d0 con 0x56021d2ac6a0 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00ad7fa700 1 -- 172.21.15.115:0/487337901 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f00bc0ad1b0 con 0x56021d2ac6a0 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00da5deb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00da5deb80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00da5deb80 20 client.15708 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00da5deb80 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56021d2c6040 0x56021d2c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00da5deb80 4 client.15708 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.608+0000 7f00da5deb80 1 -- 172.21.15.115:0/487337901 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56021d1dc9d0 con 0x56021d2c6040 2024-05-09T05:39:07.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.609+0000 7f00c9964700 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56021d2c6040 0x56021d2c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.609+0000 7f00c9964700 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56021d2c6040 0x56021d2c84b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14292616e2631dfb server_cookie=2009561989c29408 in_seq=0 out_seq=0 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.609+0000 7f00ad7fa700 10 client.15708 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00ad7fa700 1 -- 172.21.15.115:0/487337901 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56021d1dc9d0 con 0x56021d2c6040 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00ad7fa700 10 client.15708 handle_command_reply: tid=0 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 client.15708 shutdown 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 -- 172.21.15.115:0/487337901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 msgr2=0x56021d2b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d2b6bd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0a6490 tx=0x7f00bc004720 comp rx=0 tx=0).stop 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 -- 172.21.15.115:0/487337901 shutdown_connections 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56021d2c6040 0x56021d2c84b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d2b6bd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d12f4a0 0x56021d2b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00ad7fa700 0 client.15708 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 --2- 172.21.15.115:0/487337901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 -- 172.21.15.115:0/487337901 >> 172.21.15.115:0/487337901 conn(0x56021d18f6a0 msgr2=0x56021d2a71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 -- 172.21.15.115:0/487337901 shutdown_connections 2024-05-09T05:39:07.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 1 -- 172.21.15.115:0/487337901 wait complete. 2024-05-09T05:39:07.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.610+0000 7f00da5deb80 20 client.15708 trim_cache size 0 max 16384 2024-05-09T05:39:07.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00da5deb80 1 Processor -- start 2024-05-09T05:39:07.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00da5deb80 1 -- start start 2024-05-09T05:39:07.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d12f4a0 0x56021d2ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d2ac6a0 0x56021d2a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00da5deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41af180 con 0x56021d12f4a0 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00da5deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41b0cc0 con 0x56021d219160 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00da5deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41af850 con 0x56021d2ac6a0 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00c9964700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d12f4a0 0x56021d2ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.630+0000 7f00c9964700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d12f4a0 0x56021d2ac0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33364/0 (socket says 172.21.15.115:33364) 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00c9964700 1 -- 172.21.15.115:0/293090178 learned_addr learned my addr 172.21.15.115:0/293090178 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00ca165700 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d2ac6a0 0x56021d2a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00c9163700 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00c9964700 1 -- 172.21.15.115:0/293090178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d2ac6a0 msgr2=0x56021d2a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00c9964700 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d2ac6a0 0x56021d2a6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00c9964700 1 -- 172.21.15.115:0/293090178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 msgr2=0x56021d2c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00c9964700 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00c9964700 1 -- 172.21.15.115:0/293090178 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021d2a8d00 con 0x56021d12f4a0 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00c9163700 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:07.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00c9964700 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d12f4a0 0x56021d2ac0c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f00c00099c0 tx=0x7f00c001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8dfd7947c081940 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:07.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00af7fe700 1 -- 172.21.15.115:0/293090178 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0041070 con 0x56021d12f4a0 2024-05-09T05:39:07.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00af7fe700 1 -- 172.21.15.115:0/293090178 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c000fc30 con 0x56021d12f4a0 2024-05-09T05:39:07.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.631+0000 7f00af7fe700 1 -- 172.21.15.115:0/293090178 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c003c070 con 0x56021d12f4a0 2024-05-09T05:39:07.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 -- 172.21.15.115:0/293090178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d12f4a0 msgr2=0x56021d2ac0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d12f4a0 0x56021d2ac0c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f00c00099c0 tx=0x7f00c001a040 comp rx=0 tx=0).stop 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 -- 172.21.15.115:0/293090178 shutdown_connections 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d2ac6a0 0x56021d2a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d12f4a0 0x56021d2ac0c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 --2- 172.21.15.115:0/293090178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d219160 0x56021d2c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 -- 172.21.15.115:0/293090178 >> 172.21.15.115:0/293090178 conn(0x56021d18f6a0 msgr2=0x56021d2a68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 -- 172.21.15.115:0/293090178 shutdown_connections 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 -- 172.21.15.115:0/293090178 wait complete. 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.632+0000 7f00da5deb80 1 -- start start 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00da5deb80 1 Processor -- start 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d12f4a0 0x56021d2bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d219160 0x56021d2bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00da5deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d32f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00da5deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41b05f0 con 0x56021d2ac6a0 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00da5deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41aff20 con 0x56021d12f4a0 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00da5deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41ad120 con 0x56021d219160 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00da5deb80 10 client.0 _ll_register_callbacks cb 0x56021d0406c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00ca165700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d32f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00ca165700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d32f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33376/0 (socket says 172.21.15.115:33376) 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00ca165700 1 -- 172.21.15.115:0/1769175249 learned_addr learned my addr 172.21.15.115:0/1769175249 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00c9964700 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d12f4a0 0x56021d2bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00c9163700 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d219160 0x56021d2bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00ca165700 1 -- 172.21.15.115:0/1769175249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d219160 msgr2=0x56021d2bcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00ca165700 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d219160 0x56021d2bcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00ca165700 1 -- 172.21.15.115:0/1769175249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d12f4a0 msgr2=0x56021d2bc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00ca165700 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d12f4a0 0x56021d2bc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.633+0000 7f00ca165700 1 -- 172.21.15.115:0/1769175249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56021d32fdc0 con 0x56021d2ac6a0 2024-05-09T05:39:07.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00ca165700 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d32f880 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f00bc004870 tx=0x7f00bc0ad6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8b842b50f83d05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00ad7fa700 1 -- 172.21.15.115:0/1769175249 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0c3070 con 0x56021d2ac6a0 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00ad7fa700 1 -- 172.21.15.115:0/1769175249 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f00bc0e20d0 con 0x56021d2ac6a0 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00da5deb80 1 -- 172.21.15.115:0/1769175249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c000fc30 con 0x56021d2ac6a0 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00ad7fa700 1 -- 172.21.15.115:0/1769175249 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00bc0a50e0 con 0x56021d2ac6a0 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00da5deb80 10 client.15714 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00da5deb80 1 -- 172.21.15.115:0/1769175249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56021d330ef0 con 0x56021d2ac6a0 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00ad7fa700 1 -- 172.21.15.115:0/1769175249 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00c000fc30 con 0x56021d2ac6a0 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00ad7fa700 1 -- 172.21.15.115:0/1769175249 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f00bc08f310 con 0x56021d2ac6a0 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00da5deb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00da5deb80 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00da5deb80 20 client.15714 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00da5deb80 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56021d331330 0x56021d333750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00da5deb80 4 client.15714 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:39:07.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.634+0000 7f00da5deb80 1 -- 172.21.15.115:0/1769175249 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56021d1d6a70 con 0x56021d331330 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00c9964700 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56021d331330 0x56021d333750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00ad7fa700 10 client.15714 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00c9964700 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56021d331330 0x56021d333750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708629c6ae2a628b server_cookie=c2ea43a03855c4ac in_seq=0 out_seq=0 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00ad7fa700 1 -- 172.21.15.115:0/1769175249 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56021d1d6a70 con 0x56021d331330 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00ad7fa700 10 client.15714 handle_command_reply: tid=0 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00da5deb80 1 client.15714 shutdown 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00da5deb80 1 -- 172.21.15.115:0/1769175249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 msgr2=0x56021d32f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00da5deb80 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d32f880 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f00bc004870 tx=0x7f00bc0ad6c0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00da5deb80 1 -- 172.21.15.115:0/1769175249 shutdown_connections 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00da5deb80 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56021d12f4a0 0x56021d2bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00da5deb80 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56021d331330 0x56021d333750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00da5deb80 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56021d2ac6a0 0x56021d32f880 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00da5deb80 1 --2- 172.21.15.115:0/1769175249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56021d219160 0x56021d2bcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00ad7fa700 0 client.15714 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:07.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.635+0000 7f00da5deb80 1 -- 172.21.15.115:0/1769175249 >> 172.21.15.115:0/1769175249 conn(0x56021d18f6a0 msgr2=0x56021d2abe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:07.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00da5deb80 1 -- 172.21.15.115:0/1769175249 shutdown_connections 2024-05-09T05:39:07.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00da5deb80 1 -- 172.21.15.115:0/1769175249 wait complete. 2024-05-09T05:39:07.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00da5deb80 20 client.15714 trim_cache size 0 max 16384 2024-05-09T05:39:07.634 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:39:07.634 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:39:07.634 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:39:07.634 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:39:07.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00b00b2b90 msgr2=0x7f00b00b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:07.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00b00b2b90 0x7f00b00b50a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009990 tx=0x7f00c0005fb0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c4108760 msgr2=0x7f00c4199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:07.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c4108760 0x7f00c4199a60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f00bc007ee0 tx=0x7f00bc00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3019220009 shutdown_connections 2024-05-09T05:39:07.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00b00b2b90 0x7f00b00b50a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00c4108760 0x7f00c4199a60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00c41034e0 0x7f00c419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 --2- 172.21.15.115:0/3019220009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00c4107e10 0x7f00c419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:07.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.636+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3019220009 >> 172.21.15.115:0/3019220009 conn(0x7f00c40fef50 msgr2=0x7f00c41096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:07.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.637+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3019220009 shutdown_connections 2024-05-09T05:39:07.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:07.637+0000 7f00cbbc8700 1 -- 172.21.15.115:0/3019220009 wait complete. 2024-05-09T05:39:07.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:07 smithi161 ceph-mon[33805]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T05:39:07.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:07 smithi195 ceph-mon[33844]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T05:39:08.666 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:39:08.666 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:39:09.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:09 smithi115 ceph-mon[30349]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-05-09T05:39:09.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:09 smithi161 ceph-mon[33805]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-05-09T05:39:09.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:09 smithi195 ceph-mon[33844]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-05-09T05:39:09.991 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.240+0000 7f92aca8b700 1 -- 172.21.15.115:0/3368099745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8107df0 msgr2=0x7f92a8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.240+0000 7f92aca8b700 1 --2- 172.21.15.115:0/3368099745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8107df0 0x7f92a8108200 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f929c0099c0 tx=0x7f929c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.240+0000 7f92aca8b700 1 -- 172.21.15.115:0/3368099745 shutdown_connections 2024-05-09T05:39:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.240+0000 7f92aca8b700 1 --2- 172.21.15.115:0/3368099745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a81034c0 0x7f92a81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.240+0000 7f92aca8b700 1 --2- 172.21.15.115:0/3368099745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a8108740 0x7f92a81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.240+0000 7f92aca8b700 1 --2- 172.21.15.115:0/3368099745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8107df0 0x7f92a8108200 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.240+0000 7f92aca8b700 1 -- 172.21.15.115:0/3368099745 >> 172.21.15.115:0/3368099745 conn(0x7f92a80fedf0 msgr2=0x7f92a8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.241+0000 7f92aca8b700 1 -- 172.21.15.115:0/3368099745 shutdown_connections 2024-05-09T05:39:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.241+0000 7f92aca8b700 1 -- 172.21.15.115:0/3368099745 wait complete. 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.241+0000 7f92aca8b700 1 Processor -- start 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.241+0000 7f92aca8b700 1 -- start start 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92aca8b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a81034c0 0x7f92a819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92aca8b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8107df0 0x7f92a819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92aca8b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8108740 0x7f92a819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92aca8b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8113220 con 0x7f92a8108740 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92aca8b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81130a0 con 0x7f92a8107df0 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92aca8b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81133a0 con 0x7f92a81034c0 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8108740 0x7f92a819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8108740 0x7f92a819f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33400/0 (socket says 172.21.15.115:33400) 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a6d9d700 1 -- 172.21.15.115:0/2063499170 learned_addr learned my addr 172.21.15.115:0/2063499170 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a659c700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a81034c0 0x7f92a819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8107df0 0x7f92a819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a6d9d700 1 -- 172.21.15.115:0/2063499170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a81034c0 msgr2=0x7f92a819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a6d9d700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a81034c0 0x7f92a819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a6d9d700 1 -- 172.21.15.115:0/2063499170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8107df0 msgr2=0x7f92a819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a6d9d700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8107df0 0x7f92a819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a6d9d700 1 -- 172.21.15.115:0/2063499170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a819fc30 con 0x7f92a8108740 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8107df0 0x7f92a819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.242+0000 7f92a6d9d700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8108740 0x7f92a819f590 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f92a0007ee0 tx=0x7f92a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.243+0000 7f92977fe700 1 -- 172.21.15.115:0/2063499170 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a000dbd0 con 0x7f92a8108740 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.243+0000 7f92977fe700 1 -- 172.21.15.115:0/2063499170 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92a000dd70 con 0x7f92a8108740 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.243+0000 7f92aca8b700 1 -- 172.21.15.115:0/2063499170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a819ff20 con 0x7f92a8108740 2024-05-09T05:39:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.243+0000 7f92977fe700 1 -- 172.21.15.115:0/2063499170 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a00058d0 con 0x7f92a8108740 2024-05-09T05:39:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.243+0000 7f92aca8b700 1 -- 172.21.15.115:0/2063499170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a81a7840 con 0x7f92a8108740 2024-05-09T05:39:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.245+0000 7f92977fe700 1 -- 172.21.15.115:0/2063499170 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f92a001a020 con 0x7f92a8108740 2024-05-09T05:39:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.246+0000 7f92977fe700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b2b10 0x7f92900b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.247+0000 7f92a659c700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b2b10 0x7f92900b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.247+0000 7f92977fe700 1 -- 172.21.15.115:0/2063499170 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f92a00e2360 con 0x7f92a8108740 2024-05-09T05:39:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.247+0000 7f92bb4a1b80 1 Processor -- start 2024-05-09T05:39:10.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.247+0000 7f92a659c700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b2b10 0x7f92900b5020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9298005950 tx=0x7f929800b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:10.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.247+0000 7f92bb4a1b80 1 -- start start 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.247+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.247+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbf70450 0x55c2bc0f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.247+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc010050 con 0x55c2bbf70450 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2bbfb5950 con 0x55c2bbfeacb0 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc00c010 con 0x55c2bc0f17c0 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbf70450 0x55c2bc0f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbf70450 0x55c2bc0f7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33404/0 (socket says 172.21.15.115:33404) 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a5d9b700 1 -- 172.21.15.115:0/1365841159 learned_addr learned my addr 172.21.15.115:0/1365841159 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a659c700 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a6d9d700 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a5d9b700 1 -- 172.21.15.115:0/1365841159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bc0f17c0 msgr2=0x55c2bc0f1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0f1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a5d9b700 1 -- 172.21.15.115:0/1365841159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bbfeacb0 msgr2=0x55c2bc0f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a5d9b700 1 -- 172.21.15.115:0/1365841159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2bbfa64b0 con 0x55c2bbf70450 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a659c700 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a6d9d700 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbf70450 0x55c2bc0f7b40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f92a8076b40 tx=0x7f929c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515674dcc0cc38ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f9294ff9700 1 -- 172.21.15.115:0/1365841159 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c027070 con 0x55c2bbf70450 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.248+0000 7f9294ff9700 1 -- 172.21.15.115:0/1365841159 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f929c00bca0 con 0x55c2bbf70450 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.249+0000 7f9294ff9700 1 -- 172.21.15.115:0/1365841159 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c022d40 con 0x55c2bbf70450 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.249+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/1365841159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbf70450 msgr2=0x55c2bc0f7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.249+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbf70450 0x55c2bc0f7b40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f92a8076b40 tx=0x7f929c009510 comp rx=0 tx=0).stop 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.249+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/1365841159 shutdown_connections 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.250+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.250+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.250+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/1365841159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbf70450 0x55c2bc0f7b40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.250+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/1365841159 >> 172.21.15.115:0/1365841159 conn(0x55c2bbf73f20 msgr2=0x55c2bc0e81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.250+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/1365841159 shutdown_connections 2024-05-09T05:39:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.250+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/1365841159 wait complete. 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.250+0000 7f92bb4a1b80 1 -- start start 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.250+0000 7f92bb4a1b80 1 Processor -- start 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc0fb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0fb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc0f24c0 con 0x55c2bbfeacb0 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2bbfaaad0 con 0x55c2bc0f17c0 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc1075c0 con 0x55c2bbf70450 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92bb4a1b80 10 client.0 _ll_register_callbacks cb 0x55c2bbe81670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33414/0 (socket says 172.21.15.115:33414) 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a5d9b700 1 -- 172.21.15.115:0/3081549061 learned_addr learned my addr 172.21.15.115:0/3081549061 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a659c700 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc0fb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a6d9d700 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a5d9b700 1 -- 172.21.15.115:0/3081549061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 msgr2=0x55c2bc0fb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc0fb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a5d9b700 1 -- 172.21.15.115:0/3081549061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 msgr2=0x55c2bc0fbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0fbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a5d9b700 1 -- 172.21.15.115:0/3081549061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2bbe8b440 con 0x55c2bbfeacb0 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0fb920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f929c00b600 tx=0x7f929c0226a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1a78d7bf7fe428 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.251+0000 7f928dffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f928dffb700 1 -- 172.21.15.115:0/3081549061 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c032060 con 0x55c2bbfeacb0 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f928dffb700 1 -- 172.21.15.115:0/3081549061 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f929c01e040 con 0x55c2bbfeacb0 2024-05-09T05:39:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3081549061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929c00bca0 con 0x55c2bbfeacb0 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f928dffb700 1 -- 172.21.15.115:0/3081549061 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f929c039050 con 0x55c2bbfeacb0 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f92bb4a1b80 10 client.15726 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3081549061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2bc108dd0 con 0x55c2bbfeacb0 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f928dffb700 1 -- 172.21.15.115:0/3081549061 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f929c00bca0 con 0x55c2bbfeacb0 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f928dffb700 1 -- 172.21.15.115:0/3081549061 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f929c00fcd0 con 0x55c2bbfeacb0 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f92bb4a1b80 10 client.15726 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.252+0000 7f92bb4a1b80 10 client.15726 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.253+0000 7f92bb4a1b80 20 client.15726 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.253+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2bc10a120 0x55c2bc10c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.253+0000 7f92bb4a1b80 4 client.15726 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.253+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3081549061 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2bc01da30 con 0x55c2bc10a120 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.253+0000 7f92a659c700 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2bc10a120 0x55c2bc10c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.253+0000 7f928dffb700 10 client.15726 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.253+0000 7f92a659c700 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2bc10a120 0x55c2bc10c2d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d7820effdb0a8f server_cookie=2c4097ba5850fc1a in_seq=0 out_seq=0 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.254+0000 7f928dffb700 1 -- 172.21.15.115:0/3081549061 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2bc01da30 con 0x55c2bc10a120 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.254+0000 7f928dffb700 10 client.15726 handle_command_reply: tid=0 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.254+0000 7f92bb4a1b80 1 client.15726 shutdown 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.254+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3081549061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 msgr2=0x55c2bc0fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.254+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0fb920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f929c00b600 tx=0x7f929c0226a0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.254+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3081549061 shutdown_connections 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.254+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.254+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2bc10a120 0x55c2bc10c2d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.254+0000 7f928dffb700 0 client.15726 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.255+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc0fb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.255+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3081549061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0fb920 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.255+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3081549061 >> 172.21.15.115:0/3081549061 conn(0x55c2bbf73f20 msgr2=0x55c2bc0e1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.255+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3081549061 shutdown_connections 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.255+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3081549061 wait complete. 2024-05-09T05:39:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.255+0000 7f92bb4a1b80 20 client.15726 trim_cache size 0 max 16384 2024-05-09T05:39:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.276+0000 7f92bb4a1b80 1 Processor -- start 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.276+0000 7f92bb4a1b80 1 -- start start 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.276+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc0fa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc10a610 con 0x55c2bbfeacb0 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc10ace0 con 0x55c2bc0f17c0 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc10b3b0 con 0x55c2bbf70450 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33420/0 (socket says 172.21.15.115:33420) 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a5d9b700 1 -- 172.21.15.115:0/3785213248 learned_addr learned my addr 172.21.15.115:0/3785213248 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a659c700 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc0fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a6d9d700 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a659c700 1 -- 172.21.15.115:0/3785213248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 msgr2=0x55c2bc0fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a659c700 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0fbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a659c700 1 -- 172.21.15.115:0/3785213248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 msgr2=0x55c2bc0f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a659c700 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a659c700 1 -- 172.21.15.115:0/3785213248 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2bc0fdc80 con 0x55c2bbf70450 2024-05-09T05:39:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f92a659c700 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc0fa700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55c2bc0fc550 tx=0x7f9298013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c3fb50aa4223e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.277+0000 7f928ffff700 1 -- 172.21.15.115:0/3785213248 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298004830 con 0x55c2bbf70450 2024-05-09T05:39:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.278+0000 7f928ffff700 1 -- 172.21.15.115:0/3785213248 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9298005020 con 0x55c2bbf70450 2024-05-09T05:39:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.278+0000 7f928ffff700 1 -- 172.21.15.115:0/3785213248 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298004e60 con 0x55c2bbf70450 2024-05-09T05:39:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.278+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3785213248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 msgr2=0x55c2bc0fa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.278+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc0fa700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55c2bc0fc550 tx=0x7f9298013000 comp rx=0 tx=0).stop 2024-05-09T05:39:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.278+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3785213248 shutdown_connections 2024-05-09T05:39:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.278+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc0fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc0fa700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3785213248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc0f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3785213248 >> 172.21.15.115:0/3785213248 conn(0x55c2bbf73f20 msgr2=0x55c2bc0e1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3785213248 shutdown_connections 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3785213248 wait complete. 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 -- start start 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 Processor -- start 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc115950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc110700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc110c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc10c150 con 0x55c2bbfeacb0 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc10ba80 con 0x55c2bc0f17c0 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.279+0000 7f92bb4a1b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c2bc10c820 con 0x55c2bbf70450 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92bb4a1b80 10 client.0 _ll_register_callbacks cb 0x55c2bbe81670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a5d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc110700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33430/0 (socket says 172.21.15.115:33430) 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a5d9b700 1 -- 172.21.15.115:0/3497452810 learned_addr learned my addr 172.21.15.115:0/3497452810 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a659c700 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a6d9d700 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc110c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a5d9b700 1 -- 172.21.15.115:0/3497452810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 msgr2=0x55c2bc115950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc115950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a5d9b700 1 -- 172.21.15.115:0/3497452810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 msgr2=0x55c2bc110c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc110c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a5d9b700 1 -- 172.21.15.115:0/3497452810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2bc111180 con 0x55c2bbfeacb0 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a6d9d700 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92a5d9b700 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc110700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f929c0098e0 tx=0x7f929c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a79219ee338d43d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f928dffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f928dffb700 1 -- 172.21.15.115:0/3497452810 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c032040 con 0x55c2bbfeacb0 2024-05-09T05:39:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f928dffb700 1 -- 172.21.15.115:0/3497452810 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f929c028040 con 0x55c2bbfeacb0 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.280+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3497452810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9298005020 con 0x55c2bbfeacb0 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f928dffb700 1 -- 172.21.15.115:0/3497452810 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f929c027030 con 0x55c2bbfeacb0 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f92bb4a1b80 10 client.15732 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3497452810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2bc176e60 con 0x55c2bbfeacb0 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f928dffb700 1 -- 172.21.15.115:0/3497452810 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9298005020 con 0x55c2bbfeacb0 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f928dffb700 1 -- 172.21.15.115:0/3497452810 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f929c00f460 con 0x55c2bbfeacb0 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f92bb4a1b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f92bb4a1b80 10 client.15732 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f92bb4a1b80 20 client.15732 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2bc10a120 0x55c2bc17a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f92bb4a1b80 4 client.15732 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3497452810 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "status"}) -- 0x55c2bc017ad0 con 0x55c2bc10a120 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.281+0000 7f92a659c700 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2bc10a120 0x55c2bc17a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f928dffb700 10 client.15732 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92a659c700 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2bc10a120 0x55c2bc17a620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61e272052c05d04 server_cookie=912b09d27c79eecc in_seq=0 out_seq=0 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f928dffb700 1 -- 172.21.15.115:0/3497452810 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c2bc017ad0 con 0x55c2bc10a120 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f928dffb700 10 client.15732 handle_command_reply: tid=0 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92bb4a1b80 1 client.15732 shutdown 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3497452810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 msgr2=0x55c2bc110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc110700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f929c0098e0 tx=0x7f929c005b00 comp rx=0 tx=0).stop 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3497452810 shutdown_connections 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2bc0f17c0 0x55c2bc110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c2bc10a120 0x55c2bc17a620 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c2bbf70450 0x55c2bc115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92bb4a1b80 1 --2- 172.21.15.115:0/3497452810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c2bbfeacb0 0x55c2bc110700 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f928dffb700 0 client.15732 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.282+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3497452810 >> 172.21.15.115:0/3497452810 conn(0x55c2bbf73f20 msgr2=0x55c2bc0e1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3497452810 shutdown_connections 2024-05-09T05:39:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92bb4a1b80 1 -- 172.21.15.115:0/3497452810 wait complete. 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92bb4a1b80 20 client.15732 trim_cache size 0 max 16384 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "69de8a38-0dc4-11ef-bc97-c7b262605968", 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 24469, 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 315.50734228499999, 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 30, 2024-05-09T05:39:10.282 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 88, 2024-05-09T05:39:10.283 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 87, 2024-05-09T05:39:10.283 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 341.02074870299998 2024-05-09T05:39:10.283 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 -- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b2b10 msgr2=0x7f92900b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b2b10 0x7f92900b5020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9298005950 tx=0x7f929800b410 comp rx=0 tx=0).stop 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 -- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8108740 msgr2=0x7f92a819f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8108740 0x7f92a819f590 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f92a0007ee0 tx=0x7f92a000d3b0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 -- 172.21.15.115:0/2063499170 shutdown_connections 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b2b10 0x7f92900b5020 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8108740 0x7f92a819f590 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a81034c0 0x7f92a819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 --2- 172.21.15.115:0/2063499170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8107df0 0x7f92a819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 -- 172.21.15.115:0/2063499170 >> 172.21.15.115:0/2063499170 conn(0x7f92a80fedf0 msgr2=0x7f92a81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 -- 172.21.15.115:0/2063499170 shutdown_connections 2024-05-09T05:39:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:10.283+0000 7f92aca8b700 1 -- 172.21.15.115:0/2063499170 wait complete. 2024-05-09T05:39:11.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:11 smithi161 ceph-mon[33805]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T05:39:11.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:11 smithi195 ceph-mon[33844]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T05:39:11.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:11 smithi115 ceph-mon[30349]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T05:39:12.295 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.845+0000 7fcf585c7700 1 -- 172.21.15.115:0/3424224410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf50107df0 msgr2=0x7fcf50108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.845+0000 7fcf585c7700 1 --2- 172.21.15.115:0/3424224410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf50107df0 0x7fcf50108200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcf400099c0 tx=0x7fcf40009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.845+0000 7fcf585c7700 1 -- 172.21.15.115:0/3424224410 shutdown_connections 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.845+0000 7fcf585c7700 1 --2- 172.21.15.115:0/3424224410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50108740 0x7fcf501100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.845+0000 7fcf585c7700 1 --2- 172.21.15.115:0/3424224410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf501034c0 0x7fcf501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.845+0000 7fcf585c7700 1 --2- 172.21.15.115:0/3424224410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf50107df0 0x7fcf50108200 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.845+0000 7fcf585c7700 1 -- 172.21.15.115:0/3424224410 >> 172.21.15.115:0/3424224410 conn(0x7fcf500fedc0 msgr2=0x7fcf501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.845+0000 7fcf585c7700 1 -- 172.21.15.115:0/3424224410 shutdown_connections 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.845+0000 7fcf585c7700 1 -- 172.21.15.115:0/3424224410 wait complete. 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf585c7700 1 Processor -- start 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf585c7700 1 -- start start 2024-05-09T05:39:12.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf585c7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf501034c0 0x7fcf5019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf585c7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf50107df0 0x7fcf5019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf585c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50108740 0x7fcf5019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf585c7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50113270 con 0x7fcf501034c0 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf585c7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcf501130f0 con 0x7fcf50108740 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf585c7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf501133f0 con 0x7fcf50107df0 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf56363700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf501034c0 0x7fcf5019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf56363700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf501034c0 0x7fcf5019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48940/0 (socket says 172.21.15.115:48940) 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.846+0000 7fcf56363700 1 -- 172.21.15.115:0/541311322 learned_addr learned my addr 172.21.15.115:0/541311322 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf55b62700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf50107df0 0x7fcf5019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf56b64700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50108740 0x7fcf5019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf56363700 1 -- 172.21.15.115:0/541311322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf50107df0 msgr2=0x7fcf5019c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf56363700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf50107df0 0x7fcf5019c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf56363700 1 -- 172.21.15.115:0/541311322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50108740 msgr2=0x7fcf5019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:12.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf56363700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50108740 0x7fcf5019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:12.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf56363700 1 -- 172.21.15.115:0/541311322 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf501a6f40 con 0x7fcf501034c0 2024-05-09T05:39:12.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf56363700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf501034c0 0x7fcf5019bbf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0060b0 tx=0x7fcf4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:12.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf477fe700 1 -- 172.21.15.115:0/541311322 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c01a070 con 0x7fcf501034c0 2024-05-09T05:39:12.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf477fe700 1 -- 172.21.15.115:0/541311322 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf4c00dd20 con 0x7fcf501034c0 2024-05-09T05:39:12.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf477fe700 1 -- 172.21.15.115:0/541311322 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c014600 con 0x7fcf501034c0 2024-05-09T05:39:12.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf501a7230 con 0x7fcf501034c0 2024-05-09T05:39:12.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.847+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf501a7790 con 0x7fcf501034c0 2024-05-09T05:39:12.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.848+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf500512c0 con 0x7fcf501034c0 2024-05-09T05:39:12.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.851+0000 7fcf477fe700 1 -- 172.21.15.115:0/541311322 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7fcf4c01d070 con 0x7fcf501034c0 2024-05-09T05:39:12.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.851+0000 7fcf477fe700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcf3c0b2b90 0x7fcf3c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:12.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.852+0000 7fcf55b62700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcf3c0b2b90 0x7fcf3c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:12.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.852+0000 7fcf477fe700 1 -- 172.21.15.115:0/541311322 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fcf4c0e2530 con 0x7fcf501034c0 2024-05-09T05:39:12.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.852+0000 7fcf55b62700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcf3c0b2b90 0x7fcf3c0b50a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009990 tx=0x7fcf40005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:12.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:12.855+0000 7fcf477fe700 1 -- 172.21.15.115:0/541311322 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf4c010070 con 0x7fcf501034c0 2024-05-09T05:39:13.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.104+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf5006bc00 con 0x7fcf501034c0 2024-05-09T05:39:13.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.105+0000 7fcf477fe700 1 -- 172.21.15.115:0/541311322 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7fcf4c0a9330 con 0x7fcf501034c0 2024-05-09T05:39:13.098 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:39:13.099 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcf3c0b2b90 msgr2=0x7fcf3c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcf3c0b2b90 0x7fcf3c0b50a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009990 tx=0x7fcf40005fb0 comp rx=0 tx=0).stop 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf501034c0 msgr2=0x7fcf5019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf501034c0 0x7fcf5019bbf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0060b0 tx=0x7fcf4c00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 shutdown_connections 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcf3c0b2b90 0x7fcf3c0b50a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50108740 0x7fcf5019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf501034c0 0x7fcf5019bbf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 --2- 172.21.15.115:0/541311322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf50107df0 0x7fcf5019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:13.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 >> 172.21.15.115:0/541311322 conn(0x7fcf500fedc0 msgr2=0x7fcf501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:13.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 shutdown_connections 2024-05-09T05:39:13.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:13.108+0000 7fcf585c7700 1 -- 172.21.15.115:0/541311322 wait complete. 2024-05-09T05:39:13.103 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:39:13.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:13 smithi161 ceph-mon[33805]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T05:39:13.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:13 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/541311322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:13.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:13 smithi195 ceph-mon[33844]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T05:39:13.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:13 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/541311322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:13.879 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:13 smithi115 ceph-mon[30349]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T05:39:13.879 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:13 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/541311322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:13.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:39:14.160 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:15.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:15 smithi115 ceph-mon[30349]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-09T05:39:15.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:15 smithi195 ceph-mon[33844]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-09T05:39:15.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:15 smithi161 ceph-mon[33805]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.080+0000 7f896e17e700 1 -- 172.21.15.115:0/3679458536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 msgr2=0x7f8968102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.080+0000 7f896e17e700 1 --2- 172.21.15.115:0/3679458536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 0x7f8968102ba0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f895c009990 tx=0x7f895c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.080+0000 7f896e17e700 1 -- 172.21.15.115:0/3679458536 shutdown_connections 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.080+0000 7f896e17e700 1 --2- 172.21.15.115:0/3679458536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8968109300 0x7f8968109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.080+0000 7f896e17e700 1 --2- 172.21.15.115:0/3679458536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89681030e0 0x7f8968111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.080+0000 7f896e17e700 1 --2- 172.21.15.115:0/3679458536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 0x7f8968102ba0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.080+0000 7f896e17e700 1 -- 172.21.15.115:0/3679458536 >> 172.21.15.115:0/3679458536 conn(0x7f89680fe520 msgr2=0x7f8968100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.080+0000 7f896e17e700 1 -- 172.21.15.115:0/3679458536 shutdown_connections 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.080+0000 7f896e17e700 1 -- 172.21.15.115:0/3679458536 wait complete. 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f896e17e700 1 Processor -- start 2024-05-09T05:39:16.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f896e17e700 1 -- start start 2024-05-09T05:39:16.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f896e17e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 0x7f896819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f896e17e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89681030e0 0x7f896819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f896e17e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8968109300 0x7f896819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f896e17e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8968114520 con 0x7f8968102790 2024-05-09T05:39:16.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f896e17e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89681143a0 con 0x7f8968109300 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f896e17e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89681146a0 con 0x7f89681030e0 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f89677fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 0x7f896819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f8966ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89681030e0 0x7f896819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f89677fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 0x7f896819af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48962/0 (socket says 172.21.15.115:48962) 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.081+0000 7f89677fe700 1 -- 172.21.15.115:0/1090621290 learned_addr learned my addr 172.21.15.115:0/1090621290 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f8967fff700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8968109300 0x7f896819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f89677fe700 1 -- 172.21.15.115:0/1090621290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89681030e0 msgr2=0x7f896819b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f89677fe700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89681030e0 0x7f896819b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f89677fe700 1 -- 172.21.15.115:0/1090621290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8968109300 msgr2=0x7f896819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f89677fe700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8968109300 0x7f896819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f89677fe700 1 -- 172.21.15.115:0/1090621290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896819fd90 con 0x7f8968102790 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f89677fe700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 0x7f896819af10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f895800b990 tx=0x7f895800bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f8964ff9700 1 -- 172.21.15.115:0/1090621290 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895800d890 con 0x7f8968102790 2024-05-09T05:39:16.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f896e17e700 1 -- 172.21.15.115:0/1090621290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89681a0080 con 0x7f8968102790 2024-05-09T05:39:16.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f8964ff9700 1 -- 172.21.15.115:0/1090621290 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f895800e460 con 0x7f8968102790 2024-05-09T05:39:16.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f8964ff9700 1 -- 172.21.15.115:0/1090621290 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958017660 con 0x7f8968102790 2024-05-09T05:39:16.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.082+0000 7f896e17e700 1 -- 172.21.15.115:0/1090621290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89681a79a0 con 0x7f8968102790 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.085+0000 7f8964ff9700 1 -- 172.21.15.115:0/1090621290 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f89580203f0 con 0x7f8968102790 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.085+0000 7f8964ff9700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89500b2b10 0x7f89500b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f8966ffd700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89500b2b10 0x7f89500b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f8964ff9700 1 -- 172.21.15.115:0/1090621290 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f89580e2850 con 0x7f8968102790 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f897cb94b80 1 Processor -- start 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f897cb94b80 1 -- start start 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f8966ffd700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89500b2b10 0x7f89500b5020 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f895c006010 tx=0x7f895c00b560 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd72b9160 0x561cd73524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 0x561cd7352a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd734cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f897cb94b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561cd726eff0 con 0x561cd734c6a0 2024-05-09T05:39:16.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f897cb94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561cd720d7f0 con 0x561cd72b9160 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f897cb94b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561cd726afb0 con 0x561cd71cf4a0 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f8967fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd734cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f8967fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd734cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48974/0 (socket says 172.21.15.115:48974) 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f8967fff700 1 -- 172.21.15.115:0/1076468542 learned_addr learned my addr 172.21.15.115:0/1076468542 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f8966ffd700 1 --2- 172.21.15.115:0/1076468542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 0x561cd7352a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.086+0000 7f89677fe700 1 --2- 172.21.15.115:0/1076468542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd72b9160 0x561cd73524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.087+0000 7f8967fff700 1 -- 172.21.15.115:0/1076468542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 msgr2=0x561cd7352a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.087+0000 7f8967fff700 1 --2- 172.21.15.115:0/1076468542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 0x561cd7352a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.087+0000 7f8967fff700 1 -- 172.21.15.115:0/1076468542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd72b9160 msgr2=0x561cd73524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.087+0000 7f8967fff700 1 --2- 172.21.15.115:0/1076468542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd72b9160 0x561cd73524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.087+0000 7f8967fff700 1 -- 172.21.15.115:0/1076468542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd7205370 con 0x561cd734c6a0 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.087+0000 7f8967fff700 1 --2- 172.21.15.115:0/1076468542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd734cad0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8960009a70 tx=0x7f8960009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692af1f4e03c03f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.087+0000 7f894dffb700 1 -- 172.21.15.115:0/1076468542 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960017070 con 0x561cd734c6a0 2024-05-09T05:39:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.087+0000 7f894dffb700 1 -- 172.21.15.115:0/1076468542 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f896000ed60 con 0x561cd734c6a0 2024-05-09T05:39:16.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.087+0000 7f894dffb700 1 -- 172.21.15.115:0/1076468542 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960013d90 con 0x561cd734c6a0 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 -- 172.21.15.115:0/1076468542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 msgr2=0x561cd734cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1076468542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd734cad0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8960009a70 tx=0x7f8960009d80 comp rx=0 tx=0).stop 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 -- 172.21.15.115:0/1076468542 shutdown_connections 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1076468542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd72b9160 0x561cd73524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1076468542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd734cad0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1076468542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 0x561cd7352a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 -- 172.21.15.115:0/1076468542 >> 172.21.15.115:0/1076468542 conn(0x561cd722f6a0 msgr2=0x561cd7347400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 -- 172.21.15.115:0/1076468542 shutdown_connections 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 -- 172.21.15.115:0/1076468542 wait complete. 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.088+0000 7f897cb94b80 1 -- start start 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f897cb94b80 1 Processor -- start 2024-05-09T05:39:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 0x561cd73561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd734c6a0 0x561cd7356c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f897cb94b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561cd70eaba0 con 0x561cd72b9160 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f897cb94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561cd7229150 con 0x561cd734c6a0 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f897cb94b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561cd73591d0 con 0x561cd71cf4a0 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f897cb94b80 10 client.0 _ll_register_callbacks cb 0x561cd70e06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f8966ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f8966ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48990/0 (socket says 172.21.15.115:48990) 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f8966ffd700 1 -- 172.21.15.115:0/475647031 learned_addr learned my addr 172.21.15.115:0/475647031 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f89677fe700 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 0x561cd73561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f8967fff700 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd734c6a0 0x561cd7356c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f8966ffd700 1 -- 172.21.15.115:0/475647031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 msgr2=0x561cd73561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f8966ffd700 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 0x561cd73561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f8966ffd700 1 -- 172.21.15.115:0/475647031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd734c6a0 msgr2=0x561cd7356c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:16.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f8966ffd700 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd734c6a0 0x561cd7356c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.089+0000 7f8966ffd700 1 -- 172.21.15.115:0/475647031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd70ea490 con 0x561cd72b9160 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f8966ffd700 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73566f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f895c015000 tx=0x7f895c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62400adc2c13649e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f8946ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f8946ffd700 1 -- 172.21.15.115:0/475647031 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c00f460 con 0x561cd72b9160 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f8946ffd700 1 -- 172.21.15.115:0/475647031 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f895c00f600 con 0x561cd72b9160 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f897cb94b80 1 -- 172.21.15.115:0/475647031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f896000ed60 con 0x561cd72b9160 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f8946ffd700 1 -- 172.21.15.115:0/475647031 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f895c00f800 con 0x561cd72b9160 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f897cb94b80 10 client.15750 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f897cb94b80 1 -- 172.21.15.115:0/475647031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561cd7363b90 con 0x561cd72b9160 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f8946ffd700 1 -- 172.21.15.115:0/475647031 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f896000ed60 con 0x561cd72b9160 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f8946ffd700 1 -- 172.21.15.115:0/475647031 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f895c02e070 con 0x561cd72b9160 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f897cb94b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f897cb94b80 10 client.15750 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.090+0000 7f897cb94b80 20 client.15750 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.091+0000 7f897cb94b80 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x561cd7366270 0x561cd7368690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.091+0000 7f897cb94b80 4 client.15750 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.091+0000 7f897cb94b80 1 -- 172.21.15.115:0/475647031 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cd727c9d0 con 0x561cd7366270 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.091+0000 7f89677fe700 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x561cd7366270 0x561cd7368690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.091+0000 7f89677fe700 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x561cd7366270 0x561cd7368690 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=88b0ec178bf69d0c server_cookie=48f059c7936c5b95 in_seq=0 out_seq=0 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.091+0000 7f8946ffd700 10 client.15750 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.092+0000 7f8946ffd700 1 -- 172.21.15.115:0/475647031 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561cd727c9d0 con 0x561cd7366270 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.092+0000 7f8946ffd700 10 client.15750 handle_command_reply: tid=0 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.092+0000 7f897cb94b80 1 client.15750 shutdown 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 -- 172.21.15.115:0/475647031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 msgr2=0x561cd73566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73566f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f895c015000 tx=0x7f895c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 -- 172.21.15.115:0/475647031 shutdown_connections 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd734c6a0 0x561cd7356c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x561cd7366270 0x561cd7368690 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd71cf4a0 0x561cd73561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 --2- 172.21.15.115:0/475647031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73566f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f8946ffd700 0 client.15750 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:39:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 -- 172.21.15.115:0/475647031 >> 172.21.15.115:0/475647031 conn(0x561cd722f6a0 msgr2=0x561cd73471c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 -- 172.21.15.115:0/475647031 shutdown_connections 2024-05-09T05:39:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 1 -- 172.21.15.115:0/475647031 wait complete. 2024-05-09T05:39:16.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.093+0000 7f897cb94b80 20 client.15750 trim_cache size 0 max 16384 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f897cb94b80 1 Processor -- start 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f897cb94b80 1 -- start start 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 0x561cd73509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd72b9160 0x561cd7356a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd7361fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f897cb94b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89681b0310 con 0x561cd734c6a0 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f897cb94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89681b1e50 con 0x561cd71cf4a0 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f897cb94b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89681b09e0 con 0x561cd72b9160 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8967fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd7361fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8967fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd7361fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49000/0 (socket says 172.21.15.115:49000) 2024-05-09T05:39:16.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8967fff700 1 -- 172.21.15.115:0/1141154455 learned_addr learned my addr 172.21.15.115:0/1141154455 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f89677fe700 1 --2- 172.21.15.115:0/1141154455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 0x561cd73509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8967fff700 1 -- 172.21.15.115:0/1141154455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd72b9160 msgr2=0x561cd7356a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8966ffd700 1 --2- 172.21.15.115:0/1141154455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd72b9160 0x561cd7356a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8967fff700 1 --2- 172.21.15.115:0/1141154455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd72b9160 0x561cd7356a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8967fff700 1 -- 172.21.15.115:0/1141154455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 msgr2=0x561cd73509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8967fff700 1 --2- 172.21.15.115:0/1141154455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 0x561cd73509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8967fff700 1 -- 172.21.15.115:0/1141154455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd7356fb0 con 0x561cd734c6a0 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.108+0000 7f8967fff700 1 --2- 172.21.15.115:0/1141154455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd7361fd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8960019000 tx=0x7f8960009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906dd30a72bfc03a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f894d7fa700 1 -- 172.21.15.115:0/1141154455 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960029070 con 0x561cd734c6a0 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f894d7fa700 1 -- 172.21.15.115:0/1141154455 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8960013470 con 0x561cd734c6a0 2024-05-09T05:39:16.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f894d7fa700 1 -- 172.21.15.115:0/1141154455 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960017040 con 0x561cd734c6a0 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f897cb94b80 1 -- 172.21.15.115:0/1141154455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 msgr2=0x561cd7361fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1141154455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd7361fd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8960019000 tx=0x7f8960009280 comp rx=0 tx=0).stop 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f897cb94b80 1 -- 172.21.15.115:0/1141154455 shutdown_connections 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1141154455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 0x561cd73509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1141154455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd734c6a0 0x561cd7361fd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1141154455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd72b9160 0x561cd7356a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.109+0000 7f897cb94b80 1 -- 172.21.15.115:0/1141154455 >> 172.21.15.115:0/1141154455 conn(0x561cd722f6a0 msgr2=0x561cd7354ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.110+0000 7f897cb94b80 1 -- 172.21.15.115:0/1141154455 shutdown_connections 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.110+0000 7f897cb94b80 1 -- 172.21.15.115:0/1141154455 wait complete. 2024-05-09T05:39:16.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.110+0000 7f897cb94b80 1 -- start start 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.110+0000 7f897cb94b80 1 Processor -- start 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 0x561cd73cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f897cb94b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd734c6a0 0x561cd736ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f897cb94b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f89681b1780 con 0x561cd72b9160 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f897cb94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89681b10b0 con 0x561cd71cf4a0 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f897cb94b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89681ae2b0 con 0x561cd734c6a0 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f897cb94b80 10 client.0 _ll_register_callbacks cb 0x561cd70e06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8966ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8966ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49014/0 (socket says 172.21.15.115:49014) 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8966ffd700 1 -- 172.21.15.115:0/1764594530 learned_addr learned my addr 172.21.15.115:0/1764594530 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f89677fe700 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 0x561cd73cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8967fff700 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd734c6a0 0x561cd736ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8966ffd700 1 -- 172.21.15.115:0/1764594530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd734c6a0 msgr2=0x561cd736ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:16.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8966ffd700 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd734c6a0 0x561cd736ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8966ffd700 1 -- 172.21.15.115:0/1764594530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 msgr2=0x561cd73cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8966ffd700 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 0x561cd73cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8966ffd700 1 -- 172.21.15.115:0/1764594530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd736d3c0 con 0x561cd72b9160 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8966ffd700 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73ce140 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f895c030b70 tx=0x7f895c030e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78043c205bffc63f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8946ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8946ffd700 1 -- 172.21.15.115:0/1764594530 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c043060 con 0x561cd72b9160 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8946ffd700 1 -- 172.21.15.115:0/1764594530 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f895c030650 con 0x561cd72b9160 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f897cb94b80 1 -- 172.21.15.115:0/1764594530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8960013470 con 0x561cd72b9160 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.111+0000 7f8946ffd700 1 -- 172.21.15.115:0/1764594530 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f895c04a050 con 0x561cd72b9160 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f897cb94b80 10 client.15756 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f897cb94b80 1 -- 172.21.15.115:0/1764594530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561cd735e660 con 0x561cd72b9160 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f8946ffd700 1 -- 172.21.15.115:0/1764594530 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8960013470 con 0x561cd72b9160 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f8946ffd700 1 -- 172.21.15.115:0/1764594530 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f895c02e040 con 0x561cd72b9160 2024-05-09T05:39:16.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f897cb94b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f897cb94b80 10 client.15756 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f897cb94b80 20 client.15756 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x561cd735ea40 0x561cd73d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f897cb94b80 4 client.15756 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f897cb94b80 1 -- 172.21.15.115:0/1764594530 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cd7276a70 con 0x561cd735ea40 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.112+0000 7f89677fe700 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x561cd735ea40 0x561cd73d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f8946ffd700 10 client.15756 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f89677fe700 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x561cd735ea40 0x561cd73d6e20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=defa13c0ec68edff server_cookie=4636eb41e643d40d in_seq=0 out_seq=0 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f8946ffd700 1 -- 172.21.15.115:0/1764594530 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cd7276a70 con 0x561cd735ea40 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f8946ffd700 10 client.15756 handle_command_reply: tid=0 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 client.15756 shutdown 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 -- 172.21.15.115:0/1764594530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 msgr2=0x561cd73ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73ce140 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f895c030b70 tx=0x7f895c030e20 comp rx=0 tx=0).stop 2024-05-09T05:39:16.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 -- 172.21.15.115:0/1764594530 shutdown_connections 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cd71cf4a0 0x561cd73cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x561cd735ea40 0x561cd73d6e20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd734c6a0 0x561cd736ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 --2- 172.21.15.115:0/1764594530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561cd72b9160 0x561cd73ce140 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f8946ffd700 0 client.15756 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 -- 172.21.15.115:0/1764594530 >> 172.21.15.115:0/1764594530 conn(0x561cd722f6a0 msgr2=0x561cd7354780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 -- 172.21.15.115:0/1764594530 shutdown_connections 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 1 -- 172.21.15.115:0/1764594530 wait complete. 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.113+0000 7f897cb94b80 20 client.15756 trim_cache size 0 max 16384 2024-05-09T05:39:16.111 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:16.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 -- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89500b2b10 msgr2=0x7f89500b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:16.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89500b2b10 0x7f89500b5020 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f895c006010 tx=0x7f895c00b560 comp rx=0 tx=0).stop 2024-05-09T05:39:16.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 -- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 msgr2=0x7f896819af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:16.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 0x7f896819af10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f895800b990 tx=0x7f895800bca0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 -- 172.21.15.115:0/1090621290 shutdown_connections 2024-05-09T05:39:16.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89500b2b10 0x7f89500b5020 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8968109300 0x7f896819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8968102790 0x7f896819af10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 --2- 172.21.15.115:0/1090621290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89681030e0 0x7f896819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:16.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 -- 172.21.15.115:0/1090621290 >> 172.21.15.115:0/1090621290 conn(0x7f89680fe520 msgr2=0x7f8968107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:16.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 -- 172.21.15.115:0/1090621290 shutdown_connections 2024-05-09T05:39:16.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:16.114+0000 7f896e17e700 1 -- 172.21.15.115:0/1090621290 wait complete. 2024-05-09T05:39:16.964 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:39:17.208 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:17.605 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:17 smithi115 ceph-mon[30349]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T05:39:17.667 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:17 smithi195 ceph-mon[33844]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T05:39:17.712 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:17 smithi161 ceph-mon[33805]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-09T05:39:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.115+0000 7fd5a9636700 1 -- 172.21.15.115:0/2449267066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a4104240 msgr2=0x7fd5a4110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.115+0000 7fd5a9636700 1 --2- 172.21.15.115:0/2449267066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a4104240 0x7fd5a4110070 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009990 tx=0x7fd59c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.115+0000 7fd5a9636700 1 -- 172.21.15.115:0/2449267066 shutdown_connections 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.115+0000 7fd5a9636700 1 --2- 172.21.15.115:0/2449267066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4108200 0x7fd5a4108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.115+0000 7fd5a9636700 1 --2- 172.21.15.115:0/2449267066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a4104240 0x7fd5a4110070 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.115+0000 7fd5a9636700 1 --2- 172.21.15.115:0/2449267066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5a41038f0 0x7fd5a4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.115+0000 7fd5a9636700 1 -- 172.21.15.115:0/2449267066 >> 172.21.15.115:0/2449267066 conn(0x7fd5a40fe6f0 msgr2=0x7fd5a4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.115+0000 7fd5a9636700 1 -- 172.21.15.115:0/2449267066 shutdown_connections 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.115+0000 7fd5a9636700 1 -- 172.21.15.115:0/2449267066 wait complete. 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a9636700 1 Processor -- start 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a9636700 1 -- start start 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a9636700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a41038f0 0x7fd5a419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a9636700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5a4104240 0x7fd5a419c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a9636700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4108200 0x7fd5a419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a9636700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4113240 con 0x7fd5a41038f0 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a9636700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41130c0 con 0x7fd5a4108200 2024-05-09T05:39:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a9636700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41133c0 con 0x7fd5a4104240 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a41038f0 0x7fd5a419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a41038f0 0x7fd5a419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49024/0 (socket says 172.21.15.115:49024) 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/76027522 learned_addr learned my addr 172.21.15.115:0/76027522 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.116+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5a4104240 0x7fd5a419c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a37fe700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4108200 0x7fd5a419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/76027522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5a4104240 msgr2=0x7fd5a419c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5a4104240 0x7fd5a419c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/76027522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4108200 msgr2=0x7fd5a419c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4108200 0x7fd5a419c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/76027522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a41a6e40 con 0x7fd5a41038f0 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a37fe700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4108200 0x7fd5a419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a41038f0 0x7fd5a419bb20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd594007f10 tx=0x7fd59400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd59bfff700 1 -- 172.21.15.115:0/76027522 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59400dbd0 con 0x7fd5a41038f0 2024-05-09T05:39:19.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a9636700 1 -- 172.21.15.115:0/76027522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a41a7130 con 0x7fd5a41038f0 2024-05-09T05:39:19.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd59bfff700 1 -- 172.21.15.115:0/76027522 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd59400dd70 con 0x7fd5a41038f0 2024-05-09T05:39:19.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd59bfff700 1 -- 172.21.15.115:0/76027522 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594005a40 con 0x7fd5a41038f0 2024-05-09T05:39:19.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.117+0000 7fd5a9636700 1 -- 172.21.15.115:0/76027522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a41a76c0 con 0x7fd5a41038f0 2024-05-09T05:39:19.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.120+0000 7fd59bfff700 1 -- 172.21.15.115:0/76027522 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7fd59401a020 con 0x7fd5a41038f0 2024-05-09T05:39:19.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.121+0000 7fd59bfff700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd5900b2b90 0x7fd5900b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.121+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd5900b2b90 0x7fd5900b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.121+0000 7fd59bfff700 1 -- 172.21.15.115:0/76027522 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd5940e22d0 con 0x7fd5a41038f0 2024-05-09T05:39:19.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.121+0000 7fd5b804cb80 1 Processor -- start 2024-05-09T05:39:19.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.121+0000 7fd5b804cb80 1 -- start start 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.121+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd5900b2b90 0x7fd5900b50a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4106960 tx=0x7fd58c00a300 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe5c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 0x55e7ebe56810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e7ebd790a0 con 0x55e7ebd53d00 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e7ebd1e9a0 con 0x55e7ebe56400 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e7ebd75060 con 0x55e7ebcd94a0 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe5c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe5c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49026/0 (socket says 172.21.15.115:49026) 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/1258898844 learned_addr learned my addr 172.21.15.115:0/1258898844 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a37fe700 1 --2- 172.21.15.115:0/1258898844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 0x55e7ebe56810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/1258898844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/1258898844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 msgr2=0x55e7ebe5c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/1258898844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/1258898844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 msgr2=0x55e7ebe56810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/1258898844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 0x55e7ebe56810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/1258898844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ebd0f500 con 0x55e7ebd53d00 2024-05-09T05:39:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.122+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/1258898844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe5c240 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd594007ee0 tx=0x7fd5940a8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ac9c1e05caca34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.123+0000 7fd5997fa700 1 -- 172.21.15.115:0/1258898844 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5940a9020 con 0x55e7ebd53d00 2024-05-09T05:39:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.123+0000 7fd5997fa700 1 -- 172.21.15.115:0/1258898844 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5940ad3d0 con 0x55e7ebd53d00 2024-05-09T05:39:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.123+0000 7fd5997fa700 1 -- 172.21.15.115:0/1258898844 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5940b19f0 con 0x55e7ebd53d00 2024-05-09T05:39:19.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1258898844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 msgr2=0x55e7ebe5c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1258898844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe5c240 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd594007ee0 tx=0x7fd5940a8a80 comp rx=0 tx=0).stop 2024-05-09T05:39:19.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1258898844 shutdown_connections 2024-05-09T05:39:19.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1258898844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 0x55e7ebe56810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1258898844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe5c240 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1258898844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1258898844 >> 172.21.15.115:0/1258898844 conn(0x55e7ebcdcf70 msgr2=0x55e7ebe51220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1258898844 shutdown_connections 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1258898844 wait complete. 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 -- start start 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.124+0000 7fd5b804cb80 1 Processor -- start 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe60080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 0x55e7ebe60b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e7ebe57160 con 0x55e7ebd53d00 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e7ebd13b20 con 0x55e7ebcd94a0 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e7ebe67430 con 0x55e7ebe56400 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5b804cb80 10 client.0 _ll_register_callbacks cb 0x55e7ebbea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a27fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a27fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49028/0 (socket says 172.21.15.115:49028) 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a27fc700 1 -- 172.21.15.115:0/783313794 learned_addr learned my addr 172.21.15.115:0/783313794 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe60080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a37fe700 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 0x55e7ebe60b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a27fc700 1 -- 172.21.15.115:0/783313794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 msgr2=0x55e7ebe60b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 0x55e7ebe60b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a27fc700 1 -- 172.21.15.115:0/783313794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebcd94a0 msgr2=0x55e7ebe60080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe60080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a27fc700 1 -- 172.21.15.115:0/783313794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ebbf4490 con 0x55e7ebd53d00 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe605c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd58c01da70 tx=0x7fd58c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b666cc0755cc92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.125+0000 7fd58a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd58a7fc700 1 -- 172.21.15.115:0/783313794 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c016070 con 0x55e7ebd53d00 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd58a7fc700 1 -- 172.21.15.115:0/783313794 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd58c004a30 con 0x55e7ebd53d00 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd5b804cb80 1 -- 172.21.15.115:0/783313794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5940ad3d0 con 0x55e7ebd53d00 2024-05-09T05:39:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd58a7fc700 1 -- 172.21.15.115:0/783313794 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd58c030050 con 0x55e7ebd53d00 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd5b804cb80 10 client.15768 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd5b804cb80 1 -- 172.21.15.115:0/783313794 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7ebe71e70 con 0x55e7ebd53d00 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd58a7fc700 1 -- 172.21.15.115:0/783313794 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5940ad3d0 con 0x55e7ebd53d00 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd58a7fc700 1 -- 172.21.15.115:0/783313794 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd58c024030 con 0x55e7ebd53d00 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd5b804cb80 10 client.15768 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd5b804cb80 10 client.15768 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd5b804cb80 20 client.15768 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e7ebe744b0 0x55e7ebe768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd5b804cb80 4 client.15768 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.126+0000 7fd5b804cb80 1 -- 172.21.15.115:0/783313794 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ebd86a80 con 0x55e7ebe744b0 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.127+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e7ebe744b0 0x55e7ebe768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.127+0000 7fd58a7fc700 10 client.15768 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:39:19.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.127+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e7ebe744b0 0x55e7ebe768d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32c427588cd66074 server_cookie=21371c17cd4ecaa9 in_seq=0 out_seq=0 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd58a7fc700 1 -- 172.21.15.115:0/783313794 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e7ebd86a80 con 0x55e7ebe744b0 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd58a7fc700 10 client.15768 handle_command_reply: tid=0 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd5b804cb80 1 client.15768 shutdown 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd5b804cb80 1 -- 172.21.15.115:0/783313794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 msgr2=0x55e7ebe605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe605c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd58c01da70 tx=0x7fd58c0047c0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd5b804cb80 1 -- 172.21.15.115:0/783313794 shutdown_connections 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe60080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e7ebe744b0 0x55e7ebe768d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 0x55e7ebe60b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/783313794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe605c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd5b804cb80 1 -- 172.21.15.115:0/783313794 >> 172.21.15.115:0/783313794 conn(0x55e7ebcdcf70 msgr2=0x55e7ebe4aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.128+0000 7fd58a7fc700 0 client.15768 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:39:19.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.129+0000 7fd5b804cb80 1 -- 172.21.15.115:0/783313794 shutdown_connections 2024-05-09T05:39:19.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.129+0000 7fd5b804cb80 1 -- 172.21.15.115:0/783313794 wait complete. 2024-05-09T05:39:19.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.129+0000 7fd5b804cb80 20 client.15768 trim_cache size 0 max 16384 2024-05-09T05:39:19.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.143+0000 7fd5b804cb80 1 Processor -- start 2024-05-09T05:39:19.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5b804cb80 1 -- start start 2024-05-09T05:39:19.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebd53d00 0x55e7ebe60940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 0x55e7ebe4b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41ae230 con 0x55e7ebcd94a0 2024-05-09T05:39:19.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41afd70 con 0x55e7ebd53d00 2024-05-09T05:39:19.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41ae900 con 0x55e7ebe56400 2024-05-09T05:39:19.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49040/0 (socket says 172.21.15.115:49040) 2024-05-09T05:39:19.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/2708663824 learned_addr learned my addr 172.21.15.115:0/2708663824 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5a37fe700 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 0x55e7ebe4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.144+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebd53d00 0x55e7ebe60940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/2708663824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 msgr2=0x55e7ebe4b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 0x55e7ebe4b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/2708663824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebd53d00 msgr2=0x55e7ebe60940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebd53d00 0x55e7ebe60940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd5a2ffd700 1 -- 172.21.15.115:0/2708663824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ebe4d9d0 con 0x55e7ebcd94a0 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd5a37fe700 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 0x55e7ebe4b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebd53d00 0x55e7ebe60940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5ece0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd5940a50a0 tx=0x7fd5940a4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11274eee12f8108 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd598ff9700 1 -- 172.21.15.115:0/2708663824 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5940ad3d0 con 0x55e7ebcd94a0 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd598ff9700 1 -- 172.21.15.115:0/2708663824 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5940ad0b0 con 0x55e7ebcd94a0 2024-05-09T05:39:19.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.145+0000 7fd598ff9700 1 -- 172.21.15.115:0/2708663824 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5940df840 con 0x55e7ebcd94a0 2024-05-09T05:39:19.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 -- 172.21.15.115:0/2708663824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebcd94a0 msgr2=0x55e7ebe5ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5ece0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd5940a50a0 tx=0x7fd5940a4f90 comp rx=0 tx=0).stop 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 -- 172.21.15.115:0/2708663824 shutdown_connections 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebe56400 0x55e7ebe4b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe5ece0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/2708663824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebd53d00 0x55e7ebe60940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 -- 172.21.15.115:0/2708663824 >> 172.21.15.115:0/2708663824 conn(0x55e7ebcdcf70 msgr2=0x55e7ebe4aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 -- 172.21.15.115:0/2708663824 shutdown_connections 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 -- 172.21.15.115:0/2708663824 wait complete. 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.146+0000 7fd5b804cb80 1 -- start start 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5b804cb80 1 Processor -- start 2024-05-09T05:39:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe7c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe6c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5b804cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 0x55e7ebe6cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41af6a0 con 0x55e7ebd53d00 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41aefd0 con 0x55e7ebe56400 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5b804cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41ac1f0 con 0x55e7ebcd94a0 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5b804cb80 10 client.0 _ll_register_callbacks cb 0x55e7ebe51880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a27fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe6c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a27fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe6c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49050/0 (socket says 172.21.15.115:49050) 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a27fc700 1 -- 172.21.15.115:0/1874917015 learned_addr learned my addr 172.21.15.115:0/1874917015 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a37fe700 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 0x55e7ebe6cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe7c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a27fc700 1 -- 172.21.15.115:0/1874917015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 msgr2=0x55e7ebe7c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe7c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a27fc700 1 -- 172.21.15.115:0/1874917015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 msgr2=0x55e7ebe6cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 0x55e7ebe6cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.147+0000 7fd5a27fc700 1 -- 172.21.15.115:0/1874917015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ebe57600 con 0x55e7ebd53d00 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd5a27fc700 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe6c560 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd58c005950 tx=0x7fd58c01bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276090ec4c1353e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd58a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd58a7fc700 1 -- 172.21.15.115:0/1874917015 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c038020 con 0x55e7ebd53d00 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1874917015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5940ad0b0 con 0x55e7ebd53d00 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd58a7fc700 1 -- 172.21.15.115:0/1874917015 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd58c010630 con 0x55e7ebd53d00 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd58a7fc700 1 -- 172.21.15.115:0/1874917015 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd58c043050 con 0x55e7ebd53d00 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd5b804cb80 10 client.15774 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1874917015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7ebedde50 con 0x55e7ebd53d00 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd58a7fc700 1 -- 172.21.15.115:0/1874917015 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5940ad0b0 con 0x55e7ebd53d00 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd58a7fc700 1 -- 172.21.15.115:0/1874917015 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd58c024040 con 0x55e7ebd53d00 2024-05-09T05:39:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd5b804cb80 10 client.15774 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd5b804cb80 10 client.15774 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd5b804cb80 20 client.15774 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.148+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e7ebede580 0x55e7ebee09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.149+0000 7fd5b804cb80 4 client.15774 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.149+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1874917015 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7ebd80b20 con 0x55e7ebede580 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.149+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e7ebede580 0x55e7ebee09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.149+0000 7fd58a7fc700 10 client.15774 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.149+0000 7fd5a2ffd700 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e7ebede580 0x55e7ebee09a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=979eed3da759b4fe server_cookie=d33c57acf0555c0 in_seq=0 out_seq=0 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.149+0000 7fd58a7fc700 1 -- 172.21.15.115:0/1874917015 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7ebd80b20 con 0x55e7ebede580 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.149+0000 7fd58a7fc700 10 client.15774 handle_command_reply: tid=0 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.149+0000 7fd5b804cb80 1 client.15774 shutdown 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1874917015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 msgr2=0x55e7ebe6c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe6c560 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd58c005950 tx=0x7fd58c01bd00 comp rx=0 tx=0).stop 2024-05-09T05:39:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1874917015 shutdown_connections 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7ebe56400 0x55e7ebe6cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e7ebede580 0x55e7ebee09a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7ebcd94a0 0x55e7ebe7c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd58a7fc700 0 client.15774 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 --2- 172.21.15.115:0/1874917015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7ebd53d00 0x55e7ebe6c560 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1874917015 >> 172.21.15.115:0/1874917015 conn(0x55e7ebcdcf70 msgr2=0x55e7ebe59cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1874917015 shutdown_connections 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 1 -- 172.21.15.115:0/1874917015 wait complete. 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5b804cb80 20 client.15774 trim_cache size 0 max 16384 2024-05-09T05:39:19.148 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:19.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5a9636700 1 -- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd5900b2b90 msgr2=0x7fd5900b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.150+0000 7fd5a9636700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd5900b2b90 0x7fd5900b50a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4106960 tx=0x7fd58c00a300 comp rx=0 tx=0).stop 2024-05-09T05:39:19.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 -- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a41038f0 msgr2=0x7fd5a419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a41038f0 0x7fd5a419bb20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd594007f10 tx=0x7fd59400d3b0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 -- 172.21.15.115:0/76027522 shutdown_connections 2024-05-09T05:39:19.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd5900b2b90 0x7fd5900b50a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5a4108200 0x7fd5a419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5a41038f0 0x7fd5a419bb20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 --2- 172.21.15.115:0/76027522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5a4104240 0x7fd5a419c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 -- 172.21.15.115:0/76027522 >> 172.21.15.115:0/76027522 conn(0x7fd5a40fe6f0 msgr2=0x7fd5a4109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 -- 172.21.15.115:0/76027522 shutdown_connections 2024-05-09T05:39:19.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:19.151+0000 7fd5a9636700 1 -- 172.21.15.115:0/76027522 wait complete. 2024-05-09T05:39:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.231+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2506194046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d410e6b0 msgr2=0x7fb6d4110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.231+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2506194046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d410e6b0 0x7fb6d4110ad0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009900 tx=0x7fb6cc02a410 comp rx=0 tx=0).stop 2024-05-09T05:39:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.231+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2506194046 shutdown_connections 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.231+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2506194046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d410e6b0 0x7fb6d4110ad0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.231+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2506194046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6d41021b0 0x7fb6d41025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.231+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2506194046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6d4109d00 0x7fb6d4101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.231+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2506194046 >> 172.21.15.115:0/2506194046 conn(0x7fb6d40fda20 msgr2=0x7fb6d40ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.231+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2506194046 shutdown_connections 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.231+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2506194046 wait complete. 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.232+0000 7fb6d9f4d640 1 Processor -- start 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.232+0000 7fb6d9f4d640 1 -- start start 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.232+0000 7fb6d9f4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6d41021b0 0x7fb6d4102e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.232+0000 7fb6d9f4d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d4109d00 0x7fb6d4103390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.232+0000 7fb6d9f4d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6d410e6b0 0x7fb6d41038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.232+0000 7fb6d9f4d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4113d50 con 0x7fb6d4109d00 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.232+0000 7fb6d9f4d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4113bd0 con 0x7fb6d410e6b0 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.232+0000 7fb6d9f4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4113ed0 con 0x7fb6d41021b0 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d4109d00 0x7fb6d4103390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d4109d00 0x7fb6d4103390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49076/0 (socket says 172.21.15.115:49076) 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d2ffd640 1 -- 172.21.15.115:0/2790537716 learned_addr learned my addr 172.21.15.115:0/2790537716 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d37fe640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6d41021b0 0x7fb6d4102e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6d410e6b0 0x7fb6d41038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d2ffd640 1 -- 172.21.15.115:0/2790537716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6d41021b0 msgr2=0x7fb6d4102e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6d41021b0 0x7fb6d4102e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d2ffd640 1 -- 172.21.15.115:0/2790537716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6d410e6b0 msgr2=0x7fb6d41038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6d410e6b0 0x7fb6d41038d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d2ffd640 1 -- 172.21.15.115:0/2790537716 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41a8a70 con 0x7fb6d4109d00 2024-05-09T05:39:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d4109d00 0x7fb6d4103390 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800e9a0 tx=0x7fb6c800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d0ff9640 1 -- 172.21.15.115:0/2790537716 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c800cd80 con 0x7fb6d4109d00 2024-05-09T05:39:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2790537716 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d41a8d60 con 0x7fb6d4109d00 2024-05-09T05:39:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d0ff9640 1 -- 172.21.15.115:0/2790537716 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6c80035f0 con 0x7fb6d4109d00 2024-05-09T05:39:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d0ff9640 1 -- 172.21.15.115:0/2790537716 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8010690 con 0x7fb6d4109d00 2024-05-09T05:39:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.233+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2790537716 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d41a91f0 con 0x7fb6d4109d00 2024-05-09T05:39:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6d0ff9640 1 -- 172.21.15.115:0/2790537716 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7fb6c8004000 con 0x7fb6d4109d00 2024-05-09T05:39:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6e0e00740 1 Processor -- start 2024-05-09T05:39:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6e0e00740 1 -- start start 2024-05-09T05:39:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1e9e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ad22f0 0x5578d1e9ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 0x5578d1e9f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5578d1e037c0 con 0x5578d1afb1b0 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5578d1d9f0e0 con 0x5578d1ada4d0 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5578d1dff780 con 0x5578d1ad22f0 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.236+0000 7fb6d3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1e9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1e9e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49086/0 (socket says 172.21.15.115:49086) 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 -- 172.21.15.115:0/474769298 learned_addr learned my addr 172.21.15.115:0/474769298 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d37fe640 1 --2- 172.21.15.115:0/474769298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ad22f0 0x5578d1e9ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d0ff9640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb6bc0b2a70 0x7fb6bc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/474769298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 0x5578d1e9f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 -- 172.21.15.115:0/474769298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ad22f0 msgr2=0x5578d1e9ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/474769298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ad22f0 0x5578d1e9ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 -- 172.21.15.115:0/474769298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 msgr2=0x5578d1e9f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/474769298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 0x5578d1e9f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 -- 172.21.15.115:0/474769298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d1dc87c0 con 0x5578d1afb1b0 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb6bc0b2a70 0x7fb6bc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d0ff9640 1 -- 172.21.15.115:0/2790537716 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fb6c80e2fe0 con 0x7fb6d4109d00 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/474769298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1e9e680 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc02a930 tx=0x7fb6cc004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0b49ce0e0b922a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6b9ffb640 1 -- 172.21.15.115:0/474769298 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc035040 con 0x5578d1afb1b0 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6b9ffb640 1 -- 172.21.15.115:0/474769298 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6cc004b50 con 0x5578d1afb1b0 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6b9ffb640 1 -- 172.21.15.115:0/474769298 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0039b0 con 0x5578d1afb1b0 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.237+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb6bc0b2a70 0x7fb6bc0b4f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc02aa00 tx=0x7fb6cc04a000 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.238+0000 7fb6e0e00740 1 -- 172.21.15.115:0/474769298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 msgr2=0x5578d1e9e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.238+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/474769298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1e9e680 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc02a930 tx=0x7fb6cc004350 comp rx=0 tx=0).stop 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.238+0000 7fb6e0e00740 1 -- 172.21.15.115:0/474769298 shutdown_connections 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.238+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/474769298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 0x5578d1e9f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.238+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/474769298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ad22f0 0x5578d1e9ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.238+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/474769298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1e9e680 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.238+0000 7fb6e0e00740 1 -- 172.21.15.115:0/474769298 >> 172.21.15.115:0/474769298 conn(0x5578d1aef300 msgr2=0x5578d1e9ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.238+0000 7fb6e0e00740 1 -- 172.21.15.115:0/474769298 shutdown_connections 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.238+0000 7fb6e0e00740 1 -- 172.21.15.115:0/474769298 wait complete. 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6e0e00740 1 -- start start 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6e0e00740 1 Processor -- start 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 0x5578d1eaac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1afb1b0 0x5578d1ea8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5578d1eab7b0 con 0x5578d1ad22f0 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5578d1d8a9c0 con 0x5578d1afb1b0 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5578d1eab930 con 0x5578d1ada4d0 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6e0e00740 10 client.0 _ll_register_callbacks cb 0x5578d1db7020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6d37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6d37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49102/0 (socket says 172.21.15.115:49102) 2024-05-09T05:39:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6d37fe640 1 -- 172.21.15.115:0/3496091410 learned_addr learned my addr 172.21.15.115:0/3496091410 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 0x5578d1eaac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.239+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1afb1b0 0x5578d1ea8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6d37fe640 1 -- 172.21.15.115:0/3496091410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 msgr2=0x5578d1eaac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6d37fe640 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 0x5578d1eaac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6d37fe640 1 -- 172.21.15.115:0/3496091410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1afb1b0 msgr2=0x5578d1ea8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6d37fe640 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1afb1b0 0x5578d1ea8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6d37fe640 1 -- 172.21.15.115:0/3496091410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d1db11f0 con 0x5578d1ad22f0 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6d37fe640 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaa730 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4005fd0 tx=0x7fb6c4013cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a35c4ac5730e4f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/3496091410 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4011e30 con 0x5578d1ad22f0 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/3496091410 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fb6c4003970 con 0x5578d1ad22f0 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6e0e00740 1 -- 172.21.15.115:0/3496091410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6cc004b50 con 0x5578d1ad22f0 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/3496091410 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6c4003b70 con 0x5578d1ad22f0 2024-05-09T05:39:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6e0e00740 10 client.15786 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6e0e00740 1 -- 172.21.15.115:0/3496091410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578d1b66790 con 0x5578d1ad22f0 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/3496091410 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6cc004b50 con 0x5578d1ad22f0 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/3496091410 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb6c401a070 con 0x5578d1ad22f0 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.240+0000 7fb6e0e00740 10 client.15786 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.241+0000 7fb6e0e00740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.241+0000 7fb6e0e00740 20 client.15786 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.241+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5578d1ab9d80 0x5578d1ebb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.241+0000 7fb6e0e00740 4 client.15786 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.241+0000 7fb6e0e00740 1 -- 172.21.15.115:0/3496091410 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d1e113d0 con 0x5578d1ab9d80 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.241+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5578d1ab9d80 0x5578d1ebb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.241+0000 7fb6b2ffd640 10 client.15786 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.241+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5578d1ab9d80 0x5578d1ebb110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6526dc9c204908a server_cookie=6dc1b553d9ca1f1c in_seq=0 out_seq=0 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/3496091410 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578d1e113d0 con 0x5578d1ab9d80 2024-05-09T05:39:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6b2ffd640 10 client.15786 handle_command_reply: tid=0 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 client.15786 shutdown 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 -- 172.21.15.115:0/3496091410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 msgr2=0x5578d1eaa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaa730 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4005fd0 tx=0x7fb6c4013cb0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 -- 172.21.15.115:0/3496091410 shutdown_connections 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1afb1b0 0x5578d1ea8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5578d1ab9d80 0x5578d1ebb110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 0x5578d1eaac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/3496091410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaa730 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6b2ffd640 0 client.15786 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 -- 172.21.15.115:0/3496091410 >> 172.21.15.115:0/3496091410 conn(0x5578d1aef300 msgr2=0x5578d1e99700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 -- 172.21.15.115:0/3496091410 shutdown_connections 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 1 -- 172.21.15.115:0/3496091410 wait complete. 2024-05-09T05:39:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.242+0000 7fb6e0e00740 20 client.15786 trim_cache size 0 max 16384 2024-05-09T05:39:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.257+0000 7fb6e0e00740 1 Processor -- start 2024-05-09T05:39:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.257+0000 7fb6e0e00740 1 -- start start 2024-05-09T05:39:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.257+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ad22f0 0x5578d1e9ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.257+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 0x5578d1ec0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.257+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1ec0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.257+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b3280 con 0x5578d1afb1b0 2024-05-09T05:39:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.257+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b4dc0 con 0x5578d1ad22f0 2024-05-09T05:39:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.257+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b3950 con 0x5578d1ada4d0 2024-05-09T05:39:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1ec0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1ec0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49114/0 (socket says 172.21.15.115:49114) 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d3fff640 1 -- 172.21.15.115:0/2071633389 learned_addr learned my addr 172.21.15.115:0/2071633389 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d37fe640 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ad22f0 0x5578d1e9ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 0x5578d1ec0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d3fff640 1 -- 172.21.15.115:0/2071633389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 msgr2=0x5578d1ec0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 0x5578d1ec0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d3fff640 1 -- 172.21.15.115:0/2071633389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ad22f0 msgr2=0x5578d1e9ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ad22f0 0x5578d1e9ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d3fff640 1 -- 172.21.15.115:0/2071633389 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d1bdd5f0 con 0x5578d1afb1b0 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 0x5578d1ec0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6d3fff640 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1ec0e50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0057c0 tx=0x7fb6cc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8964a447b72a6ba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6b97fa640 1 -- 172.21.15.115:0/2071633389 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc035030 con 0x5578d1afb1b0 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6b97fa640 1 -- 172.21.15.115:0/2071633389 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6cc003990 con 0x5578d1afb1b0 2024-05-09T05:39:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.258+0000 7fb6b97fa640 1 -- 172.21.15.115:0/2071633389 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc004920 con 0x5578d1afb1b0 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 -- 172.21.15.115:0/2071633389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 msgr2=0x5578d1ec0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1ec0e50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0057c0 tx=0x7fb6cc004610 comp rx=0 tx=0).stop 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 -- 172.21.15.115:0/2071633389 shutdown_connections 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1afb1b0 0x5578d1ec0e50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1ada4d0 0x5578d1ec0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/2071633389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ad22f0 0x5578d1e9ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 -- 172.21.15.115:0/2071633389 >> 172.21.15.115:0/2071633389 conn(0x5578d1aef300 msgr2=0x5578d1e9a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 -- 172.21.15.115:0/2071633389 shutdown_connections 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 -- 172.21.15.115:0/2071633389 wait complete. 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.259+0000 7fb6e0e00740 1 -- start start 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6e0e00740 1 Processor -- start 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 0x5578d1eaf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6e0e00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1afb1b0 0x5578d1f224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b46f0 con 0x5578d1ad22f0 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b4020 con 0x5578d1ada4d0 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6e0e00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b1260 con 0x5578d1afb1b0 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6e0e00740 10 client.0 _ll_register_callbacks cb 0x7fb6c4009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49128/0 (socket says 172.21.15.115:49128) 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d37fe640 1 -- 172.21.15.115:0/115266321 learned_addr learned my addr 172.21.15.115:0/115266321 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 0x5578d1eaf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d37fe640 1 -- 172.21.15.115:0/115266321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1afb1b0 msgr2=0x5578d1f224d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d37fe640 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1afb1b0 0x5578d1f224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d37fe640 1 -- 172.21.15.115:0/115266321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 msgr2=0x5578d1eaf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d37fe640 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 0x5578d1eaf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d37fe640 1 -- 172.21.15.115:0/115266321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d1dc7ed0 con 0x5578d1ad22f0 2024-05-09T05:39:19.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6d37fe640 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaeb00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4005fd0 tx=0x7fb6c4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4fea1f1feb4c71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.260+0000 7fb6b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/115266321 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c402c020 con 0x5578d1ad22f0 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/115266321 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fb6c401b440 con 0x5578d1ad22f0 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6e0e00740 1 -- 172.21.15.115:0/115266321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6cc003990 con 0x5578d1ad22f0 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/115266321 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6c4037050 con 0x5578d1ad22f0 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6e0e00740 10 client.15792 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6e0e00740 1 -- 172.21.15.115:0/115266321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578d1f237c0 con 0x5578d1ad22f0 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/115266321 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6cc003990 con 0x5578d1ad22f0 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/115266321 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb6c401a040 con 0x5578d1ad22f0 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6e0e00740 10 client.15792 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6e0e00740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6e0e00740 20 client.15792 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5578d1a8f1c0 0x5578d1f26050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6e0e00740 4 client.15792 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:39:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6e0e00740 1 -- 172.21.15.115:0/115266321 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578d1e0b240 con 0x5578d1a8f1c0 2024-05-09T05:39:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.261+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5578d1a8f1c0 0x5578d1f26050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.262+0000 7fb6d2ffd640 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5578d1a8f1c0 0x5578d1f26050 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e61a99f4208076c server_cookie=12cd1b7ae45a377e in_seq=0 out_seq=0 2024-05-09T05:39:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.262+0000 7fb6b2ffd640 10 client.15792 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6b2ffd640 1 -- 172.21.15.115:0/115266321 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5578d1e0b240 con 0x5578d1a8f1c0 2024-05-09T05:39:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6b2ffd640 10 client.15792 handle_command_reply: tid=0 2024-05-09T05:39:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 client.15792 shutdown 2024-05-09T05:39:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 -- 172.21.15.115:0/115266321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 msgr2=0x5578d1eaeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaeb00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4005fd0 tx=0x7fb6c4011e60 comp rx=0 tx=0).stop 2024-05-09T05:39:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 -- 172.21.15.115:0/115266321 shutdown_connections 2024-05-09T05:39:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5578d1a8f1c0 0x5578d1f26050 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5578d1afb1b0 0x5578d1f224d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5578d1ada4d0 0x5578d1eaf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6b2ffd640 0 client.15792 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 --2- 172.21.15.115:0/115266321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5578d1ad22f0 0x5578d1eaeb00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 -- 172.21.15.115:0/115266321 >> 172.21.15.115:0/115266321 conn(0x5578d1aef300 msgr2=0x5578d1eaa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 -- 172.21.15.115:0/115266321 shutdown_connections 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 1 -- 172.21.15.115:0/115266321 wait complete. 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.266+0000 7fb6e0e00740 20 client.15792 trim_cache size 0 max 16384 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:39:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:39:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb6bc0b2a70 msgr2=0x7fb6bc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb6bc0b2a70 0x7fb6bc0b4f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc02aa00 tx=0x7fb6cc04a000 comp rx=0 tx=0).stop 2024-05-09T05:39:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d4109d00 msgr2=0x7fb6d4103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d4109d00 0x7fb6d4103390 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800e9a0 tx=0x7fb6c800ee60 comp rx=0 tx=0).stop 2024-05-09T05:39:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2790537716 shutdown_connections 2024-05-09T05:39:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb6bc0b2a70 0x7fb6bc0b4f80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6d410e6b0 0x7fb6d41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6d4109d00 0x7fb6d4103390 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 --2- 172.21.15.115:0/2790537716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6d41021b0 0x7fb6d4102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2790537716 >> 172.21.15.115:0/2790537716 conn(0x7fb6d40fda20 msgr2=0x7fb6d40ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2790537716 shutdown_connections 2024-05-09T05:39:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:19.267+0000 7fb6d9f4d640 1 -- 172.21.15.115:0/2790537716 wait complete. 2024-05-09T05:39:19.734 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:19 smithi115 ceph-mon[30349]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-05-09T05:39:19.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:19 smithi195 ceph-mon[33844]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-05-09T05:39:19.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:19 smithi161 ceph-mon[33805]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-05-09T05:39:19.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:39:20.204 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:21.751 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:21 smithi115 ceph-mon[30349]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:39:21.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:21 smithi195 ceph-mon[33844]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:39:21.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:21 smithi161 ceph-mon[33805]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:39:22.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.016+0000 7ff2127dd700 1 -- 172.21.15.115:0/617573967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 msgr2=0x7ff20c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:22.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.016+0000 7ff2127dd700 1 --2- 172.21.15.115:0/617573967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 0x7ff20c1038d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40099c0 tx=0x7ff1f4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.016+0000 7ff2127dd700 1 -- 172.21.15.115:0/617573967 shutdown_connections 2024-05-09T05:39:22.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.016+0000 7ff2127dd700 1 --2- 172.21.15.115:0/617573967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff20c108740 0x7ff20c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.016+0000 7ff2127dd700 1 --2- 172.21.15.115:0/617573967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 0x7ff20c1038d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.016+0000 7ff2127dd700 1 --2- 172.21.15.115:0/617573967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c107df0 0x7ff20c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.016+0000 7ff2127dd700 1 -- 172.21.15.115:0/617573967 >> 172.21.15.115:0/617573967 conn(0x7ff20c0fedc0 msgr2=0x7ff20c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:22.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.016+0000 7ff2127dd700 1 -- 172.21.15.115:0/617573967 shutdown_connections 2024-05-09T05:39:22.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.016+0000 7ff2127dd700 1 -- 172.21.15.115:0/617573967 wait complete. 2024-05-09T05:39:22.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.017+0000 7ff2127dd700 1 Processor -- start 2024-05-09T05:39:22.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.017+0000 7ff2127dd700 1 -- start start 2024-05-09T05:39:22.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff2127dd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 0x7ff20c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff2127dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff20c107df0 0x7ff20c19c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff2127dd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c108740 0x7ff20c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff2127dd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1132a0 con 0x7ff20c1034c0 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff2127dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c113120 con 0x7ff20c107df0 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff2127dd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c113420 con 0x7ff20c108740 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 0x7ff20c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 0x7ff20c19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49148/0 (socket says 172.21.15.115:49148) 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20bfff700 1 -- 172.21.15.115:0/5672105 learned_addr learned my addr 172.21.15.115:0/5672105 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20b7fe700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff20c107df0 0x7ff20c19c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff210d7a700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c108740 0x7ff20c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20bfff700 1 -- 172.21.15.115:0/5672105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c108740 msgr2=0x7ff20c19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20bfff700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c108740 0x7ff20c19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20bfff700 1 -- 172.21.15.115:0/5672105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff20c107df0 msgr2=0x7ff20c19c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20bfff700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff20c107df0 0x7ff20c19c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20bfff700 1 -- 172.21.15.115:0/5672105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c1a6f50 con 0x7ff20c1034c0 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.018+0000 7ff20bfff700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 0x7ff20c19bbe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4000c00 tx=0x7ff1f400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:22.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.019+0000 7ff2097fa700 1 -- 172.21.15.115:0/5672105 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4026070 con 0x7ff20c1034c0 2024-05-09T05:39:22.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.019+0000 7ff2097fa700 1 -- 172.21.15.115:0/5672105 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1f4005030 con 0x7ff20c1034c0 2024-05-09T05:39:22.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.019+0000 7ff2127dd700 1 -- 172.21.15.115:0/5672105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20c1a71e0 con 0x7ff20c1034c0 2024-05-09T05:39:22.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.019+0000 7ff2097fa700 1 -- 172.21.15.115:0/5672105 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4004b00 con 0x7ff20c1034c0 2024-05-09T05:39:22.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.019+0000 7ff2127dd700 1 -- 172.21.15.115:0/5672105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20c1a7630 con 0x7ff20c1034c0 2024-05-09T05:39:22.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.022+0000 7ff2097fa700 1 -- 172.21.15.115:0/5672105 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7ff1f4029070 con 0x7ff20c1034c0 2024-05-09T05:39:22.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.022+0000 7ff2097fa700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1f80b2b10 0x7ff1f80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.022+0000 7ff2097fa700 1 -- 172.21.15.115:0/5672105 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff1f40e35f0 con 0x7ff20c1034c0 2024-05-09T05:39:22.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.022+0000 7ff20b7fe700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1f80b2b10 0x7ff1f80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff2211f3b80 1 Processor -- start 2024-05-09T05:39:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff2211f3b80 1 -- start start 2024-05-09T05:39:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff20b7fe700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1f80b2b10 0x7ff1f80b5020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff20c106630 tx=0x7ff1fc00a300 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994aa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d1993c3ff0 con 0x55d1994aa400 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d1993627f0 con 0x55d1993244a0 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d1993bffb0 con 0x55d19940e160 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff210d7a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff210d7a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994aa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49158/0 (socket says 172.21.15.115:49158) 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff210d7a700 1 -- 172.21.15.115:0/51708328 learned_addr learned my addr 172.21.15.115:0/51708328 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff20b7fe700 1 --2- 172.21.15.115:0/51708328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff20bfff700 1 --2- 172.21.15.115:0/51708328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff210d7a700 1 -- 172.21.15.115:0/51708328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 msgr2=0x55d1994a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff210d7a700 1 --2- 172.21.15.115:0/51708328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff210d7a700 1 -- 172.21.15.115:0/51708328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 msgr2=0x55d1994a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff210d7a700 1 --2- 172.21.15.115:0/51708328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994a62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.023+0000 7ff210d7a700 1 -- 172.21.15.115:0/51708328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d19935a370 con 0x55d1994aa400 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.024+0000 7ff210d7a700 1 --2- 172.21.15.115:0/51708328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994aa810 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff20400a400 tx=0x7ff204009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce009c8ddc70b26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.024+0000 7ff2027fc700 1 -- 172.21.15.115:0/51708328 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204014620 con 0x55d1994aa400 2024-05-09T05:39:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.024+0000 7ff2027fc700 1 -- 172.21.15.115:0/51708328 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff204019070 con 0x55d1994aa400 2024-05-09T05:39:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.024+0000 7ff2027fc700 1 -- 172.21.15.115:0/51708328 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204013940 con 0x55d1994aa400 2024-05-09T05:39:22.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 -- 172.21.15.115:0/51708328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 msgr2=0x55d1994aa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:22.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/51708328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994aa810 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff20400a400 tx=0x7ff204009650 comp rx=0 tx=0).stop 2024-05-09T05:39:22.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 -- 172.21.15.115:0/51708328 shutdown_connections 2024-05-09T05:39:22.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/51708328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994aa810 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/51708328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/51708328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 -- 172.21.15.115:0/51708328 >> 172.21.15.115:0/51708328 conn(0x55d1993846a0 msgr2=0x55d19949c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 -- 172.21.15.115:0/51708328 shutdown_connections 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 -- 172.21.15.115:0/51708328 wait complete. 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 -- start start 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.025+0000 7ff2211f3b80 1 Processor -- start 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d19923fba0 con 0x55d1994aa400 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d19937e150 con 0x55d1993244a0 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d1994ab1b0 con 0x55d19940e160 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff2211f3b80 10 client.0 _ll_register_callbacks cb 0x55d1992356c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff210d7a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff210d7a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49160/0 (socket says 172.21.15.115:49160) 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff210d7a700 1 -- 172.21.15.115:0/2229428534 learned_addr learned my addr 172.21.15.115:0/2229428534 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff20b7fe700 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff20bfff700 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff210d7a700 1 -- 172.21.15.115:0/2229428534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 msgr2=0x55d1994afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff210d7a700 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff210d7a700 1 -- 172.21.15.115:0/2229428534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 msgr2=0x55d1994af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff210d7a700 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff210d7a700 1 -- 172.21.15.115:0/2229428534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d19923f490 con 0x55d1994aa400 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff210d7a700 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994b0010 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff204004b10 tx=0x7ff2040137f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf035a2c8e203b32 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.026+0000 7ff1eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2229428534 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204013cd0 con 0x55d1994aa400 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2229428534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff204019070 con 0x55d1994aa400 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2229428534 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff204013e70 con 0x55d1994aa400 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2229428534 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff20402c5f0 con 0x55d1994aa400 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff2211f3b80 10 client.15804 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2229428534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d1994bceb0 con 0x55d1994aa400 2024-05-09T05:39:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2229428534 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff204019070 con 0x55d1994aa400 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2229428534 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff204027430 con 0x55d1994aa400 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff2211f3b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff2211f3b80 10 client.15804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff2211f3b80 20 client.15804 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55d1994bf440 0x55d1994c1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff2211f3b80 4 client.15804 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.027+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2229428534 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1993d19d0 con 0x55d1994bf440 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.028+0000 7ff20bfff700 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55d1994bf440 0x55d1994c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.028+0000 7ff1eb7fe700 10 client.15804 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.028+0000 7ff20bfff700 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55d1994bf440 0x55d1994c1860 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d62d793f2e33f70a server_cookie=5ccfe7053751fde4 in_seq=0 out_seq=0 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2229428534 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1993d19d0 con 0x55d1994bf440 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff1eb7fe700 10 client.15804 handle_command_reply: tid=0 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 client.15804 shutdown 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2229428534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 msgr2=0x55d1994b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994b0010 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff204004b10 tx=0x7ff2040137f0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2229428534 shutdown_connections 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55d1994bf440 0x55d1994c1860 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994b0010 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2229428534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff1eb7fe700 0 client.15804 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2229428534 >> 172.21.15.115:0/2229428534 conn(0x55d1993846a0 msgr2=0x55d19949c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2229428534 shutdown_connections 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2229428534 wait complete. 2024-05-09T05:39:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.029+0000 7ff2211f3b80 20 client.15804 trim_cache size 0 max 16384 2024-05-09T05:39:22.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff2211f3b80 1 Processor -- start 2024-05-09T05:39:22.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff2211f3b80 1 -- start start 2024-05-09T05:39:22.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1993244a0 0x55d1994ab780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d19940e160 0x55d1994ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1994aa400 0x55d1994a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1ae0b0 con 0x55d19940e160 2024-05-09T05:39:22.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1afbf0 con 0x55d1994aa400 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1ae780 con 0x55d1993244a0 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff20b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d19940e160 0x55d1994ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff20b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d19940e160 0x55d1994ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49170/0 (socket says 172.21.15.115:49170) 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff20b7fe700 1 -- 172.21.15.115:0/3123801313 learned_addr learned my addr 172.21.15.115:0/3123801313 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff210d7a700 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1994aa400 0x55d1994a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.044+0000 7ff20bfff700 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1993244a0 0x55d1994ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff20b7fe700 1 -- 172.21.15.115:0/3123801313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1993244a0 msgr2=0x55d1994ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff20b7fe700 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1993244a0 0x55d1994ab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff20b7fe700 1 -- 172.21.15.115:0/3123801313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1994aa400 msgr2=0x55d1994a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff20b7fe700 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1994aa400 0x55d1994a63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff20b7fe700 1 -- 172.21.15.115:0/3123801313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1994abfa0 con 0x55d19940e160 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff210d7a700 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1994aa400 0x55d1994a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff20b7fe700 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d19940e160 0x55d1994ad1e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc01d530 tx=0x7ff1fc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3025df914c5ac93f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff201ffb700 1 -- 172.21.15.115:0/3123801313 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc016070 con 0x55d19940e160 2024-05-09T05:39:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff201ffb700 1 -- 172.21.15.115:0/3123801313 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1fc01dcf0 con 0x55d19940e160 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.045+0000 7ff201ffb700 1 -- 172.21.15.115:0/3123801313 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc005510 con 0x55d19940e160 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 -- 172.21.15.115:0/3123801313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d19940e160 msgr2=0x55d1994ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d19940e160 0x55d1994ad1e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc01d530 tx=0x7ff1fc012000 comp rx=0 tx=0).stop 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 -- 172.21.15.115:0/3123801313 shutdown_connections 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1994aa400 0x55d1994a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1993244a0 0x55d1994ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/3123801313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d19940e160 0x55d1994ad1e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 -- 172.21.15.115:0/3123801313 >> 172.21.15.115:0/3123801313 conn(0x55d1993846a0 msgr2=0x55d1994ade80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 -- 172.21.15.115:0/3123801313 shutdown_connections 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 -- 172.21.15.115:0/3123801313 wait complete. 2024-05-09T05:39:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.046+0000 7ff2211f3b80 1 -- start start 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff2211f3b80 1 Processor -- start 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994a7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff2211f3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1af520 con 0x55d1994aa400 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1aee50 con 0x55d1993244a0 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff2211f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1ac070 con 0x55d19940e160 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff2211f3b80 10 client.0 _ll_register_callbacks cb 0x55d1992356c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff210d7a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff210d7a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994a7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49186/0 (socket says 172.21.15.115:49186) 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff210d7a700 1 -- 172.21.15.115:0/2681247165 learned_addr learned my addr 172.21.15.115:0/2681247165 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff20bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff20b7fe700 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff210d7a700 1 -- 172.21.15.115:0/2681247165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 msgr2=0x55d1994a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff210d7a700 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994a7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff210d7a700 1 -- 172.21.15.115:0/2681247165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 msgr2=0x55d1994b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff210d7a700 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994b7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff210d7a700 1 -- 172.21.15.115:0/2681247165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1994a8280 con 0x55d1994aa400 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.047+0000 7ff20bfff700 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff210d7a700 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994a7d40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff20400afa0 tx=0x7ff2040276e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1f47fc6ce26a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff1eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2681247165 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204027c70 con 0x55d1994aa400 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2681247165 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff204013ad0 con 0x55d1994aa400 2024-05-09T05:39:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2681247165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1fc01dcf0 con 0x55d1994aa400 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2681247165 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff20400a9f0 con 0x55d1994aa400 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff2211f3b80 10 client.15810 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2681247165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d1994b7430 con 0x55d1994aa400 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2681247165 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1fc01dcf0 con 0x55d1994aa400 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2681247165 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff204033020 con 0x55d1994aa400 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff2211f3b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff2211f3b80 10 client.15810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff2211f3b80 20 client.15810 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55d199529c10 0x55d19952c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff2211f3b80 4 client.15810 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.048+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2681247165 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1993cba70 con 0x55d199529c10 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.049+0000 7ff20bfff700 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55d199529c10 0x55d19952c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.049+0000 7ff1eb7fe700 10 client.15810 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.049+0000 7ff20bfff700 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55d199529c10 0x55d19952c030 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=914b7d3ca0ea8991 server_cookie=97f17b2dfdd88cb2 in_seq=0 out_seq=0 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.049+0000 7ff1eb7fe700 1 -- 172.21.15.115:0/2681247165 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1993cba70 con 0x55d199529c10 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.049+0000 7ff1eb7fe700 10 client.15810 handle_command_reply: tid=0 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.049+0000 7ff2211f3b80 1 client.15810 shutdown 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.049+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2681247165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 msgr2=0x55d1994a7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.049+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994a7d40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff20400afa0 tx=0x7ff2040276e0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2681247165 shutdown_connections 2024-05-09T05:39:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1993244a0 0x55d1994b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55d199529c10 0x55d19952c030 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1994aa400 0x55d1994a7d40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2211f3b80 1 --2- 172.21.15.115:0/2681247165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d19940e160 0x55d1994a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff1eb7fe700 0 client.15810 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2681247165 >> 172.21.15.115:0/2681247165 conn(0x55d1993846a0 msgr2=0x55d1994afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2681247165 shutdown_connections 2024-05-09T05:39:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2211f3b80 1 -- 172.21.15.115:0/2681247165 wait complete. 2024-05-09T05:39:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2211f3b80 20 client.15810 trim_cache size 0 max 16384 2024-05-09T05:39:22.050 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:22.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 -- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1f80b2b10 msgr2=0x7ff1f80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:22.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1f80b2b10 0x7ff1f80b5020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff20c106630 tx=0x7ff1fc00a300 comp rx=0 tx=0).stop 2024-05-09T05:39:22.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 -- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 msgr2=0x7ff20c19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:22.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 0x7ff20c19bbe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4000c00 tx=0x7ff1f400f690 comp rx=0 tx=0).stop 2024-05-09T05:39:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 -- 172.21.15.115:0/5672105 shutdown_connections 2024-05-09T05:39:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1f80b2b10 0x7ff1f80b5020 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c108740 0x7ff20c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff20c1034c0 0x7ff20c19bbe0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 --2- 172.21.15.115:0/5672105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff20c107df0 0x7ff20c19c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.050+0000 7ff2127dd700 1 -- 172.21.15.115:0/5672105 >> 172.21.15.115:0/5672105 conn(0x7ff20c0fedc0 msgr2=0x7ff20c101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.051+0000 7ff2127dd700 1 -- 172.21.15.115:0/5672105 shutdown_connections 2024-05-09T05:39:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:22.051+0000 7ff2127dd700 1 -- 172.21.15.115:0/5672105 wait complete. 2024-05-09T05:39:23.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:39:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:23.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:23 smithi115 ceph-mon[30349]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:39:23.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:23 smithi195 ceph-mon[33844]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:39:23.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:23 smithi161 ceph-mon[33805]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:39:25.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.162+0000 7f18b190f700 1 -- 172.21.15.115:0/1754268506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 msgr2=0x7f18ac108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:25.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.162+0000 7f18b190f700 1 --2- 172.21.15.115:0/1754268506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 0x7f18ac108200 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f18a00099c0 tx=0x7f18a0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.162+0000 7f18b190f700 1 -- 172.21.15.115:0/1754268506 shutdown_connections 2024-05-09T05:39:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.162+0000 7f18b190f700 1 --2- 172.21.15.115:0/1754268506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18ac108740 0x7f18ac1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.162+0000 7f18b190f700 1 --2- 172.21.15.115:0/1754268506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac1034c0 0x7f18ac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.162+0000 7f18b190f700 1 --2- 172.21.15.115:0/1754268506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 0x7f18ac108200 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.163+0000 7f18b190f700 1 -- 172.21.15.115:0/1754268506 >> 172.21.15.115:0/1754268506 conn(0x7f18ac0fedc0 msgr2=0x7f18ac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.163+0000 7f18b190f700 1 -- 172.21.15.115:0/1754268506 shutdown_connections 2024-05-09T05:39:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.163+0000 7f18b190f700 1 -- 172.21.15.115:0/1754268506 wait complete. 2024-05-09T05:39:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.163+0000 7f18b190f700 1 Processor -- start 2024-05-09T05:39:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.163+0000 7f18b190f700 1 -- start start 2024-05-09T05:39:25.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18b190f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac1034c0 0x7f18ac19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18b190f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 0x7f18ac19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18b190f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18ac108740 0x7f18ac1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18b190f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac113270 con 0x7f18ac107df0 2024-05-09T05:39:25.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18b190f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1130f0 con 0x7f18ac108740 2024-05-09T05:39:25.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18b190f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1133f0 con 0x7f18ac1034c0 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 0x7f18ac19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 0x7f18ac19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47162/0 (socket says 172.21.15.115:47162) 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aa7fc700 1 -- 172.21.15.115:0/1916160328 learned_addr learned my addr 172.21.15.115:0/1916160328 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18ab7fe700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18ac108740 0x7f18ac1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aaffd700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac1034c0 0x7f18ac19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aa7fc700 1 -- 172.21.15.115:0/1916160328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac1034c0 msgr2=0x7f18ac19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac1034c0 0x7f18ac19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aa7fc700 1 -- 172.21.15.115:0/1916160328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18ac108740 msgr2=0x7f18ac1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18ac108740 0x7f18ac1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aa7fc700 1 -- 172.21.15.115:0/1916160328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac1a1ec0 con 0x7f18ac107df0 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.164+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 0x7f18ac19d590 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b580 tx=0x7f18a0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.165+0000 7f18b090d700 1 -- 172.21.15.115:0/1916160328 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0027070 con 0x7f18ac107df0 2024-05-09T05:39:25.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.165+0000 7f18b190f700 1 -- 172.21.15.115:0/1916160328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ac1a2150 con 0x7f18ac107df0 2024-05-09T05:39:25.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.165+0000 7f18b090d700 1 -- 172.21.15.115:0/1916160328 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a000fc00 con 0x7f18ac107df0 2024-05-09T05:39:25.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.165+0000 7f18b090d700 1 -- 172.21.15.115:0/1916160328 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0022cb0 con 0x7f18ac107df0 2024-05-09T05:39:25.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.165+0000 7f18b190f700 1 -- 172.21.15.115:0/1916160328 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ac1a9ad0 con 0x7f18ac107df0 2024-05-09T05:39:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.167+0000 7f18b090d700 1 -- 172.21.15.115:0/1916160328 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f18a002a070 con 0x7f18ac107df0 2024-05-09T05:39:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.168+0000 7f18b090d700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f18940b2b10 0x7f18940b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.168+0000 7f18aaffd700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f18940b2b10 0x7f18940b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.168+0000 7f18b090d700 1 -- 172.21.15.115:0/1916160328 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f18a00e3a30 con 0x7f18ac107df0 2024-05-09T05:39:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18c0325b80 1 Processor -- start 2024-05-09T05:39:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18c0325b80 1 -- start start 2024-05-09T05:39:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aaffd700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f18940b2b10 0x7f18940b5020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f189c009a10 tx=0x7f189c009190 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 0x5594ae7c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7bead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18c0325b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5594ae6e0ff0 con 0x5594ae72b160 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18c0325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5594ae67f7f0 con 0x5594ae6414a0 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18c0325b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5594ae6dcfb0 con 0x5594ae7be6a0 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47164/0 (socket says 172.21.15.115:47164) 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aaffd700 1 -- 172.21.15.115:0/4157431861 learned_addr learned my addr 172.21.15.115:0/4157431861 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18ab7fe700 1 --2- 172.21.15.115:0/4157431861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7bead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/4157431861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 0x5594ae7c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aaffd700 1 -- 172.21.15.115:0/4157431861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 msgr2=0x5594ae7bead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aaffd700 1 --2- 172.21.15.115:0/4157431861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7bead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aaffd700 1 -- 172.21.15.115:0/4157431861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 msgr2=0x5594ae7c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aaffd700 1 --2- 172.21.15.115:0/4157431861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 0x5594ae7c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.169+0000 7f18aaffd700 1 -- 172.21.15.115:0/4157431861 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ae677370 con 0x5594ae72b160 2024-05-09T05:39:25.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.170+0000 7f18aaffd700 1 --2- 172.21.15.115:0/4157431861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c44f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f189c00ea00 tx=0x7f189c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2388b206b89d7120 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:25.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.170+0000 7f1891ffb700 1 -- 172.21.15.115:0/4157431861 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c01b040 con 0x5594ae72b160 2024-05-09T05:39:25.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.170+0000 7f1891ffb700 1 -- 172.21.15.115:0/4157431861 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f189c024070 con 0x5594ae72b160 2024-05-09T05:39:25.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.170+0000 7f1891ffb700 1 -- 172.21.15.115:0/4157431861 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0136b0 con 0x5594ae72b160 2024-05-09T05:39:25.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 -- 172.21.15.115:0/4157431861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 msgr2=0x5594ae7c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:25.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 --2- 172.21.15.115:0/4157431861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c44f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f189c00ea00 tx=0x7f189c011000 comp rx=0 tx=0).stop 2024-05-09T05:39:25.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 -- 172.21.15.115:0/4157431861 shutdown_connections 2024-05-09T05:39:25.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 --2- 172.21.15.115:0/4157431861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7bead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 --2- 172.21.15.115:0/4157431861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c44f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 --2- 172.21.15.115:0/4157431861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 0x5594ae7c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 -- 172.21.15.115:0/4157431861 >> 172.21.15.115:0/4157431861 conn(0x5594ae6a16a0 msgr2=0x5594ae7b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 -- 172.21.15.115:0/4157431861 shutdown_connections 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 -- 172.21.15.115:0/4157431861 wait complete. 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 -- start start 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.171+0000 7f18c0325b80 1 Processor -- start 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae6414a0 0x5594ae7c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 0x5594ae7c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae7be6a0 0x5594ae7c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18c0325b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5594ae55cba0 con 0x5594ae7be6a0 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18c0325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5594ae69b150 con 0x5594ae72b160 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18c0325b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5594ae7cb1d0 con 0x5594ae6414a0 2024-05-09T05:39:25.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18c0325b80 10 client.0 _ll_register_callbacks cb 0x5594ae5526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18ab7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae7be6a0 0x5594ae7c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18ab7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae7be6a0 0x5594ae7c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47166/0 (socket says 172.21.15.115:47166) 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18ab7fe700 1 -- 172.21.15.115:0/3506738984 learned_addr learned my addr 172.21.15.115:0/3506738984 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18aaffd700 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae6414a0 0x5594ae7c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 0x5594ae7c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18ab7fe700 1 -- 172.21.15.115:0/3506738984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae6414a0 msgr2=0x5594ae7c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18ab7fe700 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae6414a0 0x5594ae7c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18ab7fe700 1 -- 172.21.15.115:0/3506738984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 msgr2=0x5594ae7c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18ab7fe700 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 0x5594ae7c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18ab7fe700 1 -- 172.21.15.115:0/3506738984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ae55c490 con 0x5594ae7be6a0 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f18ab7fe700 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae7be6a0 0x5594ae7c8cb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f18a40040f0 tx=0x7f18a4004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79bd01cae8c7d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.172+0000 7f188affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f188affd700 1 -- 172.21.15.115:0/3506738984 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a401a020 con 0x5594ae7be6a0 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f188affd700 1 -- 172.21.15.115:0/3506738984 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f18a40079d0 con 0x5594ae7be6a0 2024-05-09T05:39:25.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f18c0325b80 1 -- 172.21.15.115:0/3506738984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f189c024070 con 0x5594ae7be6a0 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f188affd700 1 -- 172.21.15.115:0/3506738984 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18a4025050 con 0x5594ae7be6a0 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f18c0325b80 10 client.15822 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f18c0325b80 1 -- 172.21.15.115:0/3506738984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5594ae7d5be0 con 0x5594ae7be6a0 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f188affd700 1 -- 172.21.15.115:0/3506738984 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f189c024070 con 0x5594ae7be6a0 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f188affd700 1 -- 172.21.15.115:0/3506738984 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f18a4016cd0 con 0x5594ae7be6a0 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f18c0325b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f18c0325b80 10 client.15822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f18c0325b80 20 client.15822 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5594ae7d82c0 0x5594ae7da6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f18c0325b80 4 client.15822 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.173+0000 7f18c0325b80 1 -- 172.21.15.115:0/3506738984 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594ae6ee9d0 con 0x5594ae7d82c0 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.174+0000 7f18aaffd700 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5594ae7d82c0 0x5594ae7da6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.174+0000 7f188affd700 10 client.15822 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.174+0000 7f18aaffd700 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5594ae7d82c0 0x5594ae7da6e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1333d775143d7c2b server_cookie=f25e9680765f0d64 in_seq=0 out_seq=0 2024-05-09T05:39:25.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f188affd700 1 -- 172.21.15.115:0/3506738984 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594ae6ee9d0 con 0x5594ae7d82c0 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f188affd700 10 client.15822 handle_command_reply: tid=0 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 client.15822 shutdown 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 -- 172.21.15.115:0/3506738984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae7be6a0 msgr2=0x5594ae7c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae7be6a0 0x5594ae7c8cb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f18a40040f0 tx=0x7f18a4004400 comp rx=0 tx=0).stop 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 -- 172.21.15.115:0/3506738984 shutdown_connections 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5594ae7d82c0 0x5594ae7da6e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae7be6a0 0x5594ae7c8cb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae6414a0 0x5594ae7c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3506738984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 0x5594ae7c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 -- 172.21.15.115:0/3506738984 >> 172.21.15.115:0/3506738984 conn(0x5594ae6a16a0 msgr2=0x5594ae7b91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f188affd700 0 client.15822 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 -- 172.21.15.115:0/3506738984 shutdown_connections 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 1 -- 172.21.15.115:0/3506738984 wait complete. 2024-05-09T05:39:25.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.175+0000 7f18c0325b80 20 client.15822 trim_cache size 0 max 16384 2024-05-09T05:39:25.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18c0325b80 1 Processor -- start 2024-05-09T05:39:25.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18c0325b80 1 -- start start 2024-05-09T05:39:25.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 0x5594ae7c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7bd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18c0325b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1b24e0 con 0x5594ae72b160 2024-05-09T05:39:25.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18c0325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1b4020 con 0x5594ae6414a0 2024-05-09T05:39:25.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18c0325b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1b2bb0 con 0x5594ae7be6a0 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18aa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18aa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47170/0 (socket says 172.21.15.115:47170) 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.190+0000 7f18aa7fc700 1 -- 172.21.15.115:0/3896310351 learned_addr learned my addr 172.21.15.115:0/3896310351 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18aaffd700 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 0x5594ae7c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18ab7fe700 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7bd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18aa7fc700 1 -- 172.21.15.115:0/3896310351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 msgr2=0x5594ae7bd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7bd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18aa7fc700 1 -- 172.21.15.115:0/3896310351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 msgr2=0x5594ae7c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 0x5594ae7c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18aa7fc700 1 -- 172.21.15.115:0/3896310351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ae7c7550 con 0x5594ae72b160 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18aaffd700 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 0x5594ae7c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c8af0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f18a00a7a80 tx=0x7f18a00b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acb6da1f7e84606 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18917fa700 1 -- 172.21.15.115:0/3896310351 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00cd0b0 con 0x5594ae72b160 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18917fa700 1 -- 172.21.15.115:0/3896310351 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a00aaf10 con 0x5594ae72b160 2024-05-09T05:39:25.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.191+0000 7f18917fa700 1 -- 172.21.15.115:0/3896310351 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00aacd0 con 0x5594ae72b160 2024-05-09T05:39:25.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 -- 172.21.15.115:0/3896310351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 msgr2=0x5594ae7c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c8af0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f18a00a7a80 tx=0x7f18a00b1040 comp rx=0 tx=0).stop 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 -- 172.21.15.115:0/3896310351 shutdown_connections 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae6414a0 0x5594ae7c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7bd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3896310351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae72b160 0x5594ae7c8af0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 -- 172.21.15.115:0/3896310351 >> 172.21.15.115:0/3896310351 conn(0x5594ae6a16a0 msgr2=0x5594ae7c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 -- 172.21.15.115:0/3896310351 shutdown_connections 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 -- 172.21.15.115:0/3896310351 wait complete. 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 -- start start 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.192+0000 7f18c0325b80 1 Processor -- start 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae6414a0 0x5594ae7d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 0x5594ae7d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18c0325b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7deeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18c0325b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1b3950 con 0x5594ae6414a0 2024-05-09T05:39:25.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18c0325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1b3280 con 0x5594ae72b160 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18c0325b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1b0480 con 0x5594ae7be6a0 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae6414a0 0x5594ae7d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18c0325b80 10 client.0 _ll_register_callbacks cb 0x5594ae5526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae6414a0 0x5594ae7d0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47174/0 (socket says 172.21.15.115:47174) 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aaffd700 1 -- 172.21.15.115:0/3972760462 learned_addr learned my addr 172.21.15.115:0/3972760462 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 0x5594ae7d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18ab7fe700 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7deeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aaffd700 1 -- 172.21.15.115:0/3972760462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 msgr2=0x5594ae7deeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aaffd700 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7deeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aaffd700 1 -- 172.21.15.115:0/3972760462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 msgr2=0x5594ae7d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aaffd700 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 0x5594ae7d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aaffd700 1 -- 172.21.15.115:0/3972760462 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ae7e0850 con 0x5594ae6414a0 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f18aaffd700 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae6414a0 0x5594ae7d0260 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f189c015e60 tx=0x7f189c009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50efc12b952872b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:25.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.193+0000 7f188affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f188affd700 1 -- 172.21.15.115:0/3972760462 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0159a0 con 0x5594ae6414a0 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f188affd700 1 -- 172.21.15.115:0/3972760462 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f189c015b40 con 0x5594ae6414a0 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f18c0325b80 1 -- 172.21.15.115:0/3972760462 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a00aaf10 con 0x5594ae6414a0 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f188affd700 1 -- 172.21.15.115:0/3972760462 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f189c0343c0 con 0x5594ae6414a0 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f18c0325b80 10 client.15828 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f18c0325b80 1 -- 172.21.15.115:0/3972760462 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5594ae841f50 con 0x5594ae6414a0 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f188affd700 1 -- 172.21.15.115:0/3972760462 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a00aaf10 con 0x5594ae6414a0 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f188affd700 1 -- 172.21.15.115:0/3972760462 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f189c01e030 con 0x5594ae6414a0 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f18c0325b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f18c0325b80 10 client.15828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f18c0325b80 20 client.15828 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5594ae842320 0x5594ae844740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f18c0325b80 4 client.15828 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.194+0000 7f18c0325b80 1 -- 172.21.15.115:0/3972760462 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594ae6e8a70 con 0x5594ae842320 2024-05-09T05:39:25.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5594ae842320 0x5594ae844740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18aa7fc700 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5594ae842320 0x5594ae844740 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce8dc3a6c01c25fd server_cookie=d03c5afa9403e1ff in_seq=0 out_seq=0 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f188affd700 10 client.15828 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f188affd700 1 -- 172.21.15.115:0/3972760462 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594ae6e8a70 con 0x5594ae842320 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f188affd700 10 client.15828 handle_command_reply: tid=0 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18c0325b80 1 client.15828 shutdown 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18c0325b80 1 -- 172.21.15.115:0/3972760462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae6414a0 msgr2=0x5594ae7d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae6414a0 0x5594ae7d0260 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f189c015e60 tx=0x7f189c009660 comp rx=0 tx=0).stop 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18c0325b80 1 -- 172.21.15.115:0/3972760462 shutdown_connections 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5594ae842320 0x5594ae844740 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594ae7be6a0 0x5594ae7deeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594ae6414a0 0x5594ae7d0260 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.195+0000 7f18c0325b80 1 --2- 172.21.15.115:0/3972760462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594ae72b160 0x5594ae7d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f188affd700 0 client.15828 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18c0325b80 1 -- 172.21.15.115:0/3972760462 >> 172.21.15.115:0/3972760462 conn(0x5594ae6a16a0 msgr2=0x5594ae7c1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:25.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18c0325b80 1 -- 172.21.15.115:0/3972760462 shutdown_connections 2024-05-09T05:39:25.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18c0325b80 1 -- 172.21.15.115:0/3972760462 wait complete. 2024-05-09T05:39:25.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18c0325b80 20 client.15828 trim_cache size 0 max 16384 2024-05-09T05:39:25.194 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 -- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f18940b2b10 msgr2=0x7f18940b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f18940b2b10 0x7f18940b5020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f189c009a10 tx=0x7f189c009190 comp rx=0 tx=0).stop 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 -- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 msgr2=0x7f18ac19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 0x7f18ac19d590 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b580 tx=0x7f18a0004790 comp rx=0 tx=0).stop 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 -- 172.21.15.115:0/1916160328 shutdown_connections 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f18940b2b10 0x7f18940b5020 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18ac108740 0x7f18ac1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18ac1034c0 0x7f18ac19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 --2- 172.21.15.115:0/1916160328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18ac107df0 0x7f18ac19d590 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:25.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 -- 172.21.15.115:0/1916160328 >> 172.21.15.115:0/1916160328 conn(0x7f18ac0fedc0 msgr2=0x7f18ac06b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:25.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.196+0000 7f18b190f700 1 -- 172.21.15.115:0/1916160328 shutdown_connections 2024-05-09T05:39:25.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:25.197+0000 7f18b190f700 1 -- 172.21.15.115:0/1916160328 wait complete. 2024-05-09T05:39:25.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:25 smithi195 ceph-mon[33844]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T05:39:25.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:25 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:39:25.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:25 smithi115 ceph-mon[30349]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T05:39:25.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:25 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:39:25.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:25 smithi161 ceph-mon[33805]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T05:39:25.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:25 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:39:26.016 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:39:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:26.708 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:26 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:39:26.708 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:26 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:39:26.708 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:26 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:39:26.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:26 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:39:26.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:26 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:39:26.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:26 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:39:26.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:26 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:39:26.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:26 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:39:26.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:26 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:39:27.296 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-09T05:39:27.296 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.95 2024-05-09T05:39:27.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:39:27.761 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:27 smithi115 ceph-mon[30349]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:27.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:27 smithi195 ceph-mon[33844]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:27.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:27 smithi161 ceph-mon[33805]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:27.913 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:28.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.174+0000 7f0ebeb65700 1 -- 172.21.15.115:0/142391811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb8107df0 msgr2=0x7f0eb8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:28.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.174+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/142391811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb8107df0 0x7f0eb8108200 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40099c0 tx=0x7f0ea4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.174+0000 7f0ebeb65700 1 -- 172.21.15.115:0/142391811 shutdown_connections 2024-05-09T05:39:28.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.174+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/142391811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb81034c0 0x7f0eb81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.174+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/142391811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb8108740 0x7f0eb81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.174+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/142391811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb8107df0 0x7f0eb8108200 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.174+0000 7f0ebeb65700 1 -- 172.21.15.115:0/142391811 >> 172.21.15.115:0/142391811 conn(0x7f0eb80fedf0 msgr2=0x7f0eb8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:28.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.174+0000 7f0ebeb65700 1 -- 172.21.15.115:0/142391811 shutdown_connections 2024-05-09T05:39:28.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.174+0000 7f0ebeb65700 1 -- 172.21.15.115:0/142391811 wait complete. 2024-05-09T05:39:28.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.175+0000 7f0ebeb65700 1 Processor -- start 2024-05-09T05:39:28.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.175+0000 7f0ebeb65700 1 -- start start 2024-05-09T05:39:28.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.175+0000 7f0ebeb65700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb81034c0 0x7f0eb8199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.175+0000 7f0ebeb65700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb8107df0 0x7f0eb8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.175+0000 7f0ebeb65700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb8108740 0x7f0eb819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.175+0000 7f0ebeb65700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8113270 con 0x7f0eb81034c0 2024-05-09T05:39:28.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.175+0000 7f0ebeb65700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81130f0 con 0x7f0eb8108740 2024-05-09T05:39:28.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.175+0000 7f0ebeb65700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81133f0 con 0x7f0eb8107df0 2024-05-09T05:39:28.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0ebc901700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb81034c0 0x7f0eb8199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0eb7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb8107df0 0x7f0eb8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0eb7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb8107df0 0x7f0eb8199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:54884/0 (socket says 172.21.15.115:54884) 2024-05-09T05:39:28.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0ebc901700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb81034c0 0x7f0eb8199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47194/0 (socket says 172.21.15.115:47194) 2024-05-09T05:39:28.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0eb7fff700 1 -- 172.21.15.115:0/1015725310 learned_addr learned my addr 172.21.15.115:0/1015725310 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:28.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0ebd102700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb8108740 0x7f0eb819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0ebc901700 1 -- 172.21.15.115:0/1015725310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb8107df0 msgr2=0x7f0eb8199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:28.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0ebc901700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb8107df0 0x7f0eb8199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0ebc901700 1 -- 172.21.15.115:0/1015725310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb8108740 msgr2=0x7f0eb819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:28.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0ebc901700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb8108740 0x7f0eb819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0ebc901700 1 -- 172.21.15.115:0/1015725310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb81a9080 con 0x7f0eb81034c0 2024-05-09T05:39:28.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0ebc901700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb81034c0 0x7f0eb8199a00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c9a0 tx=0x7f0eac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:28.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.176+0000 7f0eb5ffb700 1 -- 172.21.15.115:0/1015725310 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac014070 con 0x7f0eb81034c0 2024-05-09T05:39:28.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.177+0000 7f0ebeb65700 1 -- 172.21.15.115:0/1015725310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb81a9370 con 0x7f0eb81034c0 2024-05-09T05:39:28.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.177+0000 7f0eb5ffb700 1 -- 172.21.15.115:0/1015725310 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0eac004440 con 0x7f0eb81034c0 2024-05-09T05:39:28.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.177+0000 7f0eb5ffb700 1 -- 172.21.15.115:0/1015725310 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00e970 con 0x7f0eb81034c0 2024-05-09T05:39:28.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.177+0000 7f0ebeb65700 1 -- 172.21.15.115:0/1015725310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a9900 con 0x7f0eb81034c0 2024-05-09T05:39:28.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.180+0000 7f0eb5ffb700 1 -- 172.21.15.115:0/1015725310 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f0eac017070 con 0x7f0eb81034c0 2024-05-09T05:39:28.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.181+0000 7f0eb5ffb700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0ea80b2b10 0x7f0ea80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.181+0000 7f0eb7fff700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0ea80b2b10 0x7f0ea80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.181+0000 7f0eb5ffb700 1 -- 172.21.15.115:0/1015725310 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0eac0e24a0 con 0x7f0eb81034c0 2024-05-09T05:39:28.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.181+0000 7f0eb7fff700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0ea80b2b10 0x7f0ea80b5020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0eb81069a0 tx=0x7f0ea4005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:28.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ecd57bb80 1 Processor -- start 2024-05-09T05:39:28.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ecd57bb80 1 -- start start 2024-05-09T05:39:28.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b0506160 0x55b5b059f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b041c4a0 0x55b5b059fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b05996a0 0x55b5b0599ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b5b04bbff0 con 0x55b5b0506160 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b5b045a7f0 con 0x55b5b05996a0 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b5b04b7fb0 con 0x55b5b041c4a0 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ebc901700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b0506160 0x55b5b059f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ebc901700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b0506160 0x55b5b059f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47202/0 (socket says 172.21.15.115:47202) 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ebc901700 1 -- 172.21.15.115:0/502785089 learned_addr learned my addr 172.21.15.115:0/502785089 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0eb7fff700 1 --2- 172.21.15.115:0/502785089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b041c4a0 0x55b5b059fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ebc901700 1 -- 172.21.15.115:0/502785089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b041c4a0 msgr2=0x55b5b059fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:28.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ebc901700 1 --2- 172.21.15.115:0/502785089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b041c4a0 0x55b5b059fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ebc901700 1 -- 172.21.15.115:0/502785089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b05996a0 msgr2=0x55b5b0599ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ebc901700 1 --2- 172.21.15.115:0/502785089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b05996a0 0x55b5b0599ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0ebc901700 1 -- 172.21.15.115:0/502785089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b0452370 con 0x55b5b0506160 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.182+0000 7f0eb7fff700 1 --2- 172.21.15.115:0/502785089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b041c4a0 0x55b5b059fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.183+0000 7f0ebc901700 1 --2- 172.21.15.115:0/502785089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b0506160 0x55b5b059f4f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000c00 tx=0x7f0eac0a8d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a07cf63e5647809 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.183+0000 7f0ea2ffd700 1 -- 172.21.15.115:0/502785089 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0ad5e0 con 0x55b5b0506160 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.183+0000 7f0ea2ffd700 1 -- 172.21.15.115:0/502785089 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0eac0b2090 con 0x55b5b0506160 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.183+0000 7f0ea2ffd700 1 -- 172.21.15.115:0/502785089 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0a96f0 con 0x55b5b0506160 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/502785089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b0506160 msgr2=0x55b5b059f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/502785089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b0506160 0x55b5b059f4f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000c00 tx=0x7f0eac0a8d40 comp rx=0 tx=0).stop 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/502785089 shutdown_connections 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/502785089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b05996a0 0x55b5b0599ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/502785089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b0506160 0x55b5b059f4f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/502785089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b041c4a0 0x55b5b059fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/502785089 >> 172.21.15.115:0/502785089 conn(0x55b5b047c6a0 msgr2=0x55b5b0594400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/502785089 shutdown_connections 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/502785089 wait complete. 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 -- start start 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.184+0000 7f0ecd57bb80 1 Processor -- start 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b05a3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b0506160 0x55b5b05a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b05996a0 0x55b5b05a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b5b0337ba0 con 0x55b5b041c4a0 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b5b0476150 con 0x55b5b0506160 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b5b05a61d0 con 0x55b5b05996a0 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ebc901700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b05a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ebc901700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b05a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47214/0 (socket says 172.21.15.115:47214) 2024-05-09T05:39:28.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ebc901700 1 -- 172.21.15.115:0/3463140015 learned_addr learned my addr 172.21.15.115:0/3463140015 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ecd57bb80 10 client.0 _ll_register_callbacks cb 0x55b5b032d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0eb7fff700 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b0506160 0x55b5b05a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ebd102700 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b05996a0 0x55b5b05a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ebc901700 1 -- 172.21.15.115:0/3463140015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b05996a0 msgr2=0x55b5b05a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ebc901700 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b05996a0 0x55b5b05a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ebc901700 1 -- 172.21.15.115:0/3463140015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b0506160 msgr2=0x55b5b05a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ebc901700 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b0506160 0x55b5b05a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.185+0000 7f0ebc901700 1 -- 172.21.15.115:0/3463140015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b0337490 con 0x55b5b041c4a0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ebc901700 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b05a3280 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0a93f0 tx=0x7f0eac0ad420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95e457ef2d372c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0e97fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0e97fff700 1 -- 172.21.15.115:0/3463140015 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac06d060 con 0x55b5b041c4a0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0e97fff700 1 -- 172.21.15.115:0/3463140015 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0eac0a5820 con 0x55b5b041c4a0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/3463140015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eac0b2090 con 0x55b5b041c4a0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0e97fff700 1 -- 172.21.15.115:0/3463140015 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0eac0c2030 con 0x55b5b041c4a0 2024-05-09T05:39:28.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ecd57bb80 10 client.15840 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/3463140015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b5b05b0c20 con 0x55b5b041c4a0 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0e97fff700 1 -- 172.21.15.115:0/3463140015 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0eac0b2090 con 0x55b5b041c4a0 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0e97fff700 1 -- 172.21.15.115:0/3463140015 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0eac0b1c00 con 0x55b5b041c4a0 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ecd57bb80 10 client.15840 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ecd57bb80 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ecd57bb80 20 client.15840 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b5b05b30f0 0x55b5b05b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ecd57bb80 4 client.15840 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.186+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/3463140015 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5b04c99d0 con 0x55b5b05b30f0 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.187+0000 7f0eb7fff700 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b5b05b30f0 0x55b5b05b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.187+0000 7f0e97fff700 10 client.15840 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.187+0000 7f0eb7fff700 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b5b05b30f0 0x55b5b05b5510 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3aa9e5a1e5dd16 server_cookie=81d094e8c2805568 in_seq=0 out_seq=0 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0e97fff700 1 -- 172.21.15.115:0/3463140015 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5b04c99d0 con 0x55b5b05b30f0 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0e97fff700 10 client.15840 handle_command_reply: tid=0 2024-05-09T05:39:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 client.15840 shutdown 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/3463140015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 msgr2=0x55b5b05a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b05a3280 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0a93f0 tx=0x7f0eac0ad420 comp rx=0 tx=0).stop 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/3463140015 shutdown_connections 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b5b05b30f0 0x55b5b05b5510 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b05996a0 0x55b5b05a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b05a3280 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/3463140015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b0506160 0x55b5b05a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0e97fff700 0 client.15840 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/3463140015 >> 172.21.15.115:0/3463140015 conn(0x55b5b047c6a0 msgr2=0x55b5b05941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/3463140015 shutdown_connections 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.188+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/3463140015 wait complete. 2024-05-09T05:39:28.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.189+0000 7f0ecd57bb80 20 client.15840 trim_cache size 0 max 16384 2024-05-09T05:39:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.203+0000 7f0ecd57bb80 1 Processor -- start 2024-05-09T05:39:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.203+0000 7f0ecd57bb80 1 -- start start 2024-05-09T05:39:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.203+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b059d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.203+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 0x55b5b05af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b05996a0 0x55b5b05a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81b2680 con 0x55b5b041c4a0 2024-05-09T05:39:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81b41c0 con 0x55b5b05996a0 2024-05-09T05:39:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81b2d50 con 0x55b5b0506160 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebc901700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b059d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebc901700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b059d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47228/0 (socket says 172.21.15.115:47228) 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebc901700 1 -- 172.21.15.115:0/1708848574 learned_addr learned my addr 172.21.15.115:0/1708848574 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0eb7fff700 1 --2- 172.21.15.115:0/1708848574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 0x55b5b05af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebd102700 1 --2- 172.21.15.115:0/1708848574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b05996a0 0x55b5b05a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebc901700 1 -- 172.21.15.115:0/1708848574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 msgr2=0x55b5b05af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebc901700 1 --2- 172.21.15.115:0/1708848574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 0x55b5b05af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebc901700 1 -- 172.21.15.115:0/1708848574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b05996a0 msgr2=0x55b5b05a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebc901700 1 --2- 172.21.15.115:0/1708848574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b05996a0 0x55b5b05a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebc901700 1 -- 172.21.15.115:0/1708848574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b0595110 con 0x55b5b041c4a0 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ebc901700 1 --2- 172.21.15.115:0/1708848574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b059d860 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c970 tx=0x7f0eac0a8d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6081ee9062d9662 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ea27fc700 1 -- 172.21.15.115:0/1708848574 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0a96a0 con 0x55b5b041c4a0 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ea27fc700 1 -- 172.21.15.115:0/1708848574 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0eac0b2090 con 0x55b5b041c4a0 2024-05-09T05:39:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.204+0000 7f0ea27fc700 1 -- 172.21.15.115:0/1708848574 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0a4e70 con 0x55b5b041c4a0 2024-05-09T05:39:28.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.205+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/1708848574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 msgr2=0x55b5b059d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:28.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.205+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/1708848574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b059d860 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c970 tx=0x7f0eac0a8d40 comp rx=0 tx=0).stop 2024-05-09T05:39:28.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.205+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/1708848574 shutdown_connections 2024-05-09T05:39:28.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.205+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/1708848574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b05996a0 0x55b5b05a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.205+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/1708848574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b041c4a0 0x55b5b059d860 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.205+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/1708848574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 0x55b5b05af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.205+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/1708848574 >> 172.21.15.115:0/1708848574 conn(0x55b5b047c6a0 msgr2=0x55b5b059e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.205+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/1708848574 shutdown_connections 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.205+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/1708848574 wait complete. 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ecd57bb80 1 -- start start 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ecd57bb80 1 Processor -- start 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b041c4a0 0x55b5b05b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 0x55b5b05b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ecd57bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b05996a0 0x55b5b061b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81b3af0 con 0x55b5b05996a0 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81b3420 con 0x55b5b041c4a0 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ecd57bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81b0620 con 0x55b5b0506160 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ecd57bb80 10 client.0 _ll_register_callbacks cb 0x55b5b032d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ebd102700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b05996a0 0x55b5b061b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ebd102700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b05996a0 0x55b5b061b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47244/0 (socket says 172.21.15.115:47244) 2024-05-09T05:39:28.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0ebd102700 1 -- 172.21.15.115:0/2304200820 learned_addr learned my addr 172.21.15.115:0/2304200820 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.206+0000 7f0eb7fff700 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 0x55b5b05b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ebc901700 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b041c4a0 0x55b5b05b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ebd102700 1 -- 172.21.15.115:0/2304200820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 msgr2=0x55b5b05b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ebd102700 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 0x55b5b05b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ebd102700 1 -- 172.21.15.115:0/2304200820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b041c4a0 msgr2=0x55b5b05b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ebd102700 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b041c4a0 0x55b5b05b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ebd102700 1 -- 172.21.15.115:0/2304200820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b058df60 con 0x55b5b05996a0 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ebc901700 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b041c4a0 0x55b5b05b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ebd102700 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b05996a0 0x55b5b061b620 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000aaf0 tx=0x7f0eb000ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261724fada9d8a47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0e97fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0e97fff700 1 -- 172.21.15.115:0/2304200820 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb001b020 con 0x55b5b05996a0 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0e97fff700 1 -- 172.21.15.115:0/2304200820 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0eb00078c0 con 0x55b5b05996a0 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/2304200820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eac0b2090 con 0x55b5b05996a0 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0e97fff700 1 -- 172.21.15.115:0/2304200820 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0eb0026050 con 0x55b5b05996a0 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ecd57bb80 10 client.15846 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/2304200820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b5b061cc00 con 0x55b5b05996a0 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.207+0000 7f0e97fff700 1 -- 172.21.15.115:0/2304200820 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0eac0b2090 con 0x55b5b05996a0 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.208+0000 7f0e97fff700 1 -- 172.21.15.115:0/2304200820 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0eb0017c90 con 0x55b5b05996a0 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.208+0000 7f0ecd57bb80 10 client.15846 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.208+0000 7f0ecd57bb80 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.208+0000 7f0ecd57bb80 20 client.15846 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.208+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b5b061d360 0x55b5b061f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.208+0000 7f0ecd57bb80 4 client.15846 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.208+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/2304200820 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5b04c3a70 con 0x55b5b061d360 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.208+0000 7f0ebc901700 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b5b061d360 0x55b5b061f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.208+0000 7f0e97fff700 10 client.15846 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ebc901700 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b5b061d360 0x55b5b061f780 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df85578cd85beb86 server_cookie=4a7833e909531fe in_seq=0 out_seq=0 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0e97fff700 1 -- 172.21.15.115:0/2304200820 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5b04c3a70 con 0x55b5b061d360 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0e97fff700 10 client.15846 handle_command_reply: tid=0 2024-05-09T05:39:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 client.15846 shutdown 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/2304200820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b05996a0 msgr2=0x55b5b061b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b05996a0 0x55b5b061b620 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000aaf0 tx=0x7f0eb000ab20 comp rx=0 tx=0).stop 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/2304200820 shutdown_connections 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5b041c4a0 0x55b5b05b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b5b061d360 0x55b5b061f780 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b5b05996a0 0x55b5b061b620 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 --2- 172.21.15.115:0/2304200820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b5b0506160 0x55b5b05b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/2304200820 >> 172.21.15.115:0/2304200820 conn(0x55b5b047c6a0 msgr2=0x55b5b059ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0e97fff700 0 client.15846 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/2304200820 shutdown_connections 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 1 -- 172.21.15.115:0/2304200820 wait complete. 2024-05-09T05:39:28.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.209+0000 7f0ecd57bb80 20 client.15846 trim_cache size 0 max 16384 2024-05-09T05:39:28.208 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:28.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 -- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0ea80b2b10 msgr2=0x7f0ea80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:28.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0ea80b2b10 0x7f0ea80b5020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0eb81069a0 tx=0x7f0ea4005fb0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 -- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb81034c0 msgr2=0x7f0eb8199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:28.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb81034c0 0x7f0eb8199a00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c9a0 tx=0x7f0eac00ccb0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 -- 172.21.15.115:0/1015725310 shutdown_connections 2024-05-09T05:39:28.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0ea80b2b10 0x7f0ea80b5020 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb8108740 0x7f0eb819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0eb81034c0 0x7f0eb8199a00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 --2- 172.21.15.115:0/1015725310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eb8107df0 0x7f0eb8199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:28.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 -- 172.21.15.115:0/1015725310 >> 172.21.15.115:0/1015725310 conn(0x7f0eb80fedf0 msgr2=0x7f0eb8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:28.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 -- 172.21.15.115:0/1015725310 shutdown_connections 2024-05-09T05:39:28.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:28.210+0000 7f0ebeb65700 1 -- 172.21.15.115:0/1015725310 wait complete. 2024-05-09T05:39:29.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:29 smithi115 ceph-mon[30349]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T05:39:29.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:29 smithi195 ceph-mon[33844]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T05:39:29.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:29 smithi161 ceph-mon[33805]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-09T05:39:30.762 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:39:30.762 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:39:30.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.784+0000 7fc36fd48700 1 -- 172.21.15.115:0/4003109443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681034c0 msgr2=0x7fc3681038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:30.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.784+0000 7fc36fd48700 1 --2- 172.21.15.115:0/4003109443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681034c0 0x7fc3681038d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc3640099c0 tx=0x7fc364009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:30.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.784+0000 7fc36fd48700 1 -- 172.21.15.115:0/4003109443 shutdown_connections 2024-05-09T05:39:30.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.784+0000 7fc36fd48700 1 --2- 172.21.15.115:0/4003109443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc368108740 0x7fc3681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:30.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.784+0000 7fc36fd48700 1 --2- 172.21.15.115:0/4003109443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3681034c0 0x7fc3681038d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:30.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.784+0000 7fc36fd48700 1 --2- 172.21.15.115:0/4003109443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc368107df0 0x7fc368108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:30.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.784+0000 7fc36fd48700 1 -- 172.21.15.115:0/4003109443 >> 172.21.15.115:0/4003109443 conn(0x7fc3680fedc0 msgr2=0x7fc3681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:30.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.784+0000 7fc36fd48700 1 -- 172.21.15.115:0/4003109443 shutdown_connections 2024-05-09T05:39:30.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.784+0000 7fc36fd48700 1 -- 172.21.15.115:0/4003109443 wait complete. 2024-05-09T05:39:30.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36fd48700 1 Processor -- start 2024-05-09T05:39:30.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36fd48700 1 -- start start 2024-05-09T05:39:30.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36fd48700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3681034c0 0x7fc36819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36fd48700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368107df0 0x7fc36819c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36fd48700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc368108740 0x7fc36819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36fd48700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc368113220 con 0x7fc368107df0 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36fd48700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3681130a0 con 0x7fc3681034c0 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36fd48700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3681133a0 con 0x7fc368108740 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36d2e3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368107df0 0x7fc36819c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36d2e3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368107df0 0x7fc36819c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47268/0 (socket says 172.21.15.115:47268) 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.785+0000 7fc36d2e3700 1 -- 172.21.15.115:0/2562332464 learned_addr learned my addr 172.21.15.115:0/2562332464 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36e2e5700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc368108740 0x7fc36819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36dae4700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3681034c0 0x7fc36819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36d2e3700 1 -- 172.21.15.115:0/2562332464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc368108740 msgr2=0x7fc36819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36d2e3700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc368108740 0x7fc36819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36d2e3700 1 -- 172.21.15.115:0/2562332464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3681034c0 msgr2=0x7fc36819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:30.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36d2e3700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3681034c0 0x7fc36819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:30.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36d2e3700 1 -- 172.21.15.115:0/2562332464 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3681a6f00 con 0x7fc368107df0 2024-05-09T05:39:30.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36d2e3700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368107df0 0x7fc36819c120 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc35800d980 tx=0x7fc35800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:30.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc35effd700 1 -- 172.21.15.115:0/2562332464 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358017070 con 0x7fc368107df0 2024-05-09T05:39:30.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3681a71f0 con 0x7fc368107df0 2024-05-09T05:39:30.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc35effd700 1 -- 172.21.15.115:0/2562332464 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc358009dc0 con 0x7fc368107df0 2024-05-09T05:39:30.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc35effd700 1 -- 172.21.15.115:0/2562332464 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35800f3c0 con 0x7fc368107df0 2024-05-09T05:39:30.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.786+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3681a7730 con 0x7fc368107df0 2024-05-09T05:39:30.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.787+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3680512c0 con 0x7fc368107df0 2024-05-09T05:39:30.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.789+0000 7fc35effd700 1 -- 172.21.15.115:0/2562332464 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7fc358009930 con 0x7fc368107df0 2024-05-09T05:39:30.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.790+0000 7fc35effd700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3540b2b10 0x7fc3540b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:30.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.790+0000 7fc36dae4700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3540b2b10 0x7fc3540b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:30.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.790+0000 7fc35effd700 1 -- 172.21.15.115:0/2562332464 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc3580e2740 con 0x7fc368107df0 2024-05-09T05:39:30.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.790+0000 7fc36dae4700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3540b2b10 0x7fc3540b5020 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc364000c00 tx=0x7fc364005940 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:30.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:30.795+0000 7fc35effd700 1 -- 172.21.15.115:0/2562332464 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc3580a9b90 con 0x7fc368107df0 2024-05-09T05:39:31.026 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:31.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.074+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc36802e5f0 con 0x7fc368107df0 2024-05-09T05:39:31.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.075+0000 7fc35effd700 1 -- 172.21.15.115:0/2562332464 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7fc3580a9490 con 0x7fc368107df0 2024-05-09T05:39:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:39:31.068 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.077+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3540b2b10 msgr2=0x7fc3540b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.077+0000 7fc36fd48700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3540b2b10 0x7fc3540b5020 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc364000c00 tx=0x7fc364005940 comp rx=0 tx=0).stop 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.077+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368107df0 msgr2=0x7fc36819c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.077+0000 7fc36fd48700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368107df0 0x7fc36819c120 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc35800d980 tx=0x7fc35800dc90 comp rx=0 tx=0).stop 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.078+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 shutdown_connections 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.078+0000 7fc36fd48700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3540b2b10 0x7fc3540b5020 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.078+0000 7fc36fd48700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3681034c0 0x7fc36819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.078+0000 7fc36fd48700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc368108740 0x7fc36819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.078+0000 7fc36fd48700 1 --2- 172.21.15.115:0/2562332464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368107df0 0x7fc36819c120 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.078+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 >> 172.21.15.115:0/2562332464 conn(0x7fc3680fedc0 msgr2=0x7fc3681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:31.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.078+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 shutdown_connections 2024-05-09T05:39:31.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:31.078+0000 7fc36fd48700 1 -- 172.21.15.115:0/2562332464 wait complete. 2024-05-09T05:39:31.072 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:39:31.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:31 smithi115 ceph-mon[30349]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:31.752 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:31 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2562332464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:31.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:31 smithi195 ceph-mon[33844]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:31.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:31 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2562332464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:31.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:31 smithi161 ceph-mon[33805]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:31.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:31 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2562332464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.368+0000 7f9d9e842640 1 -- 172.21.15.115:0/4145815547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d981032f0 msgr2=0x7f9d9810f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.368+0000 7f9d9e842640 1 --2- 172.21.15.115:0/4145815547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d981032f0 0x7f9d9810f6c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009ca0 tx=0x7f9d9002a8c0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.368+0000 7f9d9e842640 1 -- 172.21.15.115:0/4145815547 shutdown_connections 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.368+0000 7f9d9e842640 1 --2- 172.21.15.115:0/4145815547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d981032f0 0x7f9d9810f6c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.368+0000 7f9d9e842640 1 --2- 172.21.15.115:0/4145815547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d98109040 0x7f9d98102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.368+0000 7f9d9e842640 1 --2- 172.21.15.115:0/4145815547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d98108ba0 0x7f9d98102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.368+0000 7f9d9e842640 1 -- 172.21.15.115:0/4145815547 >> 172.21.15.115:0/4145815547 conn(0x7f9d980782e0 msgr2=0x7f9d98078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.368+0000 7f9d9e842640 1 -- 172.21.15.115:0/4145815547 shutdown_connections 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.368+0000 7f9d9e842640 1 -- 172.21.15.115:0/4145815547 wait complete. 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9e842640 1 Processor -- start 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9e842640 1 -- start start 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9e842640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d981032f0 0x7f9d9819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9e842640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d98108ba0 0x7f9d9819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9e842640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d98109040 0x7f9d9819c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9e842640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98112cb0 con 0x7f9d98109040 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9e842640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98112b30 con 0x7f9d98108ba0 2024-05-09T05:39:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9e842640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98112e30 con 0x7f9d981032f0 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9cddf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d98109040 0x7f9d9819c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9cddf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d98109040 0x7f9d9819c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47292/0 (socket says 172.21.15.115:47292) 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9cddf640 1 -- 172.21.15.115:0/60612407 learned_addr learned my addr 172.21.15.115:0/60612407 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d97fff640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d981032f0 0x7f9d9819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d98108ba0 0x7f9d9819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9cddf640 1 -- 172.21.15.115:0/60612407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d981032f0 msgr2=0x7f9d9819b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9cddf640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d981032f0 0x7f9d9819b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.369+0000 7f9d9cddf640 1 -- 172.21.15.115:0/60612407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d98108ba0 msgr2=0x7f9d9819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.370+0000 7f9d9cddf640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d98108ba0 0x7f9d9819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.370+0000 7f9d9cddf640 1 -- 172.21.15.115:0/60612407 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d981a6920 con 0x7f9d98109040 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.370+0000 7f9d9cddf640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d98109040 0x7f9d9819c090 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9d90031040 tx=0x7f9d90032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.370+0000 7f9d95ffb640 1 -- 172.21.15.115:0/60612407 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90004040 con 0x7f9d98109040 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.370+0000 7f9d9e842640 1 -- 172.21.15.115:0/60612407 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d981a6bb0 con 0x7f9d98109040 2024-05-09T05:39:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.370+0000 7f9d95ffb640 1 -- 172.21.15.115:0/60612407 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d90004670 con 0x7f9d98109040 2024-05-09T05:39:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.370+0000 7f9d95ffb640 1 -- 172.21.15.115:0/60612407 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90030bb0 con 0x7f9d98109040 2024-05-09T05:39:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.370+0000 7f9d9e842640 1 -- 172.21.15.115:0/60612407 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d981a7150 con 0x7f9d98109040 2024-05-09T05:39:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9da57cf740 1 Processor -- start 2024-05-09T05:39:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9d95ffb640 1 -- 172.21.15.115:0/60612407 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f9d900041e0 con 0x7f9d98109040 2024-05-09T05:39:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9da57cf740 1 -- start start 2024-05-09T05:39:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 0x563d1a06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 0x563d1a06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a070100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9da57cf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563d19fd47c0 con 0x563d19cab4d0 2024-05-09T05:39:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9da57cf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563d19f700e0 con 0x563d19ca32f0 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9da57cf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563d19fd0780 con 0x563d19ccc1b0 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a070100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47294/0 (socket says 172.21.15.115:47294) 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9d8ffff640 1 -- 172.21.15.115:0/4237287570 learned_addr learned my addr 172.21.15.115:0/4237287570 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.373+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 0x563d1a06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d97fff640 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 0x563d1a06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d95ffb640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d840b2a70 0x7f9d840b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d8ffff640 1 -- 172.21.15.115:0/4237287570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 msgr2=0x563d1a06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 0x563d1a06f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d8ffff640 1 -- 172.21.15.115:0/4237287570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 msgr2=0x563d1a06fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 0x563d1a06fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d8ffff640 1 -- 172.21.15.115:0/4237287570 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d19f997c0 con 0x563d19cab4d0 2024-05-09T05:39:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d97fff640 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 0x563d1a06fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d9cddf640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d840b2a70 0x7f9d840b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a070100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005bc0 tx=0x7f9d8000c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7603235537feb883 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d95ffb640 1 -- 172.21.15.115:0/60612407 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9d90103fa0 con 0x7f9d98109040 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d8e7fc640 1 -- 172.21.15.115:0/4237287570 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d800142d0 con 0x563d19cab4d0 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d8e7fc640 1 -- 172.21.15.115:0/4237287570 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d800054b0 con 0x563d19cab4d0 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.374+0000 7f9d8e7fc640 1 -- 172.21.15.115:0/4237287570 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80004e10 con 0x563d19cab4d0 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9da57cf740 1 -- 172.21.15.115:0/4237287570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 msgr2=0x563d1a070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9d9cddf640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d840b2a70 0x7f9d840b4f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d901013a0 tx=0x7f9d900989d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9da57cf740 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a070100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005bc0 tx=0x7f9d8000c410 comp rx=0 tx=0).stop 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9da57cf740 1 -- 172.21.15.115:0/4237287570 shutdown_connections 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9da57cf740 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a070100 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9da57cf740 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 0x563d1a06fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9da57cf740 1 --2- 172.21.15.115:0/4237287570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 0x563d1a06f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9da57cf740 1 -- 172.21.15.115:0/4237287570 >> 172.21.15.115:0/4237287570 conn(0x563d19cc0300 msgr2=0x563d1a06be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9da57cf740 1 -- 172.21.15.115:0/4237287570 shutdown_connections 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.375+0000 7f9da57cf740 1 -- 172.21.15.115:0/4237287570 wait complete. 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9da57cf740 1 -- start start 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9da57cf740 1 Processor -- start 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 0x563d1a07b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a07bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 0x563d1a079bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9da57cf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563d1a07c7b0 con 0x563d19cab4d0 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9da57cf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563d19f5b9c0 con 0x563d19ca32f0 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9da57cf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563d1a07c930 con 0x563d19ccc1b0 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9da57cf740 10 client.0 _ll_register_callbacks cb 0x563d19f88020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9d97fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a07bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 0x563d1a07b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9d97fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a07bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47304/0 (socket says 172.21.15.115:47304) 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.376+0000 7f9d97fff640 1 -- 172.21.15.115:0/1112432678 learned_addr learned my addr 172.21.15.115:0/1112432678 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 0x563d1a079bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d97fff640 1 -- 172.21.15.115:0/1112432678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 msgr2=0x563d1a079bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d97fff640 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 0x563d1a079bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d97fff640 1 -- 172.21.15.115:0/1112432678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 msgr2=0x563d1a07b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d97fff640 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 0x563d1a07b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d97fff640 1 -- 172.21.15.115:0/1112432678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d19f821f0 con 0x563d19cab4d0 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d97fff640 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a07bcb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800b970 tx=0x7f9d8800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481648513d73aaf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d777fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d777fe640 1 -- 172.21.15.115:0/1112432678 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8801b440 con 0x563d19cab4d0 2024-05-09T05:39:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d777fe640 1 -- 172.21.15.115:0/1112432678 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9d8801b5e0 con 0x563d19cab4d0 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9da57cf740 1 -- 172.21.15.115:0/1112432678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d800054b0 con 0x563d19cab4d0 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d777fe640 1 -- 172.21.15.115:0/1112432678 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d88024050 con 0x563d19cab4d0 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9da57cf740 10 client.15864 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9da57cf740 1 -- 172.21.15.115:0/1112432678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d19d37790 con 0x563d19cab4d0 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.377+0000 7f9d777fe640 1 -- 172.21.15.115:0/1112432678 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d800054b0 con 0x563d19cab4d0 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9d777fe640 1 -- 172.21.15.115:0/1112432678 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9d88019020 con 0x563d19cab4d0 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9da57cf740 10 client.15864 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9da57cf740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9da57cf740 20 client.15864 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9da57cf740 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563d19c8ad80 0x563d1a08c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9da57cf740 4 client.15864 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9da57cf740 1 -- 172.21.15.115:0/1112432678 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d19fe23d0 con 0x563d19c8ad80 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563d19c8ad80 0x563d1a08c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9d777fe640 10 client.15864 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.378+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563d19c8ad80 0x563d1a08c050 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e301b94880536372 server_cookie=f96c7b7c1c903979 in_seq=0 out_seq=0 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.379+0000 7f9d777fe640 1 -- 172.21.15.115:0/1112432678 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d19fe23d0 con 0x563d19c8ad80 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.379+0000 7f9d777fe640 10 client.15864 handle_command_reply: tid=0 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.379+0000 7f9da57cf740 1 client.15864 shutdown 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.379+0000 7f9da57cf740 1 -- 172.21.15.115:0/1112432678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 msgr2=0x563d1a07bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.379+0000 7f9da57cf740 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a07bcb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800b970 tx=0x7f9d8800bd70 comp rx=0 tx=0).stop 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.379+0000 7f9da57cf740 1 -- 172.21.15.115:0/1112432678 shutdown_connections 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.379+0000 7f9da57cf740 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563d19c8ad80 0x563d1a08c050 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.379+0000 7f9da57cf740 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ccc1b0 0x563d1a079bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.380+0000 7f9da57cf740 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19cab4d0 0x563d1a07bcb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.380+0000 7f9d777fe640 0 client.15864 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.380+0000 7f9da57cf740 1 --2- 172.21.15.115:0/1112432678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ca32f0 0x563d1a07b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.380+0000 7f9da57cf740 1 -- 172.21.15.115:0/1112432678 >> 172.21.15.115:0/1112432678 conn(0x563d19cc0300 msgr2=0x563d1a0713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.380+0000 7f9da57cf740 1 -- 172.21.15.115:0/1112432678 shutdown_connections 2024-05-09T05:39:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.380+0000 7f9da57cf740 1 -- 172.21.15.115:0/1112432678 wait complete. 2024-05-09T05:39:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.380+0000 7f9da57cf740 20 client.15864 trim_cache size 0 max 16384 2024-05-09T05:39:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.395+0000 7f9da57cf740 1 Processor -- start 2024-05-09T05:39:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.395+0000 7f9da57cf740 1 -- start start 2024-05-09T05:39:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.395+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ca32f0 0x563d1a079b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.395+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ccc1b0 0x563d1a081c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.395+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19c601c0 0x563d1a082150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.395+0000 7f9da57cf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b1010 con 0x563d19ca32f0 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.395+0000 7f9da57cf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b2b50 con 0x563d19ccc1b0 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.395+0000 7f9da57cf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b16e0 con 0x563d19c601c0 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ca32f0 0x563d1a079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ca32f0 0x563d1a079b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47314/0 (socket says 172.21.15.115:47314) 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8ffff640 1 -- 172.21.15.115:0/3894800230 learned_addr learned my addr 172.21.15.115:0/3894800230 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d97fff640 1 --2- 172.21.15.115:0/3894800230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ccc1b0 0x563d1a081c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8ffff640 1 -- 172.21.15.115:0/3894800230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19c601c0 msgr2=0x563d1a082150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/3894800230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19c601c0 0x563d1a082150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8ffff640 1 -- 172.21.15.115:0/3894800230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ccc1b0 msgr2=0x563d1a081c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/3894800230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ccc1b0 0x563d1a081c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8ffff640 1 -- 172.21.15.115:0/3894800230 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d19dae5f0 con 0x563d19ca32f0 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/3894800230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ca32f0 0x563d1a079b90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005b90 tx=0x7f9d80005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da90708d175c6216 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8dffb640 1 -- 172.21.15.115:0/3894800230 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80014210 con 0x563d19ca32f0 2024-05-09T05:39:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8dffb640 1 -- 172.21.15.115:0/3894800230 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d80031890 con 0x563d19ca32f0 2024-05-09T05:39:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.396+0000 7f9d8dffb640 1 -- 172.21.15.115:0/3894800230 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000f730 con 0x563d19ca32f0 2024-05-09T05:39:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.397+0000 7f9da57cf740 1 -- 172.21.15.115:0/3894800230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ca32f0 msgr2=0x563d1a079b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.397+0000 7f9da57cf740 1 --2- 172.21.15.115:0/3894800230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ca32f0 0x563d1a079b90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005b90 tx=0x7f9d80005870 comp rx=0 tx=0).stop 2024-05-09T05:39:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.397+0000 7f9da57cf740 1 -- 172.21.15.115:0/3894800230 shutdown_connections 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.397+0000 7f9da57cf740 1 --2- 172.21.15.115:0/3894800230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19c601c0 0x563d1a082150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.397+0000 7f9da57cf740 1 --2- 172.21.15.115:0/3894800230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19ccc1b0 0x563d1a081c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.397+0000 7f9da57cf740 1 --2- 172.21.15.115:0/3894800230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ca32f0 0x563d1a079b90 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.397+0000 7f9da57cf740 1 -- 172.21.15.115:0/3894800230 >> 172.21.15.115:0/3894800230 conn(0x563d19cc0300 msgr2=0x563d1a079480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.397+0000 7f9da57cf740 1 -- 172.21.15.115:0/3894800230 shutdown_connections 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.397+0000 7f9da57cf740 1 -- 172.21.15.115:0/3894800230 wait complete. 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9da57cf740 1 -- start start 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9da57cf740 1 Processor -- start 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19c601c0 0x563d1a07d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ca32f0 0x563d1a07dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9da57cf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ccc1b0 0x563d1a07e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9da57cf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b2480 con 0x563d19ccc1b0 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9da57cf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b1db0 con 0x563d19c601c0 2024-05-09T05:39:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9da57cf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981aeff0 con 0x563d19ca32f0 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9da57cf740 10 client.0 _ll_register_callbacks cb 0x7f9d88004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ccc1b0 0x563d1a07e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9d8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ccc1b0 0x563d1a07e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47328/0 (socket says 172.21.15.115:47328) 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9d8ffff640 1 -- 172.21.15.115:0/779066220 learned_addr learned my addr 172.21.15.115:0/779066220 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.398+0000 7f9d97fff640 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ca32f0 0x563d1a07dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19c601c0 0x563d1a07d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d8ffff640 1 -- 172.21.15.115:0/779066220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ca32f0 msgr2=0x563d1a07dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ca32f0 0x563d1a07dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d8ffff640 1 -- 172.21.15.115:0/779066220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19c601c0 msgr2=0x563d1a07d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19c601c0 0x563d1a07d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d8ffff640 1 -- 172.21.15.115:0/779066220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d19f98ed0 con 0x563d19ccc1b0 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d8ffff640 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ccc1b0 0x563d1a07e160 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005780 tx=0x7f9d8000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ff006680ee013a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d777fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d777fe640 1 -- 172.21.15.115:0/779066220 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80039070 con 0x563d19ccc1b0 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d777fe640 1 -- 172.21.15.115:0/779066220 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9d80031600 con 0x563d19ccc1b0 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9da57cf740 1 -- 172.21.15.115:0/779066220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d80031890 con 0x563d19ccc1b0 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d777fe640 1 -- 172.21.15.115:0/779066220 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d80004020 con 0x563d19ccc1b0 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9da57cf740 10 client.15870 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9da57cf740 1 -- 172.21.15.115:0/779066220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d1a0f42a0 con 0x563d19ccc1b0 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.399+0000 7f9d777fe640 1 -- 172.21.15.115:0/779066220 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d80031890 con 0x563d19ccc1b0 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9d777fe640 1 -- 172.21.15.115:0/779066220 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9d8003e020 con 0x563d19ccc1b0 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9da57cf740 10 client.15870 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9da57cf740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9da57cf740 20 client.15870 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9da57cf740 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563d1a0f4700 0x563d1a0f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9da57cf740 4 client.15870 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9da57cf740 1 -- 172.21.15.115:0/779066220 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d19fdc240 con 0x563d1a0f4700 2024-05-09T05:39:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9d97fff640 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563d1a0f4700 0x563d1a0f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9d97fff640 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563d1a0f4700 0x563d1a0f6b20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b9eae918f4cdd0 server_cookie=96554fb99c775dde in_seq=0 out_seq=0 2024-05-09T05:39:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.400+0000 7f9d777fe640 10 client.15870 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.404+0000 7f9d777fe640 1 -- 172.21.15.115:0/779066220 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563d19fdc240 con 0x563d1a0f4700 2024-05-09T05:39:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.404+0000 7f9d777fe640 10 client.15870 handle_command_reply: tid=0 2024-05-09T05:39:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.404+0000 7f9da57cf740 1 client.15870 shutdown 2024-05-09T05:39:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 -- 172.21.15.115:0/779066220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ccc1b0 msgr2=0x563d1a07e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ccc1b0 0x563d1a07e160 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005780 tx=0x7f9d8000cb10 comp rx=0 tx=0).stop 2024-05-09T05:39:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 -- 172.21.15.115:0/779066220 shutdown_connections 2024-05-09T05:39:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563d19ccc1b0 0x563d1a07e160 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563d1a0f4700 0x563d1a0f6b20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563d19ca32f0 0x563d1a07dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 --2- 172.21.15.115:0/779066220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d19c601c0 0x563d1a07d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d777fe640 0 client.15870 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 -- 172.21.15.115:0/779066220 >> 172.21.15.115:0/779066220 conn(0x563d19cc0300 msgr2=0x563d1a078090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 -- 172.21.15.115:0/779066220 shutdown_connections 2024-05-09T05:39:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 1 -- 172.21.15.115:0/779066220 wait complete. 2024-05-09T05:39:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9da57cf740 20 client.15870 trim_cache size 0 max 16384 2024-05-09T05:39:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:39:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:39:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:39:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 -- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d840b2a70 msgr2=0x7f9d840b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d840b2a70 0x7f9d840b4f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d901013a0 tx=0x7f9d900989d0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 -- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d98109040 msgr2=0x7f9d9819c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d98109040 0x7f9d9819c090 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9d90031040 tx=0x7f9d90032bc0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 -- 172.21.15.115:0/60612407 shutdown_connections 2024-05-09T05:39:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9d840b2a70 0x7f9d840b4f80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d98109040 0x7f9d9819c090 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d98108ba0 0x7f9d9819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 --2- 172.21.15.115:0/60612407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d981032f0 0x7f9d9819b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 -- 172.21.15.115:0/60612407 >> 172.21.15.115:0/60612407 conn(0x7f9d980782e0 msgr2=0x7f9d980ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 -- 172.21.15.115:0/60612407 shutdown_connections 2024-05-09T05:39:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:32.405+0000 7f9d9e842640 1 -- 172.21.15.115:0/60612407 wait complete. 2024-05-09T05:39:32.611 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0' 2024-05-09T05:39:32.612 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0 2024-05-09T05:39:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:33.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.631+0000 7f2c755f5700 1 -- 172.21.15.115:0/3415891668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c70109300 msgr2=0x7f2c70109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:33.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.631+0000 7f2c755f5700 1 --2- 172.21.15.115:0/3415891668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c70109300 0x7f2c70109710 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009990 tx=0x7f2c60009ca0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.631+0000 7f2c755f5700 1 -- 172.21.15.115:0/3415891668 shutdown_connections 2024-05-09T05:39:33.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.631+0000 7f2c755f5700 1 --2- 172.21.15.115:0/3415891668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c701030e0 0x7f2c70111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.631+0000 7f2c755f5700 1 --2- 172.21.15.115:0/3415891668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c70109300 0x7f2c70109710 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.631+0000 7f2c755f5700 1 --2- 172.21.15.115:0/3415891668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c70102790 0x7f2c70102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.631+0000 7f2c755f5700 1 -- 172.21.15.115:0/3415891668 >> 172.21.15.115:0/3415891668 conn(0x7f2c700fe4e0 msgr2=0x7f2c70100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:33.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.631+0000 7f2c755f5700 1 -- 172.21.15.115:0/3415891668 shutdown_connections 2024-05-09T05:39:33.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.631+0000 7f2c755f5700 1 -- 172.21.15.115:0/3415891668 wait complete. 2024-05-09T05:39:33.625 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:33 smithi115 ceph-mon[30349]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.632+0000 7f2c755f5700 1 Processor -- start 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.632+0000 7f2c755f5700 1 -- start start 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c755f5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c70102790 0x7f2c7019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c755f5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c701030e0 0x7f2c7019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c755f5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c70109300 0x7f2c7019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c755f5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c70114440 con 0x7f2c701030e0 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c755f5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c701142c0 con 0x7f2c70109300 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c6e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c701030e0 0x7f2c7019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c6e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c701030e0 0x7f2c7019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40088/0 (socket says 172.21.15.115:40088) 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c6e7fc700 1 -- 172.21.15.115:0/2824382544 learned_addr learned my addr 172.21.15.115:0/2824382544 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c755f5700 1 -- 172.21.15.115:0/2824382544 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c701145c0 con 0x7f2c70102790 2024-05-09T05:39:33.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c6effd700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c70102790 0x7f2c7019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c6f7fe700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c70109300 0x7f2c7019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.633+0000 7f2c6e7fc700 1 -- 172.21.15.115:0/2824382544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c70102790 msgr2=0x7f2c7019aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:33.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c70102790 0x7f2c7019aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c6e7fc700 1 -- 172.21.15.115:0/2824382544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c70109300 msgr2=0x7f2c7019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:33.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c70109300 0x7f2c7019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c6e7fc700 1 -- 172.21.15.115:0/2824382544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7019fca0 con 0x7f2c701030e0 2024-05-09T05:39:33.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c701030e0 0x7f2c7019b470 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c9e0 tx=0x7f2c6400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:33.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c57fff700 1 -- 172.21.15.115:0/2824382544 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64007aa0 con 0x7f2c701030e0 2024-05-09T05:39:33.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c755f5700 1 -- 172.21.15.115:0/2824382544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7019ff90 con 0x7f2c701030e0 2024-05-09T05:39:33.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c57fff700 1 -- 172.21.15.115:0/2824382544 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c64007c40 con 0x7f2c701030e0 2024-05-09T05:39:33.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c57fff700 1 -- 172.21.15.115:0/2824382544 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64018400 con 0x7f2c701030e0 2024-05-09T05:39:33.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.634+0000 7f2c755f5700 1 -- 172.21.15.115:0/2824382544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c701a7960 con 0x7f2c701030e0 2024-05-09T05:39:33.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.638+0000 7f2c57fff700 1 -- 172.21.15.115:0/2824382544 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f2c64004870 con 0x7f2c701030e0 2024-05-09T05:39:33.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.639+0000 7f2c57fff700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c580b2b10 0x7f2c580b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.639+0000 7f2c8400bb80 1 Processor -- start 2024-05-09T05:39:33.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.639+0000 7f2c57fff700 1 -- 172.21.15.115:0/2824382544 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2c640e2b40 con 0x7f2c701030e0 2024-05-09T05:39:33.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.639+0000 7f2c8400bb80 1 -- start start 2024-05-09T05:39:33.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.639+0000 7f2c6effd700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c580b2b10 0x7f2c580b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594a851e0 0x55d594b1e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b1eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b18b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d594a3b070 con 0x55d594a851e0 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d5949d9870 con 0x55d594b18720 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d594a37030 con 0x55d59499b520 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594a851e0 0x55d594b1e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594a851e0 0x55d594b1e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40098/0 (socket says 172.21.15.115:40098) 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6effd700 1 -- 172.21.15.115:0/635733542 learned_addr learned my addr 172.21.15.115:0/635733542 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/635733542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b1eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6effd700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c580b2b10 0x7f2c580b5020 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c600059d0 tx=0x7f2c60005960 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6f7fe700 1 --2- 172.21.15.115:0/635733542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b18b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6e7fc700 1 -- 172.21.15.115:0/635733542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 msgr2=0x55d594b18b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/635733542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b18b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6e7fc700 1 -- 172.21.15.115:0/635733542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594a851e0 msgr2=0x55d594b1e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/635733542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594a851e0 0x55d594b1e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.640+0000 7f2c6e7fc700 1 -- 172.21.15.115:0/635733542 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5949d13f0 con 0x55d59499b520 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.641+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/635733542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b1eab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c640dfdd0 tx=0x7f2c640a9080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d68c514dd22dd71c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.641+0000 7f2c557fa700 1 -- 172.21.15.115:0/635733542 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64007df0 con 0x55d59499b520 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.641+0000 7f2c557fa700 1 -- 172.21.15.115:0/635733542 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c640a9890 con 0x55d59499b520 2024-05-09T05:39:33.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.641+0000 7f2c557fa700 1 -- 172.21.15.115:0/635733542 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c640b2260 con 0x55d59499b520 2024-05-09T05:39:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 -- 172.21.15.115:0/635733542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 msgr2=0x55d594b1eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/635733542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b1eab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c640dfdd0 tx=0x7f2c640a9080 comp rx=0 tx=0).stop 2024-05-09T05:39:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 -- 172.21.15.115:0/635733542 shutdown_connections 2024-05-09T05:39:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/635733542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b18b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/635733542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594a851e0 0x55d594b1e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/635733542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b1eab0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 -- 172.21.15.115:0/635733542 >> 172.21.15.115:0/635733542 conn(0x55d5949fb720 msgr2=0x55d594b13480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 -- 172.21.15.115:0/635733542 shutdown_connections 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 -- 172.21.15.115:0/635733542 wait complete. 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.642+0000 7f2c8400bb80 1 -- start start 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c8400bb80 1 Processor -- start 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b22290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 0x55d594b227d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b22d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d5948b6c20 con 0x55d59499b520 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d5949f51d0 con 0x55d594b18720 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d594b25250 con 0x55d594a851e0 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c8400bb80 10 client.0 _ll_register_callbacks cb 0x55d5948ac740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c6effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b22290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c6effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b22290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40102/0 (socket says 172.21.15.115:40102) 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c6effd700 1 -- 172.21.15.115:0/369845207 learned_addr learned my addr 172.21.15.115:0/369845207 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 0x55d594b227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.643+0000 7f2c6f7fe700 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b22d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c6effd700 1 -- 172.21.15.115:0/369845207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 msgr2=0x55d594b227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c6effd700 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 0x55d594b227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c6effd700 1 -- 172.21.15.115:0/369845207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 msgr2=0x55d594b22d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c6effd700 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b22d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c6effd700 1 -- 172.21.15.115:0/369845207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5948b6510 con 0x55d59499b520 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c6f7fe700 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b22d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c6effd700 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b22290 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2c60003760 tx=0x7f2c60009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d61396dfe7ea60e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c52ff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c52ff5700 1 -- 172.21.15.115:0/369845207 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000f460 con 0x55d59499b520 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c52ff5700 1 -- 172.21.15.115:0/369845207 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2c6000f600 con 0x55d59499b520 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c8400bb80 1 -- 172.21.15.115:0/369845207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c640a9890 con 0x55d59499b520 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c52ff5700 1 -- 172.21.15.115:0/369845207 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c6000f800 con 0x55d59499b520 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c8400bb80 10 client.15882 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.644+0000 7f2c8400bb80 1 -- 172.21.15.115:0/369845207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d594b384c0 con 0x55d59499b520 2024-05-09T05:39:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.645+0000 7f2c52ff5700 1 -- 172.21.15.115:0/369845207 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c640a9890 con 0x55d59499b520 2024-05-09T05:39:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.646+0000 7f2c52ff5700 1 -- 172.21.15.115:0/369845207 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2c6002e020 con 0x55d59499b520 2024-05-09T05:39:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.646+0000 7f2c8400bb80 10 client.15882 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.646+0000 7f2c8400bb80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.646+0000 7f2c8400bb80 20 client.15882 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.646+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d594b3ab00 0x55d594b3cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.646+0000 7f2c8400bb80 4 client.15882 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.646+0000 7f2c8400bb80 1 -- 172.21.15.115:0/369845207 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d594a48a50 con 0x55d594b3ab00 2024-05-09T05:39:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.646+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d594b3ab00 0x55d594b3cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.647+0000 7f2c52ff5700 10 client.15882 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.647+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d594b3ab00 0x55d594b3cf20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cc65ea453d81d5 server_cookie=919647f87c475836 in_seq=0 out_seq=0 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.647+0000 7f2c52ff5700 1 -- 172.21.15.115:0/369845207 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d594a48a50 con 0x55d594b3ab00 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.647+0000 7f2c52ff5700 10 client.15882 handle_command_reply: tid=0 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.647+0000 7f2c8400bb80 1 client.15882 shutdown 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 -- 172.21.15.115:0/369845207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 msgr2=0x55d594b22290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b22290 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2c60003760 tx=0x7f2c60009ca0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 -- 172.21.15.115:0/369845207 shutdown_connections 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b22d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d594b3ab00 0x55d594b3cf20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b22290 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/369845207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 0x55d594b227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 -- 172.21.15.115:0/369845207 >> 172.21.15.115:0/369845207 conn(0x55d5949fb720 msgr2=0x55d594b13240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c52ff5700 0 client.15882 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 -- 172.21.15.115:0/369845207 shutdown_connections 2024-05-09T05:39:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 1 -- 172.21.15.115:0/369845207 wait complete. 2024-05-09T05:39:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.648+0000 7f2c8400bb80 20 client.15882 trim_cache size 0 max 16384 2024-05-09T05:39:33.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c8400bb80 1 Processor -- start 2024-05-09T05:39:33.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c8400bb80 1 -- start start 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594a851e0 0x55d594b22b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594b18720 0x55d594b24050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c701ae110 con 0x55d594b18720 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c701afc50 con 0x55d594a851e0 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c701ae7e0 con 0x55d59499b520 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c6f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594b18720 0x55d594b24050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c6f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594b18720 0x55d594b24050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40108/0 (socket says 172.21.15.115:40108) 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c6f7fe700 1 -- 172.21.15.115:0/518554037 learned_addr learned my addr 172.21.15.115:0/518554037 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c6effd700 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.669+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594a851e0 0x55d594b22b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c6f7fe700 1 -- 172.21.15.115:0/518554037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 msgr2=0x55d594b215d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c6f7fe700 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b215d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c6f7fe700 1 -- 172.21.15.115:0/518554037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594a851e0 msgr2=0x55d594b22b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:33.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c6f7fe700 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594a851e0 0x55d594b22b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c6f7fe700 1 -- 172.21.15.115:0/518554037 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d594b1e0b0 con 0x55d594b18720 2024-05-09T05:39:33.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c6effd700 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:33.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c6f7fe700 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594b18720 0x55d594b24050 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009940 tx=0x7f2c68009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c498f68d1a7949d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:33.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c54ff9700 1 -- 172.21.15.115:0/518554037 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6801a070 con 0x55d594b18720 2024-05-09T05:39:33.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c54ff9700 1 -- 172.21.15.115:0/518554037 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c68004db0 con 0x55d594b18720 2024-05-09T05:39:33.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.670+0000 7f2c54ff9700 1 -- 172.21.15.115:0/518554037 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68007be0 con 0x55d594b18720 2024-05-09T05:39:33.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.671+0000 7f2c8400bb80 1 -- 172.21.15.115:0/518554037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594b18720 msgr2=0x55d594b24050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.671+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594b18720 0x55d594b24050 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009940 tx=0x7f2c68009970 comp rx=0 tx=0).stop 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.671+0000 7f2c8400bb80 1 -- 172.21.15.115:0/518554037 shutdown_connections 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.671+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d594b18720 0x55d594b24050 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.671+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d59499b520 0x55d594b215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.671+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/518554037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594a851e0 0x55d594b22b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.671+0000 7f2c8400bb80 1 -- 172.21.15.115:0/518554037 >> 172.21.15.115:0/518554037 conn(0x55d5949fb720 msgr2=0x55d594b23940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.671+0000 7f2c8400bb80 1 -- 172.21.15.115:0/518554037 shutdown_connections 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.671+0000 7f2c8400bb80 1 -- 172.21.15.115:0/518554037 wait complete. 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c8400bb80 1 -- start start 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c8400bb80 1 Processor -- start 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 0x55d594b2fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c8400bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c701af580 con 0x55d59499b520 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c701aeeb0 con 0x55d594b18720 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c8400bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c701ac0b0 con 0x55d594a851e0 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c8400bb80 10 client.0 _ll_register_callbacks cb 0x55d5948ac740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c6effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c6effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b331a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40112/0 (socket says 172.21.15.115:40112) 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c6effd700 1 -- 172.21.15.115:0/3011786302 learned_addr learned my addr 172.21.15.115:0/3011786302 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c6f7fe700 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.672+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 0x55d594b2fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c6effd700 1 -- 172.21.15.115:0/3011786302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 msgr2=0x55d594b2fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c6effd700 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 0x55d594b2fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c6effd700 1 -- 172.21.15.115:0/3011786302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 msgr2=0x55d594b301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c6effd700 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c6effd700 1 -- 172.21.15.115:0/3011786302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d594b30800 con 0x55d59499b520 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c6effd700 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b331a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009600 tx=0x7f2c6001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20c58dba90cb55c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:33.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c52ff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c52ff5700 1 -- 172.21.15.115:0/3011786302 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60042060 con 0x55d59499b520 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c52ff5700 1 -- 172.21.15.115:0/3011786302 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2c6000f9a0 con 0x55d59499b520 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c8400bb80 1 -- 172.21.15.115:0/3011786302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c68004db0 con 0x55d59499b520 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c52ff5700 1 -- 172.21.15.115:0/3011786302 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c60027040 con 0x55d59499b520 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c8400bb80 10 client.15888 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c8400bb80 1 -- 172.21.15.115:0/3011786302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d594ba60b0 con 0x55d59499b520 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.673+0000 7f2c52ff5700 1 -- 172.21.15.115:0/3011786302 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c68004db0 con 0x55d59499b520 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c52ff5700 1 -- 172.21.15.115:0/3011786302 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2c6002e070 con 0x55d59499b520 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c8400bb80 10 client.15888 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c8400bb80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c8400bb80 20 client.15888 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d594ba6600 0x55d594ba8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c8400bb80 4 client.15888 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:39:33.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c8400bb80 1 -- 172.21.15.115:0/3011786302 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d594a42af0 con 0x55d594ba6600 2024-05-09T05:39:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d594ba6600 0x55d594ba8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c52ff5700 10 client.15888 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.674+0000 7f2c6e7fc700 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d594ba6600 0x55d594ba8a20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a2f10bda73d9bf server_cookie=83e5f4094c5b7870 in_seq=0 out_seq=0 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c52ff5700 1 -- 172.21.15.115:0/3011786302 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d594a42af0 con 0x55d594ba6600 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c52ff5700 10 client.15888 handle_command_reply: tid=0 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 client.15888 shutdown 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 -- 172.21.15.115:0/3011786302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 msgr2=0x55d594b331a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b331a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009600 tx=0x7f2c6001e000 comp rx=0 tx=0).stop 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 -- 172.21.15.115:0/3011786302 shutdown_connections 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d594b18720 0x55d594b301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d594ba6600 0x55d594ba8a20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c52ff5700 0 client.15888 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d59499b520 0x55d594b331a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 --2- 172.21.15.115:0/3011786302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d594a851e0 0x55d594b2fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 -- 172.21.15.115:0/3011786302 >> 172.21.15.115:0/3011786302 conn(0x55d5949fb720 msgr2=0x55d594b21330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:33.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 -- 172.21.15.115:0/3011786302 shutdown_connections 2024-05-09T05:39:33.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 1 -- 172.21.15.115:0/3011786302 wait complete. 2024-05-09T05:39:33.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.676+0000 7f2c8400bb80 20 client.15888 trim_cache size 0 max 16384 2024-05-09T05:39:33.673 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:39:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:39:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "0daa7bb2-eda5-44dc-97af-84a9f86fc044", 2024-05-09T05:39:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:39:33.673 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:39:33.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 -- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c580b2b10 msgr2=0x7f2c580b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:33.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c580b2b10 0x7f2c580b5020 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c600059d0 tx=0x7f2c60005960 comp rx=0 tx=0).stop 2024-05-09T05:39:33.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 -- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c701030e0 msgr2=0x7f2c7019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:33.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c701030e0 0x7f2c7019b470 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c9e0 tx=0x7f2c6400ccf0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 -- 172.21.15.115:0/2824382544 shutdown_connections 2024-05-09T05:39:33.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c580b2b10 0x7f2c580b5020 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c70109300 0x7f2c7019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c70102790 0x7f2c7019aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 --2- 172.21.15.115:0/2824382544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c701030e0 0x7f2c7019b470 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:33.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 -- 172.21.15.115:0/2824382544 >> 172.21.15.115:0/2824382544 conn(0x7f2c700fe4e0 msgr2=0x7f2c70107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:33.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 -- 172.21.15.115:0/2824382544 shutdown_connections 2024-05-09T05:39:33.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:33.677+0000 7f2c755f5700 1 -- 172.21.15.115:0/2824382544 wait complete. 2024-05-09T05:39:33.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:33 smithi195 ceph-mon[33844]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:33.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:33 smithi161 ceph-mon[33805]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:34 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:39:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:34 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:39:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:34 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:39:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:34 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:39:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:34 smithi115 ceph-mon[30349]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-05-09T05:39:34.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:34 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:39:34.702 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:39:34.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:34 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:39:34.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:34 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:39:34.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:34 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:39:34.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:34 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:39:34.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:34 smithi195 ceph-mon[33844]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-05-09T05:39:34.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:34 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:39:34.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:34 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:39:34.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:34 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:39:34.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:34 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:39:34.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:34 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:39:34.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:34 smithi161 ceph-mon[33805]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-05-09T05:39:34.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:34 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:39:35.797 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:37.338 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:37 smithi115 ceph-mon[30349]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:37.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:37 smithi195 ceph-mon[33844]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:37.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:37 smithi161 ceph-mon[33805]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:37.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.654+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/2744469037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c4810e8f0 msgr2=0x7f2c48110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:37.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.654+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/2744469037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c4810e8f0 0x7f2c48110dd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000b0a0 tx=0x7f2c4000b3b0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.654+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/2744469037 shutdown_connections 2024-05-09T05:39:37.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.654+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/2744469037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c481054c0 0x7f2c481058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.654+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/2744469037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c4810e8f0 0x7f2c48110dd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.654+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/2744469037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c48105e10 0x7f2c4810e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.654+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/2744469037 >> 172.21.15.115:0/2744469037 conn(0x7f2c480fedc0 msgr2=0x7f2c481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:37.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.654+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/2744469037 shutdown_connections 2024-05-09T05:39:37.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.654+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/2744469037 wait complete. 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.655+0000 7f2c4f3a6700 1 Processor -- start 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.655+0000 7f2c4f3a6700 1 -- start start 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.655+0000 7f2c4f3a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c481054c0 0x7f2c481a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.655+0000 7f2c4f3a6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c48105e10 0x7f2c481a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.655+0000 7f2c4f3a6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c4810e8f0 0x7f2c4819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.655+0000 7f2c4f3a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c48114300 con 0x7f2c481054c0 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.655+0000 7f2c4f3a6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c48114180 con 0x7f2c48105e10 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.655+0000 7f2c4f3a6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c48114480 con 0x7f2c4810e8f0 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d142700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c481054c0 0x7f2c481a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d142700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c481054c0 0x7f2c481a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40136/0 (socket says 172.21.15.115:40136) 2024-05-09T05:39:37.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d142700 1 -- 172.21.15.115:0/4057485033 learned_addr learned my addr 172.21.15.115:0/4057485033 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:37.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4c941700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c48105e10 0x7f2c481a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d943700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c4810e8f0 0x7f2c4819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d142700 1 -- 172.21.15.115:0/4057485033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c4810e8f0 msgr2=0x7f2c4819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:37.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d142700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c4810e8f0 0x7f2c4819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d142700 1 -- 172.21.15.115:0/4057485033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c48105e10 msgr2=0x7f2c481a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:37.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d142700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c48105e10 0x7f2c481a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d142700 1 -- 172.21.15.115:0/4057485033 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4819b170 con 0x7f2c481054c0 2024-05-09T05:39:37.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4d142700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c481054c0 0x7f2c481a08a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e9a0 tx=0x7f2c4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:37.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c3e7fc700 1 -- 172.21.15.115:0/4057485033 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400ca10 con 0x7f2c481054c0 2024-05-09T05:39:37.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c3e7fc700 1 -- 172.21.15.115:0/4057485033 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c4400cbb0 con 0x7f2c481054c0 2024-05-09T05:39:37.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/4057485033 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4819b400 con 0x7f2c481054c0 2024-05-09T05:39:37.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c3e7fc700 1 -- 172.21.15.115:0/4057485033 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44018680 con 0x7f2c481054c0 2024-05-09T05:39:37.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.656+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/4057485033 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c481a7730 con 0x7f2c481054c0 2024-05-09T05:39:37.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.659+0000 7f2c3e7fc700 1 -- 172.21.15.115:0/4057485033 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f2c44010430 con 0x7f2c481054c0 2024-05-09T05:39:37.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.660+0000 7f2c3e7fc700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c340b2b10 0x7f2c340b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.660+0000 7f2c4c941700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c340b2b10 0x7f2c340b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.660+0000 7f2c3e7fc700 1 -- 172.21.15.115:0/4057485033 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2c440e2db0 con 0x7f2c481054c0 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.660+0000 7f2c5ddbcb80 1 Processor -- start 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c5ddbcb80 1 -- start start 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4c941700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c340b2b10 0x7f2c340b5020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2c481022c0 tx=0x7f2c3800b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe253640 0x562fbe3d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 0x562fbe3d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562fbe2f3190 con 0x562fbe253640 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562fbe291990 con 0x562fbe33d300 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562fbe2ef150 con 0x562fbe3d0bf0 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4c941700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe253640 0x562fbe3d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4c941700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe253640 0x562fbe3d7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40146/0 (socket says 172.21.15.115:40146) 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4c941700 1 -- 172.21.15.115:0/672424675 learned_addr learned my addr 172.21.15.115:0/672424675 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:37.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4d943700 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 0x562fbe3d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4d142700 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4c941700 1 -- 172.21.15.115:0/672424675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 msgr2=0x562fbe3d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4c941700 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 0x562fbe3d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4c941700 1 -- 172.21.15.115:0/672424675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 msgr2=0x562fbe3d6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4c941700 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3d6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4c941700 1 -- 172.21.15.115:0/672424675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fbe289510 con 0x562fbe253640 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.661+0000 7f2c4d142700 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3d6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.662+0000 7f2c4c941700 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe253640 0x562fbe3d7070 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2c38010dc0 tx=0x7f2c38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641d01ecc86042cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.662+0000 7f2c337fe700 1 -- 172.21.15.115:0/672424675 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38011bb0 con 0x562fbe253640 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.662+0000 7f2c337fe700 1 -- 172.21.15.115:0/672424675 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c38011d50 con 0x562fbe253640 2024-05-09T05:39:37.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.662+0000 7f2c337fe700 1 -- 172.21.15.115:0/672424675 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3801c610 con 0x562fbe253640 2024-05-09T05:39:37.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/672424675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe253640 msgr2=0x562fbe3d7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:37.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe253640 0x562fbe3d7070 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2c38010dc0 tx=0x7f2c38013000 comp rx=0 tx=0).stop 2024-05-09T05:39:37.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/672424675 shutdown_connections 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3d6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 0x562fbe3d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/672424675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe253640 0x562fbe3d7070 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/672424675 >> 172.21.15.115:0/672424675 conn(0x562fbe2b3840 msgr2=0x562fbe3cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/672424675 shutdown_connections 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/672424675 wait complete. 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 -- start start 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.663+0000 7f2c5ddbcb80 1 Processor -- start 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe253640 0x562fbe3da7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562fbe16ed40 con 0x562fbe3d0bf0 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562fbe2ad2f0 con 0x562fbe33d300 2024-05-09T05:39:37.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562fbe3e69a0 con 0x562fbe253640 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c5ddbcb80 10 client.0 _ll_register_callbacks cb 0x562fbe164860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4d943700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4d142700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe253640 0x562fbe3da7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4c941700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4d142700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe253640 0x562fbe3da7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:37542/0 (socket says 172.21.15.115:37542) 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4d142700 1 -- 172.21.15.115:0/2968673493 learned_addr learned my addr 172.21.15.115:0/2968673493 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4d943700 1 -- 172.21.15.115:0/2968673493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe253640 msgr2=0x562fbe3da7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4d943700 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe253640 0x562fbe3da7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4d943700 1 -- 172.21.15.115:0/2968673493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 msgr2=0x562fbe3dace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4d943700 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3dace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.664+0000 7f2c4d943700 1 -- 172.21.15.115:0/2968673493 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fbe16e630 con 0x562fbe3d0bf0 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c4d142700 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe253640 0x562fbe3da7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c4c941700 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:37.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c4d943700 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3db220 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009900 tx=0x7f2c4000bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fa53e7954c0c89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2968673493 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40003c00 con 0x562fbe3d0bf0 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2968673493 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2c40003da0 con 0x562fbe3d0bf0 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2968673493 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c38011d50 con 0x562fbe3d0bf0 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2968673493 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c4002f050 con 0x562fbe3d0bf0 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c5ddbcb80 10 client.15900 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2968673493 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562fbe3e8290 con 0x562fbe3d0bf0 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.665+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2968673493 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c38011d50 con 0x562fbe3d0bf0 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.666+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2968673493 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2c40011070 con 0x562fbe3d0bf0 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.666+0000 7f2c5ddbcb80 10 client.15900 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.666+0000 7f2c5ddbcb80 10 client.15900 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.666+0000 7f2c5ddbcb80 20 client.15900 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.666+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562fbe3ea8d0 0x562fbe3eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.666+0000 7f2c5ddbcb80 4 client.15900 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.666+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2968673493 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fbe300b70 con 0x562fbe3ea8d0 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.666+0000 7f2c4d142700 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562fbe3ea8d0 0x562fbe3eccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.667+0000 7f2c30ff9700 10 client.15900 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.667+0000 7f2c4d142700 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562fbe3ea8d0 0x562fbe3eccf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a372c702fb281b1 server_cookie=967d807d3909c2b0 in_seq=0 out_seq=0 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.667+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2968673493 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562fbe300b70 con 0x562fbe3ea8d0 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.667+0000 7f2c30ff9700 10 client.15900 handle_command_reply: tid=0 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.667+0000 7f2c5ddbcb80 1 client.15900 shutdown 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2968673493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 msgr2=0x562fbe3db220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3db220 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009900 tx=0x7f2c4000bdb0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2968673493 shutdown_connections 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562fbe3ea8d0 0x562fbe3eccf0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3db220 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe253640 0x562fbe3da7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2968673493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe33d300 0x562fbe3dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2968673493 >> 172.21.15.115:0/2968673493 conn(0x562fbe2b3840 msgr2=0x562fbe3cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:37.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c30ff9700 0 client.15900 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:37.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2968673493 shutdown_connections 2024-05-09T05:39:37.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2968673493 wait complete. 2024-05-09T05:39:37.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.668+0000 7f2c5ddbcb80 20 client.15900 trim_cache size 0 max 16384 2024-05-09T05:39:37.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.688+0000 7f2c5ddbcb80 1 Processor -- start 2024-05-09T05:39:37.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.688+0000 7f2c5ddbcb80 1 -- start start 2024-05-09T05:39:37.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 0x562fbe3d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe33d300 0x562fbe3d4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 0x562fbe3db060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481b02f0 con 0x562fbe33d300 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481b1e30 con 0x562fbe253640 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481b09c0 con 0x562fbe3d0bf0 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4c941700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe33d300 0x562fbe3d4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4c941700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe33d300 0x562fbe3d4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40164/0 (socket says 172.21.15.115:40164) 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4c941700 1 -- 172.21.15.115:0/1843531335 learned_addr learned my addr 172.21.15.115:0/1843531335 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4d142700 1 --2- 172.21.15.115:0/1843531335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 0x562fbe3d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4d943700 1 --2- 172.21.15.115:0/1843531335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 0x562fbe3db060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4c941700 1 -- 172.21.15.115:0/1843531335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 msgr2=0x562fbe3db060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4c941700 1 --2- 172.21.15.115:0/1843531335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 0x562fbe3db060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4c941700 1 -- 172.21.15.115:0/1843531335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 msgr2=0x562fbe3d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:37.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4c941700 1 --2- 172.21.15.115:0/1843531335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 0x562fbe3d0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4c941700 1 -- 172.21.15.115:0/1843531335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fbe3cc9f0 con 0x562fbe33d300 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c4c941700 1 --2- 172.21.15.115:0/1843531335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe33d300 0x562fbe3d4d60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b9b0 tx=0x7f2c3801fff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad113dd5d9cd2ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c32ffd700 1 -- 172.21.15.115:0/1843531335 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38011db0 con 0x562fbe33d300 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c32ffd700 1 -- 172.21.15.115:0/1843531335 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c38036070 con 0x562fbe33d300 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.689+0000 7f2c32ffd700 1 -- 172.21.15.115:0/1843531335 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38004ab0 con 0x562fbe33d300 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.690+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/1843531335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe33d300 msgr2=0x562fbe3d4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.690+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/1843531335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe33d300 0x562fbe3d4d60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b9b0 tx=0x7f2c3801fff0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.690+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/1843531335 shutdown_connections 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.690+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/1843531335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 0x562fbe3d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.690+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/1843531335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe3d0bf0 0x562fbe3db060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.690+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/1843531335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe33d300 0x562fbe3d4d60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.690+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/1843531335 >> 172.21.15.115:0/1843531335 conn(0x562fbe2b3840 msgr2=0x562fbe3d57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.690+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/1843531335 shutdown_connections 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.690+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/1843531335 wait complete. 2024-05-09T05:39:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c5ddbcb80 1 -- start start 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c5ddbcb80 1 Processor -- start 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 0x562fbe3e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe33d300 0x562fbe3dd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c5ddbcb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481b1760 con 0x562fbe3d0bf0 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481b1090 con 0x562fbe253640 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c5ddbcb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481ae290 con 0x562fbe33d300 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c5ddbcb80 10 client.0 _ll_register_callbacks cb 0x562fbe3d63f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c4d943700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c4d943700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3dde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40170/0 (socket says 172.21.15.115:40170) 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c4d943700 1 -- 172.21.15.115:0/2346123253 learned_addr learned my addr 172.21.15.115:0/2346123253 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.691+0000 7f2c4c941700 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe33d300 0x562fbe3dd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c4d142700 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 0x562fbe3e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c4d943700 1 -- 172.21.15.115:0/2346123253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe33d300 msgr2=0x562fbe3dd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c4d943700 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe33d300 0x562fbe3dd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c4d943700 1 -- 172.21.15.115:0/2346123253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 msgr2=0x562fbe3e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c4d943700 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 0x562fbe3e2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c4d943700 1 -- 172.21.15.115:0/2346123253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fbe3de5d0 con 0x562fbe3d0bf0 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c4d943700 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3dde00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2c40026980 tx=0x7f2c40026db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baabf84bf5c776b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2346123253 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40003a00 con 0x562fbe3d0bf0 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2346123253 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2c40003ba0 con 0x562fbe3d0bf0 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2346123253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c38036070 con 0x562fbe3d0bf0 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2346123253 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c40003da0 con 0x562fbe3d0bf0 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c5ddbcb80 10 client.15906 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2346123253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562fbe455f60 con 0x562fbe3d0bf0 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.692+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2346123253 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c38036070 con 0x562fbe3d0bf0 2024-05-09T05:39:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.693+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2346123253 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2c40011040 con 0x562fbe3d0bf0 2024-05-09T05:39:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.693+0000 7f2c5ddbcb80 10 client.15906 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.693+0000 7f2c5ddbcb80 10 client.15906 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:39:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.693+0000 7f2c5ddbcb80 20 client.15906 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.693+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562fbe456690 0x562fbe458ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.693+0000 7f2c5ddbcb80 4 client.15906 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}] 2024-05-09T05:39:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.693+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2346123253 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}) -- 0x562fbe2fac10 con 0x562fbe456690 2024-05-09T05:39:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.693+0000 7f2c4d142700 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562fbe456690 0x562fbe458ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.694+0000 7f2c30ff9700 10 client.15906 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.694+0000 7f2c4d142700 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562fbe456690 0x562fbe458ab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ed8dc51e391ac3 server_cookie=d0a5ac59971904e1 in_seq=0 out_seq=0 2024-05-09T05:39:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c30ff9700 1 -- 172.21.15.115:0/2346123253 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562fbe2fac10 con 0x562fbe456690 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c30ff9700 10 client.15906 handle_command_reply: tid=0 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 client.15906 shutdown 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2346123253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 msgr2=0x562fbe3dde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3dde00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2c40026980 tx=0x7f2c40026db0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2346123253 shutdown_connections 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fbe253640 0x562fbe3e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562fbe456690 0x562fbe458ab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562fbe3d0bf0 0x562fbe3dde00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 --2- 172.21.15.115:0/2346123253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562fbe33d300 0x562fbe3dd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c30ff9700 0 client.15906 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2346123253 >> 172.21.15.115:0/2346123253 conn(0x562fbe2b3840 msgr2=0x562fbe3dad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2346123253 shutdown_connections 2024-05-09T05:39:37.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.704+0000 7f2c5ddbcb80 1 -- 172.21.15.115:0/2346123253 wait complete. 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c5ddbcb80 20 client.15906 trim_cache size 0 max 16384 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 30, 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 2, 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stdout: "9c695dab": { 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 2, 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:39:37.700 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 144.9, 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:39:37.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:39:37.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:39:37.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:39:37.702 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:39:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c340b2b10 msgr2=0x7f2c340b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c340b2b10 0x7f2c340b5020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2c481022c0 tx=0x7f2c3800b410 comp rx=0 tx=0).stop 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c481054c0 msgr2=0x7f2c481a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c481054c0 0x7f2c481a08a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e9a0 tx=0x7f2c4400ecb0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/4057485033 shutdown_connections 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c340b2b10 0x7f2c340b5020 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c4810e8f0 0x7f2c4819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c481054c0 0x7f2c481a08a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 --2- 172.21.15.115:0/4057485033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c48105e10 0x7f2c481a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/4057485033 >> 172.21.15.115:0/4057485033 conn(0x7f2c480fedc0 msgr2=0x7f2c480ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/4057485033 shutdown_connections 2024-05-09T05:39:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:37.705+0000 7f2c4f3a6700 1 -- 172.21.15.115:0/4057485033 wait complete. 2024-05-09T05:39:38.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.152+0000 7f7be64dc700 1 -- 172.21.15.115:0/2892198335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be01027e0 msgr2=0x7f7be0102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:38.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.152+0000 7f7be64dc700 1 --2- 172.21.15.115:0/2892198335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be01027e0 0x7f7be0102bf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009990 tx=0x7f7bd4009ca0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.152+0000 7f7be64dc700 1 -- 172.21.15.115:0/2892198335 shutdown_connections 2024-05-09T05:39:38.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.152+0000 7f7be64dc700 1 --2- 172.21.15.115:0/2892198335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010a3e0 0x7f7be010a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.152+0000 7f7be64dc700 1 --2- 172.21.15.115:0/2892198335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7be0103130 0x7f7be0111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.152+0000 7f7be64dc700 1 --2- 172.21.15.115:0/2892198335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be01027e0 0x7f7be0102bf0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.152+0000 7f7be64dc700 1 -- 172.21.15.115:0/2892198335 >> 172.21.15.115:0/2892198335 conn(0x7f7be00fe6f0 msgr2=0x7f7be0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:38.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.152+0000 7f7be64dc700 1 -- 172.21.15.115:0/2892198335 shutdown_connections 2024-05-09T05:39:38.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.152+0000 7f7be64dc700 1 -- 172.21.15.115:0/2892198335 wait complete. 2024-05-09T05:39:38.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.153+0000 7f7be64dc700 1 Processor -- start 2024-05-09T05:39:38.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.153+0000 7f7be64dc700 1 -- start start 2024-05-09T05:39:38.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.153+0000 7f7be64dc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7be01027e0 0x7f7be01039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.153+0000 7f7be64dc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be0103130 0x7f7be0103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.153+0000 7f7be64dc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010a3e0 0x7f7be0104440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.153+0000 7f7be64dc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0114360 con 0x7f7be0103130 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.153+0000 7f7be64dc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01141e0 con 0x7f7be010a3e0 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.153+0000 7f7be64dc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01144e0 con 0x7f7be01027e0 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be0103130 0x7f7be0103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be0103130 0x7f7be0103f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40192/0 (socket says 172.21.15.115:40192) 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdf7fe700 1 -- 172.21.15.115:0/1079749805 learned_addr learned my addr 172.21.15.115:0/1079749805 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdffff700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7be01027e0 0x7f7be01039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7be4a79700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010a3e0 0x7f7be0104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdf7fe700 1 -- 172.21.15.115:0/1079749805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7be01027e0 msgr2=0x7f7be01039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdf7fe700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7be01027e0 0x7f7be01039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdf7fe700 1 -- 172.21.15.115:0/1079749805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010a3e0 msgr2=0x7f7be0104440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdf7fe700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010a3e0 0x7f7be0104440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdf7fe700 1 -- 172.21.15.115:0/1079749805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be01a9020 con 0x7f7be0103130 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7be4a79700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010a3e0 0x7f7be0104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdf7fe700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be0103130 0x7f7be0103f00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4006010 tx=0x7f7bd40057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdd7fa700 1 -- 172.21.15.115:0/1079749805 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4027070 con 0x7f7be0103130 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7bdd7fa700 1 -- 172.21.15.115:0/1079749805 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bd400fc00 con 0x7f7be0103130 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.154+0000 7f7be64dc700 1 -- 172.21.15.115:0/1079749805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be01a92b0 con 0x7f7be0103130 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.155+0000 7f7bdd7fa700 1 -- 172.21.15.115:0/1079749805 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4022c90 con 0x7f7be0103130 2024-05-09T05:39:38.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.155+0000 7f7be64dc700 1 -- 172.21.15.115:0/1079749805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be01a9700 con 0x7f7be0103130 2024-05-09T05:39:38.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.157+0000 7f7bdd7fa700 1 -- 172.21.15.115:0/1079749805 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7f7bd402a070 con 0x7f7be0103130 2024-05-09T05:39:38.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.158+0000 7f7bdd7fa700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7bc80b2b10 0x7f7bc80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.158+0000 7f7bdffff700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7bc80b2b10 0x7f7bc80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.158+0000 7f7bdd7fa700 1 -- 172.21.15.115:0/1079749805 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7bd40e3c30 con 0x7f7be0103130 2024-05-09T05:39:38.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.158+0000 7f7bdffff700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7bc80b2b10 0x7f7bc80b5020 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000abd0 tx=0x7f7bd000a380 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:38.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bf4ef2b80 1 Processor -- start 2024-05-09T05:39:38.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bf4ef2b80 1 -- start start 2024-05-09T05:39:38.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x5600326794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600324f64a0 0x560032679a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600326736a0 0x560032673ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560032595ff0 con 0x5600324f64a0 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5600325347f0 con 0x5600325e0160 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560032591fb0 con 0x5600326736a0 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bdf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600324f64a0 0x560032679a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bdf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600324f64a0 0x560032679a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40194/0 (socket says 172.21.15.115:40194) 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bdf7fe700 1 -- 172.21.15.115:0/671564226 learned_addr learned my addr 172.21.15.115:0/671564226 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7be4a79700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600326736a0 0x560032673ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.160+0000 7f7bdffff700 1 --2- 172.21.15.115:0/671564226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x5600326794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7be4a79700 1 -- 172.21.15.115:0/671564226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 msgr2=0x5600326794f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7be4a79700 1 --2- 172.21.15.115:0/671564226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x5600326794f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7be4a79700 1 -- 172.21.15.115:0/671564226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600324f64a0 msgr2=0x560032679a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7be4a79700 1 --2- 172.21.15.115:0/671564226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600324f64a0 0x560032679a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7be4a79700 1 -- 172.21.15.115:0/671564226 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003252c370 con 0x5600326736a0 2024-05-09T05:39:38.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7bdf7fe700 1 --2- 172.21.15.115:0/671564226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600324f64a0 0x560032679a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:38.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7be4a79700 1 --2- 172.21.15.115:0/671564226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600326736a0 0x560032673ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8007fd0 tx=0x7f7bd800e440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d822b6e93da5ca30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:38.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7bc67fc700 1 -- 172.21.15.115:0/671564226 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd800ecd0 con 0x5600326736a0 2024-05-09T05:39:38.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7bc67fc700 1 -- 172.21.15.115:0/671564226 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bd800ee70 con 0x5600326736a0 2024-05-09T05:39:38.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.161+0000 7f7bc67fc700 1 -- 172.21.15.115:0/671564226 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8014da0 con 0x5600326736a0 2024-05-09T05:39:38.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.162+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/671564226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600326736a0 msgr2=0x560032673ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:38.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.162+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/671564226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600326736a0 0x560032673ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8007fd0 tx=0x7f7bd800e440 comp rx=0 tx=0).stop 2024-05-09T05:39:38.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.162+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/671564226 shutdown_connections 2024-05-09T05:39:38.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.162+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/671564226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x5600326794f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.162+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/671564226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600326736a0 0x560032673ad0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.162+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/671564226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600324f64a0 0x560032679a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.162+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/671564226 >> 172.21.15.115:0/671564226 conn(0x5600325566a0 msgr2=0x56003266e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.162+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/671564226 shutdown_connections 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.162+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/671564226 wait complete. 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.163+0000 7f7bf4ef2b80 1 -- start start 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.163+0000 7f7bf4ef2b80 1 Processor -- start 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.163+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x56003267d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.163+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x56003267d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.163+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x56003267dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.163+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560032411ba0 con 0x5600326736a0 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.163+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560032550150 con 0x5600325e0160 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.163+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5600326801d0 con 0x5600324f64a0 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.163+0000 7f7bf4ef2b80 10 client.0 _ll_register_callbacks cb 0x5600324076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7be4a79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x56003267dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7bdffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x56003267d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7bdffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x56003267d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:37602/0 (socket says 172.21.15.115:37602) 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7be4a79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x56003267dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40206/0 (socket says 172.21.15.115:40206) 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7bdffff700 1 -- 172.21.15.115:0/203092142 learned_addr learned my addr 172.21.15.115:0/203092142 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7bdf7fe700 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x56003267d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7be4a79700 1 -- 172.21.15.115:0/203092142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 msgr2=0x56003267d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7be4a79700 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x56003267d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7be4a79700 1 -- 172.21.15.115:0/203092142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 msgr2=0x56003267d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7be4a79700 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x56003267d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7be4a79700 1 -- 172.21.15.115:0/203092142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560032411490 con 0x5600326736a0 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7bdffff700 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x56003267d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:38.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7be4a79700 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x56003267dc30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8007fd0 tx=0x7f7bd8015de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d02e603deb952f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7bbf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/203092142 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8022030 con 0x5600326736a0 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/203092142 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7bd8010070 con 0x5600326736a0 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.164+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/203092142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bd800ee70 con 0x5600326736a0 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/203092142 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bd802f050 con 0x5600326736a0 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bf4ef2b80 10 client.15918 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/203092142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56003268ab90 con 0x5600326736a0 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/203092142 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bd800ee70 con 0x5600326736a0 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/203092142 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7bd801e7c0 con 0x5600326736a0 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bf4ef2b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bf4ef2b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bf4ef2b80 20 client.15918 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:38.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56003268d060 0x56003268f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bf4ef2b80 4 client.15918 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:38.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/203092142 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600325a39d0 con 0x56003268d060 2024-05-09T05:39:38.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.165+0000 7f7bdffff700 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56003268d060 0x56003268f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.166+0000 7f7bdffff700 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56003268d060 0x56003268f480 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=585f6fece1de02c1 server_cookie=7173c1fc07bbc376 in_seq=0 out_seq=0 2024-05-09T05:39:38.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.166+0000 7f7bbf7fe700 10 client.15918 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:38.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/203092142 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600325a39d0 con 0x56003268d060 2024-05-09T05:39:38.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bbf7fe700 10 client.15918 handle_command_reply: tid=0 2024-05-09T05:39:38.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 client.15918 shutdown 2024-05-09T05:39:38.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/203092142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 msgr2=0x56003267dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x56003267dc30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8007fd0 tx=0x7f7bd8015de0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/203092142 shutdown_connections 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56003268d060 0x56003268f480 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x56003267dc30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x56003267d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/203092142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x56003267d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/203092142 >> 172.21.15.115:0/203092142 conn(0x5600325566a0 msgr2=0x56003266e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bbf7fe700 0 client.15918 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/203092142 shutdown_connections 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/203092142 wait complete. 2024-05-09T05:39:38.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.167+0000 7f7bf4ef2b80 20 client.15918 trim_cache size 0 max 16384 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bf4ef2b80 1 Processor -- start 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bf4ef2b80 1 -- start start 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x5600326893a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x56003267c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032677510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01b2360 con 0x5600326736a0 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01b3ea0 con 0x5600325e0160 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01b2a30 con 0x5600324f64a0 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7be4a79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032677510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7be4a79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032677510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40222/0 (socket says 172.21.15.115:40222) 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7be4a79700 1 -- 172.21.15.115:0/3940925033 learned_addr learned my addr 172.21.15.115:0/3940925033 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bdffff700 1 --2- 172.21.15.115:0/3940925033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x5600326893a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.188+0000 7f7bdf7fe700 1 --2- 172.21.15.115:0/3940925033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x56003267c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.189+0000 7f7be4a79700 1 -- 172.21.15.115:0/3940925033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 msgr2=0x5600326893a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:38.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.189+0000 7f7be4a79700 1 --2- 172.21.15.115:0/3940925033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x5600326893a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.189+0000 7f7be4a79700 1 -- 172.21.15.115:0/3940925033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 msgr2=0x56003267c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:38.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.189+0000 7f7be4a79700 1 --2- 172.21.15.115:0/3940925033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x56003267c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.189+0000 7f7be4a79700 1 -- 172.21.15.115:0/3940925033 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560032679050 con 0x5600326736a0 2024-05-09T05:39:38.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.189+0000 7f7be4a79700 1 --2- 172.21.15.115:0/3940925033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032677510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8002230 tx=0x7f7bd8007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3a6c548c544843 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:38.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.189+0000 7f7bc5ffb700 1 -- 172.21.15.115:0/3940925033 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8022070 con 0x5600326736a0 2024-05-09T05:39:38.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.189+0000 7f7bc5ffb700 1 -- 172.21.15.115:0/3940925033 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bd800edb0 con 0x5600326736a0 2024-05-09T05:39:38.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.189+0000 7f7bc5ffb700 1 -- 172.21.15.115:0/3940925033 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd801e680 con 0x5600326736a0 2024-05-09T05:39:38.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.190+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/3940925033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 msgr2=0x560032677510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:38.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.190+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/3940925033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032677510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8002230 tx=0x7f7bd8007a30 comp rx=0 tx=0).stop 2024-05-09T05:39:38.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.190+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/3940925033 shutdown_connections 2024-05-09T05:39:38.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.190+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/3940925033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032677510 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.190+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/3940925033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600324f64a0 0x5600326893a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.190+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/3940925033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600325e0160 0x56003267c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.190+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/3940925033 >> 172.21.15.115:0/3940925033 conn(0x5600325566a0 msgr2=0x560032676e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.190+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/3940925033 shutdown_connections 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.190+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/3940925033 wait complete. 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.191+0000 7f7bf4ef2b80 1 -- start start 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.191+0000 7f7bf4ef2b80 1 Processor -- start 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.191+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600324f64a0 0x5600326926f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.191+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600325e0160 0x56003268fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7bf4ef2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032690070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01b37d0 con 0x5600326736a0 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01b3100 con 0x5600324f64a0 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7bf4ef2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01b0300 con 0x5600325e0160 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7bf4ef2b80 10 client.0 _ll_register_callbacks cb 0x5600324076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:38.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7be4a79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032690070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7be4a79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032690070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40230/0 (socket says 172.21.15.115:40230) 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7be4a79700 1 -- 172.21.15.115:0/4235604670 learned_addr learned my addr 172.21.15.115:0/4235604670 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7bdffff700 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600324f64a0 0x5600326926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7bdf7fe700 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600325e0160 0x56003268fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7be4a79700 1 -- 172.21.15.115:0/4235604670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600325e0160 msgr2=0x56003268fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7be4a79700 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600325e0160 0x56003268fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7be4a79700 1 -- 172.21.15.115:0/4235604670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600324f64a0 msgr2=0x5600326926f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7be4a79700 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600324f64a0 0x5600326926f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7be4a79700 1 -- 172.21.15.115:0/4235604670 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560032690860 con 0x5600326736a0 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7be4a79700 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032690070 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7bd802c040 tx=0x7f7bd8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4b76832b769fb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7bbf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.192+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/4235604670 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8038030 con 0x5600326736a0 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/4235604670 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bd800edb0 con 0x5600326736a0 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/4235604670 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7bd8010050 con 0x5600326736a0 2024-05-09T05:39:38.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/4235604670 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bd8045050 con 0x5600326736a0 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bf4ef2b80 10 client.15924 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/4235604670 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600326f75a0 con 0x5600326736a0 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/4235604670 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bd800edb0 con 0x5600326736a0 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/4235604670 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7bd803f4d0 con 0x5600326736a0 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bf4ef2b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bf4ef2b80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bf4ef2b80 20 client.15924 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5600326f7b30 0x5600326f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bf4ef2b80 4 client.15924 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.193+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/4235604670 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56003259da70 con 0x5600326f7b30 2024-05-09T05:39:38.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bdffff700 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5600326f7b30 0x5600326f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:38.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bbf7fe700 10 client.15924 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bdffff700 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5600326f7b30 0x5600326f9f50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec6c318ab4a3f62e server_cookie=e41d8ce60fa8289c in_seq=0 out_seq=0 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bbf7fe700 1 -- 172.21.15.115:0/4235604670 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56003259da70 con 0x5600326f7b30 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bbf7fe700 10 client.15924 handle_command_reply: tid=0 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bf4ef2b80 1 client.15924 shutdown 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/4235604670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 msgr2=0x560032690070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032690070 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7bd802c040 tx=0x7f7bd8005f10 comp rx=0 tx=0).stop 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/4235604670 shutdown_connections 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600324f64a0 0x5600326926f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5600326f7b30 0x5600326f9f50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5600326736a0 0x560032690070 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bf4ef2b80 1 --2- 172.21.15.115:0/4235604670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600325e0160 0x56003268fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/4235604670 >> 172.21.15.115:0/4235604670 conn(0x5600325566a0 msgr2=0x560032678940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.194+0000 7f7bbf7fe700 0 client.15924 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/4235604670 shutdown_connections 2024-05-09T05:39:38.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7bf4ef2b80 1 -- 172.21.15.115:0/4235604670 wait complete. 2024-05-09T05:39:38.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7bf4ef2b80 20 client.15924 trim_cache size 0 max 16384 2024-05-09T05:39:38.193 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:39:38.193 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:39:38.193 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:39:38.193 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 -- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7bc80b2b10 msgr2=0x7f7bc80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7bc80b2b10 0x7f7bc80b5020 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000abd0 tx=0x7f7bd000a380 comp rx=0 tx=0).stop 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 -- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be0103130 msgr2=0x7f7be0103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be0103130 0x7f7be0103f00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4006010 tx=0x7f7bd40057a0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 -- 172.21.15.115:0/1079749805 shutdown_connections 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7bc80b2b10 0x7f7bc80b5020 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be010a3e0 0x7f7be0104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7be01027e0 0x7f7be01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 --2- 172.21.15.115:0/1079749805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7be0103130 0x7f7be0103f00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:38.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.195+0000 7f7be64dc700 1 -- 172.21.15.115:0/1079749805 >> 172.21.15.115:0/1079749805 conn(0x7f7be00fe6f0 msgr2=0x7f7be0100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:38.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.196+0000 7f7be64dc700 1 -- 172.21.15.115:0/1079749805 shutdown_connections 2024-05-09T05:39:38.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:38.196+0000 7f7be64dc700 1 -- 172.21.15.115:0/1079749805 wait complete. 2024-05-09T05:39:38.867 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9c695dab', quiesce took 11.6 seconds. Will release after: 13.35 2024-05-09T05:39:39.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:39 smithi115 ceph-mon[30349]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 14 op/s 2024-05-09T05:39:39.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:39 smithi195 ceph-mon[33844]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 14 op/s 2024-05-09T05:39:39.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:39 smithi161 ceph-mon[33805]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 14 op/s 2024-05-09T05:39:39.695 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:39:39.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:39:39.964 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:41.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:41 smithi115 ceph-mon[30349]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T05:39:41.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:41 smithi195 ceph-mon[33844]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T05:39:41.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:41 smithi161 ceph-mon[33805]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T05:39:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.764+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/1411492790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8107df0 msgr2=0x7fcfd8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.764+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/1411492790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8107df0 0x7fcfd8108200 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80099c0 tx=0x7fcfc8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:39:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.764+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/1411492790 shutdown_connections 2024-05-09T05:39:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.764+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/1411492790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfd8108740 0x7fcfd81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.764+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/1411492790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfd81034c0 0x7fcfd81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.764+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/1411492790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8107df0 0x7fcfd8108200 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.764+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/1411492790 >> 172.21.15.115:0/1411492790 conn(0x7fcfd80fedc0 msgr2=0x7fcfd81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.764+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/1411492790 shutdown_connections 2024-05-09T05:39:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.764+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/1411492790 wait complete. 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfdfdcd700 1 Processor -- start 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfdfdcd700 1 -- start start 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfdfdcd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfd81034c0 0x7fcfd8199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfdfdcd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfd8107df0 0x7fcfd819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfdfdcd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8108740 0x7fcfd819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfdfdcd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd8113200 con 0x7fcfd8108740 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfdfdcd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd8113080 con 0x7fcfd81034c0 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfdfdcd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd8113380 con 0x7fcfd8107df0 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfde36a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8108740 0x7fcfd819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.765+0000 7fcfde36a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8108740 0x7fcfd819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40260/0 (socket says 172.21.15.115:40260) 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfde36a700 1 -- 172.21.15.115:0/160372196 learned_addr learned my addr 172.21.15.115:0/160372196 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfddb69700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfd81034c0 0x7fcfd8199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfdd368700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfd8107df0 0x7fcfd819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:41.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfde36a700 1 -- 172.21.15.115:0/160372196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfd8107df0 msgr2=0x7fcfd819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfde36a700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfd8107df0 0x7fcfd819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfde36a700 1 -- 172.21.15.115:0/160372196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfd81034c0 msgr2=0x7fcfd8199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfde36a700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfd81034c0 0x7fcfd8199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfde36a700 1 -- 172.21.15.115:0/160372196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd819ebe0 con 0x7fcfd8108740 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfde36a700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8108740 0x7fcfd819e540 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000ada0 tx=0x7fcfd000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfceffd700 1 -- 172.21.15.115:0/160372196 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0017020 con 0x7fcfd8108740 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfceffd700 1 -- 172.21.15.115:0/160372196 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfd000cd60 con 0x7fcfd8108740 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.766+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd819eed0 con 0x7fcfd8108740 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.767+0000 7fcfceffd700 1 -- 172.21.15.115:0/160372196 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0005280 con 0x7fcfd8108740 2024-05-09T05:39:41.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.767+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd81a67f0 con 0x7fcfd8108740 2024-05-09T05:39:41.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.768+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd80512c0 con 0x7fcfd8108740 2024-05-09T05:39:41.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.769+0000 7fcfceffd700 1 -- 172.21.15.115:0/160372196 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) ==== 149886+0+0 (secure 0 0 0) 0x7fcfd000c8d0 con 0x7fcfd8108740 2024-05-09T05:39:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.770+0000 7fcfceffd700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfc40b2b90 0x7fcfc40b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.770+0000 7fcfceffd700 1 -- 172.21.15.115:0/160372196 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fcfd00e2880 con 0x7fcfd8108740 2024-05-09T05:39:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.770+0000 7fcfddb69700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfc40b2b90 0x7fcfc40b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.771+0000 7fcfddb69700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfc40b2b90 0x7fcfc40b50a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4005950 tx=0x7fcfd400b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:41.775+0000 7fcfceffd700 1 -- 172.21.15.115:0/160372196 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcfd0021020 con 0x7fcfd8108740 2024-05-09T05:39:42.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.060+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfd802e5f0 con 0x7fcfd8108740 2024-05-09T05:39:42.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.061+0000 7fcfceffd700 1 -- 172.21.15.115:0/160372196 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7fcfd00a9d60 con 0x7fcfd8108740 2024-05-09T05:39:42.055 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:39:42.055 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:39:42.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfc40b2b90 msgr2=0x7fcfc40b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:42.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfc40b2b90 0x7fcfc40b50a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4005950 tx=0x7fcfd400b410 comp rx=0 tx=0).stop 2024-05-09T05:39:42.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8108740 msgr2=0x7fcfd819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:42.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8108740 0x7fcfd819e540 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000ada0 tx=0x7fcfd000c330 comp rx=0 tx=0).stop 2024-05-09T05:39:42.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 shutdown_connections 2024-05-09T05:39:42.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfc40b2b90 0x7fcfc40b50a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:42.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfd81034c0 0x7fcfd8199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:42.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfd8108740 0x7fcfd819e540 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:42.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 --2- 172.21.15.115:0/160372196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfd8107df0 0x7fcfd819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:42.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 >> 172.21.15.115:0/160372196 conn(0x7fcfd80fedc0 msgr2=0x7fcfd81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:42.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 shutdown_connections 2024-05-09T05:39:42.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:42.064+0000 7fcfdfdcd700 1 -- 172.21.15.115:0/160372196 wait complete. 2024-05-09T05:39:42.059 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:39:42.790 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:39:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:43.310 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:43 smithi115 ceph-mon[30349]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T05:39:43.310 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:43 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/160372196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:43.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:43 smithi195 ceph-mon[33844]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T05:39:43.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:43 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/160372196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:43.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:43 smithi161 ceph-mon[33805]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-09T05:39:43.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:43 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/160372196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:44.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.823+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/2068804055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 msgr2=0x7fc6ac105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.823+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/2068804055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 0x7fc6ac105890 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009990 tx=0x7fc69c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.823+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/2068804055 shutdown_connections 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.823+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/2068804055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6ac10e8f0 0x7fc6ac110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.824+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/2068804055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 0x7fc6ac105890 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.824+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/2068804055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac105dd0 0x7fc6ac10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.824+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/2068804055 >> 172.21.15.115:0/2068804055 conn(0x7fc6ac0fef30 msgr2=0x7fc6ac101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.824+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/2068804055 shutdown_connections 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.824+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/2068804055 wait complete. 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6b0fc3700 1 Processor -- start 2024-05-09T05:39:44.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6b0fc3700 1 -- start start 2024-05-09T05:39:44.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6b0fc3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 0x7fc6ac1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6b0fc3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac105dd0 0x7fc6ac199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6b0fc3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6ac10e8f0 0x7fc6ac1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 0x7fc6ac1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 0x7fc6ac1997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50472/0 (socket says 172.21.15.115:50472) 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac1142f0 con 0x7fc6ac105480 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac114170 con 0x7fc6ac10e8f0 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac114470 con 0x7fc6ac105dd0 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6aa59c700 1 -- 172.21.15.115:0/54665487 learned_addr learned my addr 172.21.15.115:0/54665487 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.825+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac105dd0 0x7fc6ac199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc6aad9d700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6ac10e8f0 0x7fc6ac1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc6a9d9b700 1 -- 172.21.15.115:0/54665487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6ac10e8f0 msgr2=0x7fc6ac1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6ac10e8f0 0x7fc6ac1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc6a9d9b700 1 -- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 msgr2=0x7fc6ac1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:44.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 0x7fc6ac1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc6a9d9b700 1 -- 172.21.15.115:0/54665487 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac1a0d60 con 0x7fc6ac105dd0 2024-05-09T05:39:44.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 0x7fc6ac1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:44.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac105dd0 0x7fc6ac199d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000e9f0 tx=0x7fc6a000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:44.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc69b7fe700 1 -- 172.21.15.115:0/54665487 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a000c9c0 con 0x7fc6ac105dd0 2024-05-09T05:39:44.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.826+0000 7fc69b7fe700 1 -- 172.21.15.115:0/54665487 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6a0010430 con 0x7fc6ac105dd0 2024-05-09T05:39:44.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.827+0000 7fc69b7fe700 1 -- 172.21.15.115:0/54665487 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0018680 con 0x7fc6ac105dd0 2024-05-09T05:39:44.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.827+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac1a1050 con 0x7fc6ac105dd0 2024-05-09T05:39:44.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.827+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac1a1540 con 0x7fc6ac105dd0 2024-05-09T05:39:44.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.830+0000 7fc69b7fe700 1 -- 172.21.15.115:0/54665487 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc6a000cb60 con 0x7fc6ac105dd0 2024-05-09T05:39:44.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.831+0000 7fc69b7fe700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc6940b29f0 0x7fc6940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.831+0000 7fc69b7fe700 1 -- 172.21.15.115:0/54665487 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc6a00e2a70 con 0x7fc6ac105dd0 2024-05-09T05:39:44.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.831+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc6940b29f0 0x7fc6940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.832+0000 7fc6bf9d9b80 1 Processor -- start 2024-05-09T05:39:44.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.832+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc6940b29f0 0x7fc6940b4f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac10e830 tx=0x7fc69c005960 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:44.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.832+0000 7fc6bf9d9b80 1 -- start start 2024-05-09T05:39:44.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.832+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865de8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865de26a0 0x556865de2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6a9d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865de8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6a9d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865de8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50474/0 (socket says 172.21.15.115:50474) 2024-05-09T05:39:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33278/0 (socket says 172.21.15.115:33278) 2024-05-09T05:39:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6a9d9b700 1 -- 172.21.15.115:0/3277513237 learned_addr learned my addr 172.21.15.115:0/3277513237 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/3277513237 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556865d04ff0 con 0x556865c654a0 2024-05-09T05:39:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/3277513237 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556865ca37f0 con 0x556865d4f160 2024-05-09T05:39:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/3277513237 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556865d00fb0 con 0x556865de26a0 2024-05-09T05:39:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.833+0000 7fc6aad9d700 1 --2- 172.21.15.115:0/3277513237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865de26a0 0x556865de2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.834+0000 7fc6a9d9b700 1 -- 172.21.15.115:0/3277513237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865de26a0 msgr2=0x556865de2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.834+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/3277513237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865de26a0 0x556865de2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.834+0000 7fc6a9d9b700 1 -- 172.21.15.115:0/3277513237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 msgr2=0x556865de84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.834+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/3277513237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.834+0000 7fc6a9d9b700 1 -- 172.21.15.115:0/3277513237 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556865c9b370 con 0x556865c654a0 2024-05-09T05:39:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.834+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/3277513237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865de8a30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00a5a90 tx=0x7fc6a00b21b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bfced8664d5aad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.834+0000 7fc698ff9700 1 -- 172.21.15.115:0/3277513237 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a00bee60 con 0x556865c654a0 2024-05-09T05:39:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.834+0000 7fc698ff9700 1 -- 172.21.15.115:0/3277513237 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6a00aa110 con 0x556865c654a0 2024-05-09T05:39:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.834+0000 7fc698ff9700 1 -- 172.21.15.115:0/3277513237 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a00a9c70 con 0x556865c654a0 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.835+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/3277513237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 msgr2=0x556865de8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.836+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/3277513237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865de8a30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00a5a90 tx=0x7fc6a00b21b0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.836+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/3277513237 shutdown_connections 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.836+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/3277513237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.836+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/3277513237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865de26a0 0x556865de2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.836+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/3277513237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865de8a30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.836+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/3277513237 >> 172.21.15.115:0/3277513237 conn(0x556865cc56a0 msgr2=0x556865ddd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.836+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/3277513237 shutdown_connections 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.836+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/3277513237 wait complete. 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.837+0000 7fc6bf9d9b80 1 -- start start 2024-05-09T05:39:44.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.837+0000 7fc6bf9d9b80 1 Processor -- start 2024-05-09T05:39:44.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865dec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865d4f160 0x556865dec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865de26a0 0x556865decd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865dec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865dec2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50486/0 (socket says 172.21.15.115:50486) 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6aa59c700 1 -- 172.21.15.115:0/2475262771 learned_addr learned my addr 172.21.15.115:0/2475262771 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556865b80ba0 con 0x556865c654a0 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556865cbf150 con 0x556865de26a0 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556865def1d0 con 0x556865d4f160 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865d4f160 0x556865dec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.838+0000 7fc6aad9d700 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865de26a0 0x556865decd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc6bf9d9b80 10 client.0 _ll_register_callbacks cb 0x556865b766c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc6aa59c700 1 -- 172.21.15.115:0/2475262771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865d4f160 msgr2=0x556865dec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865d4f160 0x556865dec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc6aa59c700 1 -- 172.21.15.115:0/2475262771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865de26a0 msgr2=0x556865decd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:44.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865de26a0 0x556865decd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc6aa59c700 1 -- 172.21.15.115:0/2475262771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556865b80490 con 0x556865c654a0 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc6aad9d700 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865de26a0 0x556865decd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a00aa110 con 0x556865c654a0 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865dec2c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000c00 tx=0x7fc69c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655a2abfeff24bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc691ffb700 10 client.15942 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc691ffb700 1 -- 172.21.15.115:0/2475262771 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c004a30 con 0x556865c654a0 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.839+0000 7fc691ffb700 1 -- 172.21.15.115:0/2475262771 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc69c004bd0 con 0x556865c654a0 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc691ffb700 1 -- 172.21.15.115:0/2475262771 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc69c004e30 con 0x556865c654a0 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc6bf9d9b80 10 client.15942 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:44.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=30}) -- 0x556865df9910 con 0x556865c654a0 2024-05-09T05:39:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc691ffb700 1 -- 172.21.15.115:0/2475262771 <== mon.0 v2:172.21.15.115:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc69c005020 con 0x556865c654a0 2024-05-09T05:39:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc6bf9d9b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc6bf9d9b80 10 client.15942 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:39:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc6bf9d9b80 20 client.15942 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556865dfbe40 0x556865dfe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc6bf9d9b80 4 client.15942 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.840+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556865d129d0 con 0x556865dfbe40 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.841+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556865dfbe40 0x556865dfe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.841+0000 7fc691ffb700 10 client.15942 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.841+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556865dfbe40 0x556865dfe2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eac0046c1dc84e26 server_cookie=63e94b95431ec6cc in_seq=0 out_seq=0 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc691ffb700 1 -- 172.21.15.115:0/2475262771 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556865d129d0 con 0x556865dfbe40 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc691ffb700 10 client.15942 handle_command_reply: tid=0 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 client.15942 shutdown 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 msgr2=0x556865dec2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865dec2c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000c00 tx=0x7fc69c01e000 comp rx=0 tx=0).stop 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 shutdown_connections 2024-05-09T05:39:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865de26a0 0x556865decd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556865dfbe40 0x556865dfe2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865dec2c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc691ffb700 0 client.15942 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:39:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2475262771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865d4f160 0x556865dec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 >> 172.21.15.115:0/2475262771 conn(0x556865cc56a0 msgr2=0x556865ddd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 shutdown_connections 2024-05-09T05:39:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.842+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2475262771 wait complete. 2024-05-09T05:39:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.843+0000 7fc6bf9d9b80 20 client.15942 trim_cache size 0 max 16384 2024-05-09T05:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.858+0000 7fc6bf9d9b80 1 Processor -- start 2024-05-09T05:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.858+0000 7fc6bf9d9b80 1 -- start start 2024-05-09T05:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.858+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865cc56a0 0x556865deb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.858+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865c654a0 0x556865df8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.858+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.858+0000 7fc6bf9d9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac1bf2c0 con 0x556865cc56a0 2024-05-09T05:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.858+0000 7fc6bf9d9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac1c0e00 con 0x556865d4f160 2024-05-09T05:39:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6bf9d9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac1bf990 con 0x556865c654a0 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865cc56a0 0x556865deb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865cc56a0 0x556865deb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50498/0 (socket says 172.21.15.115:50498) 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aa59c700 1 -- 172.21.15.115:0/43703964 learned_addr learned my addr 172.21.15.115:0/43703964 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865c654a0 0x556865df8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aad9d700 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aa59c700 1 -- 172.21.15.115:0/43703964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865c654a0 msgr2=0x556865df8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865c654a0 0x556865df8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aa59c700 1 -- 172.21.15.115:0/43703964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 msgr2=0x556865de5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aa59c700 1 -- 172.21.15.115:0/43703964 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556865dd6f60 con 0x556865cc56a0 2024-05-09T05:39:44.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aad9d700 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:44.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865cc56a0 0x556865deb190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0098e0 tx=0x7fc69c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c1b36932d045c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:44.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc693fff700 1 -- 172.21.15.115:0/43703964 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c004fd0 con 0x556865cc56a0 2024-05-09T05:39:44.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc693fff700 1 -- 172.21.15.115:0/43703964 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6a00aa110 con 0x556865cc56a0 2024-05-09T05:39:44.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.859+0000 7fc693fff700 1 -- 172.21.15.115:0/43703964 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c022b60 con 0x556865cc56a0 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/43703964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865cc56a0 msgr2=0x556865deb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865cc56a0 0x556865deb190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0098e0 tx=0x7fc69c021000 comp rx=0 tx=0).stop 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/43703964 shutdown_connections 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865de5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865cc56a0 0x556865deb190 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/43703964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865c654a0 0x556865df8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/43703964 >> 172.21.15.115:0/43703964 conn(0x556865de2e10 msgr2=0x556865deaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/43703964 shutdown_connections 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/43703964 wait complete. 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 -- start start 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.861+0000 7fc6bf9d9b80 1 Processor -- start 2024-05-09T05:39:44.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865e033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865cc56a0 0x556865e038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6bf9d9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865e03e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6bf9d9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac1c0730 con 0x556865c654a0 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6bf9d9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac1c0060 con 0x556865d4f160 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6bf9d9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac1bd260 con 0x556865cc56a0 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6bf9d9b80 10 client.0 _ll_register_callbacks cb 0x556865b766c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865e033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6aad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865e03e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6aa59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865e033b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50500/0 (socket says 172.21.15.115:50500) 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6aa59c700 1 -- 172.21.15.115:0/2621960765 learned_addr learned my addr 172.21.15.115:0/2621960765 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865cc56a0 0x556865e038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6aa59c700 1 -- 172.21.15.115:0/2621960765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865cc56a0 msgr2=0x556865e038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.862+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865cc56a0 0x556865e038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc6aa59c700 1 -- 172.21.15.115:0/2621960765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 msgr2=0x556865e03e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:44.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865e03e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc6aa59c700 1 -- 172.21.15.115:0/2621960765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556865e656b0 con 0x556865c654a0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865cc56a0 0x556865e038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc6aad9d700 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865e03e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc6aa59c700 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865e033b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000c00 tx=0x7fc69c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bd26b2bf7e94e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc691ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc691ffb700 1 -- 172.21.15.115:0/2621960765 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c004a40 con 0x556865c654a0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc691ffb700 1 -- 172.21.15.115:0/2621960765 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc69c03f040 con 0x556865c654a0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2621960765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a00aa110 con 0x556865c654a0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc691ffb700 1 -- 172.21.15.115:0/2621960765 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc69c004be0 con 0x556865c654a0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc6bf9d9b80 10 client.15948 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.863+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2621960765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556865e667c0 con 0x556865c654a0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.864+0000 7fc691ffb700 1 -- 172.21.15.115:0/2621960765 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc69c022c50 con 0x556865c654a0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.864+0000 7fc691ffb700 1 -- 172.21.15.115:0/2621960765 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc69c039a90 con 0x556865c654a0 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.864+0000 7fc6bf9d9b80 10 client.15948 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:44.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.864+0000 7fc6bf9d9b80 10 client.15948 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:39:44.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.864+0000 7fc6bf9d9b80 20 client.15948 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:44.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.864+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556865de26a0 0x556865e68c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:44.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.864+0000 7fc6bf9d9b80 4 client.15948 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:44.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.864+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2621960765 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc6a00aa110 con 0x556865de26a0 2024-05-09T05:39:44.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.864+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556865de26a0 0x556865e68c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:44.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.865+0000 7fc691ffb700 10 client.15948 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:39:44.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.865+0000 7fc6a9d9b700 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556865de26a0 0x556865e68c90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a490f6f0c0472903 server_cookie=9455b48dbc2d3ebf in_seq=0 out_seq=0 2024-05-09T05:39:44.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.865+0000 7fc691ffb700 1 -- 172.21.15.115:0/2621960765 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc6a00aa110 con 0x556865de26a0 2024-05-09T05:39:44.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.865+0000 7fc691ffb700 10 client.15948 handle_command_reply: tid=0 2024-05-09T05:39:44.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.865+0000 7fc6bf9d9b80 1 client.15948 shutdown 2024-05-09T05:39:44.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2621960765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 msgr2=0x556865e033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:44.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865e033b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000c00 tx=0x7fc69c005050 comp rx=0 tx=0).stop 2024-05-09T05:39:44.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2621960765 shutdown_connections 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556865d4f160 0x556865e03e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x556865de26a0 0x556865e68c90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556865c654a0 0x556865e033b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 --2- 172.21.15.115:0/2621960765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556865cc56a0 0x556865e038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2621960765 >> 172.21.15.115:0/2621960765 conn(0x556865de2e10 msgr2=0x556865de5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc691ffb700 0 client.15948 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2621960765 shutdown_connections 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 1 -- 172.21.15.115:0/2621960765 wait complete. 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.866+0000 7fc6bf9d9b80 20 client.15948 trim_cache size 0 max 16384 2024-05-09T05:39:44.864 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:44.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc6940b29f0 msgr2=0x7fc6940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:44.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc6940b29f0 0x7fc6940b4f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac10e830 tx=0x7fc69c005960 comp rx=0 tx=0).stop 2024-05-09T05:39:44.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac105dd0 msgr2=0x7fc6ac199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:44.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac105dd0 0x7fc6ac199d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000e9f0 tx=0x7fc6a000ed00 comp rx=0 tx=0).stop 2024-05-09T05:39:44.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 shutdown_connections 2024-05-09T05:39:44.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc6940b29f0 0x7fc6940b4f00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6ac10e8f0 0x7fc6ac1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6ac105480 0x7fc6ac1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 --2- 172.21.15.115:0/54665487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6ac105dd0 0x7fc6ac199d60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:44.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 >> 172.21.15.115:0/54665487 conn(0x7fc6ac0fef30 msgr2=0x7fc6ac1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:44.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 shutdown_connections 2024-05-09T05:39:44.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:44.867+0000 7fc6b0fc3700 1 -- 172.21.15.115:0/54665487 wait complete. 2024-05-09T05:39:45.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:45 smithi115 ceph-mon[30349]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-05-09T05:39:45.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:45 smithi115 ceph-mon[30349]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-05-09T05:39:45.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:45 smithi195 ceph-mon[33844]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-05-09T05:39:45.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:45 smithi195 ceph-mon[33844]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-05-09T05:39:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:45 smithi161 ceph-mon[33805]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-05-09T05:39:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:45 smithi161 ceph-mon[33805]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-05-09T05:39:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.539+0000 7f9b8d29a640 1 -- 172.21.15.115:0/1857311300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88109040 msgr2=0x7f9b881032e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.539+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/1857311300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88109040 0x7f9b881032e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009900 tx=0x7f9b7c02a410 comp rx=0 tx=0).stop 2024-05-09T05:39:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.539+0000 7f9b8d29a640 1 -- 172.21.15.115:0/1857311300 shutdown_connections 2024-05-09T05:39:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.539+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/1857311300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b88103950 0x7f9b8810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.539+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/1857311300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88109040 0x7f9b881032e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.539+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/1857311300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b88108ba0 0x7f9b88102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.539+0000 7f9b8d29a640 1 -- 172.21.15.115:0/1857311300 >> 172.21.15.115:0/1857311300 conn(0x7f9b880779e0 msgr2=0x7f9b88077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.540+0000 7f9b8d29a640 1 -- 172.21.15.115:0/1857311300 shutdown_connections 2024-05-09T05:39:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.540+0000 7f9b8d29a640 1 -- 172.21.15.115:0/1857311300 wait complete. 2024-05-09T05:39:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.540+0000 7f9b8d29a640 1 Processor -- start 2024-05-09T05:39:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.540+0000 7f9b8d29a640 1 -- start start 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b8d29a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b88103950 0x7f9b8819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b8d29a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88108ba0 0x7f9b8819bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b8d29a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b88109040 0x7f9b8819c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b867fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88108ba0 0x7f9b8819bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b867fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88108ba0 0x7f9b8819bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50532/0 (socket says 172.21.15.115:50532) 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b8d29a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9b88112c40 con 0x7f9b88108ba0 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b867fc640 1 -- 172.21.15.115:0/2021463888 learned_addr learned my addr 172.21.15.115:0/2021463888 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9b88112ac0 con 0x7f9b88103950 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b88112dc0 con 0x7f9b88109040 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b88103950 0x7f9b8819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b877fe640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b88109040 0x7f9b8819c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b867fc640 1 -- 172.21.15.115:0/2021463888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b88109040 msgr2=0x7f9b8819c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b867fc640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b88109040 0x7f9b8819c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b867fc640 1 -- 172.21.15.115:0/2021463888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b88103950 msgr2=0x7f9b8819b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b867fc640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b88103950 0x7f9b8819b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.541+0000 7f9b867fc640 1 -- 172.21.15.115:0/2021463888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b881a68d0 con 0x7f9b88108ba0 2024-05-09T05:39:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.542+0000 7f9b867fc640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88108ba0 0x7f9b8819bb60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c005940 tx=0x7f9b7c0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.542+0000 7f9b6ffff640 1 -- 172.21.15.115:0/2021463888 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c03f030 con 0x7f9b88108ba0 2024-05-09T05:39:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.542+0000 7f9b6ffff640 1 -- 172.21.15.115:0/2021463888 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b7c003a80 con 0x7f9b88108ba0 2024-05-09T05:39:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.542+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b881a6b60 con 0x7f9b88108ba0 2024-05-09T05:39:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.542+0000 7f9b6ffff640 1 -- 172.21.15.115:0/2021463888 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c035070 con 0x7f9b88108ba0 2024-05-09T05:39:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.542+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b881a70a0 con 0x7f9b88108ba0 2024-05-09T05:39:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.545+0000 7f9b9421f740 1 Processor -- start 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.545+0000 7f9b6ffff640 1 -- 172.21.15.115:0/2021463888 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f9b7c042030 con 0x7f9b88108ba0 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b9421f740 1 -- start start 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e415680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e415bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e416100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b9421f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561a0e37a7c0 con 0x561a0e0514d0 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b9421f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561a0e3160e0 con 0x561a0e0721b0 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b9421f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561a0e376780 con 0x561a0e0492f0 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b867fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e416100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e415680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e415680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33354/0 (socket says 172.21.15.115:33354) 2024-05-09T05:39:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b877fe640 1 -- 172.21.15.115:0/3674011402 learned_addr learned my addr 172.21.15.115:0/3674011402 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/3674011402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e415bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.546+0000 7f9b6ffff640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9b700b29d0 0x7f9b700b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b86ffd640 1 -- 172.21.15.115:0/3674011402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 msgr2=0x561a0e415680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/3674011402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e415680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b86ffd640 1 -- 172.21.15.115:0/3674011402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 msgr2=0x561a0e416100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/3674011402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e416100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b86ffd640 1 -- 172.21.15.115:0/3674011402 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0e33f7c0 con 0x561a0e0492f0 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b877fe640 1 --2- 172.21.15.115:0/3674011402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e415680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b6ffff640 1 -- 172.21.15.115:0/2021463888 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9b7c102e20 con 0x7f9b88108ba0 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b877fe640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9b700b29d0 0x7f9b700b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/3674011402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e415bc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800ab40 tx=0x7f9b7800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c5acafaa83034b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b6d7fa640 1 -- 172.21.15.115:0/3674011402 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78005020 con 0x561a0e0492f0 2024-05-09T05:39:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b6d7fa640 1 -- 172.21.15.115:0/3674011402 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b78005650 con 0x561a0e0492f0 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.547+0000 7f9b6d7fa640 1 -- 172.21.15.115:0/3674011402 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b780186a0 con 0x561a0e0492f0 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.548+0000 7f9b9421f740 1 -- 172.21.15.115:0/3674011402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 msgr2=0x561a0e415bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.548+0000 7f9b877fe640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9b700b29d0 0x7f9b700b4ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x561a0e4127f0 tx=0x7f9b80004a40 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.548+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3674011402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e415bc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800ab40 tx=0x7f9b7800f3c0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.548+0000 7f9b9421f740 1 -- 172.21.15.115:0/3674011402 shutdown_connections 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.548+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3674011402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e416100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.548+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3674011402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e415bc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.548+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3674011402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e415680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.548+0000 7f9b9421f740 1 -- 172.21.15.115:0/3674011402 >> 172.21.15.115:0/3674011402 conn(0x561a0e066300 msgr2=0x561a0e411e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.548+0000 7f9b9421f740 1 -- 172.21.15.115:0/3674011402 shutdown_connections 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.549+0000 7f9b9421f740 1 -- 172.21.15.115:0/3674011402 wait complete. 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.549+0000 7f9b9421f740 1 -- start start 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.549+0000 7f9b9421f740 1 Processor -- start 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e421720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 0x561a0e421c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e41fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b9421f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561a0e4227b0 con 0x561a0e0721b0 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b9421f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561a0e3019c0 con 0x561a0e0514d0 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b9421f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561a0e422930 con 0x561a0e0492f0 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b9421f740 10 client.0 _ll_register_callbacks cb 0x561a0e32e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e41fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e41fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50552/0 (socket says 172.21.15.115:50552) 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b877fe640 1 -- 172.21.15.115:0/1874414205 learned_addr learned my addr 172.21.15.115:0/1874414205 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e421720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b867fc640 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 0x561a0e421c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b877fe640 1 -- 172.21.15.115:0/1874414205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 msgr2=0x561a0e421720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b877fe640 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e421720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b877fe640 1 -- 172.21.15.115:0/1874414205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 msgr2=0x561a0e421c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b877fe640 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 0x561a0e421c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b877fe640 1 -- 172.21.15.115:0/1874414205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0e3281f0 con 0x561a0e0721b0 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b867fc640 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 0x561a0e421c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b877fe640 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e41fbc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9b80005fd0 tx=0x7f9b80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1152d713e601bb34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.550+0000 7f9b6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b6aff5640 1 -- 172.21.15.115:0/1874414205 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8001b480 con 0x561a0e0721b0 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b6aff5640 1 -- 172.21.15.115:0/1874414205 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9b8001b620 con 0x561a0e0721b0 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b9421f740 1 -- 172.21.15.115:0/1874414205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b78005650 con 0x561a0e0721b0 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b6aff5640 1 -- 172.21.15.115:0/1874414205 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b8001b820 con 0x561a0e0721b0 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b9421f740 10 client.15960 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b9421f740 1 -- 172.21.15.115:0/1874414205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a0e0dd790 con 0x561a0e0721b0 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b6aff5640 1 -- 172.21.15.115:0/1874414205 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b78005650 con 0x561a0e0721b0 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b6aff5640 1 -- 172.21.15.115:0/1874414205 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9b8002b9f0 con 0x561a0e0721b0 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b9421f740 10 client.15960 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b9421f740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b9421f740 20 client.15960 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.551+0000 7f9b9421f740 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a0e030d80 0x561a0e432010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.552+0000 7f9b9421f740 4 client.15960 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.552+0000 7f9b9421f740 1 -- 172.21.15.115:0/1874414205 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a0e3883d0 con 0x561a0e030d80 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.552+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a0e030d80 0x561a0e432010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.552+0000 7f9b6aff5640 10 client.15960 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.552+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a0e030d80 0x561a0e432010 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0dd520f0103f70 server_cookie=7070ad982968b184 in_seq=0 out_seq=0 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b6aff5640 1 -- 172.21.15.115:0/1874414205 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a0e3883d0 con 0x561a0e030d80 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b6aff5640 10 client.15960 handle_command_reply: tid=0 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 client.15960 shutdown 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 -- 172.21.15.115:0/1874414205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 msgr2=0x561a0e41fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e41fbc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9b80005fd0 tx=0x7f9b80004ab0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 -- 172.21.15.115:0/1874414205 shutdown_connections 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e41fbc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 0x561a0e421c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a0e030d80 0x561a0e432010 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 --2- 172.21.15.115:0/1874414205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e421720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b6aff5640 0 client.15960 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 -- 172.21.15.115:0/1874414205 >> 172.21.15.115:0/1874414205 conn(0x561a0e066300 msgr2=0x561a0e4173a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 -- 172.21.15.115:0/1874414205 shutdown_connections 2024-05-09T05:39:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.553+0000 7f9b9421f740 1 -- 172.21.15.115:0/1874414205 wait complete. 2024-05-09T05:39:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.554+0000 7f9b9421f740 20 client.15960 trim_cache size 0 max 16384 2024-05-09T05:39:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b9421f740 1 Processor -- start 2024-05-09T05:39:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b9421f740 1 -- start start 2024-05-09T05:39:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e416420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e427c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e428150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b9421f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9b881aee20 con 0x561a0e0514d0 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b9421f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9b881b0960 con 0x561a0e0721b0 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b9421f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b881af4f0 con 0x561a0e0492f0 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b867fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e427c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b867fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e427c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50562/0 (socket says 172.21.15.115:50562) 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b867fc640 1 -- 172.21.15.115:0/2161946539 learned_addr learned my addr 172.21.15.115:0/2161946539 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e416420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b877fe640 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e428150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b867fc640 1 -- 172.21.15.115:0/2161946539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 msgr2=0x561a0e416420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b867fc640 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e416420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b867fc640 1 -- 172.21.15.115:0/2161946539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 msgr2=0x561a0e428150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b867fc640 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e428150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b867fc640 1 -- 172.21.15.115:0/2161946539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0e1545f0 con 0x561a0e0514d0 2024-05-09T05:39:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.582+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e416420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.583+0000 7f9b877fe640 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e428150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.583+0000 7f9b867fc640 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e427c10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0c7030 tx=0x7f9b7c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509b5378489e239d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.583+0000 7f9b6cff9640 1 -- 172.21.15.115:0/2161946539 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c049050 con 0x561a0e0514d0 2024-05-09T05:39:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.583+0000 7f9b6cff9640 1 -- 172.21.15.115:0/2161946539 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b7c0c98d0 con 0x561a0e0514d0 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.583+0000 7f9b6cff9640 1 -- 172.21.15.115:0/2161946539 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0ce170 con 0x561a0e0514d0 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 -- 172.21.15.115:0/2161946539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 msgr2=0x561a0e427c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e427c10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0c7030 tx=0x7f9b7c005d50 comp rx=0 tx=0).stop 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 -- 172.21.15.115:0/2161946539 shutdown_connections 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0721b0 0x561a0e428150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0514d0 0x561a0e427c10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 --2- 172.21.15.115:0/2161946539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e416420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 -- 172.21.15.115:0/2161946539 >> 172.21.15.115:0/2161946539 conn(0x561a0e066300 msgr2=0x561a0e41f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 -- 172.21.15.115:0/2161946539 shutdown_connections 2024-05-09T05:39:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 -- 172.21.15.115:0/2161946539 wait complete. 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.584+0000 7f9b9421f740 1 -- start start 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b9421f740 1 Processor -- start 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e4235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 0x561a0e423b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b9421f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e424060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b9421f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9b881b0290 con 0x561a0e0721b0 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b9421f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9b881afbc0 con 0x561a0e0514d0 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b9421f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b881ace00 con 0x561a0e0492f0 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b9421f740 10 client.0 _ll_register_callbacks cb 0x7f9b80009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e424060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e424060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50568/0 (socket says 172.21.15.115:50568) 2024-05-09T05:39:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b877fe640 1 -- 172.21.15.115:0/3513068589 learned_addr learned my addr 172.21.15.115:0/3513068589 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b867fc640 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 0x561a0e423b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e4235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.585+0000 7f9b877fe640 1 -- 172.21.15.115:0/3513068589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 msgr2=0x561a0e4235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b877fe640 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e4235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b877fe640 1 -- 172.21.15.115:0/3513068589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 msgr2=0x561a0e423b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b877fe640 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 0x561a0e423b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b877fe640 1 -- 172.21.15.115:0/3513068589 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0e33eed0 con 0x561a0e0721b0 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e4235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b877fe640 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e424060 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9b8002be90 tx=0x7f9b8000fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0258987803dcfb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b6aff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b6aff5640 1 -- 172.21.15.115:0/3513068589 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8002b670 con 0x561a0e0721b0 2024-05-09T05:39:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b6aff5640 1 -- 172.21.15.115:0/3513068589 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9b8002b810 con 0x561a0e0721b0 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b9421f740 1 -- 172.21.15.115:0/3513068589 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b7c0c98d0 con 0x561a0e0721b0 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b6aff5640 1 -- 172.21.15.115:0/3513068589 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b80046050 con 0x561a0e0721b0 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b9421f740 10 client.15966 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b9421f740 1 -- 172.21.15.115:0/3513068589 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a0e49a380 con 0x561a0e0721b0 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.586+0000 7f9b6aff5640 1 -- 172.21.15.115:0/3513068589 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b7c0c98d0 con 0x561a0e0721b0 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b6aff5640 1 -- 172.21.15.115:0/3513068589 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9b80038420 con 0x561a0e0721b0 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b9421f740 10 client.15966 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b9421f740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b9421f740 20 client.15966 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a0e0061c0 0x561a0e49cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b9421f740 4 client.15966 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b9421f740 1 -- 172.21.15.115:0/3513068589 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a0e382240 con 0x561a0e0061c0 2024-05-09T05:39:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a0e0061c0 0x561a0e49cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b6aff5640 10 client.15966 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.587+0000 7f9b86ffd640 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a0e0061c0 0x561a0e49cb60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e666a5c1209299c server_cookie=e5cba27e21073045 in_seq=0 out_seq=0 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b6aff5640 1 -- 172.21.15.115:0/3513068589 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561a0e382240 con 0x561a0e0061c0 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b6aff5640 10 client.15966 handle_command_reply: tid=0 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 client.15966 shutdown 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 -- 172.21.15.115:0/3513068589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 msgr2=0x561a0e424060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e424060 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9b8002be90 tx=0x7f9b8000fbd0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 -- 172.21.15.115:0/3513068589 shutdown_connections 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a0e0721b0 0x561a0e424060 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a0e0514d0 0x561a0e423b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a0e0061c0 0x561a0e49cb60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 --2- 172.21.15.115:0/3513068589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a0e0492f0 0x561a0e4235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 -- 172.21.15.115:0/3513068589 >> 172.21.15.115:0/3513068589 conn(0x561a0e066300 msgr2=0x561a0e41fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b6aff5640 0 client.15966 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 -- 172.21.15.115:0/3513068589 shutdown_connections 2024-05-09T05:39:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 1 -- 172.21.15.115:0/3513068589 wait complete. 2024-05-09T05:39:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.594+0000 7f9b9421f740 20 client.15966 trim_cache size 0 max 16384 2024-05-09T05:39:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:39:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:39:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:39:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9b700b29d0 msgr2=0x7f9b700b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9b700b29d0 0x7f9b700b4ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x561a0e4127f0 tx=0x7f9b80004a40 comp rx=0 tx=0).stop 2024-05-09T05:39:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88108ba0 msgr2=0x7f9b8819bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88108ba0 0x7f9b8819bb60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c005940 tx=0x7f9b7c0040e0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 shutdown_connections 2024-05-09T05:39:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9b700b29d0 0x7f9b700b4ee0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b88109040 0x7f9b8819c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b88108ba0 0x7f9b8819bb60 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 --2- 172.21.15.115:0/2021463888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b88103950 0x7f9b8819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 >> 172.21.15.115:0/2021463888 conn(0x7f9b880779e0 msgr2=0x7f9b88100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 shutdown_connections 2024-05-09T05:39:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:45.595+0000 7f9b8d29a640 1 -- 172.21.15.115:0/2021463888 wait complete. 2024-05-09T05:39:45.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:39:45.965 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:47.323 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:47 smithi115 ceph-mon[30349]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T05:39:47.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:47 smithi195 ceph-mon[33844]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T05:39:47.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:47 smithi161 ceph-mon[33805]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T05:39:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.969+0000 7ff89ad77700 1 -- 172.21.15.115:0/597157641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 msgr2=0x7ff894110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.969+0000 7ff89ad77700 1 --2- 172.21.15.115:0/597157641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 0x7ff894110de0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00b0a0 tx=0x7ff88c00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.969+0000 7ff89ad77700 1 -- 172.21.15.115:0/597157641 shutdown_connections 2024-05-09T05:39:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.970+0000 7ff89ad77700 1 --2- 172.21.15.115:0/597157641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 0x7ff894110de0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.970+0000 7ff89ad77700 1 --2- 172.21.15.115:0/597157641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894105540 0x7ff894105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.970+0000 7ff89ad77700 1 --2- 172.21.15.115:0/597157641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff894105e90 0x7ff89410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.970+0000 7ff89ad77700 1 -- 172.21.15.115:0/597157641 >> 172.21.15.115:0/597157641 conn(0x7ff8940fef30 msgr2=0x7ff894101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:47.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.970+0000 7ff89ad77700 1 -- 172.21.15.115:0/597157641 shutdown_connections 2024-05-09T05:39:47.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.970+0000 7ff89ad77700 1 -- 172.21.15.115:0/597157641 wait complete. 2024-05-09T05:39:47.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.970+0000 7ff89ad77700 1 Processor -- start 2024-05-09T05:39:47.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff89ad77700 1 -- start start 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff89ad77700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894105540 0x7ff894075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff89ad77700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff894105e90 0x7ff89406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff89ad77700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 0x7ff89406ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff899314700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 0x7ff89406ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff893fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff894105e90 0x7ff89406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff899314700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 0x7ff89406ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50596/0 (socket says 172.21.15.115:50596) 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff898b13700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894105540 0x7ff894075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff898b13700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894105540 0x7ff894075340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:45050/0 (socket says 172.21.15.115:45050) 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff899314700 1 -- 172.21.15.115:0/2518950572 learned_addr learned my addr 172.21.15.115:0/2518950572 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff894112190 con 0x7ff89410e920 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff894112010 con 0x7ff894105e90 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff894112310 con 0x7ff894105540 2024-05-09T05:39:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.971+0000 7ff893fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff894105e90 0x7ff89406e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33392/0 (socket says 172.21.15.115:33392) 2024-05-09T05:39:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff899314700 1 -- 172.21.15.115:0/2518950572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894105540 msgr2=0x7ff894075340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff899314700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894105540 0x7ff894075340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff899314700 1 -- 172.21.15.115:0/2518950572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff894105e90 msgr2=0x7ff89406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff899314700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff894105e90 0x7ff89406e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff899314700 1 -- 172.21.15.115:0/2518950572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89406f290 con 0x7ff89410e920 2024-05-09T05:39:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff899314700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 0x7ff89406ea30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00b780 tx=0x7ff88c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff891ffb700 1 -- 172.21.15.115:0/2518950572 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c021070 con 0x7ff89410e920 2024-05-09T05:39:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff894072f30 con 0x7ff89410e920 2024-05-09T05:39:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff891ffb700 1 -- 172.21.15.115:0/2518950572 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff88c007b80 con 0x7ff89410e920 2024-05-09T05:39:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.972+0000 7ff891ffb700 1 -- 172.21.15.115:0/2518950572 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0052a0 con 0x7ff89410e920 2024-05-09T05:39:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.973+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff894073460 con 0x7ff89410e920 2024-05-09T05:39:47.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.976+0000 7ff891ffb700 1 -- 172.21.15.115:0/2518950572 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff88c024070 con 0x7ff89410e920 2024-05-09T05:39:47.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.977+0000 7ff8a978db80 1 Processor -- start 2024-05-09T05:39:47.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.977+0000 7ff891ffb700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff8840b29f0 0x7ff8840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff8a978db80 1 -- start start 2024-05-09T05:39:47.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff893fff700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff8840b29f0 0x7ff8840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff891ffb700 1 -- 172.21.15.115:0/2518950572 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff88c00f020 con 0x7ff89410e920 2024-05-09T05:39:47.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182ed4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 0x5599182eda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x5599182e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff8a978db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559918209ff0 con 0x559918254160 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff8a978db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5599181a87f0 con 0x5599182e76a0 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff8a978db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559918205fb0 con 0x55991816a4a0 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff898b13700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182ed4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff898b13700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182ed4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50610/0 (socket says 172.21.15.115:50610) 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff898b13700 1 -- 172.21.15.115:0/2677062529 learned_addr learned my addr 172.21.15.115:0/2677062529 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff899314700 1 --2- 172.21.15.115:0/2677062529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x5599182e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff893fff700 1 --2- 172.21.15.115:0/2677062529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 0x5599182eda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.978+0000 7ff893fff700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff8840b29f0 0x7ff8840b4f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff894102490 tx=0x7ff8800103d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.979+0000 7ff899314700 1 -- 172.21.15.115:0/2677062529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 msgr2=0x5599182eda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.979+0000 7ff899314700 1 --2- 172.21.15.115:0/2677062529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 0x5599182eda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.979+0000 7ff899314700 1 -- 172.21.15.115:0/2677062529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 msgr2=0x5599182ed4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.979+0000 7ff899314700 1 --2- 172.21.15.115:0/2677062529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182ed4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.979+0000 7ff899314700 1 -- 172.21.15.115:0/2677062529 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599181a0370 con 0x5599182e76a0 2024-05-09T05:39:47.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.979+0000 7ff899314700 1 --2- 172.21.15.115:0/2677062529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x5599182e7ad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0e1280 tx=0x7ff88c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03ed7b51a8ebb4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:47.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.979+0000 7ff87effd700 1 -- 172.21.15.115:0/2677062529 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0aaea0 con 0x5599182e76a0 2024-05-09T05:39:47.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.979+0000 7ff87effd700 1 -- 172.21.15.115:0/2677062529 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff88c0af250 con 0x5599182e76a0 2024-05-09T05:39:47.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.979+0000 7ff87effd700 1 -- 172.21.15.115:0/2677062529 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0b37e0 con 0x5599182e76a0 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 -- 172.21.15.115:0/2677062529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 msgr2=0x5599182e7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2677062529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x5599182e7ad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0e1280 tx=0x7ff88c0099e0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 -- 172.21.15.115:0/2677062529 shutdown_connections 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2677062529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x5599182e7ad0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2677062529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182ed4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2677062529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 0x5599182eda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 -- 172.21.15.115:0/2677062529 >> 172.21.15.115:0/2677062529 conn(0x5599181ca6a0 msgr2=0x5599182e2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 -- 172.21.15.115:0/2677062529 shutdown_connections 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 -- 172.21.15.115:0/2677062529 wait complete. 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 -- start start 2024-05-09T05:39:47.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.981+0000 7ff8a978db80 1 Processor -- start 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.982+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.982+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182f7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.982+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.982+0000 7ff8a978db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559918085ba0 con 0x559918254160 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.982+0000 7ff8a978db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5599181c4150 con 0x55991816a4a0 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.982+0000 7ff8a978db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5599182e8360 con 0x5599182e76a0 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff893fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff898b13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff898b13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33418/0 (socket says 172.21.15.115:33418) 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff898b13700 1 -- 172.21.15.115:0/537053102 learned_addr learned my addr 172.21.15.115:0/537053102 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff893fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182f7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50614/0 (socket says 172.21.15.115:50614) 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff899314700 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff8a978db80 10 client.0 _ll_register_callbacks cb 0x55991807b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff898b13700 1 -- 172.21.15.115:0/537053102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 msgr2=0x5599182f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff898b13700 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff898b13700 1 -- 172.21.15.115:0/537053102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 msgr2=0x5599182f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff898b13700 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff898b13700 1 -- 172.21.15.115:0/537053102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559918085490 con 0x55991816a4a0 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff899314700 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.983+0000 7ff893fff700 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.984+0000 7ff898b13700 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f6cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff888004450 tx=0x7ff888003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b565f5639c0da085 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.984+0000 7ff873fff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.984+0000 7ff873fff700 1 -- 172.21.15.115:0/537053102 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888020060 con 0x55991816a4a0 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.984+0000 7ff8a978db80 1 -- 172.21.15.115:0/537053102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff88c0af250 con 0x55991816a4a0 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.984+0000 7ff873fff700 1 -- 172.21.15.115:0/537053102 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff88801b670 con 0x55991816a4a0 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.984+0000 7ff873fff700 1 -- 172.21.15.115:0/537053102 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff888027050 con 0x55991816a4a0 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.984+0000 7ff8a978db80 10 client.25511 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:47.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.984+0000 7ff8a978db80 1 -- 172.21.15.115:0/537053102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5599182fe770 con 0x55991816a4a0 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.984+0000 7ff873fff700 1 -- 172.21.15.115:0/537053102 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff88c0af250 con 0x55991816a4a0 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.985+0000 7ff873fff700 1 -- 172.21.15.115:0/537053102 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff888019370 con 0x55991816a4a0 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.985+0000 7ff8a978db80 10 client.25511 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.985+0000 7ff8a978db80 10 client.25511 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.985+0000 7ff8a978db80 20 client.25511 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.985+0000 7ff8a978db80 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x559918301130 0x559918303600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.985+0000 7ff8a978db80 4 client.25511 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.985+0000 7ff8a978db80 1 -- 172.21.15.115:0/537053102 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599182179d0 con 0x559918301130 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.985+0000 7ff893fff700 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x559918301130 0x559918303600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.986+0000 7ff873fff700 10 client.25511 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:39:47.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.986+0000 7ff893fff700 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x559918301130 0x559918303600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20343ca60a231e51 server_cookie=1c41a1364077dbe4 in_seq=0 out_seq=0 2024-05-09T05:39:47.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.986+0000 7ff873fff700 1 -- 172.21.15.115:0/537053102 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599182179d0 con 0x559918301130 2024-05-09T05:39:47.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.986+0000 7ff873fff700 10 client.25511 handle_command_reply: tid=0 2024-05-09T05:39:47.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.986+0000 7ff8a978db80 1 client.25511 shutdown 2024-05-09T05:39:47.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 -- 172.21.15.115:0/537053102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 msgr2=0x5599182f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f6cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff888004450 tx=0x7ff888003960 comp rx=0 tx=0).stop 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 -- 172.21.15.115:0/537053102 shutdown_connections 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f6cf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x559918301130 0x559918303600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 --2- 172.21.15.115:0/537053102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 -- 172.21.15.115:0/537053102 >> 172.21.15.115:0/537053102 conn(0x5599181ca6a0 msgr2=0x5599182e21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff873fff700 0 client.25511 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 -- 172.21.15.115:0/537053102 shutdown_connections 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 1 -- 172.21.15.115:0/537053102 wait complete. 2024-05-09T05:39:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:47.987+0000 7ff8a978db80 20 client.25511 trim_cache size 0 max 16384 2024-05-09T05:39:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.016+0000 7ff8a978db80 1 Processor -- start 2024-05-09T05:39:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.016+0000 7ff8a978db80 1 -- start start 2024-05-09T05:39:48.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.016+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 0x5599182efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:48.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.016+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:48.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.016+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x559918308dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.016+0000 7ff8a978db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b68e0 con 0x559918254160 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.016+0000 7ff8a978db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b8420 con 0x5599182e76a0 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.016+0000 7ff8a978db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b6fb0 con 0x55991816a4a0 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff893fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff893fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182eff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50626/0 (socket says 172.21.15.115:50626) 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff893fff700 1 -- 172.21.15.115:0/4244072027 learned_addr learned my addr 172.21.15.115:0/4244072027 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff899314700 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x559918308dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff898b13700 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 0x5599182efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff899314700 1 -- 172.21.15.115:0/4244072027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 msgr2=0x5599182efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff899314700 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 0x5599182efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff899314700 1 -- 172.21.15.115:0/4244072027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 msgr2=0x5599182eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff899314700 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff899314700 1 -- 172.21.15.115:0/4244072027 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599182e8050 con 0x5599182e76a0 2024-05-09T05:39:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff893fff700 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:48.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.017+0000 7ff899314700 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x559918308dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0e12b0 tx=0x7ff88c009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa1221a9f07a74c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:48.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.018+0000 7ff87e7fc700 1 -- 172.21.15.115:0/4244072027 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0af250 con 0x5599182e76a0 2024-05-09T05:39:48.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.018+0000 7ff87e7fc700 1 -- 172.21.15.115:0/4244072027 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff88c0b3c70 con 0x5599182e76a0 2024-05-09T05:39:48.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.018+0000 7ff87e7fc700 1 -- 172.21.15.115:0/4244072027 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0b37e0 con 0x5599182e76a0 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 -- 172.21.15.115:0/4244072027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 msgr2=0x559918308dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x559918308dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0e12b0 tx=0x7ff88c009600 comp rx=0 tx=0).stop 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 -- 172.21.15.115:0/4244072027 shutdown_connections 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599182e76a0 0x559918308dc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55991816a4a0 0x5599182efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 --2- 172.21.15.115:0/4244072027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 -- 172.21.15.115:0/4244072027 >> 172.21.15.115:0/4244072027 conn(0x5599181ca6a0 msgr2=0x559918303140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 -- 172.21.15.115:0/4244072027 shutdown_connections 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 -- 172.21.15.115:0/4244072027 wait complete. 2024-05-09T05:39:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.019+0000 7ff8a978db80 1 -- start start 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff8a978db80 1 Processor -- start 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182fa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff8a978db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff8a978db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b7d50 con 0x559918254160 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff8a978db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b7680 con 0x55991816a4a0 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff8a978db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b4880 con 0x5599182e76a0 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff8a978db80 10 client.0 _ll_register_callbacks cb 0x5599182e20a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff893fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182fa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff898b13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff893fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182fa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50630/0 (socket says 172.21.15.115:50630) 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.020+0000 7ff893fff700 1 -- 172.21.15.115:0/2819643662 learned_addr learned my addr 172.21.15.115:0/2819643662 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff899314700 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:48.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff893fff700 1 -- 172.21.15.115:0/2819643662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 msgr2=0x5599182fa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff893fff700 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182fa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff893fff700 1 -- 172.21.15.115:0/2819643662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 msgr2=0x5599182f9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff893fff700 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff893fff700 1 -- 172.21.15.115:0/2819643662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599182f1f30 con 0x559918254160 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff898b13700 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff899314700 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff893fff700 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182fa060 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x5599182e3ed0 tx=0x7ff880004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbad9b9eb6f8a13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff873fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff873fff700 1 -- 172.21.15.115:0/2819643662 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880036020 con 0x559918254160 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff873fff700 1 -- 172.21.15.115:0/2819643662 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff8800322b0 con 0x559918254160 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.021+0000 7ff8a978db80 1 -- 172.21.15.115:0/2819643662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559918211a70 con 0x559918254160 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff873fff700 1 -- 172.21.15.115:0/2819643662 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff880041050 con 0x559918254160 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff8a978db80 10 client.15984 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:48.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff8a978db80 1 -- 172.21.15.115:0/2819643662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5599182ed080 con 0x559918254160 2024-05-09T05:39:48.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff873fff700 1 -- 172.21.15.115:0/2819643662 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559918211a70 con 0x559918254160 2024-05-09T05:39:48.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff873fff700 1 -- 172.21.15.115:0/2819643662 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff880030880 con 0x559918254160 2024-05-09T05:39:48.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff8a978db80 10 client.15984 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:48.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff8a978db80 10 client.15984 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:39:48.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff8a978db80 20 client.15984 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:48.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55991836b010 0x55991836d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:48.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff8a978db80 4 client.15984 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:48.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.022+0000 7ff8a978db80 1 -- 172.21.15.115:0/2819643662 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559918085970 con 0x55991836b010 2024-05-09T05:39:48.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.023+0000 7ff898b13700 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55991836b010 0x55991836d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:48.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.023+0000 7ff873fff700 10 client.15984 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:39:48.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.023+0000 7ff898b13700 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55991836b010 0x55991836d430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6334de003ae3da01 server_cookie=ff9d607d6be1f51c in_seq=0 out_seq=0 2024-05-09T05:39:48.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.023+0000 7ff873fff700 1 -- 172.21.15.115:0/2819643662 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559918085970 con 0x55991836b010 2024-05-09T05:39:48.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.023+0000 7ff873fff700 10 client.15984 handle_command_reply: tid=0 2024-05-09T05:39:48.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.023+0000 7ff8a978db80 1 client.15984 shutdown 2024-05-09T05:39:48.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 -- 172.21.15.115:0/2819643662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 msgr2=0x5599182fa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:48.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182fa060 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x5599182e3ed0 tx=0x7ff880004750 comp rx=0 tx=0).stop 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 -- 172.21.15.115:0/2819643662 shutdown_connections 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55991816a4a0 0x5599182f9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55991836b010 0x55991836d430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff873fff700 0 client.15984 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599182e76a0 0x5599182fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 --2- 172.21.15.115:0/2819643662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559918254160 0x5599182fa060 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 -- 172.21.15.115:0/2819643662 >> 172.21.15.115:0/2819643662 conn(0x5599181ca6a0 msgr2=0x5599182ef9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 -- 172.21.15.115:0/2819643662 shutdown_connections 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 1 -- 172.21.15.115:0/2819643662 wait complete. 2024-05-09T05:39:48.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.024+0000 7ff8a978db80 20 client.15984 trim_cache size 0 max 16384 2024-05-09T05:39:48.023 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:48.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff8840b29f0 msgr2=0x7ff8840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:48.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff8840b29f0 0x7ff8840b4f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff894102490 tx=0x7ff8800103d0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 msgr2=0x7ff89406ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:48.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 0x7ff89406ea30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00b780 tx=0x7ff88c004d00 comp rx=0 tx=0).stop 2024-05-09T05:39:48.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 shutdown_connections 2024-05-09T05:39:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff8840b29f0 0x7ff8840b4f00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff89410e920 0x7ff89406ea30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff894105540 0x7ff894075340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 --2- 172.21.15.115:0/2518950572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff894105e90 0x7ff89406e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.025+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 >> 172.21.15.115:0/2518950572 conn(0x7ff8940fef30 msgr2=0x7ff8940ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.026+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 shutdown_connections 2024-05-09T05:39:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:48.026+0000 7ff89ad77700 1 -- 172.21.15.115:0/2518950572 wait complete. 2024-05-09T05:39:48.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:39:49.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:49 smithi115 ceph-mon[30349]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-05-09T05:39:49.184 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:49.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:49 smithi195 ceph-mon[33844]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-05-09T05:39:49.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:49 smithi161 ceph-mon[33805]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-05-09T05:39:51.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.124+0000 7f92bec13700 1 -- 172.21.15.115:0/2152437704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8102880 msgr2=0x7f92b8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:51.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.124+0000 7f92bec13700 1 --2- 172.21.15.115:0/2152437704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8102880 0x7f92b8102c90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009990 tx=0x7f92a4009ca0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.124+0000 7f92bec13700 1 -- 172.21.15.115:0/2152437704 shutdown_connections 2024-05-09T05:39:51.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.124+0000 7f92bec13700 1 --2- 172.21.15.115:0/2152437704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b81031d0 0x7f92b8111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.124+0000 7f92bec13700 1 --2- 172.21.15.115:0/2152437704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92b8109300 0x7f92b8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.124+0000 7f92bec13700 1 --2- 172.21.15.115:0/2152437704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8102880 0x7f92b8102c90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.124+0000 7f92bec13700 1 -- 172.21.15.115:0/2152437704 >> 172.21.15.115:0/2152437704 conn(0x7f92b80fe6f0 msgr2=0x7f92b8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:51.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.124+0000 7f92bec13700 1 -- 172.21.15.115:0/2152437704 shutdown_connections 2024-05-09T05:39:51.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.124+0000 7f92bec13700 1 -- 172.21.15.115:0/2152437704 wait complete. 2024-05-09T05:39:51.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.126+0000 7f92bec13700 1 Processor -- start 2024-05-09T05:39:51.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.126+0000 7f92bec13700 1 -- start start 2024-05-09T05:39:51.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92bec13700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8102880 0x7f92b819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92bec13700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92b81031d0 0x7f92b819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92bec13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8109300 0x7f92b819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92bec13700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92b81144a0 con 0x7f92b8102880 2024-05-09T05:39:51.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92bc9af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8102880 0x7f92b819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92b81031d0 0x7f92b819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92b81031d0 0x7f92b819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:45094/0 (socket says 172.21.15.115:45094) 2024-05-09T05:39:51.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92b7fff700 1 -- 172.21.15.115:0/1221578029 learned_addr learned my addr 172.21.15.115:0/1221578029 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:51.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92bd1b0700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8109300 0x7f92b819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8114320 con 0x7f92b8109300 2024-05-09T05:39:51.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.127+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8114620 con 0x7f92b81031d0 2024-05-09T05:39:51.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.128+0000 7f92bc9af700 1 -- 172.21.15.115:0/1221578029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92b81031d0 msgr2=0x7f92b819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:51.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.128+0000 7f92bc9af700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92b81031d0 0x7f92b819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.128+0000 7f92bc9af700 1 -- 172.21.15.115:0/1221578029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8109300 msgr2=0x7f92b819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:51.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.128+0000 7f92bc9af700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8109300 0x7f92b819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.128+0000 7f92bc9af700 1 -- 172.21.15.115:0/1221578029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b819fd20 con 0x7f92b8102880 2024-05-09T05:39:51.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.129+0000 7f92bc9af700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8102880 0x7f92b819ae50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f92b806b050 tx=0x7f92ac00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:51.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.129+0000 7f92b5ffb700 1 -- 172.21.15.115:0/1221578029 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004220 con 0x7f92b8102880 2024-05-09T05:39:51.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.129+0000 7f92b5ffb700 1 -- 172.21.15.115:0/1221578029 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92ac0043c0 con 0x7f92b8102880 2024-05-09T05:39:51.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.130+0000 7f92b5ffb700 1 -- 172.21.15.115:0/1221578029 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac003b80 con 0x7f92b8102880 2024-05-09T05:39:51.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.130+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b81a0010 con 0x7f92b8102880 2024-05-09T05:39:51.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.130+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b81a7930 con 0x7f92b8102880 2024-05-09T05:39:51.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.134+0000 7f92b5ffb700 1 -- 172.21.15.115:0/1221578029 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f92ac00c7c0 con 0x7f92b8102880 2024-05-09T05:39:51.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92cd629b80 1 Processor -- start 2024-05-09T05:39:51.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92cd629b80 1 -- start start 2024-05-09T05:39:51.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7825d2160 0x55e7826694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7824e84a0 0x55e7826656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 0x55e782666150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92cd629b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e782587ff0 con 0x55e7824e84a0 2024-05-09T05:39:51.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92cd629b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e7825267f0 con 0x55e7825d2160 2024-05-09T05:39:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92cd629b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e782583fb0 con 0x55e782665c80 2024-05-09T05:39:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92bc9af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7824e84a0 0x55e7826656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92bc9af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7824e84a0 0x55e7826656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50678/0 (socket says 172.21.15.115:50678) 2024-05-09T05:39:51.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 0x55e782666150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.136+0000 7f92bc9af700 1 -- 172.21.15.115:0/486498736 learned_addr learned my addr 172.21.15.115:0/486498736 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92bd1b0700 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7825d2160 0x55e7826694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92b7fff700 1 -- 172.21.15.115:0/486498736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7825d2160 msgr2=0x55e7826694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92b7fff700 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7825d2160 0x55e7826694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92b7fff700 1 -- 172.21.15.115:0/486498736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7824e84a0 msgr2=0x55e7826656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92b7fff700 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7824e84a0 0x55e7826656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92b5ffb700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92a80b2af0 0x7f92a80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92b7fff700 1 -- 172.21.15.115:0/486498736 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e78251e370 con 0x55e782665c80 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92bc9af700 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7824e84a0 0x55e7826656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92bd1b0700 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7825d2160 0x55e7826694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92b7fff700 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 0x55e782666150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92a400b2e0 tx=0x7f92a40096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5d37b79079eea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92b5ffb700 1 -- 172.21.15.115:0/1221578029 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f92ac0e2b40 con 0x7f92b8102880 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92bd1b0700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92a80b2af0 0x7f92a80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92a2ffd700 1 -- 172.21.15.115:0/486498736 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4027070 con 0x55e782665c80 2024-05-09T05:39:51.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.137+0000 7f92a2ffd700 1 -- 172.21.15.115:0/486498736 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92a4003b10 con 0x55e782665c80 2024-05-09T05:39:51.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.138+0000 7f92a2ffd700 1 -- 172.21.15.115:0/486498736 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a400bbd0 con 0x55e782665c80 2024-05-09T05:39:51.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92cd629b80 1 -- 172.21.15.115:0/486498736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 msgr2=0x55e782666150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:51.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92bd1b0700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92a80b2af0 0x7f92a80b5000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009510 tx=0x7f92b000b040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:51.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92cd629b80 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 0x55e782666150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92a400b2e0 tx=0x7f92a40096a0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92cd629b80 1 -- 172.21.15.115:0/486498736 shutdown_connections 2024-05-09T05:39:51.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92cd629b80 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7825d2160 0x55e7826694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92cd629b80 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 0x55e782666150 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92cd629b80 1 --2- 172.21.15.115:0/486498736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7824e84a0 0x55e7826656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92cd629b80 1 -- 172.21.15.115:0/486498736 >> 172.21.15.115:0/486498736 conn(0x55e7825486a0 msgr2=0x55e782660400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:51.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92cd629b80 1 -- 172.21.15.115:0/486498736 shutdown_connections 2024-05-09T05:39:51.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.139+0000 7f92cd629b80 1 -- 172.21.15.115:0/486498736 wait complete. 2024-05-09T05:39:51.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.140+0000 7f92cd629b80 1 -- start start 2024-05-09T05:39:51.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.140+0000 7f92cd629b80 1 Processor -- start 2024-05-09T05:39:51.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.140+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 0x55e782673d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.140+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7825d2160 0x55e782674260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.140+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e782665c80 0x55e782679a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.140+0000 7f92cd629b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e782403ba0 con 0x55e782665c80 2024-05-09T05:39:51.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.140+0000 7f92cd629b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e782542150 con 0x55e7824e84a0 2024-05-09T05:39:51.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.140+0000 7f92cd629b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e782669c90 con 0x55e7825d2160 2024-05-09T05:39:51.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.140+0000 7f92cd629b80 10 client.0 _ll_register_callbacks cb 0x55e7823f96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:51.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7825d2160 0x55e782674260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7825d2160 0x55e782674260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:45110/0 (socket says 172.21.15.115:45110) 2024-05-09T05:39:51.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92b7fff700 1 -- 172.21.15.115:0/1254735828 learned_addr learned my addr 172.21.15.115:0/1254735828 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:51.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92bd1b0700 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e782665c80 0x55e782679a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92bc9af700 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 0x55e782673d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92b7fff700 1 -- 172.21.15.115:0/1254735828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 msgr2=0x55e782673d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:51.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92b7fff700 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 0x55e782673d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92b7fff700 1 -- 172.21.15.115:0/1254735828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e782665c80 msgr2=0x55e782679a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:51.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92b7fff700 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e782665c80 0x55e782679a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92b7fff700 1 -- 172.21.15.115:0/1254735828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e782403490 con 0x55e7825d2160 2024-05-09T05:39:51.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92bd1b0700 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e782665c80 0x55e782679a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:51.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f92b7fff700 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7825d2160 0x55e782674260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f92a40094e0 tx=0x7f92a40054f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fc1042a32f0076 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:51.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.141+0000 7f9297fff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:39:51.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f9297fff700 1 -- 172.21.15.115:0/1254735828 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4033060 con 0x55e7825d2160 2024-05-09T05:39:51.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f9297fff700 1 -- 172.21.15.115:0/1254735828 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f92a400b6b0 con 0x55e7825d2160 2024-05-09T05:39:51.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f92cd629b80 1 -- 172.21.15.115:0/1254735828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a4003b10 con 0x55e7825d2160 2024-05-09T05:39:51.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f9297fff700 1 -- 172.21.15.115:0/1254735828 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92a4027070 con 0x55e7825d2160 2024-05-09T05:39:51.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f92cd629b80 10 client.25600 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:51.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f92cd629b80 1 -- 172.21.15.115:0/1254735828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e78267cb10 con 0x55e7825d2160 2024-05-09T05:39:51.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f9297fff700 1 -- 172.21.15.115:0/1254735828 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92a4028040 con 0x55e7825d2160 2024-05-09T05:39:51.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f9297fff700 1 -- 172.21.15.115:0/1254735828 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f92a402e890 con 0x55e7825d2160 2024-05-09T05:39:51.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f92cd629b80 10 client.25600 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:51.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f92cd629b80 10 client.25600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:39:51.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f92cd629b80 20 client.25600 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:51.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.142+0000 7f92cd629b80 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e78267f0a0 0x55e7826814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.143+0000 7f92cd629b80 4 client.25600 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.143+0000 7f92cd629b80 1 -- 172.21.15.115:0/1254735828 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f92a4003b10 con 0x55e78267f0a0 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.143+0000 7f92bc9af700 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e78267f0a0 0x55e7826814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.143+0000 7f9297fff700 10 client.25600 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.143+0000 7f92bc9af700 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e78267f0a0 0x55e7826814c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3cee09b76bfd8ca server_cookie=35aa3a481f086b28 in_seq=0 out_seq=0 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f9297fff700 1 -- 172.21.15.115:0/1254735828 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f92a4003b10 con 0x55e78267f0a0 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f9297fff700 10 client.25600 handle_command_reply: tid=0 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f92cd629b80 1 client.25600 shutdown 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f92cd629b80 1 -- 172.21.15.115:0/1254735828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7825d2160 msgr2=0x55e782674260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f92cd629b80 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7825d2160 0x55e782674260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f92a40094e0 tx=0x7f92a40054f0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f92cd629b80 1 -- 172.21.15.115:0/1254735828 shutdown_connections 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f92cd629b80 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 0x55e782673d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f92cd629b80 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e78267f0a0 0x55e7826814c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f92cd629b80 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e782665c80 0x55e782679a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f9297fff700 0 client.25600 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:51.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f92cd629b80 1 --2- 172.21.15.115:0/1254735828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7825d2160 0x55e782674260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.144+0000 7f92cd629b80 1 -- 172.21.15.115:0/1254735828 >> 172.21.15.115:0/1254735828 conn(0x55e7825486a0 msgr2=0x55e7826601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:51.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.145+0000 7f92cd629b80 1 -- 172.21.15.115:0/1254735828 shutdown_connections 2024-05-09T05:39:51.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.145+0000 7f92cd629b80 1 -- 172.21.15.115:0/1254735828 wait complete. 2024-05-09T05:39:51.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.145+0000 7f92cd629b80 20 client.25600 trim_cache size 0 max 16384 2024-05-09T05:39:51.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92cd629b80 1 Processor -- start 2024-05-09T05:39:51.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92cd629b80 1 -- start start 2024-05-09T05:39:51.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 0x55e7826735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e782679a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 0x55e782664da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92cd629b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8073740 con 0x55e7825d2160 2024-05-09T05:39:51.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92cd629b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8075280 con 0x55e7824e84a0 2024-05-09T05:39:51.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92cd629b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8073e10 con 0x55e782665c80 2024-05-09T05:39:51.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e782679a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e782679a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50690/0 (socket says 172.21.15.115:50690) 2024-05-09T05:39:51.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92b7fff700 1 -- 172.21.15.115:0/684565885 learned_addr learned my addr 172.21.15.115:0/684565885 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:51.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.175+0000 7f92bc9af700 1 --2- 172.21.15.115:0/684565885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 0x55e7826735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92bd1b0700 1 --2- 172.21.15.115:0/684565885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 0x55e782664da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92b7fff700 1 -- 172.21.15.115:0/684565885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 msgr2=0x55e782664da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:51.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92b7fff700 1 --2- 172.21.15.115:0/684565885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 0x55e782664da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92b7fff700 1 -- 172.21.15.115:0/684565885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 msgr2=0x55e7826735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:51.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92b7fff700 1 --2- 172.21.15.115:0/684565885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 0x55e7826735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92b7fff700 1 -- 172.21.15.115:0/684565885 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e78267a280 con 0x55e7825d2160 2024-05-09T05:39:51.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92b7fff700 1 --2- 172.21.15.115:0/684565885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e782679a10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f92a402ece0 tx=0x7f92a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9371c813d874d1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:51.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92a27fc700 1 -- 172.21.15.115:0/684565885 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4033040 con 0x55e7825d2160 2024-05-09T05:39:51.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92a27fc700 1 -- 172.21.15.115:0/684565885 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92a4003a90 con 0x55e7825d2160 2024-05-09T05:39:51.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.176+0000 7f92a27fc700 1 -- 172.21.15.115:0/684565885 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a400fd40 con 0x55e7825d2160 2024-05-09T05:39:51.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 -- 172.21.15.115:0/684565885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 msgr2=0x55e782679a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:51.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 --2- 172.21.15.115:0/684565885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e782679a10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f92a402ece0 tx=0x7f92a4000bc0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 -- 172.21.15.115:0/684565885 shutdown_connections 2024-05-09T05:39:51.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 --2- 172.21.15.115:0/684565885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7824e84a0 0x55e7826735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 --2- 172.21.15.115:0/684565885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e782665c80 0x55e782664da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 --2- 172.21.15.115:0/684565885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e782679a10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 -- 172.21.15.115:0/684565885 >> 172.21.15.115:0/684565885 conn(0x55e7825486a0 msgr2=0x55e782664690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:51.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 -- 172.21.15.115:0/684565885 shutdown_connections 2024-05-09T05:39:51.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 -- 172.21.15.115:0/684565885 wait complete. 2024-05-09T05:39:51.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.178+0000 7f92cd629b80 1 -- start start 2024-05-09T05:39:51.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.179+0000 7f92cd629b80 1 Processor -- start 2024-05-09T05:39:51.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.179+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7824e84a0 0x55e782681470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.179+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e7826819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92cd629b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e782665c80 0x55e78272b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92bc9af700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7824e84a0 0x55e782681470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92bc9af700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7824e84a0 0x55e782681470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:45132/0 (socket says 172.21.15.115:45132) 2024-05-09T05:39:51.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e7826819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e7826819b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50700/0 (socket says 172.21.15.115:50700) 2024-05-09T05:39:51.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92bc9af700 1 -- 172.21.15.115:0/2788948109 learned_addr learned my addr 172.21.15.115:0/2788948109 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8074bb0 con 0x55e7825d2160 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80744e0 con 0x55e782665c80 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80716e0 con 0x55e7824e84a0 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92bd1b0700 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e782665c80 0x55e78272b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92cd629b80 10 client.0 _ll_register_callbacks cb 0x55e7823f96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92bc9af700 1 -- 172.21.15.115:0/2788948109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e782665c80 msgr2=0x55e78272b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92bc9af700 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e782665c80 0x55e78272b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92bc9af700 1 -- 172.21.15.115:0/2788948109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 msgr2=0x55e7826819b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92bc9af700 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e7826819b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92bc9af700 1 -- 172.21.15.115:0/2788948109 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e782682290 con 0x55e7824e84a0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.180+0000 7f92b7fff700 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e7826819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.181+0000 7f92bc9af700 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7824e84a0 0x55e782681470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0e2e70 tx=0x7f92ac003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2bec372292b8fac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.181+0000 7f9297fff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.181+0000 7f9297fff700 1 -- 172.21.15.115:0/2788948109 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0a91f0 con 0x55e7824e84a0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.181+0000 7f9297fff700 1 -- 172.21.15.115:0/2788948109 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f92ac0e61c0 con 0x55e7824e84a0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.181+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a4003a90 con 0x55e7824e84a0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.181+0000 7f9297fff700 1 -- 172.21.15.115:0/2788948109 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92ac0a5210 con 0x55e7824e84a0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.181+0000 7f92cd629b80 10 client.25606 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.181+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e78272cb00 con 0x55e7824e84a0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.182+0000 7f9297fff700 1 -- 172.21.15.115:0/2788948109 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92ac0b2220 con 0x55e7824e84a0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.182+0000 7f9297fff700 1 -- 172.21.15.115:0/2788948109 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f92ac016330 con 0x55e7824e84a0 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.182+0000 7f92cd629b80 10 client.25606 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.182+0000 7f92cd629b80 10 client.25606 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:39:51.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.182+0000 7f92cd629b80 20 client.25606 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.182+0000 7f92cd629b80 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e78272d290 0x55e78272f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.182+0000 7f92cd629b80 4 client.25606 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.182+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f92a4003a90 con 0x55e78272d290 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.183+0000 7f92b7fff700 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e78272d290 0x55e78272f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.183+0000 7f9297fff700 10 client.25606 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.183+0000 7f92b7fff700 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e78272d290 0x55e78272f6b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f06f0b1652cb8e0 server_cookie=cf302168850586b8 in_seq=0 out_seq=0 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.183+0000 7f9297fff700 1 -- 172.21.15.115:0/2788948109 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f92a4003a90 con 0x55e78272d290 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.183+0000 7f9297fff700 10 client.25606 handle_command_reply: tid=0 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.183+0000 7f92cd629b80 1 client.25606 shutdown 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.183+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7824e84a0 msgr2=0x55e782681470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.183+0000 7f92cd629b80 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7824e84a0 0x55e782681470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0e2e70 tx=0x7f92ac003f50 comp rx=0 tx=0).stop 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 shutdown_connections 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f92cd629b80 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e782665c80 0x55e78272b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f92cd629b80 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e78272d290 0x55e78272f6b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f92cd629b80 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7824e84a0 0x55e782681470 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f92cd629b80 1 --2- 172.21.15.115:0/2788948109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7825d2160 0x55e7826819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 >> 172.21.15.115:0/2788948109 conn(0x55e7825486a0 msgr2=0x55e782669ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f9297fff700 0 client.25606 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:39:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 shutdown_connections 2024-05-09T05:39:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f92cd629b80 1 -- 172.21.15.115:0/2788948109 wait complete. 2024-05-09T05:39:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.184+0000 7f92cd629b80 20 client.25606 trim_cache size 0 max 16384 2024-05-09T05:39:51.182 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92a80b2af0 msgr2=0x7f92a80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92a80b2af0 0x7f92a80b5000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009510 tx=0x7f92b000b040 comp rx=0 tx=0).stop 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8102880 msgr2=0x7f92b819ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8102880 0x7f92b819ae50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f92b806b050 tx=0x7f92ac00ee00 comp rx=0 tx=0).stop 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 shutdown_connections 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92a80b2af0 0x7f92a80b5000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92b8109300 0x7f92b819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92b8102880 0x7f92b819ae50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 --2- 172.21.15.115:0/1221578029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92b81031d0 0x7f92b819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:51.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 >> 172.21.15.115:0/1221578029 conn(0x7f92b80fe6f0 msgr2=0x7f92b810a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:51.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 shutdown_connections 2024-05-09T05:39:51.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:51.185+0000 7f92bec13700 1 -- 172.21.15.115:0/1221578029 wait complete. 2024-05-09T05:39:51.197 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:51 smithi115 ceph-mon[30349]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T05:39:51.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:51 smithi195 ceph-mon[33844]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T05:39:51.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:51 smithi161 ceph-mon[33805]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T05:39:52.095 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:39:52.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:39:52.369 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:53.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:53 smithi195 ceph-mon[33844]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T05:39:53.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:53 smithi115 ceph-mon[30349]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T05:39:53.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:53 smithi161 ceph-mon[33805]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-09T05:39:53.706 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:54.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.664+0000 7f363087f700 1 -- 172.21.15.115:0/571059008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f362c1029c0 msgr2=0x7f362c102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:54.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.664+0000 7f363087f700 1 --2- 172.21.15.115:0/571059008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f362c1029c0 0x7f362c102dd0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3620009990 tx=0x7f3620009ca0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.664+0000 7f363087f700 1 -- 172.21.15.115:0/571059008 shutdown_connections 2024-05-09T05:39:54.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.664+0000 7f363087f700 1 --2- 172.21.15.115:0/571059008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f362c1089f0 0x7f362c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.664+0000 7f363087f700 1 --2- 172.21.15.115:0/571059008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f362c103310 0x7f362c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.664+0000 7f363087f700 1 --2- 172.21.15.115:0/571059008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f362c1029c0 0x7f362c102dd0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.664+0000 7f363087f700 1 -- 172.21.15.115:0/571059008 >> 172.21.15.115:0/571059008 conn(0x7f362c0fe6f0 msgr2=0x7f362c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:54.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.664+0000 7f363087f700 1 -- 172.21.15.115:0/571059008 shutdown_connections 2024-05-09T05:39:54.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.664+0000 7f363087f700 1 -- 172.21.15.115:0/571059008 wait complete. 2024-05-09T05:39:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f363087f700 1 Processor -- start 2024-05-09T05:39:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f363087f700 1 -- start start 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f363087f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f362c1029c0 0x7f362c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f363087f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f362c103310 0x7f362c19d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f363087f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f362c1089f0 0x7f362c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f363087f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1129a0 con 0x7f362c103310 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f363087f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f362c112820 con 0x7f362c1089f0 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f363087f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f362c112b20 con 0x7f362c1029c0 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f3629d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f362c103310 0x7f362c19d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f362ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f362c1089f0 0x7f362c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f3629d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f362c103310 0x7f362c19d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41664/0 (socket says 172.21.15.115:41664) 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f3629d9b700 1 -- 172.21.15.115:0/330157458 learned_addr learned my addr 172.21.15.115:0/330157458 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.666+0000 7f362a59c700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f362c1029c0 0x7f362c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f362ad9d700 1 -- 172.21.15.115:0/330157458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f362c1029c0 msgr2=0x7f362c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f362ad9d700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f362c1029c0 0x7f362c19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f362ad9d700 1 -- 172.21.15.115:0/330157458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f362c103310 msgr2=0x7f362c19d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:54.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f362ad9d700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f362c103310 0x7f362c19d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f362ad9d700 1 -- 172.21.15.115:0/330157458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c1a1df0 con 0x7f362c1089f0 2024-05-09T05:39:54.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f362ad9d700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f362c1089f0 0x7f362c1a1750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f362400efc0 tx=0x7f362400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:54.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f361b7fe700 1 -- 172.21.15.115:0/330157458 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36240092e0 con 0x7f362c1089f0 2024-05-09T05:39:54.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f363087f700 1 -- 172.21.15.115:0/330157458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c1a2080 con 0x7f362c1089f0 2024-05-09T05:39:54.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f361b7fe700 1 -- 172.21.15.115:0/330157458 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3624004bb0 con 0x7f362c1089f0 2024-05-09T05:39:54.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.667+0000 7f361b7fe700 1 -- 172.21.15.115:0/330157458 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624004020 con 0x7f362c1089f0 2024-05-09T05:39:54.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.668+0000 7f363087f700 1 -- 172.21.15.115:0/330157458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c1a2570 con 0x7f362c1089f0 2024-05-09T05:39:54.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.670+0000 7f361b7fe700 1 -- 172.21.15.115:0/330157458 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f362401b030 con 0x7f362c1089f0 2024-05-09T05:39:54.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.672+0000 7f361b7fe700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36140b29f0 0x7f36140b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.672+0000 7f361b7fe700 1 -- 172.21.15.115:0/330157458 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f36240e2440 con 0x7f362c1089f0 2024-05-09T05:39:54.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.672+0000 7f362a59c700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36140b29f0 0x7f36140b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.673+0000 7f363f295b80 1 Processor -- start 2024-05-09T05:39:54.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.673+0000 7f362a59c700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36140b29f0 0x7f36140b4f00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f362c103b00 tx=0x7f361c009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:54.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.673+0000 7f363f295b80 1 -- start start 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.673+0000 7f363f295b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 0x56209c2d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.673+0000 7f363f295b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2d8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.673+0000 7f363f295b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f363f295b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56209c1f50a0 con 0x56209c1554a0 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f363f295b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56209c19a9a0 con 0x56209c2d2400 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f363f295b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56209c1f1060 con 0x56209c1cfd00 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 0x56209c2d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f3629d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2d8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 0x56209c2d8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:55642/0 (socket says 172.21.15.115:55642) 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362a59c700 1 -- 172.21.15.115:0/4176316870 learned_addr learned my addr 172.21.15.115:0/4176316870 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362ad9d700 1 --2- 172.21.15.115:0/4176316870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362ad9d700 1 -- 172.21.15.115:0/4176316870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 msgr2=0x56209c2d8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:54.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362ad9d700 1 --2- 172.21.15.115:0/4176316870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 0x56209c2d8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362ad9d700 1 -- 172.21.15.115:0/4176316870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 msgr2=0x56209c2d8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:54.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362ad9d700 1 --2- 172.21.15.115:0/4176316870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2d8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362ad9d700 1 -- 172.21.15.115:0/4176316870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209c18b500 con 0x56209c2d2400 2024-05-09T05:39:54.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f3629d9b700 1 --2- 172.21.15.115:0/4176316870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2d8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:54.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.674+0000 7f362ad9d700 1 --2- 172.21.15.115:0/4176316870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2d2810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36240df850 tx=0x7f36240a9050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8734b4a70db00381 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:54.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.675+0000 7f3618ff9700 1 -- 172.21.15.115:0/4176316870 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36240ad820 con 0x56209c2d2400 2024-05-09T05:39:54.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.675+0000 7f3618ff9700 1 -- 172.21.15.115:0/4176316870 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36240b2240 con 0x56209c2d2400 2024-05-09T05:39:54.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.675+0000 7f3618ff9700 1 -- 172.21.15.115:0/4176316870 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36240b2010 con 0x56209c2d2400 2024-05-09T05:39:54.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.676+0000 7f363f295b80 1 -- 172.21.15.115:0/4176316870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 msgr2=0x56209c2d2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:54.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.676+0000 7f363f295b80 1 --2- 172.21.15.115:0/4176316870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2d2810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36240df850 tx=0x7f36240a9050 comp rx=0 tx=0).stop 2024-05-09T05:39:54.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.676+0000 7f363f295b80 1 -- 172.21.15.115:0/4176316870 shutdown_connections 2024-05-09T05:39:54.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.676+0000 7f363f295b80 1 --2- 172.21.15.115:0/4176316870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2d2810 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.676+0000 7f363f295b80 1 --2- 172.21.15.115:0/4176316870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 0x56209c2d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.676+0000 7f363f295b80 1 --2- 172.21.15.115:0/4176316870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2d8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.676+0000 7f363f295b80 1 -- 172.21.15.115:0/4176316870 >> 172.21.15.115:0/4176316870 conn(0x56209c158f70 msgr2=0x56209c2cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:54.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.677+0000 7f363f295b80 1 -- 172.21.15.115:0/4176316870 shutdown_connections 2024-05-09T05:39:54.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.677+0000 7f363f295b80 1 -- 172.21.15.115:0/4176316870 wait complete. 2024-05-09T05:39:54.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.677+0000 7f363f295b80 1 -- start start 2024-05-09T05:39:54.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.677+0000 7f363f295b80 1 Processor -- start 2024-05-09T05:39:54.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f363f295b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f363f295b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1cfd00 0x56209c2dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f363f295b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2dcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f363f295b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56209c2d3160 con 0x56209c1cfd00 2024-05-09T05:39:54.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f363f295b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56209c18fb20 con 0x56209c2d2400 2024-05-09T05:39:54.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f363f295b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56209c2df020 con 0x56209c1554a0 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f362a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f362a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2dc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:55652/0 (socket says 172.21.15.115:55652) 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f362a59c700 1 -- 172.21.15.115:0/1789378199 learned_addr learned my addr 172.21.15.115:0/1789378199 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f363f295b80 10 client.0 _ll_register_callbacks cb 0x56209c0666c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f3629d9b700 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1cfd00 0x56209c2dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f362a59c700 1 -- 172.21.15.115:0/1789378199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 msgr2=0x56209c2dcbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f362a59c700 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2dcbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f362a59c700 1 -- 172.21.15.115:0/1789378199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1cfd00 msgr2=0x56209c2dc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f362a59c700 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1cfd00 0x56209c2dc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.678+0000 7f362a59c700 1 -- 172.21.15.115:0/1789378199 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209c070490 con 0x56209c1554a0 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.679+0000 7f3629d9b700 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1cfd00 0x56209c2dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.679+0000 7f362a59c700 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2dc160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f361c01f8f0 tx=0x7f361c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=764747462a156f73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:54.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.679+0000 7f3611ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:39:54.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.679+0000 7f3611ffb700 1 -- 172.21.15.115:0/1789378199 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c01db10 con 0x56209c1554a0 2024-05-09T05:39:54.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.679+0000 7f363f295b80 1 -- 172.21.15.115:0/1789378199 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36240b2240 con 0x56209c1554a0 2024-05-09T05:39:54.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.679+0000 7f3611ffb700 1 -- 172.21.15.115:0/1789378199 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f361c01dcb0 con 0x56209c1554a0 2024-05-09T05:39:54.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.679+0000 7f3611ffb700 1 -- 172.21.15.115:0/1789378199 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f361c02d5f0 con 0x56209c1554a0 2024-05-09T05:39:54.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.679+0000 7f363f295b80 10 client.25618 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:54.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.679+0000 7f363f295b80 1 -- 172.21.15.115:0/1789378199 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56209c2f2340 con 0x56209c1554a0 2024-05-09T05:39:54.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.680+0000 7f3611ffb700 1 -- 172.21.15.115:0/1789378199 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36240b2240 con 0x56209c1554a0 2024-05-09T05:39:54.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.680+0000 7f3611ffb700 1 -- 172.21.15.115:0/1789378199 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f361c026070 con 0x56209c1554a0 2024-05-09T05:39:54.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.680+0000 7f363f295b80 10 client.25618 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:54.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.680+0000 7f363f295b80 10 client.25618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:39:54.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.680+0000 7f363f295b80 20 client.25618 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.680+0000 7f363f295b80 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x56209c2f48d0 0x56209c2f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.680+0000 7f363f295b80 4 client.25618 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.680+0000 7f363f295b80 1 -- 172.21.15.115:0/1789378199 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56209c202a80 con 0x56209c2f48d0 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.680+0000 7f3629d9b700 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x56209c2f48d0 0x56209c2f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.681+0000 7f3611ffb700 10 client.25618 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.681+0000 7f3629d9b700 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x56209c2f48d0 0x56209c2f6d10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35bec068a599d222 server_cookie=8a49604c057f135c in_seq=0 out_seq=0 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f3611ffb700 1 -- 172.21.15.115:0/1789378199 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56209c202a80 con 0x56209c2f48d0 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f3611ffb700 10 client.25618 handle_command_reply: tid=0 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f363f295b80 1 client.25618 shutdown 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f363f295b80 1 -- 172.21.15.115:0/1789378199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 msgr2=0x56209c2dc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f363f295b80 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2dc160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f361c01f8f0 tx=0x7f361c0047c0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f363f295b80 1 -- 172.21.15.115:0/1789378199 shutdown_connections 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f363f295b80 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x56209c2f48d0 0x56209c2f6d10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f363f295b80 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2dcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f3611ffb700 0 client.25618 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:54.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f363f295b80 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2dc160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f363f295b80 1 --2- 172.21.15.115:0/1789378199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1cfd00 0x56209c2dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.682+0000 7f363f295b80 1 -- 172.21.15.115:0/1789378199 >> 172.21.15.115:0/1789378199 conn(0x56209c158f70 msgr2=0x56209c2c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:54.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.683+0000 7f363f295b80 1 -- 172.21.15.115:0/1789378199 shutdown_connections 2024-05-09T05:39:54.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.683+0000 7f363f295b80 1 -- 172.21.15.115:0/1789378199 wait complete. 2024-05-09T05:39:54.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.683+0000 7f363f295b80 20 client.25618 trim_cache size 0 max 16384 2024-05-09T05:39:54.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.698+0000 7f363f295b80 1 Processor -- start 2024-05-09T05:39:54.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.698+0000 7f363f295b80 1 -- start start 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.698+0000 7f363f295b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.698+0000 7f363f295b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c1cfd00 0x56209c2db480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.698+0000 7f363f295b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c2d2400 0x56209c2dca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.698+0000 7f363f295b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ae210 con 0x56209c2d2400 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.698+0000 7f363f295b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1afd50 con 0x56209c1cfd00 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.698+0000 7f363f295b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ae8e0 con 0x56209c1554a0 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c2d2400 0x56209c2dca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c2d2400 0x56209c2dca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41696/0 (socket says 172.21.15.115:41696) 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362ad9d700 1 -- 172.21.15.115:0/2802919429 learned_addr learned my addr 172.21.15.115:0/2802919429 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362a59c700 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f3629d9b700 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c1cfd00 0x56209c2db480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362a59c700 1 -- 172.21.15.115:0/2802919429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c1cfd00 msgr2=0x56209c2db480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:54.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362a59c700 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c1cfd00 0x56209c2db480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362a59c700 1 -- 172.21.15.115:0/2802919429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c2d2400 msgr2=0x56209c2dca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:54.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362a59c700 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c2d2400 0x56209c2dca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362a59c700 1 -- 172.21.15.115:0/2802919429 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209c2dea00 con 0x56209c1554a0 2024-05-09T05:39:54.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362ad9d700 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c2d2400 0x56209c2dca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:54.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.699+0000 7f362a59c700 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2d9e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f361c028ce0 tx=0x7f361c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38df6886c31209a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:54.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.700+0000 7f3613fff700 1 -- 172.21.15.115:0/2802919429 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c039020 con 0x56209c1554a0 2024-05-09T05:39:54.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.700+0000 7f3613fff700 1 -- 172.21.15.115:0/2802919429 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f361c01d690 con 0x56209c1554a0 2024-05-09T05:39:54.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.700+0000 7f3613fff700 1 -- 172.21.15.115:0/2802919429 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c0286a0 con 0x56209c1554a0 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 -- 172.21.15.115:0/2802919429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 msgr2=0x56209c2d9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2d9e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f361c028ce0 tx=0x7f361c011ab0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 -- 172.21.15.115:0/2802919429 shutdown_connections 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c2d2400 0x56209c2dca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1554a0 0x56209c2d9e70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 --2- 172.21.15.115:0/2802919429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c1cfd00 0x56209c2db480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 -- 172.21.15.115:0/2802919429 >> 172.21.15.115:0/2802919429 conn(0x56209c158f70 msgr2=0x56209c2c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 -- 172.21.15.115:0/2802919429 shutdown_connections 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 -- 172.21.15.115:0/2802919429 wait complete. 2024-05-09T05:39:54.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 -- start start 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.701+0000 7f363f295b80 1 Processor -- start 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f363f295b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2ed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f363f295b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 0x56209c2fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f363f295b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f362a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f363f295b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1af680 con 0x56209c1554a0 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f363f295b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1aefb0 con 0x56209c2d2400 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f363f295b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ac1b0 con 0x56209c1cfd00 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f362a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2ed140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41712/0 (socket says 172.21.15.115:41712) 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f362a59c700 1 -- 172.21.15.115:0/459222785 learned_addr learned my addr 172.21.15.115:0/459222785 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f363f295b80 10 client.0 _ll_register_callbacks cb 0x56209c0666c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f362ad9d700 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f362a59c700 1 -- 172.21.15.115:0/459222785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 msgr2=0x56209c2fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:54.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f362a59c700 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 0x56209c2fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f362a59c700 1 -- 172.21.15.115:0/459222785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 msgr2=0x56209c2fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f362a59c700 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.702+0000 7f362a59c700 1 -- 172.21.15.115:0/459222785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209c2fc910 con 0x56209c1554a0 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.703+0000 7f362a59c700 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2ed140 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f361c028ea0 tx=0x7f361c01dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3164c996db9486 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.703+0000 7f3611ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.703+0000 7f3611ffb700 1 -- 172.21.15.115:0/459222785 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c039050 con 0x56209c1554a0 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.703+0000 7f3611ffb700 1 -- 172.21.15.115:0/459222785 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f361c028850 con 0x56209c1554a0 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.703+0000 7f363f295b80 1 -- 172.21.15.115:0/459222785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f361c01d690 con 0x56209c1554a0 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.703+0000 7f3611ffb700 1 -- 172.21.15.115:0/459222785 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f361c04c050 con 0x56209c1554a0 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.703+0000 7f363f295b80 10 client.16017 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:54.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.703+0000 7f363f295b80 1 -- 172.21.15.115:0/459222785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56209c35f060 con 0x56209c1554a0 2024-05-09T05:39:54.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.704+0000 7f3611ffb700 1 -- 172.21.15.115:0/459222785 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f361c01d690 con 0x56209c1554a0 2024-05-09T05:39:54.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.704+0000 7f3611ffb700 1 -- 172.21.15.115:0/459222785 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f361c047030 con 0x56209c1554a0 2024-05-09T05:39:54.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.704+0000 7f363f295b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:54.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.704+0000 7f363f295b80 10 client.16017 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.704+0000 7f363f295b80 20 client.16017 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.704+0000 7f363f295b80 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x56209c2f48d0 0x56209c3616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.704+0000 7f363f295b80 4 client.16017 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.704+0000 7f363f295b80 1 -- 172.21.15.115:0/459222785 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56209c1fcb20 con 0x56209c2f48d0 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.704+0000 7f3629d9b700 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x56209c2f48d0 0x56209c3616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f3611ffb700 10 client.16017 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f3629d9b700 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x56209c2f48d0 0x56209c3616c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=851dc44cb61037b3 server_cookie=963a05401d77aa23 in_seq=0 out_seq=0 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f3611ffb700 1 -- 172.21.15.115:0/459222785 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56209c1fcb20 con 0x56209c2f48d0 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f3611ffb700 10 client.16017 handle_command_reply: tid=0 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 client.16017 shutdown 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 -- 172.21.15.115:0/459222785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 msgr2=0x56209c2ed140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2ed140 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f361c028ea0 tx=0x7f361c01dbc0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 -- 172.21.15.115:0/459222785 shutdown_connections 2024-05-09T05:39:54.703 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:54.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x56209c2f48d0 0x56209c3616c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f3611ffb700 0 client.16017 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:39:54.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56209c2d2400 0x56209c2fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56209c1554a0 0x56209c2ed140 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 --2- 172.21.15.115:0/459222785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56209c1cfd00 0x56209c2fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 -- 172.21.15.115:0/459222785 >> 172.21.15.115:0/459222785 conn(0x56209c158f70 msgr2=0x56209c2db1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:54.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 -- 172.21.15.115:0/459222785 shutdown_connections 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 1 -- 172.21.15.115:0/459222785 wait complete. 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.705+0000 7f363f295b80 20 client.16017 trim_cache size 0 max 16384 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 -- 172.21.15.115:0/330157458 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36140b29f0 msgr2=0x7f36140b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36140b29f0 0x7f36140b4f00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f362c103b00 tx=0x7f361c009260 comp rx=0 tx=0).stop 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 -- 172.21.15.115:0/330157458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f362c1089f0 msgr2=0x7f362c1a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f362c1089f0 0x7f362c1a1750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f362400efc0 tx=0x7f362400c3a0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 -- 172.21.15.115:0/330157458 shutdown_connections 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36140b29f0 0x7f36140b4f00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f362c1089f0 0x7f362c1a1750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f362c1029c0 0x7f362c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 --2- 172.21.15.115:0/330157458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f362c103310 0x7f362c19d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.706+0000 7f363087f700 1 -- 172.21.15.115:0/330157458 >> 172.21.15.115:0/330157458 conn(0x7f362c0fe6f0 msgr2=0x7f362c105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.708+0000 7f363087f700 1 -- 172.21.15.115:0/330157458 shutdown_connections 2024-05-09T05:39:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:54.708+0000 7f363087f700 1 -- 172.21.15.115:0/330157458 wait complete. 2024-05-09T05:39:55.038 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:55 smithi115 ceph-mon[30349]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-05-09T05:39:55.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:55 smithi195 ceph-mon[33844]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-05-09T05:39:55.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:55 smithi161 ceph-mon[33805]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-05-09T05:39:55.546 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:39:57.030 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:57.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.109+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1350267542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee14107df0 msgr2=0x7fee14108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:57.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.109+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1350267542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee14107df0 0x7fee14108200 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fee04009a10 tx=0x7fee04009d20 comp rx=0 tx=0).stop 2024-05-09T05:39:57.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.109+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1350267542 shutdown_connections 2024-05-09T05:39:57.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.109+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1350267542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee14108740 0x7fee141100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.109+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1350267542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee141034c0 0x7fee141038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.109+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1350267542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee14107df0 0x7fee14108200 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.109+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1350267542 >> 172.21.15.115:0/1350267542 conn(0x7fee140fedc0 msgr2=0x7fee141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.109+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1350267542 shutdown_connections 2024-05-09T05:39:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.110+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1350267542 wait complete. 2024-05-09T05:39:57.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.111+0000 7fee1bfb3700 1 Processor -- start 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.111+0000 7fee1bfb3700 1 -- start start 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee1bfb3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee141034c0 0x7fee1419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee1bfb3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee14107df0 0x7fee1419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee19d4f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee141034c0 0x7fee1419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee19d4f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee141034c0 0x7fee1419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41730/0 (socket says 172.21.15.115:41730) 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee19d4f700 1 -- 172.21.15.115:0/1789215319 learned_addr learned my addr 172.21.15.115:0/1789215319 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee14108740 0x7fee14199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fee141132a0 con 0x7fee141034c0 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee14113120 con 0x7fee14107df0 2024-05-09T05:39:57.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee14113420 con 0x7fee14108740 2024-05-09T05:39:57.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.112+0000 7fee1954e700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee14107df0 0x7fee1419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:57.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.113+0000 7fee1a550700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee14108740 0x7fee14199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:57.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.113+0000 7fee1954e700 1 -- 172.21.15.115:0/1789215319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee14108740 msgr2=0x7fee14199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:57.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.113+0000 7fee1954e700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee14108740 0x7fee14199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:57.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.113+0000 7fee1954e700 1 -- 172.21.15.115:0/1789215319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee141034c0 msgr2=0x7fee1419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:57.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.113+0000 7fee1954e700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee141034c0 0x7fee1419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:57.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.113+0000 7fee1954e700 1 -- 172.21.15.115:0/1789215319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee14199fd0 con 0x7fee14107df0 2024-05-09T05:39:57.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.113+0000 7fee1954e700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee14107df0 0x7fee1419fc40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee14076ab0 tx=0x7fee0400f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:57.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.114+0000 7fee0affd700 1 -- 172.21.15.115:0/1789215319 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0400be70 con 0x7fee14107df0 2024-05-09T05:39:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.114+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1419a260 con 0x7fee14107df0 2024-05-09T05:39:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.114+0000 7fee0affd700 1 -- 172.21.15.115:0/1789215319 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee04004830 con 0x7fee14107df0 2024-05-09T05:39:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.114+0000 7fee0affd700 1 -- 172.21.15.115:0/1789215319 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04005280 con 0x7fee14107df0 2024-05-09T05:39:57.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.114+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1419a6e0 con 0x7fee14107df0 2024-05-09T05:39:57.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.116+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee140512c0 con 0x7fee14107df0 2024-05-09T05:39:57.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.118+0000 7fee0affd700 1 -- 172.21.15.115:0/1789215319 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fee04028060 con 0x7fee14107df0 2024-05-09T05:39:57.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.119+0000 7fee0affd700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fee000b2a70 0x7fee000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:57.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.119+0000 7fee0affd700 1 -- 172.21.15.115:0/1789215319 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fee040e2f10 con 0x7fee14107df0 2024-05-09T05:39:57.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.119+0000 7fee19d4f700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fee000b2a70 0x7fee000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:57.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.120+0000 7fee19d4f700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fee000b2a70 0x7fee000b4f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee1419aa60 tx=0x7fee10009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:57.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.124+0000 7fee0affd700 1 -- 172.21.15.115:0/1789215319 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee04013070 con 0x7fee14107df0 2024-05-09T05:39:57.320 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:57 smithi115 ceph-mon[30349]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:57.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:57 smithi195 ceph-mon[33844]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:57.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:57 smithi161 ceph-mon[33805]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T05:39:57.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.512+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee1410b1d0 con 0x7fee14107df0 2024-05-09T05:39:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.514+0000 7fee0affd700 1 -- 172.21.15.115:0/1789215319 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7fee04005420 con 0x7fee14107df0 2024-05-09T05:39:57.508 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:39:57.508 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:39:57.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.516+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fee000b2a70 msgr2=0x7fee000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:57.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.516+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fee000b2a70 0x7fee000b4f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee1419aa60 tx=0x7fee10009260 comp rx=0 tx=0).stop 2024-05-09T05:39:57.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.516+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee14107df0 msgr2=0x7fee1419fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:57.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.516+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee14107df0 0x7fee1419fc40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee14076ab0 tx=0x7fee0400f670 comp rx=0 tx=0).stop 2024-05-09T05:39:57.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.517+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 shutdown_connections 2024-05-09T05:39:57.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.517+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fee000b2a70 0x7fee000b4f80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:57.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.517+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee14108740 0x7fee14199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:57.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.517+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee141034c0 0x7fee1419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:57.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.517+0000 7fee1bfb3700 1 --2- 172.21.15.115:0/1789215319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee14107df0 0x7fee1419fc40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:57.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.517+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 >> 172.21.15.115:0/1789215319 conn(0x7fee140fedc0 msgr2=0x7fee14107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:57.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.517+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 shutdown_connections 2024-05-09T05:39:57.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:57.517+0000 7fee1bfb3700 1 -- 172.21.15.115:0/1789215319 wait complete. 2024-05-09T05:39:57.512 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:39:58.287 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:58 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1789215319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:58.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:58 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1789215319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:58.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:58 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1789215319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.747+0000 7f017639d640 1 -- 172.21.15.115:0/2788064219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01701089b0 msgr2=0x7f0170102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.747+0000 7f017639d640 1 --2- 172.21.15.115:0/2788064219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01701089b0 0x7f0170102bc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f016c009900 tx=0x7f016c02a410 comp rx=0 tx=0).stop 2024-05-09T05:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.747+0000 7f017639d640 1 -- 172.21.15.115:0/2788064219 shutdown_connections 2024-05-09T05:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.747+0000 7f017639d640 1 --2- 172.21.15.115:0/2788064219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01701037b0 0x7f017010f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.747+0000 7f017639d640 1 --2- 172.21.15.115:0/2788064219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0170108e50 0x7f0170103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.747+0000 7f017639d640 1 --2- 172.21.15.115:0/2788064219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01701089b0 0x7f0170102bc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.747+0000 7f017639d640 1 -- 172.21.15.115:0/2788064219 >> 172.21.15.115:0/2788064219 conn(0x7f01700fd960 msgr2=0x7f01700ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.747+0000 7f017639d640 1 -- 172.21.15.115:0/2788064219 shutdown_connections 2024-05-09T05:39:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.747+0000 7f017639d640 1 -- 172.21.15.115:0/2788064219 wait complete. 2024-05-09T05:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.748+0000 7f017639d640 1 Processor -- start 2024-05-09T05:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.748+0000 7f017639d640 1 -- start start 2024-05-09T05:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.748+0000 7f017639d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01701089b0 0x7f017019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.748+0000 7f017639d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0170108e50 0x7f017019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.748+0000 7f017639d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017019ea30 0x7f017019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.748+0000 7f017639d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0170112a50 con 0x7f0170108e50 2024-05-09T05:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.748+0000 7f017639d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f01701128d0 con 0x7f01701089b0 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.748+0000 7f017639d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0170112bd0 con 0x7f017019ea30 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0174b9a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0170108e50 0x7f017019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0174b9a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0170108e50 0x7f017019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41744/0 (socket says 172.21.15.115:41744) 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0174b9a640 1 -- 172.21.15.115:0/3335639562 learned_addr learned my addr 172.21.15.115:0/3335639562 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0175b9c640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017019ea30 0x7f017019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f017539b640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01701089b0 0x7f017019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0174b9a640 1 -- 172.21.15.115:0/3335639562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017019ea30 msgr2=0x7f017019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0174b9a640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017019ea30 0x7f017019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0174b9a640 1 -- 172.21.15.115:0/3335639562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01701089b0 msgr2=0x7f017019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0174b9a640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01701089b0 0x7f017019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0174b9a640 1 -- 172.21.15.115:0/3335639562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017019f480 con 0x7f0170108e50 2024-05-09T05:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0175b9c640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017019ea30 0x7f017019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.749+0000 7f0174b9a640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0170108e50 0x7f017019a770 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f016000d740 tx=0x7f016000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.750+0000 7f01667fc640 1 -- 172.21.15.115:0/3335639562 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160017070 con 0x7f0170108e50 2024-05-09T05:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.750+0000 7f01667fc640 1 -- 172.21.15.115:0/3335639562 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f016000de50 con 0x7f0170108e50 2024-05-09T05:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.750+0000 7f017639d640 1 -- 172.21.15.115:0/3335639562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017019f770 con 0x7f0170108e50 2024-05-09T05:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.750+0000 7f01667fc640 1 -- 172.21.15.115:0/3335639562 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016000f620 con 0x7f0170108e50 2024-05-09T05:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.750+0000 7f017639d640 1 -- 172.21.15.115:0/3335639562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01701a6fd0 con 0x7f0170108e50 2024-05-09T05:39:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.753+0000 7f01667fc640 1 -- 172.21.15.115:0/3335639562 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f016001a070 con 0x7f0170108e50 2024-05-09T05:39:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017e587740 1 Processor -- start 2024-05-09T05:39:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017e587740 1 -- start start 2024-05-09T05:39:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f01667fc640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f015c0b29d0 0x7f015c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017e587740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ac7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017e587740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ac7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f01667fc640 1 -- 172.21.15.115:0/3335639562 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f01600e2210 con 0x7f0170108e50 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017e587740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ac82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017e587740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555702a2c7c0 con 0x5557027034d0 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017e587740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5557029c80e0 con 0x5557027241b0 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017e587740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555702a28780 con 0x5557026fb2f0 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017539b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ac7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017539b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ac7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33228/0 (socket says 172.21.15.115:33228) 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.754+0000 7f017539b640 1 -- 172.21.15.115:0/712854548 learned_addr learned my addr 172.21.15.115:0/712854548 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f0175b9c640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f015c0b29d0 0x7f015c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f0174b9a640 1 --2- 172.21.15.115:0/712854548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ac7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f017539b640 1 -- 172.21.15.115:0/712854548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 msgr2=0x555702ac7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f017539b640 1 --2- 172.21.15.115:0/712854548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ac7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f017539b640 1 -- 172.21.15.115:0/712854548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 msgr2=0x555702ac82c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:39:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f017539b640 1 --2- 172.21.15.115:0/712854548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ac82c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f017539b640 1 -- 172.21.15.115:0/712854548 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557029f17c0 con 0x5557027241b0 2024-05-09T05:39:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f0175b9c640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f015c0b29d0 0x7f015c0b4ee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0168005fd0 tx=0x7f01680096a0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f017539b640 1 --2- 172.21.15.115:0/712854548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ac7840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f016c0057c0 tx=0x7f016c031ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c8fe835259a21e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.755+0000 7f015b7fe640 1 -- 172.21.15.115:0/712854548 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c040070 con 0x5557027241b0 2024-05-09T05:39:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.756+0000 7f015b7fe640 1 -- 172.21.15.115:0/712854548 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f016c031e50 con 0x5557027241b0 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.756+0000 7f015b7fe640 1 -- 172.21.15.115:0/712854548 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c035040 con 0x5557027241b0 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 -- 172.21.15.115:0/712854548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 msgr2=0x555702ac7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 --2- 172.21.15.115:0/712854548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ac7840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f016c0057c0 tx=0x7f016c031ba0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 -- 172.21.15.115:0/712854548 shutdown_connections 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 --2- 172.21.15.115:0/712854548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ac82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 --2- 172.21.15.115:0/712854548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ac7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 --2- 172.21.15.115:0/712854548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ac7840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 -- 172.21.15.115:0/712854548 >> 172.21.15.115:0/712854548 conn(0x555702718300 msgr2=0x555702ac3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 -- 172.21.15.115:0/712854548 shutdown_connections 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 -- 172.21.15.115:0/712854548 wait complete. 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 -- start start 2024-05-09T05:39:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.757+0000 7f017e587740 1 Processor -- start 2024-05-09T05:39:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017e587740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557026fb2f0 0x555702ad39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017e587740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017e587740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557027241b0 0x555702ad1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017e587740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555702ad4910 con 0x5557027034d0 2024-05-09T05:39:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017e587740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5557029b39c0 con 0x5557026fb2f0 2024-05-09T05:39:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017e587740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555702ad4a90 con 0x5557027241b0 2024-05-09T05:39:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017539b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557026fb2f0 0x555702ad39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017e587740 10 client.0 _ll_register_callbacks cb 0x5557029e0020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017539b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557026fb2f0 0x555702ad39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33238/0 (socket says 172.21.15.115:33238) 2024-05-09T05:39:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f017539b640 1 -- 172.21.15.115:0/4050375030 learned_addr learned my addr 172.21.15.115:0/4050375030 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.758+0000 7f0174b9a640 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f0175b9c640 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557027241b0 0x555702ad1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f017539b640 1 -- 172.21.15.115:0/4050375030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557027241b0 msgr2=0x555702ad1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f017539b640 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557027241b0 0x555702ad1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f017539b640 1 -- 172.21.15.115:0/4050375030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 msgr2=0x555702ad3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f017539b640 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f017539b640 1 -- 172.21.15.115:0/4050375030 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557029da1f0 con 0x5557026fb2f0 2024-05-09T05:39:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f0174b9a640 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f017539b640 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557026fb2f0 0x555702ad39e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f016c02a930 tx=0x7f016c031630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e50d53e807df67f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f0158ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f0158ff9640 1 -- 172.21.15.115:0/4050375030 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c040070 con 0x5557026fb2f0 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f0158ff9640 1 -- 172.21.15.115:0/4050375030 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f016c03c850 con 0x5557026fb2f0 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.759+0000 7f017e587740 1 -- 172.21.15.115:0/4050375030 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f016c031e50 con 0x5557026fb2f0 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.760+0000 7f0158ff9640 1 -- 172.21.15.115:0/4050375030 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f016c035040 con 0x5557026fb2f0 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.760+0000 7f017e587740 10 client.25565 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.760+0000 7f017e587740 1 -- 172.21.15.115:0/4050375030 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55570278f790 con 0x5557026fb2f0 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.760+0000 7f0158ff9640 1 -- 172.21.15.115:0/4050375030 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f016c04e400 con 0x5557026fb2f0 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.761+0000 7f0158ff9640 1 -- 172.21.15.115:0/4050375030 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f016c044400 con 0x5557026fb2f0 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.761+0000 7f017e587740 10 client.25565 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.761+0000 7f017e587740 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.761+0000 7f017e587740 20 client.25565 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.761+0000 7f017e587740 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5557026e2d80 0x555702ae4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.761+0000 7f017e587740 4 client.25565 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.761+0000 7f017e587740 1 -- 172.21.15.115:0/4050375030 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f016c031e50 con 0x5557026e2d80 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.761+0000 7f0174b9a640 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5557026e2d80 0x555702ae4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.762+0000 7f0174b9a640 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5557026e2d80 0x555702ae4200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16f07757aebfb3f server_cookie=c47014c6a30aeedd in_seq=0 out_seq=0 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.762+0000 7f0158ff9640 10 client.25565 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f0158ff9640 1 -- 172.21.15.115:0/4050375030 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f016c031e50 con 0x5557026e2d80 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f0158ff9640 10 client.25565 handle_command_reply: tid=0 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 client.25565 shutdown 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 -- 172.21.15.115:0/4050375030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557026fb2f0 msgr2=0x555702ad39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557026fb2f0 0x555702ad39e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f016c02a930 tx=0x7f016c031630 comp rx=0 tx=0).stop 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 -- 172.21.15.115:0/4050375030 shutdown_connections 2024-05-09T05:39:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5557026e2d80 0x555702ae4200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557027241b0 0x555702ad1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 --2- 172.21.15.115:0/4050375030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557026fb2f0 0x555702ad39e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 -- 172.21.15.115:0/4050375030 >> 172.21.15.115:0/4050375030 conn(0x555702718300 msgr2=0x555702ac2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f0158ff9640 0 client.25565 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 -- 172.21.15.115:0/4050375030 shutdown_connections 2024-05-09T05:39:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 1 -- 172.21.15.115:0/4050375030 wait complete. 2024-05-09T05:39:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.763+0000 7f017e587740 20 client.25565 trim_cache size 0 max 16384 2024-05-09T05:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.778+0000 7f017e587740 1 Processor -- start 2024-05-09T05:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.778+0000 7f017e587740 1 -- start start 2024-05-09T05:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.778+0000 7f017e587740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ad9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.778+0000 7f017e587740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027034d0 0x555702ada260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f017e587740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027241b0 0x555702ac2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f017e587740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01701aec60 con 0x5557027241b0 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f017e587740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f01701b07a0 con 0x5557027034d0 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f017e587740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01701af330 con 0x5557026fb2f0 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0175b9c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027241b0 0x555702ac2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0175b9c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027241b0 0x555702ac2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41770/0 (socket says 172.21.15.115:41770) 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0175b9c640 1 -- 172.21.15.115:0/3192456735 learned_addr learned my addr 172.21.15.115:0/3192456735 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0174b9a640 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027034d0 0x555702ada260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f017539b640 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ad9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0175b9c640 1 -- 172.21.15.115:0/3192456735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 msgr2=0x555702ad9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0175b9c640 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ad9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0175b9c640 1 -- 172.21.15.115:0/3192456735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027034d0 msgr2=0x555702ada260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0175b9c640 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027034d0 0x555702ada260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0175b9c640 1 -- 172.21.15.115:0/3192456735 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557028065f0 con 0x5557027241b0 2024-05-09T05:39:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f017539b640 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ad9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.779+0000 7f0175b9c640 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027241b0 0x555702ac2860 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0168010c30 tx=0x7f0168013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05fea3cf09a6346 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.780+0000 7f015affd640 1 -- 172.21.15.115:0/3192456735 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168011cc0 con 0x5557027241b0 2024-05-09T05:39:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.780+0000 7f015affd640 1 -- 172.21.15.115:0/3192456735 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0168023070 con 0x5557027241b0 2024-05-09T05:39:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.780+0000 7f015affd640 1 -- 172.21.15.115:0/3192456735 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168004770 con 0x5557027241b0 2024-05-09T05:39:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 -- 172.21.15.115:0/3192456735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027241b0 msgr2=0x555702ac2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027241b0 0x555702ac2860 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0168010c30 tx=0x7f0168013000 comp rx=0 tx=0).stop 2024-05-09T05:39:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 -- 172.21.15.115:0/3192456735 shutdown_connections 2024-05-09T05:39:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027241b0 0x555702ac2860 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027034d0 0x555702ada260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 --2- 172.21.15.115:0/3192456735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ad9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 -- 172.21.15.115:0/3192456735 >> 172.21.15.115:0/3192456735 conn(0x555702718300 msgr2=0x555702ad06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 -- 172.21.15.115:0/3192456735 shutdown_connections 2024-05-09T05:39:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 -- 172.21.15.115:0/3192456735 wait complete. 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 -- start start 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.781+0000 7f017e587740 1 Processor -- start 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f017e587740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ada2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f017e587740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f017e587740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ad6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f017e587740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f01701b00d0 con 0x5557027034d0 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f017e587740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f01701afa00 con 0x5557027241b0 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f017e587740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01701acc40 con 0x5557026fb2f0 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f017e587740 10 client.0 _ll_register_callbacks cb 0x7f0160010f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0174b9a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0174b9a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41772/0 (socket says 172.21.15.115:41772) 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0174b9a640 1 -- 172.21.15.115:0/2716361304 learned_addr learned my addr 172.21.15.115:0/2716361304 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f017539b640 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ada2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0174b9a640 1 -- 172.21.15.115:0/2716361304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 msgr2=0x555702ada2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0174b9a640 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ada2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0174b9a640 1 -- 172.21.15.115:0/2716361304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 msgr2=0x555702ad6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0175b9c640 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ad6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0174b9a640 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ad6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0174b9a640 1 -- 172.21.15.115:0/2716361304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557029f0ed0 con 0x5557027034d0 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f017539b640 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ada2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.782+0000 7f0175b9c640 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ad6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f0174b9a640 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad6630 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f01600a64d0 tx=0x7f01600a8a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396c10b4e5ca3185 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f0158ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f0158ff9640 1 -- 172.21.15.115:0/2716361304 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160021040 con 0x5557027034d0 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f0158ff9640 1 -- 172.21.15.115:0/2716361304 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f01600a54f0 con 0x5557027034d0 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f017e587740 1 -- 172.21.15.115:0/2716361304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0168023070 con 0x5557027034d0 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f0158ff9640 1 -- 172.21.15.115:0/2716361304 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f01600ad150 con 0x5557027034d0 2024-05-09T05:39:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f017e587740 10 client.16038 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f017e587740 1 -- 172.21.15.115:0/2716361304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555702b4b2e0 con 0x5557027034d0 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f0158ff9640 1 -- 172.21.15.115:0/2716361304 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0168023070 con 0x5557027034d0 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.783+0000 7f0158ff9640 1 -- 172.21.15.115:0/2716361304 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f01600a4d60 con 0x5557027034d0 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.784+0000 7f017e587740 10 client.16038 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.784+0000 7f017e587740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.784+0000 7f017e587740 20 client.16038 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.784+0000 7f017e587740 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5557026b81c0 0x555702b4da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.784+0000 7f017e587740 4 client.16038 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.784+0000 7f017e587740 1 -- 172.21.15.115:0/2716361304 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555702a3a3d0 con 0x5557026b81c0 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.784+0000 7f017539b640 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5557026b81c0 0x555702b4da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.784+0000 7f0158ff9640 10 client.16038 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.784+0000 7f017539b640 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5557026b81c0 0x555702b4da20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e149aeb1d3d6636 server_cookie=12e0cb99fd2afd88 in_seq=0 out_seq=0 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.788+0000 7f0158ff9640 1 -- 172.21.15.115:0/2716361304 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555702a3a3d0 con 0x5557026b81c0 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.788+0000 7f0158ff9640 10 client.16038 handle_command_reply: tid=0 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.788+0000 7f017e587740 1 client.16038 shutdown 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.788+0000 7f017e587740 1 -- 172.21.15.115:0/2716361304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 msgr2=0x555702ad6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.788+0000 7f017e587740 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad6630 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f01600a64d0 tx=0x7f01600a8a90 comp rx=0 tx=0).stop 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017e587740 1 -- 172.21.15.115:0/2716361304 shutdown_connections 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017e587740 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557027241b0 0x555702ad6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017e587740 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557027034d0 0x555702ad6630 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017e587740 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5557026b81c0 0x555702b4da20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f0158ff9640 0 client.16038 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017e587740 1 --2- 172.21.15.115:0/2716361304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557026fb2f0 0x555702ada2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017e587740 1 -- 172.21.15.115:0/2716361304 >> 172.21.15.115:0/2716361304 conn(0x555702718300 msgr2=0x555702ad0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017e587740 1 -- 172.21.15.115:0/2716361304 shutdown_connections 2024-05-09T05:39:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017e587740 1 -- 172.21.15.115:0/2716361304 wait complete. 2024-05-09T05:39:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017e587740 20 client.16038 trim_cache size 0 max 16384 2024-05-09T05:39:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:39:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:39:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:39:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017639d640 1 -- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f015c0b29d0 msgr2=0x7f015c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017639d640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f015c0b29d0 0x7f015c0b4ee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0168005fd0 tx=0x7f01680096a0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017639d640 1 -- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0170108e50 msgr2=0x7f017019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.789+0000 7f017639d640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0170108e50 0x7f017019a770 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f016000d740 tx=0x7f016000dc00 comp rx=0 tx=0).stop 2024-05-09T05:39:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.790+0000 7f017639d640 1 -- 172.21.15.115:0/3335639562 shutdown_connections 2024-05-09T05:39:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.790+0000 7f017639d640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f015c0b29d0 0x7f015c0b4ee0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.790+0000 7f017639d640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f017019ea30 0x7f017019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.790+0000 7f017639d640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0170108e50 0x7f017019a770 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.790+0000 7f017639d640 1 --2- 172.21.15.115:0/3335639562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01701089b0 0x7f017019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.790+0000 7f017639d640 1 -- 172.21.15.115:0/3335639562 >> 172.21.15.115:0/3335639562 conn(0x7f01700fd960 msgr2=0x7f01700fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.790+0000 7f017639d640 1 -- 172.21.15.115:0/3335639562 shutdown_connections 2024-05-09T05:39:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:39:58.790+0000 7f017639d640 1 -- 172.21.15.115:0/3335639562 wait complete. 2024-05-09T05:39:59.015 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 9c695dab --release --await' 2024-05-09T05:39:59.015 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id 9c695dab --release --await 2024-05-09T05:39:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:39:59 smithi115 ceph-mon[30349]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-09T05:39:59.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:39:59 smithi195 ceph-mon[33844]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-09T05:39:59.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:39:59 smithi161 ceph-mon[33805]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-09T05:39:59.637 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:39:59.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.673+0000 7f8eda078700 1 -- 172.21.15.115:0/2253169498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed410e8e0 msgr2=0x7f8ed4110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:59.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.673+0000 7f8eda078700 1 --2- 172.21.15.115:0/2253169498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed410e8e0 0x7f8ed4110dc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b0a0 tx=0x7f8ecc00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.674+0000 7f8eda078700 1 -- 172.21.15.115:0/2253169498 shutdown_connections 2024-05-09T05:39:59.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.674+0000 7f8eda078700 1 --2- 172.21.15.115:0/2253169498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed410e8e0 0x7f8ed4110dc0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.674+0000 7f8eda078700 1 --2- 172.21.15.115:0/2253169498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4105500 0x7f8ed4105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.674+0000 7f8eda078700 1 --2- 172.21.15.115:0/2253169498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed4105e50 0x7f8ed410e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.674+0000 7f8eda078700 1 -- 172.21.15.115:0/2253169498 >> 172.21.15.115:0/2253169498 conn(0x7f8ed40fedc0 msgr2=0x7f8ed41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:59.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.674+0000 7f8eda078700 1 -- 172.21.15.115:0/2253169498 shutdown_connections 2024-05-09T05:39:59.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.674+0000 7f8eda078700 1 -- 172.21.15.115:0/2253169498 wait complete. 2024-05-09T05:39:59.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.675+0000 7f8eda078700 1 Processor -- start 2024-05-09T05:39:59.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.675+0000 7f8eda078700 1 -- start start 2024-05-09T05:39:59.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8eda078700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4105500 0x7f8ed4104d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8eda078700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed4105e50 0x7f8ed41031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8eda078700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed410e8e0 0x7f8ed41037d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8eda078700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4114340 con 0x7f8ed4105e50 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8eda078700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41141c0 con 0x7f8ed410e8e0 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8eda078700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41144c0 con 0x7f8ed4105500 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8ed3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed410e8e0 0x7f8ed41037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8ed3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed410e8e0 0x7f8ed41037d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33278/0 (socket says 172.21.15.115:33278) 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8ed3fff700 1 -- 172.21.15.115:0/303769037 learned_addr learned my addr 172.21.15.115:0/303769037 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4105500 0x7f8ed4104d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.676+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed4105e50 0x7f8ed41031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8ed37fe700 1 -- 172.21.15.115:0/303769037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed410e8e0 msgr2=0x7f8ed41037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed410e8e0 0x7f8ed41037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8ed37fe700 1 -- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed4105e50 msgr2=0x7f8ed41031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:59.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed4105e50 0x7f8ed41031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8ed37fe700 1 -- 172.21.15.115:0/303769037 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed4104000 con 0x7f8ed4105500 2024-05-09T05:39:59.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4105500 0x7f8ed4104d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400e9f0 tx=0x7f8ec400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:59.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8ed0ff9700 1 -- 172.21.15.115:0/303769037 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400c9c0 con 0x7f8ed4105500 2024-05-09T05:39:59.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8eda078700 1 -- 172.21.15.115:0/303769037 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed41ad550 con 0x7f8ed4105500 2024-05-09T05:39:59.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8eda078700 1 -- 172.21.15.115:0/303769037 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41adab0 con 0x7f8ed4105500 2024-05-09T05:39:59.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8ed0ff9700 1 -- 172.21.15.115:0/303769037 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec4010430 con 0x7f8ed4105500 2024-05-09T05:39:59.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.677+0000 7f8ed0ff9700 1 -- 172.21.15.115:0/303769037 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4026060 con 0x7f8ed4105500 2024-05-09T05:39:59.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.682+0000 7f8ed0ff9700 1 -- 172.21.15.115:0/303769037 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f8ec4018680 con 0x7f8ed4105500 2024-05-09T05:39:59.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.682+0000 7f8ee8a8eb80 1 Processor -- start 2024-05-09T05:39:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.682+0000 7f8ee8a8eb80 1 -- start start 2024-05-09T05:39:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb20240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb20780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb1a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c48ba3d0a0 con 0x55c48ba17d00 2024-05-09T05:39:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c48b9e29a0 con 0x55c48b99d4a0 2024-05-09T05:39:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c48ba39060 con 0x55c48bb1a400 2024-05-09T05:39:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb20780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb20780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33296/0 (socket says 172.21.15.115:33296) 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed37fe700 1 -- 172.21.15.115:0/160925782 learned_addr learned my addr 172.21.15.115:0/160925782 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed3fff700 1 --2- 172.21.15.115:0/160925782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb20240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed0ff9700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8ebc0b2af0 0x7f8ebc0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/160925782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb1a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed37fe700 1 -- 172.21.15.115:0/160925782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 msgr2=0x55c48bb1a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/160925782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb1a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed37fe700 1 -- 172.21.15.115:0/160925782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 msgr2=0x55c48bb20240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/160925782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb20240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.683+0000 7f8ed37fe700 1 -- 172.21.15.115:0/160925782 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c48b9d3500 con 0x55c48b99d4a0 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.684+0000 7f8ed3fff700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8ebc0b2af0 0x7f8ebc0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.684+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/160925782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb20780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40dfc70 tx=0x7f8ec40761f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab5b36a0b554a8da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.684+0000 7f8ed0ff9700 1 -- 172.21.15.115:0/303769037 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f8ec40e2980 con 0x7f8ed4105500 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.684+0000 7f8eb9ffb700 1 -- 172.21.15.115:0/160925782 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40a9690 con 0x55c48b99d4a0 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.684+0000 7f8eb9ffb700 1 -- 172.21.15.115:0/160925782 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec40b2070 con 0x55c48b99d4a0 2024-05-09T05:39:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.684+0000 7f8eb9ffb700 1 -- 172.21.15.115:0/160925782 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4076b10 con 0x55c48b99d4a0 2024-05-09T05:39:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.685+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/160925782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 msgr2=0x55c48bb20780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.685+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/160925782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb20780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40dfc70 tx=0x7f8ec40761f0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.685+0000 7f8ed3fff700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8ebc0b2af0 0x7f8ebc0b5000 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b780 tx=0x7f8ecc00b3b0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.685+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/160925782 shutdown_connections 2024-05-09T05:39:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.685+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/160925782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb1a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.685+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/160925782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb20240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.685+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/160925782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb20780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.685+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/160925782 >> 172.21.15.115:0/160925782 conn(0x55c48b9a0f70 msgr2=0x55c48bb15220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.686+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/160925782 shutdown_connections 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.686+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/160925782 wait complete. 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.686+0000 7f8ee8a8eb80 1 -- start start 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.686+0000 7f8ee8a8eb80 1 Processor -- start 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb23f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48ba17d00 0x55c48bb244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48bb1a400 0x55c48bb24a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c48bb1b100 con 0x55c48b99d4a0 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c48b9d7b20 con 0x55c48bb1a400 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c48bb30230 con 0x55c48ba17d00 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ed37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb23f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ee8a8eb80 10 client.0 _ll_register_callbacks cb 0x55c48b8ae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ed37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb23f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41820/0 (socket says 172.21.15.115:41820) 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ed37fe700 1 -- 172.21.15.115:0/2379260635 learned_addr learned my addr 172.21.15.115:0/2379260635 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ed3fff700 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48bb1a400 0x55c48bb24a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.687+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48ba17d00 0x55c48bb244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.688+0000 7f8ed37fe700 1 -- 172.21.15.115:0/2379260635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48ba17d00 msgr2=0x55c48bb244d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.688+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48ba17d00 0x55c48bb244d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.688+0000 7f8ed37fe700 1 -- 172.21.15.115:0/2379260635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48bb1a400 msgr2=0x55c48bb24a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.688+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48bb1a400 0x55c48bb24a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.688+0000 7f8ed37fe700 1 -- 172.21.15.115:0/2379260635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c48b8b8490 con 0x55c48b99d4a0 2024-05-09T05:39:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.688+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb23f90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40a95c0 tx=0x7f8ec40df4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e666fa134e1377 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.688+0000 7f8eb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:59.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.688+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/2379260635 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4077100 con 0x55c48b99d4a0 2024-05-09T05:39:59.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.688+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/2379260635 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f8ec4075fc0 con 0x55c48b99d4a0 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.689+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/2379260635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec40b2070 con 0x55c48b99d4a0 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.689+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/2379260635 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ec40772f0 con 0x55c48b99d4a0 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.689+0000 7f8ee8a8eb80 10 client.16050 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.689+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/2379260635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c48bb31aa0 con 0x55c48b99d4a0 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.689+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/2379260635 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec40b2070 con 0x55c48b99d4a0 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.689+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/2379260635 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec40dffb0 con 0x55c48b99d4a0 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.690+0000 7f8ee8a8eb80 10 client.16050 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.690+0000 7f8ee8a8eb80 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.690+0000 7f8ee8a8eb80 20 client.16050 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.690+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c48bb33f70 0x55c48bb36390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.690+0000 7f8ee8a8eb80 4 client.16050 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.690+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/2379260635 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c48ba4aa80 con 0x55c48bb33f70 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.690+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c48bb33f70 0x55c48bb36390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.691+0000 7f8eb2ffd700 10 client.16050 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.691+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c48bb33f70 0x55c48bb36390 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620124c8ea28e83a server_cookie=530d9388f46d1c64 in_seq=0 out_seq=0 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.691+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/2379260635 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c48ba4aa80 con 0x55c48bb33f70 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.691+0000 7f8eb2ffd700 10 client.16050 handle_command_reply: tid=0 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.691+0000 7f8ee8a8eb80 1 client.16050 shutdown 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/2379260635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 msgr2=0x55c48bb23f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb23f90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40a95c0 tx=0x7f8ec40df4a0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/2379260635 shutdown_connections 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48bb1a400 0x55c48bb24a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c48bb33f70 0x55c48bb36390 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb23f90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/2379260635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48ba17d00 0x55c48bb244d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8eb2ffd700 0 client.16050 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/2379260635 >> 172.21.15.115:0/2379260635 conn(0x55c48b9a0f70 msgr2=0x55c48bb0eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/2379260635 shutdown_connections 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.692+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/2379260635 wait complete. 2024-05-09T05:39:59.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.693+0000 7f8ee8a8eb80 20 client.16050 trim_cache size 0 max 16384 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.708+0000 7f8ee8a8eb80 1 Processor -- start 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.708+0000 7f8ee8a8eb80 1 -- start start 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb1f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb15330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb22f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec000eab0 con 0x55c48ba17d00 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec00105f0 con 0x55c48b99d4a0 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec000f180 con 0x55c48bb1a400 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb15330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb15330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41830/0 (socket says 172.21.15.115:41830) 2024-05-09T05:39:59.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed2ffd700 1 -- 172.21.15.115:0/775358685 learned_addr learned my addr 172.21.15.115:0/775358685 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb1f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed3fff700 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb22f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed2ffd700 1 -- 172.21.15.115:0/775358685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 msgr2=0x55c48bb22f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb22f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed2ffd700 1 -- 172.21.15.115:0/775358685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 msgr2=0x55c48bb1f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb1f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed2ffd700 1 -- 172.21.15.115:0/775358685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c48bb20af0 con 0x55c48ba17d00 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.709+0000 7f8ed3fff700 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb22f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.710+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb15330 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55c48b9e29a0 tx=0x7f8ec8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30711c33e2217805 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.710+0000 7f8eb97fa700 1 -- 172.21.15.115:0/775358685 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8022030 con 0x55c48ba17d00 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.710+0000 7f8eb97fa700 1 -- 172.21.15.115:0/775358685 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec8004cc0 con 0x55c48ba17d00 2024-05-09T05:39:59.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.710+0000 7f8eb97fa700 1 -- 172.21.15.115:0/775358685 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec801d410 con 0x55c48ba17d00 2024-05-09T05:39:59.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/775358685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 msgr2=0x55c48bb15330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb15330 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55c48b9e29a0 tx=0x7f8ec8009260 comp rx=0 tx=0).stop 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/775358685 shutdown_connections 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48b99d4a0 0x55c48bb1f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb22f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/775358685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48ba17d00 0x55c48bb15330 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/775358685 >> 172.21.15.115:0/775358685 conn(0x55c48b9a0f70 msgr2=0x55c48bb22880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/775358685 shutdown_connections 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/775358685 wait complete. 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.711+0000 7f8ee8a8eb80 1 -- start start 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ee8a8eb80 1 Processor -- start 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48ba17d00 0x55c48bb38040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ee8a8eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb38580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ed37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ed37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb395d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41836/0 (socket says 172.21.15.115:41836) 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ee8a8eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec000ff20 con 0x55c48b99d4a0 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ed37fe700 1 -- 172.21.15.115:0/896934703 learned_addr learned my addr 172.21.15.115:0/896934703 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec000f850 con 0x55c48ba17d00 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec000ca50 con 0x55c48bb1a400 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.712+0000 7f8ee8a8eb80 10 client.0 _ll_register_callbacks cb 0x55c48b8ae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48ba17d00 0x55c48bb38040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8ed3fff700 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb38580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8ed37fe700 1 -- 172.21.15.115:0/896934703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 msgr2=0x55c48bb38580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb38580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8ed37fe700 1 -- 172.21.15.115:0/896934703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48ba17d00 msgr2=0x55c48bb38040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48ba17d00 0x55c48bb38040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8ed37fe700 1 -- 172.21.15.115:0/896934703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c48bb39df0 con 0x55c48b99d4a0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8ed3fff700 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb38580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8ed37fe700 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb395d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40a95c0 tx=0x7f8ec40b2870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1fd517e943af0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8eb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/896934703 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec406c230 con 0x55c48b99d4a0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.713+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/896934703 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f8ec40a5230 con 0x55c48b99d4a0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec8004cc0 con 0x55c48b99d4a0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/896934703 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ec406d180 con 0x55c48b99d4a0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8ee8a8eb80 10 client.16056 fetch_fsmap learned FSMap version 30 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c48bb9df80 con 0x55c48b99d4a0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/896934703 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec8004cc0 con 0x55c48b99d4a0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/896934703 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec40b2290 con 0x55c48b99d4a0 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8ee8a8eb80 10 client.16056 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8ee8a8eb80 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:39:59.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8ee8a8eb80 20 client.16056 populate_metadata read hostname 'smithi115' 2024-05-09T05:39:59.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.714+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c48bb9e2b0 0x55c48bba06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:39:59.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.715+0000 7f8ee8a8eb80 4 client.16056 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:39:59.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.715+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c48ba44b20 con 0x55c48bb9e2b0 2024-05-09T05:39:59.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.715+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c48bb9e2b0 0x55c48bba06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.715+0000 7f8eb2ffd700 10 client.16056 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.715+0000 7f8ed2ffd700 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c48bb9e2b0 0x55c48bba06d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3c81ac2bd92e57 server_cookie=5cc1d108fdff6d58 in_seq=0 out_seq=0 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8eb2ffd700 1 -- 172.21.15.115:0/896934703 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c48ba44b20 con 0x55c48bb9e2b0 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8eb2ffd700 10 client.16056 handle_command_reply: tid=0 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8ee8a8eb80 1 client.16056 shutdown 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 msgr2=0x55c48bb395d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb395d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40a95c0 tx=0x7f8ec40b2870 comp rx=0 tx=0).stop 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 shutdown_connections 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c48bb9e2b0 0x55c48bba06d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c48bb1a400 0x55c48bb38580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c48b99d4a0 0x55c48bb395d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8ee8a8eb80 1 --2- 172.21.15.115:0/896934703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c48ba17d00 0x55c48bb38040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 >> 172.21.15.115:0/896934703 conn(0x55c48b9a0f70 msgr2=0x55c48bb1e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:59.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.716+0000 7f8eb2ffd700 0 client.16056 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:39:59.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.717+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 shutdown_connections 2024-05-09T05:39:59.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.717+0000 7f8ee8a8eb80 1 -- 172.21.15.115:0/896934703 wait complete. 2024-05-09T05:39:59.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.717+0000 7f8ee8a8eb80 20 client.16056 trim_cache size 0 max 16384 2024-05-09T05:39:59.715 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:39:59.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.719+0000 7f8eda078700 1 -- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8ebc0b2af0 msgr2=0x7f8ebc0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:59.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.719+0000 7f8eda078700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8ebc0b2af0 0x7f8ebc0b5000 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b780 tx=0x7f8ecc00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.719+0000 7f8eda078700 1 -- 172.21.15.115:0/303769037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4105500 msgr2=0x7f8ed4104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:39:59.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.719+0000 7f8eda078700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4105500 0x7f8ed4104d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400e9f0 tx=0x7f8ec400ed00 comp rx=0 tx=0).stop 2024-05-09T05:39:59.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.720+0000 7f8eda078700 1 -- 172.21.15.115:0/303769037 shutdown_connections 2024-05-09T05:39:59.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.720+0000 7f8eda078700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8ebc0b2af0 0x7f8ebc0b5000 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.720+0000 7f8eda078700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed410e8e0 0x7f8ed41037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.720+0000 7f8eda078700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4105500 0x7f8ed4104d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.720+0000 7f8eda078700 1 --2- 172.21.15.115:0/303769037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed4105e50 0x7f8ed41031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:39:59.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.720+0000 7f8eda078700 1 -- 172.21.15.115:0/303769037 >> 172.21.15.115:0/303769037 conn(0x7f8ed40fedc0 msgr2=0x7f8ed40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:39:59.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.720+0000 7f8eda078700 1 -- 172.21.15.115:0/303769037 shutdown_connections 2024-05-09T05:39:59.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:39:59.720+0000 7f8eda078700 1 -- 172.21.15.115:0/303769037 wait complete. 2024-05-09T05:40:00.158 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:00 smithi115 ceph-mon[30349]: overall HEALTH_OK 2024-05-09T05:40:00.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:00 smithi195 ceph-mon[33844]: overall HEALTH_OK 2024-05-09T05:40:00.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:00 smithi161 ceph-mon[33805]: overall HEALTH_OK 2024-05-09T05:40:01.318 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:01 smithi115 ceph-mon[30349]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:40:01.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:01 smithi195 ceph-mon[33844]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:40:01.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:01 smithi161 ceph-mon[33805]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:40:01.710 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:40:01.710 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:40:02.299 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:02.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.369+0000 7fb09b831700 1 -- 172.21.15.115:0/348321034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105480 msgr2=0x7fb094105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:02.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.369+0000 7fb09b831700 1 --2- 172.21.15.115:0/348321034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105480 0x7fb094105890 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb090009990 tx=0x7fb090009ca0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.369+0000 7fb09b831700 1 -- 172.21.15.115:0/348321034 shutdown_connections 2024-05-09T05:40:02.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.369+0000 7fb09b831700 1 --2- 172.21.15.115:0/348321034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb09410e8f0 0x7fb094110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.369+0000 7fb09b831700 1 --2- 172.21.15.115:0/348321034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105480 0x7fb094105890 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.369+0000 7fb09b831700 1 --2- 172.21.15.115:0/348321034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb094105dd0 0x7fb09410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.369+0000 7fb09b831700 1 -- 172.21.15.115:0/348321034 >> 172.21.15.115:0/348321034 conn(0x7fb0940fef90 msgr2=0x7fb0941013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:02.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.369+0000 7fb09b831700 1 -- 172.21.15.115:0/348321034 shutdown_connections 2024-05-09T05:40:02.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.372+0000 7fb09b831700 1 -- 172.21.15.115:0/348321034 wait complete. 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.372+0000 7fb09b831700 1 Processor -- start 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.372+0000 7fb09b831700 1 -- start start 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb09b831700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb094105480 0x7fb09419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb09b831700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105dd0 0x7fb09419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb09b831700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb09410e8f0 0x7fb0941988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb09b831700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb094112130 con 0x7fb094105dd0 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb09b831700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb094111fb0 con 0x7fb09410e8f0 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb09b831700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941122b0 con 0x7fb094105480 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb098dcc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105dd0 0x7fb09419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb098dcc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105dd0 0x7fb09419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41860/0 (socket says 172.21.15.115:41860) 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb098dcc700 1 -- 172.21.15.115:0/1217245066 learned_addr learned my addr 172.21.15.115:0/1217245066 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:02.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb098dcc700 1 -- 172.21.15.115:0/1217245066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb094105480 msgr2=0x7fb09419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb098dcc700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb094105480 0x7fb09419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb098dcc700 1 -- 172.21.15.115:0/1217245066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb09410e8f0 msgr2=0x7fb0941988e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb098dcc700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb09410e8f0 0x7fb0941988e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb098dcc700 1 -- 172.21.15.115:0/1217245066 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb094198f90 con 0x7fb094105dd0 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.373+0000 7fb098dcc700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105dd0 0x7fb09419ec00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb084009c70 tx=0x7fb08400b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.374+0000 7fb08a7fc700 1 -- 172.21.15.115:0/1217245066 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08401e030 con 0x7fb094105dd0 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.374+0000 7fb09b831700 1 -- 172.21.15.115:0/1217245066 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb094199280 con 0x7fb094105dd0 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.374+0000 7fb08a7fc700 1 -- 172.21.15.115:0/1217245066 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb084009db0 con 0x7fb094105dd0 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.374+0000 7fb08a7fc700 1 -- 172.21.15.115:0/1217245066 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08400e640 con 0x7fb094105dd0 2024-05-09T05:40:02.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.374+0000 7fb09b831700 1 -- 172.21.15.115:0/1217245066 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0941a5550 con 0x7fb094105dd0 2024-05-09T05:40:02.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.378+0000 7fb08a7fc700 1 -- 172.21.15.115:0/1217245066 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fb084021030 con 0x7fb094105dd0 2024-05-09T05:40:02.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.379+0000 7fb08a7fc700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb0800b2a70 0x7fb0800b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.379+0000 7fb08a7fc700 1 -- 172.21.15.115:0/1217245066 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fb0840ea7f0 con 0x7fb094105dd0 2024-05-09T05:40:02.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0995cd700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb0800b2a70 0x7fb0800b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0aa247b80 1 Processor -- start 2024-05-09T05:40:02.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0aa247b80 1 -- start start 2024-05-09T05:40:02.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0995cd700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb0800b2a70 0x7fb0800b4f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb0900098e0 tx=0x7fb09000b610 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:02.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b336260 0x55753b3cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b24c5a0 0x55753b3cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55753b2ec0f0 con 0x55753b3c9b50 2024-05-09T05:40:02.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55753b28a8f0 con 0x55753b24c5a0 2024-05-09T05:40:02.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.380+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55753b2e80b0 con 0x55753b336260 2024-05-09T05:40:02.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb099dce700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb099dce700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41864/0 (socket says 172.21.15.115:41864) 2024-05-09T05:40:02.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb099dce700 1 -- 172.21.15.115:0/1867632393 learned_addr learned my addr 172.21.15.115:0/1867632393 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:02.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb0995cd700 1 --2- 172.21.15.115:0/1867632393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b336260 0x55753b3cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb099dce700 1 -- 172.21.15.115:0/1867632393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b336260 msgr2=0x55753b3cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:02.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb099dce700 1 --2- 172.21.15.115:0/1867632393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b336260 0x55753b3cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb099dce700 1 -- 172.21.15.115:0/1867632393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b24c5a0 msgr2=0x55753b3cffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:40:02.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb099dce700 1 --2- 172.21.15.115:0/1867632393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b24c5a0 0x55753b3cffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb099dce700 1 -- 172.21.15.115:0/1867632393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55753b282470 con 0x55753b3c9b50 2024-05-09T05:40:02.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb099dce700 1 --2- 172.21.15.115:0/1867632393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3c9f60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009b60 tx=0x7fb08c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28006d344801d95c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:02.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb07f7fe700 1 -- 172.21.15.115:0/1867632393 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c007cc0 con 0x55753b3c9b50 2024-05-09T05:40:02.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb07f7fe700 1 -- 172.21.15.115:0/1867632393 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb08c004bb0 con 0x55753b3c9b50 2024-05-09T05:40:02.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.381+0000 7fb07f7fe700 1 -- 172.21.15.115:0/1867632393 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0133f0 con 0x55753b3c9b50 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.382+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1867632393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 msgr2=0x55753b3c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.382+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1867632393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3c9f60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009b60 tx=0x7fb08c009e70 comp rx=0 tx=0).stop 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.382+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1867632393 shutdown_connections 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.382+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1867632393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3c9f60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.382+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1867632393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b336260 0x55753b3cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.382+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1867632393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b24c5a0 0x55753b3cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.382+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1867632393 >> 172.21.15.115:0/1867632393 conn(0x55753b2ac7a0 msgr2=0x55753b3c4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.382+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1867632393 shutdown_connections 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.382+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1867632393 wait complete. 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.383+0000 7fb0aa247b80 1 -- start start 2024-05-09T05:40:02.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.383+0000 7fb0aa247b80 1 Processor -- start 2024-05-09T05:40:02.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.383+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 0x55753b3d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.383+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b336260 0x55753b3d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.383+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3d4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55753b167ca0 con 0x55753b3c9b50 2024-05-09T05:40:02.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55753b2a6250 con 0x55753b336260 2024-05-09T05:40:02.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55753b3d6920 con 0x55753b24c5a0 2024-05-09T05:40:02.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb0aa247b80 10 client.0 _ll_register_callbacks cb 0x55753b15d7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:02.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb0995cd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 0x55753b3d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb099dce700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3d4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb099dce700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3d4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41876/0 (socket says 172.21.15.115:41876) 2024-05-09T05:40:02.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb0995cd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 0x55753b3d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:55824/0 (socket says 172.21.15.115:55824) 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb0995cd700 1 -- 172.21.15.115:0/4029022134 learned_addr learned my addr 172.21.15.115:0/4029022134 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb099dce700 1 -- 172.21.15.115:0/4029022134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 msgr2=0x55753b3d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb099dce700 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 0x55753b3d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb099dce700 1 -- 172.21.15.115:0/4029022134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b336260 msgr2=0x55753b3d3e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb099dce700 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b336260 0x55753b3d3e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb099dce700 1 -- 172.21.15.115:0/4029022134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55753b167590 con 0x55753b3c9b50 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb099dce700 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3d4390 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009270 tx=0x7fb08c005010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c154afa9b861da4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb07cff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb07cff9700 1 -- 172.21.15.115:0/4029022134 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c021070 con 0x55753b3c9b50 2024-05-09T05:40:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb0aa247b80 1 -- 172.21.15.115:0/4029022134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb08c004bb0 con 0x55753b3c9b50 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.384+0000 7fb07cff9700 1 -- 172.21.15.115:0/4029022134 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fb08c003a00 con 0x55753b3c9b50 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb07cff9700 1 -- 172.21.15.115:0/4029022134 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb08c02d050 con 0x55753b3c9b50 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb0aa247b80 10 client.16068 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb0aa247b80 1 -- 172.21.15.115:0/4029022134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55753b3d8090 con 0x55753b3c9b50 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb07cff9700 1 -- 172.21.15.115:0/4029022134 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb08c004bb0 con 0x55753b3c9b50 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb07cff9700 1 -- 172.21.15.115:0/4029022134 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb08c004700 con 0x55753b3c9b50 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb0aa247b80 10 client.16068 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb0aa247b80 10 client.16068 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb0aa247b80 20 client.16068 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55753b3da5c0 0x55753b3dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb0aa247b80 4 client.16068 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb0aa247b80 1 -- 172.21.15.115:0/4029022134 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55753b2f9ad0 con 0x55753b3da5c0 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.385+0000 7fb0995cd700 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55753b3da5c0 0x55753b3dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.386+0000 7fb07cff9700 10 client.16068 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:02.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.386+0000 7fb0995cd700 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55753b3da5c0 0x55753b3dc9e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dfca49eb345b5d8 server_cookie=76102d1467c0ca2 in_seq=0 out_seq=0 2024-05-09T05:40:02.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb07cff9700 1 -- 172.21.15.115:0/4029022134 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55753b2f9ad0 con 0x55753b3da5c0 2024-05-09T05:40:02.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb07cff9700 10 client.16068 handle_command_reply: tid=0 2024-05-09T05:40:02.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb0aa247b80 1 client.16068 shutdown 2024-05-09T05:40:02.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb0aa247b80 1 -- 172.21.15.115:0/4029022134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 msgr2=0x55753b3d4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:02.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3d4390 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009270 tx=0x7fb08c005010 comp rx=0 tx=0).stop 2024-05-09T05:40:02.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb0aa247b80 1 -- 172.21.15.115:0/4029022134 shutdown_connections 2024-05-09T05:40:02.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55753b3da5c0 0x55753b3dc9e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb07cff9700 0 client.16068 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:02.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b3c9b50 0x55753b3d4390 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 0x55753b3d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/4029022134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b336260 0x55753b3d3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.387+0000 7fb0aa247b80 1 -- 172.21.15.115:0/4029022134 >> 172.21.15.115:0/4029022134 conn(0x55753b2ac7a0 msgr2=0x55753b3c4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:02.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.389+0000 7fb0aa247b80 1 -- 172.21.15.115:0/4029022134 shutdown_connections 2024-05-09T05:40:02.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.390+0000 7fb0aa247b80 1 -- 172.21.15.115:0/4029022134 wait complete. 2024-05-09T05:40:02.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.390+0000 7fb0aa247b80 20 client.16068 trim_cache size 0 max 16384 2024-05-09T05:40:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.410+0000 7fb0aa247b80 1 Processor -- start 2024-05-09T05:40:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.410+0000 7fb0aa247b80 1 -- start start 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.410+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b24c5a0 0x55753b3cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b336260 0x55753b3cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b3c9b50 0x55753b3cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941b6a80 con 0x55753b24c5a0 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941b85c0 con 0x55753b336260 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941b7150 con 0x55753b3c9b50 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0995cd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b24c5a0 0x55753b3cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0995cd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b24c5a0 0x55753b3cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41884/0 (socket says 172.21.15.115:41884) 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0995cd700 1 -- 172.21.15.115:0/2916090606 learned_addr learned my addr 172.21.15.115:0/2916090606 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb098dcc700 1 --2- 172.21.15.115:0/2916090606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b336260 0x55753b3cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0995cd700 1 -- 172.21.15.115:0/2916090606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b3c9b50 msgr2=0x55753b3cc8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0995cd700 1 --2- 172.21.15.115:0/2916090606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b3c9b50 0x55753b3cc8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0995cd700 1 -- 172.21.15.115:0/2916090606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b336260 msgr2=0x55753b3cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:02.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0995cd700 1 --2- 172.21.15.115:0/2916090606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b336260 0x55753b3cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.411+0000 7fb0995cd700 1 -- 172.21.15.115:0/2916090606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55753b3d61b0 con 0x55753b24c5a0 2024-05-09T05:40:02.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.412+0000 7fb0995cd700 1 --2- 172.21.15.115:0/2916090606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b24c5a0 0x55753b3cdf80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb090015000 tx=0x7fb09001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa9a02a54017149 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:02.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.412+0000 7fb07effd700 1 -- 172.21.15.115:0/2916090606 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090037070 con 0x55753b24c5a0 2024-05-09T05:40:02.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.412+0000 7fb07effd700 1 -- 172.21.15.115:0/2916090606 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb090004c40 con 0x55753b24c5a0 2024-05-09T05:40:02.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.412+0000 7fb07effd700 1 -- 172.21.15.115:0/2916090606 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09000f750 con 0x55753b24c5a0 2024-05-09T05:40:02.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.413+0000 7fb0aa247b80 1 -- 172.21.15.115:0/2916090606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b24c5a0 msgr2=0x55753b3cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:02.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.413+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/2916090606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b24c5a0 0x55753b3cdf80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb090015000 tx=0x7fb09001f000 comp rx=0 tx=0).stop 2024-05-09T05:40:02.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.413+0000 7fb0aa247b80 1 -- 172.21.15.115:0/2916090606 shutdown_connections 2024-05-09T05:40:02.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.413+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/2916090606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b3c9b50 0x55753b3cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.413+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/2916090606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b24c5a0 0x55753b3cdf80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.413+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/2916090606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b336260 0x55753b3cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.413+0000 7fb0aa247b80 1 -- 172.21.15.115:0/2916090606 >> 172.21.15.115:0/2916090606 conn(0x55753b2ac7a0 msgr2=0x55753b3d4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:02.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.414+0000 7fb0aa247b80 1 -- 172.21.15.115:0/2916090606 shutdown_connections 2024-05-09T05:40:02.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.414+0000 7fb0aa247b80 1 -- 172.21.15.115:0/2916090606 wait complete. 2024-05-09T05:40:02.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.414+0000 7fb0aa247b80 1 -- start start 2024-05-09T05:40:02.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.414+0000 7fb0aa247b80 1 Processor -- start 2024-05-09T05:40:02.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 0x55753b468de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b336260 0x55753b469320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb0aa247b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b3c9b50 0x55753b3def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941b7ef0 con 0x55753b336260 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941b7820 con 0x55753b3c9b50 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb0aa247b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb094073c20 con 0x55753b24c5a0 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb0aa247b80 10 client.0 _ll_register_callbacks cb 0x55753b15d7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb0995cd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 0x55753b468de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb098dcc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b336260 0x55753b469320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb098dcc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b336260 0x55753b469320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41890/0 (socket says 172.21.15.115:41890) 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb098dcc700 1 -- 172.21.15.115:0/1877154983 learned_addr learned my addr 172.21.15.115:0/1877154983 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.415+0000 7fb099dce700 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b3c9b50 0x55753b3def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb098dcc700 1 -- 172.21.15.115:0/1877154983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 msgr2=0x55753b468de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb098dcc700 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 0x55753b468de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb098dcc700 1 -- 172.21.15.115:0/1877154983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b3c9b50 msgr2=0x55753b3def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb098dcc700 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b3c9b50 0x55753b3def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb098dcc700 1 -- 172.21.15.115:0/1877154983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55753b3df5c0 con 0x55753b336260 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb099dce700 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b3c9b50 0x55753b3def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb098dcc700 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b336260 0x55753b469320 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb0840ad810 tx=0x7fb0840b1cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0505dbd7750b00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb07cff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb07cff9700 1 -- 172.21.15.115:0/1877154983 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084028070 con 0x55753b336260 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb07cff9700 1 -- 172.21.15.115:0/1877154983 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fb0840990b0 con 0x55753b336260 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.416+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1877154983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb090004c40 con 0x55753b336260 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb07cff9700 1 -- 172.21.15.115:0/1877154983 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb0840b1790 con 0x55753b336260 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb0aa247b80 10 client.16074 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1877154983 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55753b4704f0 con 0x55753b336260 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb07cff9700 1 -- 172.21.15.115:0/1877154983 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb090004c40 con 0x55753b336260 2024-05-09T05:40:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb07cff9700 1 -- 172.21.15.115:0/1877154983 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb0840ad190 con 0x55753b336260 2024-05-09T05:40:02.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb0aa247b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:02.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb0aa247b80 10 client.16074 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:40:02.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb0aa247b80 20 client.16074 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:02.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55753b470b30 0x55753b472f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:02.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb0aa247b80 4 client.16074 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9c695dab", "release": true, "await": true}] 2024-05-09T05:40:02.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.417+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1877154983 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9c695dab", "release": true, "await": true}) -- 0x55753b2f3b70 con 0x55753b470b30 2024-05-09T05:40:02.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.418+0000 7fb0995cd700 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55753b470b30 0x55753b472f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:02.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.418+0000 7fb07cff9700 10 client.16074 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:02.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.419+0000 7fb0995cd700 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55753b470b30 0x55753b472f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf82d204ed4ca9ca server_cookie=8875fe57534461aa in_seq=0 out_seq=0 2024-05-09T05:40:02.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.421+0000 7fb07cff9700 1 -- 172.21.15.115:0/1877154983 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55753b2f3b70 con 0x55753b470b30 2024-05-09T05:40:02.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.421+0000 7fb07cff9700 10 client.16074 handle_command_reply: tid=0 2024-05-09T05:40:02.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.421+0000 7fb0aa247b80 1 client.16074 shutdown 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.421+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1877154983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b336260 msgr2=0x55753b469320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.421+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b336260 0x55753b469320 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb0840ad810 tx=0x7fb0840b1cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.421+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1877154983 shutdown_connections 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.422+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55753b3c9b50 0x55753b3def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.422+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55753b470b30 0x55753b472f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.422+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55753b24c5a0 0x55753b468de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.422+0000 7fb07cff9700 0 client.16074 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.422+0000 7fb0aa247b80 1 --2- 172.21.15.115:0/1877154983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55753b336260 0x55753b469320 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.422+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1877154983 >> 172.21.15.115:0/1877154983 conn(0x55753b2ac7a0 msgr2=0x55753b3c4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.422+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1877154983 shutdown_connections 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.422+0000 7fb0aa247b80 1 -- 172.21.15.115:0/1877154983 wait complete. 2024-05-09T05:40:02.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.422+0000 7fb0aa247b80 20 client.16074 trim_cache size 0 max 16384 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 30, 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 4, 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "9c695dab": { 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 4, 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:40:02.417 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 144.9, 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:02.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:02.419 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:40:02.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 -- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb0800b2a70 msgr2=0x7fb0800b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:02.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb0800b2a70 0x7fb0800b4f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb0900098e0 tx=0x7fb09000b610 comp rx=0 tx=0).stop 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 -- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105dd0 msgr2=0x7fb09419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105dd0 0x7fb09419ec00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb084009c70 tx=0x7fb08400b540 comp rx=0 tx=0).stop 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 -- 172.21.15.115:0/1217245066 shutdown_connections 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb0800b2a70 0x7fb0800b4f80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb09410e8f0 0x7fb0941988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb094105480 0x7fb09419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 --2- 172.21.15.115:0/1217245066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb094105dd0 0x7fb09419ec00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.423+0000 7fb09b831700 1 -- 172.21.15.115:0/1217245066 >> 172.21.15.115:0/1217245066 conn(0x7fb0940fef90 msgr2=0x7fb094100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.424+0000 7fb09b831700 1 -- 172.21.15.115:0/1217245066 shutdown_connections 2024-05-09T05:40:02.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:02.424+0000 7fb09b831700 1 -- 172.21.15.115:0/1217245066 wait complete. 2024-05-09T05:40:03.066 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:03 smithi115 ceph-mon[30349]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:40:03.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:03 smithi195 ceph-mon[33844]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:40:03.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:03 smithi161 ceph-mon[33805]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:40:03.446 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9c695dab', total seconds elapsed: 36.2 2024-05-09T05:40:03.446 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.95 2024-05-09T05:40:03.447 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:40:04.403 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:05.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:05 smithi195 ceph-mon[33844]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-09T05:40:05.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:05 smithi115 ceph-mon[30349]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-09T05:40:05.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:05 smithi161 ceph-mon[33805]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-09T05:40:07.071 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:07 smithi115 ceph-mon[30349]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T05:40:07.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:07 smithi195 ceph-mon[33844]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T05:40:07.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:07 smithi161 ceph-mon[33805]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T05:40:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.592+0000 7f4b88249700 1 -- 172.21.15.115:0/2721304652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80107e00 msgr2=0x7f4b80108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.592+0000 7f4b88249700 1 --2- 172.21.15.115:0/2721304652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80107e00 0x7f4b80108210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4b700098f0 tx=0x7f4b70009c00 comp rx=0 tx=0).stop 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.592+0000 7f4b88249700 1 -- 172.21.15.115:0/2721304652 shutdown_connections 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.592+0000 7f4b88249700 1 --2- 172.21.15.115:0/2721304652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80108750 0x7f4b801100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.592+0000 7f4b88249700 1 --2- 172.21.15.115:0/2721304652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b80103c60 0x7f4b80104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.592+0000 7f4b88249700 1 --2- 172.21.15.115:0/2721304652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80107e00 0x7f4b80108210 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.592+0000 7f4b88249700 1 -- 172.21.15.115:0/2721304652 >> 172.21.15.115:0/2721304652 conn(0x7f4b800fe6f0 msgr2=0x7f4b80100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.593+0000 7f4b88249700 1 -- 172.21.15.115:0/2721304652 shutdown_connections 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.593+0000 7f4b88249700 1 -- 172.21.15.115:0/2721304652 wait complete. 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b88249700 1 Processor -- start 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b88249700 1 -- start start 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b88249700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80103c60 0x7f4b8019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b88249700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b80107e00 0x7f4b8019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b85fe5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80103c60 0x7f4b8019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b85fe5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80103c60 0x7f4b8019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59592/0 (socket says 172.21.15.115:59592) 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b85fe5700 1 -- 172.21.15.115:0/1785999533 learned_addr learned my addr 172.21.15.115:0/1785999533 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b88249700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80108750 0x7f4b801a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80112190 con 0x7f4b80103c60 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80112010 con 0x7f4b80108750 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80112310 con 0x7f4b80107e00 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.594+0000 7f4b857e4700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b80107e00 0x7f4b8019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b867e6700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80108750 0x7f4b801a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b85fe5700 1 -- 172.21.15.115:0/1785999533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b80107e00 msgr2=0x7f4b8019d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b80107e00 0x7f4b8019d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b85fe5700 1 -- 172.21.15.115:0/1785999533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80108750 msgr2=0x7f4b801a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80108750 0x7f4b801a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b85fe5700 1 -- 172.21.15.115:0/1785999533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b801a1e40 con 0x7f4b80103c60 2024-05-09T05:40:08.094 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:40:08.094 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:40:08.094 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "cbf294cc-ec0a-4351-903e-924360875aac", 2024-05-09T05:40:08.094 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:40:08.094 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:40:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80103c60 0x7f4b8019cf70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009a20 tx=0x7f4b7c009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b76ffd700 1 -- 172.21.15.115:0/1785999533 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c01b070 con 0x7f4b80103c60 2024-05-09T05:40:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b76ffd700 1 -- 172.21.15.115:0/1785999533 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b7c00cbf0 con 0x7f4b80103c60 2024-05-09T05:40:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b76ffd700 1 -- 172.21.15.115:0/1785999533 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c016c10 con 0x7f4b80103c60 2024-05-09T05:40:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.595+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b801a2130 con 0x7f4b80103c60 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.596+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b801a9a50 con 0x7f4b80103c60 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.600+0000 7f4b76ffd700 1 -- 172.21.15.115:0/1785999533 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4b7c01e070 con 0x7f4b80103c60 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.600+0000 7f4b76ffd700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4b6c0b29f0 0x7f4b6c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.601+0000 7f4b76ffd700 1 -- 172.21.15.115:0/1785999533 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4b7c0e1c10 con 0x7f4b80103c60 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.601+0000 7f4b857e4700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4b6c0b29f0 0x7f4b6c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.601+0000 7f4b96c5fb80 1 Processor -- start 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.601+0000 7f4b96c5fb80 1 -- start start 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b857e4700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4b6c0b29f0 0x7f4b6c0b4f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b700098c0 tx=0x7f4b70019040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b7dc1e0 0x55a94b875570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b6f2520 0x55a94b875ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b86fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b96c5fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a94b792070 con 0x55a94b86f720 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b96c5fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a94b730870 con 0x55a94b6f2520 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b867e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b86fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b96c5fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a94b78e030 con 0x55a94b7dc1e0 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b867e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b86fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59596/0 (socket says 172.21.15.115:59596) 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b867e6700 1 -- 172.21.15.115:0/4066845968 learned_addr learned my addr 172.21.15.115:0/4066845968 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b857e4700 1 --2- 172.21.15.115:0/4066845968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b6f2520 0x55a94b875ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/4066845968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b7dc1e0 0x55a94b875570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b867e6700 1 -- 172.21.15.115:0/4066845968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b7dc1e0 msgr2=0x55a94b875570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b867e6700 1 --2- 172.21.15.115:0/4066845968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b7dc1e0 0x55a94b875570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b867e6700 1 -- 172.21.15.115:0/4066845968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b6f2520 msgr2=0x55a94b875ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b867e6700 1 --2- 172.21.15.115:0/4066845968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b6f2520 0x55a94b875ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.602+0000 7f4b867e6700 1 -- 172.21.15.115:0/4066845968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94b7283f0 con 0x55a94b86f720 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.603+0000 7f4b867e6700 1 --2- 172.21.15.115:0/4066845968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b86fb50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4b78011670 tx=0x7f4b780099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0aececd865eb8eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.603+0000 7f4b6bfff700 1 -- 172.21.15.115:0/4066845968 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78019070 con 0x55a94b86f720 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.603+0000 7f4b6bfff700 1 -- 172.21.15.115:0/4066845968 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b78004020 con 0x55a94b86f720 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.603+0000 7f4b6bfff700 1 -- 172.21.15.115:0/4066845968 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78007800 con 0x55a94b86f720 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.604+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/4066845968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 msgr2=0x55a94b86fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.604+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/4066845968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b86fb50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4b78011670 tx=0x7f4b780099e0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.604+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/4066845968 shutdown_connections 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.604+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/4066845968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b86fb50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.604+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/4066845968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b7dc1e0 0x55a94b875570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.604+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/4066845968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b6f2520 0x55a94b875ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.604+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/4066845968 >> 172.21.15.115:0/4066845968 conn(0x55a94b752720 msgr2=0x55a94b86a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.604+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/4066845968 shutdown_connections 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.604+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/4066845968 wait complete. 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.605+0000 7f4b96c5fb80 1 -- start start 2024-05-09T05:40:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.605+0000 7f4b96c5fb80 1 Processor -- start 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.605+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b6f2520 0x55a94b879220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.605+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b7dc1e0 0x55a94b879760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.605+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b879ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.605+0000 7f4b96c5fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a94b60dc20 con 0x55a94b7dc1e0 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.605+0000 7f4b96c5fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a94b74c1d0 con 0x55a94b86f720 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.605+0000 7f4b96c5fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a94b87c250 con 0x55a94b6f2520 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.605+0000 7f4b96c5fb80 10 client.0 _ll_register_callbacks cb 0x55a94b603740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b857e4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b7dc1e0 0x55a94b879760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b857e4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b7dc1e0 0x55a94b879760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59600/0 (socket says 172.21.15.115:59600) 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b857e4700 1 -- 172.21.15.115:0/3653719722 learned_addr learned my addr 172.21.15.115:0/3653719722 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b867e6700 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b879ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b867e6700 1 -- 172.21.15.115:0/3653719722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b6f2520 msgr2=0x55a94b879220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b867e6700 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b6f2520 0x55a94b879220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b867e6700 1 -- 172.21.15.115:0/3653719722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b7dc1e0 msgr2=0x55a94b879760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b867e6700 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b7dc1e0 0x55a94b879760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b867e6700 1 -- 172.21.15.115:0/3653719722 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a94b60d510 con 0x55a94b86f720 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.606+0000 7f4b857e4700 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b7dc1e0 0x55a94b879760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.607+0000 7f4b867e6700 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b879ca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b78000f80 tx=0x7f4b78004940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=465663a789f8d3aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.607+0000 7f4b697fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.607+0000 7f4b697fa700 1 -- 172.21.15.115:0/3653719722 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78019040 con 0x55a94b86f720 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.607+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3653719722 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b78004020 con 0x55a94b86f720 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.607+0000 7f4b697fa700 1 -- 172.21.15.115:0/3653719722 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4b78003a00 con 0x55a94b86f720 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.607+0000 7f4b697fa700 1 -- 172.21.15.115:0/3653719722 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4b78030050 con 0x55a94b86f720 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.607+0000 7f4b96c5fb80 10 client.25607 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.607+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3653719722 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a94b88f3b0 con 0x55a94b86f720 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.607+0000 7f4b697fa700 1 -- 172.21.15.115:0/3653719722 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b78004020 con 0x55a94b86f720 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.608+0000 7f4b697fa700 1 -- 172.21.15.115:0/3653719722 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4b78025020 con 0x55a94b86f720 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.608+0000 7f4b96c5fb80 10 client.25607 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.608+0000 7f4b96c5fb80 10 client.25607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.608+0000 7f4b96c5fb80 20 client.25607 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.608+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a94b8918e0 0x55a94b893d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.608+0000 7f4b96c5fb80 4 client.25607 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.608+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3653719722 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a94b79fa50 con 0x55a94b8918e0 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.608+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a94b8918e0 0x55a94b893d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.609+0000 7f4b697fa700 10 client.25607 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.609+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a94b8918e0 0x55a94b893d00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad90c0893e8a0ce server_cookie=1cb738718ed90610 in_seq=0 out_seq=0 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.609+0000 7f4b697fa700 1 -- 172.21.15.115:0/3653719722 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a94b79fa50 con 0x55a94b8918e0 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.609+0000 7f4b697fa700 10 client.25607 handle_command_reply: tid=0 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.609+0000 7f4b96c5fb80 1 client.25607 shutdown 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.609+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3653719722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 msgr2=0x55a94b879ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.609+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b879ca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b78000f80 tx=0x7f4b78004940 comp rx=0 tx=0).stop 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3653719722 shutdown_connections 2024-05-09T05:40:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b879ca0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a94b8918e0 0x55a94b893d00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b6f2520 0x55a94b879220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3653719722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b7dc1e0 0x55a94b879760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b697fa700 0 client.25607 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3653719722 >> 172.21.15.115:0/3653719722 conn(0x55a94b752720 msgr2=0x55a94b86a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3653719722 shutdown_connections 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3653719722 wait complete. 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.610+0000 7f4b96c5fb80 20 client.25607 trim_cache size 0 max 16384 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.642+0000 7f4b96c5fb80 1 Processor -- start 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.642+0000 7f4b96c5fb80 1 -- start start 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.642+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b6f2520 0x55a94b87b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.642+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b7dc1e0 0x55a94b879ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.642+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b871e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.642+0000 7f4b96c5fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801ae200 con 0x55a94b6f2520 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.642+0000 7f4b96c5fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801afd40 con 0x55a94b86f720 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.642+0000 7f4b96c5fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801ae8d0 con 0x55a94b7dc1e0 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b85fe5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b6f2520 0x55a94b87b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b85fe5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b6f2520 0x55a94b87b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59608/0 (socket says 172.21.15.115:59608) 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b85fe5700 1 -- 172.21.15.115:0/3045298032 learned_addr learned my addr 172.21.15.115:0/3045298032 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b85fe5700 1 -- 172.21.15.115:0/3045298032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b7dc1e0 msgr2=0x55a94b879ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/3045298032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b7dc1e0 0x55a94b879ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b85fe5700 1 -- 172.21.15.115:0/3045298032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 msgr2=0x55a94b871e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b867e6700 1 --2- 172.21.15.115:0/3045298032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b871e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/3045298032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b871e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b85fe5700 1 -- 172.21.15.115:0/3045298032 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94b87a020 con 0x55a94b6f2520 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.643+0000 7f4b867e6700 1 --2- 172.21.15.115:0/3045298032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b871e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.644+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/3045298032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b6f2520 0x55a94b87b2b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0df620 tx=0x7f4b7c0a4ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d088aee43919a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.644+0000 7f4b6b7fe700 1 -- 172.21.15.115:0/3045298032 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0c9e60 con 0x55a94b6f2520 2024-05-09T05:40:08.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.644+0000 7f4b6b7fe700 1 -- 172.21.15.115:0/3045298032 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b7c0a8bb0 con 0x55a94b6f2520 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.644+0000 7f4b6b7fe700 1 -- 172.21.15.115:0/3045298032 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0b1580 con 0x55a94b6f2520 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.645+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3045298032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b6f2520 msgr2=0x55a94b87b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.645+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3045298032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b6f2520 0x55a94b87b2b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0df620 tx=0x7f4b7c0a4ac0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.645+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3045298032 shutdown_connections 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.645+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3045298032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b86f720 0x55a94b871e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.645+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3045298032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b6f2520 0x55a94b87b2b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.645+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/3045298032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b7dc1e0 0x55a94b879ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.645+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3045298032 >> 172.21.15.115:0/3045298032 conn(0x55a94b752720 msgr2=0x55a94b874930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.645+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3045298032 shutdown_connections 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.645+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/3045298032 wait complete. 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.646+0000 7f4b96c5fb80 1 -- start start 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.646+0000 7f4b96c5fb80 1 Processor -- start 2024-05-09T05:40:08.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.647+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b6f2520 0x55a94b8814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.647+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b7dc1e0 0x55a94b881a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.647+0000 7f4b96c5fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b87c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.647+0000 7f4b867e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b87c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.647+0000 7f4b867e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b87c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59614/0 (socket says 172.21.15.115:59614) 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.647+0000 7f4b867e6700 1 -- 172.21.15.115:0/248310691 learned_addr learned my addr 172.21.15.115:0/248310691 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.647+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801af670 con 0x55a94b86f720 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.647+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801aefa0 con 0x55a94b7dc1e0 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.647+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4b801ac1a0 con 0x55a94b6f2520 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.648+0000 7f4b867e6700 1 -- 172.21.15.115:0/248310691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b6f2520 msgr2=0x55a94b8814f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.648+0000 7f4b867e6700 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b6f2520 0x55a94b8814f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.648+0000 7f4b867e6700 1 -- 172.21.15.115:0/248310691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b7dc1e0 msgr2=0x55a94b881a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.648+0000 7f4b867e6700 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b7dc1e0 0x55a94b881a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.648+0000 7f4b867e6700 1 -- 172.21.15.115:0/248310691 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a94b87cba0 con 0x55a94b86f720 2024-05-09T05:40:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.648+0000 7f4b96c5fb80 10 client.0 _ll_register_callbacks cb 0x55a94b603740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.648+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b7c0a8bb0 con 0x55a94b86f720 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.648+0000 7f4b697fa700 10 client.16092 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.648+0000 7f4b867e6700 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b87c400 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009a20 tx=0x7f4b78005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da973ea894cbc6ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b697fa700 1 -- 172.21.15.115:0/248310691 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78019040 con 0x55a94b86f720 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b697fa700 1 -- 172.21.15.115:0/248310691 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4b78004460 con 0x55a94b86f720 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b697fa700 1 -- 172.21.15.115:0/248310691 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4b78004660 con 0x55a94b86f720 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b96c5fb80 10 client.16092 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55a94b8fc900 con 0x55a94b86f720 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b697fa700 1 -- 172.21.15.115:0/248310691 <== mon.0 v2:172.21.15.115:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4b78025050 con 0x55a94b86f720 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b96c5fb80 10 client.16092 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b96c5fb80 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b96c5fb80 20 client.16092 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a94b8fcef0 0x55a94b8ff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:08.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b96c5fb80 4 client.16092 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.649+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a94b799af0 con 0x55a94b8fcef0 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.650+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a94b8fcef0 0x55a94b8ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.650+0000 7f4b697fa700 10 client.16092 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.650+0000 7f4b85fe5700 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a94b8fcef0 0x55a94b8ff390 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4541a6149cf7cd10 server_cookie=d0c7e617a489eff0 in_seq=0 out_seq=0 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.652+0000 7f4b697fa700 1 -- 172.21.15.115:0/248310691 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a94b799af0 con 0x55a94b8fcef0 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.652+0000 7f4b697fa700 10 client.16092 handle_command_reply: tid=0 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.652+0000 7f4b96c5fb80 1 client.16092 shutdown 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.652+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 msgr2=0x55a94b87c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.652+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b87c400 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009a20 tx=0x7f4b78005f40 comp rx=0 tx=0).stop 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 shutdown_connections 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a94b8fcef0 0x55a94b8ff390 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a94b86f720 0x55a94b87c400 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94b6f2520 0x55a94b8814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b697fa700 0 client.16092 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:08.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b96c5fb80 1 --2- 172.21.15.115:0/248310691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a94b7dc1e0 0x55a94b881a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 >> 172.21.15.115:0/248310691 conn(0x55a94b752720 msgr2=0x55a94b872e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 shutdown_connections 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b96c5fb80 1 -- 172.21.15.115:0/248310691 wait complete. 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.653+0000 7f4b96c5fb80 20 client.16092 trim_cache size 0 max 16384 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.654+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4b6c0b29f0 msgr2=0x7f4b6c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.654+0000 7f4b88249700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4b6c0b29f0 0x7f4b6c0b4f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4b700098c0 tx=0x7f4b70019040 comp rx=0 tx=0).stop 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.654+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80103c60 msgr2=0x7f4b8019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.654+0000 7f4b88249700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80103c60 0x7f4b8019cf70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009a20 tx=0x7f4b7c009d30 comp rx=0 tx=0).stop 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.655+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 shutdown_connections 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.655+0000 7f4b88249700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4b6c0b29f0 0x7f4b6c0b4f00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.655+0000 7f4b88249700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80108750 0x7f4b801a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.655+0000 7f4b88249700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b80103c60 0x7f4b8019cf70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.655+0000 7f4b88249700 1 --2- 172.21.15.115:0/1785999533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b80107e00 0x7f4b8019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:08.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.655+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 >> 172.21.15.115:0/1785999533 conn(0x7f4b800fe6f0 msgr2=0x7f4b800feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:08.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.655+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 shutdown_connections 2024-05-09T05:40:08.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:07.655+0000 7f4b88249700 1 -- 172.21.15.115:0/1785999533 wait complete. 2024-05-09T05:40:08.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:40:09.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:09 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:40:09.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:09 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:09.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:09 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:40:09.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:09 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:09.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:09 smithi195 ceph-mon[33844]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-05-09T05:40:09.375 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:09 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:40:09.375 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:09 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:09.375 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:09 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:40:09.375 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:09 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:09.376 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:09 smithi115 ceph-mon[30349]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-05-09T05:40:09.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:09 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:40:09.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:09 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:09.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:09 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:40:09.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:09 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:09.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:09 smithi161 ceph-mon[33805]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-05-09T05:40:10.081 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:10 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:40:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:10.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:10 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:40:10.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:10 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:40:10.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.974+0000 7f893eef4700 1 -- 172.21.15.115:0/27837301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938108f30 msgr2=0x7f8938109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:10.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.974+0000 7f893eef4700 1 --2- 172.21.15.115:0/27837301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938108f30 0x7f8938109340 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f89240099c0 tx=0x7f8924009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:10.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.974+0000 7f893eef4700 1 -- 172.21.15.115:0/27837301 shutdown_connections 2024-05-09T05:40:10.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.974+0000 7f893eef4700 1 --2- 172.21.15.115:0/27837301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8938104650 0x7f8938104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:10.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.974+0000 7f893eef4700 1 --2- 172.21.15.115:0/27837301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8938109880 0x7f89381111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:10.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.974+0000 7f893eef4700 1 --2- 172.21.15.115:0/27837301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938108f30 0x7f8938109340 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:10.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.974+0000 7f893eef4700 1 -- 172.21.15.115:0/27837301 >> 172.21.15.115:0/27837301 conn(0x7f8938079f00 msgr2=0x7f893807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:10.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.974+0000 7f893eef4700 1 -- 172.21.15.115:0/27837301 shutdown_connections 2024-05-09T05:40:10.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.974+0000 7f893eef4700 1 -- 172.21.15.115:0/27837301 wait complete. 2024-05-09T05:40:10.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893eef4700 1 Processor -- start 2024-05-09T05:40:10.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893eef4700 1 -- start start 2024-05-09T05:40:10.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893eef4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8938104650 0x7f893819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:10.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893eef4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8938108f30 0x7f893819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:10.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893eef4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938109880 0x7f893819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893eef4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8938114340 con 0x7f8938109880 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893eef4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89381141c0 con 0x7f8938108f30 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893eef4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89381144c0 con 0x7f8938104650 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893d491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938109880 0x7f893819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893d491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938109880 0x7f893819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59620/0 (socket says 172.21.15.115:59620) 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.975+0000 7f893d491700 1 -- 172.21.15.115:0/2748638359 learned_addr learned my addr 172.21.15.115:0/2748638359 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f893cc90700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8938104650 0x7f893819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f8937fff700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8938108f30 0x7f893819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f893d491700 1 -- 172.21.15.115:0/2748638359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8938104650 msgr2=0x7f893819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f893d491700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8938104650 0x7f893819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:10.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f893d491700 1 -- 172.21.15.115:0/2748638359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8938108f30 msgr2=0x7f893819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:10.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f893d491700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8938108f30 0x7f893819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:10.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f893d491700 1 -- 172.21.15.115:0/2748638359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893819fbf0 con 0x7f8938109880 2024-05-09T05:40:10.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f8937fff700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8938108f30 0x7f893819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:10.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f893d491700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938109880 0x7f893819f600 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f893000ef30 tx=0x7f893000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:10.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f8935ffb700 1 -- 172.21.15.115:0/2748638359 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8930004bb0 con 0x7f8938109880 2024-05-09T05:40:10.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.976+0000 7f8935ffb700 1 -- 172.21.15.115:0/2748638359 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8930004d50 con 0x7f8938109880 2024-05-09T05:40:10.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.977+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893819fee0 con 0x7f8938109880 2024-05-09T05:40:10.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.977+0000 7f8935ffb700 1 -- 172.21.15.115:0/2748638359 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8930004280 con 0x7f8938109880 2024-05-09T05:40:10.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.977+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89381a03f0 con 0x7f8938109880 2024-05-09T05:40:10.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.979+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89380512c0 con 0x7f8938109880 2024-05-09T05:40:10.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.981+0000 7f8935ffb700 1 -- 172.21.15.115:0/2748638359 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f893000c9f0 con 0x7f8938109880 2024-05-09T05:40:10.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.982+0000 7f8935ffb700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89280b2a70 0x7f89280b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:10.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.982+0000 7f893cc90700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89280b2a70 0x7f89280b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:10.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.983+0000 7f8935ffb700 1 -- 172.21.15.115:0/2748638359 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f89300e6200 con 0x7f8938109880 2024-05-09T05:40:10.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.983+0000 7f893cc90700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89280b2a70 0x7f89280b4f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f892c00ac50 tx=0x7f892c00a380 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:10.991+0000 7f8935ffb700 1 -- 172.21.15.115:0/2748638359 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89300ad9b0 con 0x7f8938109880 2024-05-09T05:40:11.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:11 smithi115 ceph-mon[30349]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-05-09T05:40:11.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:11 smithi195 ceph-mon[33844]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-05-09T05:40:11.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:11 smithi161 ceph-mon[33805]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-05-09T05:40:11.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.418+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f893802e5f0 con 0x7f8938109880 2024-05-09T05:40:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.419+0000 7f8935ffb700 1 -- 172.21.15.115:0/2748638359 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7f89300ad2b0 con 0x7f8938109880 2024-05-09T05:40:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:40:11.412 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.422+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89280b2a70 msgr2=0x7f89280b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.422+0000 7f893eef4700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89280b2a70 0x7f89280b4f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f892c00ac50 tx=0x7f892c00a380 comp rx=0 tx=0).stop 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.422+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938109880 msgr2=0x7f893819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.422+0000 7f893eef4700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938109880 0x7f893819f600 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f893000ef30 tx=0x7f893000c3a0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.423+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 shutdown_connections 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.423+0000 7f893eef4700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f89280b2a70 0x7f89280b4f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.423+0000 7f893eef4700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8938109880 0x7f893819f600 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.423+0000 7f893eef4700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8938104650 0x7f893819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.423+0000 7f893eef4700 1 --2- 172.21.15.115:0/2748638359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8938108f30 0x7f893819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.423+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 >> 172.21.15.115:0/2748638359 conn(0x7f8938079f00 msgr2=0x7f8938102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:11.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.423+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 shutdown_connections 2024-05-09T05:40:11.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:11.423+0000 7f893eef4700 1 -- 172.21.15.115:0/2748638359 wait complete. 2024-05-09T05:40:11.417 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:40:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.940+0000 7f0f91d9b640 1 -- 172.21.15.115:0/1189913137 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c031800 con 0x7f0f94109030 2024-05-09T05:40:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.941+0000 7f0f98e0c640 1 -- 172.21.15.115:0/1189913137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f94109030 msgr2=0x7f0f94102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.941+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/1189913137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f94109030 0x7f0f94102c70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009900 tx=0x7f0f7c02a3f0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.941+0000 7f0f98e0c640 1 -- 172.21.15.115:0/1189913137 shutdown_connections 2024-05-09T05:40:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.941+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/1189913137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f941032e0 0x7f0f9410f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.941+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/1189913137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f94109030 0x7f0f94102c70 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.941+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/1189913137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94108b90 0x7f0f94102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.941+0000 7f0f98e0c640 1 -- 172.21.15.115:0/1189913137 >> 172.21.15.115:0/1189913137 conn(0x7f0f940fe390 msgr2=0x7f0f941007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.941+0000 7f0f98e0c640 1 -- 172.21.15.115:0/1189913137 shutdown_connections 2024-05-09T05:40:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.941+0000 7f0f98e0c640 1 -- 172.21.15.115:0/1189913137 wait complete. 2024-05-09T05:40:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.942+0000 7f0f98e0c640 1 Processor -- start 2024-05-09T05:40:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.942+0000 7f0f98e0c640 1 -- start start 2024-05-09T05:40:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f98e0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f941032e0 0x7f0f941a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f98e0c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94108b90 0x7f0f941a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f98e0c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f94109030 0x7f0f9419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f98e0c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0f94112c80 con 0x7f0f941032e0 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f98e0c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0f94112b00 con 0x7f0f94108b90 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f98e0c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f94112e00 con 0x7f0f94109030 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f9259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f941032e0 0x7f0f941a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f9259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f941032e0 0x7f0f941a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59642/0 (socket says 172.21.15.115:59642) 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f9259c640 1 -- 172.21.15.115:0/2147852476 learned_addr learned my addr 172.21.15.115:0/2147852476 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f89d9b640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94108b90 0x7f0f941a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f94109030 0x7f0f9419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f9259c640 1 -- 172.21.15.115:0/2147852476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f94109030 msgr2=0x7f0f9419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f9259c640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f94109030 0x7f0f9419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f9259c640 1 -- 172.21.15.115:0/2147852476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94108b90 msgr2=0x7f0f941a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f9259c640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94108b90 0x7f0f941a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f9259c640 1 -- 172.21.15.115:0/2147852476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9419ad70 con 0x7f0f941032e0 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f89d9b640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94108b90 0x7f0f941a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.943+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f94109030 0x7f0f9419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.944+0000 7f0f9259c640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f941032e0 0x7f0f941a0510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400d770 tx=0x7f0f8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.944+0000 7f0f8bfff640 1 -- 172.21.15.115:0/2147852476 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84017070 con 0x7f0f941032e0 2024-05-09T05:40:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.944+0000 7f0f8bfff640 1 -- 172.21.15.115:0/2147852476 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f84009bd0 con 0x7f0f941032e0 2024-05-09T05:40:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.944+0000 7f0f8bfff640 1 -- 172.21.15.115:0/2147852476 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400f3c0 con 0x7f0f941032e0 2024-05-09T05:40:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.944+0000 7f0f98e0c640 1 -- 172.21.15.115:0/2147852476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9419b060 con 0x7f0f941032e0 2024-05-09T05:40:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.944+0000 7f0f98e0c640 1 -- 172.21.15.115:0/2147852476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f940fe7a0 con 0x7f0f941032e0 2024-05-09T05:40:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.948+0000 7f0f9fd9a740 1 Processor -- start 2024-05-09T05:40:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.948+0000 7f0f9fd9a740 1 -- start start 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.949+0000 7f0f8bfff640 1 -- 172.21.15.115:0/2147852476 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0f8401a070 con 0x7f0f941032e0 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.949+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba518680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.949+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 0x55fdba518bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.949+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 0x55fdba519100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.949+0000 7f0f92d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba518680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.949+0000 7f0f92d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba518680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59644/0 (socket says 172.21.15.115:59644) 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.949+0000 7f0f92d9d640 1 -- 172.21.15.115:0/1418386177 learned_addr learned my addr 172.21.15.115:0/1418386177 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1418386177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fdba47d7c0 con 0x55fdba1751b0 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f9259c640 1 --2- 172.21.15.115:0/1418386177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 0x55fdba518bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1418386177 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fdba4190e0 con 0x55fdba1544d0 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1418386177 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fdba479780 con 0x55fdba14c2f0 2024-05-09T05:40:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f89d9b640 1 --2- 172.21.15.115:0/1418386177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 0x55fdba519100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f92d9d640 1 -- 172.21.15.115:0/1418386177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 msgr2=0x55fdba518bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/1418386177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 0x55fdba518bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f92d9d640 1 -- 172.21.15.115:0/1418386177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 msgr2=0x55fdba519100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f8bfff640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f800b2aa0 0x7f0f800b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/1418386177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 0x55fdba519100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f92d9d640 1 -- 172.21.15.115:0/1418386177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdba4427c0 con 0x55fdba1751b0 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.950+0000 7f0f8bfff640 1 -- 172.21.15.115:0/2147852476 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0f840e2360 con 0x7f0f941032e0 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.951+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f800b2aa0 0x7f0f800b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.951+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f800b2aa0 0x7f0f800b4fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c011010 tx=0x7f0f8c004ab0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.951+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/1418386177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba518680 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009fb0 tx=0x7f0f8c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ed5acf341633cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.952+0000 7f0f88d99640 1 -- 172.21.15.115:0/1418386177 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c022030 con 0x55fdba1751b0 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.952+0000 7f0f88d99640 1 -- 172.21.15.115:0/1418386177 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f8c012510 con 0x55fdba1751b0 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.952+0000 7f0f88d99640 1 -- 172.21.15.115:0/1418386177 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c018040 con 0x55fdba1751b0 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.953+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1418386177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 msgr2=0x55fdba518680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.953+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1418386177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba518680 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009fb0 tx=0x7f0f8c010800 comp rx=0 tx=0).stop 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.953+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1418386177 shutdown_connections 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.954+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1418386177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 0x55fdba519100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.954+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1418386177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 0x55fdba518bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.954+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1418386177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba518680 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.954+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1418386177 >> 172.21.15.115:0/1418386177 conn(0x55fdba169300 msgr2=0x55fdba514e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.954+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1418386177 shutdown_connections 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.954+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1418386177 wait complete. 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.955+0000 7f0f9fd9a740 1 -- start start 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.955+0000 7f0f9fd9a740 1 Processor -- start 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 0x55fdba524ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba522d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9fd9a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fdba5257b0 con 0x55fdba1751b0 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f92d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba522d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5247a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:36644/0 (socket says 172.21.15.115:36644) 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9259c640 1 -- 172.21.15.115:0/1627482911 learned_addr learned my addr 172.21.15.115:0/1627482911 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f89d9b640 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 0x55fdba524ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fdba4049c0 con 0x55fdba14c2f0 2024-05-09T05:40:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fdba525930 con 0x55fdba1544d0 2024-05-09T05:40:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.956+0000 7f0f9fd9a740 10 client.0 _ll_register_callbacks cb 0x55fdba431020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f92d9d640 1 -- 172.21.15.115:0/1627482911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 msgr2=0x55fdba524ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 0x55fdba524ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f92d9d640 1 -- 172.21.15.115:0/1627482911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 msgr2=0x55fdba5247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5247a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f92d9d640 1 -- 172.21.15.115:0/1627482911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdba42b1f0 con 0x55fdba1751b0 2024-05-09T05:40:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba522d00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009f80 tx=0x7f0f8c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14643b742f6623bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f79ffb640 1 -- 172.21.15.115:0/1627482911 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c018030 con 0x55fdba1751b0 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8c012510 con 0x55fdba1751b0 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f79ffb640 1 -- 172.21.15.115:0/1627482911 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0f8c01e670 con 0x55fdba1751b0 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.957+0000 7f0f79ffb640 1 -- 172.21.15.115:0/1627482911 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0f8c022040 con 0x55fdba1751b0 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.958+0000 7f0f9fd9a740 10 client.16110 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.958+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdba1e0790 con 0x55fdba1751b0 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.958+0000 7f0f79ffb640 1 -- 172.21.15.115:0/1627482911 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f8c012510 con 0x55fdba1751b0 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.958+0000 7f0f79ffb640 1 -- 172.21.15.115:0/1627482911 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0f8c02e070 con 0x55fdba1751b0 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.959+0000 7f0f9fd9a740 10 client.16110 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.959+0000 7f0f9fd9a740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.959+0000 7f0f9fd9a740 20 client.16110 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.959+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fdba133d80 0x55fdba534db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.959+0000 7f0f9fd9a740 4 client.16110 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.959+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdba48b3d0 con 0x55fdba133d80 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.959+0000 7f0f9259c640 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fdba133d80 0x55fdba534db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.960+0000 7f0f9259c640 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fdba133d80 0x55fdba534db0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75fb7333c6a4aa4c server_cookie=542e3875fc42110e in_seq=0 out_seq=0 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.960+0000 7f0f79ffb640 10 client.16110 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f79ffb640 1 -- 172.21.15.115:0/1627482911 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fdba48b3d0 con 0x55fdba133d80 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f79ffb640 10 client.16110 handle_command_reply: tid=0 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 client.16110 shutdown 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 msgr2=0x55fdba522d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba522d00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009f80 tx=0x7f0f8c011ac0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 shutdown_connections 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba522d00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fdba133d80 0x55fdba534db0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 0x55fdba524ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f79ffb640 0 client.16110 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/1627482911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 >> 172.21.15.115:0/1627482911 conn(0x55fdba169300 msgr2=0x55fdba51a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 shutdown_connections 2024-05-09T05:40:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/1627482911 wait complete. 2024-05-09T05:40:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.961+0000 7f0f9fd9a740 20 client.16110 trim_cache size 0 max 16384 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.981+0000 7f0f9fd9a740 1 Processor -- start 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.981+0000 7f0f9fd9a740 1 -- start start 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.981+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5225c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.981+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 0x55fdba518d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.981+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba51c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f9fd9a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941b33c0 con 0x55fdba1751b0 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f9fd9a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941b4f00 con 0x55fdba14c2f0 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f9fd9a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941b3a90 con 0x55fdba1544d0 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f92d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba51c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f92d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba51c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59670/0 (socket says 172.21.15.115:59670) 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f92d9d640 1 -- 172.21.15.115:0/3816105001 learned_addr learned my addr 172.21.15.115:0/3816105001 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f9259c640 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f89d9b640 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 0x55fdba518d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f92d9d640 1 -- 172.21.15.115:0/3816105001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 msgr2=0x55fdba518d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 0x55fdba518d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f92d9d640 1 -- 172.21.15.115:0/3816105001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 msgr2=0x55fdba5225c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5225c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f92d9d640 1 -- 172.21.15.115:0/3816105001 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdba2575f0 con 0x55fdba1751b0 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.982+0000 7f0f9259c640 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5225c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.983+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba51c330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c002230 tx=0x7f0f8c009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2519f57302e482 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.983+0000 7f0f7bfff640 1 -- 172.21.15.115:0/3816105001 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c018030 con 0x55fdba1751b0 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.983+0000 7f0f7bfff640 1 -- 172.21.15.115:0/3816105001 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f8c012510 con 0x55fdba1751b0 2024-05-09T05:40:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.983+0000 7f0f7bfff640 1 -- 172.21.15.115:0/3816105001 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c011d90 con 0x55fdba1751b0 2024-05-09T05:40:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3816105001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 msgr2=0x55fdba51c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba51c330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c002230 tx=0x7f0f8c009e80 comp rx=0 tx=0).stop 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3816105001 shutdown_connections 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba51c330 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba1544d0 0x55fdba518d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3816105001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba14c2f0 0x55fdba5225c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3816105001 >> 172.21.15.115:0/3816105001 conn(0x55fdba169300 msgr2=0x55fdba518620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3816105001 shutdown_connections 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3816105001 wait complete. 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.984+0000 7f0f9fd9a740 1 -- start start 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9fd9a740 1 Processor -- start 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 0x55fdba5383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 0x55fdba535720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9fd9a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba535c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9fd9a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941b4830 con 0x55fdba1751b0 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9fd9a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941b4160 con 0x55fdba1544d0 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9fd9a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941b13a0 con 0x55fdba14c2f0 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9fd9a740 10 client.0 _ll_register_callbacks cb 0x7f0f840e5d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f92d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba535c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f92d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba535c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59686/0 (socket says 172.21.15.115:59686) 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f92d9d640 1 -- 172.21.15.115:0/3855582183 learned_addr learned my addr 172.21.15.115:0/3855582183 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9259c640 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 0x55fdba5383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f89d9b640 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 0x55fdba535720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f92d9d640 1 -- 172.21.15.115:0/3855582183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 msgr2=0x55fdba5383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 0x55fdba5383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f92d9d640 1 -- 172.21.15.115:0/3855582183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 msgr2=0x55fdba535720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 0x55fdba535720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f92d9d640 1 -- 172.21.15.115:0/3855582183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdba441ed0 con 0x55fdba1751b0 2024-05-09T05:40:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.985+0000 7f0f9259c640 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 0x55fdba5383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.986+0000 7f0f92d9d640 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba535c60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c000f80 tx=0x7f0f8c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47aa3310cf89931 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.986+0000 7f0f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.986+0000 7f0f79ffb640 1 -- 172.21.15.115:0/3855582183 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c018040 con 0x55fdba1751b0 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.986+0000 7f0f79ffb640 1 -- 172.21.15.115:0/3855582183 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0f8c045070 con 0x55fdba1751b0 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.986+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3855582183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8c012510 con 0x55fdba1751b0 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.986+0000 7f0f79ffb640 1 -- 172.21.15.115:0/3855582183 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0f8c052050 con 0x55fdba1751b0 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.986+0000 7f0f9fd9a740 10 client.16116 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.986+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3855582183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdba59cc60 con 0x55fdba1751b0 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.986+0000 7f0f79ffb640 1 -- 172.21.15.115:0/3855582183 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f8c012510 con 0x55fdba1751b0 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.987+0000 7f0f79ffb640 1 -- 172.21.15.115:0/3855582183 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0f8c02e040 con 0x55fdba1751b0 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.987+0000 7f0f9fd9a740 10 client.16116 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.987+0000 7f0f9fd9a740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.987+0000 7f0f9fd9a740 20 client.16116 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.987+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fdba1091c0 0x55fdba59f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.987+0000 7f0f9fd9a740 4 client.16116 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:40:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.987+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3855582183 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdba485240 con 0x55fdba1091c0 2024-05-09T05:40:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.987+0000 7f0f9259c640 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fdba1091c0 0x55fdba59f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.987+0000 7f0f79ffb640 10 client.16116 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.988+0000 7f0f9259c640 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fdba1091c0 0x55fdba59f300 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a8aa69eff6e22e server_cookie=98a007a0822bbdfe in_seq=0 out_seq=0 2024-05-09T05:40:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.993+0000 7f0f79ffb640 1 -- 172.21.15.115:0/3855582183 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fdba485240 con 0x55fdba1091c0 2024-05-09T05:40:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.993+0000 7f0f79ffb640 10 client.16116 handle_command_reply: tid=0 2024-05-09T05:40:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 client.16116 shutdown 2024-05-09T05:40:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3855582183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 msgr2=0x55fdba535c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba535c60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c000f80 tx=0x7f0f8c010d80 comp rx=0 tx=0).stop 2024-05-09T05:40:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3855582183 shutdown_connections 2024-05-09T05:40:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fdba1751b0 0x55fdba535c60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fdba1544d0 0x55fdba535720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fdba1091c0 0x55fdba59f300 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f79ffb640 0 client.16116 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 --2- 172.21.15.115:0/3855582183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fdba14c2f0 0x55fdba5383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3855582183 >> 172.21.15.115:0/3855582183 conn(0x55fdba169300 msgr2=0x55fdba5215b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3855582183 shutdown_connections 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 1 -- 172.21.15.115:0/3855582183 wait complete. 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.994+0000 7f0f9fd9a740 20 client.16116 trim_cache size 0 max 16384 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:40:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:40:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 -- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f800b2aa0 msgr2=0x7f0f800b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f800b2aa0 0x7f0f800b4fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c011010 tx=0x7f0f8c004ab0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 -- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f941032e0 msgr2=0x7f0f941a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f941032e0 0x7f0f941a0510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400d770 tx=0x7f0f8400dc30 comp rx=0 tx=0).stop 2024-05-09T05:40:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 -- 172.21.15.115:0/2147852476 shutdown_connections 2024-05-09T05:40:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f800b2aa0 0x7f0f800b4fb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f94109030 0x7f0f9419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94108b90 0x7f0f941a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 --2- 172.21.15.115:0/2147852476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f941032e0 0x7f0f941a0510 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 -- 172.21.15.115:0/2147852476 >> 172.21.15.115:0/2147852476 conn(0x7f0f940fe390 msgr2=0x7f0f94075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 -- 172.21.15.115:0/2147852476 shutdown_connections 2024-05-09T05:40:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:11.995+0000 7f0f98e0c640 1 -- 172.21.15.115:0/2147852476 wait complete. 2024-05-09T05:40:12.088 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:12 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2748638359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:12.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:12 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2748638359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:12.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:12 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2748638359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:12.709 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0' 2024-05-09T05:40:12.710 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0 2024-05-09T05:40:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.347+0000 7f84a8cba700 1 -- 172.21.15.115:0/3466265126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a4107df0 msgr2=0x7f84a4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.347+0000 7f84a8cba700 1 --2- 172.21.15.115:0/3466265126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a4107df0 0x7f84a4108200 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f84980099c0 tx=0x7f8498009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.341 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:13 smithi115 ceph-mon[30349]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-05-09T05:40:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.348+0000 7f84a8cba700 1 -- 172.21.15.115:0/3466265126 shutdown_connections 2024-05-09T05:40:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.348+0000 7f84a8cba700 1 --2- 172.21.15.115:0/3466265126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a41034c0 0x7f84a41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.348+0000 7f84a8cba700 1 --2- 172.21.15.115:0/3466265126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84a4108740 0x7f84a41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.348+0000 7f84a8cba700 1 --2- 172.21.15.115:0/3466265126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a4107df0 0x7f84a4108200 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.348+0000 7f84a8cba700 1 -- 172.21.15.115:0/3466265126 >> 172.21.15.115:0/3466265126 conn(0x7f84a40fedc0 msgr2=0x7f84a41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.348+0000 7f84a8cba700 1 -- 172.21.15.115:0/3466265126 shutdown_connections 2024-05-09T05:40:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.348+0000 7f84a8cba700 1 -- 172.21.15.115:0/3466265126 wait complete. 2024-05-09T05:40:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.349+0000 7f84a8cba700 1 Processor -- start 2024-05-09T05:40:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.350+0000 7f84a8cba700 1 -- start start 2024-05-09T05:40:13.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.350+0000 7f84a8cba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a41034c0 0x7f84a419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a8cba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a4107df0 0x7f84a419d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a8cba700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84a4108740 0x7f84a41a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a8cba700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4116520 con 0x7f84a41034c0 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a4107df0 0x7f84a419d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a4107df0 0x7f84a419d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:37052/0 (socket says 172.21.15.115:37052) 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a1d9b700 1 -- 172.21.15.115:0/1645322376 learned_addr learned my addr 172.21.15.115:0/1645322376 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a259c700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a41034c0 0x7f84a419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a2d9d700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84a4108740 0x7f84a41a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41163a0 con 0x7f84a4107df0 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.351+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41166a0 con 0x7f84a4108740 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.352+0000 7f84a1d9b700 1 -- 172.21.15.115:0/1645322376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84a4108740 msgr2=0x7f84a41a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.352+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84a4108740 0x7f84a41a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.352+0000 7f84a1d9b700 1 -- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a41034c0 msgr2=0x7f84a419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.352+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a41034c0 0x7f84a419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.352+0000 7f84a1d9b700 1 -- 172.21.15.115:0/1645322376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a41a1c00 con 0x7f84a4107df0 2024-05-09T05:40:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.352+0000 7f84a259c700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a41034c0 0x7f84a419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.352+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a4107df0 0x7f84a419d2d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f849800f690 tx=0x7f849800f6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.353+0000 7f84937fe700 1 -- 172.21.15.115:0/1645322376 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498027070 con 0x7f84a4107df0 2024-05-09T05:40:13.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.353+0000 7f84937fe700 1 -- 172.21.15.115:0/1645322376 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8498022470 con 0x7f84a4107df0 2024-05-09T05:40:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.354+0000 7f84937fe700 1 -- 172.21.15.115:0/1645322376 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498017650 con 0x7f84a4107df0 2024-05-09T05:40:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.354+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a41a1e90 con 0x7f84a4107df0 2024-05-09T05:40:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.354+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41a2370 con 0x7f84a4107df0 2024-05-09T05:40:13.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.358+0000 7f84b76d0b80 1 Processor -- start 2024-05-09T05:40:13.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.358+0000 7f84b76d0b80 1 -- start start 2024-05-09T05:40:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f740232240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7400af4a0 0x55f740232780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74022c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f74014f0a0 con 0x55f74022c400 2024-05-09T05:40:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f7400f49a0 con 0x55f740129d00 2024-05-09T05:40:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f74014b060 con 0x55f7400af4a0 2024-05-09T05:40:13.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84a2d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f740232240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84a2d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f740232240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:37068/0 (socket says 172.21.15.115:37068) 2024-05-09T05:40:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84a2d9d700 1 -- 172.21.15.115:0/1012276085 learned_addr learned my addr 172.21.15.115:0/1012276085 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.359+0000 7f84a259c700 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7400af4a0 0x55f740232780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.360+0000 7f84937fe700 1 -- 172.21.15.115:0/1645322376 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f849802a070 con 0x7f84a4107df0 2024-05-09T05:40:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.360+0000 7f84a2d9d700 1 -- 172.21.15.115:0/1012276085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7400af4a0 msgr2=0x55f740232780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.360+0000 7f84a2d9d700 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7400af4a0 0x55f740232780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.360+0000 7f84a2d9d700 1 -- 172.21.15.115:0/1012276085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 msgr2=0x55f74022c810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:40:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.361+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74022c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.361+0000 7f84a2d9d700 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74022c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.361+0000 7f84a2d9d700 1 -- 172.21.15.115:0/1012276085 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7400e5500 con 0x55f740129d00 2024-05-09T05:40:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.361+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74022c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:40:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.361+0000 7f84a2d9d700 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f740232240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f849c00a730 tx=0x7f849c0078d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16711118586ed341 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.361+0000 7f84937fe700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f848c0b2a70 0x7f848c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.362+0000 7f8490ff9700 1 -- 172.21.15.115:0/1012276085 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c003a00 con 0x55f740129d00 2024-05-09T05:40:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.362+0000 7f8490ff9700 1 -- 172.21.15.115:0/1012276085 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f849c003ba0 con 0x55f740129d00 2024-05-09T05:40:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.362+0000 7f8490ff9700 1 -- 172.21.15.115:0/1012276085 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c014660 con 0x55f740129d00 2024-05-09T05:40:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.362+0000 7f84937fe700 1 -- 172.21.15.115:0/1645322376 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f84980e3860 con 0x7f84a4107df0 2024-05-09T05:40:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.362+0000 7f84a259c700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f848c0b2a70 0x7f848c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.363+0000 7f84b76d0b80 1 -- 172.21.15.115:0/1012276085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 msgr2=0x55f740232240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:13.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.363+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f740232240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f849c00a730 tx=0x7f849c0078d0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.363+0000 7f84b76d0b80 1 -- 172.21.15.115:0/1012276085 shutdown_connections 2024-05-09T05:40:13.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.363+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f740232240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.363+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74022c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.363+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/1012276085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7400af4a0 0x55f740232780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.363+0000 7f84b76d0b80 1 -- 172.21.15.115:0/1012276085 >> 172.21.15.115:0/1012276085 conn(0x55f7400b2f70 msgr2=0x55f740227220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.363+0000 7f84b76d0b80 1 -- 172.21.15.115:0/1012276085 shutdown_connections 2024-05-09T05:40:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.363+0000 7f84b76d0b80 1 -- 172.21.15.115:0/1012276085 wait complete. 2024-05-09T05:40:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.364+0000 7f84a259c700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f848c0b2a70 0x7f848c0b4f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f740228a00 tx=0x7f84940073d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.364+0000 7f84b76d0b80 1 -- start start 2024-05-09T05:40:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.364+0000 7f84b76d0b80 1 Processor -- start 2024-05-09T05:40:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.364+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7400af4a0 0x55f740236020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740129d00 0x55f740236560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f74022c400 0x55f740236aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f74022d100 con 0x55f7400af4a0 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f7400e9b20 con 0x55f74022c400 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f740242200 con 0x55f740129d00 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84b76d0b80 10 client.0 _ll_register_callbacks cb 0x55f73ffc06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84a259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7400af4a0 0x55f740236020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84a1d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740129d00 0x55f740236560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84a259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7400af4a0 0x55f740236020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38898/0 (socket says 172.21.15.115:38898) 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84a259c700 1 -- 172.21.15.115:0/3818821492 learned_addr learned my addr 172.21.15.115:0/3818821492 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84a2d9d700 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f74022c400 0x55f740236aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84a259c700 1 -- 172.21.15.115:0/3818821492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740129d00 msgr2=0x55f740236560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84a259c700 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740129d00 0x55f740236560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84a259c700 1 -- 172.21.15.115:0/3818821492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f74022c400 msgr2=0x55f740236aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.365+0000 7f84a259c700 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f74022c400 0x55f740236aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.366+0000 7f84a259c700 1 -- 172.21.15.115:0/3818821492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73ffca490 con 0x55f7400af4a0 2024-05-09T05:40:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.366+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740129d00 0x55f740236560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.366+0000 7f84a2d9d700 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f74022c400 0x55f740236aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.366+0000 7f84a259c700 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7400af4a0 0x55f740236020 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f84940109f0 tx=0x7f8494004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c43d4c801a858 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.366+0000 7f8489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.366+0000 7f8489ffb700 1 -- 172.21.15.115:0/3818821492 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849402a060 con 0x55f7400af4a0 2024-05-09T05:40:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.366+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3818821492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f849c003ba0 con 0x55f7400af4a0 2024-05-09T05:40:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.366+0000 7f8489ffb700 1 -- 172.21.15.115:0/3818821492 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f849401d850 con 0x55f7400af4a0 2024-05-09T05:40:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.367+0000 7f8489ffb700 1 -- 172.21.15.115:0/3818821492 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8494031050 con 0x55f7400af4a0 2024-05-09T05:40:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.367+0000 7f84b76d0b80 10 client.16125 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.367+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3818821492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f740243940 con 0x55f7400af4a0 2024-05-09T05:40:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.367+0000 7f8489ffb700 1 -- 172.21.15.115:0/3818821492 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f849c003ba0 con 0x55f7400af4a0 2024-05-09T05:40:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.367+0000 7f8489ffb700 1 -- 172.21.15.115:0/3818821492 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8494019040 con 0x55f7400af4a0 2024-05-09T05:40:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.367+0000 7f84b76d0b80 10 client.16125 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.367+0000 7f84b76d0b80 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.368+0000 7f84b76d0b80 20 client.16125 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.368+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f740246280 0x55f740248750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.368+0000 7f84b76d0b80 4 client.16125 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.368+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3818821492 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f74015ca80 con 0x55f740246280 2024-05-09T05:40:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.368+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f740246280 0x55f740248750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.369+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f740246280 0x55f740248750 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec90d3507e1fc43 server_cookie=ce57b29e05c3dfd1 in_seq=0 out_seq=0 2024-05-09T05:40:13.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.369+0000 7f8489ffb700 10 client.16125 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:13.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f8489ffb700 1 -- 172.21.15.115:0/3818821492 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f74015ca80 con 0x55f740246280 2024-05-09T05:40:13.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f8489ffb700 10 client.16125 handle_command_reply: tid=0 2024-05-09T05:40:13.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 client.16125 shutdown 2024-05-09T05:40:13.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3818821492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7400af4a0 msgr2=0x55f740236020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:13.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7400af4a0 0x55f740236020 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f84940109f0 tx=0x7f8494004750 comp rx=0 tx=0).stop 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3818821492 shutdown_connections 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f74022c400 0x55f740236aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f740246280 0x55f740248750 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f7400af4a0 0x55f740236020 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3818821492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740129d00 0x55f740236560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3818821492 >> 172.21.15.115:0/3818821492 conn(0x55f7400b2f70 msgr2=0x55f740220ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f8489ffb700 0 client.16125 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3818821492 shutdown_connections 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.370+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3818821492 wait complete. 2024-05-09T05:40:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.371+0000 7f84b76d0b80 20 client.16125 trim_cache size 0 max 16384 2024-05-09T05:40:13.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:13 smithi195 ceph-mon[33844]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-05-09T05:40:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.396+0000 7f84b76d0b80 1 Processor -- start 2024-05-09T05:40:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.396+0000 7f84b76d0b80 1 -- start start 2024-05-09T05:40:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f740129d00 0x55f740221700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f74022c400 0x55f740241de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740242320 0x55f74024df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41b2200 con 0x55f740129d00 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41b3d40 con 0x55f740242320 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41b28d0 con 0x55f74022c400 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84a259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f740129d00 0x55f740221700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84a259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f740129d00 0x55f740221700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38902/0 (socket says 172.21.15.115:38902) 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84a259c700 1 -- 172.21.15.115:0/3963301806 learned_addr learned my addr 172.21.15.115:0/3963301806 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/3963301806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f74022c400 0x55f740241de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.397+0000 7f84a2d9d700 1 --2- 172.21.15.115:0/3963301806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740242320 0x55f74024df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.398+0000 7f84a259c700 1 -- 172.21.15.115:0/3963301806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f74022c400 msgr2=0x55f740241de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.398+0000 7f84a259c700 1 --2- 172.21.15.115:0/3963301806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f74022c400 0x55f740241de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.398+0000 7f84a259c700 1 -- 172.21.15.115:0/3963301806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740242320 msgr2=0x55f74024df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.398+0000 7f84a259c700 1 --2- 172.21.15.115:0/3963301806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740242320 0x55f74024df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.398+0000 7f84a259c700 1 -- 172.21.15.115:0/3963301806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7402239d0 con 0x55f740129d00 2024-05-09T05:40:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.398+0000 7f84a259c700 1 --2- 172.21.15.115:0/3963301806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f740129d00 0x55f740221700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f849401bcd0 tx=0x7f849401a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1446c742946cf72c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:13.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.398+0000 7f848bfff700 1 -- 172.21.15.115:0/3963301806 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849401d230 con 0x55f740129d00 2024-05-09T05:40:13.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.398+0000 7f848bfff700 1 -- 172.21.15.115:0/3963301806 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f849401d3d0 con 0x55f740129d00 2024-05-09T05:40:13.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.398+0000 7f848bfff700 1 -- 172.21.15.115:0/3963301806 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849401de20 con 0x55f740129d00 2024-05-09T05:40:13.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.399+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3963301806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f740129d00 msgr2=0x55f740221700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:13.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.399+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3963301806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f740129d00 0x55f740221700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f849401bcd0 tx=0x7f849401a550 comp rx=0 tx=0).stop 2024-05-09T05:40:13.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.399+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3963301806 shutdown_connections 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.399+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3963301806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740242320 0x55f74024df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.399+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3963301806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f740129d00 0x55f740221700 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.399+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3963301806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f74022c400 0x55f740241de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.399+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3963301806 >> 172.21.15.115:0/3963301806 conn(0x55f7400b2f70 msgr2=0x55f740220ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.399+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3963301806 shutdown_connections 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.400+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3963301806 wait complete. 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.400+0000 7f84b76d0b80 1 -- start start 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.400+0000 7f84b76d0b80 1 Processor -- start 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.400+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f74023e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74023e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84b76d0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740242320 0x55f740239860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41b3670 con 0x55f74022c400 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84b76d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41b2fa0 con 0x55f740129d00 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74023e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74023e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38910/0 (socket says 172.21.15.115:38910) 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a1d9b700 1 -- 172.21.15.115:0/3746730283 learned_addr learned my addr 172.21.15.115:0/3746730283 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a259c700 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f74023e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3746730283 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41b01a0 con 0x55f740242320 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84b76d0b80 10 client.0 _ll_register_callbacks cb 0x55f73ffc06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a259c700 1 -- 172.21.15.115:0/3746730283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740242320 msgr2=0x55f740239860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a259c700 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740242320 0x55f740239860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a259c700 1 -- 172.21.15.115:0/3746730283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 msgr2=0x55f74023e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a259c700 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74023e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a259c700 1 -- 172.21.15.115:0/3746730283 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f740239fe0 con 0x55f740129d00 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f84a259c700 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f74023e2e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8494017040 tx=0x7f849401eb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0fd4be3270fcaf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.401+0000 7f8489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.402+0000 7f8489ffb700 1 -- 172.21.15.115:0/3746730283 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849402a040 con 0x55f740129d00 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.402+0000 7f8489ffb700 1 -- 172.21.15.115:0/3746730283 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f849401b660 con 0x55f740129d00 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.402+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3746730283 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f849401d3d0 con 0x55f740129d00 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.402+0000 7f8489ffb700 1 -- 172.21.15.115:0/3746730283 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8494048050 con 0x55f740129d00 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.402+0000 7f84b76d0b80 10 client.25646 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.402+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3746730283 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f7402b05f0 con 0x55f740129d00 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.402+0000 7f8489ffb700 1 -- 172.21.15.115:0/3746730283 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f849403a070 con 0x55f740129d00 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f8489ffb700 1 -- 172.21.15.115:0/3746730283 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8494019070 con 0x55f740129d00 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f84b76d0b80 10 client.25646 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f84b76d0b80 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f84b76d0b80 20 client.25646 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f7402b0be0 0x55f7402b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f84b76d0b80 4 client.25646 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:40:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3746730283 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f849401d3d0 con 0x55f7402b0be0 2024-05-09T05:40:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f7402b0be0 0x55f7402b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f84a1d9b700 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f7402b0be0 0x55f7402b3000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d66c8c4f8732758 server_cookie=645cf80f95d1522c in_seq=0 out_seq=0 2024-05-09T05:40:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.403+0000 7f8489ffb700 10 client.25646 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f8489ffb700 1 -- 172.21.15.115:0/3746730283 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f849401d3d0 con 0x55f7402b0be0 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f8489ffb700 10 client.25646 handle_command_reply: tid=0 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 client.25646 shutdown 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3746730283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 msgr2=0x55f74023e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f74023e2e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8494017040 tx=0x7f849401eb10 comp rx=0 tx=0).stop 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3746730283 shutdown_connections 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f740129d00 0x55f74023e2e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f7402b0be0 0x55f7402b3000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f8489ffb700 0 client.25646 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f740242320 0x55f740239860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 --2- 172.21.15.115:0/3746730283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f74022c400 0x55f74023e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3746730283 >> 172.21.15.115:0/3746730283 conn(0x55f7400b2f70 msgr2=0x55f74022b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3746730283 shutdown_connections 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.404+0000 7f84b76d0b80 1 -- 172.21.15.115:0/3746730283 wait complete. 2024-05-09T05:40:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84b76d0b80 20 client.25646 trim_cache size 0 max 16384 2024-05-09T05:40:13.401 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:40:13.401 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:40:13.401 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:40:13.401 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f848c0b2a70 msgr2=0x7f848c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f848c0b2a70 0x7f848c0b4f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f740228a00 tx=0x7f84940073d0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a4107df0 msgr2=0x7f84a419d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a4107df0 0x7f84a419d2d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f849800f690 tx=0x7f849800f6c0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 shutdown_connections 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f848c0b2a70 0x7f848c0b4f80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84a4108740 0x7f84a41a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a41034c0 0x7f84a419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 --2- 172.21.15.115:0/1645322376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84a4107df0 0x7f84a419d2d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.405+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 >> 172.21.15.115:0/1645322376 conn(0x7f84a40fedc0 msgr2=0x7f84a4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.406+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 shutdown_connections 2024-05-09T05:40:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:13.406+0000 7f84a8cba700 1 -- 172.21.15.115:0/1645322376 wait complete. 2024-05-09T05:40:13.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:13 smithi161 ceph-mon[33805]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-05-09T05:40:14.463 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:40:14.463 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:40:15.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:15 smithi195 ceph-mon[33844]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-09T05:40:15.373 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:15 smithi115 ceph-mon[30349]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-09T05:40:15.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:15 smithi161 ceph-mon[33805]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-09T05:40:15.440 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:16.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.119+0000 7f56b1d4f700 1 -- 172.21.15.115:0/1645301431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac107df0 msgr2=0x7f56ac108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:16.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.119+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/1645301431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac107df0 0x7f56ac108200 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f56940099c0 tx=0x7f5694009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.120+0000 7f56b1d4f700 1 -- 172.21.15.115:0/1645301431 shutdown_connections 2024-05-09T05:40:16.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.120+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/1645301431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56ac108740 0x7f56ac1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.120+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/1645301431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56ac1034c0 0x7f56ac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.120+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/1645301431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac107df0 0x7f56ac108200 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.120+0000 7f56b1d4f700 1 -- 172.21.15.115:0/1645301431 >> 172.21.15.115:0/1645301431 conn(0x7f56ac0fedf0 msgr2=0x7f56ac101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.120+0000 7f56b1d4f700 1 -- 172.21.15.115:0/1645301431 shutdown_connections 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.120+0000 7f56b1d4f700 1 -- 172.21.15.115:0/1645301431 wait complete. 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.120+0000 7f56b1d4f700 1 Processor -- start 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.120+0000 7f56b1d4f700 1 -- start start 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56b1d4f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56ac1034c0 0x7f56ac19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56b1d4f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56ac107df0 0x7f56ac19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56b1d4f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac108740 0x7f56ac19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56b1d4f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac113270 con 0x7f56ac108740 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56b1d4f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1130f0 con 0x7f56ac1034c0 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56b1d4f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1133f0 con 0x7f56ac107df0 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56abfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac108740 0x7f56ac19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56abfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac108740 0x7f56ac19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38922/0 (socket says 172.21.15.115:38922) 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56abfff700 1 -- 172.21.15.115:0/2792222074 learned_addr learned my addr 172.21.15.115:0/2792222074 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56ac1034c0 0x7f56ac19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56a3fff700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56ac107df0 0x7f56ac19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56abfff700 1 -- 172.21.15.115:0/2792222074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56ac107df0 msgr2=0x7f56ac19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56abfff700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56ac107df0 0x7f56ac19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56abfff700 1 -- 172.21.15.115:0/2792222074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56ac1034c0 msgr2=0x7f56ac19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56abfff700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56ac1034c0 0x7f56ac19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56abfff700 1 -- 172.21.15.115:0/2792222074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac19fce0 con 0x7f56ac108740 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56abfff700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac108740 0x7f56ac19f640 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f56a4007ee0 tx=0x7f56a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56a97fa700 1 -- 172.21.15.115:0/2792222074 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a400ddd0 con 0x7f56ac108740 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56b1d4f700 1 -- 172.21.15.115:0/2792222074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac19ffd0 con 0x7f56ac108740 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.121+0000 7f56a97fa700 1 -- 172.21.15.115:0/2792222074 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56a4004020 con 0x7f56ac108740 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.122+0000 7f56a97fa700 1 -- 172.21.15.115:0/2792222074 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4015660 con 0x7f56ac108740 2024-05-09T05:40:16.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.122+0000 7f56b1d4f700 1 -- 172.21.15.115:0/2792222074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac1a78f0 con 0x7f56ac108740 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.124+0000 7f56a97fa700 1 -- 172.21.15.115:0/2792222074 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f56a40041d0 con 0x7f56ac108740 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.125+0000 7f56a97fa700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f56980b29f0 0x7f56980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.125+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f56980b29f0 0x7f56980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.125+0000 7f56a97fa700 1 -- 172.21.15.115:0/2792222074 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f56a40e2440 con 0x7f56ac108740 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.125+0000 7f56c0765b80 1 Processor -- start 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.125+0000 7f56c0765b80 1 -- start start 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.125+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f56980b29f0 0x7f56980b4f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f569c009a10 tx=0x7f569c009190 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.125+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634675f3ea0 0x5634676f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.125+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 0x5634676f6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634677009a0 0x563467702dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56c0765b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563467619240 con 0x5634677009a0 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56c0765b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634675beb40 con 0x5634675f3ea0 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56c0765b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563467615200 con 0x563467579640 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56abfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634677009a0 0x563467702dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56abfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634677009a0 0x563467702dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38938/0 (socket says 172.21.15.115:38938) 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56abfff700 1 -- 172.21.15.115:0/79018972 learned_addr learned my addr 172.21.15.115:0/79018972 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56a3fff700 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 0x5634676f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634675f3ea0 0x5634676f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56abfff700 1 -- 172.21.15.115:0/79018972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 msgr2=0x5634676f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56abfff700 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 0x5634676f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56abfff700 1 -- 172.21.15.115:0/79018972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634675f3ea0 msgr2=0x5634676f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56abfff700 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634675f3ea0 0x5634676f5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56abfff700 1 -- 172.21.15.115:0/79018972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634675af6a0 con 0x5634677009a0 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634675f3ea0 0x5634676f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56abfff700 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634677009a0 0x563467702dc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f56a4004870 tx=0x7f56a40a9b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f029ca1dce858f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56a1ffb700 1 -- 172.21.15.115:0/79018972 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a40b21e0 con 0x5634677009a0 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56a1ffb700 1 -- 172.21.15.115:0/79018972 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56a4009410 con 0x5634677009a0 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.126+0000 7f56a1ffb700 1 -- 172.21.15.115:0/79018972 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a40a9840 con 0x5634677009a0 2024-05-09T05:40:16.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.127+0000 7f56c0765b80 1 -- 172.21.15.115:0/79018972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634677009a0 msgr2=0x563467702dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.127+0000 7f56c0765b80 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634677009a0 0x563467702dc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f56a4004870 tx=0x7f56a40a9b50 comp rx=0 tx=0).stop 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.127+0000 7f56c0765b80 1 -- 172.21.15.115:0/79018972 shutdown_connections 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.127+0000 7f56c0765b80 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634675f3ea0 0x5634676f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.127+0000 7f56c0765b80 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634677009a0 0x563467702dc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.127+0000 7f56c0765b80 1 --2- 172.21.15.115:0/79018972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 0x5634676f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.127+0000 7f56c0765b80 1 -- 172.21.15.115:0/79018972 >> 172.21.15.115:0/79018972 conn(0x56346757d110 msgr2=0x5634676f1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.127+0000 7f56c0765b80 1 -- 172.21.15.115:0/79018972 shutdown_connections 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.127+0000 7f56c0765b80 1 -- 172.21.15.115:0/79018972 wait complete. 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56c0765b80 1 -- start start 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56c0765b80 1 Processor -- start 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 0x5634676fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x5634676fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 0x5634676fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56c0765b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563467703300 con 0x5634675f3ea0 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56c0765b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634675b3cc0 con 0x5634677009a0 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56c0765b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563467703480 con 0x563467579640 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56c0765b80 10 client.0 _ll_register_callbacks cb 0x56346748a860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x5634676fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x5634676fdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38942/0 (socket says 172.21.15.115:38942) 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.128+0000 7f56a3fff700 1 -- 172.21.15.115:0/4288809197 learned_addr learned my addr 172.21.15.115:0/4288809197 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56abfff700 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 0x5634676fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 0x5634676fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56a3fff700 1 -- 172.21.15.115:0/4288809197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 msgr2=0x5634676fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56a3fff700 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 0x5634676fd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56a3fff700 1 -- 172.21.15.115:0/4288809197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 msgr2=0x5634676fe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56a3fff700 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 0x5634676fe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56a3fff700 1 -- 172.21.15.115:0/4288809197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563467494630 con 0x5634675f3ea0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56a3fff700 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x5634676fdeb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f56ac076b40 tx=0x7f56940057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffb4bd9d674dd79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f568affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f568affd700 1 -- 172.21.15.115:0/4288809197 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694027060 con 0x5634675f3ea0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f568affd700 1 -- 172.21.15.115:0/4288809197 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f569400bba0 con 0x5634675f3ea0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56c0765b80 1 -- 172.21.15.115:0/4288809197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56a4009410 con 0x5634675f3ea0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f568affd700 1 -- 172.21.15.115:0/4288809197 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f569402e050 con 0x5634675f3ea0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56c0765b80 10 client.16143 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f56c0765b80 1 -- 172.21.15.115:0/4288809197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56346770def0 con 0x5634675f3ea0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.129+0000 7f568affd700 1 -- 172.21.15.115:0/4288809197 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56a4009410 con 0x5634675f3ea0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.130+0000 7f568affd700 1 -- 172.21.15.115:0/4288809197 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5694021070 con 0x5634675f3ea0 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.130+0000 7f56c0765b80 10 client.16143 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.130+0000 7f56c0765b80 10 client.16143 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.130+0000 7f56c0765b80 20 client.16143 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:16.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.130+0000 7f56c0765b80 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563467710530 0x563467712950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.130+0000 7f56c0765b80 4 client.16143 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.130+0000 7f56c0765b80 1 -- 172.21.15.115:0/4288809197 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563467626c20 con 0x563467710530 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.130+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563467710530 0x563467712950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.131+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563467710530 0x563467712950 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e70697eb27a8a87 server_cookie=6a2e1dbeca8ec9b9 in_seq=0 out_seq=0 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.131+0000 7f568affd700 10 client.16143 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.131+0000 7f568affd700 1 -- 172.21.15.115:0/4288809197 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563467626c20 con 0x563467710530 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.131+0000 7f568affd700 10 client.16143 handle_command_reply: tid=0 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.131+0000 7f56c0765b80 1 client.16143 shutdown 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.131+0000 7f56c0765b80 1 -- 172.21.15.115:0/4288809197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 msgr2=0x5634676fdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x5634676fdeb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f56ac076b40 tx=0x7f56940057c0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 1 -- 172.21.15.115:0/4288809197 shutdown_connections 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 0x5634676fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563467710530 0x563467712950 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563467579640 0x5634676fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 1 --2- 172.21.15.115:0/4288809197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x5634676fdeb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 1 -- 172.21.15.115:0/4288809197 >> 172.21.15.115:0/4288809197 conn(0x56346757d110 msgr2=0x5634676f0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f568affd700 0 client.16143 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 1 -- 172.21.15.115:0/4288809197 shutdown_connections 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 1 -- 172.21.15.115:0/4288809197 wait complete. 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.132+0000 7f56c0765b80 20 client.16143 trim_cache size 0 max 16384 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.151+0000 7f56c0765b80 1 Processor -- start 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.151+0000 7f56c0765b80 1 -- start start 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563467579640 0x5634676ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x56346770c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634677009a0 0x5634676f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56c0765b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1b0300 con 0x5634675f3ea0 2024-05-09T05:40:16.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56c0765b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1b1e40 con 0x563467579640 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56c0765b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1b09d0 con 0x5634677009a0 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x56346770c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 30, 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 6, 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stdout: "9333f06f": { 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 6, 2024-05-09T05:40:16.466 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 144.9, 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:16.467 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x56346770c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38946/0 (socket says 172.21.15.115:38946) 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a3fff700 1 -- 172.21.15.115:0/1810263172 learned_addr learned my addr 172.21.15.115:0/1810263172 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/1810263172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563467579640 0x5634676ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56abfff700 1 --2- 172.21.15.115:0/1810263172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634677009a0 0x5634676f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a3fff700 1 -- 172.21.15.115:0/1810263172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634677009a0 msgr2=0x5634676f7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:16.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a3fff700 1 --2- 172.21.15.115:0/1810263172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634677009a0 0x5634676f7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a3fff700 1 -- 172.21.15.115:0/1810263172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563467579640 msgr2=0x5634676ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a3fff700 1 --2- 172.21.15.115:0/1810263172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563467579640 0x5634676ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a3fff700 1 -- 172.21.15.115:0/1810263172 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563467700770 con 0x5634675f3ea0 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a3fff700 1 --2- 172.21.15.115:0/1810263172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x56346770c390 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f56940098e0 tx=0x7f5694009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0b02bc3b95b1e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a17fa700 1 -- 172.21.15.115:0/1810263172 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569401b070 con 0x5634675f3ea0 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a17fa700 1 -- 172.21.15.115:0/1810263172 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f569400fa50 con 0x5634675f3ea0 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.152+0000 7f56a17fa700 1 -- 172.21.15.115:0/1810263172 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694027030 con 0x5634675f3ea0 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.153+0000 7f56c0765b80 1 -- 172.21.15.115:0/1810263172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 msgr2=0x56346770c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.153+0000 7f56c0765b80 1 --2- 172.21.15.115:0/1810263172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x56346770c390 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f56940098e0 tx=0x7f5694009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.153+0000 7f56c0765b80 1 -- 172.21.15.115:0/1810263172 shutdown_connections 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.153+0000 7f56c0765b80 1 --2- 172.21.15.115:0/1810263172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563467579640 0x5634676ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.153+0000 7f56c0765b80 1 --2- 172.21.15.115:0/1810263172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634677009a0 0x5634676f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.153+0000 7f56c0765b80 1 --2- 172.21.15.115:0/1810263172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5634675f3ea0 0x56346770c390 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.153+0000 7f56c0765b80 1 -- 172.21.15.115:0/1810263172 >> 172.21.15.115:0/1810263172 conn(0x56346757d110 msgr2=0x5634676f7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:16.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.153+0000 7f56c0765b80 1 -- 172.21.15.115:0/1810263172 shutdown_connections 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.153+0000 7f56c0765b80 1 -- 172.21.15.115:0/1810263172 wait complete. 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56c0765b80 1 -- start start 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56c0765b80 1 Processor -- start 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563467579640 0x563467708920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634675f3ea0 0x5634677036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56c0765b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 0x563467703c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56c0765b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1b1770 con 0x563467579640 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56c0765b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1b10a0 con 0x5634677009a0 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56c0765b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1ae2a0 con 0x5634675f3ea0 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56c0765b80 10 client.0 _ll_register_callbacks cb 0x5634676f6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56ab7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563467579640 0x563467708920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56ab7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563467579640 0x563467708920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38948/0 (socket says 172.21.15.115:38948) 2024-05-09T05:40:16.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56ab7fe700 1 -- 172.21.15.115:0/3347575352 learned_addr learned my addr 172.21.15.115:0/3347575352 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56a3fff700 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634675f3ea0 0x5634677036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.154+0000 7f56abfff700 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 0x563467703c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f56ab7fe700 1 -- 172.21.15.115:0/3347575352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634675f3ea0 msgr2=0x5634677036d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634675f3ea0 0x5634677036d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f56ab7fe700 1 -- 172.21.15.115:0/3347575352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 msgr2=0x563467703c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 0x563467703c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f56ab7fe700 1 -- 172.21.15.115:0/3347575352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563467709140 con 0x563467579640 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f56ab7fe700 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563467579640 0x563467708920 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5634676fd1f0 tx=0x7f569c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8225e2d70209e831 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f568affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f568affd700 1 -- 172.21.15.115:0/3347575352 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c027340 con 0x563467579640 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f568affd700 1 -- 172.21.15.115:0/3347575352 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f569c00f040 con 0x563467579640 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f56c0765b80 1 -- 172.21.15.115:0/3347575352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569400fa50 con 0x563467579640 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f568affd700 1 -- 172.21.15.115:0/3347575352 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f569c0274e0 con 0x563467579640 2024-05-09T05:40:16.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f56c0765b80 10 client.16149 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f56c0765b80 1 -- 172.21.15.115:0/3347575352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56346777b800 con 0x563467579640 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.155+0000 7f568affd700 1 -- 172.21.15.115:0/3347575352 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f569400fa50 con 0x563467579640 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f568affd700 1 -- 172.21.15.115:0/3347575352 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f569c02e070 con 0x563467579640 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f56c0765b80 10 client.16149 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f56c0765b80 10 client.16149 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f56c0765b80 20 client.16149 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f56c0765b80 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56346777bfb0 0x56346777e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f56c0765b80 4 client.16149 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}] 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f56c0765b80 1 -- 172.21.15.115:0/3347575352 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}) -- 0x563467620cc0 con 0x56346777bfb0 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f56a3fff700 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56346777bfb0 0x56346777e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f568affd700 10 client.16149 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.156+0000 7f56a3fff700 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56346777bfb0 0x56346777e3d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67aa032a76c55c5d server_cookie=b246d8354920a406 in_seq=0 out_seq=0 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f568affd700 1 -- 172.21.15.115:0/3347575352 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x563467620cc0 con 0x56346777bfb0 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f568affd700 10 client.16149 handle_command_reply: tid=0 2024-05-09T05:40:16.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 client.16149 shutdown 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 -- 172.21.15.115:0/3347575352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563467579640 msgr2=0x563467708920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563467579640 0x563467708920 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5634676fd1f0 tx=0x7f569c004750 comp rx=0 tx=0).stop 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 -- 172.21.15.115:0/3347575352 shutdown_connections 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634677009a0 0x563467703c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56346777bfb0 0x56346777e3d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563467579640 0x563467708920 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 --2- 172.21.15.115:0/3347575352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5634675f3ea0 0x5634677036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f568affd700 0 client.16149 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 -- 172.21.15.115:0/3347575352 >> 172.21.15.115:0/3347575352 conn(0x56346757d110 msgr2=0x5634676f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 -- 172.21.15.115:0/3347575352 shutdown_connections 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.165+0000 7f56c0765b80 1 -- 172.21.15.115:0/3347575352 wait complete. 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56c0765b80 20 client.16149 trim_cache size 0 max 16384 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 -- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f56980b29f0 msgr2=0x7f56980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:16.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f56980b29f0 0x7f56980b4f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f569c009a10 tx=0x7f569c009190 comp rx=0 tx=0).stop 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 -- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac108740 msgr2=0x7f56ac19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac108740 0x7f56ac19f640 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f56a4007ee0 tx=0x7f56a400d3b0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 -- 172.21.15.115:0/2792222074 shutdown_connections 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f56980b29f0 0x7f56980b4f00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56ac1034c0 0x7f56ac19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56ac108740 0x7f56ac19f640 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 --2- 172.21.15.115:0/2792222074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56ac107df0 0x7f56ac19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 -- 172.21.15.115:0/2792222074 >> 172.21.15.115:0/2792222074 conn(0x7f56ac0fedf0 msgr2=0x7f56ac075e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 -- 172.21.15.115:0/2792222074 shutdown_connections 2024-05-09T05:40:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:16.166+0000 7f56b1d4f700 1 -- 172.21.15.115:0/2792222074 wait complete. 2024-05-09T05:40:17.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:17 smithi195 ceph-mon[33844]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T05:40:17.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:17 smithi115 ceph-mon[30349]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T05:40:17.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:17 smithi161 ceph-mon[33805]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-09T05:40:18.192 INFO:tasks.workunit.client.0.smithi115.stderr:Updating files: 94% (11617/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-09T05:40:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 -- 172.21.15.115:0/2223620970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec64103310 msgr2=0x7fec6410f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 --2- 172.21.15.115:0/2223620970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec64103310 0x7fec6410f890 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c210 tx=0x7fec5c00c520 comp rx=0 tx=0).stop 2024-05-09T05:40:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 -- 172.21.15.115:0/2223620970 shutdown_connections 2024-05-09T05:40:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 --2- 172.21.15.115:0/2223620970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec64103310 0x7fec6410f890 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 --2- 172.21.15.115:0/2223620970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec641089f0 0x7fec64108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 --2- 172.21.15.115:0/2223620970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec641029c0 0x7fec64102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 -- 172.21.15.115:0/2223620970 >> 172.21.15.115:0/2223620970 conn(0x7fec640fe6f0 msgr2=0x7fec64100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 -- 172.21.15.115:0/2223620970 shutdown_connections 2024-05-09T05:40:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 -- 172.21.15.115:0/2223620970 wait complete. 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.525+0000 7fec69a72700 1 Processor -- start 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec69a72700 1 -- start start 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec69a72700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec641029c0 0x7fec6419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec69a72700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec64103310 0x7fec6419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec69a72700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec641089f0 0x7fec6419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec69a72700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fec641121a0 con 0x7fec641089f0 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec69a72700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec64112020 con 0x7fec641029c0 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec69a72700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fec64112320 con 0x7fec64103310 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec63fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec641089f0 0x7fec6419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec63fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec641089f0 0x7fec6419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38960/0 (socket says 172.21.15.115:38960) 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec63fff700 1 -- 172.21.15.115:0/2072905127 learned_addr learned my addr 172.21.15.115:0/2072905127 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec637fe700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec641029c0 0x7fec6419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec62ffd700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec64103310 0x7fec6419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec63fff700 1 -- 172.21.15.115:0/2072905127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec64103310 msgr2=0x7fec6419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec63fff700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec64103310 0x7fec6419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec63fff700 1 -- 172.21.15.115:0/2072905127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec641029c0 msgr2=0x7fec6419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec63fff700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec641029c0 0x7fec6419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.526+0000 7fec63fff700 1 -- 172.21.15.115:0/2072905127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6419fca0 con 0x7fec641089f0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.527+0000 7fec63fff700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec641089f0 0x7fec6419f600 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00a720 tx=0x7fec5c0084b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.527+0000 7fec60ff9700 1 -- 172.21.15.115:0/2072905127 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c022070 con 0x7fec641089f0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.527+0000 7fec60ff9700 1 -- 172.21.15.115:0/2072905127 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec5c009d20 con 0x7fec641089f0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.527+0000 7fec60ff9700 1 -- 172.21.15.115:0/2072905127 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0052a0 con 0x7fec641089f0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.527+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6419ff30 con 0x7fec641089f0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.527+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec641a0420 con 0x7fec641089f0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.528+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec640512c0 con 0x7fec641089f0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.530+0000 7fec60ff9700 1 -- 172.21.15.115:0/2072905127 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fec5c025070 con 0x7fec641089f0 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.530+0000 7fec60ff9700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fec4c0b2a70 0x7fec4c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.530+0000 7fec637fe700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fec4c0b2a70 0x7fec4c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:18.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.530+0000 7fec60ff9700 1 -- 172.21.15.115:0/2072905127 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fec5c010020 con 0x7fec641089f0 2024-05-09T05:40:18.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.531+0000 7fec637fe700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fec4c0b2a70 0x7fec4c0b4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec54005950 tx=0x7fec5400a300 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:18.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.539+0000 7fec60ff9700 1 -- 172.21.15.115:0/2072905127 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec5c02c020 con 0x7fec641089f0 2024-05-09T05:40:18.637 INFO:tasks.workunit.client.0.smithi115.stderr:Note: switching to 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'. 2024-05-09T05:40:18.637 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-05-09T05:40:18.637 INFO:tasks.workunit.client.0.smithi115.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T05:40:18.637 INFO:tasks.workunit.client.0.smithi115.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T05:40:18.637 INFO:tasks.workunit.client.0.smithi115.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T05:40:18.637 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-05-09T05:40:18.637 INFO:tasks.workunit.client.0.smithi115.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T05:40:18.637 INFO:tasks.workunit.client.0.smithi115.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr: git switch -c 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr:Or undo this operation with: 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr: git switch - 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr: 2024-05-09T05:40:18.638 INFO:tasks.workunit.client.0.smithi115.stderr:HEAD is now at c9da4049f63 Merge PR #57081 into wip-pdonnell-testing-20240508.183908-debug 2024-05-09T05:40:18.643 DEBUG:teuthology.orchestra.run.smithi115:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T05:40:18.730 INFO:tasks.workunit.client.0.smithi115.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T05:40:18.732 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T05:40:18.732 INFO:tasks.workunit.client.0.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T05:40:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.796+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec6402e5f0 con 0x7fec641089f0 2024-05-09T05:40:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.797+0000 7fec60ff9700 1 -- 172.21.15.115:0/2072905127 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7fec5c0aba80 con 0x7fec641089f0 2024-05-09T05:40:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:40:18.790 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:40:18.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fec4c0b2a70 msgr2=0x7fec4c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:18.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fec4c0b2a70 0x7fec4c0b4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec54005950 tx=0x7fec5400a300 comp rx=0 tx=0).stop 2024-05-09T05:40:18.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec641089f0 msgr2=0x7fec6419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:18.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec641089f0 0x7fec6419f600 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00a720 tx=0x7fec5c0084b0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 shutdown_connections 2024-05-09T05:40:18.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fec4c0b2a70 0x7fec4c0b4f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec641029c0 0x7fec6419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec641089f0 0x7fec6419f600 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 --2- 172.21.15.115:0/2072905127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec64103310 0x7fec6419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:18.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 >> 172.21.15.115:0/2072905127 conn(0x7fec640fe6f0 msgr2=0x7fec64105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:18.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 shutdown_connections 2024-05-09T05:40:18.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:18.800+0000 7fec69a72700 1 -- 172.21.15.115:0/2072905127 wait complete. 2024-05-09T05:40:18.796 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:40:19.159 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9333f06f', quiesce took 15.7 seconds. Will release after: 9.25 2024-05-09T05:40:19.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:19 smithi195 ceph-mon[33844]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 682 B/s wr, 14 op/s 2024-05-09T05:40:19.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:19 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2072905127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:19.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:19 smithi115 ceph-mon[30349]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 682 B/s wr, 14 op/s 2024-05-09T05:40:19.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:19 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2072905127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:19.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:19 smithi161 ceph-mon[33805]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 682 B/s wr, 14 op/s 2024-05-09T05:40:19.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:19 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2072905127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:20.075 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:40:20.327 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:20.661 INFO:tasks.workunit.client.0.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T05:40:20.731 INFO:tasks.workunit.client.0.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T05:40:20.770 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T05:40:20.772 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T05:40:20.772 INFO:tasks.workunit.client.0.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T05:40:20.816 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T05:40:20.819 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T05:40:20.820 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T05:40:20.836 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-05-09T05:40:20.837 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-05-09T05:40:20.837 DEBUG:teuthology.orchestra.run.smithi115:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c9da4049f638c0f84bdd9b4e33c42e86046ac554 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-09T05:40:21.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:21 smithi195 ceph-mon[33844]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T05:40:21.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:21 smithi115 ceph-mon[30349]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T05:40:21.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:21 smithi161 ceph-mon[33805]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T05:40:22.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.291+0000 7f4414b74700 1 -- 172.21.15.115:0/4020688355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410108f10 msgr2=0x7f4410109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:22.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.291+0000 7f4414b74700 1 --2- 172.21.15.115:0/4020688355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410108f10 0x7f4410109320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f44040099c0 tx=0x7f4404009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.291+0000 7f4414b74700 1 -- 172.21.15.115:0/4020688355 shutdown_connections 2024-05-09T05:40:22.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.291+0000 7f4414b74700 1 --2- 172.21.15.115:0/4020688355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44101045e0 0x7f44101049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.291+0000 7f4414b74700 1 --2- 172.21.15.115:0/4020688355 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4410109860 0x7f44101111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.291+0000 7f4414b74700 1 --2- 172.21.15.115:0/4020688355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410108f10 0x7f4410109320 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.291+0000 7f4414b74700 1 -- 172.21.15.115:0/4020688355 >> 172.21.15.115:0/4020688355 conn(0x7f44100ffef0 msgr2=0x7f4410102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:22.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.291+0000 7f4414b74700 1 -- 172.21.15.115:0/4020688355 shutdown_connections 2024-05-09T05:40:22.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.291+0000 7f4414b74700 1 -- 172.21.15.115:0/4020688355 wait complete. 2024-05-09T05:40:22.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f4414b74700 1 Processor -- start 2024-05-09T05:40:22.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f4414b74700 1 -- start start 2024-05-09T05:40:22.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f4414b74700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44101045e0 0x7f441019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f4414b74700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4410108f10 0x7f441019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f4414b74700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410109860 0x7f441019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f4414b74700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4410114320 con 0x7f4410109860 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f4414b74700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44101141a0 con 0x7f4410108f10 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f4414b74700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f44101144a0 con 0x7f44101045e0 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f440ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410109860 0x7f441019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f440ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410109860 0x7f441019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38978/0 (socket says 172.21.15.115:38978) 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f440ed9d700 1 -- 172.21.15.115:0/3942582376 learned_addr learned my addr 172.21.15.115:0/3942582376 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.292+0000 7f440e59c700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44101045e0 0x7f441019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f440dd9b700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4410108f10 0x7f441019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f440ed9d700 1 -- 172.21.15.115:0/3942582376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44101045e0 msgr2=0x7f441019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f440ed9d700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44101045e0 0x7f441019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f440ed9d700 1 -- 172.21.15.115:0/3942582376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4410108f10 msgr2=0x7f441019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f440ed9d700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4410108f10 0x7f441019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f440ed9d700 1 -- 172.21.15.115:0/3942582376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441019fc40 con 0x7f4410109860 2024-05-09T05:40:22.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f440ed9d700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410109860 0x7f441019f5a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f440800ada0 tx=0x7f440800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:22.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f43ff7fe700 1 -- 172.21.15.115:0/3942582376 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408017020 con 0x7f4410109860 2024-05-09T05:40:22.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f43ff7fe700 1 -- 172.21.15.115:0/3942582376 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f440800cd60 con 0x7f4410109860 2024-05-09T05:40:22.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f4414b74700 1 -- 172.21.15.115:0/3942582376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441019ff30 con 0x7f4410109860 2024-05-09T05:40:22.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f43ff7fe700 1 -- 172.21.15.115:0/3942582376 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408005280 con 0x7f4410109860 2024-05-09T05:40:22.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.293+0000 7f4414b74700 1 -- 172.21.15.115:0/3942582376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44101a7850 con 0x7f4410109860 2024-05-09T05:40:22.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.296+0000 7f43ff7fe700 1 -- 172.21.15.115:0/3942582376 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4408005420 con 0x7f4410109860 2024-05-09T05:40:22.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.297+0000 7f43ff7fe700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43f80b29f0 0x7f43f80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.297+0000 7f442358ab80 1 Processor -- start 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.297+0000 7f440e59c700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43f80b29f0 0x7f43f80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.297+0000 7f43ff7fe700 1 -- 172.21.15.115:0/3942582376 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f44080e2170 con 0x7f4410109860 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.297+0000 7f442358ab80 1 -- start start 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440e59c700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43f80b29f0 0x7f43f80b4f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4400005950 tx=0x7f440000a300 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f442358ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f784ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f442358ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f785030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f442358ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f442358ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557a4f6a30a0 con 0x557a4f6034a0 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f442358ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557a4f6489a0 con 0x557a4f67dd00 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f442358ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557a4f69f060 con 0x557a4f7891e0 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f785030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f784ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f785030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38990/0 (socket says 172.21.15.115:38990) 2024-05-09T05:40:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440dd9b700 1 -- 172.21.15.115:0/3814502007 learned_addr learned my addr 172.21.15.115:0/3814502007 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440ed9d700 1 --2- 172.21.15.115:0/3814502007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440dd9b700 1 -- 172.21.15.115:0/3814502007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 msgr2=0x557a4f7895f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440dd9b700 1 --2- 172.21.15.115:0/3814502007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7895f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440dd9b700 1 -- 172.21.15.115:0/3814502007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 msgr2=0x557a4f784ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440dd9b700 1 --2- 172.21.15.115:0/3814502007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f784ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440dd9b700 1 -- 172.21.15.115:0/3814502007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4f639500 con 0x557a4f6034a0 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440e59c700 1 --2- 172.21.15.115:0/3814502007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f784ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.298+0000 7f440dd9b700 1 --2- 172.21.15.115:0/3814502007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f785030 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f441006c1b0 tx=0x7f4404005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e280cfe76671d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.299+0000 7f43fcff9700 1 -- 172.21.15.115:0/3814502007 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404027070 con 0x557a4f6034a0 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.299+0000 7f43fcff9700 1 -- 172.21.15.115:0/3814502007 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44040054f0 con 0x557a4f6034a0 2024-05-09T05:40:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.299+0000 7f43fcff9700 1 -- 172.21.15.115:0/3814502007 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404003b10 con 0x557a4f6034a0 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 -- 172.21.15.115:0/3814502007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 msgr2=0x557a4f785030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 --2- 172.21.15.115:0/3814502007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f785030 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f441006c1b0 tx=0x7f4404005100 comp rx=0 tx=0).stop 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 -- 172.21.15.115:0/3814502007 shutdown_connections 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 --2- 172.21.15.115:0/3814502007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f784ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 --2- 172.21.15.115:0/3814502007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 --2- 172.21.15.115:0/3814502007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f785030 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 -- 172.21.15.115:0/3814502007 >> 172.21.15.115:0/3814502007 conn(0x557a4f606f70 msgr2=0x557a4f77b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 -- 172.21.15.115:0/3814502007 shutdown_connections 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 -- 172.21.15.115:0/3814502007 wait complete. 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 -- start start 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.300+0000 7f442358ab80 1 Processor -- start 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f442358ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f6034a0 0x557a4f78e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f442358ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f67dd00 0x557a4f78ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f442358ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f7891e0 0x557a4f78ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f442358ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557a4f789ea0 con 0x557a4f7891e0 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f442358ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557a4f63db20 con 0x557a4f6034a0 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f442358ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557a4f78a020 con 0x557a4f67dd00 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f442358ab80 10 client.0 _ll_register_callbacks cb 0x557a4f5146c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f7891e0 0x557a4f78ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f7891e0 0x557a4f78ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39006/0 (socket says 172.21.15.115:39006) 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440ed9d700 1 -- 172.21.15.115:0/2078749821 learned_addr learned my addr 172.21.15.115:0/2078749821 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440e59c700 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f6034a0 0x557a4f78e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440dd9b700 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f67dd00 0x557a4f78ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440ed9d700 1 -- 172.21.15.115:0/2078749821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f67dd00 msgr2=0x557a4f78ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440ed9d700 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f67dd00 0x557a4f78ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440ed9d700 1 -- 172.21.15.115:0/2078749821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f6034a0 msgr2=0x557a4f78e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440ed9d700 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f6034a0 0x557a4f78e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440ed9d700 1 -- 172.21.15.115:0/2078749821 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4f51e490 con 0x557a4f7891e0 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440dd9b700 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f67dd00 0x557a4f78ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f440ed9d700 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f7891e0 0x557a4f78ef50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f44080df580 tx=0x7f44080a8d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60d3396013a0700 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f43f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2078749821 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44080ad4f0 con 0x557a4f7891e0 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f442358ab80 1 -- 172.21.15.115:0/2078749821 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44040054f0 con 0x557a4f7891e0 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2078749821 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f44080ad150 con 0x557a4f7891e0 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.301+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2078749821 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44080b1f10 con 0x557a4f7891e0 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f442358ab80 10 client.16167 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f442358ab80 1 -- 172.21.15.115:0/2078749821 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a4f79bdf0 con 0x557a4f7891e0 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2078749821 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44040054f0 con 0x557a4f7891e0 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2078749821 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4408012070 con 0x557a4f7891e0 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f442358ab80 10 client.16167 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f442358ab80 10 client.16167 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f442358ab80 20 client.16167 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f442358ab80 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x557a4f79e420 0x557a4f7a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f442358ab80 4 client.16167 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f442358ab80 1 -- 172.21.15.115:0/2078749821 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a4f6b0a80 con 0x557a4f79e420 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.302+0000 7f440e59c700 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x557a4f79e420 0x557a4f7a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.303+0000 7f43f5ffb700 10 client.16167 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.303+0000 7f440e59c700 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x557a4f79e420 0x557a4f7a08d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=890fa8ffa203f07d server_cookie=e60f7392e89f5f9d in_seq=0 out_seq=0 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2078749821 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a4f6b0a80 con 0x557a4f79e420 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f43f5ffb700 10 client.16167 handle_command_reply: tid=0 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 client.16167 shutdown 2024-05-09T05:40:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 -- 172.21.15.115:0/2078749821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f7891e0 msgr2=0x557a4f78ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f7891e0 0x557a4f78ef50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f44080df580 tx=0x7f44080a8d00 comp rx=0 tx=0).stop 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 -- 172.21.15.115:0/2078749821 shutdown_connections 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f6034a0 0x557a4f78e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x557a4f79e420 0x557a4f7a08d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f7891e0 0x557a4f78ef50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 --2- 172.21.15.115:0/2078749821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f67dd00 0x557a4f78ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 -- 172.21.15.115:0/2078749821 >> 172.21.15.115:0/2078749821 conn(0x557a4f606f70 msgr2=0x557a4f774ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f43f5ffb700 0 client.16167 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 -- 172.21.15.115:0/2078749821 shutdown_connections 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 1 -- 172.21.15.115:0/2078749821 wait complete. 2024-05-09T05:40:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.304+0000 7f442358ab80 20 client.16167 trim_cache size 0 max 16384 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f442358ab80 1 Processor -- start 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f442358ab80 1 -- start start 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f442358ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f77b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f442358ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f775700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f442358ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f442358ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44101ae050 con 0x557a4f6034a0 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f442358ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44101afb90 con 0x557a4f67dd00 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f442358ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f44101ae720 con 0x557a4f7891e0 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f77b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f77b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39018/0 (socket says 172.21.15.115:39018) 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440e59c700 1 -- 172.21.15.115:0/2035630052 learned_addr learned my addr 172.21.15.115:0/2035630052 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440dd9b700 1 --2- 172.21.15.115:0/2035630052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f775700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440e59c700 1 -- 172.21.15.115:0/2035630052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 msgr2=0x557a4f7a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440e59c700 1 --2- 172.21.15.115:0/2035630052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440e59c700 1 -- 172.21.15.115:0/2035630052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 msgr2=0x557a4f775700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440e59c700 1 --2- 172.21.15.115:0/2035630052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f775700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440e59c700 1 -- 172.21.15.115:0/2035630052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4f790d20 con 0x557a4f6034a0 2024-05-09T05:40:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.319+0000 7f440e59c700 1 --2- 172.21.15.115:0/2035630052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f77b330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f440000f750 tx=0x7f4400012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236211b369685e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f43f7fff700 1 -- 172.21.15.115:0/2035630052 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400016070 con 0x557a4f6034a0 2024-05-09T05:40:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f43f7fff700 1 -- 172.21.15.115:0/2035630052 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f440002d070 con 0x557a4f6034a0 2024-05-09T05:40:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f43f7fff700 1 -- 172.21.15.115:0/2035630052 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000279b0 con 0x557a4f6034a0 2024-05-09T05:40:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f442358ab80 1 -- 172.21.15.115:0/2035630052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 msgr2=0x557a4f77b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f442358ab80 1 --2- 172.21.15.115:0/2035630052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f77b330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f440000f750 tx=0x7f4400012000 comp rx=0 tx=0).stop 2024-05-09T05:40:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f442358ab80 1 -- 172.21.15.115:0/2035630052 shutdown_connections 2024-05-09T05:40:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f442358ab80 1 --2- 172.21.15.115:0/2035630052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f442358ab80 1 --2- 172.21.15.115:0/2035630052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f77b330 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f442358ab80 1 --2- 172.21.15.115:0/2035630052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f775700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.320+0000 7f442358ab80 1 -- 172.21.15.115:0/2035630052 >> 172.21.15.115:0/2035630052 conn(0x557a4f606f70 msgr2=0x557a4f774ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 -- 172.21.15.115:0/2035630052 shutdown_connections 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 -- 172.21.15.115:0/2035630052 wait complete. 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 -- start start 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 Processor -- start 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f795ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f796240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44101af4c0 con 0x557a4f6034a0 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44101aedf0 con 0x557a4f67dd00 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f44101abff0 con 0x557a4f7891e0 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f442358ab80 10 client.0 _ll_register_callbacks cb 0x557a4f5146c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f440e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f795ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f440e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f795ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39034/0 (socket says 172.21.15.115:39034) 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f440e59c700 1 -- 172.21.15.115:0/2098676521 learned_addr learned my addr 172.21.15.115:0/2098676521 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.321+0000 7f440dd9b700 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f796240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f440ed9d700 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f440e59c700 1 -- 172.21.15.115:0/2098676521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 msgr2=0x557a4f7ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f440e59c700 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f440e59c700 1 -- 172.21.15.115:0/2098676521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 msgr2=0x557a4f796240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f440e59c700 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f796240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f440e59c700 1 -- 172.21.15.115:0/2098676521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4f8002c0 con 0x557a4f6034a0 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f440ed9d700 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f440e59c700 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f795ce0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f44000275d0 tx=0x7f44000285e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a8758ffedbfdc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f43f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2098676521 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400037070 con 0x557a4f6034a0 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2098676521 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4400027630 con 0x557a4f6034a0 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f442358ab80 1 -- 172.21.15.115:0/2098676521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440002d070 con 0x557a4f6034a0 2024-05-09T05:40:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2098676521 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4400043050 con 0x557a4f6034a0 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f442358ab80 10 client.16173 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f442358ab80 1 -- 172.21.15.115:0/2098676521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a4f808790 con 0x557a4f6034a0 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.322+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2098676521 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f440002d070 con 0x557a4f6034a0 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2098676521 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4400031700 con 0x557a4f6034a0 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f442358ab80 10 client.16173 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f442358ab80 10 client.16173 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f442358ab80 20 client.16173 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f442358ab80 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x557a4f808e10 0x557a4f80b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f442358ab80 4 client.16173 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f442358ab80 1 -- 172.21.15.115:0/2098676521 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a4f7a0e20 con 0x557a4f808e10 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f440dd9b700 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x557a4f808e10 0x557a4f80b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f43f5ffb700 10 client.16173 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.323+0000 7f440dd9b700 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x557a4f808e10 0x557a4f80b230 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=87045bdf4248017f server_cookie=a49ca9b54023635c in_seq=0 out_seq=0 2024-05-09T05:40:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f43f5ffb700 1 -- 172.21.15.115:0/2098676521 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a4f7a0e20 con 0x557a4f808e10 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f43f5ffb700 10 client.16173 handle_command_reply: tid=0 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 client.16173 shutdown 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 -- 172.21.15.115:0/2098676521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 msgr2=0x557a4f795ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f795ce0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f44000275d0 tx=0x7f44000285e0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 -- 172.21.15.115:0/2098676521 shutdown_connections 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x557a4f808e10 0x557a4f80b230 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557a4f7891e0 0x557a4f7ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557a4f6034a0 0x557a4f795ce0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f43f5ffb700 0 client.16173 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 --2- 172.21.15.115:0/2098676521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a4f67dd00 0x557a4f796240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 -- 172.21.15.115:0/2098676521 >> 172.21.15.115:0/2098676521 conn(0x557a4f606f70 msgr2=0x557a4f77f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 -- 172.21.15.115:0/2098676521 shutdown_connections 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 1 -- 172.21.15.115:0/2098676521 wait complete. 2024-05-09T05:40:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f442358ab80 20 client.16173 trim_cache size 0 max 16384 2024-05-09T05:40:22.323 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f4414b74700 1 -- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43f80b29f0 msgr2=0x7f43f80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f4414b74700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43f80b29f0 0x7f43f80b4f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4400005950 tx=0x7f440000a300 comp rx=0 tx=0).stop 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f4414b74700 1 -- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410109860 msgr2=0x7f441019f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f4414b74700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410109860 0x7f441019f5a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f440800ada0 tx=0x7f440800c330 comp rx=0 tx=0).stop 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.324+0000 7f4414b74700 1 -- 172.21.15.115:0/3942582376 shutdown_connections 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.325+0000 7f4414b74700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43f80b29f0 0x7f43f80b4f00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.325+0000 7f4414b74700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4410109860 0x7f441019f5a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.325+0000 7f4414b74700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44101045e0 0x7f441019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.325+0000 7f4414b74700 1 --2- 172.21.15.115:0/3942582376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4410108f10 0x7f441019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.325+0000 7f4414b74700 1 -- 172.21.15.115:0/3942582376 >> 172.21.15.115:0/3942582376 conn(0x7f44100ffef0 msgr2=0x7f4410102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.325+0000 7f4414b74700 1 -- 172.21.15.115:0/3942582376 shutdown_connections 2024-05-09T05:40:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:22.325+0000 7f4414b74700 1 -- 172.21.15.115:0/3942582376 wait complete. 2024-05-09T05:40:23.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:40:23.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:23 smithi195 ceph-mon[33844]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T05:40:23.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:23 smithi115 ceph-mon[30349]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T05:40:23.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:23 smithi161 ceph-mon[33805]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T05:40:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.103+0000 7fed32419640 1 -- 172.21.15.115:0/329784336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c104de0 msgr2=0x7fed2c1051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.103+0000 7fed32419640 1 --2- 172.21.15.115:0/329784336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c104de0 0x7fed2c1051f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009900 tx=0x7fed1c02a480 comp rx=0 tx=0).stop 2024-05-09T05:40:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.103+0000 7fed32419640 1 -- 172.21.15.115:0/329784336 shutdown_connections 2024-05-09T05:40:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.103+0000 7fed32419640 1 --2- 172.21.15.115:0/329784336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c10e230 0x7fed2c110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.103+0000 7fed32419640 1 --2- 172.21.15.115:0/329784336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed2c10b840 0x7fed2c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.103+0000 7fed32419640 1 --2- 172.21.15.115:0/329784336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c104de0 0x7fed2c1051f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.103+0000 7fed32419640 1 -- 172.21.15.115:0/329784336 >> 172.21.15.115:0/329784336 conn(0x7fed2c0782e0 msgr2=0x7fed2c078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.103+0000 7fed32419640 1 -- 172.21.15.115:0/329784336 shutdown_connections 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.103+0000 7fed32419640 1 -- 172.21.15.115:0/329784336 wait complete. 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.104+0000 7fed32419640 1 Processor -- start 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.104+0000 7fed32419640 1 -- start start 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed32419640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c104de0 0x7fed2c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed32419640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed2c10b840 0x7fed2c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed32419640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c10e230 0x7fed2c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed32419640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c113d20 con 0x7fed2c10e230 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed32419640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c113ba0 con 0x7fed2c10b840 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed32419640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c113ea0 con 0x7fed2c104de0 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed309b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c10e230 0x7fed2c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed309b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c10e230 0x7fed2c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34410/0 (socket says 172.21.15.115:34410) 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed309b6640 1 -- 172.21.15.115:0/2510231948 learned_addr learned my addr 172.21.15.115:0/2510231948 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed2bfff640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c104de0 0x7fed2c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed2c10b840 0x7fed2c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed309b6640 1 -- 172.21.15.115:0/2510231948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c104de0 msgr2=0x7fed2c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed309b6640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c104de0 0x7fed2c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed309b6640 1 -- 172.21.15.115:0/2510231948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed2c10b840 msgr2=0x7fed2c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed309b6640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed2c10b840 0x7fed2c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed309b6640 1 -- 172.21.15.115:0/2510231948 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c19f640 con 0x7fed2c10e230 2024-05-09T05:40:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed2bfff640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c104de0 0x7fed2c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed309b6640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c10e230 0x7fed2c19f000 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fed24009fd0 tx=0x7fed2400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed297fa640 1 -- 172.21.15.115:0/2510231948 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24017070 con 0x7fed2c10e230 2024-05-09T05:40:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed297fa640 1 -- 172.21.15.115:0/2510231948 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed24003e00 con 0x7fed2c10e230 2024-05-09T05:40:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed32419640 1 -- 172.21.15.115:0/2510231948 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c19f930 con 0x7fed2c10e230 2024-05-09T05:40:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed297fa640 1 -- 172.21.15.115:0/2510231948 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24010460 con 0x7fed2c10e230 2024-05-09T05:40:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.105+0000 7fed32419640 1 -- 172.21.15.115:0/2510231948 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c19fe30 con 0x7fed2c10e230 2024-05-09T05:40:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.108+0000 7fed297fa640 1 -- 172.21.15.115:0/2510231948 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fed24003970 con 0x7fed2c10e230 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed393ab740 1 Processor -- start 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed297fa640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fed140b2950 0x7fed140b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed393ab740 1 -- start start 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fed140b2950 0x7fed140b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed297fa640 1 -- 172.21.15.115:0/2510231948 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fed240e2b00 con 0x7fed2c10e230 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed393ab740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 0x558936fde680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed393ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 0x558936fdebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed393ab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936fdf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed393ab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558936f437c0 con 0x558936c1a4d0 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed393ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558936edf0e0 con 0x558936c122f0 2024-05-09T05:40:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed393ab740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558936f3f780 con 0x558936c3b1b0 2024-05-09T05:40:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed309b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936fdf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 0x558936fde680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed309b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936fdf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34414/0 (socket says 172.21.15.115:34414) 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed309b6640 1 -- 172.21.15.115:0/2977511692 learned_addr learned my addr 172.21.15.115:0/2977511692 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.109+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fed140b2950 0x7fed140b4e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fed20005fd0 tx=0x7fed20005850 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.110+0000 7fed309b6640 1 -- 172.21.15.115:0/2977511692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 msgr2=0x558936fde680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.110+0000 7fed309b6640 1 --2- 172.21.15.115:0/2977511692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 0x558936fde680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.110+0000 7fed309b6640 1 -- 172.21.15.115:0/2977511692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 msgr2=0x558936fdebc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.110+0000 7fed309b6640 1 --2- 172.21.15.115:0/2977511692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 0x558936fdebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.110+0000 7fed309b6640 1 -- 172.21.15.115:0/2977511692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558936f087c0 con 0x558936c1a4d0 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.110+0000 7fed309b6640 1 --2- 172.21.15.115:0/2977511692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936fdf100 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fed240a97a0 tx=0x7fed24003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d5ab95e892c756 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.110+0000 7fed127fc640 1 -- 172.21.15.115:0/2977511692 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24020070 con 0x558936c1a4d0 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.110+0000 7fed127fc640 1 -- 172.21.15.115:0/2977511692 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed240aa1a0 con 0x558936c1a4d0 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.110+0000 7fed127fc640 1 -- 172.21.15.115:0/2977511692 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240a5d30 con 0x558936c1a4d0 2024-05-09T05:40:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 -- 172.21.15.115:0/2977511692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 msgr2=0x558936fdf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 --2- 172.21.15.115:0/2977511692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936fdf100 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fed240a97a0 tx=0x7fed24003d20 comp rx=0 tx=0).stop 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 -- 172.21.15.115:0/2977511692 shutdown_connections 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 --2- 172.21.15.115:0/2977511692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936fdf100 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 --2- 172.21.15.115:0/2977511692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 0x558936fdebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 --2- 172.21.15.115:0/2977511692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 0x558936fde680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 -- 172.21.15.115:0/2977511692 >> 172.21.15.115:0/2977511692 conn(0x558936c2f300 msgr2=0x558936fdae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 -- 172.21.15.115:0/2977511692 shutdown_connections 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 -- 172.21.15.115:0/2977511692 wait complete. 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 -- start start 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.111+0000 7fed393ab740 1 Processor -- start 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed393ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 0x558936fea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed393ab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936feade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed393ab740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 0x558936fe8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed393ab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558936feb7b0 con 0x558936c1a4d0 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed393ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558936eca9c0 con 0x558936c122f0 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed393ab740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558936feb930 con 0x558936c3b1b0 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed393ab740 10 client.0 _ll_register_callbacks cb 0x558936ef7020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936feade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936feade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34426/0 (socket says 172.21.15.115:34426) 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2b7fe640 1 -- 172.21.15.115:0/3998459712 learned_addr learned my addr 172.21.15.115:0/3998459712 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2bfff640 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 0x558936fea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed309b6640 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 0x558936fe8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2b7fe640 1 -- 172.21.15.115:0/3998459712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 msgr2=0x558936fe8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 0x558936fe8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2b7fe640 1 -- 172.21.15.115:0/3998459712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 msgr2=0x558936fea8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 0x558936fea8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2b7fe640 1 -- 172.21.15.115:0/3998459712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558936ef11f0 con 0x558936c1a4d0 2024-05-09T05:40:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2bfff640 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 0x558936fea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936feade0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fed2000bf50 tx=0x7fed200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487e24be81456b86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.112+0000 7fed0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed0b7fe640 1 -- 172.21.15.115:0/3998459712 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20022070 con 0x558936c1a4d0 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed0b7fe640 1 -- 172.21.15.115:0/3998459712 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fed2001e860 con 0x558936c1a4d0 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed393ab740 1 -- 172.21.15.115:0/3998459712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed240aa1a0 con 0x558936c1a4d0 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed0b7fe640 1 -- 172.21.15.115:0/3998459712 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed2002e050 con 0x558936c1a4d0 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed393ab740 10 client.16185 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed393ab740 1 -- 172.21.15.115:0/3998459712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558936ca6790 con 0x558936c1a4d0 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed0b7fe640 1 -- 172.21.15.115:0/3998459712 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed240aa1a0 con 0x558936c1a4d0 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed0b7fe640 1 -- 172.21.15.115:0/3998459712 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed20010970 con 0x558936c1a4d0 2024-05-09T05:40:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed393ab740 10 client.16185 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed393ab740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed393ab740 20 client.16185 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed393ab740 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558936bf9d80 0x558936ffb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed393ab740 4 client.16185 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.113+0000 7fed393ab740 1 -- 172.21.15.115:0/3998459712 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558936f513d0 con 0x558936bf9d80 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.114+0000 7fed2bfff640 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558936bf9d80 0x558936ffb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.114+0000 7fed0b7fe640 10 client.16185 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.114+0000 7fed2bfff640 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558936bf9d80 0x558936ffb030 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c27519f4238a2f server_cookie=5f235eed19ce6157 in_seq=0 out_seq=0 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed0b7fe640 1 -- 172.21.15.115:0/3998459712 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558936f513d0 con 0x558936bf9d80 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed0b7fe640 10 client.16185 handle_command_reply: tid=0 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 client.16185 shutdown 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 -- 172.21.15.115:0/3998459712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 msgr2=0x558936feade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936feade0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fed2000bf50 tx=0x7fed200046c0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 -- 172.21.15.115:0/3998459712 shutdown_connections 2024-05-09T05:40:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558936bf9d80 0x558936ffb030 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c3b1b0 0x558936fe8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c1a4d0 0x558936feade0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed0b7fe640 0 client.16185 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 --2- 172.21.15.115:0/3998459712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c122f0 0x558936fea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 -- 172.21.15.115:0/3998459712 >> 172.21.15.115:0/3998459712 conn(0x558936c2f300 msgr2=0x558936fd9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 -- 172.21.15.115:0/3998459712 shutdown_connections 2024-05-09T05:40:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 1 -- 172.21.15.115:0/3998459712 wait complete. 2024-05-09T05:40:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.115+0000 7fed393ab740 20 client.16185 trim_cache size 0 max 16384 2024-05-09T05:40:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed393ab740 1 Processor -- start 2024-05-09T05:40:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed393ab740 1 -- start start 2024-05-09T05:40:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed393ab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936fe7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed393ab740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 0x558936fe7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed393ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 0x558936ff0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed393ab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b1110 con 0x558936c122f0 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed393ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b2c50 con 0x558936c3b1b0 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed393ab740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b17e0 con 0x558936c1a4d0 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936fe7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936fe7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34442/0 (socket says 172.21.15.115:34442) 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed2bfff640 1 -- 172.21.15.115:0/4133427551 learned_addr learned my addr 172.21.15.115:0/4133427551 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed309b6640 1 --2- 172.21.15.115:0/4133427551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 0x558936ff0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/4133427551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 0x558936fe7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed2bfff640 1 -- 172.21.15.115:0/4133427551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 msgr2=0x558936fe7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed2bfff640 1 --2- 172.21.15.115:0/4133427551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 0x558936fe7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed2bfff640 1 -- 172.21.15.115:0/4133427551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 msgr2=0x558936ff0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed2bfff640 1 --2- 172.21.15.115:0/4133427551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 0x558936ff0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.131+0000 7fed2bfff640 1 -- 172.21.15.115:0/4133427551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558936d1d5f0 con 0x558936c122f0 2024-05-09T05:40:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.132+0000 7fed2bfff640 1 --2- 172.21.15.115:0/4133427551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936fe7560 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fed1c000c00 tx=0x7fed1c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c09715c4fbe2d2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.132+0000 7fed11ffb640 1 -- 172.21.15.115:0/4133427551 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c049070 con 0x558936c122f0 2024-05-09T05:40:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.132+0000 7fed11ffb640 1 -- 172.21.15.115:0/4133427551 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed1c0044b0 con 0x558936c122f0 2024-05-09T05:40:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.132+0000 7fed11ffb640 1 -- 172.21.15.115:0/4133427551 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c035040 con 0x558936c122f0 2024-05-09T05:40:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 -- 172.21.15.115:0/4133427551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 msgr2=0x558936fe7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 --2- 172.21.15.115:0/4133427551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936fe7560 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fed1c000c00 tx=0x7fed1c009560 comp rx=0 tx=0).stop 2024-05-09T05:40:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 -- 172.21.15.115:0/4133427551 shutdown_connections 2024-05-09T05:40:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 --2- 172.21.15.115:0/4133427551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 0x558936ff0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 --2- 172.21.15.115:0/4133427551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 0x558936fe7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 --2- 172.21.15.115:0/4133427551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936fe7560 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 -- 172.21.15.115:0/4133427551 >> 172.21.15.115:0/4133427551 conn(0x558936c2f300 msgr2=0x558936fe3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 -- 172.21.15.115:0/4133427551 shutdown_connections 2024-05-09T05:40:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 -- 172.21.15.115:0/4133427551 wait complete. 2024-05-09T05:40:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 -- start start 2024-05-09T05:40:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.133+0000 7fed393ab740 1 Processor -- start 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed393ab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936ff11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed393ab740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 0x558936fee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed393ab740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 0x558936feead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed393ab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b2580 con 0x558936c122f0 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed393ab740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b1eb0 con 0x558936c3b1b0 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed393ab740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1af0f0 con 0x558936c1a4d0 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed393ab740 10 client.0 _ll_register_callbacks cb 0x7fed20015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936ff11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936ff11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34458/0 (socket says 172.21.15.115:34458) 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2bfff640 1 -- 172.21.15.115:0/1858308669 learned_addr learned my addr 172.21.15.115:0/1858308669 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed309b6640 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 0x558936feead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 0x558936fee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2bfff640 1 -- 172.21.15.115:0/1858308669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 msgr2=0x558936fee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2bfff640 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 0x558936fee550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2bfff640 1 -- 172.21.15.115:0/1858308669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 msgr2=0x558936feead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2bfff640 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 0x558936feead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2bfff640 1 -- 172.21.15.115:0/1858308669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558936f07ed0 con 0x558936c122f0 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 0x558936fee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed2bfff640 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936ff11e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009230 tx=0x7fed1c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c4c09d5b74112f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed0b7fe640 1 -- 172.21.15.115:0/1858308669 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c049040 con 0x558936c122f0 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed0b7fe640 1 -- 172.21.15.115:0/1858308669 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fed1c004cf0 con 0x558936c122f0 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.134+0000 7fed393ab740 1 -- 172.21.15.115:0/1858308669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed1c0044b0 con 0x558936c122f0 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed0b7fe640 1 -- 172.21.15.115:0/1858308669 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed1c05a050 con 0x558936c122f0 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed393ab740 10 client.16191 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed393ab740 1 -- 172.21.15.115:0/1858308669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589370631b0 con 0x558936c122f0 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed0b7fe640 1 -- 172.21.15.115:0/1858308669 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed1c0044b0 con 0x558936c122f0 2024-05-09T05:40:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed0b7fe640 1 -- 172.21.15.115:0/1858308669 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed1c055030 con 0x558936c122f0 2024-05-09T05:40:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed393ab740 10 client.16191 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed393ab740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed393ab740 20 client.16191 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed393ab740 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558936bcf1c0 0x5589370658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed393ab740 4 client.16191 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:40:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed393ab740 1 -- 172.21.15.115:0/1858308669 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558936f4b240 con 0x558936bcf1c0 2024-05-09T05:40:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.135+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558936bcf1c0 0x5589370658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.140+0000 7fed0b7fe640 10 client.16191 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.140+0000 7fed2b7fe640 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558936bcf1c0 0x5589370658b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0009ef511880322 server_cookie=555aa87b906d4459 in_seq=0 out_seq=0 2024-05-09T05:40:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed0b7fe640 1 -- 172.21.15.115:0/1858308669 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558936f4b240 con 0x558936bcf1c0 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed0b7fe640 10 client.16191 handle_command_reply: tid=0 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 client.16191 shutdown 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 -- 172.21.15.115:0/1858308669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 msgr2=0x558936ff11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936ff11e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009230 tx=0x7fed1c004740 comp rx=0 tx=0).stop 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 -- 172.21.15.115:0/1858308669 shutdown_connections 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558936c3b1b0 0x558936feead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558936bcf1c0 0x5589370658b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558936c1a4d0 0x558936fee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed0b7fe640 0 client.16191 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 --2- 172.21.15.115:0/1858308669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558936c122f0 0x558936ff11e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 -- 172.21.15.115:0/1858308669 >> 172.21.15.115:0/1858308669 conn(0x558936c2f300 msgr2=0x558936fe7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 -- 172.21.15.115:0/1858308669 shutdown_connections 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 1 -- 172.21.15.115:0/1858308669 wait complete. 2024-05-09T05:40:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.146+0000 7fed393ab740 20 client.16191 trim_cache size 0 max 16384 2024-05-09T05:40:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:40:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:40:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 -- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fed140b2950 msgr2=0x7fed140b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fed140b2950 0x7fed140b4e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fed20005fd0 tx=0x7fed20005850 comp rx=0 tx=0).stop 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 -- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c10e230 msgr2=0x7fed2c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c10e230 0x7fed2c19f000 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fed24009fd0 tx=0x7fed2400b260 comp rx=0 tx=0).stop 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 -- 172.21.15.115:0/2510231948 shutdown_connections 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fed140b2950 0x7fed140b4e60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2c10e230 0x7fed2c19f000 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed2c10b840 0x7fed2c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 --2- 172.21.15.115:0/2510231948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed2c104de0 0x7fed2c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 -- 172.21.15.115:0/2510231948 >> 172.21.15.115:0/2510231948 conn(0x7fed2c0782e0 msgr2=0x7fed2c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 -- 172.21.15.115:0/2510231948 shutdown_connections 2024-05-09T05:40:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:25.147+0000 7fed32419640 1 -- 172.21.15.115:0/2510231948 wait complete. 2024-05-09T05:40:25.169 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:25 smithi115 ceph-mon[30349]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-09T05:40:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.290+0000 7fc953650700 1 -- 172.21.15.115:0/2105939013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 msgr2=0x7fc94c108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.290+0000 7fc953650700 1 --2- 172.21.15.115:0/2105939013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 0x7fc94c108610 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009990 tx=0x7fc93c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.290+0000 7fc953650700 1 -- 172.21.15.115:0/2105939013 shutdown_connections 2024-05-09T05:40:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.290+0000 7fc953650700 1 --2- 172.21.15.115:0/2105939013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc94c1042f0 0x7fc94c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.290+0000 7fc953650700 1 --2- 172.21.15.115:0/2105939013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 0x7fc94c108610 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.290+0000 7fc953650700 1 --2- 172.21.15.115:0/2105939013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc94c1039a0 0x7fc94c103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.290+0000 7fc953650700 1 -- 172.21.15.115:0/2105939013 >> 172.21.15.115:0/2105939013 conn(0x7fc94c0fe4e0 msgr2=0x7fc94c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.290+0000 7fc953650700 1 -- 172.21.15.115:0/2105939013 shutdown_connections 2024-05-09T05:40:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.290+0000 7fc953650700 1 -- 172.21.15.115:0/2105939013 wait complete. 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.291+0000 7fc953650700 1 Processor -- start 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.291+0000 7fc953650700 1 -- start start 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.291+0000 7fc953650700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc94c1039a0 0x7fc94c19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.291+0000 7fc953650700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc94c1042f0 0x7fc94c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.291+0000 7fc953650700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 0x7fc94c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.291+0000 7fc953650700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c113240 con 0x7fc94c108200 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.291+0000 7fc953650700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1130c0 con 0x7fc94c1042f0 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.291+0000 7fc953650700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1133c0 con 0x7fc94c1039a0 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.291+0000 7fc951bed700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 0x7fc94c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc951bed700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 0x7fc94c199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34468/0 (socket says 172.21.15.115:34468) 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc951bed700 1 -- 172.21.15.115:0/2425146249 learned_addr learned my addr 172.21.15.115:0/2425146249 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc9513ec700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc94c1039a0 0x7fc94c19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc950beb700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc94c1042f0 0x7fc94c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc951bed700 1 -- 172.21.15.115:0/2425146249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc94c1039a0 msgr2=0x7fc94c19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc951bed700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc94c1039a0 0x7fc94c19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc951bed700 1 -- 172.21.15.115:0/2425146249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc94c1042f0 msgr2=0x7fc94c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc951bed700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc94c1042f0 0x7fc94c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc951bed700 1 -- 172.21.15.115:0/2425146249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c19a1e0 con 0x7fc94c108200 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc950beb700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc94c1042f0 0x7fc94c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc951bed700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 0x7fc94c199b00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc94400e580 tx=0x7fc94400e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc9427fc700 1 -- 172.21.15.115:0/2425146249 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944004db0 con 0x7fc94c108200 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc953650700 1 -- 172.21.15.115:0/2425146249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c19a4d0 con 0x7fc94c108200 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc9427fc700 1 -- 172.21.15.115:0/2425146249 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9440090d0 con 0x7fc94c108200 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc953650700 1 -- 172.21.15.115:0/2425146249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c102890 con 0x7fc94c108200 2024-05-09T05:40:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.292+0000 7fc9427fc700 1 -- 172.21.15.115:0/2425146249 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440045b0 con 0x7fc94c108200 2024-05-09T05:40:25.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.295+0000 7fc9427fc700 1 -- 172.21.15.115:0/2425146249 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc944003a00 con 0x7fc94c108200 2024-05-09T05:40:25.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.296+0000 7fc9427fc700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc9380b29f0 0x7fc9380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.296+0000 7fc9513ec700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc9380b29f0 0x7fc9380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.296+0000 7fc9427fc700 1 -- 172.21.15.115:0/2425146249 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc9440e49c0 con 0x7fc94c108200 2024-05-09T05:40:25.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.296+0000 7fc9513ec700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc9380b29f0 0x7fc9380b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc94c19ac40 tx=0x7fc93c005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.298+0000 7fc962066b80 1 Processor -- start 2024-05-09T05:40:25.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.298+0000 7fc962066b80 1 -- start start 2024-05-09T05:40:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.298+0000 7fc962066b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x5557323434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.298+0000 7fc962066b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732343a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.298+0000 7fc962066b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55573233d6a0 0x55573233dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.298+0000 7fc962066b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55573225fff0 con 0x5557321c04a0 2024-05-09T05:40:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.298+0000 7fc962066b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5557321fe7f0 con 0x55573233d6a0 2024-05-09T05:40:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.298+0000 7fc962066b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55573225bfb0 con 0x5557322aa160 2024-05-09T05:40:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.298+0000 7fc950beb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732343a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc950beb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732343a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34476/0 (socket says 172.21.15.115:34476) 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc950beb700 1 -- 172.21.15.115:0/193115252 learned_addr learned my addr 172.21.15.115:0/193115252 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9513ec700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x5557323434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc951bed700 1 --2- 172.21.15.115:0/193115252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55573233d6a0 0x55573233dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9513ec700 1 -- 172.21.15.115:0/193115252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55573233d6a0 msgr2=0x55573233dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9513ec700 1 --2- 172.21.15.115:0/193115252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55573233d6a0 0x55573233dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9513ec700 1 -- 172.21.15.115:0/193115252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 msgr2=0x555732343a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9513ec700 1 --2- 172.21.15.115:0/193115252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732343a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9513ec700 1 -- 172.21.15.115:0/193115252 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557321f6370 con 0x5557322aa160 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9513ec700 1 --2- 172.21.15.115:0/193115252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x5557323434f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc93c000c00 tx=0x7fc93c01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91d30f89a7f83b5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9377fe700 1 -- 172.21.15.115:0/193115252 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00fac0 con 0x5557322aa160 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9377fe700 1 -- 172.21.15.115:0/193115252 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc93c00be90 con 0x5557322aa160 2024-05-09T05:40:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.299+0000 7fc9377fe700 1 -- 172.21.15.115:0/193115252 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0233f0 con 0x5557322aa160 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.300+0000 7fc962066b80 1 -- 172.21.15.115:0/193115252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 msgr2=0x5557323434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.300+0000 7fc962066b80 1 --2- 172.21.15.115:0/193115252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x5557323434f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc93c000c00 tx=0x7fc93c01e000 comp rx=0 tx=0).stop 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.300+0000 7fc962066b80 1 -- 172.21.15.115:0/193115252 shutdown_connections 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.300+0000 7fc962066b80 1 --2- 172.21.15.115:0/193115252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55573233d6a0 0x55573233dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.300+0000 7fc962066b80 1 --2- 172.21.15.115:0/193115252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x5557323434f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.300+0000 7fc962066b80 1 --2- 172.21.15.115:0/193115252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732343a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.300+0000 7fc962066b80 1 -- 172.21.15.115:0/193115252 >> 172.21.15.115:0/193115252 conn(0x5557322206a0 msgr2=0x555732338400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.300+0000 7fc962066b80 1 -- 172.21.15.115:0/193115252 shutdown_connections 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.300+0000 7fc962066b80 1 -- 172.21.15.115:0/193115252 wait complete. 2024-05-09T05:40:25.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.301+0000 7fc962066b80 1 -- start start 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.301+0000 7fc962066b80 1 Processor -- start 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc962066b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732347200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc962066b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557322aa160 0x555732347740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc962066b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573233d6a0 0x555732347c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc962066b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5557320dbba0 con 0x5557321c04a0 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc962066b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55573221a150 con 0x5557322aa160 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc962066b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55573234e5e0 con 0x55573233d6a0 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc962066b80 10 client.0 _ll_register_callbacks cb 0x5557320d16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc9513ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732347200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc9513ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732347200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34480/0 (socket says 172.21.15.115:34480) 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc9513ec700 1 -- 172.21.15.115:0/972014912 learned_addr learned my addr 172.21.15.115:0/972014912 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc951bed700 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573233d6a0 0x555732347c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc950beb700 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557322aa160 0x555732347740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc9513ec700 1 -- 172.21.15.115:0/972014912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573233d6a0 msgr2=0x555732347c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc9513ec700 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573233d6a0 0x555732347c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc9513ec700 1 -- 172.21.15.115:0/972014912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557322aa160 msgr2=0x555732347740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc9513ec700 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557322aa160 0x555732347740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc9513ec700 1 -- 172.21.15.115:0/972014912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557320db490 con 0x5557321c04a0 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc9513ec700 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732347200 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc93c005a30 tx=0x7fc93c00ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25af52a19c326df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.302+0000 7fc934ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc934ff9700 1 -- 172.21.15.115:0/972014912 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c023b80 con 0x5557321c04a0 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc934ff9700 1 -- 172.21.15.115:0/972014912 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc93c023d20 con 0x5557321c04a0 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc962066b80 1 -- 172.21.15.115:0/972014912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc93c00be90 con 0x5557321c04a0 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc934ff9700 1 -- 172.21.15.115:0/972014912 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc93c042050 con 0x5557321c04a0 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc962066b80 10 client.16203 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc962066b80 1 -- 172.21.15.115:0/972014912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555732358fb0 con 0x5557321c04a0 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc934ff9700 1 -- 172.21.15.115:0/972014912 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc93c00be90 con 0x5557321c04a0 2024-05-09T05:40:25.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc934ff9700 1 -- 172.21.15.115:0/972014912 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc93c022070 con 0x5557321c04a0 2024-05-09T05:40:25.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc962066b80 10 client.16203 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:25.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc962066b80 10 client.16203 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:40:25.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc962066b80 20 client.16203 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:25.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc962066b80 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55573235b480 0x55573235d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc962066b80 4 client.16203 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:25.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.303+0000 7fc962066b80 1 -- 172.21.15.115:0/972014912 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55573226d9d0 con 0x55573235b480 2024-05-09T05:40:25.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.304+0000 7fc950beb700 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55573235b480 0x55573235d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.304+0000 7fc934ff9700 10 client.16203 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:40:25.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.304+0000 7fc950beb700 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55573235b480 0x55573235d8a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b70c679242af004 server_cookie=808a79b40450db2 in_seq=0 out_seq=0 2024-05-09T05:40:25.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc934ff9700 1 -- 172.21.15.115:0/972014912 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55573226d9d0 con 0x55573235b480 2024-05-09T05:40:25.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc934ff9700 10 client.16203 handle_command_reply: tid=0 2024-05-09T05:40:25.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 client.16203 shutdown 2024-05-09T05:40:25.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 -- 172.21.15.115:0/972014912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 msgr2=0x555732347200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732347200 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc93c005a30 tx=0x7fc93c00ff20 comp rx=0 tx=0).stop 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 -- 172.21.15.115:0/972014912 shutdown_connections 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55573235b480 0x55573235d8a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55573233d6a0 0x555732347c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557321c04a0 0x555732347200 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc934ff9700 0 client.16203 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 --2- 172.21.15.115:0/972014912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557322aa160 0x555732347740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 -- 172.21.15.115:0/972014912 >> 172.21.15.115:0/972014912 conn(0x5557322206a0 msgr2=0x5557323381c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 -- 172.21.15.115:0/972014912 shutdown_connections 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 1 -- 172.21.15.115:0/972014912 wait complete. 2024-05-09T05:40:25.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.305+0000 7fc962066b80 20 client.16203 trim_cache size 0 max 16384 2024-05-09T05:40:25.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc962066b80 1 Processor -- start 2024-05-09T05:40:25.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc962066b80 1 -- start start 2024-05-09T05:40:25.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc962066b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x555732344db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc962066b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x555732347af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc962066b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 0x5557323608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc962066b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1ae1c0 con 0x55573233d6a0 2024-05-09T05:40:25.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc962066b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1afd00 con 0x555732346200 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc962066b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1ae890 con 0x5557322aa160 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc950beb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x555732347af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc950beb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x555732347af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34482/0 (socket says 172.21.15.115:34482) 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc950beb700 1 -- 172.21.15.115:0/2633899304 learned_addr learned my addr 172.21.15.115:0/2633899304 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc9513ec700 1 --2- 172.21.15.115:0/2633899304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x555732344db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.336+0000 7fc951bed700 1 --2- 172.21.15.115:0/2633899304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 0x5557323608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.337+0000 7fc950beb700 1 -- 172.21.15.115:0/2633899304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 msgr2=0x555732344db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.337+0000 7fc950beb700 1 --2- 172.21.15.115:0/2633899304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x555732344db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.337+0000 7fc950beb700 1 -- 172.21.15.115:0/2633899304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 msgr2=0x5557323608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.337+0000 7fc950beb700 1 --2- 172.21.15.115:0/2633899304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 0x5557323608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.337+0000 7fc950beb700 1 -- 172.21.15.115:0/2633899304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55573234a000 con 0x55573233d6a0 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.337+0000 7fc950beb700 1 --2- 172.21.15.115:0/2633899304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x555732347af0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x555732339ed0 tx=0x7fc94800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360acc7a25c243ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.337+0000 7fc936ffd700 1 -- 172.21.15.115:0/2633899304 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948021020 con 0x55573233d6a0 2024-05-09T05:40:25.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.337+0000 7fc936ffd700 1 -- 172.21.15.115:0/2633899304 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc94801d8b0 con 0x55573233d6a0 2024-05-09T05:40:25.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.337+0000 7fc936ffd700 1 -- 172.21.15.115:0/2633899304 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94801cba0 con 0x55573233d6a0 2024-05-09T05:40:25.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.338+0000 7fc962066b80 1 -- 172.21.15.115:0/2633899304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 msgr2=0x555732347af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.338+0000 7fc962066b80 1 --2- 172.21.15.115:0/2633899304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x555732347af0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x555732339ed0 tx=0x7fc94800b6c0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.338+0000 7fc962066b80 1 -- 172.21.15.115:0/2633899304 shutdown_connections 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.338+0000 7fc962066b80 1 --2- 172.21.15.115:0/2633899304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 0x5557323608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.338+0000 7fc962066b80 1 --2- 172.21.15.115:0/2633899304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x555732344db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.338+0000 7fc962066b80 1 --2- 172.21.15.115:0/2633899304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x555732347af0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.338+0000 7fc962066b80 1 -- 172.21.15.115:0/2633899304 >> 172.21.15.115:0/2633899304 conn(0x5557322206a0 msgr2=0x555732345af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.338+0000 7fc962066b80 1 -- 172.21.15.115:0/2633899304 shutdown_connections 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.338+0000 7fc962066b80 1 -- 172.21.15.115:0/2633899304 wait complete. 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.339+0000 7fc962066b80 1 -- start start 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.339+0000 7fc962066b80 1 Processor -- start 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.339+0000 7fc962066b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x555732344be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.339+0000 7fc962066b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x55573235ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.339+0000 7fc962066b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 0x55573235f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.339+0000 7fc962066b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1af630 con 0x55573233d6a0 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.339+0000 7fc962066b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1aef60 con 0x555732346200 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.339+0000 7fc962066b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1ac160 con 0x5557322aa160 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc962066b80 10 client.0 _ll_register_callbacks cb 0x5557320d16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc950beb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x55573235ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc950beb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x55573235ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34486/0 (socket says 172.21.15.115:34486) 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc950beb700 1 -- 172.21.15.115:0/1968244423 learned_addr learned my addr 172.21.15.115:0/1968244423 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc9513ec700 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x555732344be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc951bed700 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 0x55573235f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc950beb700 1 -- 172.21.15.115:0/1968244423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 msgr2=0x555732344be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc950beb700 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x555732344be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc950beb700 1 -- 172.21.15.115:0/1968244423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 msgr2=0x55573235f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc950beb700 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 0x55573235f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc950beb700 1 -- 172.21.15.115:0/1968244423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555732339d00 con 0x55573233d6a0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc951bed700 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 0x55573235f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc950beb700 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x55573235ee80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc94801dbc0 tx=0x7fc94801dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac0b6ba3939603a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc934ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc934ff9700 1 -- 172.21.15.115:0/1968244423 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948021030 con 0x55573233d6a0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc962066b80 1 -- 172.21.15.115:0/1968244423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc94801d8b0 con 0x55573233d6a0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.340+0000 7fc934ff9700 1 -- 172.21.15.115:0/1968244423 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc94801ca20 con 0x55573233d6a0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc934ff9700 1 -- 172.21.15.115:0/1968244423 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc94802c070 con 0x55573233d6a0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc962066b80 10 client.16209 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc962066b80 1 -- 172.21.15.115:0/1968244423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557323c5550 con 0x55573233d6a0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc934ff9700 1 -- 172.21.15.115:0/1968244423 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc94801d8b0 con 0x55573233d6a0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc934ff9700 1 -- 172.21.15.115:0/1968244423 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc948026c20 con 0x55573233d6a0 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc962066b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc962066b80 10 client.16209 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:40:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc962066b80 20 client.16209 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc962066b80 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5557323c5b70 0x5557323c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc962066b80 4 client.16209 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc962066b80 1 -- 172.21.15.115:0/1968244423 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555732267a70 con 0x5557323c5b70 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.341+0000 7fc9513ec700 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5557323c5b70 0x5557323c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc934ff9700 10 client.16209 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc9513ec700 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5557323c5b70 0x5557323c7f90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7689f2d1db23688a server_cookie=862d11b8d91ade01 in_seq=0 out_seq=0 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc934ff9700 1 -- 172.21.15.115:0/1968244423 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555732267a70 con 0x5557323c5b70 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc934ff9700 10 client.16209 handle_command_reply: tid=0 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 client.16209 shutdown 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 -- 172.21.15.115:0/1968244423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 msgr2=0x55573235ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x55573235ee80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc94801dbc0 tx=0x7fc94801dbf0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 -- 172.21.15.115:0/1968244423 shutdown_connections 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555732346200 0x55573235f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5557323c5b70 0x5557323c7f90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557322aa160 0x555732344be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 --2- 172.21.15.115:0/1968244423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55573233d6a0 0x55573235ee80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 -- 172.21.15.115:0/1968244423 >> 172.21.15.115:0/1968244423 conn(0x5557322206a0 msgr2=0x555732340e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc934ff9700 0 client.16209 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:40:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 -- 172.21.15.115:0/1968244423 shutdown_connections 2024-05-09T05:40:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 1 -- 172.21.15.115:0/1968244423 wait complete. 2024-05-09T05:40:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.342+0000 7fc962066b80 20 client.16209 trim_cache size 0 max 16384 2024-05-09T05:40:25.340 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 -- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc9380b29f0 msgr2=0x7fc9380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc9380b29f0 0x7fc9380b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc94c19ac40 tx=0x7fc93c005fb0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 -- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 msgr2=0x7fc94c199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 0x7fc94c199b00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc94400e580 tx=0x7fc94400e890 comp rx=0 tx=0).stop 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 -- 172.21.15.115:0/2425146249 shutdown_connections 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc9380b29f0 0x7fc9380b4f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc94c108200 0x7fc94c199b00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc94c1039a0 0x7fc94c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 --2- 172.21.15.115:0/2425146249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc94c1042f0 0x7fc94c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 -- 172.21.15.115:0/2425146249 >> 172.21.15.115:0/2425146249 conn(0x7fc94c0fe4e0 msgr2=0x7fc94c109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 -- 172.21.15.115:0/2425146249 shutdown_connections 2024-05-09T05:40:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:25.343+0000 7fc953650700 1 -- 172.21.15.115:0/2425146249 wait complete. 2024-05-09T05:40:25.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:25 smithi195 ceph-mon[33844]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-09T05:40:25.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:25 smithi161 ceph-mon[33805]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-09T05:40:26.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:40:26.513 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:27.373 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:27 smithi115 ceph-mon[30349]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:40:27.373 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:27 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:40:27.373 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:27 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:40:27.373 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:27 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:40:27.373 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:27 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:40:27.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:27 smithi161 ceph-mon[33805]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:40:27.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:27 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:40:27.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:27 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:40:27.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:27 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:40:27.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:27 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:40:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:27 smithi195 ceph-mon[33844]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-09T05:40:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:27 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:40:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:27 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:40:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:27 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:40:27.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:27 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:40:28.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 -- 172.21.15.115:0/3258663221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c81034c0 msgr2=0x7f48c81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:28.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 --2- 172.21.15.115:0/3258663221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c81034c0 0x7f48c81038d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0099c0 tx=0x7f48bc009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 -- 172.21.15.115:0/3258663221 shutdown_connections 2024-05-09T05:40:28.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 --2- 172.21.15.115:0/3258663221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c8108740 0x7f48c81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 --2- 172.21.15.115:0/3258663221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c81034c0 0x7f48c81038d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 --2- 172.21.15.115:0/3258663221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c8107df0 0x7f48c8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 -- 172.21.15.115:0/3258663221 >> 172.21.15.115:0/3258663221 conn(0x7f48c80fedf0 msgr2=0x7f48c8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:28.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 -- 172.21.15.115:0/3258663221 shutdown_connections 2024-05-09T05:40:28.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 -- 172.21.15.115:0/3258663221 wait complete. 2024-05-09T05:40:28.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.392+0000 7f48cf280700 1 Processor -- start 2024-05-09T05:40:28.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cf280700 1 -- start start 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cf280700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c81034c0 0x7f48c819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cf280700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c8107df0 0x7f48c819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cf280700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c8108740 0x7f48c819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cf280700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8114390 con 0x7f48c8108740 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cf280700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8114210 con 0x7f48c81034c0 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cf280700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8114510 con 0x7f48c8107df0 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd81d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c8108740 0x7f48c819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd81d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c8108740 0x7f48c819f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34490/0 (socket says 172.21.15.115:34490) 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd01c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c81034c0 0x7f48c819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd81d700 1 -- 172.21.15.115:0/1810922684 learned_addr learned my addr 172.21.15.115:0/1810922684 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cc81b700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c8107df0 0x7f48c819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd81d700 1 -- 172.21.15.115:0/1810922684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c8107df0 msgr2=0x7f48c819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd81d700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c8107df0 0x7f48c819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd81d700 1 -- 172.21.15.115:0/1810922684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c81034c0 msgr2=0x7f48c819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd81d700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c81034c0 0x7f48c819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd81d700 1 -- 172.21.15.115:0/1810922684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c819fc30 con 0x7f48c8108740 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.393+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c81034c0 0x7f48c819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.394+0000 7f48cd81d700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c8108740 0x7f48c819f590 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f48c000b970 tx=0x7f48c000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.394+0000 7f48ba7fc700 1 -- 172.21.15.115:0/1810922684 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c000d860 con 0x7f48c8108740 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.394+0000 7f48ba7fc700 1 -- 172.21.15.115:0/1810922684 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c000e430 con 0x7f48c8108740 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.394+0000 7f48ba7fc700 1 -- 172.21.15.115:0/1810922684 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0017660 con 0x7f48c8108740 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.394+0000 7f48cf280700 1 -- 172.21.15.115:0/1810922684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c819ff20 con 0x7f48c8108740 2024-05-09T05:40:28.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.394+0000 7f48cf280700 1 -- 172.21.15.115:0/1810922684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c81a7840 con 0x7f48c8108740 2024-05-09T05:40:28.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.397+0000 7f48ba7fc700 1 -- 172.21.15.115:0/1810922684 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f48c00203f0 con 0x7f48c8108740 2024-05-09T05:40:28.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.398+0000 7f48ba7fc700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48b40b29f0 0x7f48b40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.398+0000 7f48ddc96b80 1 Processor -- start 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.398+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48b40b29f0 0x7f48b40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.398+0000 7f48ddc96b80 1 -- start start 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.398+0000 7f48ba7fc700 1 -- 172.21.15.115:0/1810922684 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f48c00e2550 con 0x7f48c8108740 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84beb0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beaa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e84bdcd0a0 con 0x55e84beaa400 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e84bd729a0 con 0x55e84bda7d00 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e84bdc9060 con 0x55e84bd2d4a0 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd81d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beaa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd81d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beaa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34492/0 (socket says 172.21.15.115:34492) 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd81d700 1 -- 172.21.15.115:0/3210433310 learned_addr learned my addr 172.21.15.115:0/3210433310 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:28.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48b40b29f0 0x7f48b40b4f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48bc000c00 tx=0x7f48bc005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd01c700 1 --2- 172.21.15.115:0/3210433310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84beb0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cc81b700 1 --2- 172.21.15.115:0/3210433310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd81d700 1 -- 172.21.15.115:0/3210433310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 msgr2=0x55e84beb0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd81d700 1 --2- 172.21.15.115:0/3210433310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd81d700 1 -- 172.21.15.115:0/3210433310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 msgr2=0x55e84beb0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd81d700 1 --2- 172.21.15.115:0/3210433310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84beb0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd81d700 1 -- 172.21.15.115:0/3210433310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e84bd63500 con 0x55e84beaa400 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.399+0000 7f48cd81d700 1 --2- 172.21.15.115:0/3210433310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beaa810 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f48c00a5510 tx=0x7f48c00a9020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3709d836f27d60a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.400+0000 7f48b37fe700 1 -- 172.21.15.115:0/3210433310 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c00ad840 con 0x55e84beaa400 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.400+0000 7f48b37fe700 1 -- 172.21.15.115:0/3210433310 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c00b22f0 con 0x55e84beaa400 2024-05-09T05:40:28.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.400+0000 7f48b37fe700 1 -- 172.21.15.115:0/3210433310 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c00a96f0 con 0x55e84beaa400 2024-05-09T05:40:28.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 -- 172.21.15.115:0/3210433310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 msgr2=0x55e84beaa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:28.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/3210433310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beaa810 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f48c00a5510 tx=0x7f48c00a9020 comp rx=0 tx=0).stop 2024-05-09T05:40:28.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 -- 172.21.15.115:0/3210433310 shutdown_connections 2024-05-09T05:40:28.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/3210433310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84beb0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/3210433310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beaa810 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/3210433310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 -- 172.21.15.115:0/3210433310 >> 172.21.15.115:0/3210433310 conn(0x55e84bd30f70 msgr2=0x55e84bea5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 -- 172.21.15.115:0/3210433310 shutdown_connections 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 -- 172.21.15.115:0/3210433310 wait complete. 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 -- start start 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.401+0000 7f48ddc96b80 1 Processor -- start 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84beb4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beb4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e84beab160 con 0x55e84beaa400 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e84bd67b20 con 0x55e84bda7d00 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e84beb7020 con 0x55e84bd2d4a0 2024-05-09T05:40:28.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48ddc96b80 10 client.0 _ll_register_callbacks cb 0x55e84bc3e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd81d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beb4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd81d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beb4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34504/0 (socket says 172.21.15.115:34504) 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd81d700 1 -- 172.21.15.115:0/642054094 learned_addr learned my addr 172.21.15.115:0/642054094 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cc81b700 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84beb4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd01c700 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd81d700 1 -- 172.21.15.115:0/642054094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 msgr2=0x55e84beb3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd81d700 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd81d700 1 -- 172.21.15.115:0/642054094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 msgr2=0x55e84beb4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd81d700 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84beb4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd81d700 1 -- 172.21.15.115:0/642054094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e84bc48490 con 0x55e84beaa400 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48cd81d700 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beb4a50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f48c001f000 tx=0x7f48c00a5350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2db5906721dc17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48b0ff9700 1 -- 172.21.15.115:0/642054094 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c00e2150 con 0x55e84beaa400 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48b0ff9700 1 -- 172.21.15.115:0/642054094 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f48c00e22f0 con 0x55e84beaa400 2024-05-09T05:40:28.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.402+0000 7f48ddc96b80 1 -- 172.21.15.115:0/642054094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48c00b22f0 con 0x55e84beaa400 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48b0ff9700 1 -- 172.21.15.115:0/642054094 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f48c00a96f0 con 0x55e84beaa400 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48ddc96b80 10 client.16221 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48ddc96b80 1 -- 172.21.15.115:0/642054094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e84bec19b0 con 0x55e84beaa400 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48b0ff9700 1 -- 172.21.15.115:0/642054094 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c00b22f0 con 0x55e84beaa400 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48b0ff9700 1 -- 172.21.15.115:0/642054094 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f48c001f030 con 0x55e84beaa400 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48ddc96b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48ddc96b80 10 client.16221 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48ddc96b80 20 client.16221 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e84bec3e80 0x55e84bec62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48ddc96b80 4 client.16221 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48ddc96b80 1 -- 172.21.15.115:0/642054094 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e84bddaa80 con 0x55e84bec3e80 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.403+0000 7f48cd01c700 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e84bec3e80 0x55e84bec62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.404+0000 7f48cd01c700 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e84bec3e80 0x55e84bec62a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8be5de721148fdff server_cookie=4815cf79ff7ab7ce in_seq=0 out_seq=0 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.404+0000 7f48b0ff9700 10 client.16221 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:40:28.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48b0ff9700 1 -- 172.21.15.115:0/642054094 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e84bddaa80 con 0x55e84bec3e80 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48b0ff9700 10 client.16221 handle_command_reply: tid=0 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 client.16221 shutdown 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 -- 172.21.15.115:0/642054094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 msgr2=0x55e84beb4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beb4a50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f48c001f000 tx=0x7f48c00a5350 comp rx=0 tx=0).stop 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 -- 172.21.15.115:0/642054094 shutdown_connections 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e84bec3e80 0x55e84bec62a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48b0ff9700 0 client.16221 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84beaa400 0x55e84beb4a50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/642054094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84beb4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 -- 172.21.15.115:0/642054094 >> 172.21.15.115:0/642054094 conn(0x55e84bd30f70 msgr2=0x55e84be9eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 -- 172.21.15.115:0/642054094 shutdown_connections 2024-05-09T05:40:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 1 -- 172.21.15.115:0/642054094 wait complete. 2024-05-09T05:40:28.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.405+0000 7f48ddc96b80 20 client.16221 trim_cache size 0 max 16384 2024-05-09T05:40:28.410 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:40:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48ddc96b80 1 Processor -- start 2024-05-09T05:40:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48ddc96b80 1 -- start start 2024-05-09T05:40:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bd30f70 0x55e84be9fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84bec01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48c81b03e0 con 0x55e84bd30f70 2024-05-09T05:40:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f48c81b1f20 con 0x55e84bda7d00 2024-05-09T05:40:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48c81b0ab0 con 0x55e84bd2d4a0 2024-05-09T05:40:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48cd01c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bd30f70 0x55e84be9fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48cd01c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bd30f70 0x55e84be9fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34514/0 (socket says 172.21.15.115:34514) 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48cd01c700 1 -- 172.21.15.115:0/1686813065 learned_addr learned my addr 172.21.15.115:0/1686813065 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48cc81b700 1 --2- 172.21.15.115:0/1686813065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.420+0000 7f48cd81d700 1 --2- 172.21.15.115:0/1686813065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84bec01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.421+0000 7f48cd01c700 1 -- 172.21.15.115:0/1686813065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 msgr2=0x55e84beb6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.421+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1686813065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.421+0000 7f48cd01c700 1 -- 172.21.15.115:0/1686813065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 msgr2=0x55e84bec01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.421+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1686813065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84bec01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.421+0000 7f48cd01c700 1 -- 172.21.15.115:0/1686813065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e84bea6aa0 con 0x55e84bd30f70 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.421+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1686813065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bd30f70 0x55e84be9fa20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0099c0 tx=0x7f48bc022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa68a152eccfedfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.421+0000 7f48b2ffd700 1 -- 172.21.15.115:0/1686813065 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc03a070 con 0x55e84bd30f70 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.421+0000 7f48b2ffd700 1 -- 172.21.15.115:0/1686813065 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48bc0056a0 con 0x55e84bd30f70 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.421+0000 7f48b2ffd700 1 -- 172.21.15.115:0/1686813065 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc031800 con 0x55e84bd30f70 2024-05-09T05:40:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1686813065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bd30f70 msgr2=0x55e84be9fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1686813065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bd30f70 0x55e84be9fa20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0099c0 tx=0x7f48bc022000 comp rx=0 tx=0).stop 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1686813065 shutdown_connections 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1686813065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bda7d00 0x55e84bec01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1686813065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bd30f70 0x55e84be9fa20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1686813065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84bd2d4a0 0x55e84beb6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1686813065 >> 172.21.15.115:0/1686813065 conn(0x55e84bea5520 msgr2=0x55e84beadcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1686813065 shutdown_connections 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1686813065 wait complete. 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 -- start start 2024-05-09T05:40:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.422+0000 7f48ddc96b80 1 Processor -- start 2024-05-09T05:40:28.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bd2d4a0 0x55e84bec90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bda7d00 0x55e84bec9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48ddc96b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84beaa400 0x55e84bec66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f48c81b1850 con 0x55e84bda7d00 2024-05-09T05:40:28.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f48c81b1180 con 0x55e84bd2d4a0 2024-05-09T05:40:28.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48ddc96b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48c81ae380 con 0x55e84beaa400 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48ddc96b80 10 client.0 _ll_register_callbacks cb 0x55e84bc3e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48cc81b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bda7d00 0x55e84bec9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48cc81b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bda7d00 0x55e84bec9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34524/0 (socket says 172.21.15.115:34524) 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48cc81b700 1 -- 172.21.15.115:0/1945756138 learned_addr learned my addr 172.21.15.115:0/1945756138 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bd2d4a0 0x55e84bec90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.423+0000 7f48cd81d700 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84beaa400 0x55e84bec66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48cc81b700 1 -- 172.21.15.115:0/1945756138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84beaa400 msgr2=0x55e84bec66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48cc81b700 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84beaa400 0x55e84bec66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48cc81b700 1 -- 172.21.15.115:0/1945756138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bd2d4a0 msgr2=0x55e84bec90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48cc81b700 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bd2d4a0 0x55e84bec90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48cc81b700 1 -- 172.21.15.115:0/1945756138 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e84bec6d10 con 0x55e84bda7d00 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bd2d4a0 0x55e84bec90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48cc81b700 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bda7d00 0x55e84bec9610 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f48c4005950 tx=0x7f48c4010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c1f2b60743f7b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48b0ff9700 1 -- 172.21.15.115:0/1945756138 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c401f020 con 0x55e84bda7d00 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48b0ff9700 1 -- 172.21.15.115:0/1945756138 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f48c400fc70 con 0x55e84bda7d00 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1945756138 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48bc0056a0 con 0x55e84bda7d00 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48b0ff9700 1 -- 172.21.15.115:0/1945756138 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f48c400fe70 con 0x55e84bda7d00 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48ddc96b80 10 client.16227 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1945756138 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e84bf2dd10 con 0x55e84bda7d00 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.424+0000 7f48b0ff9700 1 -- 172.21.15.115:0/1945756138 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48bc0056a0 con 0x55e84bda7d00 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48b0ff9700 1 -- 172.21.15.115:0/1945756138 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f48c4003c30 con 0x55e84bda7d00 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48ddc96b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48ddc96b80 10 client.16227 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48ddc96b80 20 client.16227 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e84bf2e440 0x55e84bf30860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48ddc96b80 4 client.16227 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1945756138 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e84bdd4b20 con 0x55e84bf2e440 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e84bf2e440 0x55e84bf30860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48b0ff9700 10 client.16227 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.425+0000 7f48cd01c700 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e84bf2e440 0x55e84bf30860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5aa51171aba5c488 server_cookie=9fffc3ee213731f in_seq=0 out_seq=0 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48b0ff9700 1 -- 172.21.15.115:0/1945756138 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e84bdd4b20 con 0x55e84bf2e440 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48b0ff9700 10 client.16227 handle_command_reply: tid=0 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 client.16227 shutdown 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1945756138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bda7d00 msgr2=0x55e84bec9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bda7d00 0x55e84bec9610 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f48c4005950 tx=0x7f48c4010d00 comp rx=0 tx=0).stop 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1945756138 shutdown_connections 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e84bd2d4a0 0x55e84bec90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55e84bf2e440 0x55e84bf30860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e84beaa400 0x55e84bec66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 --2- 172.21.15.115:0/1945756138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e84bda7d00 0x55e84bec9610 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1945756138 >> 172.21.15.115:0/1945756138 conn(0x55e84bea5520 msgr2=0x55e84beaf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48b0ff9700 0 client.16227 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:40:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1945756138 shutdown_connections 2024-05-09T05:40:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 1 -- 172.21.15.115:0/1945756138 wait complete. 2024-05-09T05:40:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.426+0000 7f48ddc96b80 20 client.16227 trim_cache size 0 max 16384 2024-05-09T05:40:28.424 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 -- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48b40b29f0 msgr2=0x7f48b40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48b40b29f0 0x7f48b40b4f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48bc000c00 tx=0x7f48bc005fb0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 -- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c8108740 msgr2=0x7f48c819f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c8108740 0x7f48c819f590 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f48c000b970 tx=0x7f48c000bc80 comp rx=0 tx=0).stop 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 -- 172.21.15.115:0/1810922684 shutdown_connections 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f48b40b29f0 0x7f48b40b4f00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48c81034c0 0x7f48c819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48c8108740 0x7f48c819f590 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 --2- 172.21.15.115:0/1810922684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48c8107df0 0x7f48c819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 -- 172.21.15.115:0/1810922684 >> 172.21.15.115:0/1810922684 conn(0x7f48c80fedf0 msgr2=0x7f48c8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 -- 172.21.15.115:0/1810922684 shutdown_connections 2024-05-09T05:40:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:28.427+0000 7f48cf280700 1 -- 172.21.15.115:0/1810922684 wait complete. 2024-05-09T05:40:29.232 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:40:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:29.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:29 smithi115 ceph-mon[30349]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-09T05:40:29.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:29 smithi161 ceph-mon[33805]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-09T05:40:29.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:29 smithi195 ceph-mon[33844]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-09T05:40:30.057 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:31.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.364+0000 7f0b25146700 1 -- 172.21.15.115:0/2932439346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b201034c0 msgr2=0x7f0b201038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:31.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.364+0000 7f0b25146700 1 --2- 172.21.15.115:0/2932439346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b201034c0 0x7f0b201038d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0b100099c0 tx=0x7f0b10009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.364+0000 7f0b25146700 1 -- 172.21.15.115:0/2932439346 shutdown_connections 2024-05-09T05:40:31.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.364+0000 7f0b25146700 1 --2- 172.21.15.115:0/2932439346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b20108740 0x7f0b201100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.364+0000 7f0b25146700 1 --2- 172.21.15.115:0/2932439346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b201034c0 0x7f0b201038d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.364+0000 7f0b25146700 1 --2- 172.21.15.115:0/2932439346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b20107df0 0x7f0b20108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.364+0000 7f0b25146700 1 -- 172.21.15.115:0/2932439346 >> 172.21.15.115:0/2932439346 conn(0x7f0b200fedc0 msgr2=0x7f0b201011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:31.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.364+0000 7f0b25146700 1 -- 172.21.15.115:0/2932439346 shutdown_connections 2024-05-09T05:40:31.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.364+0000 7f0b25146700 1 -- 172.21.15.115:0/2932439346 wait complete. 2024-05-09T05:40:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.365+0000 7f0b25146700 1 Processor -- start 2024-05-09T05:40:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.365+0000 7f0b25146700 1 -- start start 2024-05-09T05:40:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b25146700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b201034c0 0x7f0b2019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b25146700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b20107df0 0x7f0b2019c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b25146700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b20108740 0x7f0b2019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b25146700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20113220 con 0x7f0b20107df0 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b25146700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201130a0 con 0x7f0b20108740 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b25146700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201133a0 con 0x7f0b201034c0 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b20107df0 0x7f0b2019c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b20107df0 0x7f0b2019c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34552/0 (socket says 172.21.15.115:34552) 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1e59c700 1 -- 172.21.15.115:0/3259112870 learned_addr learned my addr 172.21.15.115:0/3259112870 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1ed9d700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b201034c0 0x7f0b2019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1f59e700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b20108740 0x7f0b2019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1e59c700 1 -- 172.21.15.115:0/3259112870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b201034c0 msgr2=0x7f0b2019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1e59c700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b201034c0 0x7f0b2019bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1e59c700 1 -- 172.21.15.115:0/3259112870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b20108740 msgr2=0x7f0b2019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1e59c700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b20108740 0x7f0b2019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1e59c700 1 -- 172.21.15.115:0/3259112870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b201a6e50 con 0x7f0b20107df0 2024-05-09T05:40:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b1e59c700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b20107df0 0x7f0b2019c100 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c9e0 tx=0x7f0b1400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b07fff700 1 -- 172.21.15.115:0/3259112870 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14007aa0 con 0x7f0b20107df0 2024-05-09T05:40:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b07fff700 1 -- 172.21.15.115:0/3259112870 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b14007c40 con 0x7f0b20107df0 2024-05-09T05:40:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b201a7140 con 0x7f0b20107df0 2024-05-09T05:40:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b07fff700 1 -- 172.21.15.115:0/3259112870 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14018660 con 0x7f0b20107df0 2024-05-09T05:40:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.366+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a7680 con 0x7f0b20107df0 2024-05-09T05:40:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.367+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b200512c0 con 0x7f0b20107df0 2024-05-09T05:40:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.369+0000 7f0b07fff700 1 -- 172.21.15.115:0/3259112870 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0b14018800 con 0x7f0b20107df0 2024-05-09T05:40:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.370+0000 7f0b07fff700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0b080b29f0 0x7f0b080b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.370+0000 7f0b1ed9d700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0b080b29f0 0x7f0b080b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.370+0000 7f0b07fff700 1 -- 172.21.15.115:0/3259112870 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0b140e2b20 con 0x7f0b20107df0 2024-05-09T05:40:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.370+0000 7f0b1ed9d700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0b080b29f0 0x7f0b080b4f00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b10005940 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:31.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.373+0000 7f0b07fff700 1 -- 172.21.15.115:0/3259112870 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b14054df0 con 0x7f0b20107df0 2024-05-09T05:40:31.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:31 smithi115 ceph-mon[30349]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-09T05:40:31.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:31 smithi161 ceph-mon[33805]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-09T05:40:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:31 smithi195 ceph-mon[33844]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-09T05:40:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.630+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b2002e5f0 con 0x7f0b20107df0 2024-05-09T05:40:31.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.631+0000 7f0b07fff700 1 -- 172.21.15.115:0/3259112870 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11953 (secure 0 0 0) 0x7f0b14014030 con 0x7f0b20107df0 2024-05-09T05:40:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:40:31.624 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:38:10.901453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:40:31.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0b080b29f0 msgr2=0x7f0b080b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0b080b29f0 0x7f0b080b4f00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b10005940 comp rx=0 tx=0).stop 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b20107df0 msgr2=0x7f0b2019c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b20107df0 0x7f0b2019c100 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c9e0 tx=0x7f0b1400cda0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 shutdown_connections 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0b080b29f0 0x7f0b080b4f00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b20108740 0x7f0b2019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b201034c0 0x7f0b2019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 --2- 172.21.15.115:0/3259112870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b20107df0 0x7f0b2019c100 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.633+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 >> 172.21.15.115:0/3259112870 conn(0x7f0b200fedc0 msgr2=0x7f0b201011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.634+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 shutdown_connections 2024-05-09T05:40:31.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:31.634+0000 7f0b25146700 1 -- 172.21.15.115:0/3259112870 wait complete. 2024-05-09T05:40:31.628 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 30 2024-05-09T05:40:32.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:32 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3259112870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:32.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:32 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3259112870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:32.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:32 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3259112870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:32.767 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 9333f06f --release --await' 2024-05-09T05:40:32.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id 9333f06f --release --await 2024-05-09T05:40:33.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:33 smithi115 ceph-mon[30349]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-09T05:40:33.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:33 smithi161 ceph-mon[33805]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-09T05:40:33.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:33 smithi195 ceph-mon[33844]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-09T05:40:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.778+0000 7fe65c022700 1 -- 172.21.15.115:0/3152560477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6541029c0 msgr2=0x7fe654102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.778+0000 7fe65c022700 1 --2- 172.21.15.115:0/3152560477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6541029c0 0x7fe654102dd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe650009990 tx=0x7fe650009ca0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.778+0000 7fe65c022700 1 -- 172.21.15.115:0/3152560477 shutdown_connections 2024-05-09T05:40:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.778+0000 7fe65c022700 1 --2- 172.21.15.115:0/3152560477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6541089f0 0x7fe654108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.778+0000 7fe65c022700 1 --2- 172.21.15.115:0/3152560477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe654103310 0x7fe65410f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.778+0000 7fe65c022700 1 --2- 172.21.15.115:0/3152560477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6541029c0 0x7fe654102dd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.778+0000 7fe65c022700 1 -- 172.21.15.115:0/3152560477 >> 172.21.15.115:0/3152560477 conn(0x7fe6540fe520 msgr2=0x7fe654100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.778+0000 7fe65c022700 1 -- 172.21.15.115:0/3152560477 shutdown_connections 2024-05-09T05:40:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.778+0000 7fe65c022700 1 -- 172.21.15.115:0/3152560477 wait complete. 2024-05-09T05:40:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65c022700 1 Processor -- start 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65c022700 1 -- start start 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65c022700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6541029c0 0x7fe65419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65c022700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe654103310 0x7fe65419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65c022700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6541089f0 0x7fe654198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65c022700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541121a0 con 0x7fe654103310 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65c022700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe654112020 con 0x7fe6541089f0 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65c022700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe654112320 con 0x7fe6541029c0 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe6595bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe654103310 0x7fe65419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe6595bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe654103310 0x7fe65419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54502/0 (socket says 172.21.15.115:54502) 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe6595bd700 1 -- 172.21.15.115:0/3546041167 learned_addr learned my addr 172.21.15.115:0/3546041167 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe659dbe700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6541029c0 0x7fe65419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65a5bf700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6541089f0 0x7fe654198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe6595bd700 1 -- 172.21.15.115:0/3546041167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6541029c0 msgr2=0x7fe65419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe6595bd700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6541029c0 0x7fe65419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe6595bd700 1 -- 172.21.15.115:0/3546041167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6541089f0 msgr2=0x7fe654198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe6595bd700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6541089f0 0x7fe654198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe6595bd700 1 -- 172.21.15.115:0/3546041167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe654199030 con 0x7fe654103310 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.779+0000 7fe65a5bf700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6541089f0 0x7fe654198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.780+0000 7fe6595bd700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe654103310 0x7fe65419eca0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe6500059d0 tx=0x7fe65000bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.780+0000 7fe646ffd700 1 -- 172.21.15.115:0/3546041167 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650027070 con 0x7fe654103310 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.780+0000 7fe65c022700 1 -- 172.21.15.115:0/3546041167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6541992c0 con 0x7fe654103310 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.780+0000 7fe646ffd700 1 -- 172.21.15.115:0/3546041167 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe650022470 con 0x7fe654103310 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.780+0000 7fe646ffd700 1 -- 172.21.15.115:0/3546041167 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650017650 con 0x7fe654103310 2024-05-09T05:40:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.780+0000 7fe65c022700 1 -- 172.21.15.115:0/3546041167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6541997c0 con 0x7fe654103310 2024-05-09T05:40:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.782+0000 7fe646ffd700 1 -- 172.21.15.115:0/3546041167 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fe650009d10 con 0x7fe654103310 2024-05-09T05:40:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.783+0000 7fe646ffd700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe6400b3b00 0x7fe6400b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe659dbe700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe6400b3b00 0x7fe6400b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe646ffd700 1 -- 172.21.15.115:0/3546041167 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fe6500e35d0 con 0x7fe654103310 2024-05-09T05:40:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe66aa38b80 1 Processor -- start 2024-05-09T05:40:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe66aa38b80 1 -- start start 2024-05-09T05:40:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe659dbe700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe6400b3b00 0x7fe6400b6010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe654199ac0 tx=0x7fe648009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea712d6d00 0x55ea713d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 0x55ea713d6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea713d6e60 0x55ea713e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ea712fc0a0 con 0x55ea713d6e60 2024-05-09T05:40:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ea712a19a0 con 0x55ea712d6d00 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ea712f8060 con 0x55ea7125c4a0 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe659dbe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea712d6d00 0x55ea713d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.784+0000 7fe65a5bf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea713d6e60 0x55ea713e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe659dbe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea712d6d00 0x55ea713d63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49492/0 (socket says 172.21.15.115:49492) 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe65a5bf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea713d6e60 0x55ea713e9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54518/0 (socket says 172.21.15.115:54518) 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe659dbe700 1 -- 172.21.15.115:0/1330458131 learned_addr learned my addr 172.21.15.115:0/1330458131 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe6595bd700 1 --2- 172.21.15.115:0/1330458131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 0x55ea713d6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe65a5bf700 1 -- 172.21.15.115:0/1330458131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 msgr2=0x55ea713d6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe65a5bf700 1 --2- 172.21.15.115:0/1330458131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 0x55ea713d6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe65a5bf700 1 -- 172.21.15.115:0/1330458131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea712d6d00 msgr2=0x55ea713d63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe65a5bf700 1 --2- 172.21.15.115:0/1330458131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea712d6d00 0x55ea713d63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe65a5bf700 1 -- 172.21.15.115:0/1330458131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea71292500 con 0x55ea713d6e60 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe659dbe700 1 --2- 172.21.15.115:0/1330458131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea712d6d00 0x55ea713d63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe65a5bf700 1 --2- 172.21.15.115:0/1330458131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea713d6e60 0x55ea713e9a90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009d90 tx=0x7fe64c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c4f036f1dbe9a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe63ffff700 1 -- 172.21.15.115:0/1330458131 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c014610 con 0x55ea713d6e60 2024-05-09T05:40:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe63ffff700 1 -- 172.21.15.115:0/1330458131 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe64c019070 con 0x55ea713d6e60 2024-05-09T05:40:33.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.785+0000 7fe63ffff700 1 -- 172.21.15.115:0/1330458131 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0137f0 con 0x55ea713d6e60 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.786+0000 7fe66aa38b80 1 -- 172.21.15.115:0/1330458131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea713d6e60 msgr2=0x55ea713e9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.786+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/1330458131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea713d6e60 0x55ea713e9a90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009d90 tx=0x7fe64c0077e0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.786+0000 7fe66aa38b80 1 -- 172.21.15.115:0/1330458131 shutdown_connections 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.786+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/1330458131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea712d6d00 0x55ea713d63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.786+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/1330458131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea713d6e60 0x55ea713e9a90 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.786+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/1330458131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 0x55ea713d6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.786+0000 7fe66aa38b80 1 -- 172.21.15.115:0/1330458131 >> 172.21.15.115:0/1330458131 conn(0x55ea7125ff70 msgr2=0x55ea713d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 -- 172.21.15.115:0/1330458131 shutdown_connections 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 -- 172.21.15.115:0/1330458131 wait complete. 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 -- start start 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 Processor -- start 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 0x55ea713e74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea713e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ea713ea180 con 0x55ea7125c4a0 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ea71296b20 con 0x55ea713d6e60 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.787+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ea713ea300 con 0x55ea712d6d00 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe66aa38b80 10 client.0 _ll_register_callbacks cb 0x55ea7116d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe659dbe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe659dbe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713e6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54520/0 (socket says 172.21.15.115:54520) 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe659dbe700 1 -- 172.21.15.115:0/494020198 learned_addr learned my addr 172.21.15.115:0/494020198 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe6595bd700 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 0x55ea713e74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe65a5bf700 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea713e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe659dbe700 1 -- 172.21.15.115:0/494020198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 msgr2=0x55ea713e74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe659dbe700 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 0x55ea713e74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe659dbe700 1 -- 172.21.15.115:0/494020198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 msgr2=0x55ea713e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe659dbe700 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea713e7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe659dbe700 1 -- 172.21.15.115:0/494020198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea71177490 con 0x55ea7125c4a0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe659dbe700 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713e6f80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe64801f8e0 tx=0x7fe6480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d185fa699fe1c667 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe63d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe63d7fa700 1 -- 172.21.15.115:0/494020198 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648004830 con 0x55ea7125c4a0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe63d7fa700 1 -- 172.21.15.115:0/494020198 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fe648009040 con 0x55ea7125c4a0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe66aa38b80 1 -- 172.21.15.115:0/494020198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe64c019070 con 0x55ea7125c4a0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe63d7fa700 1 -- 172.21.15.115:0/494020198 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe6480049d0 con 0x55ea7125c4a0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe66aa38b80 10 client.16245 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.788+0000 7fe66aa38b80 1 -- 172.21.15.115:0/494020198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea713f4e00 con 0x55ea7125c4a0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.789+0000 7fe63d7fa700 1 -- 172.21.15.115:0/494020198 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe64c019070 con 0x55ea7125c4a0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.789+0000 7fe63d7fa700 1 -- 172.21.15.115:0/494020198 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe6480051b0 con 0x55ea7125c4a0 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.789+0000 7fe66aa38b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.789+0000 7fe66aa38b80 10 client.16245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.789+0000 7fe66aa38b80 20 client.16245 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:33.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.789+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55ea713f7440 0x55ea713f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.789+0000 7fe66aa38b80 4 client.16245 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.789+0000 7fe66aa38b80 1 -- 172.21.15.115:0/494020198 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea71309a80 con 0x55ea713f7440 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.789+0000 7fe6595bd700 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55ea713f7440 0x55ea713f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.790+0000 7fe63d7fa700 10 client.16245 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.790+0000 7fe6595bd700 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55ea713f7440 0x55ea713f98c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d762ebc66142284d server_cookie=177b65fa6edb03a6 in_seq=0 out_seq=0 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.790+0000 7fe63d7fa700 1 -- 172.21.15.115:0/494020198 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea71309a80 con 0x55ea713f7440 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.790+0000 7fe63d7fa700 10 client.16245 handle_command_reply: tid=0 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.790+0000 7fe66aa38b80 1 client.16245 shutdown 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 -- 172.21.15.115:0/494020198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 msgr2=0x55ea713e6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713e6f80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe64801f8e0 tx=0x7fe6480047c0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 -- 172.21.15.115:0/494020198 shutdown_connections 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55ea713f7440 0x55ea713f98c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea713e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713e6f80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe63d7fa700 0 client.16245 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:40:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/494020198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 0x55ea713e74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 -- 172.21.15.115:0/494020198 >> 172.21.15.115:0/494020198 conn(0x55ea7125ff70 msgr2=0x55ea713cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:33.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 -- 172.21.15.115:0/494020198 shutdown_connections 2024-05-09T05:40:33.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 1 -- 172.21.15.115:0/494020198 wait complete. 2024-05-09T05:40:33.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.791+0000 7fe66aa38b80 20 client.16245 trim_cache size 0 max 16384 2024-05-09T05:40:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe66aa38b80 1 Processor -- start 2024-05-09T05:40:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe66aa38b80 1 -- start start 2024-05-09T05:40:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 0x55ea713e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea713f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541ae080 con 0x55ea7125c4a0 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541afbc0 con 0x55ea713d6e60 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541ae750 con 0x55ea712d6d00 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe659dbe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe659dbe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54536/0 (socket says 172.21.15.115:54536) 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe659dbe700 1 -- 172.21.15.115:0/793352335 learned_addr learned my addr 172.21.15.115:0/793352335 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe65a5bf700 1 --2- 172.21.15.115:0/793352335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea713f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.806+0000 7fe6595bd700 1 --2- 172.21.15.115:0/793352335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 0x55ea713e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.807+0000 7fe659dbe700 1 -- 172.21.15.115:0/793352335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 msgr2=0x55ea713e5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.807+0000 7fe659dbe700 1 --2- 172.21.15.115:0/793352335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 0x55ea713e5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.807+0000 7fe659dbe700 1 -- 172.21.15.115:0/793352335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 msgr2=0x55ea713f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:33.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.807+0000 7fe659dbe700 1 --2- 172.21.15.115:0/793352335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea713f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.807+0000 7fe659dbe700 1 -- 172.21.15.115:0/793352335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea713e54f0 con 0x55ea7125c4a0 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.807+0000 7fe659dbe700 1 --2- 172.21.15.115:0/793352335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713cea20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe648013010 tx=0x7fe648009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad1ca6efb3dab06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.807+0000 7fe63f7fe700 1 -- 172.21.15.115:0/793352335 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64801d9f0 con 0x55ea7125c4a0 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.807+0000 7fe63f7fe700 1 -- 172.21.15.115:0/793352335 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe648004040 con 0x55ea7125c4a0 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.807+0000 7fe63f7fe700 1 -- 172.21.15.115:0/793352335 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648003ae0 con 0x55ea7125c4a0 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 -- 172.21.15.115:0/793352335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 msgr2=0x55ea713cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/793352335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713cea20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe648013010 tx=0x7fe648009040 comp rx=0 tx=0).stop 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 -- 172.21.15.115:0/793352335 shutdown_connections 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/793352335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea713f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/793352335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea7125c4a0 0x55ea713cea20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/793352335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea712d6d00 0x55ea713e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 -- 172.21.15.115:0/793352335 >> 172.21.15.115:0/793352335 conn(0x55ea7125ff70 msgr2=0x55ea713da590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 -- 172.21.15.115:0/793352335 shutdown_connections 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 -- 172.21.15.115:0/793352335 wait complete. 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 -- start start 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.808+0000 7fe66aa38b80 1 Processor -- start 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 0x55ea713ea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea712d6d00 0x55ea713eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe66aa38b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea7145fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541af4f0 con 0x55ea712d6d00 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541aee20 con 0x55ea713d6e60 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe66aa38b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541ac020 con 0x55ea7125c4a0 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe66aa38b80 10 client.0 _ll_register_callbacks cb 0x55ea7116d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe6595bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea712d6d00 0x55ea713eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe6595bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea712d6d00 0x55ea713eadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54552/0 (socket says 172.21.15.115:54552) 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe659dbe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 0x55ea713ea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe6595bd700 1 -- 172.21.15.115:0/2460594749 learned_addr learned my addr 172.21.15.115:0/2460594749 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe65a5bf700 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea7145fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe6595bd700 1 -- 172.21.15.115:0/2460594749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 msgr2=0x55ea713ea870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe6595bd700 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 0x55ea713ea870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe6595bd700 1 -- 172.21.15.115:0/2460594749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 msgr2=0x55ea7145fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe6595bd700 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea7145fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe6595bd700 1 -- 172.21.15.115:0/2460594749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea71460160 con 0x55ea712d6d00 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe6595bd700 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea712d6d00 0x55ea713eadb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe6500e0a40 tx=0x7fe65000b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2853b8dcffcd338c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.809+0000 7fe63d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe63d7fa700 1 -- 172.21.15.115:0/2460594749 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65006e070 con 0x55ea712d6d00 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe63d7fa700 1 -- 172.21.15.115:0/2460594749 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fe6500a6cd0 con 0x55ea712d6d00 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe66aa38b80 1 -- 172.21.15.115:0/2460594749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe648004040 con 0x55ea712d6d00 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe63d7fa700 1 -- 172.21.15.115:0/2460594749 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe6500e3420 con 0x55ea712d6d00 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe66aa38b80 10 client.16251 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:33.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe66aa38b80 1 -- 172.21.15.115:0/2460594749 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea71461380 con 0x55ea712d6d00 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe63d7fa700 1 -- 172.21.15.115:0/2460594749 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe648004040 con 0x55ea712d6d00 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe63d7fa700 1 -- 172.21.15.115:0/2460594749 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe648011ce0 con 0x55ea712d6d00 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe66aa38b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe66aa38b80 10 client.16251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe66aa38b80 20 client.16251 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55ea71461a10 0x55ea71463e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe66aa38b80 4 client.16251 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe66aa38b80 1 -- 172.21.15.115:0/2460594749 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea71303b20 con 0x55ea71461a10 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.810+0000 7fe659dbe700 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55ea71461a10 0x55ea71463e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe63d7fa700 10 client.16251 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe659dbe700 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55ea71461a10 0x55ea71463e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd5ffdd15ca56918 server_cookie=2849545fab5feeba in_seq=0 out_seq=0 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe63d7fa700 1 -- 172.21.15.115:0/2460594749 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea71303b20 con 0x55ea71461a10 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe63d7fa700 10 client.16251 handle_command_reply: tid=0 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 client.16251 shutdown 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 -- 172.21.15.115:0/2460594749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea712d6d00 msgr2=0x55ea713eadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea712d6d00 0x55ea713eadb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe6500e0a40 tx=0x7fe65000b490 comp rx=0 tx=0).stop 2024-05-09T05:40:33.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 -- 172.21.15.115:0/2460594749 shutdown_connections 2024-05-09T05:40:33.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55ea71461a10 0x55ea71463e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea713d6e60 0x55ea7145fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ea7125c4a0 0x55ea713ea870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe63d7fa700 0 client.16251 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:40:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 --2- 172.21.15.115:0/2460594749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ea712d6d00 0x55ea713eadb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 -- 172.21.15.115:0/2460594749 >> 172.21.15.115:0/2460594749 conn(0x55ea7125ff70 msgr2=0x55ea713ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 -- 172.21.15.115:0/2460594749 shutdown_connections 2024-05-09T05:40:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 1 -- 172.21.15.115:0/2460594749 wait complete. 2024-05-09T05:40:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.811+0000 7fe66aa38b80 20 client.16251 trim_cache size 0 max 16384 2024-05-09T05:40:33.813 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:40:33.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 -- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe6400b3b00 msgr2=0x7fe6400b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe6400b3b00 0x7fe6400b6010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe654199ac0 tx=0x7fe648009260 comp rx=0 tx=0).stop 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 -- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe654103310 msgr2=0x7fe65419eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe654103310 0x7fe65419eca0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe6500059d0 tx=0x7fe65000bfc0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 -- 172.21.15.115:0/3546041167 shutdown_connections 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe6400b3b00 0x7fe6400b6010 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6541089f0 0x7fe654198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6541029c0 0x7fe65419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 --2- 172.21.15.115:0/3546041167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe654103310 0x7fe65419eca0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 -- 172.21.15.115:0/3546041167 >> 172.21.15.115:0/3546041167 conn(0x7fe6540fe520 msgr2=0x7fe654100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 -- 172.21.15.115:0/3546041167 shutdown_connections 2024-05-09T05:40:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:33.812+0000 7fe65c022700 1 -- 172.21.15.115:0/3546041167 wait complete. 2024-05-09T05:40:34.057 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:34.682 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:40:35.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:35 smithi115 ceph-mon[30349]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-09T05:40:35.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:35 smithi161 ceph-mon[33805]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-09T05:40:35.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:35 smithi195 ceph-mon[33844]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-09T05:40:35.949 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:36.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.463+0000 7f96948dd700 1 -- 172.21.15.115:0/2446426328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690103520 msgr2=0x7f9690103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:36.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.463+0000 7f96948dd700 1 --2- 172.21.15.115:0/2446426328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690103520 0x7f9690103930 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f96800099c0 tx=0x7f9680009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.463+0000 7f96948dd700 1 -- 172.21.15.115:0/2446426328 shutdown_connections 2024-05-09T05:40:36.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.463+0000 7f96948dd700 1 --2- 172.21.15.115:0/2446426328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690108750 0x7f96901100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.463+0000 7f96948dd700 1 --2- 172.21.15.115:0/2446426328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690103520 0x7f9690103930 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.463+0000 7f96948dd700 1 --2- 172.21.15.115:0/2446426328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690107e00 0x7f9690108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.463+0000 7f96948dd700 1 -- 172.21.15.115:0/2446426328 >> 172.21.15.115:0/2446426328 conn(0x7f96900fef90 msgr2=0x7f96901013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:36.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.463+0000 7f96948dd700 1 -- 172.21.15.115:0/2446426328 shutdown_connections 2024-05-09T05:40:36.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.463+0000 7f96948dd700 1 -- 172.21.15.115:0/2446426328 wait complete. 2024-05-09T05:40:36.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f96948dd700 1 Processor -- start 2024-05-09T05:40:36.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f96948dd700 1 -- start start 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f96948dd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690103520 0x7f969019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f96948dd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690107e00 0x7f969019c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f96948dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690108750 0x7f969019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f96948dd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9690113230 con 0x7f9690107e00 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f96948dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f96901130b0 con 0x7f9690108750 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f96948dd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96901133b0 con 0x7f9690103520 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f968dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690107e00 0x7f969019c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f968dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690107e00 0x7f969019c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54566/0 (socket says 172.21.15.115:54566) 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f968dd9b700 1 -- 172.21.15.115:0/3185133779 learned_addr learned my addr 172.21.15.115:0/3185133779 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f968ed9d700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690108750 0x7f969019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.464+0000 7f968e59c700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690103520 0x7f969019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f968dd9b700 1 -- 172.21.15.115:0/3185133779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690103520 msgr2=0x7f969019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f968dd9b700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690103520 0x7f969019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f968dd9b700 1 -- 172.21.15.115:0/3185133779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690108750 msgr2=0x7f969019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:36.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f968dd9b700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690108750 0x7f969019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f968dd9b700 1 -- 172.21.15.115:0/3185133779 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96901a6f40 con 0x7f9690107e00 2024-05-09T05:40:36.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f968dd9b700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690107e00 0x7f969019c110 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f968400d980 tx=0x7f968400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:36.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f967f7fe700 1 -- 172.21.15.115:0/3185133779 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684017070 con 0x7f9690107e00 2024-05-09T05:40:36.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f96948dd700 1 -- 172.21.15.115:0/3185133779 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96901a7230 con 0x7f9690107e00 2024-05-09T05:40:36.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f967f7fe700 1 -- 172.21.15.115:0/3185133779 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f968400fd20 con 0x7f9690107e00 2024-05-09T05:40:36.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f967f7fe700 1 -- 172.21.15.115:0/3185133779 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968400e3b0 con 0x7f9690107e00 2024-05-09T05:40:36.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.465+0000 7f96948dd700 1 -- 172.21.15.115:0/3185133779 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96901a7770 con 0x7f9690107e00 2024-05-09T05:40:36.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.468+0000 7f967f7fe700 1 -- 172.21.15.115:0/3185133779 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f968400f890 con 0x7f9690107e00 2024-05-09T05:40:36.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.469+0000 7f967f7fe700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96780b29f0 0x7f96780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.469+0000 7f968e59c700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96780b29f0 0x7f96780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.469+0000 7f967f7fe700 1 -- 172.21.15.115:0/3185133779 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f96840e23e0 con 0x7f9690107e00 2024-05-09T05:40:36.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.469+0000 7f96a32f3b80 1 Processor -- start 2024-05-09T05:40:36.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.469+0000 7f96a32f3b80 1 -- start start 2024-05-09T05:40:36.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.469+0000 7f968e59c700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96780b29f0 0x7f96780b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9680000c00 tx=0x7f9680005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:36.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc5ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc5efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdc58f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561dfdb7b0f0 con 0x561dfdbc5260 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561dfdb198f0 con 0x561dfdc58b50 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561dfdb770b0 con 0x561dfdadb5a0 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc5ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc5ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54568/0 (socket says 172.21.15.115:54568) 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968e59c700 1 -- 172.21.15.115:0/3694887636 learned_addr learned my addr 172.21.15.115:0/3694887636 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968dd9b700 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc5efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968ed9d700 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdc58f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968e59c700 1 -- 172.21.15.115:0/3694887636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 msgr2=0x561dfdc5efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968e59c700 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc5efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968e59c700 1 -- 172.21.15.115:0/3694887636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 msgr2=0x561dfdc58f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968e59c700 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdc58f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968e59c700 1 -- 172.21.15.115:0/3694887636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dfdb11470 con 0x561dfdbc5260 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968dd9b700 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc5efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f968e59c700 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc5ea90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9680015000 tx=0x7f968001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6a7b40c9dd0349 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.470+0000 7f967cff9700 1 -- 172.21.15.115:0/3694887636 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680005690 con 0x561dfdbc5260 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.471+0000 7f967cff9700 1 -- 172.21.15.115:0/3694887636 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9680003b10 con 0x561dfdbc5260 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.471+0000 7f967cff9700 1 -- 172.21.15.115:0/3694887636 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968000fb00 con 0x561dfdbc5260 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.471+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3694887636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 msgr2=0x561dfdc5ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.471+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc5ea90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9680015000 tx=0x7f968001e000 comp rx=0 tx=0).stop 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.471+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3694887636 shutdown_connections 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.471+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdc58f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.471+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc5ea90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.471+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3694887636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc5efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3694887636 >> 172.21.15.115:0/3694887636 conn(0x561dfdb3b7a0 msgr2=0x561dfdc53a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3694887636 shutdown_connections 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3694887636 wait complete. 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 -- start start 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 Processor -- start 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdadb5a0 0x561dfdc62790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdbc5260 0x561dfdc62cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdc58b50 0x561dfdc63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561dfd9f6ca0 con 0x561dfdc58b50 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561dfdb35250 con 0x561dfdadb5a0 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.472+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561dfdc59910 con 0x561dfdbc5260 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f96a32f3b80 10 client.0 _ll_register_callbacks cb 0x561dfd9ec7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdc58b50 0x561dfdc63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdc58b50 0x561dfdc63210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54572/0 (socket says 172.21.15.115:54572) 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968ed9d700 1 -- 172.21.15.115:0/891018844 learned_addr learned my addr 172.21.15.115:0/891018844 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968dd9b700 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdbc5260 0x561dfdc62cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968e59c700 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdadb5a0 0x561dfdc62790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968ed9d700 1 -- 172.21.15.115:0/891018844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdbc5260 msgr2=0x561dfdc62cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968ed9d700 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdbc5260 0x561dfdc62cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968ed9d700 1 -- 172.21.15.115:0/891018844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdadb5a0 msgr2=0x561dfdc62790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968ed9d700 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdadb5a0 0x561dfdc62790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968ed9d700 1 -- 172.21.15.115:0/891018844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dfd9f6590 con 0x561dfdc58b50 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f968ed9d700 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdc58b50 0x561dfdc63210 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f968800ae80 tx=0x7f9688012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341e7e06ab4e7a23 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f9675ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f9675ffb700 1 -- 172.21.15.115:0/891018844 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968801a070 con 0x561dfdc58b50 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f9675ffb700 1 -- 172.21.15.115:0/891018844 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9688012bf0 con 0x561dfdc58b50 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.473+0000 7f96a32f3b80 1 -- 172.21.15.115:0/891018844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9680003b10 con 0x561dfdc58b50 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f9675ffb700 1 -- 172.21.15.115:0/891018844 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9688026050 con 0x561dfdc58b50 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f96a32f3b80 10 client.16263 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f96a32f3b80 1 -- 172.21.15.115:0/891018844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dfdc74490 con 0x561dfdc58b50 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f9675ffb700 1 -- 172.21.15.115:0/891018844 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9680003b10 con 0x561dfdc58b50 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f9675ffb700 1 -- 172.21.15.115:0/891018844 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9688016c60 con 0x561dfdc58b50 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f96a32f3b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f96a32f3b80 10 client.16263 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f96a32f3b80 20 client.16263 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:36.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561dfdc76a70 0x561dfdc78ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f96a32f3b80 4 client.16263 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.474+0000 7f96a32f3b80 1 -- 172.21.15.115:0/891018844 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dfdb88ad0 con 0x561dfdc76a70 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.475+0000 7f968e59c700 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561dfdc76a70 0x561dfdc78ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.475+0000 7f9675ffb700 10 client.16263 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.475+0000 7f968e59c700 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561dfdc76a70 0x561dfdc78ef0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8254acec00f91173 server_cookie=85421d9b5a453409 in_seq=0 out_seq=0 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f9675ffb700 1 -- 172.21.15.115:0/891018844 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561dfdb88ad0 con 0x561dfdc76a70 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f9675ffb700 10 client.16263 handle_command_reply: tid=0 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 client.16263 shutdown 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 -- 172.21.15.115:0/891018844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdc58b50 msgr2=0x561dfdc63210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdc58b50 0x561dfdc63210 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f968800ae80 tx=0x7f9688012510 comp rx=0 tx=0).stop 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 -- 172.21.15.115:0/891018844 shutdown_connections 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdadb5a0 0x561dfdc62790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561dfdc76a70 0x561dfdc78ef0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdc58b50 0x561dfdc63210 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/891018844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdbc5260 0x561dfdc62cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 -- 172.21.15.115:0/891018844 >> 172.21.15.115:0/891018844 conn(0x561dfdb3b7a0 msgr2=0x561dfdc53750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:36.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f9675ffb700 0 client.16263 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:36.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 -- 172.21.15.115:0/891018844 shutdown_connections 2024-05-09T05:40:36.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 1 -- 172.21.15.115:0/891018844 wait complete. 2024-05-09T05:40:36.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.476+0000 7f96a32f3b80 20 client.16263 trim_cache size 0 max 16384 2024-05-09T05:40:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f96a32f3b80 1 Processor -- start 2024-05-09T05:40:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f96a32f3b80 1 -- start start 2024-05-09T05:40:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc5cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdc72820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ae220 con 0x561dfdbc5260 2024-05-09T05:40:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f96901afd60 con 0x561dfdc58b50 2024-05-09T05:40:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ae8f0 con 0x561dfdadb5a0 2024-05-09T05:40:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f968dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f968dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc58580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54578/0 (socket says 172.21.15.115:54578) 2024-05-09T05:40:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f968dd9b700 1 -- 172.21.15.115:0/410717866 learned_addr learned my addr 172.21.15.115:0/410717866 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f968e59c700 1 --2- 172.21.15.115:0/410717866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc5cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.497+0000 7f968ed9d700 1 --2- 172.21.15.115:0/410717866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdc72820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.498+0000 7f968dd9b700 1 -- 172.21.15.115:0/410717866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 msgr2=0x561dfdc5cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.498+0000 7f968dd9b700 1 --2- 172.21.15.115:0/410717866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc5cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.498+0000 7f968dd9b700 1 -- 172.21.15.115:0/410717866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 msgr2=0x561dfdc72820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.498+0000 7f968dd9b700 1 --2- 172.21.15.115:0/410717866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdc72820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.498+0000 7f968dd9b700 1 -- 172.21.15.115:0/410717866 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dfdc4d410 con 0x561dfdbc5260 2024-05-09T05:40:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.498+0000 7f968dd9b700 1 --2- 172.21.15.115:0/410717866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc58580 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f96840a53b0 tx=0x7f968400fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48ca39c0862d512 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.498+0000 7f9677fff700 1 -- 172.21.15.115:0/410717866 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684020040 con 0x561dfdbc5260 2024-05-09T05:40:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.498+0000 7f9677fff700 1 -- 172.21.15.115:0/410717866 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96840a9a80 con 0x561dfdbc5260 2024-05-09T05:40:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.498+0000 7f9677fff700 1 -- 172.21.15.115:0/410717866 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96840a95e0 con 0x561dfdbc5260 2024-05-09T05:40:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.499+0000 7f96a32f3b80 1 -- 172.21.15.115:0/410717866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 msgr2=0x561dfdc58580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.499+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/410717866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc58580 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f96840a53b0 tx=0x7f968400fbd0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.499+0000 7f96a32f3b80 1 -- 172.21.15.115:0/410717866 shutdown_connections 2024-05-09T05:40:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.499+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/410717866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdc72820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.499+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/410717866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc5cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.499+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/410717866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc58580 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.499+0000 7f96a32f3b80 1 -- 172.21.15.115:0/410717866 >> 172.21.15.115:0/410717866 conn(0x561dfdb3b7a0 msgr2=0x561dfdc52e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.499+0000 7f96a32f3b80 1 -- 172.21.15.115:0/410717866 shutdown_connections 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 1 -- 172.21.15.115:0/410717866 wait complete. 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 1 -- start start 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 1 Processor -- start 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc6ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc6b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdce0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f96901af690 con 0x561dfdbc5260 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f96901aefc0 con 0x561dfdc58b50 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ac1c0 con 0x561dfdadb5a0 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.500+0000 7f96a32f3b80 10 client.0 _ll_register_callbacks cb 0x561dfd9ec7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc6b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc6b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54580/0 (socket says 172.21.15.115:54580) 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968dd9b700 1 -- 172.21.15.115:0/3537586965 learned_addr learned my addr 172.21.15.115:0/3537586965 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968e59c700 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc6ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968ed9d700 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdce0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968dd9b700 1 -- 172.21.15.115:0/3537586965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 msgr2=0x561dfdc6ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968dd9b700 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc6ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968dd9b700 1 -- 172.21.15.115:0/3537586965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 msgr2=0x561dfdce0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968dd9b700 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdce0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968dd9b700 1 -- 172.21.15.115:0/3537586965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dfdce07a0 con 0x561dfdbc5260 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f968dd9b700 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc6b370 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f96840df8a0 tx=0x7f96840a58c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afda8a2fa9e750b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f9675ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f9675ffb700 1 -- 172.21.15.115:0/3537586965 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684020030 con 0x561dfdbc5260 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f9675ffb700 1 -- 172.21.15.115:0/3537586965 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f96840a97f0 con 0x561dfdbc5260 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3537586965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96840a9a80 con 0x561dfdbc5260 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f9675ffb700 1 -- 172.21.15.115:0/3537586965 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96840a9380 con 0x561dfdbc5260 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f96a32f3b80 10 client.16269 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.501+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3537586965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dfdce1970 con 0x561dfdbc5260 2024-05-09T05:40:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.502+0000 7f9675ffb700 1 -- 172.21.15.115:0/3537586965 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96840ad730 con 0x561dfdbc5260 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.502+0000 7f9675ffb700 1 -- 172.21.15.115:0/3537586965 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f96840a4ef0 con 0x561dfdbc5260 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.502+0000 7f96a32f3b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.502+0000 7f96a32f3b80 10 client.16269 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.502+0000 7f96a32f3b80 20 client.16269 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.502+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561dfdce1f60 0x561dfdce4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.502+0000 7f96a32f3b80 4 client.16269 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9333f06f", "release": true, "await": true}] 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.502+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3537586965 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9333f06f", "release": true, "await": true}) -- 0x7f96840a9a80 con 0x561dfdce1f60 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.502+0000 7f968e59c700 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561dfdce1f60 0x561dfdce4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.503+0000 7f9675ffb700 10 client.16269 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.503+0000 7f968e59c700 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561dfdce1f60 0x561dfdce4380 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f054e508e9e3304 server_cookie=e1efe8041316d70c in_seq=0 out_seq=0 2024-05-09T05:40:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f9675ffb700 1 -- 172.21.15.115:0/3537586965 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f96840a9a80 con 0x561dfdce1f60 2024-05-09T05:40:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f9675ffb700 10 client.16269 handle_command_reply: tid=0 2024-05-09T05:40:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 client.16269 shutdown 2024-05-09T05:40:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3537586965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 msgr2=0x561dfdc6b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc6b370 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f96840df8a0 tx=0x7f96840a58c0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3537586965 shutdown_connections 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561dfdc58b50 0x561dfdce0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561dfdce1f60 0x561dfdce4380 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561dfdadb5a0 0x561dfdc6ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 --2- 172.21.15.115:0/3537586965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561dfdbc5260 0x561dfdc6b370 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3537586965 >> 172.21.15.115:0/3537586965 conn(0x561dfdb3b7a0 msgr2=0x561dfdc53860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f9675ffb700 0 client.16269 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3537586965 shutdown_connections 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 1 -- 172.21.15.115:0/3537586965 wait complete. 2024-05-09T05:40:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.507+0000 7f96a32f3b80 20 client.16269 trim_cache size 0 max 16384 2024-05-09T05:40:36.506 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:40:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 30, 2024-05-09T05:40:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:40:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 8, 2024-05-09T05:40:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:40:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "9333f06f": { 2024-05-09T05:40:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 8, 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 144.9, 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:40:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:40:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:40:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:40:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:40:36.508 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 -- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96780b29f0 msgr2=0x7f96780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96780b29f0 0x7f96780b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9680000c00 tx=0x7f9680005fb0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 -- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690107e00 msgr2=0x7f969019c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690107e00 0x7f969019c110 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f968400d980 tx=0x7f968400dc90 comp rx=0 tx=0).stop 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 -- 172.21.15.115:0/3185133779 shutdown_connections 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96780b29f0 0x7f96780b4f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9690108750 0x7f969019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690103520 0x7f969019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 --2- 172.21.15.115:0/3185133779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9690107e00 0x7f969019c110 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 -- 172.21.15.115:0/3185133779 >> 172.21.15.115:0/3185133779 conn(0x7f96900fef90 msgr2=0x7f96901096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:36.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 -- 172.21.15.115:0/3185133779 shutdown_connections 2024-05-09T05:40:36.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:36.508+0000 7f96948dd700 1 -- 172.21.15.115:0/3185133779 wait complete. 2024-05-09T05:40:36.514 INFO:tasks.workunit.client.0.smithi115.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-09T05:40:36.515 INFO:tasks.workunit.client.0.smithi115.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-05-09T05:40:36.515 INFO:tasks.workunit.client.0.smithi115.stderr:+ trap -- retry EXIT 2024-05-09T05:40:36.515 INFO:tasks.workunit.client.0.smithi115.stderr:+ rm -rf ffsb 2024-05-09T05:40:36.516 INFO:tasks.workunit.client.0.smithi115.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-05-09T05:40:36.520 INFO:tasks.workunit.client.0.smithi115.stderr:Cloning into 'ffsb'... 2024-05-09T05:40:37.346 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:37 smithi115 ceph-mon[30349]: pgmap v75: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-09T05:40:37.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:37 smithi161 ceph-mon[33805]: pgmap v75: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-09T05:40:37.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:37 smithi195 ceph-mon[33844]: pgmap v75: 129 pgs: 129 active+clean; 1.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-09T05:40:37.940 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9333f06f', total seconds elapsed: 34.5 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.266+0000 7f1db39ce640 1 -- 172.21.15.115:0/835439845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dac108b90 msgr2=0x7f1dac102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.266+0000 7f1db39ce640 1 --2- 172.21.15.115:0/835439845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dac108b90 0x7f1dac102730 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009900 tx=0x7f1da802a410 comp rx=0 tx=0).stop 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.266+0000 7f1db39ce640 1 -- 172.21.15.115:0/835439845 shutdown_connections 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.266+0000 7f1db39ce640 1 --2- 172.21.15.115:0/835439845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac1032e0 0x7f1dac10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.266+0000 7f1db39ce640 1 --2- 172.21.15.115:0/835439845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dac109030 0x7f1dac102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.266+0000 7f1db39ce640 1 --2- 172.21.15.115:0/835439845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dac108b90 0x7f1dac102730 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.266+0000 7f1db39ce640 1 -- 172.21.15.115:0/835439845 >> 172.21.15.115:0/835439845 conn(0x7f1dac0fe390 msgr2=0x7f1dac1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.266+0000 7f1db39ce640 1 -- 172.21.15.115:0/835439845 shutdown_connections 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.266+0000 7f1db39ce640 1 -- 172.21.15.115:0/835439845 wait complete. 2024-05-09T05:40:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.267+0000 7f1db39ce640 1 Processor -- start 2024-05-09T05:40:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.267+0000 7f1db39ce640 1 -- start start 2024-05-09T05:40:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.267+0000 7f1db39ce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dac1032e0 0x7f1dac1a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.267+0000 7f1db39ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac108b90 0x7f1dac1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db39ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dac109030 0x7f1dac19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db39ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac075580 con 0x7f1dac109030 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db39ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac075400 con 0x7f1dac108b90 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db39ce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac075700 con 0x7f1dac1032e0 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db176a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dac1032e0 0x7f1dac1a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db0f69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac108b90 0x7f1dac1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db176a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dac1032e0 0x7f1dac1a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:51468/0 (socket says 172.21.15.115:51468) 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db176a640 1 -- 172.21.15.115:0/1502681074 learned_addr learned my addr 172.21.15.115:0/1502681074 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dac109030 0x7f1dac19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db176a640 1 -- 172.21.15.115:0/1502681074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac108b90 msgr2=0x7f1dac1a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db176a640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac108b90 0x7f1dac1a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db176a640 1 -- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dac109030 msgr2=0x7f1dac19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db176a640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dac109030 0x7f1dac19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db176a640 1 -- 172.21.15.115:0/1502681074 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac19ada0 con 0x7f1dac1032e0 2024-05-09T05:40:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db0f69640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac108b90 0x7f1dac1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1db176a640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dac1032e0 0x7f1dac1a04f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1da802a930 tx=0x7f1da8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.268+0000 7f1da27fc640 1 -- 172.21.15.115:0/1502681074 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da803f070 con 0x7f1dac1032e0 2024-05-09T05:40:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.269+0000 7f1da27fc640 1 -- 172.21.15.115:0/1502681074 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1da803baf0 con 0x7f1dac1032e0 2024-05-09T05:40:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.269+0000 7f1db39ce640 1 -- 172.21.15.115:0/1502681074 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac19afd0 con 0x7f1dac1032e0 2024-05-09T05:40:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.269+0000 7f1da27fc640 1 -- 172.21.15.115:0/1502681074 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8035030 con 0x7f1dac1032e0 2024-05-09T05:40:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.269+0000 7f1db39ce640 1 -- 172.21.15.115:0/1502681074 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac19b4c0 con 0x7f1dac1032e0 2024-05-09T05:40:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1da27fc640 1 -- 172.21.15.115:0/1502681074 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1da8042070 con 0x7f1dac1032e0 2024-05-09T05:40:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1dba522740 1 Processor -- start 2024-05-09T05:40:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1dba522740 1 -- start start 2024-05-09T05:40:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1dba522740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c0561680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1dba522740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01952f0 0x55f5c0561bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1dba522740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c019d4d0 0x55f5c0562100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1dba522740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f5c04c67c0 con 0x55f5c01952f0 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1dba522740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f5c04620e0 con 0x55f5c01be1b0 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1dba522740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f5c04c2780 con 0x55f5c019d4d0 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1db176a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01952f0 0x55f5c0561bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1db1f6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c0561680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1db1f6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c0561680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49584/0 (socket says 172.21.15.115:49584) 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1db1f6b640 1 -- 172.21.15.115:0/2015422299 learned_addr learned my addr 172.21.15.115:0/2015422299 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.272+0000 7f1db0f69640 1 --2- 172.21.15.115:0/2015422299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c019d4d0 0x55f5c0562100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1db1f6b640 1 -- 172.21.15.115:0/2015422299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c019d4d0 msgr2=0x55f5c0562100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/2015422299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c019d4d0 0x55f5c0562100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1db1f6b640 1 -- 172.21.15.115:0/2015422299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01952f0 msgr2=0x55f5c0561bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/2015422299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01952f0 0x55f5c0561bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1db1f6b640 1 -- 172.21.15.115:0/2015422299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c048b7c0 con 0x55f5c01be1b0 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1da27fc640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d980b2950 0x7f1d980b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/2015422299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c0561680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009fd0 tx=0x7f1da40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3203563a2a40dc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1db176a640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d980b2950 0x7f1d980b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1da27fc640 1 -- 172.21.15.115:0/1502681074 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1da8102fb0 con 0x7f1dac1032e0 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1d977fe640 1 -- 172.21.15.115:0/2015422299 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4004d20 con 0x55f5c01be1b0 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1d977fe640 1 -- 172.21.15.115:0/2015422299 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1da4004050 con 0x55f5c01be1b0 2024-05-09T05:40:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.273+0000 7f1d977fe640 1 -- 172.21.15.115:0/2015422299 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4005260 con 0x55f5c01be1b0 2024-05-09T05:40:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1dba522740 1 -- 172.21.15.115:0/2015422299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 msgr2=0x55f5c0561680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1db176a640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d980b2950 0x7f1d980b4e60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55f5c055f170 tx=0x7f1da8106aa0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1dba522740 1 --2- 172.21.15.115:0/2015422299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c0561680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009fd0 tx=0x7f1da40118d0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1dba522740 1 -- 172.21.15.115:0/2015422299 shutdown_connections 2024-05-09T05:40:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1dba522740 1 --2- 172.21.15.115:0/2015422299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c019d4d0 0x55f5c0562100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1dba522740 1 --2- 172.21.15.115:0/2015422299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01952f0 0x55f5c0561bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1dba522740 1 --2- 172.21.15.115:0/2015422299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c0561680 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1dba522740 1 -- 172.21.15.115:0/2015422299 >> 172.21.15.115:0/2015422299 conn(0x55f5c01b2300 msgr2=0x55f5c055de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1dba522740 1 -- 172.21.15.115:0/2015422299 shutdown_connections 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.274+0000 7f1dba522740 1 -- 172.21.15.115:0/2015422299 wait complete. 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.275+0000 7f1dba522740 1 -- start start 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.275+0000 7f1dba522740 1 Processor -- start 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1dba522740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01952f0 0x55f5c056d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1dba522740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c019d4d0 0x55f5c056dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1dba522740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c056bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1dba522740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f5c056e7b0 con 0x55f5c019d4d0 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1dba522740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f5c044d9c0 con 0x55f5c01be1b0 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1dba522740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f5c056e930 con 0x55f5c01952f0 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1dba522740 10 client.0 _ll_register_callbacks cb 0x55f5c047a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db0f69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c019d4d0 0x55f5c056dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db0f69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c019d4d0 0x55f5c056dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54606/0 (socket says 172.21.15.115:54606) 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db0f69640 1 -- 172.21.15.115:0/2231012979 learned_addr learned my addr 172.21.15.115:0/2231012979 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01952f0 0x55f5c056d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c056bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db0f69640 1 -- 172.21.15.115:0/2231012979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01952f0 msgr2=0x55f5c056d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db0f69640 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01952f0 0x55f5c056d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db0f69640 1 -- 172.21.15.115:0/2231012979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 msgr2=0x55f5c056bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db0f69640 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c056bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db0f69640 1 -- 172.21.15.115:0/2231012979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c04741f0 con 0x55f5c019d4d0 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.276+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01952f0 0x55f5c056d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.277+0000 7f1db0f69640 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c019d4d0 0x55f5c056dc90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009f20 tx=0x7f1d9c00b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee723a69e0ece75d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.277+0000 7f1d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.277+0000 7f1d94ff9640 1 -- 172.21.15.115:0/2231012979 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c004230 con 0x55f5c019d4d0 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.277+0000 7f1dba522740 1 -- 172.21.15.115:0/2231012979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da4004050 con 0x55f5c019d4d0 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.277+0000 7f1d94ff9640 1 -- 172.21.15.115:0/2231012979 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1d9c0043d0 con 0x55f5c019d4d0 2024-05-09T05:40:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1d94ff9640 1 -- 172.21.15.115:0/2231012979 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d9c02a050 con 0x55f5c019d4d0 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1dba522740 10 client.16281 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1dba522740 1 -- 172.21.15.115:0/2231012979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5c0229790 con 0x55f5c019d4d0 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1d94ff9640 1 -- 172.21.15.115:0/2231012979 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d9c01bae0 con 0x55f5c019d4d0 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1d94ff9640 1 -- 172.21.15.115:0/2231012979 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1d9c003800 con 0x55f5c019d4d0 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1dba522740 10 client.16281 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1dba522740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1dba522740 20 client.16281 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1dba522740 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f5c017cd80 0x55f5c057e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1dba522740 4 client.16281 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1dba522740 1 -- 172.21.15.115:0/2231012979 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1da4004050 con 0x55f5c017cd80 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.278+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f5c017cd80 0x55f5c057e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.279+0000 7f1d94ff9640 10 client.16281 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.279+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f5c017cd80 0x55f5c057e080 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428544dc6cc3c66c server_cookie=e2e3545369e093db in_seq=0 out_seq=0 2024-05-09T05:40:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1d94ff9640 1 -- 172.21.15.115:0/2231012979 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f1da4004050 con 0x55f5c017cd80 2024-05-09T05:40:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1d94ff9640 10 client.16281 handle_command_reply: tid=0 2024-05-09T05:40:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 client.16281 shutdown 2024-05-09T05:40:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 -- 172.21.15.115:0/2231012979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c019d4d0 msgr2=0x55f5c056dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c019d4d0 0x55f5c056dc90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009f20 tx=0x7f1d9c00b750 comp rx=0 tx=0).stop 2024-05-09T05:40:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 -- 172.21.15.115:0/2231012979 shutdown_connections 2024-05-09T05:40:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01be1b0 0x55f5c056bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c019d4d0 0x55f5c056dc90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f5c017cd80 0x55f5c057e080 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 --2- 172.21.15.115:0/2231012979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01952f0 0x55f5c056d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1d94ff9640 0 client.16281 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 -- 172.21.15.115:0/2231012979 >> 172.21.15.115:0/2231012979 conn(0x55f5c01b2300 msgr2=0x55f5c055c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 -- 172.21.15.115:0/2231012979 shutdown_connections 2024-05-09T05:40:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 1 -- 172.21.15.115:0/2231012979 wait complete. 2024-05-09T05:40:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.280+0000 7f1dba522740 20 client.16281 trim_cache size 0 max 16384 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.302+0000 7f1dba522740 1 Processor -- start 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.302+0000 7f1dba522740 1 -- start start 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1dba522740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01b2300 0x55f5c05838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1dba522740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0583e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1dba522740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01be1b0 0x55f5c055c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1dba522740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1aef80 con 0x55f5c01be1b0 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1dba522740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1b0ac0 con 0x55f5c01952f0 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1dba522740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1af650 con 0x55f5c01b2300 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db0f69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0583e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db1f6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01be1b0 0x55f5c055c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db0f69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0583e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49598/0 (socket says 172.21.15.115:49598) 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db0f69640 1 -- 172.21.15.115:0/3236123337 learned_addr learned my addr 172.21.15.115:0/3236123337 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01be1b0 0x55f5c055c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54614/0 (socket says 172.21.15.115:54614) 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01b2300 0x55f5c05838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db1f6b640 1 -- 172.21.15.115:0/3236123337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01b2300 msgr2=0x55f5c05838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01b2300 0x55f5c05838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db1f6b640 1 -- 172.21.15.115:0/3236123337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 msgr2=0x55f5c0583e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0583e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db1f6b640 1 -- 172.21.15.115:0/3236123337 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c02a05f0 con 0x55f5c01be1b0 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.303+0000 7f1db0f69640 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0583e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.304+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01be1b0 0x55f5c055c700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1da40049d0 tx=0x7f1da40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb3efc95dd3d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.304+0000 7f1d96ffd640 1 -- 172.21.15.115:0/3236123337 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4004490 con 0x55f5c01be1b0 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.304+0000 7f1d96ffd640 1 -- 172.21.15.115:0/3236123337 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1da4004630 con 0x55f5c01be1b0 2024-05-09T05:40:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.304+0000 7f1d96ffd640 1 -- 172.21.15.115:0/3236123337 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da400f4e0 con 0x55f5c01be1b0 2024-05-09T05:40:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 -- 172.21.15.115:0/3236123337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01be1b0 msgr2=0x55f5c055c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01be1b0 0x55f5c055c700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1da40049d0 tx=0x7f1da40118d0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 -- 172.21.15.115:0/3236123337 shutdown_connections 2024-05-09T05:40:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01be1b0 0x55f5c055c700 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0583e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 --2- 172.21.15.115:0/3236123337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01b2300 0x55f5c05838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 -- 172.21.15.115:0/3236123337 >> 172.21.15.115:0/3236123337 conn(0x55f5c0567530 msgr2=0x55f5c056a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 -- 172.21.15.115:0/3236123337 shutdown_connections 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 -- 172.21.15.115:0/3236123337 wait complete. 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.305+0000 7f1dba522740 1 -- start start 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1dba522740 1 Processor -- start 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1dba522740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0573f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1dba522740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01b2300 0x55f5c0571280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1dba522740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01be1b0 0x55f5c05717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1dba522740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1b03f0 con 0x55f5c01b2300 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1dba522740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1afd20 con 0x55f5c01952f0 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1dba522740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1acf60 con 0x55f5c01be1b0 2024-05-09T05:40:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1dba522740 10 client.0 _ll_register_callbacks cb 0x7f1d9c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1db1f6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0573f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1db1f6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0573f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49604/0 (socket says 172.21.15.115:49604) 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1db1f6b640 1 -- 172.21.15.115:0/345096061 learned_addr learned my addr 172.21.15.115:0/345096061 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01be1b0 0x55f5c05717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.306+0000 7f1db0f69640 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01b2300 0x55f5c0571280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1db1f6b640 1 -- 172.21.15.115:0/345096061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01be1b0 msgr2=0x55f5c05717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01be1b0 0x55f5c05717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1db1f6b640 1 -- 172.21.15.115:0/345096061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01b2300 msgr2=0x55f5c0571280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01b2300 0x55f5c0571280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1db1f6b640 1 -- 172.21.15.115:0/345096061 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c048aed0 con 0x55f5c01952f0 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1db1f6b640 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0573f10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1da402a040 tx=0x7f1da40096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc6108e7bb9e6b8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1d94ff9640 1 -- 172.21.15.115:0/345096061 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da40047e0 con 0x55f5c01952f0 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1d94ff9640 1 -- 172.21.15.115:0/345096061 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1da4004290 con 0x55f5c01952f0 2024-05-09T05:40:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1dba522740 1 -- 172.21.15.115:0/345096061 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da4004630 con 0x55f5c01952f0 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1d94ff9640 1 -- 172.21.15.115:0/345096061 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1da402e8c0 con 0x55f5c01952f0 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1dba522740 10 client.25784 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1dba522740 1 -- 172.21.15.115:0/345096061 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5c05e5fa0 con 0x55f5c01952f0 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.307+0000 7f1d94ff9640 1 -- 172.21.15.115:0/345096061 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1da4004630 con 0x55f5c01952f0 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1d94ff9640 1 -- 172.21.15.115:0/345096061 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1da4025040 con 0x55f5c01952f0 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1dba522740 10 client.25784 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1dba522740 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1dba522740 20 client.25784 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1dba522740 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f5c01521c0 0x55f5c05e8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1dba522740 4 client.25784 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1dba522740 1 -- 172.21.15.115:0/345096061 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5c04d43d0 con 0x55f5c01521c0 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1db0f69640 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f5c01521c0 0x55f5c05e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1d94ff9640 10 client.25784 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.308+0000 7f1db0f69640 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f5c01521c0 0x55f5c05e8420 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0bb87378c8da92 server_cookie=361187ab997eb4d5 in_seq=0 out_seq=0 2024-05-09T05:40:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.410+0000 7f1d94ff9640 1 -- 172.21.15.115:0/345096061 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f5c04d43d0 con 0x55f5c01521c0 2024-05-09T05:40:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1d94ff9640 10 client.25784 handle_command_reply: tid=0 2024-05-09T05:40:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 client.25784 shutdown 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 -- 172.21.15.115:0/345096061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 msgr2=0x55f5c0573f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0573f10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1da402a040 tx=0x7f1da40096c0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 -- 172.21.15.115:0/345096061 shutdown_connections 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f5c01521c0 0x55f5c05e8420 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f5c01be1b0 0x55f5c05717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5c01b2300 0x55f5c0571280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 --2- 172.21.15.115:0/345096061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c01952f0 0x55f5c0573f10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 -- 172.21.15.115:0/345096061 >> 172.21.15.115:0/345096061 conn(0x55f5c0567530 msgr2=0x55f5c055c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1d94ff9640 0 client.25784 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.411+0000 7f1dba522740 1 -- 172.21.15.115:0/345096061 shutdown_connections 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.412+0000 7f1dba522740 1 -- 172.21.15.115:0/345096061 wait complete. 2024-05-09T05:40:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.412+0000 7f1dba522740 20 client.25784 trim_cache size 0 max 16384 2024-05-09T05:40:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:40:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:40:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.412+0000 7f1db39ce640 1 -- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d980b2950 msgr2=0x7f1d980b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.412+0000 7f1db39ce640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d980b2950 0x7f1d980b4e60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55f5c055f170 tx=0x7f1da8106aa0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.412+0000 7f1db39ce640 1 -- 172.21.15.115:0/1502681074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dac1032e0 msgr2=0x7f1dac1a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.412+0000 7f1db39ce640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dac1032e0 0x7f1dac1a04f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1da802a930 tx=0x7f1da8003b90 comp rx=0 tx=0).stop 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.413+0000 7f1db39ce640 1 -- 172.21.15.115:0/1502681074 shutdown_connections 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.413+0000 7f1db39ce640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d980b2950 0x7f1d980b4e60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.413+0000 7f1db39ce640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dac109030 0x7f1dac19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.413+0000 7f1db39ce640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dac108b90 0x7f1dac1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.413+0000 7f1db39ce640 1 --2- 172.21.15.115:0/1502681074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dac1032e0 0x7f1dac1a04f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.413+0000 7f1db39ce640 1 -- 172.21.15.115:0/1502681074 >> 172.21.15.115:0/1502681074 conn(0x7f1dac0fe390 msgr2=0x7f1dac0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.413+0000 7f1db39ce640 1 -- 172.21.15.115:0/1502681074 shutdown_connections 2024-05-09T05:40:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:38.413+0000 7f1db39ce640 1 -- 172.21.15.115:0/1502681074 wait complete. 2024-05-09T05:40:38.530 INFO:tasks.workunit.client.0.smithi115.stderr:+ trap - EXIT 2024-05-09T05:40:38.531 INFO:tasks.workunit.client.0.smithi115.stderr:+ cd ffsb 2024-05-09T05:40:38.532 INFO:tasks.workunit.client.0.smithi115.stderr:+ ./configure 2024-05-09T05:40:38.725 INFO:tasks.workunit.client.0.smithi115.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-09T05:40:39.341 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:39 smithi115 ceph-mon[30349]: pgmap v76: 129 pgs: 129 active+clean; 2.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 130 KiB/s wr, 28 op/s 2024-05-09T05:40:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.361+0000 7f1dc1afc700 1 -- 172.21.15.115:0/2758782761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc1045d0 msgr2=0x7f1dbc1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.361+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/2758782761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc1045d0 0x7f1dbc1049e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1da40099c0 tx=0x7f1da4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.362+0000 7f1dc1afc700 1 -- 172.21.15.115:0/2758782761 shutdown_connections 2024-05-09T05:40:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.362+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/2758782761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dbc109850 0x7f1dbc1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.362+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/2758782761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc1045d0 0x7f1dbc1049e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.362+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/2758782761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc108f00 0x7f1dbc109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.362+0000 7f1dc1afc700 1 -- 172.21.15.115:0/2758782761 >> 172.21.15.115:0/2758782761 conn(0x7f1dbc0ffea0 msgr2=0x7f1dbc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.362+0000 7f1dc1afc700 1 -- 172.21.15.115:0/2758782761 shutdown_connections 2024-05-09T05:40:39.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.362+0000 7f1dc1afc700 1 -- 172.21.15.115:0/2758782761 wait complete. 2024-05-09T05:40:39.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.362+0000 7f1dc1afc700 1 Processor -- start 2024-05-09T05:40:39.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dc1afc700 1 -- start start 2024-05-09T05:40:39.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dc1afc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dbc1045d0 0x7f1dbc076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dc1afc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc108f00 0x7f1dbc077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dc1afc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc109850 0x7f1dbc077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dc1afc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc114330 con 0x7f1dbc108f00 2024-05-09T05:40:39.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dc1afc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1141b0 con 0x7f1dbc1045d0 2024-05-09T05:40:39.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dc1afc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1144b0 con 0x7f1dbc109850 2024-05-09T05:40:39.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc108f00 0x7f1dbc077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc108f00 0x7f1dbc077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54640/0 (socket says 172.21.15.115:54640) 2024-05-09T05:40:39.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbaffd700 1 -- 172.21.15.115:0/1288457950 learned_addr learned my addr 172.21.15.115:0/1288457950 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:39.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc109850 0x7f1dbc077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dbc1045d0 0x7f1dbc076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbaffd700 1 -- 172.21.15.115:0/1288457950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc109850 msgr2=0x7f1dbc077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:39.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc109850 0x7f1dbc077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbaffd700 1 -- 172.21.15.115:0/1288457950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dbc1045d0 msgr2=0x7f1dbc076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:39.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dbc1045d0 0x7f1dbc076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbaffd700 1 -- 172.21.15.115:0/1288457950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc1a9140 con 0x7f1dbc108f00 2024-05-09T05:40:39.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.363+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dbc1045d0 0x7f1dbc076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:39.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.364+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc108f00 0x7f1dbc077110 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e970 tx=0x7f1dac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:39.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.364+0000 7f1db8ff9700 1 -- 172.21.15.115:0/1288457950 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00cbd0 con 0x7f1dbc108f00 2024-05-09T05:40:39.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.364+0000 7f1dc1afc700 1 -- 172.21.15.115:0/1288457950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc1a9430 con 0x7f1dbc108f00 2024-05-09T05:40:39.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.364+0000 7f1db8ff9700 1 -- 172.21.15.115:0/1288457950 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1dac00cd70 con 0x7f1dbc108f00 2024-05-09T05:40:39.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.364+0000 7f1db8ff9700 1 -- 172.21.15.115:0/1288457950 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac005570 con 0x7f1dbc108f00 2024-05-09T05:40:39.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.364+0000 7f1dc1afc700 1 -- 172.21.15.115:0/1288457950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc1a99c0 con 0x7f1dbc108f00 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.366+0000 7f1db8ff9700 1 -- 172.21.15.115:0/1288457950 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1dac021440 con 0x7f1dbc108f00 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.367+0000 7f1db8ff9700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1da80b29f0 0x7f1da80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.367+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1da80b29f0 0x7f1da80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.367+0000 7f1db8ff9700 1 -- 172.21.15.115:0/1288457950 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1dac0e2ba0 con 0x7f1dbc108f00 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.367+0000 7f1dd0512b80 1 Processor -- start 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dd0512b80 1 -- start start 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1da80b29f0 0x7f1da80b4f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1da4003760 tx=0x7f1da401a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68e9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 0x5645d68e3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5645d6805ff0 con 0x5645d6850160 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5645d67a47f0 con 0x5645d67664a0 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5645d6801fb0 con 0x5645d68e36a0 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68e94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54652/0 (socket says 172.21.15.115:54652) 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 -- 172.21.15.115:0/3673031634 learned_addr learned my addr 172.21.15.115:0/3673031634 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/3673031634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68e9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/3673031634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 0x5645d68e3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 -- 172.21.15.115:0/3673031634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 msgr2=0x5645d68e3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/3673031634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 0x5645d68e3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 -- 172.21.15.115:0/3673031634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 msgr2=0x5645d68e9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/3673031634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68e9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 -- 172.21.15.115:0/3673031634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d679c370 con 0x5645d6850160 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.368+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/3673031634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68e94f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1da40099c0 tx=0x7f1da401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cac7f124f1ce26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.369+0000 7f1db1ffb700 1 -- 172.21.15.115:0/3673031634 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da400b790 con 0x5645d6850160 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.369+0000 7f1db1ffb700 1 -- 172.21.15.115:0/3673031634 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1da400bdc0 con 0x5645d6850160 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.369+0000 7f1db1ffb700 1 -- 172.21.15.115:0/3673031634 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da402fcf0 con 0x5645d6850160 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3673031634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 msgr2=0x5645d68e94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3673031634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68e94f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1da40099c0 tx=0x7f1da401e000 comp rx=0 tx=0).stop 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3673031634 shutdown_connections 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3673031634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 0x5645d68e3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3673031634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68e94f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3673031634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68e9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3673031634 >> 172.21.15.115:0/3673031634 conn(0x5645d67c66a0 msgr2=0x5645d68de400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3673031634 shutdown_connections 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3673031634 wait complete. 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 -- start start 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.370+0000 7f1dd0512b80 1 Processor -- start 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68ed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 0x5645d68ed7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d68edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5645d6681ba0 con 0x5645d68e36a0 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5645d67c0150 con 0x5645d67664a0 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5645d68f01d0 con 0x5645d6850160 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dd0512b80 10 client.0 _ll_register_callbacks cb 0x5645d66776c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d68edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d68edce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54654/0 (socket says 172.21.15.115:54654) 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbbfff700 1 -- 172.21.15.115:0/3960636704 learned_addr learned my addr 172.21.15.115:0/3960636704 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:39.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 0x5645d68ed7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbbfff700 1 -- 172.21.15.115:0/3960636704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 msgr2=0x5645d68ed7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 0x5645d68ed7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbbfff700 1 -- 172.21.15.115:0/3960636704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 msgr2=0x5645d68ed260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68ed260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbbfff700 1 -- 172.21.15.115:0/3960636704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d6681490 con 0x5645d68e36a0 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d68edce0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009420 tx=0x7f1db4004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66b6d8c141f065d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.371+0000 7f1d9affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1d9affd700 1 -- 172.21.15.115:0/3960636704 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4019020 con 0x5645d68e36a0 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1d9affd700 1 -- 172.21.15.115:0/3960636704 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1db4007710 con 0x5645d68e36a0 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3960636704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da400bdc0 con 0x5645d68e36a0 2024-05-09T05:40:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1d9affd700 1 -- 172.21.15.115:0/3960636704 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1db4024050 con 0x5645d68e36a0 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dd0512b80 10 client.16299 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3960636704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645d68fac60 con 0x5645d68e36a0 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1d9affd700 1 -- 172.21.15.115:0/3960636704 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1da400bdc0 con 0x5645d68e36a0 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1d9affd700 1 -- 172.21.15.115:0/3960636704 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1db4015820 con 0x5645d68e36a0 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dd0512b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dd0512b80 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dd0512b80 20 client.16299 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645d68fd340 0x5645d68ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dd0512b80 4 client.16299 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3960636704 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645d68139d0 con 0x5645d68fd340 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.372+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645d68fd340 0x5645d68ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.373+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645d68fd340 0x5645d68ff760 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d38f10e55d0dbca server_cookie=712e2309d345fb3b in_seq=0 out_seq=0 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.373+0000 7f1d9affd700 10 client.16299 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1d9affd700 1 -- 172.21.15.115:0/3960636704 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645d68139d0 con 0x5645d68fd340 2024-05-09T05:40:39.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1d9affd700 10 client.16299 handle_command_reply: tid=0 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 client.16299 shutdown 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3960636704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 msgr2=0x5645d68edce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d68edce0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009420 tx=0x7f1db4004320 comp rx=0 tx=0).stop 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3960636704 shutdown_connections 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68ed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645d68fd340 0x5645d68ff760 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d68edce0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1d9affd700 0 client.16299 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/3960636704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 0x5645d68ed7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3960636704 >> 172.21.15.115:0/3960636704 conn(0x5645d67c66a0 msgr2=0x5645d68de1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3960636704 shutdown_connections 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 1 -- 172.21.15.115:0/3960636704 wait complete. 2024-05-09T05:40:39.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.374+0000 7f1dd0512b80 20 client.16299 trim_cache size 0 max 16384 2024-05-09T05:40:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.388+0000 7f1dd0512b80 1 Processor -- start 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.388+0000 7f1dd0512b80 1 -- start start 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 0x5645d68f9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b2580 con 0x5645d6850160 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b40c0 con 0x5645d67664a0 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b2c50 con 0x5645d68e36a0 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68f8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54664/0 (socket says 172.21.15.115:54664) 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbaffd700 1 -- 172.21.15.115:0/1098944495 learned_addr learned my addr 172.21.15.115:0/1098944495 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 0x5645d68f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbaffd700 1 -- 172.21.15.115:0/1098944495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 msgr2=0x5645d68f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:39.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 0x5645d68f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbaffd700 1 -- 172.21.15.115:0/1098944495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 msgr2=0x5645d68e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:39.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbaffd700 1 -- 172.21.15.115:0/1098944495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d68e4610 con 0x5645d6850160 2024-05-09T05:40:39.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 0x5645d68f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:40:39.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:39.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68f8fd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0e0010 tx=0x7f1dac0a9560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9071d95803f4be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:39.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.389+0000 7f1db17fa700 1 -- 172.21.15.115:0/1098944495 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0a9650 con 0x5645d6850160 2024-05-09T05:40:39.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1db17fa700 1 -- 172.21.15.115:0/1098944495 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1dac0c3070 con 0x5645d6850160 2024-05-09T05:40:39.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1db17fa700 1 -- 172.21.15.115:0/1098944495 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0adef0 con 0x5645d6850160 2024-05-09T05:40:39.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1dd0512b80 1 -- 172.21.15.115:0/1098944495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 msgr2=0x5645d68f8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:39.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68f8fd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0e0010 tx=0x7f1dac0a9560 comp rx=0 tx=0).stop 2024-05-09T05:40:39.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1dd0512b80 1 -- 172.21.15.115:0/1098944495 shutdown_connections 2024-05-09T05:40:39.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d68e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d68e36a0 0x5645d68f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/1098944495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d6850160 0x5645d68f8fd0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1dd0512b80 1 -- 172.21.15.115:0/1098944495 >> 172.21.15.115:0/1098944495 conn(0x5645d67c66a0 msgr2=0x5645d68e2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1dd0512b80 1 -- 172.21.15.115:0/1098944495 shutdown_connections 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.390+0000 7f1dd0512b80 1 -- 172.21.15.115:0/1098944495 wait complete. 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dd0512b80 1 -- start start 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dd0512b80 1 Processor -- start 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d6904740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 0x5645d6904c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dd0512b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d69051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b39f0 con 0x5645d68e36a0 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b3320 con 0x5645d67664a0 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dd0512b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b0520 con 0x5645d6850160 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dd0512b80 10 client.0 _ll_register_callbacks cb 0x5645d66776c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:39.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.391+0000 7f1dbbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d69051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d69051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54666/0 (socket says 172.21.15.115:54666) 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbbfff700 1 -- 172.21.15.115:0/2898399927 learned_addr learned my addr 172.21.15.115:0/2898399927 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 0x5645d6904c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d6904740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbbfff700 1 -- 172.21.15.115:0/2898399927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 msgr2=0x5645d6904c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 0x5645d6904c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbbfff700 1 -- 172.21.15.115:0/2898399927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 msgr2=0x5645d6904740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d6904740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbbfff700 1 -- 172.21.15.115:0/2898399927 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d6966af0 con 0x5645d68e36a0 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbaffd700 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 0x5645d6904c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dbbfff700 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d69051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009790 tx=0x7f1db401d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50aa1ee45f2a156 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1d9affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1d9affd700 1 -- 172.21.15.115:0/2898399927 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4019070 con 0x5645d68e36a0 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1d9affd700 1 -- 172.21.15.115:0/2898399927 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1db4004990 con 0x5645d68e36a0 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dd0512b80 1 -- 172.21.15.115:0/2898399927 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dac0c3070 con 0x5645d68e36a0 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1d9affd700 1 -- 172.21.15.115:0/2898399927 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1db4004b90 con 0x5645d68e36a0 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dd0512b80 10 client.16305 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.392+0000 7f1dd0512b80 1 -- 172.21.15.115:0/2898399927 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645d6967b30 con 0x5645d68e36a0 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.393+0000 7f1d9affd700 1 -- 172.21.15.115:0/2898399927 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1dac0c3070 con 0x5645d68e36a0 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.393+0000 7f1d9affd700 1 -- 172.21.15.115:0/2898399927 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1db40077a0 con 0x5645d68e36a0 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.393+0000 7f1dd0512b80 10 client.16305 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.393+0000 7f1dd0512b80 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.393+0000 7f1dd0512b80 20 client.16305 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.393+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645d69681e0 0x5645d696a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.393+0000 7f1dd0512b80 4 client.16305 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:40:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.393+0000 7f1dd0512b80 1 -- 172.21.15.115:0/2898399927 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645d680da70 con 0x5645d69681e0 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.393+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645d69681e0 0x5645d696a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dbb7fe700 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645d69681e0 0x5645d696a600 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3791cf02ad151c2b server_cookie=831c9eaa0b792d2c in_seq=0 out_seq=0 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1d9affd700 10 client.16305 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1d9affd700 1 -- 172.21.15.115:0/2898399927 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645d680da70 con 0x5645d69681e0 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1d9affd700 10 client.16305 handle_command_reply: tid=0 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 client.16305 shutdown 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 -- 172.21.15.115:0/2898399927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 msgr2=0x5645d69051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d69051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009790 tx=0x7f1db401d5d0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 -- 172.21.15.115:0/2898399927 shutdown_connections 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d67664a0 0x5645d6904740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5645d69681e0 0x5645d696a600 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5645d68e36a0 0x5645d69051c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 --2- 172.21.15.115:0/2898399927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5645d6850160 0x5645d6904c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1d9affd700 0 client.16305 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 -- 172.21.15.115:0/2898399927 >> 172.21.15.115:0/2898399927 conn(0x5645d67c66a0 msgr2=0x5645d68e6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 -- 172.21.15.115:0/2898399927 shutdown_connections 2024-05-09T05:40:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 1 -- 172.21.15.115:0/2898399927 wait complete. 2024-05-09T05:40:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.394+0000 7f1dd0512b80 20 client.16305 trim_cache size 0 max 16384 2024-05-09T05:40:39.394 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:40:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 -- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1da80b29f0 msgr2=0x7f1da80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1da80b29f0 0x7f1da80b4f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1da4003760 tx=0x7f1da401a040 comp rx=0 tx=0).stop 2024-05-09T05:40:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 -- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc108f00 msgr2=0x7f1dbc077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc108f00 0x7f1dbc077110 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e970 tx=0x7f1dac00ec80 comp rx=0 tx=0).stop 2024-05-09T05:40:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 -- 172.21.15.115:0/1288457950 shutdown_connections 2024-05-09T05:40:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1da80b29f0 0x7f1da80b4f00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dbc1045d0 0x7f1dbc076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc109850 0x7f1dbc077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 --2- 172.21.15.115:0/1288457950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc108f00 0x7f1dbc077110 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 -- 172.21.15.115:0/1288457950 >> 172.21.15.115:0/1288457950 conn(0x7f1dbc0ffea0 msgr2=0x7f1dbc102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 -- 172.21.15.115:0/1288457950 shutdown_connections 2024-05-09T05:40:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:39.395+0000 7f1dc1afc700 1 -- 172.21.15.115:0/1288457950 wait complete. 2024-05-09T05:40:39.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:39 smithi161 ceph-mon[33805]: pgmap v76: 129 pgs: 129 active+clean; 2.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 130 KiB/s wr, 28 op/s 2024-05-09T05:40:39.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:39 smithi195 ceph-mon[33844]: pgmap v76: 129 pgs: 129 active+clean; 2.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 130 KiB/s wr, 28 op/s 2024-05-09T05:40:39.736 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether build environment is sane... yes 2024-05-09T05:40:39.750 INFO:tasks.workunit.client.0.smithi115.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-09T05:40:39.751 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gawk... gawk 2024-05-09T05:40:39.773 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether make sets $(MAKE)... yes 2024-05-09T05:40:39.816 INFO:tasks.workunit.client.0.smithi115.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-09T05:40:39.816 INFO:tasks.workunit.client.0.smithi115.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-09T05:40:39.837 INFO:tasks.workunit.client.0.smithi115.stdout:checking for style of include used by make... GNU 2024-05-09T05:40:39.843 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gcc... gcc 2024-05-09T05:40:39.928 INFO:tasks.workunit.client.0.smithi115.stdout:checking for C compiler default output file name... a.out 2024-05-09T05:40:39.931 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether the C compiler works... yes 2024-05-09T05:40:39.935 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether we are cross compiling... no 2024-05-09T05:40:39.991 INFO:tasks.workunit.client.0.smithi115.stdout:checking for suffix of executables... 2024-05-09T05:40:40.035 INFO:tasks.workunit.client.0.smithi115.stdout:checking for suffix of object files... o 2024-05-09T05:40:40.080 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether we are using the GNU C compiler... yes 2024-05-09T05:40:40.130 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether gcc accepts -g... yes 2024-05-09T05:40:40.288 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-09T05:40:40.469 INFO:tasks.workunit.client.0.smithi115.stdout:checking dependency style of gcc... gcc3 2024-05-09T05:40:40.586 INFO:tasks.workunit.client.0.smithi115.stdout:checking how to run the C preprocessor... gcc -E 2024-05-09T05:40:40.648 INFO:tasks.workunit.client.0.smithi115.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-09T05:40:40.650 INFO:tasks.workunit.client.0.smithi115.stdout:checking for egrep... /usr/bin/grep -E 2024-05-09T05:40:40.663 INFO:tasks.workunit.client.0.smithi115.stdout:checking for AIX... no 2024-05-09T05:40:40.690 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gcc... (cached) gcc 2024-05-09T05:40:40.702 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-09T05:40:40.702 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-09T05:40:40.703 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-09T05:40:40.704 INFO:tasks.workunit.client.0.smithi115.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-09T05:40:40.807 INFO:tasks.workunit.client.0.smithi115.stdout:checking for main in -lm... yes 2024-05-09T05:40:40.895 INFO:tasks.workunit.client.0.smithi115.stdout:checking for main in -lpthread... yes 2024-05-09T05:40:41.106 INFO:tasks.workunit.client.0.smithi115.stdout:checking for ANSI C header files... yes 2024-05-09T05:40:41.185 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-09T05:40:41.246 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/types.h... yes 2024-05-09T05:40:41.299 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:40:41.300 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:40:41.318 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/stat.h... yes 2024-05-09T05:40:41.385 INFO:tasks.workunit.client.0.smithi115.stdout:checking for stdlib.h... yes 2024-05-09T05:40:41.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:41 smithi115 ceph-mon[30349]: pgmap v77: 129 pgs: 129 active+clean; 2.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 23 op/s 2024-05-09T05:40:41.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:41 smithi161 ceph-mon[33805]: pgmap v77: 129 pgs: 129 active+clean; 2.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 23 op/s 2024-05-09T05:40:41.454 INFO:tasks.workunit.client.0.smithi115.stdout:checking for string.h... yes 2024-05-09T05:40:41.529 INFO:tasks.workunit.client.0.smithi115.stdout:checking for memory.h... yes 2024-05-09T05:40:41.568 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:41.603 INFO:tasks.workunit.client.0.smithi115.stdout:checking for strings.h... yes 2024-05-09T05:40:41.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:41 smithi195 ceph-mon[33844]: pgmap v77: 129 pgs: 129 active+clean; 2.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 23 op/s 2024-05-09T05:40:41.671 INFO:tasks.workunit.client.0.smithi115.stdout:checking for inttypes.h... yes 2024-05-09T05:40:41.742 INFO:tasks.workunit.client.0.smithi115.stdout:checking for stdint.h... yes 2024-05-09T05:40:41.812 INFO:tasks.workunit.client.0.smithi115.stdout:checking for unistd.h... yes 2024-05-09T05:40:41.903 INFO:tasks.workunit.client.0.smithi115.stdout:checking pthread.h usability... yes 2024-05-09T05:40:41.937 INFO:tasks.workunit.client.0.smithi115.stdout:checking pthread.h presence... yes 2024-05-09T05:40:41.937 INFO:tasks.workunit.client.0.smithi115.stdout:checking for pthread.h... yes 2024-05-09T05:40:42.035 INFO:tasks.workunit.client.0.smithi115.stdout:checking fcntl.h usability... yes 2024-05-09T05:40:42.069 INFO:tasks.workunit.client.0.smithi115.stdout:checking fcntl.h presence... yes 2024-05-09T05:40:42.069 INFO:tasks.workunit.client.0.smithi115.stdout:checking for fcntl.h... yes 2024-05-09T05:40:42.147 INFO:tasks.workunit.client.0.smithi115.stdout:checking limits.h usability... yes 2024-05-09T05:40:42.177 INFO:tasks.workunit.client.0.smithi115.stdout:checking limits.h presence... yes 2024-05-09T05:40:42.178 INFO:tasks.workunit.client.0.smithi115.stdout:checking for limits.h... yes 2024-05-09T05:40:42.201 INFO:tasks.workunit.client.0.smithi115.stdout:checking for stdint.h... (cached) yes 2024-05-09T05:40:42.270 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/time.h usability... yes 2024-05-09T05:40:42.298 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/time.h presence... yes 2024-05-09T05:40:42.299 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/time.h... yes 2024-05-09T05:40:42.322 INFO:tasks.workunit.client.0.smithi115.stdout:checking for unistd.h... (cached) yes 2024-05-09T05:40:42.384 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/vfs.h usability... yes 2024-05-09T05:40:42.413 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/vfs.h presence... yes 2024-05-09T05:40:42.413 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/vfs.h... yes 2024-05-09T05:40:42.501 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/limits.h usability... no 2024-05-09T05:40:42.534 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/limits.h presence... no 2024-05-09T05:40:42.534 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/limits.h... no 2024-05-09T05:40:42.581 INFO:tasks.workunit.client.0.smithi115.stdout:checking for an ANSI C-conforming const... yes 2024-05-09T05:40:42.627 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-09T05:40:42.629 INFO:tasks.workunit.client.0.smithi115.stdout:checking for function prototypes... yes 2024-05-09T05:40:42.720 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether setvbuf arguments are reversed... no 2024-05-09T05:40:42.808 INFO:tasks.workunit.client.0.smithi115.stdout:checking for system... yes 2024-05-09T05:40:42.901 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gettimeofday... yes 2024-05-09T05:40:43.012 INFO:tasks.workunit.client.0.smithi115.stdout:checking for mkdir... yes 2024-05-09T05:40:43.111 INFO:tasks.workunit.client.0.smithi115.stdout:checking for strerror... yes 2024-05-09T05:40:43.201 INFO:tasks.workunit.client.0.smithi115.stdout:checking for open64... yes 2024-05-09T05:40:43.344 INFO:tasks.workunit.client.0.smithi115.stdout:checking for stat64... yes 2024-05-09T05:40:43.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:43 smithi115 ceph-mon[30349]: pgmap v78: 129 pgs: 129 active+clean; 2.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 23 op/s 2024-05-09T05:40:43.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:43 smithi161 ceph-mon[33805]: pgmap v78: 129 pgs: 129 active+clean; 2.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 23 op/s 2024-05-09T05:40:43.445 INFO:tasks.workunit.client.0.smithi115.stdout:checking for fseeko64... yes 2024-05-09T05:40:43.549 INFO:tasks.workunit.client.0.smithi115.stdout:checking for lrand48_r... yes 2024-05-09T05:40:43.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:43 smithi195 ceph-mon[33844]: pgmap v78: 129 pgs: 129 active+clean; 2.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 23 op/s 2024-05-09T05:40:43.648 INFO:tasks.workunit.client.0.smithi115.stdout:checking for srand48_r... yes 2024-05-09T05:40:43.681 INFO:tasks.workunit.client.0.smithi115.stdout:configure: creating ./config.status 2024-05-09T05:40:43.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.774+0000 7f4303f4d700 1 -- 172.21.15.115:0/2279674401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc108f10 msgr2=0x7f42fc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:43.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.774+0000 7f4303f4d700 1 --2- 172.21.15.115:0/2279674401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc108f10 0x7f42fc109320 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0099c0 tx=0x7f42ec009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.774+0000 7f4303f4d700 1 -- 172.21.15.115:0/2279674401 shutdown_connections 2024-05-09T05:40:43.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.774+0000 7f4303f4d700 1 --2- 172.21.15.115:0/2279674401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42fc1045e0 0x7f42fc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.774+0000 7f4303f4d700 1 --2- 172.21.15.115:0/2279674401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42fc109860 0x7f42fc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.774+0000 7f4303f4d700 1 --2- 172.21.15.115:0/2279674401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc108f10 0x7f42fc109320 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.774+0000 7f4303f4d700 1 -- 172.21.15.115:0/2279674401 >> 172.21.15.115:0/2279674401 conn(0x7f42fc0ffe70 msgr2=0x7f42fc1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:43.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.774+0000 7f4303f4d700 1 -- 172.21.15.115:0/2279674401 shutdown_connections 2024-05-09T05:40:43.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.774+0000 7f4303f4d700 1 -- 172.21.15.115:0/2279674401 wait complete. 2024-05-09T05:40:43.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.775+0000 7f4303f4d700 1 Processor -- start 2024-05-09T05:40:43.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.775+0000 7f4303f4d700 1 -- start start 2024-05-09T05:40:43.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4303f4d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc1045e0 0x7f42fc075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4303f4d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42fc108f10 0x7f42fc076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4303f4d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42fc109860 0x7f42fc076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4303f4d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc114320 con 0x7f42fc1045e0 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4303f4d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1141a0 con 0x7f42fc109860 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4303f4d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1144a0 con 0x7f42fc108f10 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4301ce9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc1045e0 0x7f42fc075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4301ce9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc1045e0 0x7f42fc075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53322/0 (socket says 172.21.15.115:53322) 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4301ce9700 1 -- 172.21.15.115:0/1479401201 learned_addr learned my addr 172.21.15.115:0/1479401201 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f43014e8700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42fc108f10 0x7f42fc076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f43024ea700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42fc109860 0x7f42fc076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4301ce9700 1 -- 172.21.15.115:0/1479401201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42fc108f10 msgr2=0x7f42fc076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4301ce9700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42fc108f10 0x7f42fc076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4301ce9700 1 -- 172.21.15.115:0/1479401201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42fc109860 msgr2=0x7f42fc076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4301ce9700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42fc109860 0x7f42fc076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4301ce9700 1 -- 172.21.15.115:0/1479401201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc1a90f0 con 0x7f42fc1045e0 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f43024ea700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42fc109860 0x7f42fc076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:43.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f4301ce9700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc1045e0 0x7f42fc075ac0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f42f800c920 tx=0x7f42f800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:43.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.776+0000 7f42f2ffd700 1 -- 172.21.15.115:0/1479401201 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8007b00 con 0x7f42fc1045e0 2024-05-09T05:40:43.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.777+0000 7f42f2ffd700 1 -- 172.21.15.115:0/1479401201 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42f8004cc0 con 0x7f42fc1045e0 2024-05-09T05:40:43.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.777+0000 7f42f2ffd700 1 -- 172.21.15.115:0/1479401201 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8005440 con 0x7f42fc1045e0 2024-05-09T05:40:43.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.777+0000 7f4303f4d700 1 -- 172.21.15.115:0/1479401201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42fc1a93e0 con 0x7f42fc1045e0 2024-05-09T05:40:43.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.777+0000 7f4303f4d700 1 -- 172.21.15.115:0/1479401201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42fc1a98f0 con 0x7f42fc1045e0 2024-05-09T05:40:43.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.780+0000 7f42f2ffd700 1 -- 172.21.15.115:0/1479401201 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f42f800f450 con 0x7f42fc1045e0 2024-05-09T05:40:43.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.781+0000 7f4312963b80 1 Processor -- start 2024-05-09T05:40:43.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.781+0000 7f4312963b80 1 -- start start 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f42f2ffd700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42e80b29f0 0x7f42e80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4312963b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b508570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4312963b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b508ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4312963b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b502b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4312963b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f87b425070 con 0x55f87b46f1e0 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4312963b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f87b3c3870 con 0x55f87b385520 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4312963b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f87b421030 con 0x55f87b502720 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4301ce9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b508570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4301ce9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b508570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53334/0 (socket says 172.21.15.115:53334) 2024-05-09T05:40:43.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4301ce9700 1 -- 172.21.15.115:0/3428879618 learned_addr learned my addr 172.21.15.115:0/3428879618 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f43024ea700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42e80b29f0 0x7f42e80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f43024ea700 1 --2- 172.21.15.115:0/3428879618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b502b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f42f2ffd700 1 -- 172.21.15.115:0/1479401201 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f42f80e2450 con 0x7f42fc1045e0 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f43014e8700 1 --2- 172.21.15.115:0/3428879618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b508ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4301ce9700 1 -- 172.21.15.115:0/3428879618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 msgr2=0x55f87b502b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4301ce9700 1 --2- 172.21.15.115:0/3428879618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b502b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4301ce9700 1 -- 172.21.15.115:0/3428879618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 msgr2=0x55f87b508ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4301ce9700 1 --2- 172.21.15.115:0/3428879618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b508ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4301ce9700 1 -- 172.21.15.115:0/3428879618 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87b3bb3f0 con 0x55f87b46f1e0 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f4301ce9700 1 --2- 172.21.15.115:0/3428879618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b508570 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f42f80df8a0 tx=0x7f42f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c40990a78fe643b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.782+0000 7f43024ea700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42e80b29f0 0x7f42e80b4f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55f87b509140 tx=0x7f42f4009920 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:43.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.783+0000 7f42e7fff700 1 -- 172.21.15.115:0/3428879618 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f806c020 con 0x55f87b46f1e0 2024-05-09T05:40:43.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.783+0000 7f42e7fff700 1 -- 172.21.15.115:0/3428879618 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42f80a8fa0 con 0x55f87b46f1e0 2024-05-09T05:40:43.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.783+0000 7f42e7fff700 1 -- 172.21.15.115:0/3428879618 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f806be00 con 0x55f87b46f1e0 2024-05-09T05:40:43.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 -- 172.21.15.115:0/3428879618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 msgr2=0x55f87b508570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:43.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 --2- 172.21.15.115:0/3428879618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b508570 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f42f80df8a0 tx=0x7f42f8000bc0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 -- 172.21.15.115:0/3428879618 shutdown_connections 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 --2- 172.21.15.115:0/3428879618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b502b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 --2- 172.21.15.115:0/3428879618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b508570 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 --2- 172.21.15.115:0/3428879618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b508ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 -- 172.21.15.115:0/3428879618 >> 172.21.15.115:0/3428879618 conn(0x55f87b3e5720 msgr2=0x55f87b4fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 -- 172.21.15.115:0/3428879618 shutdown_connections 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 -- 172.21.15.115:0/3428879618 wait complete. 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 -- start start 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.784+0000 7f4312963b80 1 Processor -- start 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4312963b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b385520 0x55f87b50c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4312963b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b46f1e0 0x55f87b50c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4312963b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b502720 0x55f87b50ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4312963b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f87b2a0c20 con 0x55f87b385520 2024-05-09T05:40:43.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4312963b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f87b3df1d0 con 0x55f87b502720 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4312963b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f87b50f250 con 0x55f87b46f1e0 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4312963b80 10 client.0 _ll_register_callbacks cb 0x55f87b296740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4301ce9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b385520 0x55f87b50c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4301ce9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b385520 0x55f87b50c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53342/0 (socket says 172.21.15.115:53342) 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4301ce9700 1 -- 172.21.15.115:0/4184152727 learned_addr learned my addr 172.21.15.115:0/4184152727 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f43014e8700 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b46f1e0 0x55f87b50c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f43024ea700 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b502720 0x55f87b50ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4301ce9700 1 -- 172.21.15.115:0/4184152727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b46f1e0 msgr2=0x55f87b50c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4301ce9700 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b46f1e0 0x55f87b50c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4301ce9700 1 -- 172.21.15.115:0/4184152727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b502720 msgr2=0x55f87b50ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4301ce9700 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b502720 0x55f87b50ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.785+0000 7f4301ce9700 1 -- 172.21.15.115:0/4184152727 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87b2a0510 con 0x55f87b385520 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f4301ce9700 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b385520 0x55f87b50c230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f42f80a6700 tx=0x7f42f8007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324f6d092fc1d681 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:43.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f42e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f42e57fa700 1 -- 172.21.15.115:0/4184152727 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f801d070 con 0x55f87b385520 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f42e57fa700 1 -- 172.21.15.115:0/4184152727 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f42f80580e0 con 0x55f87b385520 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f4312963b80 1 -- 172.21.15.115:0/4184152727 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f80a8fa0 con 0x55f87b385520 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f42e57fa700 1 -- 172.21.15.115:0/4184152727 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42f80ad840 con 0x55f87b385520 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f4312963b80 10 client.16317 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f4312963b80 1 -- 172.21.15.115:0/4184152727 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f87b519c10 con 0x55f87b385520 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f42e57fa700 1 -- 172.21.15.115:0/4184152727 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42f80a8fa0 con 0x55f87b385520 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.786+0000 7f42e57fa700 1 -- 172.21.15.115:0/4184152727 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f42f80a96f0 con 0x55f87b385520 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.787+0000 7f4312963b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.787+0000 7f4312963b80 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.787+0000 7f4312963b80 20 client.16317 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.787+0000 7f4312963b80 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f87b51c0e0 0x55f87b51e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.787+0000 7f4312963b80 4 client.16317 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:43.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.787+0000 7f4312963b80 1 -- 172.21.15.115:0/4184152727 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f87b432a50 con 0x55f87b51c0e0 2024-05-09T05:40:43.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.787+0000 7f43014e8700 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f87b51c0e0 0x55f87b51e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.787+0000 7f42e57fa700 10 client.16317 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:43.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.788+0000 7f43014e8700 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f87b51c0e0 0x55f87b51e500 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b8c314c73e382f server_cookie=45063bd2246b0e1a in_seq=0 out_seq=0 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.788+0000 7f42e57fa700 1 -- 172.21.15.115:0/4184152727 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f87b432a50 con 0x55f87b51c0e0 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f42e57fa700 10 client.16317 handle_command_reply: tid=0 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 client.16317 shutdown 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 -- 172.21.15.115:0/4184152727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b385520 msgr2=0x55f87b50c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b385520 0x55f87b50c230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f42f80a6700 tx=0x7f42f8007ed0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 -- 172.21.15.115:0/4184152727 shutdown_connections 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b502720 0x55f87b50ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f87b51c0e0 0x55f87b51e500 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b385520 0x55f87b50c230 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 --2- 172.21.15.115:0/4184152727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b46f1e0 0x55f87b50c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f42e57fa700 0 client.16317 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:43.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 -- 172.21.15.115:0/4184152727 >> 172.21.15.115:0/4184152727 conn(0x55f87b3e5720 msgr2=0x55f87b4fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:43.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 -- 172.21.15.115:0/4184152727 shutdown_connections 2024-05-09T05:40:43.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 1 -- 172.21.15.115:0/4184152727 wait complete. 2024-05-09T05:40:43.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.789+0000 7f4312963b80 20 client.16317 trim_cache size 0 max 16384 2024-05-09T05:40:43.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.812+0000 7f4312963b80 1 Processor -- start 2024-05-09T05:40:43.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.812+0000 7f4312963b80 1 -- start start 2024-05-09T05:40:43.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f4312963b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b507af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f4312963b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b50cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f4312963b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b518260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f4312963b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b2620 con 0x55f87b46f1e0 2024-05-09T05:40:43.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f4312963b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b4160 con 0x55f87b385520 2024-05-09T05:40:43.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f4312963b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b2cf0 con 0x55f87b502720 2024-05-09T05:40:43.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43014e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b50cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43014e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b50cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53346/0 (socket says 172.21.15.115:53346) 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43014e8700 1 -- 172.21.15.115:0/233699158 learned_addr learned my addr 172.21.15.115:0/233699158 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43024ea700 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b518260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f4301ce9700 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b507af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43014e8700 1 -- 172.21.15.115:0/233699158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 msgr2=0x55f87b518260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43014e8700 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b518260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43014e8700 1 -- 172.21.15.115:0/233699158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 msgr2=0x55f87b507af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43014e8700 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b507af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43014e8700 1 -- 172.21.15.115:0/233699158 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87b508bb0 con 0x55f87b46f1e0 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f4301ce9700 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b507af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.813+0000 7f43014e8700 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b50cc20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00b700 tx=0x7f42ec005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c86805372b4a5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.814+0000 7f42e77fe700 1 -- 172.21.15.115:0/233699158 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0054d0 con 0x55f87b46f1e0 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.814+0000 7f42e77fe700 1 -- 172.21.15.115:0/233699158 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42ec02e070 con 0x55f87b46f1e0 2024-05-09T05:40:43.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.814+0000 7f42e77fe700 1 -- 172.21.15.115:0/233699158 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00bb90 con 0x55f87b46f1e0 2024-05-09T05:40:43.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 -- 172.21.15.115:0/233699158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 msgr2=0x55f87b50cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b50cc20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00b700 tx=0x7f42ec005040 comp rx=0 tx=0).stop 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 -- 172.21.15.115:0/233699158 shutdown_connections 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b507af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b518260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 --2- 172.21.15.115:0/233699158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b50cc20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 -- 172.21.15.115:0/233699158 >> 172.21.15.115:0/233699158 conn(0x55f87b3e5720 msgr2=0x55f87b501710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 -- 172.21.15.115:0/233699158 shutdown_connections 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 -- 172.21.15.115:0/233699158 wait complete. 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 -- start start 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.815+0000 7f4312963b80 1 Processor -- start 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4312963b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b51f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4312963b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b51ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4312963b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b57f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4312963b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b3a90 con 0x55f87b46f1e0 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f43014e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b51ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4312963b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b33c0 con 0x55f87b385520 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4312963b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1b05c0 con 0x55f87b502720 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f43014e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b51ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53352/0 (socket says 172.21.15.115:53352) 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f43014e8700 1 -- 172.21.15.115:0/93772125 learned_addr learned my addr 172.21.15.115:0/93772125 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4301ce9700 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b51f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4312963b80 10 client.0 _ll_register_callbacks cb 0x55f87b296740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f43024ea700 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b57f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4301ce9700 1 -- 172.21.15.115:0/93772125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 msgr2=0x55f87b57f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4301ce9700 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b57f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4301ce9700 1 -- 172.21.15.115:0/93772125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 msgr2=0x55f87b51ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:43.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4301ce9700 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b51ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4301ce9700 1 -- 172.21.15.115:0/93772125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87b50c000 con 0x55f87b385520 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f43014e8700 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b51ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f4301ce9700 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b51f9b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55f87b502df0 tx=0x7f42f80dfa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdbced3005feefb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f42e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.816+0000 7f42e57fa700 1 -- 172.21.15.115:0/93772125 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f801d070 con 0x55f87b385520 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f4312963b80 1 -- 172.21.15.115:0/93772125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ec02e070 con 0x55f87b385520 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f42e57fa700 1 -- 172.21.15.115:0/93772125 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f42f80580b0 con 0x55f87b385520 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f42e57fa700 1 -- 172.21.15.115:0/93772125 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42f80a8de0 con 0x55f87b385520 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f4312963b80 10 client.25817 fetch_fsmap learned FSMap version 30 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f4312963b80 1 -- 172.21.15.115:0/93772125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f87b586d20 con 0x55f87b385520 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f42e57fa700 1 -- 172.21.15.115:0/93772125 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42ec02e070 con 0x55f87b385520 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f42e57fa700 1 -- 172.21.15.115:0/93772125 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f42f80a3bb0 con 0x55f87b385520 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f4312963b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 30 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f4312963b80 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:43.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f4312963b80 20 client.25817 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:43.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f4312963b80 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f87b5871b0 0x55f87b5895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:43.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f4312963b80 4 client.25817 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:40:43.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f4312963b80 1 -- 172.21.15.115:0/93772125 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f87b42caf0 con 0x55f87b5871b0 2024-05-09T05:40:43.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.817+0000 7f43014e8700 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f87b5871b0 0x55f87b5895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:43.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.818+0000 7f42e57fa700 10 client.25817 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:43.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.818+0000 7f43014e8700 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f87b5871b0 0x55f87b5895d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=604492496aaf4f17 server_cookie=eb2edfaaf804ebce in_seq=0 out_seq=0 2024-05-09T05:40:43.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f42e57fa700 1 -- 172.21.15.115:0/93772125 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f87b42caf0 con 0x55f87b5871b0 2024-05-09T05:40:43.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f42e57fa700 10 client.25817 handle_command_reply: tid=0 2024-05-09T05:40:43.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 client.25817 shutdown 2024-05-09T05:40:43.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 -- 172.21.15.115:0/93772125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 msgr2=0x55f87b51f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:43.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b51f9b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55f87b502df0 tx=0x7f42f80dfa00 comp rx=0 tx=0).stop 2024-05-09T05:40:43.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 -- 172.21.15.115:0/93772125 shutdown_connections 2024-05-09T05:40:43.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87b385520 0x55f87b51f9b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f87b5871b0 0x55f87b5895d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f42e57fa700 0 client.25817 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f87b502720 0x55f87b57f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 --2- 172.21.15.115:0/93772125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f87b46f1e0 0x55f87b51ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 -- 172.21.15.115:0/93772125 >> 172.21.15.115:0/93772125 conn(0x55f87b3e5720 msgr2=0x55f87b505c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 -- 172.21.15.115:0/93772125 shutdown_connections 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 1 -- 172.21.15.115:0/93772125 wait complete. 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.825+0000 7f4312963b80 20 client.25817 trim_cache size 0 max 16384 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:40:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:40:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "e194d4af-7cf3-405b-80f5-b98eab9b4e30", 2024-05-09T05:40:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:40:43.821 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:40:43.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 -- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42e80b29f0 msgr2=0x7f42e80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42e80b29f0 0x7f42e80b4f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55f87b509140 tx=0x7f42f4009920 comp rx=0 tx=0).stop 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 -- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc1045e0 msgr2=0x7f42fc075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc1045e0 0x7f42fc075ac0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f42f800c920 tx=0x7f42f800cc30 comp rx=0 tx=0).stop 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 -- 172.21.15.115:0/1479401201 shutdown_connections 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42e80b29f0 0x7f42e80b4f00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42fc109860 0x7f42fc076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42fc1045e0 0x7f42fc075ac0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 --2- 172.21.15.115:0/1479401201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42fc108f10 0x7f42fc076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 -- 172.21.15.115:0/1479401201 >> 172.21.15.115:0/1479401201 conn(0x7f42fc0ffe70 msgr2=0x7f42fc102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 -- 172.21.15.115:0/1479401201 shutdown_connections 2024-05-09T05:40:43.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:43.826+0000 7f4303f4d700 1 -- 172.21.15.115:0/1479401201 wait complete. 2024-05-09T05:40:43.837 INFO:tasks.workunit.client.0.smithi115.stdout:config.status: creating Makefile 2024-05-09T05:40:43.860 INFO:tasks.workunit.client.0.smithi115.stdout:config.status: creating config.h 2024-05-09T05:40:43.887 INFO:tasks.workunit.client.0.smithi115.stdout:config.status: executing depfiles commands 2024-05-09T05:40:43.990 INFO:tasks.workunit.client.0.smithi115.stdout:config.status: executing default commands 2024-05-09T05:40:44.094 INFO:tasks.workunit.client.0.smithi115.stderr:+ make 2024-05-09T05:40:44.134 INFO:tasks.workunit.client.0.smithi115.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-05-09T05:40:44.141 INFO:tasks.workunit.client.0.smithi115.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-05-09T05:40:44.142 INFO:tasks.workunit.client.0.smithi115.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-05-09T05:40:44.142 INFO:tasks.workunit.client.0.smithi115.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-05-09T05:40:44.142 INFO:tasks.workunit.client.0.smithi115.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-05-09T05:40:44.142 INFO:tasks.workunit.client.0.smithi115.stderr: any GNU archive site. 2024-05-09T05:40:44.148 INFO:tasks.workunit.client.0.smithi115.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-05-09T05:40:44.153 INFO:tasks.workunit.client.0.smithi115.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-05-09T05:40:44.154 INFO:tasks.workunit.client.0.smithi115.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-05-09T05:40:44.154 INFO:tasks.workunit.client.0.smithi115.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-05-09T05:40:44.155 INFO:tasks.workunit.client.0.smithi115.stderr: You might want to install the `Automake' and `Perl' packages. 2024-05-09T05:40:44.155 INFO:tasks.workunit.client.0.smithi115.stderr: Grab them from any GNU archive site. 2024-05-09T05:40:44.351 INFO:tasks.workunit.client.0.smithi115.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-05-09T05:40:44.751 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:40:44.979 INFO:tasks.workunit.client.0.smithi115.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-05-09T05:40:44.979 INFO:tasks.workunit.client.0.smithi115.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-05-09T05:40:44.979 INFO:tasks.workunit.client.0.smithi115.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-05-09T05:40:44.979 INFO:tasks.workunit.client.0.smithi115.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-05-09T05:40:45.062 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:45.136 INFO:tasks.workunit.client.0.smithi115.stdout:/bin/sh ./config.status --recheck 2024-05-09T05:40:45.208 INFO:tasks.workunit.client.0.smithi115.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-05-09T05:40:45.367 INFO:tasks.workunit.client.0.smithi115.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-09T05:40:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:45 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:40:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:45 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:45 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:40:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:45 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:45 smithi161 ceph-mon[33805]: pgmap v79: 129 pgs: 129 active+clean; 5.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 440 KiB/s wr, 146 op/s 2024-05-09T05:40:45.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:45 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:40:45.436 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:45 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:40:45.436 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:45 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:45.436 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:45 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:40:45.436 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:45 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:45.437 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:45 smithi115 ceph-mon[30349]: pgmap v79: 129 pgs: 129 active+clean; 5.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 440 KiB/s wr, 146 op/s 2024-05-09T05:40:45.437 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:45 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:40:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:45 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:40:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:45 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:45 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:40:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:45 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:40:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:45 smithi195 ceph-mon[33844]: pgmap v79: 129 pgs: 129 active+clean; 5.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 440 KiB/s wr, 146 op/s 2024-05-09T05:40:45.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:45 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:40:46.376 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether build environment is sane... yes 2024-05-09T05:40:46.385 INFO:tasks.workunit.client.0.smithi115.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-09T05:40:46.385 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gawk... gawk 2024-05-09T05:40:46.409 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether make sets $(MAKE)... yes 2024-05-09T05:40:46.453 INFO:tasks.workunit.client.0.smithi115.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-09T05:40:46.453 INFO:tasks.workunit.client.0.smithi115.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-09T05:40:46.473 INFO:tasks.workunit.client.0.smithi115.stdout:checking for style of include used by make... GNU 2024-05-09T05:40:46.479 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gcc... gcc 2024-05-09T05:40:46.506 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:46 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:40:46.606 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether the C compiler works... yes 2024-05-09T05:40:46.606 INFO:tasks.workunit.client.0.smithi115.stdout:checking for C compiler default output file name... a.out 2024-05-09T05:40:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:46 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:40:46.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:46 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:40:46.663 INFO:tasks.workunit.client.0.smithi115.stdout:checking for suffix of executables... 2024-05-09T05:40:46.720 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether we are cross compiling... no 2024-05-09T05:40:46.771 INFO:tasks.workunit.client.0.smithi115.stdout:checking for suffix of object files... o 2024-05-09T05:40:46.805 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether we are using the GNU C compiler... yes 2024-05-09T05:40:46.847 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether gcc accepts -g... yes 2024-05-09T05:40:46.902 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-09T05:40:46.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.949+0000 7fccdf30f700 1 -- 172.21.15.115:0/134231677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd8107e10 msgr2=0x7fccd8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:46.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.949+0000 7fccdf30f700 1 --2- 172.21.15.115:0/134231677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd8107e10 0x7fccd8108220 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fccc80099c0 tx=0x7fccc8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.949+0000 7fccdf30f700 1 -- 172.21.15.115:0/134231677 shutdown_connections 2024-05-09T05:40:46.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.949+0000 7fccdf30f700 1 --2- 172.21.15.115:0/134231677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd81034e0 0x7fccd81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.949+0000 7fccdf30f700 1 --2- 172.21.15.115:0/134231677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccd8108760 0x7fccd81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.949+0000 7fccdf30f700 1 --2- 172.21.15.115:0/134231677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd8107e10 0x7fccd8108220 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.949+0000 7fccdf30f700 1 -- 172.21.15.115:0/134231677 >> 172.21.15.115:0/134231677 conn(0x7fccd80fef30 msgr2=0x7fccd8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:46.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.950+0000 7fccdf30f700 1 -- 172.21.15.115:0/134231677 shutdown_connections 2024-05-09T05:40:46.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.950+0000 7fccdf30f700 1 -- 172.21.15.115:0/134231677 wait complete. 2024-05-09T05:40:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.950+0000 7fccdf30f700 1 Processor -- start 2024-05-09T05:40:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.950+0000 7fccdf30f700 1 -- start start 2024-05-09T05:40:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdf30f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd81034e0 0x7fccd819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdf30f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccd8107e10 0x7fccd819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdf30f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd8108760 0x7fccd819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdf30f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8114390 con 0x7fccd81034e0 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdf30f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8114210 con 0x7fccd8108760 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdf30f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8114510 con 0x7fccd8107e10 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd0ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd81034e0 0x7fccd819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd0ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd81034e0 0x7fccd819ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53368/0 (socket says 172.21.15.115:53368) 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd0ab700 1 -- 172.21.15.115:0/3753622611 learned_addr learned my addr 172.21.15.115:0/3753622611 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd8108760 0x7fccd819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccd8107e10 0x7fccd819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd0ab700 1 -- 172.21.15.115:0/3753622611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccd8107e10 msgr2=0x7fccd819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccd8107e10 0x7fccd819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd0ab700 1 -- 172.21.15.115:0/3753622611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd8108760 msgr2=0x7fccd819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd8108760 0x7fccd819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd0ab700 1 -- 172.21.15.115:0/3753622611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd819fc50 con 0x7fccd81034e0 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccd8107e10 0x7fccd819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.951+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd8108760 0x7fccd819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.952+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd81034e0 0x7fccd819ad80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e9a0 tx=0x7fccd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.952+0000 7fccce7fc700 1 -- 172.21.15.115:0/3753622611 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd400c9c0 con 0x7fccd81034e0 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.952+0000 7fccdf30f700 1 -- 172.21.15.115:0/3753622611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd819ff40 con 0x7fccd81034e0 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.952+0000 7fccce7fc700 1 -- 172.21.15.115:0/3753622611 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccd4010430 con 0x7fccd81034e0 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.952+0000 7fccdf30f700 1 -- 172.21.15.115:0/3753622611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a7860 con 0x7fccd81034e0 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.952+0000 7fccce7fc700 1 -- 172.21.15.115:0/3753622611 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4018680 con 0x7fccd81034e0 2024-05-09T05:40:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.955+0000 7fccce7fc700 1 -- 172.21.15.115:0/3753622611 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fccd4018820 con 0x7fccd81034e0 2024-05-09T05:40:46.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.955+0000 7fccce7fc700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fccc40b29f0 0x7fccc40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.956+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fccc40b29f0 0x7fccc40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.956+0000 7fccce7fc700 1 -- 172.21.15.115:0/3753622611 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fccd40e2a40 con 0x7fccd81034e0 2024-05-09T05:40:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.956+0000 7fccedd25b80 1 Processor -- start 2024-05-09T05:40:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.956+0000 7fccedd25b80 1 -- start start 2024-05-09T05:40:46.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.956+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c62240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.956+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9adf4a0 0x5587f9c62780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.956+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c5c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.956+0000 7fccedd25b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5587f9b7f0a0 con 0x5587f9c5c400 2024-05-09T05:40:46.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccedd25b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5587f9b249a0 con 0x5587f9adf4a0 2024-05-09T05:40:46.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccedd25b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5587f9b7b060 con 0x5587f9b59d00 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd8ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c5c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd8ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c5c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53382/0 (socket says 172.21.15.115:53382) 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd8ac700 1 -- 172.21.15.115:0/2744454422 learned_addr learned my addr 172.21.15.115:0/2744454422 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fccc40b29f0 0x7fccc40b4f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fccc8000c00 tx=0x7fccc801a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c62240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9adf4a0 0x5587f9c62780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd8ac700 1 -- 172.21.15.115:0/2744454422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 msgr2=0x5587f9c62240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c62240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd8ac700 1 -- 172.21.15.115:0/2744454422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9adf4a0 msgr2=0x5587f9c62780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9adf4a0 0x5587f9c62780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd8ac700 1 -- 172.21.15.115:0/2744454422 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587f9b15500 con 0x5587f9c5c400 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c62240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c5c810 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009510 tx=0x7fccd0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdf28bdc05d6c5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccc37fe700 1 -- 172.21.15.115:0/2744454422 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd001b020 con 0x5587f9c5c400 2024-05-09T05:40:46.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccc37fe700 1 -- 172.21.15.115:0/2744454422 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccd000eb80 con 0x5587f9c5c400 2024-05-09T05:40:46.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.957+0000 7fccc37fe700 1 -- 172.21.15.115:0/2744454422 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0017cf0 con 0x5587f9c5c400 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.958+0000 7fccedd25b80 1 -- 172.21.15.115:0/2744454422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 msgr2=0x5587f9c5c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.958+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c5c810 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009510 tx=0x7fccd0009920 comp rx=0 tx=0).stop 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.958+0000 7fccedd25b80 1 -- 172.21.15.115:0/2744454422 shutdown_connections 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.958+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c5c810 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.958+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c62240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.958+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2744454422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9adf4a0 0x5587f9c62780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.958+0000 7fccedd25b80 1 -- 172.21.15.115:0/2744454422 >> 172.21.15.115:0/2744454422 conn(0x5587f9ae2f70 msgr2=0x5587f9c57220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.959+0000 7fccedd25b80 1 -- 172.21.15.115:0/2744454422 shutdown_connections 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.959+0000 7fccedd25b80 1 -- 172.21.15.115:0/2744454422 wait complete. 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.959+0000 7fccedd25b80 1 -- start start 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.959+0000 7fccedd25b80 1 Processor -- start 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9adf4a0 0x5587f9c65fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c66500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c66a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccedd25b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5587f9c5d160 con 0x5587f9adf4a0 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccedd25b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5587f9b19b20 con 0x5587f9c5c400 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccedd25b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5587f9c69020 con 0x5587f9b59d00 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccedd25b80 10 client.0 _ll_register_callbacks cb 0x5587f99f06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdd0ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9adf4a0 0x5587f9c65fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdc8aa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c66500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdc8aa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c66500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:34074/0 (socket says 172.21.15.115:34074) 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdc8aa700 1 -- 172.21.15.115:0/1637397008 learned_addr learned my addr 172.21.15.115:0/1637397008 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c66a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdc8aa700 1 -- 172.21.15.115:0/1637397008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 msgr2=0x5587f9c66a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c66a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdc8aa700 1 -- 172.21.15.115:0/1637397008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9adf4a0 msgr2=0x5587f9c65fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9adf4a0 0x5587f9c65fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdc8aa700 1 -- 172.21.15.115:0/1637397008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587f99fa490 con 0x5587f9b59d00 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.960+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c66a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c66500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fccc80099c0 tx=0x7fccc8005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b0eb4ef78d78b32 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccc0ff9700 1 -- 172.21.15.115:0/1637397008 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8003a50 con 0x5587f9b59d00 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccedd25b80 1 -- 172.21.15.115:0/1637397008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd000eb80 con 0x5587f9b59d00 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccc0ff9700 1 -- 172.21.15.115:0/1637397008 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fccc8003bf0 con 0x5587f9b59d00 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccc0ff9700 1 -- 172.21.15.115:0/1637397008 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fccc8003df0 con 0x5587f9b59d00 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccedd25b80 10 client.25894 fetch_fsmap learned FSMap version 31 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccedd25b80 1 -- 172.21.15.115:0/1637397008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5587f9c73a20 con 0x5587f9b59d00 2024-05-09T05:40:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccc0ff9700 1 -- 172.21.15.115:0/1637397008 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccd000eb80 con 0x5587f9b59d00 2024-05-09T05:40:46.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccc0ff9700 1 -- 172.21.15.115:0/1637397008 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fccc80264c0 con 0x5587f9b59d00 2024-05-09T05:40:46.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccedd25b80 10 client.25894 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T05:40:46.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccedd25b80 10 client.25894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:46.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccedd25b80 20 client.25894 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:46.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccedd25b80 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5587f9c76420 0x5587f9c788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.961+0000 7fccedd25b80 4 client.25894 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:46.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.962+0000 7fccedd25b80 1 -- 172.21.15.115:0/1637397008 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587f9b8ca80 con 0x5587f9c76420 2024-05-09T05:40:46.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.962+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5587f9c76420 0x5587f9c788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.962+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5587f9c76420 0x5587f9c788b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30f200f9fca8b8d server_cookie=e1e654f4aa116607 in_seq=0 out_seq=0 2024-05-09T05:40:46.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.962+0000 7fccc0ff9700 10 client.25894 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:46.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.963+0000 7fccc0ff9700 1 -- 172.21.15.115:0/1637397008 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587f9b8ca80 con 0x5587f9c76420 2024-05-09T05:40:46.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.963+0000 7fccc0ff9700 10 client.25894 handle_command_reply: tid=0 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.963+0000 7fccedd25b80 1 client.25894 shutdown 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 -- 172.21.15.115:0/1637397008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 msgr2=0x5587f9c66500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c66500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fccc80099c0 tx=0x7fccc8005040 comp rx=0 tx=0).stop 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 -- 172.21.15.115:0/1637397008 shutdown_connections 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c66a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5587f9c76420 0x5587f9c788b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9adf4a0 0x5587f9c65fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 --2- 172.21.15.115:0/1637397008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9b59d00 0x5587f9c66500 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccc0ff9700 0 client.25894 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 -- 172.21.15.115:0/1637397008 >> 172.21.15.115:0/1637397008 conn(0x5587f9ae2f70 msgr2=0x5587f9c50ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 -- 172.21.15.115:0/1637397008 shutdown_connections 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 1 -- 172.21.15.115:0/1637397008 wait complete. 2024-05-09T05:40:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.964+0000 7fccedd25b80 20 client.25894 trim_cache size 0 max 16384 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.992+0000 7fccedd25b80 1 Processor -- start 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.992+0000 7fccedd25b80 1 -- start start 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 0x5587f9c573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9b59d00 0x5587f9c51a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c71be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccedd25b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc00eba0 con 0x5587f9b59d00 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccedd25b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc010710 con 0x5587f9c5c400 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccedd25b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc00f270 con 0x5587f9adf4a0 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdc8aa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9b59d00 0x5587f9c51a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdc8aa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9b59d00 0x5587f9c51a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53410/0 (socket says 172.21.15.115:53410) 2024-05-09T05:40:46.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdc8aa700 1 -- 172.21.15.115:0/3732366612 learned_addr learned my addr 172.21.15.115:0/3732366612 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:46.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c71be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 0x5587f9c573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdc8aa700 1 -- 172.21.15.115:0/3732366612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 msgr2=0x5587f9c573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:46.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 0x5587f9c573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdc8aa700 1 -- 172.21.15.115:0/3732366612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 msgr2=0x5587f9c71be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:46.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c71be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdc8aa700 1 -- 172.21.15.115:0/3732366612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587f9c539d0 con 0x5587f9b59d00 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c71be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.993+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 0x5587f9c573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.994+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9b59d00 0x5587f9c51a20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fccc8026a80 tx=0x7fccc800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0516856bfefc243 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.994+0000 7fccc2ffd700 1 -- 172.21.15.115:0/3732366612 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8026b80 con 0x5587f9b59d00 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.994+0000 7fccc2ffd700 1 -- 172.21.15.115:0/3732366612 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccc8003a50 con 0x5587f9b59d00 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.994+0000 7fccc2ffd700 1 -- 172.21.15.115:0/3732366612 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8031b30 con 0x5587f9b59d00 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 -- 172.21.15.115:0/3732366612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9b59d00 msgr2=0x5587f9c51a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9b59d00 0x5587f9c51a20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fccc8026a80 tx=0x7fccc800b790 comp rx=0 tx=0).stop 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 -- 172.21.15.115:0/3732366612 shutdown_connections 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9c5c400 0x5587f9c71be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 0x5587f9c573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 --2- 172.21.15.115:0/3732366612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9b59d00 0x5587f9c51a20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 -- 172.21.15.115:0/3732366612 >> 172.21.15.115:0/3732366612 conn(0x5587f9ae2f70 msgr2=0x5587f9c61720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:46.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 -- 172.21.15.115:0/3732366612 shutdown_connections 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 -- 172.21.15.115:0/3732366612 wait complete. 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 -- start start 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.995+0000 7fccedd25b80 1 Processor -- start 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 0x5587f9c6e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9b59d00 0x5587f9c6e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccedd25b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c70fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccedd25b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc010010 con 0x5587f9c5c400 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccedd25b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc00f940 con 0x5587f9b59d00 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccedd25b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc00cb40 con 0x5587f9adf4a0 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccedd25b80 10 client.0 _ll_register_callbacks cb 0x5587f9c5cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccdd8ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c70fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccdc8aa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9b59d00 0x5587f9c6e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccdd0ab700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 0x5587f9c6e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccdd8ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c70fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53422/0 (socket says 172.21.15.115:53422) 2024-05-09T05:40:46.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccdc8aa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9b59d00 0x5587f9c6e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:39994/0 (socket says 172.21.15.115:39994) 2024-05-09T05:40:46.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.996+0000 7fccdd8ac700 1 -- 172.21.15.115:0/2884029101 learned_addr learned my addr 172.21.15.115:0/2884029101 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:46.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccdd8ac700 1 -- 172.21.15.115:0/2884029101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 msgr2=0x5587f9c6e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:46.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 0x5587f9c6e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccdd8ac700 1 -- 172.21.15.115:0/2884029101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9b59d00 msgr2=0x5587f9c6e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:46.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9b59d00 0x5587f9c6e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccdd8ac700 1 -- 172.21.15.115:0/2884029101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587f9c71750 con 0x5587f9c5c400 2024-05-09T05:40:46.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccdc8aa700 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9b59d00 0x5587f9c6e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:46.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccdd8ac700 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c70fd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fccd00090d0 tx=0x7fccd0017c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46aa163576891b0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:46.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:46.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccc0ff9700 1 -- 172.21.15.115:0/2884029101 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd001b040 con 0x5587f9c5c400 2024-05-09T05:40:46.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccc0ff9700 1 -- 172.21.15.115:0/2884029101 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fccd000ea00 con 0x5587f9c5c400 2024-05-09T05:40:46.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.997+0000 7fccedd25b80 1 -- 172.21.15.115:0/2884029101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc8003a50 con 0x5587f9c5c400 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccc0ff9700 1 -- 172.21.15.115:0/2884029101 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fccd000ec00 con 0x5587f9c5c400 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccedd25b80 10 client.16341 fetch_fsmap learned FSMap version 31 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccedd25b80 1 -- 172.21.15.115:0/2884029101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5587f9ce0670 con 0x5587f9c5c400 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccc0ff9700 1 -- 172.21.15.115:0/2884029101 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccc8003a50 con 0x5587f9c5c400 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccc0ff9700 1 -- 172.21.15.115:0/2884029101 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fccd0021430 con 0x5587f9c5c400 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccedd25b80 10 client.16341 fetch_fsmap finished waiting for FSMap version 31 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccedd25b80 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccedd25b80 20 client.16341 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5587f9ce0b80 0x5587f9ce2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccedd25b80 4 client.16341 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccedd25b80 1 -- 172.21.15.115:0/2884029101 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587f9b86b20 con 0x5587f9ce0b80 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.998+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5587f9ce0b80 0x5587f9ce2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccc0ff9700 10 client.16341 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccdd0ab700 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5587f9ce0b80 0x5587f9ce2fa0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eecf6921aac44f47 server_cookie=a6bce7fbdd53796b in_seq=0 out_seq=0 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccc0ff9700 1 -- 172.21.15.115:0/2884029101 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5587f9b86b20 con 0x5587f9ce0b80 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccc0ff9700 10 client.16341 handle_command_reply: tid=0 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 client.16341 shutdown 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 -- 172.21.15.115:0/2884029101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 msgr2=0x5587f9c70fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c70fd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fccd00090d0 tx=0x7fccd0017c30 comp rx=0 tx=0).stop 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 -- 172.21.15.115:0/2884029101 shutdown_connections 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5587f9ce0b80 0x5587f9ce2fa0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5587f9c5c400 0x5587f9c70fd0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5587f9adf4a0 0x5587f9c6e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccc0ff9700 0 client.16341 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 --2- 172.21.15.115:0/2884029101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f9b59d00 0x5587f9c6e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 -- 172.21.15.115:0/2884029101 >> 172.21.15.115:0/2884029101 conn(0x5587f9ae2f70 msgr2=0x5587f9c5fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 -- 172.21.15.115:0/2884029101 shutdown_connections 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:46.999+0000 7fccedd25b80 1 -- 172.21.15.115:0/2884029101 wait complete. 2024-05-09T05:40:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccedd25b80 20 client.16341 trim_cache size 0 max 16384 2024-05-09T05:40:46.998 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:40:46.998 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:40:46.998 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:40:46.998 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 -- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fccc40b29f0 msgr2=0x7fccc40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fccc40b29f0 0x7fccc40b4f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fccc8000c00 tx=0x7fccc801a040 comp rx=0 tx=0).stop 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 -- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd81034e0 msgr2=0x7fccd819ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd81034e0 0x7fccd819ad80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e9a0 tx=0x7fccd400ecb0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 -- 172.21.15.115:0/3753622611 shutdown_connections 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fccc40b29f0 0x7fccc40b4f00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd8108760 0x7fccd819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd81034e0 0x7fccd819ad80 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 --2- 172.21.15.115:0/3753622611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccd8107e10 0x7fccd819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 -- 172.21.15.115:0/3753622611 >> 172.21.15.115:0/3753622611 conn(0x7fccd80fef30 msgr2=0x7fccd8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 -- 172.21.15.115:0/3753622611 shutdown_connections 2024-05-09T05:40:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:47.000+0000 7fccdf30f700 1 -- 172.21.15.115:0/3753622611 wait complete. 2024-05-09T05:40:47.084 INFO:tasks.workunit.client.0.smithi115.stdout:checking dependency style of gcc... gcc3 2024-05-09T05:40:47.152 INFO:tasks.workunit.client.0.smithi115.stdout:checking how to run the C preprocessor... gcc -E 2024-05-09T05:40:47.221 INFO:tasks.workunit.client.0.smithi115.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-09T05:40:47.223 INFO:tasks.workunit.client.0.smithi115.stdout:checking for egrep... /usr/bin/grep -E 2024-05-09T05:40:47.398 INFO:tasks.workunit.client.0.smithi115.stdout:checking for ANSI C header files... yes 2024-05-09T05:40:47.446 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/types.h... yes 2024-05-09T05:40:47.504 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/stat.h... yes 2024-05-09T05:40:47.515 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:47 smithi195 ceph-mon[33844]: pgmap v80: 129 pgs: 129 active+clean; 5.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 440 KiB/s wr, 142 op/s 2024-05-09T05:40:47.560 INFO:tasks.workunit.client.0.smithi115.stdout:checking for stdlib.h... yes 2024-05-09T05:40:47.561 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:47 smithi161 ceph-mon[33805]: pgmap v80: 129 pgs: 129 active+clean; 5.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 440 KiB/s wr, 142 op/s 2024-05-09T05:40:47.618 INFO:tasks.workunit.client.0.smithi115.stdout:checking for string.h... yes 2024-05-09T05:40:47.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:47 smithi115 ceph-mon[30349]: pgmap v80: 129 pgs: 129 active+clean; 5.9 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 440 KiB/s wr, 142 op/s 2024-05-09T05:40:47.674 INFO:tasks.workunit.client.0.smithi115.stdout:checking for memory.h... yes 2024-05-09T05:40:47.732 INFO:tasks.workunit.client.0.smithi115.stdout:checking for strings.h... yes 2024-05-09T05:40:47.788 INFO:tasks.workunit.client.0.smithi115.stdout:checking for inttypes.h... yes 2024-05-09T05:40:47.850 INFO:tasks.workunit.client.0.smithi115.stdout:checking for stdint.h... yes 2024-05-09T05:40:47.853 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:40:47.853 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:40:47.912 INFO:tasks.workunit.client.0.smithi115.stdout:checking for unistd.h... yes 2024-05-09T05:40:47.982 INFO:tasks.workunit.client.0.smithi115.stdout:checking minix/config.h usability... no 2024-05-09T05:40:48.029 INFO:tasks.workunit.client.0.smithi115.stdout:checking minix/config.h presence... no 2024-05-09T05:40:48.029 INFO:tasks.workunit.client.0.smithi115.stdout:checking for minix/config.h... no 2024-05-09T05:40:48.085 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-05-09T05:40:48.099 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gcc... (cached) gcc 2024-05-09T05:40:48.139 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:48.160 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-09T05:40:48.160 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-09T05:40:48.160 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-09T05:40:48.161 INFO:tasks.workunit.client.0.smithi115.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-09T05:40:48.224 INFO:tasks.workunit.client.0.smithi115.stdout:checking for main in -lm... yes 2024-05-09T05:40:48.290 INFO:tasks.workunit.client.0.smithi115.stdout:checking for main in -lpthread... yes 2024-05-09T05:40:48.295 INFO:tasks.workunit.client.0.smithi115.stdout:checking for ANSI C header files... (cached) yes 2024-05-09T05:40:48.359 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-09T05:40:48.468 INFO:tasks.workunit.client.0.smithi115.stdout:checking pthread.h usability... yes 2024-05-09T05:40:48.494 INFO:tasks.workunit.client.0.smithi115.stdout:checking pthread.h presence... yes 2024-05-09T05:40:48.494 INFO:tasks.workunit.client.0.smithi115.stdout:checking for pthread.h... yes 2024-05-09T05:40:48.607 INFO:tasks.workunit.client.0.smithi115.stdout:checking fcntl.h usability... yes 2024-05-09T05:40:48.631 INFO:tasks.workunit.client.0.smithi115.stdout:checking fcntl.h presence... yes 2024-05-09T05:40:48.631 INFO:tasks.workunit.client.0.smithi115.stdout:checking for fcntl.h... yes 2024-05-09T05:40:48.884 INFO:tasks.workunit.client.0.smithi115.stdout:checking limits.h usability... yes 2024-05-09T05:40:48.910 INFO:tasks.workunit.client.0.smithi115.stdout:checking limits.h presence... yes 2024-05-09T05:40:48.910 INFO:tasks.workunit.client.0.smithi115.stdout:checking for limits.h... yes 2024-05-09T05:40:48.935 INFO:tasks.workunit.client.0.smithi115.stdout:checking for stdint.h... (cached) yes 2024-05-09T05:40:49.015 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/time.h usability... yes 2024-05-09T05:40:49.045 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/time.h presence... yes 2024-05-09T05:40:49.046 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/time.h... yes 2024-05-09T05:40:49.072 INFO:tasks.workunit.client.0.smithi115.stdout:checking for unistd.h... (cached) yes 2024-05-09T05:40:49.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:49 smithi115 ceph-mon[30349]: pgmap v81: 129 pgs: 129 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 712 KiB/s wr, 217 op/s 2024-05-09T05:40:49.402 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/vfs.h usability... yes 2024-05-09T05:40:49.426 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/vfs.h presence... yes 2024-05-09T05:40:49.426 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/vfs.h... yes 2024-05-09T05:40:49.586 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/limits.h usability... no 2024-05-09T05:40:49.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:49 smithi195 ceph-mon[33844]: pgmap v81: 129 pgs: 129 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 712 KiB/s wr, 217 op/s 2024-05-09T05:40:49.631 INFO:tasks.workunit.client.0.smithi115.stdout:checking sys/limits.h presence... no 2024-05-09T05:40:49.631 INFO:tasks.workunit.client.0.smithi115.stdout:checking for sys/limits.h... no 2024-05-09T05:40:49.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:49 smithi161 ceph-mon[33805]: pgmap v81: 129 pgs: 129 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 712 KiB/s wr, 217 op/s 2024-05-09T05:40:49.737 INFO:tasks.workunit.client.0.smithi115.stdout:checking for an ANSI C-conforming const... yes 2024-05-09T05:40:49.898 INFO:tasks.workunit.client.0.smithi115.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-09T05:40:50.153 INFO:tasks.workunit.client.0.smithi115.stdout:checking for system... yes 2024-05-09T05:40:50.331 INFO:tasks.workunit.client.0.smithi115.stdout:checking for gettimeofday... yes 2024-05-09T05:40:50.556 INFO:tasks.workunit.client.0.smithi115.stdout:checking for mkdir... yes 2024-05-09T05:40:50.799 INFO:tasks.workunit.client.0.smithi115.stdout:checking for strerror... yes 2024-05-09T05:40:51.014 INFO:tasks.workunit.client.0.smithi115.stdout:checking for open64... yes 2024-05-09T05:40:51.182 INFO:tasks.workunit.client.0.smithi115.stdout:checking for stat64... yes 2024-05-09T05:40:51.355 INFO:tasks.workunit.client.0.smithi115.stdout:checking for fseeko64... yes 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.515+0000 7efea8061640 1 -- 172.21.15.115:0/3509661065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 msgr2=0x7efea010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.515+0000 7efea8061640 1 --2- 172.21.15.115:0/3509661065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 0x7efea010f8e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7efe98009ca0 tx=0x7efe9802a8c0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.515+0000 7efea8061640 1 -- 172.21.15.115:0/3509661065 shutdown_connections 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.515+0000 7efea8061640 1 --2- 172.21.15.115:0/3509661065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 0x7efea010f8e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.515+0000 7efea8061640 1 --2- 172.21.15.115:0/3509661065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea0109260 0x7efea01033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.515+0000 7efea8061640 1 --2- 172.21.15.115:0/3509661065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea0108dc0 0x7efea0102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.515+0000 7efea8061640 1 -- 172.21.15.115:0/3509661065 >> 172.21.15.115:0/3509661065 conn(0x7efea00779e0 msgr2=0x7efea0077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.515+0000 7efea8061640 1 -- 172.21.15.115:0/3509661065 shutdown_connections 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.515+0000 7efea8061640 1 -- 172.21.15.115:0/3509661065 wait complete. 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.516+0000 7efea8061640 1 Processor -- start 2024-05-09T05:40:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.516+0000 7efea8061640 1 -- start start 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.516+0000 7efea8061640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 0x7efea019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.516+0000 7efea8061640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea0108dc0 0x7efea019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.516+0000 7efea8061640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea0109260 0x7efea019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.516+0000 7efea8061640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efea0112ed0 con 0x7efea0103a30 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.516+0000 7efea8061640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efea0112d50 con 0x7efea0108dc0 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.516+0000 7efea8061640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efea0113050 con 0x7efea0109260 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.516+0000 7efea5dfd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 0x7efea019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea5dfd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 0x7efea019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53444/0 (socket says 172.21.15.115:53444) 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea5dfd640 1 -- 172.21.15.115:0/3275881514 learned_addr learned my addr 172.21.15.115:0/3275881514 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea55fc640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea0108dc0 0x7efea019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea65fe640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea0109260 0x7efea019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea5dfd640 1 -- 172.21.15.115:0/3275881514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea0109260 msgr2=0x7efea019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea5dfd640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea0109260 0x7efea019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea5dfd640 1 -- 172.21.15.115:0/3275881514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea0108dc0 msgr2=0x7efea019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea5dfd640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea0108dc0 0x7efea019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea5dfd640 1 -- 172.21.15.115:0/3275881514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea01a6b00 con 0x7efea0103a30 2024-05-09T05:40:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea5dfd640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 0x7efea019b7f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00e9f0 tx=0x7efe9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efe96ffd640 1 -- 172.21.15.115:0/3275881514 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00ccc0 con 0x7efea0103a30 2024-05-09T05:40:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efe96ffd640 1 -- 172.21.15.115:0/3275881514 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe9c0054b0 con 0x7efea0103a30 2024-05-09T05:40:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea8061640 1 -- 172.21.15.115:0/3275881514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea01a6df0 con 0x7efea0103a30 2024-05-09T05:40:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efe96ffd640 1 -- 172.21.15.115:0/3275881514 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c010430 con 0x7efea0103a30 2024-05-09T05:40:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.517+0000 7efea8061640 1 -- 172.21.15.115:0/3275881514 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea01a7200 con 0x7efea0103a30 2024-05-09T05:40:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.520+0000 7efeaeffa740 1 Processor -- start 2024-05-09T05:40:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.520+0000 7efeaeffa740 1 -- start start 2024-05-09T05:40:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.520+0000 7efe96ffd640 1 -- 172.21.15.115:0/3275881514 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7efe9c0046c0 con 0x7efea0103a30 2024-05-09T05:40:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.520+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b82fe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.520+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b82febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.520+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b82ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efeaeffa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5617b82637c0 con 0x5617b7f3a4d0 2024-05-09T05:40:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efeaeffa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5617b81ff0e0 con 0x5617b7f322f0 2024-05-09T05:40:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efeaeffa740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5617b825f780 con 0x5617b7f5b1b0 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea55fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b82ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea55fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b82ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53458/0 (socket says 172.21.15.115:53458) 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea55fc640 1 -- 172.21.15.115:0/311171392 learned_addr learned my addr 172.21.15.115:0/311171392 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea5dfd640 1 --2- 172.21.15.115:0/311171392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b82febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea65fe640 1 --2- 172.21.15.115:0/311171392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b82fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea55fc640 1 -- 172.21.15.115:0/311171392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 msgr2=0x5617b82fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea55fc640 1 --2- 172.21.15.115:0/311171392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b82fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea55fc640 1 -- 172.21.15.115:0/311171392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 msgr2=0x5617b82febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea55fc640 1 --2- 172.21.15.115:0/311171392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b82febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea55fc640 1 -- 172.21.15.115:0/311171392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b82287c0 con 0x5617b7f3a4d0 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea55fc640 1 --2- 172.21.15.115:0/311171392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b82ff100 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efe90009ba0 tx=0x7efe90011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ffe3fef5a4c7a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efe96ffd640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe8c0b2950 0x7efe8c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efe8bfff640 1 -- 172.21.15.115:0/311171392 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90004220 con 0x5617b7f3a4d0 2024-05-09T05:40:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efea65fe640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe8c0b2950 0x7efe8c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efe8bfff640 1 -- 172.21.15.115:0/311171392 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe90009e10 con 0x5617b7f3a4d0 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.521+0000 7efe8bfff640 1 -- 172.21.15.115:0/311171392 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90019690 con 0x5617b7f3a4d0 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.522+0000 7efe96ffd640 1 -- 172.21.15.115:0/3275881514 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efe9c0e2c90 con 0x7efea0103a30 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.522+0000 7efeaeffa740 1 -- 172.21.15.115:0/311171392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 msgr2=0x5617b82ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.522+0000 7efea65fe640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe8c0b2950 0x7efe8c0b4e60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efe98006010 tx=0x7efe98009360 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.522+0000 7efeaeffa740 1 --2- 172.21.15.115:0/311171392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b82ff100 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efe90009ba0 tx=0x7efe90011810 comp rx=0 tx=0).stop 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.522+0000 7efeaeffa740 1 -- 172.21.15.115:0/311171392 shutdown_connections 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.522+0000 7efeaeffa740 1 --2- 172.21.15.115:0/311171392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b82ff100 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.522+0000 7efeaeffa740 1 --2- 172.21.15.115:0/311171392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b82febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.522+0000 7efeaeffa740 1 --2- 172.21.15.115:0/311171392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b82fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.522+0000 7efeaeffa740 1 -- 172.21.15.115:0/311171392 >> 172.21.15.115:0/311171392 conn(0x5617b7f4f300 msgr2=0x5617b82fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 -- 172.21.15.115:0/311171392 shutdown_connections 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 -- 172.21.15.115:0/311171392 wait complete. 2024-05-09T05:40:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 -- start start 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 Processor -- start 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8308d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b8309250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5617b830b7b0 con 0x5617b7f3a4d0 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5617b81ea9c0 con 0x5617b7f322f0 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.523+0000 7efeaeffa740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5617b830b930 con 0x5617b7f5b1b0 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efeaeffa740 10 client.0 _ll_register_callbacks cb 0x5617b8217020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea55fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b8309250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea55fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b8309250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53470/0 (socket says 172.21.15.115:53470) 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea55fc640 1 -- 172.21.15.115:0/4190954127 learned_addr learned my addr 172.21.15.115:0/4190954127 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea5dfd640 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8308d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea65fe640 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea55fc640 1 -- 172.21.15.115:0/4190954127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 msgr2=0x5617b830ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea55fc640 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea55fc640 1 -- 172.21.15.115:0/4190954127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 msgr2=0x5617b8308d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea55fc640 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8308d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea55fc640 1 -- 172.21.15.115:0/4190954127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b82111f0 con 0x5617b7f3a4d0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efea55fc640 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b8309250 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efe90000c00 tx=0x7efe90005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e639fec63ca2f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efe897fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efe897fa640 1 -- 172.21.15.115:0/4190954127 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90028070 con 0x5617b7f3a4d0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efe897fa640 1 -- 172.21.15.115:0/4190954127 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efe90003800 con 0x5617b7f3a4d0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efeaeffa740 1 -- 172.21.15.115:0/4190954127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe90009e10 con 0x5617b7f3a4d0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efe897fa640 1 -- 172.21.15.115:0/4190954127 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe90034050 con 0x5617b7f3a4d0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efeaeffa740 10 client.16353 fetch_fsmap learned FSMap version 32 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.524+0000 7efeaeffa740 1 -- 172.21.15.115:0/4190954127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5617b7fc6790 con 0x5617b7f3a4d0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.525+0000 7efe897fa640 1 -- 172.21.15.115:0/4190954127 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe90009e10 con 0x5617b7f3a4d0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.525+0000 7efe897fa640 1 -- 172.21.15.115:0/4190954127 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7efe900048c0 con 0x5617b7f3a4d0 2024-05-09T05:40:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.525+0000 7efeaeffa740 10 client.16353 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.525+0000 7efeaeffa740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.525+0000 7efeaeffa740 20 client.16353 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.525+0000 7efeaeffa740 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5617b7f19d80 0x5617b831ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.525+0000 7efeaeffa740 4 client.16353 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.525+0000 7efeaeffa740 1 -- 172.21.15.115:0/4190954127 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617b82713d0 con 0x5617b7f19d80 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.525+0000 7efea5dfd640 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5617b7f19d80 0x5617b831ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.526+0000 7efea5dfd640 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5617b7f19d80 0x5617b831ade0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5979a27f1b7be9c server_cookie=27a1c2cf3b72e8c9 in_seq=0 out_seq=0 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.526+0000 7efe897fa640 10 client.16353 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.526+0000 7efe897fa640 1 -- 172.21.15.115:0/4190954127 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5617b82713d0 con 0x5617b7f19d80 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.526+0000 7efe897fa640 10 client.16353 handle_command_reply: tid=0 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.526+0000 7efeaeffa740 1 client.16353 shutdown 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.526+0000 7efeaeffa740 1 -- 172.21.15.115:0/4190954127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 msgr2=0x5617b8309250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.526+0000 7efeaeffa740 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b8309250 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efe90000c00 tx=0x7efe90005d30 comp rx=0 tx=0).stop 2024-05-09T05:40:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.526+0000 7efeaeffa740 1 -- 172.21.15.115:0/4190954127 shutdown_connections 2024-05-09T05:40:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.527+0000 7efeaeffa740 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5617b7f19d80 0x5617b831ade0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.527+0000 7efeaeffa740 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.527+0000 7efeaeffa740 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b8309250 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.527+0000 7efe897fa640 0 client.16353 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.527+0000 7efeaeffa740 1 --2- 172.21.15.115:0/4190954127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8308d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.527+0000 7efeaeffa740 1 -- 172.21.15.115:0/4190954127 >> 172.21.15.115:0/4190954127 conn(0x5617b7f4f300 msgr2=0x5617b83003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.527+0000 7efeaeffa740 1 -- 172.21.15.115:0/4190954127 shutdown_connections 2024-05-09T05:40:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.527+0000 7efeaeffa740 1 -- 172.21.15.115:0/4190954127 wait complete. 2024-05-09T05:40:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.527+0000 7efeaeffa740 20 client.16353 trim_cache size 0 max 16384 2024-05-09T05:40:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efeaeffa740 1 Processor -- start 2024-05-09T05:40:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efeaeffa740 1 -- start start 2024-05-09T05:40:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8300a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f3a4d0 0x5617b83023e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f5b1b0 0x5617b8302920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efeaeffa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efea01b1090 con 0x5617b7f5b1b0 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efeaeffa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efea01b2c30 con 0x5617b7f322f0 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efeaeffa740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efea01b1760 con 0x5617b7f3a4d0 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efea65fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f5b1b0 0x5617b8302920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efea65fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f5b1b0 0x5617b8302920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53482/0 (socket says 172.21.15.115:53482) 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efea65fe640 1 -- 172.21.15.115:0/2945082354 learned_addr learned my addr 172.21.15.115:0/2945082354 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.542+0000 7efea55fc640 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f3a4d0 0x5617b83023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efea5dfd640 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8300a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efea55fc640 1 -- 172.21.15.115:0/2945082354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 msgr2=0x5617b8300a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efea55fc640 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8300a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efea55fc640 1 -- 172.21.15.115:0/2945082354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f5b1b0 msgr2=0x5617b8302920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efea55fc640 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f5b1b0 0x5617b8302920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efea55fc640 1 -- 172.21.15.115:0/2945082354 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b803d5f0 con 0x5617b7f3a4d0 2024-05-09T05:40:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efea65fe640 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f5b1b0 0x5617b8302920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efea55fc640 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f3a4d0 0x5617b83023e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe900099b0 tx=0x7efe90005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b76daf4b266b17ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efe8b7fe640 1 -- 172.21.15.115:0/2945082354 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90028040 con 0x5617b7f3a4d0 2024-05-09T05:40:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efe8b7fe640 1 -- 172.21.15.115:0/2945082354 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe90004640 con 0x5617b7f3a4d0 2024-05-09T05:40:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.543+0000 7efe8b7fe640 1 -- 172.21.15.115:0/2945082354 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90005260 con 0x5617b7f3a4d0 2024-05-09T05:40:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.544+0000 7efeaeffa740 1 -- 172.21.15.115:0/2945082354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f3a4d0 msgr2=0x5617b83023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.544+0000 7efeaeffa740 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f3a4d0 0x5617b83023e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe900099b0 tx=0x7efe90005b90 comp rx=0 tx=0).stop 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.544+0000 7efeaeffa740 1 -- 172.21.15.115:0/2945082354 shutdown_connections 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.544+0000 7efeaeffa740 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f5b1b0 0x5617b8302920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.544+0000 7efeaeffa740 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f3a4d0 0x5617b83023e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.544+0000 7efeaeffa740 1 --2- 172.21.15.115:0/2945082354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8300a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.544+0000 7efeaeffa740 1 -- 172.21.15.115:0/2945082354 >> 172.21.15.115:0/2945082354 conn(0x5617b7f4f300 msgr2=0x5617b8300360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.544+0000 7efeaeffa740 1 -- 172.21.15.115:0/2945082354 shutdown_connections 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.544+0000 7efeaeffa740 1 -- 172.21.15.115:0/2945082354 wait complete. 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efeaeffa740 1 -- start start 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efeaeffa740 1 Processor -- start 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8315280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b83157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efeaeffa740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efeaeffa740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efea01b2500 con 0x5617b7f3a4d0 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efeaeffa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efea01b1e30 con 0x5617b7f322f0 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efeaeffa740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efea01af070 con 0x5617b7f5b1b0 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efeaeffa740 10 client.0 _ll_register_callbacks cb 0x7efe9c0e66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efea55fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b83157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efea65fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efea5dfd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8315280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efea55fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b83157e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53492/0 (socket says 172.21.15.115:53492) 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efea55fc640 1 -- 172.21.15.115:0/3651580426 learned_addr learned my addr 172.21.15.115:0/3651580426 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.545+0000 7efea65fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:34148/0 (socket says 172.21.15.115:34148) 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efea55fc640 1 -- 172.21.15.115:0/3651580426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 msgr2=0x5617b830e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efea55fc640 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efea55fc640 1 -- 172.21.15.115:0/3651580426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 msgr2=0x5617b8315280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efea55fc640 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8315280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efea55fc640 1 -- 172.21.15.115:0/3651580426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b8227ed0 con 0x5617b7f3a4d0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efea65fe640 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efea5dfd640 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8315280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efea55fc640 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b83157e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efe90012fd0 tx=0x7efe90004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab017a1a2feb561d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efe897fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efe897fa640 1 -- 172.21.15.115:0/3651580426 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9003c070 con 0x5617b7f3a4d0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efe897fa640 1 -- 172.21.15.115:0/3651580426 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efe90003a10 con 0x5617b7f3a4d0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efeaeffa740 1 -- 172.21.15.115:0/3651580426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe90004640 con 0x5617b7f3a4d0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efe897fa640 1 -- 172.21.15.115:0/3651580426 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe9004b050 con 0x5617b7f3a4d0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efeaeffa740 10 client.16359 fetch_fsmap learned FSMap version 32 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efeaeffa740 1 -- 172.21.15.115:0/3651580426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5617b8313740 con 0x5617b7f3a4d0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.546+0000 7efe897fa640 1 -- 172.21.15.115:0/3651580426 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe90004640 con 0x5617b7f3a4d0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efe897fa640 1 -- 172.21.15.115:0/3651580426 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7efe90024620 con 0x5617b7f3a4d0 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efeaeffa740 10 client.16359 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efeaeffa740 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:40:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efeaeffa740 20 client.16359 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efeaeffa740 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5617b7eef1c0 0x5617b8385110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efeaeffa740 4 client.16359 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:40:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efeaeffa740 1 -- 172.21.15.115:0/3651580426 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617b826b240 con 0x5617b7eef1c0 2024-05-09T05:40:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efea5dfd640 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5617b7eef1c0 0x5617b8385110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efe897fa640 10 client.16359 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.547+0000 7efea5dfd640 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5617b7eef1c0 0x5617b8385110 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb555bdc54b13096 server_cookie=67a41331eb41d502 in_seq=0 out_seq=0 2024-05-09T05:40:51.592 INFO:tasks.workunit.client.0.smithi115.stdout:checking for lrand48_r... yes 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efe897fa640 1 -- 172.21.15.115:0/3651580426 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5617b826b240 con 0x5617b7eef1c0 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efe897fa640 10 client.16359 handle_command_reply: tid=0 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efeaeffa740 1 client.16359 shutdown 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efeaeffa740 1 -- 172.21.15.115:0/3651580426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 msgr2=0x5617b83157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efeaeffa740 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b83157e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efe90012fd0 tx=0x7efe90004a20 comp rx=0 tx=0).stop 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efeaeffa740 1 -- 172.21.15.115:0/3651580426 shutdown_connections 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efeaeffa740 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5617b7eef1c0 0x5617b8385110 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efe897fa640 0 client.16359 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efeaeffa740 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617b7f5b1b0 0x5617b830e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efeaeffa740 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5617b7f3a4d0 0x5617b83157e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efeaeffa740 1 --2- 172.21.15.115:0/3651580426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617b7f322f0 0x5617b8315280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.601+0000 7efeaeffa740 1 -- 172.21.15.115:0/3651580426 >> 172.21.15.115:0/3651580426 conn(0x5617b7f4f300 msgr2=0x5617b830a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efeaeffa740 1 -- 172.21.15.115:0/3651580426 shutdown_connections 2024-05-09T05:40:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efeaeffa740 1 -- 172.21.15.115:0/3651580426 wait complete. 2024-05-09T05:40:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efeaeffa740 20 client.16359 trim_cache size 0 max 16384 2024-05-09T05:40:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:40:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:40:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:40:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 -- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe8c0b2950 msgr2=0x7efe8c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe8c0b2950 0x7efe8c0b4e60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efe98006010 tx=0x7efe98009360 comp rx=0 tx=0).stop 2024-05-09T05:40:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 -- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 msgr2=0x7efea019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 0x7efea019b7f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00e9f0 tx=0x7efe9c00eeb0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 -- 172.21.15.115:0/3275881514 shutdown_connections 2024-05-09T05:40:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efe8c0b2950 0x7efe8c0b4e60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea0109260 0x7efea019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea0108dc0 0x7efea019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 --2- 172.21.15.115:0/3275881514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efea0103a30 0x7efea019b7f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 -- 172.21.15.115:0/3275881514 >> 172.21.15.115:0/3275881514 conn(0x7efea00779e0 msgr2=0x7efea00ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 -- 172.21.15.115:0/3275881514 shutdown_connections 2024-05-09T05:40:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:40:51.602+0000 7efea8061640 1 -- 172.21.15.115:0/3275881514 wait complete. 2024-05-09T05:40:51.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:51 smithi195 ceph-mon[33844]: pgmap v82: 129 pgs: 129 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 583 KiB/s wr, 197 op/s 2024-05-09T05:40:51.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:51 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:40:51.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:51 smithi115 ceph-mon[30349]: pgmap v82: 129 pgs: 129 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 583 KiB/s wr, 197 op/s 2024-05-09T05:40:51.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:51 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:40:51.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:51 smithi161 ceph-mon[33805]: pgmap v82: 129 pgs: 129 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 583 KiB/s wr, 197 op/s 2024-05-09T05:40:51.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:51 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:40:51.856 INFO:tasks.workunit.client.0.smithi115.stdout:checking for srand48_r... yes 2024-05-09T05:40:51.885 INFO:tasks.workunit.client.0.smithi115.stdout:configure: creating ./config.status 2024-05-09T05:40:52.010 INFO:tasks.workunit.client.0.smithi115.stdout: /bin/sh ./config.status 2024-05-09T05:40:52.078 INFO:tasks.workunit.client.0.smithi115.stdout:config.status: creating Makefile 2024-05-09T05:40:52.098 INFO:tasks.workunit.client.0.smithi115.stdout:config.status: creating config.h 2024-05-09T05:40:52.169 INFO:tasks.workunit.client.0.smithi115.stdout:config.status: executing depfiles commands 2024-05-09T05:40:52.205 INFO:tasks.workunit.client.0.smithi115.stdout:config.status: executing default commands 2024-05-09T05:40:52.263 INFO:tasks.workunit.client.0.smithi115.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-05-09T05:40:52.505 INFO:tasks.workunit.client.0.smithi115.stdout:rm -f stamp-h1 2024-05-09T05:40:52.508 INFO:tasks.workunit.client.0.smithi115.stdout:touch config.h.in 2024-05-09T05:40:52.510 INFO:tasks.workunit.client.0.smithi115.stdout:cd . && /bin/sh ./config.status config.h 2024-05-09T05:40:52.567 INFO:tasks.workunit.client.0.smithi115.stdout:config.status: creating config.h 2024-05-09T05:40:52.607 INFO:tasks.workunit.client.0.smithi115.stdout:make all-am 2024-05-09T05:40:52.632 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-09T05:40:52.632 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-05-09T05:40:52.880 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-05-09T05:40:52.891 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-05-09T05:40:53.023 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-05-09T05:40:53.036 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-05-09T05:40:53.176 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-05-09T05:40:53.190 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-05-09T05:40:53.309 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-05-09T05:40:53.322 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-05-09T05:40:53.435 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:53 smithi115 ceph-mon[30349]: pgmap v83: 129 pgs: 129 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 583 KiB/s wr, 197 op/s 2024-05-09T05:40:53.480 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-05-09T05:40:53.494 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-05-09T05:40:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.560+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3724732008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed0107df0 msgr2=0x7f3ed0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.560+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3724732008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed0107df0 0x7f3ed0108200 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40099c0 tx=0x7f3ec4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.560+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3724732008 shutdown_connections 2024-05-09T05:40:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.560+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3724732008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed01034c0 0x7f3ed01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.560+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3724732008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed0108740 0x7f3ed01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.560+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3724732008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed0107df0 0x7f3ed0108200 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.560+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3724732008 >> 172.21.15.115:0/3724732008 conn(0x7f3ed00fedc0 msgr2=0x7f3ed01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.560+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3724732008 shutdown_connections 2024-05-09T05:40:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.560+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3724732008 wait complete. 2024-05-09T05:40:53.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ed56f1700 1 Processor -- start 2024-05-09T05:40:53.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ed56f1700 1 -- start start 2024-05-09T05:40:53.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ed56f1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed01034c0 0x7f3ed019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ed56f1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed0107df0 0x7f3ed019d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ed56f1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed0108740 0x7f3ed01a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ed56f1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0113270 con 0x7f3ed0108740 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ed56f1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01130f0 con 0x7f3ed01034c0 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ed56f1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01133f0 con 0x7f3ed0107df0 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ece7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed0107df0 0x7f3ed019d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ecf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed0108740 0x7f3ed01a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ece7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed0107df0 0x7f3ed019d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:45596/0 (socket says 172.21.15.115:45596) 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3ece7fc700 1 -- 172.21.15.115:0/3614933404 learned_addr learned my addr 172.21.15.115:0/3614933404 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.561+0000 7f3eceffd700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed01034c0 0x7f3ed019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3ece7fc700 1 -- 172.21.15.115:0/3614933404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed01034c0 msgr2=0x7f3ed019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3ece7fc700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed01034c0 0x7f3ed019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3ece7fc700 1 -- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed0108740 msgr2=0x7f3ed01a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3ece7fc700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed0108740 0x7f3ed01a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3ece7fc700 1 -- 172.21.15.115:0/3614933404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed01a1d70 con 0x7f3ed0107df0 2024-05-09T05:40:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3ecf7fe700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed0108740 0x7f3ed01a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:40:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3ece7fc700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed0107df0 0x7f3ed019d4f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b580 tx=0x7f3ec400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3eb7fff700 1 -- 172.21.15.115:0/3614933404 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4027070 con 0x7f3ed0107df0 2024-05-09T05:40:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed01a2000 con 0x7f3ed0107df0 2024-05-09T05:40:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3eb7fff700 1 -- 172.21.15.115:0/3614933404 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ec4022470 con 0x7f3ed0107df0 2024-05-09T05:40:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3eb7fff700 1 -- 172.21.15.115:0/3614933404 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4017650 con 0x7f3ed0107df0 2024-05-09T05:40:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.562+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed01a24e0 con 0x7f3ed0107df0 2024-05-09T05:40:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.563+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed00512c0 con 0x7f3ed0107df0 2024-05-09T05:40:53.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.565+0000 7f3eb7fff700 1 -- 172.21.15.115:0/3614933404 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f3ec402a070 con 0x7f3ed0107df0 2024-05-09T05:40:53.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.565+0000 7f3eb7fff700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3eb80b29f0 0x7f3eb80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:53.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.566+0000 7f3eceffd700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3eb80b29f0 0x7f3eb80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:53.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.566+0000 7f3eb7fff700 1 -- 172.21.15.115:0/3614933404 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f3ec40e3e20 con 0x7f3ed0107df0 2024-05-09T05:40:53.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.566+0000 7f3eceffd700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3eb80b29f0 0x7f3eb80b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000ac50 tx=0x7f3ec000a380 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.569+0000 7f3eb7fff700 1 -- 172.21.15.115:0/3614933404 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ec4031020 con 0x7f3ed0107df0 2024-05-09T05:40:53.575 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-05-09T05:40:53.588 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-05-09T05:40:53.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:53 smithi195 ceph-mon[33844]: pgmap v83: 129 pgs: 129 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 583 KiB/s wr, 197 op/s 2024-05-09T05:40:53.651 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-05-09T05:40:53.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:53 smithi161 ceph-mon[33805]: pgmap v83: 129 pgs: 129 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 583 KiB/s wr, 197 op/s 2024-05-09T05:40:53.663 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-05-09T05:40:53.745 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-05-09T05:40:53.757 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-05-09T05:40:53.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.895+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed002e5f0 con 0x7f3ed0107df0 2024-05-09T05:40:53.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.896+0000 7f3eb7fff700 1 -- 172.21.15.115:0/3614933404 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11956 (secure 0 0 0) 0x7f3ec4013070 con 0x7f3ed0107df0 2024-05-09T05:40:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:40:53.890 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:40:49.275890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:40:53.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3eb80b29f0 msgr2=0x7f3eb80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:53.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3eb80b29f0 0x7f3eb80b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000ac50 tx=0x7f3ec000a380 comp rx=0 tx=0).stop 2024-05-09T05:40:53.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed0107df0 msgr2=0x7f3ed019d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:53.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed0107df0 0x7f3ed019d4f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b580 tx=0x7f3ec400f690 comp rx=0 tx=0).stop 2024-05-09T05:40:53.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 shutdown_connections 2024-05-09T05:40:53.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3eb80b29f0 0x7f3eb80b4f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed01034c0 0x7f3ed019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed0108740 0x7f3ed01a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 --2- 172.21.15.115:0/3614933404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed0107df0 0x7f3ed019d4f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:53.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.898+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 >> 172.21.15.115:0/3614933404 conn(0x7f3ed00fedc0 msgr2=0x7f3ed01097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:53.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.899+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 shutdown_connections 2024-05-09T05:40:53.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:53.899+0000 7f3ed56f1700 1 -- 172.21.15.115:0/3614933404 wait complete. 2024-05-09T05:40:53.893 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 32 2024-05-09T05:40:54.153 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-05-09T05:40:54.166 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-05-09T05:40:54.320 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-05-09T05:40:54.336 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-05-09T05:40:54.462 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:54 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3614933404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:54.527 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-05-09T05:40:54.539 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-05-09T05:40:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:54 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3614933404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:54.631 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-05-09T05:40:54.643 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-05-09T05:40:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:54 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3614933404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:40:54.742 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-05-09T05:40:54.754 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-05-09T05:40:54.894 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-05-09T05:40:54.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:40:54.907 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-05-09T05:40:54.953 INFO:tasks.workunit.client.0.smithi115.stderr:In file included from parser.h:22, 2024-05-09T05:40:54.954 INFO:tasks.workunit.client.0.smithi115.stderr: from parser.c:25: 2024-05-09T05:40:54.954 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c: In function ‘init_filesys’: 2024-05-09T05:40:54.955 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-09T05:40:54.955 INFO:tasks.workunit.client.0.smithi115.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-09T05:40:54.955 INFO:tasks.workunit.client.0.smithi115.stderr: | ^ 2024-05-09T05:40:54.955 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: 36 | container_of(ptr, type, member) 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr: | ^ 2024-05-09T05:40:54.956 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~ 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: 36 | container_of(ptr, type, member) 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~ 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~ 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-09T05:40:54.957 INFO:tasks.workunit.client.0.smithi115.stderr: | ^ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr: 36 | container_of(ptr, type, member) 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-09T05:40:54.958 INFO:tasks.workunit.client.0.smithi115.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: | ^ 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~ 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: 36 | container_of(ptr, type, member) 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~ 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~ 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c: In function ‘init_tg_stats’: 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-09T05:40:54.959 INFO:tasks.workunit.client.0.smithi115.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr: | ^ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr: 36 | container_of(ptr, type, member) 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:54.960 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: | ^ 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~ 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: 36 | container_of(ptr, type, member) 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~ 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~ 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-09T05:40:54.961 INFO:tasks.workunit.client.0.smithi115.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: | ^ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: 36 | container_of(ptr, type, member) 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-09T05:40:54.962 INFO:tasks.workunit.client.0.smithi115.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr: | ^ 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~ 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr: 36 | container_of(ptr, type, member) 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~ 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~ 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-09T05:40:54.963 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:55.014 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c: In function ‘get_optstr’: 2024-05-09T05:40:55.014 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-05-09T05:40:55.014 INFO:tasks.workunit.client.0.smithi115.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-05-09T05:40:55.014 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:55.014 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:200:14: note: source object declared here 2024-05-09T05:40:55.015 INFO:tasks.workunit.client.0.smithi115.stderr: 200 | char temp[BUFSIZE]; 2024-05-09T05:40:55.015 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~ 2024-05-09T05:40:55.021 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c: In function ‘size64_convert’: 2024-05-09T05:40:55.021 INFO:tasks.workunit.client.0.smithi115.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-05-09T05:40:55.021 INFO:tasks.workunit.client.0.smithi115.stderr: 119 | memcpy(unit, "\0", 3); 2024-05-09T05:40:55.021 INFO:tasks.workunit.client.0.smithi115.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-09T05:40:55.160 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:40:55.344 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-05-09T05:40:55.357 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-05-09T05:40:55.440 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:55 smithi115 ceph-mon[30349]: pgmap v84: 129 pgs: 129 active+clean; 6.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 797 KiB/s wr, 277 op/s 2024-05-09T05:40:55.450 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-05-09T05:40:55.463 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-05-09T05:40:55.584 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-05-09T05:40:55.598 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-05-09T05:40:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:55 smithi195 ceph-mon[33844]: pgmap v84: 129 pgs: 129 active+clean; 6.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 797 KiB/s wr, 277 op/s 2024-05-09T05:40:55.646 INFO:tasks.workunit.client.0.smithi115.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-05-09T05:40:55.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:55 smithi161 ceph-mon[33805]: pgmap v84: 129 pgs: 129 active+clean; 6.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 797 KiB/s wr, 277 op/s 2024-05-09T05:40:55.660 INFO:tasks.workunit.client.0.smithi115.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-05-09T05:40:55.777 INFO:tasks.workunit.client.0.smithi115.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-09T05:40:55.778 INFO:tasks.workunit.client.0.smithi115.stderr:+ cd .. 2024-05-09T05:40:55.779 INFO:tasks.workunit.client.0.smithi115.stderr:+ mkdir tmp 2024-05-09T05:40:55.782 INFO:tasks.workunit.client.0.smithi115.stderr:+ cd tmp 2024-05-09T05:40:55.783 INFO:tasks.workunit.client.0.smithi115.stderr:+ for f in $mydir/*.ffsb 2024-05-09T05:40:55.783 INFO:tasks.workunit.client.0.smithi115.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-05-09T05:40:55.784 INFO:tasks.workunit.client.0.smithi115.stdout:FFSB version 6.0-RC2 started 2024-05-09T05:40:55.784 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:40:55.787 INFO:tasks.workunit.client.0.smithi115.stdout:benchmark time = 300 2024-05-09T05:40:55.787 INFO:tasks.workunit.client.0.smithi115.stdout:ThreadGroup 0 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout:================ 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: num_threads = 32 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: read_random = off 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: read_size = 0 (0B) 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: read_blocksize = 0 (0B) 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: read_skip = off 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: read_skipsize = 0 (0B) 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: write_random = on 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: write_size = 5242880 (5MB) 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: fsync_file = 0 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: write_blocksize = 4096 (4KB) 2024-05-09T05:40:55.788 INFO:tasks.workunit.client.0.smithi115.stdout: wait time = 0 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: op weights 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: read = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: readall = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: write = 1 (100.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: create = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: append = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: delete = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: metaop = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: createdir = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: stat = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: writeall = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: writeall_fsync = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: open_close = 0 (0.00%) 2024-05-09T05:40:55.789 INFO:tasks.workunit.client.0.smithi115.stdout: write_fsync = 0 (0.00%) 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: create_fsync = 0 (0.00%) 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: append_fsync = 0 (0.00%) 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout:FileSystem . 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout:========== 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: num_dirs = 0 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: starting files = 128 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: min file size = 104857600 (100MB) 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: max file size = 104857600 (100MB) 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: directio = off 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: alignedio = on 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: bufferedio = off 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:40:55.790 INFO:tasks.workunit.client.0.smithi115.stdout: aging is off 2024-05-09T05:40:55.791 INFO:tasks.workunit.client.0.smithi115.stdout: current utilization = 0.04% 2024-05-09T05:40:55.791 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:40:55.791 INFO:tasks.workunit.client.0.smithi115.stdout:checking existing fs: . 2024-05-09T05:40:55.791 INFO:tasks.workunit.client.0.smithi115.stdout:recreating new fileset 2024-05-09T05:40:55.791 INFO:tasks.workunit.client.0.smithi115.stderr:opendir: No such file or directory 2024-05-09T05:40:57.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.131+0000 7fd4a36e1700 1 -- 172.21.15.115:0/2291599890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c10e970 msgr2=0x7fd49c110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:57.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.131+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/2291599890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c10e970 0x7fd49c110e10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd49400b0a0 tx=0x7fd49400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.131+0000 7fd4a36e1700 1 -- 172.21.15.115:0/2291599890 shutdown_connections 2024-05-09T05:40:57.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.131+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/2291599890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c10e970 0x7fd49c110e10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.131+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/2291599890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd49c105590 0x7fd49c1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.131+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/2291599890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd49c105ee0 0x7fd49c10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.131+0000 7fd4a36e1700 1 -- 172.21.15.115:0/2291599890 >> 172.21.15.115:0/2291599890 conn(0x7fd49c078e00 msgr2=0x7fd49c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:57.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.131+0000 7fd4a36e1700 1 -- 172.21.15.115:0/2291599890 shutdown_connections 2024-05-09T05:40:57.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.131+0000 7fd4a36e1700 1 -- 172.21.15.115:0/2291599890 wait complete. 2024-05-09T05:40:57.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.132+0000 7fd4a36e1700 1 Processor -- start 2024-05-09T05:40:57.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.132+0000 7fd4a36e1700 1 -- start start 2024-05-09T05:40:57.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.132+0000 7fd4a36e1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c105590 0x7fd49c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.132+0000 7fd4a36e1700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd49c105ee0 0x7fd49c1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.132+0000 7fd4a36e1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd49c10e970 0x7fd49c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.132+0000 7fd4a36e1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c114390 con 0x7fd49c105590 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.132+0000 7fd4a36e1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c114210 con 0x7fd49c10e970 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.132+0000 7fd4a36e1700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c114510 con 0x7fd49c105ee0 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a147d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c105590 0x7fd49c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a147d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c105590 0x7fd49c1a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37558/0 (socket says 172.21.15.115:37558) 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a147d700 1 -- 172.21.15.115:0/1389589845 learned_addr learned my addr 172.21.15.115:0/1389589845 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd49c105ee0 0x7fd49c1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd49c10e970 0x7fd49c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a147d700 1 -- 172.21.15.115:0/1389589845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd49c105ee0 msgr2=0x7fd49c1a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:57.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a147d700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd49c105ee0 0x7fd49c1a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a147d700 1 -- 172.21.15.115:0/1389589845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd49c10e970 msgr2=0x7fd49c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:57.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a147d700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd49c10e970 0x7fd49c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a147d700 1 -- 172.21.15.115:0/1389589845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c19b280 con 0x7fd49c105590 2024-05-09T05:40:57.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a147d700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c105590 0x7fd49c1a09c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e9a0 tx=0x7fd49800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:57.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4927fc700 1 -- 172.21.15.115:0/1389589845 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49800ca10 con 0x7fd49c105590 2024-05-09T05:40:57.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4927fc700 1 -- 172.21.15.115:0/1389589845 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd49800cbb0 con 0x7fd49c105590 2024-05-09T05:40:57.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4a36e1700 1 -- 172.21.15.115:0/1389589845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49c19b510 con 0x7fd49c105590 2024-05-09T05:40:57.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.133+0000 7fd4927fc700 1 -- 172.21.15.115:0/1389589845 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498018680 con 0x7fd49c105590 2024-05-09T05:40:57.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.134+0000 7fd4a36e1700 1 -- 172.21.15.115:0/1389589845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49c1031b0 con 0x7fd49c105590 2024-05-09T05:40:57.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.137+0000 7fd4927fc700 1 -- 172.21.15.115:0/1389589845 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fd498010430 con 0x7fd49c105590 2024-05-09T05:40:57.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4b20f7b80 1 Processor -- start 2024-05-09T05:40:57.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4b20f7b80 1 -- start start 2024-05-09T05:40:57.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4927fc700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4880b2a70 0x7fd4880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7df780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4880b2a70 0x7fd4880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b23b6fc0a0 con 0x55b23b6d6d00 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4927fc700 1 -- 172.21.15.115:0/1389589845 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd4980e2a50 con 0x7fd49c105590 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b23b6a19a0 con 0x55b23b65c4a0 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b23b6f8060 con 0x55b23b7d9400 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4a0c7c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4a0c7c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7df780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:34468/0 (socket says 172.21.15.115:34468) 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.138+0000 7fd4a0c7c700 1 -- 172.21.15.115:0/2931104308 learned_addr learned my addr 172.21.15.115:0/2931104308 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.139+0000 7fd4a147d700 1 --2- 172.21.15.115:0/2931104308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.139+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/2931104308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.139+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4880b2a70 0x7fd4880b4f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd494009e60 tx=0x7fd4940093a0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.139+0000 7fd4a0c7c700 1 -- 172.21.15.115:0/2931104308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 msgr2=0x55b23b7d9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.139+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/2931104308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7d9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.139+0000 7fd4a0c7c700 1 -- 172.21.15.115:0/2931104308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 msgr2=0x55b23b7df240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.139+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/2931104308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7df240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.139+0000 7fd4a0c7c700 1 -- 172.21.15.115:0/2931104308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b23b692500 con 0x55b23b65c4a0 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.139+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/2931104308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7df780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00bc70 tx=0x7fd48c00bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcbab7aed9fb187d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.140+0000 7fd4877fe700 1 -- 172.21.15.115:0/2931104308 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c010be0 con 0x55b23b65c4a0 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.140+0000 7fd4877fe700 1 -- 172.21.15.115:0/2931104308 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd48c0044c0 con 0x55b23b65c4a0 2024-05-09T05:40:57.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.140+0000 7fd4877fe700 1 -- 172.21.15.115:0/2931104308 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c003e50 con 0x55b23b65c4a0 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.141+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/2931104308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 msgr2=0x55b23b7df780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.141+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/2931104308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7df780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00bc70 tx=0x7fd48c00bf80 comp rx=0 tx=0).stop 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.141+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/2931104308 shutdown_connections 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.141+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/2931104308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7d9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.141+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/2931104308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.141+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/2931104308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7df780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.141+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/2931104308 >> 172.21.15.115:0/2931104308 conn(0x55b23b65ff70 msgr2=0x55b23b7d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.141+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/2931104308 shutdown_connections 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.141+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/2931104308 wait complete. 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.142+0000 7fd4b20f7b80 1 -- start start 2024-05-09T05:40:57.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.142+0000 7fd4b20f7b80 1 Processor -- start 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.142+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7e3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.142+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.142+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.142+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b23b7da160 con 0x55b23b6d6d00 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.142+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b23b696b20 con 0x55b23b65c4a0 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.142+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b23b7e6020 con 0x55b23b7d9400 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.142+0000 7fd4b20f7b80 10 client.0 _ll_register_callbacks cb 0x55b23b56d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a147d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a1c7e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a147d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7e3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:34472/0 (socket says 172.21.15.115:34472) 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a147d700 1 -- 172.21.15.115:0/1598485153 learned_addr learned my addr 172.21.15.115:0/1598485153 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a1c7e700 1 -- 172.21.15.115:0/1598485153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 msgr2=0x55b23b7e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a1c7e700 1 -- 172.21.15.115:0/1598485153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 msgr2=0x55b23b7e3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a1c7e700 1 -- 172.21.15.115:0/1598485153 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b23b577490 con 0x55b23b7d9400 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.143+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.144+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7e3aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd494009fd0 tx=0x7fd49400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecaf729a325fb36e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.144+0000 7fd484ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.144+0000 7fd484ff9700 1 -- 172.21.15.115:0/1598485153 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49402d070 con 0x55b23b7d9400 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.144+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1598485153 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd48c0044c0 con 0x55b23b7d9400 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.144+0000 7fd484ff9700 1 -- 172.21.15.115:0/1598485153 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd4940050c0 con 0x55b23b7d9400 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.144+0000 7fd484ff9700 1 -- 172.21.15.115:0/1598485153 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd494039050 con 0x55b23b7d9400 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.144+0000 7fd4b20f7b80 10 client.25933 fetch_fsmap learned FSMap version 32 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.144+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1598485153 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b23b7f0a40 con 0x55b23b7d9400 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.145+0000 7fd484ff9700 1 -- 172.21.15.115:0/1598485153 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd48c0044c0 con 0x55b23b7d9400 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.145+0000 7fd484ff9700 1 -- 172.21.15.115:0/1598485153 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd494011070 con 0x55b23b7d9400 2024-05-09T05:40:57.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.145+0000 7fd4b20f7b80 10 client.25933 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:40:57.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.145+0000 7fd4b20f7b80 10 client.25933 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:40:57.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.145+0000 7fd4b20f7b80 20 client.25933 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:57.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.145+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b23b7f2fd0 0x55b23b7f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.145+0000 7fd4b20f7b80 4 client.25933 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:40:57.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.145+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1598485153 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b23b709a80 con 0x55b23b7f2fd0 2024-05-09T05:40:57.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.145+0000 7fd4a147d700 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b23b7f2fd0 0x55b23b7f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.146+0000 7fd484ff9700 10 client.25933 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:40:57.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.146+0000 7fd4a147d700 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b23b7f2fd0 0x55b23b7f5410 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5dccac8ed9a306d server_cookie=4cfd991286a40c6b in_seq=0 out_seq=0 2024-05-09T05:40:57.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd484ff9700 1 -- 172.21.15.115:0/1598485153 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b23b709a80 con 0x55b23b7f2fd0 2024-05-09T05:40:57.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd484ff9700 10 client.25933 handle_command_reply: tid=0 2024-05-09T05:40:57.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd4b20f7b80 1 client.25933 shutdown 2024-05-09T05:40:57.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1598485153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 msgr2=0x55b23b7e3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:57.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7e3aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd494009fd0 tx=0x7fd49400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1598485153 shutdown_connections 2024-05-09T05:40:57.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b23b7f2fd0 0x55b23b7f5410 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7e3aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1598485153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd484ff9700 0 client.25933 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:40:57.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.147+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1598485153 >> 172.21.15.115:0/1598485153 conn(0x55b23b65ff70 msgr2=0x55b23b7cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:57.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.148+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1598485153 shutdown_connections 2024-05-09T05:40:57.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.148+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1598485153 wait complete. 2024-05-09T05:40:57.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.148+0000 7fd4b20f7b80 20 client.25933 trim_cache size 0 max 16384 2024-05-09T05:40:57.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.165+0000 7fd4b20f7b80 1 Processor -- start 2024-05-09T05:40:57.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.165+0000 7fd4b20f7b80 1 -- start start 2024-05-09T05:40:57.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.165+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.165+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7de7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b2560 con 0x55b23b6d6d00 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b4100 con 0x55b23b65c4a0 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b2c30 con 0x55b23b7d9400 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a0c7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a0c7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37588/0 (socket says 172.21.15.115:37588) 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a0c7c700 1 -- 172.21.15.115:0/3027687662 learned_addr learned my addr 172.21.15.115:0/3027687662 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a147d700 1 --2- 172.21.15.115:0/3027687662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/3027687662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7de7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a0c7c700 1 -- 172.21.15.115:0/3027687662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 msgr2=0x55b23b7de7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/3027687662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7de7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a0c7c700 1 -- 172.21.15.115:0/3027687662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 msgr2=0x55b23b7e2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:57.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/3027687662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7e2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a0c7c700 1 -- 172.21.15.115:0/3027687662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b23b7dfaf0 con 0x55b23b6d6d00 2024-05-09T05:40:57.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.166+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/3027687662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e54c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00b4f0 tx=0x7fd48c010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3940346a679d1dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:57.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.167+0000 7fd486ffd700 1 -- 172.21.15.115:0/3027687662 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c010c80 con 0x55b23b6d6d00 2024-05-09T05:40:57.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.167+0000 7fd486ffd700 1 -- 172.21.15.115:0/3027687662 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd48c010e20 con 0x55b23b6d6d00 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.167+0000 7fd486ffd700 1 -- 172.21.15.115:0/3027687662 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c004cf0 con 0x55b23b6d6d00 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/3027687662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 msgr2=0x55b23b7e54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/3027687662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e54c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00b4f0 tx=0x7fd48c010b90 comp rx=0 tx=0).stop 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/3027687662 shutdown_connections 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/3027687662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/3027687662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b7d9400 0x55b23b7de7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/3027687662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b6d6d00 0x55b23b7e54c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/3027687662 >> 172.21.15.115:0/3027687662 conn(0x55b23b65ff70 msgr2=0x55b23b7cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/3027687662 shutdown_connections 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/3027687662 wait complete. 2024-05-09T05:40:57.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 -- start start 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.168+0000 7fd4b20f7b80 1 Processor -- start 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b6d6d00 0x55b23b7eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4b20f7b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b7d9400 0x55b23b7f9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b39d0 con 0x55b23b7d9400 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b3300 con 0x55b23b65c4a0 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4b20f7b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b0520 con 0x55b23b6d6d00 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4b20f7b80 10 client.0 _ll_register_callbacks cb 0x55b23b7f5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a1c7e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b7d9400 0x55b23b7f9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a1c7e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b7d9400 0x55b23b7f9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37596/0 (socket says 172.21.15.115:37596) 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a147d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a1c7e700 1 -- 172.21.15.115:0/1659924009 learned_addr learned my addr 172.21.15.115:0/1659924009 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a0c7c700 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b6d6d00 0x55b23b7eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a1c7e700 1 -- 172.21.15.115:0/1659924009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b6d6d00 msgr2=0x55b23b7eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b6d6d00 0x55b23b7eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a1c7e700 1 -- 172.21.15.115:0/1659924009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 msgr2=0x55b23b7eb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:40:57.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7eb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.169+0000 7fd4a1c7e700 1 -- 172.21.15.115:0/1659924009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b23b7fa100 con 0x55b23b7d9400 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.170+0000 7fd4a1c7e700 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b7d9400 0x55b23b7f9bc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd49401e510 tx=0x7fd49401e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b034b4e167ce6f9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.170+0000 7fd484ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.170+0000 7fd484ff9700 1 -- 172.21.15.115:0/1659924009 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494036070 con 0x55b23b7d9400 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.170+0000 7fd484ff9700 1 -- 172.21.15.115:0/1659924009 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd49401eb80 con 0x55b23b7d9400 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.170+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1659924009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd48c010e20 con 0x55b23b7d9400 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.170+0000 7fd484ff9700 1 -- 172.21.15.115:0/1659924009 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd49402d070 con 0x55b23b7d9400 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.170+0000 7fd4b20f7b80 10 client.16380 fetch_fsmap learned FSMap version 32 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.170+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1659924009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b23b85cec0 con 0x55b23b7d9400 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.170+0000 7fd484ff9700 1 -- 172.21.15.115:0/1659924009 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd48c010e20 con 0x55b23b7d9400 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd484ff9700 1 -- 172.21.15.115:0/1659924009 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd494011040 con 0x55b23b7d9400 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd4b20f7b80 10 client.16380 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd4b20f7b80 10 client.16380 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd4b20f7b80 20 client.16380 populate_metadata read hostname 'smithi115' 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b23b7f2fd0 0x55b23b85f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:40:57.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd4b20f7b80 4 client.16380 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1659924009 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b23b703b20 con 0x55b23b7f2fd0 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd4a147d700 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b23b7f2fd0 0x55b23b85f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd484ff9700 10 client.16380 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.171+0000 7fd4a147d700 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b23b7f2fd0 0x55b23b85f600 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1472d504ac34c16 server_cookie=92684b15b093933 in_seq=0 out_seq=0 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd484ff9700 1 -- 172.21.15.115:0/1659924009 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b23b703b20 con 0x55b23b7f2fd0 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd484ff9700 10 client.16380 handle_command_reply: tid=0 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 client.16380 shutdown 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1659924009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b7d9400 msgr2=0x55b23b7f9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b7d9400 0x55b23b7f9bc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd49401e510 tx=0x7fd49401e910 comp rx=0 tx=0).stop 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1659924009 shutdown_connections 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b23b65c4a0 0x55b23b7eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55b23b7f2fd0 0x55b23b85f600 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd484ff9700 0 client.16380 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b23b7d9400 0x55b23b7f9bc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 --2- 172.21.15.115:0/1659924009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b23b6d6d00 0x55b23b7eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1659924009 >> 172.21.15.115:0/1659924009 conn(0x55b23b65ff70 msgr2=0x55b23b7cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:57.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1659924009 shutdown_connections 2024-05-09T05:40:57.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.172+0000 7fd4b20f7b80 1 -- 172.21.15.115:0/1659924009 wait complete. 2024-05-09T05:40:57.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4b20f7b80 20 client.16380 trim_cache size 0 max 16384 2024-05-09T05:40:57.171 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:40:57.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 -- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4880b2a70 msgr2=0x7fd4880b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:57.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4880b2a70 0x7fd4880b4f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd494009e60 tx=0x7fd4940093a0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 -- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c105590 msgr2=0x7fd49c1a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:40:57.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c105590 0x7fd49c1a09c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e9a0 tx=0x7fd49800ecb0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 -- 172.21.15.115:0/1389589845 shutdown_connections 2024-05-09T05:40:57.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4880b2a70 0x7fd4880b4f80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd49c10e970 0x7fd49c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd49c105590 0x7fd49c1a09c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 --2- 172.21.15.115:0/1389589845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd49c105ee0 0x7fd49c1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:40:57.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 -- 172.21.15.115:0/1389589845 >> 172.21.15.115:0/1389589845 conn(0x7fd49c078e00 msgr2=0x7fd49c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:40:57.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.173+0000 7fd4a36e1700 1 -- 172.21.15.115:0/1389589845 shutdown_connections 2024-05-09T05:40:57.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:40:57.174+0000 7fd4a36e1700 1 -- 172.21.15.115:0/1389589845 wait complete. 2024-05-09T05:40:57.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:57 smithi115 ceph-mon[30349]: pgmap v85: 129 pgs: 129 active+clean; 6.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 486 KiB/s wr, 154 op/s 2024-05-09T05:40:57.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:57 smithi195 ceph-mon[33844]: pgmap v85: 129 pgs: 129 active+clean; 6.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 486 KiB/s wr, 154 op/s 2024-05-09T05:40:57.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:57 smithi161 ceph-mon[33805]: pgmap v85: 129 pgs: 129 active+clean; 6.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 486 KiB/s wr, 154 op/s 2024-05-09T05:40:59.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:40:59 smithi195 ceph-mon[33844]: pgmap v86: 129 pgs: 129 active+clean; 59 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 4.9 MiB/s wr, 184 op/s 2024-05-09T05:40:59.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:40:59 smithi115 ceph-mon[30349]: pgmap v86: 129 pgs: 129 active+clean; 59 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 4.9 MiB/s wr, 184 op/s 2024-05-09T05:40:59.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:40:59 smithi161 ceph-mon[33805]: pgmap v86: 129 pgs: 129 active+clean; 59 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 4.9 MiB/s wr, 184 op/s 2024-05-09T05:40:59.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:40:59.970 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:01.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:01 smithi115 ceph-mon[30349]: pgmap v87: 129 pgs: 129 active+clean; 59 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.7 MiB/s wr, 109 op/s 2024-05-09T05:41:01.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:01 smithi195 ceph-mon[33844]: pgmap v87: 129 pgs: 129 active+clean; 59 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.7 MiB/s wr, 109 op/s 2024-05-09T05:41:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:01 smithi161 ceph-mon[33805]: pgmap v87: 129 pgs: 129 active+clean; 59 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.7 MiB/s wr, 109 op/s 2024-05-09T05:41:01.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.940+0000 7fad9514c700 1 -- 172.21.15.115:0/3526123035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90109850 msgr2=0x7fad901111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:01.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.940+0000 7fad9514c700 1 --2- 172.21.15.115:0/3526123035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90109850 0x7fad901111b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fad8800b0d0 tx=0x7fad8800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.940+0000 7fad9514c700 1 -- 172.21.15.115:0/3526123035 shutdown_connections 2024-05-09T05:41:01.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.940+0000 7fad9514c700 1 --2- 172.21.15.115:0/3526123035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90109850 0x7fad901111b0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.940+0000 7fad9514c700 1 --2- 172.21.15.115:0/3526123035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad90102b50 0x7fad90102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.940+0000 7fad9514c700 1 --2- 172.21.15.115:0/3526123035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad90108f00 0x7fad90109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.940+0000 7fad9514c700 1 -- 172.21.15.115:0/3526123035 >> 172.21.15.115:0/3526123035 conn(0x7fad900fe6f0 msgr2=0x7fad90100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:01.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.940+0000 7fad9514c700 1 -- 172.21.15.115:0/3526123035 shutdown_connections 2024-05-09T05:41:01.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.940+0000 7fad9514c700 1 -- 172.21.15.115:0/3526123035 wait complete. 2024-05-09T05:41:01.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad9514c700 1 Processor -- start 2024-05-09T05:41:01.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad9514c700 1 -- start start 2024-05-09T05:41:01.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad9514c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad90102b50 0x7fad90103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad9514c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90108f00 0x7fad90103fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad9514c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad90109850 0x7fad90104520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad9514c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fad90114330 con 0x7fad90108f00 2024-05-09T05:41:01.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad9514c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fad901141b0 con 0x7fad90109850 2024-05-09T05:41:01.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad9514c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fad901144b0 con 0x7fad90102b50 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad8e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90108f00 0x7fad90103fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad8e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90108f00 0x7fad90103fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37626/0 (socket says 172.21.15.115:37626) 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad8e59c700 1 -- 172.21.15.115:0/3492718078 learned_addr learned my addr 172.21.15.115:0/3492718078 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad90102b50 0x7fad90103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.941+0000 7fad8f59e700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad90109850 0x7fad90104520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad8e59c700 1 -- 172.21.15.115:0/3492718078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad90102b50 msgr2=0x7fad90103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad8e59c700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad90102b50 0x7fad90103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad8e59c700 1 -- 172.21.15.115:0/3492718078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad90109850 msgr2=0x7fad90104520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad8e59c700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad90109850 0x7fad90104520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad8e59c700 1 -- 172.21.15.115:0/3492718078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad901a9100 con 0x7fad90108f00 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad8e59c700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90108f00 0x7fad90103fe0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fad8400d950 tx=0x7fad8400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad77fff700 1 -- 172.21.15.115:0/3492718078 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84016070 con 0x7fad90108f00 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad77fff700 1 -- 172.21.15.115:0/3492718078 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad84004830 con 0x7fad90108f00 2024-05-09T05:41:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad9514c700 1 -- 172.21.15.115:0/3492718078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad901a93f0 con 0x7fad90108f00 2024-05-09T05:41:01.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad77fff700 1 -- 172.21.15.115:0/3492718078 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84005410 con 0x7fad90108f00 2024-05-09T05:41:01.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.942+0000 7fad9514c700 1 -- 172.21.15.115:0/3492718078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad901a9930 con 0x7fad90108f00 2024-05-09T05:41:01.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.945+0000 7fad77fff700 1 -- 172.21.15.115:0/3492718078 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fad840055f0 con 0x7fad90108f00 2024-05-09T05:41:01.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.946+0000 7fada3b62b80 1 Processor -- start 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.946+0000 7fada3b62b80 1 -- start start 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.946+0000 7fad77fff700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fad780b29f0 0x7fad780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.946+0000 7fad8f59e700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fad780b29f0 0x7fad780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.946+0000 7fad77fff700 1 -- 172.21.15.115:0/3492718078 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fad840e2630 con 0x7fad90108f00 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.946+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0175d00 0x55aca027e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.946+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 0x55aca027e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.946+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0278810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fada3b62b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55aca019b0a0 con 0x55aca0278400 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fada3b62b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aca01409a0 con 0x55aca00fb4a0 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fada3b62b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aca0197060 con 0x55aca0175d00 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8ed9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0175d00 0x55aca027e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8ed9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0175d00 0x55aca027e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:45676/0 (socket says 172.21.15.115:45676) 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8ed9d700 1 -- 172.21.15.115:0/160108995 learned_addr learned my addr 172.21.15.115:0/160108995 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8e59c700 1 --2- 172.21.15.115:0/160108995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 0x55aca027e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8f59e700 1 --2- 172.21.15.115:0/160108995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0278810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8ed9d700 1 -- 172.21.15.115:0/160108995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 msgr2=0x55aca027e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/160108995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 0x55aca027e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8ed9d700 1 -- 172.21.15.115:0/160108995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 msgr2=0x55aca0278810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/160108995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0278810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8ed9d700 1 -- 172.21.15.115:0/160108995 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca0131500 con 0x55aca0175d00 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8f59e700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fad780b29f0 0x7fad780b4f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad88009fd0 tx=0x7fad88009c30 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.947+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/160108995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0175d00 0x55aca027e240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad80007cf0 tx=0x7fad800103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b708433ba2cdaaa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.948+0000 7fad757fa700 1 -- 172.21.15.115:0/160108995 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80010d10 con 0x55aca0175d00 2024-05-09T05:41:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.948+0000 7fad757fa700 1 -- 172.21.15.115:0/160108995 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad800118d0 con 0x55aca0175d00 2024-05-09T05:41:01.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.948+0000 7fad757fa700 1 -- 172.21.15.115:0/160108995 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad800196c0 con 0x55aca0175d00 2024-05-09T05:41:01.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 -- 172.21.15.115:0/160108995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0175d00 msgr2=0x55aca027e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:01.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 --2- 172.21.15.115:0/160108995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0175d00 0x55aca027e240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad80007cf0 tx=0x7fad800103d0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 -- 172.21.15.115:0/160108995 shutdown_connections 2024-05-09T05:41:01.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 --2- 172.21.15.115:0/160108995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0278810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 --2- 172.21.15.115:0/160108995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0175d00 0x55aca027e240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 --2- 172.21.15.115:0/160108995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 0x55aca027e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 -- 172.21.15.115:0/160108995 >> 172.21.15.115:0/160108995 conn(0x55aca00fef70 msgr2=0x55aca0273220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:01.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 -- 172.21.15.115:0/160108995 shutdown_connections 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 -- 172.21.15.115:0/160108995 wait complete. 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 -- start start 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.949+0000 7fada3b62b80 1 Processor -- start 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 0x55aca0281f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca02824a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca02829e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fada3b62b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55aca0279160 con 0x55aca0278400 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fada3b62b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aca0135b20 con 0x55aca0175d00 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fada3b62b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aca0285020 con 0x55aca00fb4a0 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca02824a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca02824a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:34526/0 (socket says 172.21.15.115:34526) 2024-05-09T05:41:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8e59c700 1 -- 172.21.15.115:0/4241862936 learned_addr learned my addr 172.21.15.115:0/4241862936 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8f59e700 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca02829e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 0x55aca0281f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fada3b62b80 10 client.0 _ll_register_callbacks cb 0x55aca000c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8e59c700 1 -- 172.21.15.115:0/4241862936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 msgr2=0x55aca0281f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8e59c700 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 0x55aca0281f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8e59c700 1 -- 172.21.15.115:0/4241862936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 msgr2=0x55aca02829e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8e59c700 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca02829e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.950+0000 7fad8e59c700 1 -- 172.21.15.115:0/4241862936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca0016490 con 0x55aca0175d00 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fad8e59c700 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca02824a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad840dfaa0 tx=0x7fad84004c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=420b065ebb83a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fad72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:41:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fad72ff5700 1 -- 172.21.15.115:0/4241862936 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840e2480 con 0x55aca0175d00 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fada3b62b80 1 -- 172.21.15.115:0/4241862936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad800118d0 con 0x55aca0175d00 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fad72ff5700 1 -- 172.21.15.115:0/4241862936 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fad840910d0 con 0x55aca0175d00 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fad72ff5700 1 -- 172.21.15.115:0/4241862936 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad840b2430 con 0x55aca0175d00 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fada3b62b80 10 client.25886 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fada3b62b80 1 -- 172.21.15.115:0/4241862936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aca028f8f0 con 0x55aca0175d00 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fad72ff5700 1 -- 172.21.15.115:0/4241862936 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad800118d0 con 0x55aca0175d00 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fad72ff5700 1 -- 172.21.15.115:0/4241862936 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fad840b2010 con 0x55aca0175d00 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fada3b62b80 10 client.25886 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fada3b62b80 10 client.25886 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fada3b62b80 20 client.25886 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fada3b62b80 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55aca0291f80 0x55aca02943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fada3b62b80 4 client.25886 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:01.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.951+0000 7fada3b62b80 1 -- 172.21.15.115:0/4241862936 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aca01a8a80 con 0x55aca0291f80 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.952+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55aca0291f80 0x55aca02943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.952+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55aca0291f80 0x55aca02943c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d9e84bdadc20625 server_cookie=d6e4af42a3cd1da3 in_seq=0 out_seq=0 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.952+0000 7fad72ff5700 10 client.25886 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fad72ff5700 1 -- 172.21.15.115:0/4241862936 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aca01a8a80 con 0x55aca0291f80 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fad72ff5700 10 client.25886 handle_command_reply: tid=0 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 client.25886 shutdown 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 -- 172.21.15.115:0/4241862936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 msgr2=0x55aca02824a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca02824a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad840dfaa0 tx=0x7fad84004c10 comp rx=0 tx=0).stop 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 -- 172.21.15.115:0/4241862936 shutdown_connections 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55aca0291f80 0x55aca02943c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca02829e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 0x55aca0281f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 --2- 172.21.15.115:0/4241862936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca02824a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 -- 172.21.15.115:0/4241862936 >> 172.21.15.115:0/4241862936 conn(0x55aca00fef70 msgr2=0x55aca026cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:01.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fad72ff5700 0 client.25886 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:41:01.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 -- 172.21.15.115:0/4241862936 shutdown_connections 2024-05-09T05:41:01.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 1 -- 172.21.15.115:0/4241862936 wait complete. 2024-05-09T05:41:01.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.953+0000 7fada3b62b80 20 client.25886 trim_cache size 0 max 16384 2024-05-09T05:41:01.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fada3b62b80 1 Processor -- start 2024-05-09T05:41:01.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fada3b62b80 1 -- start start 2024-05-09T05:41:01.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 0x55aca0280ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0175d00 0x55aca027d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0278400 0x55aca0282820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fada3b62b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b25f0 con 0x55aca0175d00 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fada3b62b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b4190 con 0x55aca00fb4a0 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fada3b62b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b2cc0 con 0x55aca0278400 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fad8e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0175d00 0x55aca027d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fad8e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0175d00 0x55aca027d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37668/0 (socket says 172.21.15.115:37668) 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fad8e59c700 1 -- 172.21.15.115:0/374118191 learned_addr learned my addr 172.21.15.115:0/374118191 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fad8f59e700 1 --2- 172.21.15.115:0/374118191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0278400 0x55aca0282820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.969+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/374118191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 0x55aca0280ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.970+0000 7fad8e59c700 1 -- 172.21.15.115:0/374118191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0278400 msgr2=0x55aca0282820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.970+0000 7fad8e59c700 1 --2- 172.21.15.115:0/374118191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0278400 0x55aca0282820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.970+0000 7fad8e59c700 1 -- 172.21.15.115:0/374118191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 msgr2=0x55aca0280ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.970+0000 7fad8e59c700 1 --2- 172.21.15.115:0/374118191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 0x55aca0280ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.970+0000 7fad8e59c700 1 -- 172.21.15.115:0/374118191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca027eaf0 con 0x55aca0175d00 2024-05-09T05:41:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.970+0000 7fad8e59c700 1 --2- 172.21.15.115:0/374118191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0175d00 0x55aca027d7d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fad840a5650 tx=0x7fad84004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9319bffcb2e97b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.970+0000 7fad74ff9700 1 -- 172.21.15.115:0/374118191 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840910d0 con 0x55aca0175d00 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.970+0000 7fad74ff9700 1 -- 172.21.15.115:0/374118191 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad840a89f0 con 0x55aca0175d00 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.970+0000 7fad74ff9700 1 -- 172.21.15.115:0/374118191 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840b1fa0 con 0x55aca0175d00 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.971+0000 7fada3b62b80 1 -- 172.21.15.115:0/374118191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0175d00 msgr2=0x55aca027d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.971+0000 7fada3b62b80 1 --2- 172.21.15.115:0/374118191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0175d00 0x55aca027d7d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fad840a5650 tx=0x7fad84004f70 comp rx=0 tx=0).stop 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.971+0000 7fada3b62b80 1 -- 172.21.15.115:0/374118191 shutdown_connections 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.971+0000 7fada3b62b80 1 --2- 172.21.15.115:0/374118191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca00fb4a0 0x55aca0280ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.971+0000 7fada3b62b80 1 --2- 172.21.15.115:0/374118191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca0278400 0x55aca0282820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.971+0000 7fada3b62b80 1 --2- 172.21.15.115:0/374118191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0175d00 0x55aca027d7d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.971+0000 7fada3b62b80 1 -- 172.21.15.115:0/374118191 >> 172.21.15.115:0/374118191 conn(0x55aca00fef70 msgr2=0x55aca027d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.971+0000 7fada3b62b80 1 -- 172.21.15.115:0/374118191 shutdown_connections 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.971+0000 7fada3b62b80 1 -- 172.21.15.115:0/374118191 wait complete. 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fada3b62b80 1 -- start start 2024-05-09T05:41:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fada3b62b80 1 Processor -- start 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 0x55aca028a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca028a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fada3b62b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0298b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fada3b62b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b3a60 con 0x55aca0278400 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fada3b62b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b3390 con 0x55aca0175d00 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fada3b62b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fad901b0590 con 0x55aca00fb4a0 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fada3b62b80 10 client.0 _ll_register_callbacks cb 0x55aca000c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0298b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0298b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37672/0 (socket says 172.21.15.115:37672) 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8f59e700 1 -- 172.21.15.115:0/622347500 learned_addr learned my addr 172.21.15.115:0/622347500 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8e59c700 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca028a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 0x55aca028a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8e59c700 1 -- 172.21.15.115:0/622347500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 msgr2=0x55aca028a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8e59c700 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 0x55aca028a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8e59c700 1 -- 172.21.15.115:0/622347500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 msgr2=0x55aca0298b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8e59c700 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0298b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.972+0000 7fad8e59c700 1 -- 172.21.15.115:0/622347500 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca029a4e0 con 0x55aca0175d00 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fad8f59e700 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0298b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fad8e59c700 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca028a670 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad8401afd0 tx=0x7fad840074f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54cb7cef23c1c7a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fad72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fad72ff5700 1 -- 172.21.15.115:0/622347500 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840c7070 con 0x55aca0175d00 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fad72ff5700 1 -- 172.21.15.115:0/622347500 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fad840a9a40 con 0x55aca0175d00 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fada3b62b80 1 -- 172.21.15.115:0/622347500 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad840a89f0 con 0x55aca0175d00 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fad72ff5700 1 -- 172.21.15.115:0/622347500 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad840e2480 con 0x55aca0175d00 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fada3b62b80 10 client.25892 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fada3b62b80 1 -- 172.21.15.115:0/622347500 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aca02fbc10 con 0x55aca0175d00 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fad72ff5700 1 -- 172.21.15.115:0/622347500 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad840a89f0 con 0x55aca0175d00 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fad72ff5700 1 -- 172.21.15.115:0/622347500 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fad840ca020 con 0x55aca0175d00 2024-05-09T05:41:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fada3b62b80 10 client.25892 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fada3b62b80 10 client.25892 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.973+0000 7fada3b62b80 20 client.25892 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.974+0000 7fada3b62b80 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55aca02fc0c0 0x55aca02fe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.974+0000 7fada3b62b80 4 client.25892 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.974+0000 7fada3b62b80 1 -- 172.21.15.115:0/622347500 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aca01a2b20 con 0x55aca02fc0c0 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.974+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55aca02fc0c0 0x55aca02fe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.974+0000 7fad72ff5700 10 client.25892 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.974+0000 7fad8ed9d700 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55aca02fc0c0 0x55aca02fe4e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=630a9094090b349e server_cookie=b1e06b7341dded89 in_seq=0 out_seq=0 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fad72ff5700 1 -- 172.21.15.115:0/622347500 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aca01a2b20 con 0x55aca02fc0c0 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fad72ff5700 10 client.25892 handle_command_reply: tid=0 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 client.25892 shutdown 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 -- 172.21.15.115:0/622347500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 msgr2=0x55aca028a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca028a670 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad8401afd0 tx=0x7fad840074f0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 -- 172.21.15.115:0/622347500 shutdown_connections 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55aca02fc0c0 0x55aca02fe4e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55aca0278400 0x55aca0298b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aca00fb4a0 0x55aca028a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fad72ff5700 0 client.25892 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 --2- 172.21.15.115:0/622347500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aca0175d00 0x55aca028a670 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 -- 172.21.15.115:0/622347500 >> 172.21.15.115:0/622347500 conn(0x55aca00fef70 msgr2=0x55aca027bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 -- 172.21.15.115:0/622347500 shutdown_connections 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 1 -- 172.21.15.115:0/622347500 wait complete. 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.975+0000 7fada3b62b80 20 client.25892 trim_cache size 0 max 16384 2024-05-09T05:41:01.976 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:41:01.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 -- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fad780b29f0 msgr2=0x7fad780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fad780b29f0 0x7fad780b4f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad88009fd0 tx=0x7fad88009c30 comp rx=0 tx=0).stop 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 -- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90108f00 msgr2=0x7fad90103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90108f00 0x7fad90103fe0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fad8400d950 tx=0x7fad8400dc60 comp rx=0 tx=0).stop 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 -- 172.21.15.115:0/3492718078 shutdown_connections 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fad780b29f0 0x7fad780b4f00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad90109850 0x7fad90104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad90102b50 0x7fad90103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 --2- 172.21.15.115:0/3492718078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad90108f00 0x7fad90103fe0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 -- 172.21.15.115:0/3492718078 >> 172.21.15.115:0/3492718078 conn(0x7fad900fe6f0 msgr2=0x7fad90108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 -- 172.21.15.115:0/3492718078 shutdown_connections 2024-05-09T05:41:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:01.976+0000 7fad9514c700 1 -- 172.21.15.115:0/3492718078 wait complete. 2024-05-09T05:41:02.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:41:03.059 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:03.435 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:03 smithi115 ceph-mon[30349]: pgmap v88: 129 pgs: 129 active+clean; 59 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.7 MiB/s wr, 109 op/s 2024-05-09T05:41:03.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:03 smithi195 ceph-mon[33844]: pgmap v88: 129 pgs: 129 active+clean; 59 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.7 MiB/s wr, 109 op/s 2024-05-09T05:41:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:03 smithi161 ceph-mon[33805]: pgmap v88: 129 pgs: 129 active+clean; 59 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.7 MiB/s wr, 109 op/s 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.720+0000 7f82e30f9640 1 -- 172.21.15.115:0/402244627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 msgr2=0x7f82dc103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.720+0000 7f82e30f9640 1 --2- 172.21.15.115:0/402244627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 0x7f82dc103dc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009900 tx=0x7f82c802a410 comp rx=0 tx=0).stop 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.720+0000 7f82e30f9640 1 -- 172.21.15.115:0/402244627 shutdown_connections 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.720+0000 7f82e30f9640 1 --2- 172.21.15.115:0/402244627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82dc104430 0x7f82dc110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.720+0000 7f82e30f9640 1 --2- 172.21.15.115:0/402244627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 0x7f82dc103dc0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.720+0000 7f82e30f9640 1 --2- 172.21.15.115:0/402244627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc109ce0 0x7f82dc103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.720+0000 7f82e30f9640 1 -- 172.21.15.115:0/402244627 >> 172.21.15.115:0/402244627 conn(0x7f82dc079450 msgr2=0x7f82dc079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.721+0000 7f82e30f9640 1 -- 172.21.15.115:0/402244627 shutdown_connections 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.721+0000 7f82e30f9640 1 -- 172.21.15.115:0/402244627 wait complete. 2024-05-09T05:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.721+0000 7f82e30f9640 1 Processor -- start 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.721+0000 7f82e30f9640 1 -- start start 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e30f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82dc104430 0x7f82dc1a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e30f9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc109ce0 0x7f82dc1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e30f9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 0x7f82dc19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e30f9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc113d80 con 0x7f82dc10a180 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e30f9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc113c00 con 0x7f82dc109ce0 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e30f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc113f00 con 0x7f82dc104430 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e1696640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 0x7f82dc19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e1696640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 0x7f82dc19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50480/0 (socket says 172.21.15.115:50480) 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e1696640 1 -- 172.21.15.115:0/349367186 learned_addr learned my addr 172.21.15.115:0/349367186 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82dbfff640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc109ce0 0x7f82dc1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e0e95640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82dc104430 0x7f82dc1a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e1696640 1 -- 172.21.15.115:0/349367186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82dc104430 msgr2=0x7f82dc1a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e1696640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82dc104430 0x7f82dc1a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e1696640 1 -- 172.21.15.115:0/349367186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc109ce0 msgr2=0x7f82dc1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e1696640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc109ce0 0x7f82dc1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.722+0000 7f82e1696640 1 -- 172.21.15.115:0/349367186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc19ae10 con 0x7f82dc10a180 2024-05-09T05:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.723+0000 7f82e1696640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 0x7f82dc19a6e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f82d400eaa0 tx=0x7f82d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.723+0000 7f82d9ffb640 1 -- 172.21.15.115:0/349367186 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d400cdb0 con 0x7f82dc10a180 2024-05-09T05:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.723+0000 7f82d9ffb640 1 -- 172.21.15.115:0/349367186 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82d4005020 con 0x7f82dc10a180 2024-05-09T05:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.723+0000 7f82e30f9640 1 -- 172.21.15.115:0/349367186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc19b100 con 0x7f82dc10a180 2024-05-09T05:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.723+0000 7f82d9ffb640 1 -- 172.21.15.115:0/349367186 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d40116d0 con 0x7f82dc10a180 2024-05-09T05:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.723+0000 7f82e30f9640 1 -- 172.21.15.115:0/349367186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc1a7300 con 0x7f82dc10a180 2024-05-09T05:41:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.726+0000 7f82d9ffb640 1 -- 172.21.15.115:0/349367186 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f82d40051d0 con 0x7f82dc10a180 2024-05-09T05:41:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.726+0000 7f82e9fae740 1 Processor -- start 2024-05-09T05:41:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e9fae740 1 -- start start 2024-05-09T05:41:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82d9ffb640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f82cc0b29d0 0x7f82cc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b831b0 0x5623e0f26680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b5a2f0 0x5623e0f26bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f27100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e9fae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5623e0e8b7c0 con 0x5623e0b831b0 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e9fae740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5623e0e270e0 con 0x5623e0b624d0 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e9fae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623e0e87780 con 0x5623e0b5a2f0 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e0e95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b831b0 0x5623e0f26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e0e95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b831b0 0x5623e0f26680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50492/0 (socket says 172.21.15.115:50492) 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e0e95640 1 -- 172.21.15.115:0/459198709 learned_addr learned my addr 172.21.15.115:0/459198709 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82dbfff640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f82cc0b29d0 0x7f82cc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82d9ffb640 1 -- 172.21.15.115:0/349367186 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f82d40e2b20 con 0x7f82dc10a180 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e0e95640 1 -- 172.21.15.115:0/459198709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b5a2f0 msgr2=0x5623e0f26bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e0e95640 1 --2- 172.21.15.115:0/459198709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b5a2f0 0x5623e0f26bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e0e95640 1 -- 172.21.15.115:0/459198709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 msgr2=0x5623e0f27100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e0e95640 1 --2- 172.21.15.115:0/459198709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f27100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.727+0000 7f82e0e95640 1 -- 172.21.15.115:0/459198709 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e0e507c0 con 0x5623e0b831b0 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.728+0000 7f82e0e95640 1 --2- 172.21.15.115:0/459198709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b831b0 0x5623e0f26680 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009b70 tx=0x7f82d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9feb98c3c94925f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.728+0000 7f82dbfff640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f82cc0b29d0 0x7f82cc0b4ee0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f82dc10fa60 tx=0x7f82c8032b60 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.728+0000 7f82c6ffd640 1 -- 172.21.15.115:0/459198709 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0009e60 con 0x5623e0b831b0 2024-05-09T05:41:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.728+0000 7f82c6ffd640 1 -- 172.21.15.115:0/459198709 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82d00128e0 con 0x5623e0b831b0 2024-05-09T05:41:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.728+0000 7f82c6ffd640 1 -- 172.21.15.115:0/459198709 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d000f940 con 0x5623e0b831b0 2024-05-09T05:41:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.729+0000 7f82e9fae740 1 -- 172.21.15.115:0/459198709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b831b0 msgr2=0x5623e0f26680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.729+0000 7f82e9fae740 1 --2- 172.21.15.115:0/459198709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b831b0 0x5623e0f26680 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009b70 tx=0x7f82d00117b0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.729+0000 7f82e9fae740 1 -- 172.21.15.115:0/459198709 shutdown_connections 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.729+0000 7f82e9fae740 1 --2- 172.21.15.115:0/459198709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f27100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.729+0000 7f82e9fae740 1 --2- 172.21.15.115:0/459198709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b5a2f0 0x5623e0f26bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.729+0000 7f82e9fae740 1 --2- 172.21.15.115:0/459198709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b831b0 0x5623e0f26680 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.729+0000 7f82e9fae740 1 -- 172.21.15.115:0/459198709 >> 172.21.15.115:0/459198709 conn(0x5623e0b77300 msgr2=0x5623e0f22e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.729+0000 7f82e9fae740 1 -- 172.21.15.115:0/459198709 shutdown_connections 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.729+0000 7f82e9fae740 1 -- 172.21.15.115:0/459198709 wait complete. 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.730+0000 7f82e9fae740 1 -- start start 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.730+0000 7f82e9fae740 1 Processor -- start 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.730+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b5a2f0 0x5623e0f32730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.730+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f32c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.730+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 0x5623e0f30bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.730+0000 7f82e9fae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5623e0f337b0 con 0x5623e0b5a2f0 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.730+0000 7f82e9fae740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5623e0e129c0 con 0x5623e0b624d0 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.730+0000 7f82e9fae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623e0f33930 con 0x5623e0b831b0 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e9fae740 10 client.0 _ll_register_callbacks cb 0x5623e0e3f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e0e95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b5a2f0 0x5623e0f32730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e0e95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b5a2f0 0x5623e0f32730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50504/0 (socket says 172.21.15.115:50504) 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e0e95640 1 -- 172.21.15.115:0/1952996969 learned_addr learned my addr 172.21.15.115:0/1952996969 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e1696640 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 0x5623e0f30bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e0e95640 1 -- 172.21.15.115:0/1952996969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 msgr2=0x5623e0f30bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e0e95640 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 0x5623e0f30bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e0e95640 1 -- 172.21.15.115:0/1952996969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 msgr2=0x5623e0f32c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e0e95640 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f32c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e0e95640 1 -- 172.21.15.115:0/1952996969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e0e391f0 con 0x5623e0b5a2f0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82e0e95640 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b5a2f0 0x5623e0f32730 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f82d0000c00 tx=0x7f82d0012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327ddc5f80274932 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.731+0000 7f82bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82bbfff640 1 -- 172.21.15.115:0/1952996969 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0027030 con 0x5623e0b5a2f0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82bbfff640 1 -- 172.21.15.115:0/1952996969 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f82d001f070 con 0x5623e0b5a2f0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82e9fae740 1 -- 172.21.15.115:0/1952996969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d00128e0 con 0x5623e0b5a2f0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82bbfff640 1 -- 172.21.15.115:0/1952996969 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82d0031050 con 0x5623e0b5a2f0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82e9fae740 10 client.16410 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82e9fae740 1 -- 172.21.15.115:0/1952996969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5623e0bee790 con 0x5623e0b5a2f0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82bbfff640 1 -- 172.21.15.115:0/1952996969 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82d00128e0 con 0x5623e0b5a2f0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82bbfff640 1 -- 172.21.15.115:0/1952996969 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f82d0004810 con 0x5623e0b5a2f0 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82e9fae740 10 client.16410 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82e9fae740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.732+0000 7f82e9fae740 20 client.16410 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.733+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623e0b41d80 0x5623e0f42ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.733+0000 7f82e9fae740 4 client.16410 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.733+0000 7f82e9fae740 1 -- 172.21.15.115:0/1952996969 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623e0e993d0 con 0x5623e0b41d80 2024-05-09T05:41:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.733+0000 7f82dbfff640 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623e0b41d80 0x5623e0f42ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.734+0000 7f82dbfff640 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623e0b41d80 0x5623e0f42ef0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6fd6e1c149df794 server_cookie=8fd2d46b75a6c8a5 in_seq=0 out_seq=0 2024-05-09T05:41:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.734+0000 7f82bbfff640 10 client.16410 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.734+0000 7f82bbfff640 1 -- 172.21.15.115:0/1952996969 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5623e0e993d0 con 0x5623e0b41d80 2024-05-09T05:41:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.734+0000 7f82bbfff640 10 client.16410 handle_command_reply: tid=0 2024-05-09T05:41:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.734+0000 7f82e9fae740 1 client.16410 shutdown 2024-05-09T05:41:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.734+0000 7f82e9fae740 1 -- 172.21.15.115:0/1952996969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b5a2f0 msgr2=0x5623e0f32730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.734+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b5a2f0 0x5623e0f32730 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f82d0000c00 tx=0x7f82d0012590 comp rx=0 tx=0).stop 2024-05-09T05:41:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82e9fae740 1 -- 172.21.15.115:0/1952996969 shutdown_connections 2024-05-09T05:41:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623e0b41d80 0x5623e0f42ef0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 0x5623e0f30bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f32c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1952996969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b5a2f0 0x5623e0f32730 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82e9fae740 1 -- 172.21.15.115:0/1952996969 >> 172.21.15.115:0/1952996969 conn(0x5623e0b77300 msgr2=0x5623e0f21700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82bbfff640 0 client.16410 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82e9fae740 1 -- 172.21.15.115:0/1952996969 shutdown_connections 2024-05-09T05:41:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82e9fae740 1 -- 172.21.15.115:0/1952996969 wait complete. 2024-05-09T05:41:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.735+0000 7f82e9fae740 20 client.16410 trim_cache size 0 max 16384 2024-05-09T05:41:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e9fae740 1 Processor -- start 2024-05-09T05:41:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e9fae740 1 -- start start 2024-05-09T05:41:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b77300 0x5623e0f21e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f45f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 0x5623e0f46460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e9fae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1b11d0 con 0x5623e0b77300 2024-05-09T05:41:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e9fae740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1b2d70 con 0x5623e0b624d0 2024-05-09T05:41:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e9fae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1b18a0 con 0x5623e0b831b0 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e0e95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b77300 0x5623e0f21e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e0e95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b77300 0x5623e0f21e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50516/0 (socket says 172.21.15.115:50516) 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e0e95640 1 -- 172.21.15.115:0/1065061012 learned_addr learned my addr 172.21.15.115:0/1065061012 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e1696640 1 --2- 172.21.15.115:0/1065061012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 0x5623e0f46460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82dbfff640 1 --2- 172.21.15.115:0/1065061012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f45f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e0e95640 1 -- 172.21.15.115:0/1065061012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 msgr2=0x5623e0f46460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.750+0000 7f82e0e95640 1 --2- 172.21.15.115:0/1065061012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 0x5623e0f46460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.751+0000 7f82e0e95640 1 -- 172.21.15.115:0/1065061012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 msgr2=0x5623e0f45f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.751+0000 7f82e0e95640 1 --2- 172.21.15.115:0/1065061012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f45f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.751+0000 7f82e0e95640 1 -- 172.21.15.115:0/1065061012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e0c655f0 con 0x5623e0b77300 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.751+0000 7f82e0e95640 1 --2- 172.21.15.115:0/1065061012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b77300 0x5623e0f21e10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009a90 tx=0x7f82d00095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234f94cbee25ccd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.751+0000 7f82c67fc640 1 -- 172.21.15.115:0/1065061012 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0027040 con 0x5623e0b77300 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.751+0000 7f82c67fc640 1 -- 172.21.15.115:0/1065061012 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82d000f5f0 con 0x5623e0b77300 2024-05-09T05:41:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.751+0000 7f82c67fc640 1 -- 172.21.15.115:0/1065061012 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0004e50 con 0x5623e0b77300 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 -- 172.21.15.115:0/1065061012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b77300 msgr2=0x5623e0f21e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1065061012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b77300 0x5623e0f21e10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009a90 tx=0x7f82d00095b0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 -- 172.21.15.115:0/1065061012 shutdown_connections 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1065061012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b831b0 0x5623e0f46460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1065061012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b624d0 0x5623e0f45f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 --2- 172.21.15.115:0/1065061012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b77300 0x5623e0f21e10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 -- 172.21.15.115:0/1065061012 >> 172.21.15.115:0/1065061012 conn(0x5623e0f2c530 msgr2=0x5623e0f21700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 -- 172.21.15.115:0/1065061012 shutdown_connections 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 -- 172.21.15.115:0/1065061012 wait complete. 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.752+0000 7f82e9fae740 1 -- start start 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e9fae740 1 Processor -- start 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b624d0 0x5623e0f45490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b77300 0x5623e0f459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e9fae740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b831b0 0x5623e0fa3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e9fae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1b2640 con 0x5623e0b624d0 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e9fae740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1b1f70 con 0x5623e0b831b0 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e9fae740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1af1b0 con 0x5623e0b77300 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e9fae740 10 client.0 _ll_register_callbacks cb 0x7f82d00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e0e95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b624d0 0x5623e0f45490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e0e95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b624d0 0x5623e0f45490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50532/0 (socket says 172.21.15.115:50532) 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e0e95640 1 -- 172.21.15.115:0/101233780 learned_addr learned my addr 172.21.15.115:0/101233780 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82e1696640 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b831b0 0x5623e0fa3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.753+0000 7f82dbfff640 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b77300 0x5623e0f459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82e0e95640 1 -- 172.21.15.115:0/101233780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b77300 msgr2=0x5623e0f459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82e0e95640 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b77300 0x5623e0f459d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82e0e95640 1 -- 172.21.15.115:0/101233780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b831b0 msgr2=0x5623e0fa3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82e0e95640 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b831b0 0x5623e0fa3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82e0e95640 1 -- 172.21.15.115:0/101233780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e0e4fed0 con 0x5623e0b624d0 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82e0e95640 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b624d0 0x5623e0f45490 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f82d0005860 tx=0x7f82d0005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb17fa7cda1f2c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82bbfff640 1 -- 172.21.15.115:0/101233780 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0027040 con 0x5623e0b624d0 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82bbfff640 1 -- 172.21.15.115:0/101233780 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f82d000fb20 con 0x5623e0b624d0 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82e9fae740 1 -- 172.21.15.115:0/101233780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d000f5f0 con 0x5623e0b624d0 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82bbfff640 1 -- 172.21.15.115:0/101233780 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82d0047050 con 0x5623e0b624d0 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82e9fae740 10 client.16416 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82e9fae740 1 -- 172.21.15.115:0/101233780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5623e0faa760 con 0x5623e0b624d0 2024-05-09T05:41:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.754+0000 7f82bbfff640 1 -- 172.21.15.115:0/101233780 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82d000f5f0 con 0x5623e0b624d0 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82bbfff640 1 -- 172.21.15.115:0/101233780 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f82d003d030 con 0x5623e0b624d0 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82e9fae740 10 client.16416 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82e9fae740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82e9fae740 20 client.16416 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82e9fae740 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623e0b171c0 0x5623e0face90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82e9fae740 4 client.16416 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82e9fae740 1 -- 172.21.15.115:0/101233780 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623e0e93240 con 0x5623e0b171c0 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82dbfff640 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623e0b171c0 0x5623e0face90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82bbfff640 10 client.16416 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.755+0000 7f82dbfff640 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623e0b171c0 0x5623e0face90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c4bf75a2dace3f server_cookie=3876270c48236c54 in_seq=0 out_seq=0 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.784+0000 7f82bbfff640 1 -- 172.21.15.115:0/101233780 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5623e0e93240 con 0x5623e0b171c0 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.784+0000 7f82bbfff640 10 client.16416 handle_command_reply: tid=0 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.784+0000 7f82e9fae740 1 client.16416 shutdown 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.784+0000 7f82e9fae740 1 -- 172.21.15.115:0/101233780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b624d0 msgr2=0x5623e0f45490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.784+0000 7f82e9fae740 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b624d0 0x5623e0f45490 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f82d0005860 tx=0x7f82d0005cc0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.784+0000 7f82e9fae740 1 -- 172.21.15.115:0/101233780 shutdown_connections 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.784+0000 7f82e9fae740 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623e0b831b0 0x5623e0fa3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.784+0000 7f82e9fae740 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623e0b171c0 0x5623e0face90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.785+0000 7f82e9fae740 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623e0b77300 0x5623e0f459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.785+0000 7f82e9fae740 1 --2- 172.21.15.115:0/101233780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623e0b624d0 0x5623e0f45490 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.785+0000 7f82bbfff640 0 client.16416 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.785+0000 7f82e9fae740 1 -- 172.21.15.115:0/101233780 >> 172.21.15.115:0/101233780 conn(0x5623e0f2c530 msgr2=0x5623e0f22720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.785+0000 7f82e9fae740 1 -- 172.21.15.115:0/101233780 shutdown_connections 2024-05-09T05:41:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.785+0000 7f82e9fae740 1 -- 172.21.15.115:0/101233780 wait complete. 2024-05-09T05:41:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.785+0000 7f82e9fae740 20 client.16416 trim_cache size 0 max 16384 2024-05-09T05:41:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:41:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:41:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:41:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.785+0000 7f82e30f9640 1 -- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f82cc0b29d0 msgr2=0x7f82cc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.785+0000 7f82e30f9640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f82cc0b29d0 0x7f82cc0b4ee0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f82dc10fa60 tx=0x7f82c8032b60 comp rx=0 tx=0).stop 2024-05-09T05:41:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 -- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 msgr2=0x7f82dc19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 0x7f82dc19a6e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f82d400eaa0 tx=0x7f82d400ef60 comp rx=0 tx=0).stop 2024-05-09T05:41:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 -- 172.21.15.115:0/349367186 shutdown_connections 2024-05-09T05:41:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f82cc0b29d0 0x7f82cc0b4ee0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82dc10a180 0x7f82dc19a6e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82dc109ce0 0x7f82dc1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 --2- 172.21.15.115:0/349367186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82dc104430 0x7f82dc1a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 -- 172.21.15.115:0/349367186 >> 172.21.15.115:0/349367186 conn(0x7f82dc079450 msgr2=0x7f82dc105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 -- 172.21.15.115:0/349367186 shutdown_connections 2024-05-09T05:41:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:04.786+0000 7f82e30f9640 1 -- 172.21.15.115:0/349367186 wait complete. 2024-05-09T05:41:04.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.949+0000 7f001acd4700 1 -- 172.21.15.115:0/2386802064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 msgr2=0x7f0014108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.949+0000 7f001acd4700 1 --2- 172.21.15.115:0/2386802064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 0x7f0014108200 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f00000099c0 tx=0x7f0000009cd0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.949+0000 7f001acd4700 1 -- 172.21.15.115:0/2386802064 shutdown_connections 2024-05-09T05:41:04.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.949+0000 7f001acd4700 1 --2- 172.21.15.115:0/2386802064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00141034c0 0x7f00141038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.949+0000 7f001acd4700 1 --2- 172.21.15.115:0/2386802064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0014108740 0x7f00141100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.949+0000 7f001acd4700 1 --2- 172.21.15.115:0/2386802064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 0x7f0014108200 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.949+0000 7f001acd4700 1 -- 172.21.15.115:0/2386802064 >> 172.21.15.115:0/2386802064 conn(0x7f00140fedc0 msgr2=0x7f00141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.949+0000 7f001acd4700 1 -- 172.21.15.115:0/2386802064 shutdown_connections 2024-05-09T05:41:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.949+0000 7f001acd4700 1 -- 172.21.15.115:0/2386802064 wait complete. 2024-05-09T05:41:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.950+0000 7f001acd4700 1 Processor -- start 2024-05-09T05:41:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.950+0000 7f001acd4700 1 -- start start 2024-05-09T05:41:04.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.950+0000 7f001acd4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00141034c0 0x7f001419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.950+0000 7f001acd4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 0x7f001419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.950+0000 7f001acd4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0014108740 0x7f001419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f001acd4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0014113200 con 0x7f0014107df0 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f001acd4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0014113080 con 0x7f00141034c0 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f001acd4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0014113380 con 0x7f0014108740 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0018a70700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00141034c0 0x7f001419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0013fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 0x7f001419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0013fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 0x7f001419c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50552/0 (socket says 172.21.15.115:50552) 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0013fff700 1 -- 172.21.15.115:0/537752427 learned_addr learned my addr 172.21.15.115:0/537752427 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0019271700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0014108740 0x7f001419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0013fff700 1 -- 172.21.15.115:0/537752427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0014108740 msgr2=0x7f001419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0013fff700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0014108740 0x7f001419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0013fff700 1 -- 172.21.15.115:0/537752427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00141034c0 msgr2=0x7f001419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0013fff700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00141034c0 0x7f001419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0013fff700 1 -- 172.21.15.115:0/537752427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00141a6ed0 con 0x7f0014107df0 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0018a70700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00141034c0 0x7f001419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.951+0000 7f0013fff700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 0x7f001419c0c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f000000b580 tx=0x7f0000004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.952+0000 7f0011ffb700 1 -- 172.21.15.115:0/537752427 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000000f8a0 con 0x7f0014107df0 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.952+0000 7f0011ffb700 1 -- 172.21.15.115:0/537752427 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0000004830 con 0x7f0014107df0 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.952+0000 7f001acd4700 1 -- 172.21.15.115:0/537752427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00141a7160 con 0x7f0014107df0 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.952+0000 7f0011ffb700 1 -- 172.21.15.115:0/537752427 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000005350 con 0x7f0014107df0 2024-05-09T05:41:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.952+0000 7f001acd4700 1 -- 172.21.15.115:0/537752427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00141a7550 con 0x7f0014107df0 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.955+0000 7f0011ffb700 1 -- 172.21.15.115:0/537752427 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0000029060 con 0x7f0014107df0 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f0011ffb700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00040b29f0 0x7f00040b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f00296eab80 1 Processor -- start 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f00296eab80 1 -- start start 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f0018a70700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00040b29f0 0x7f00040b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f0011ffb700 1 -- 172.21.15.115:0/537752427 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f00000e3500 con 0x7f0014107df0 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f00296eab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206dc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f00296eab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 0x5648206d8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f00296eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648206d8a20 0x5648206ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f00296eab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5648205fb0a0 con 0x5648205d5d00 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f00296eab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5648205a09a0 con 0x56482055b4a0 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f00296eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5648205f7060 con 0x5648206d8a20 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f0018a70700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206dc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f0018a70700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206dc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50558/0 (socket says 172.21.15.115:50558) 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f0018a70700 1 -- 172.21.15.115:0/1712828342 learned_addr learned my addr 172.21.15.115:0/1712828342 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f0019271700 1 --2- 172.21.15.115:0/1712828342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648206d8a20 0x5648206ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.956+0000 7f0013fff700 1 --2- 172.21.15.115:0/1712828342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 0x5648206d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7f0018a70700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00040b29f0 0x7f00040b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00080139a0 tx=0x7f0008013d60 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7f0018a70700 1 -- 172.21.15.115:0/1712828342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648206d8a20 msgr2=0x5648206ed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7f0018a70700 1 --2- 172.21.15.115:0/1712828342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648206d8a20 0x5648206ed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7f0018a70700 1 -- 172.21.15.115:0/1712828342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 msgr2=0x5648206d8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7f0018a70700 1 --2- 172.21.15.115:0/1712828342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 0x5648206d8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7f0018a70700 1 -- 172.21.15.115:0/1712828342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564820591500 con 0x5648205d5d00 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7f0018a70700 1 --2- 172.21.15.115:0/1712828342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206dc2a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f000800fba0 tx=0x7f000801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fc304c119e2961 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7efffeffd700 1 -- 172.21.15.115:0/1712828342 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008013230 con 0x5648205d5d00 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7efffeffd700 1 -- 172.21.15.115:0/1712828342 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0008004cc0 con 0x5648205d5d00 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.957+0000 7efffeffd700 1 -- 172.21.15.115:0/1712828342 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008005680 con 0x5648205d5d00 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.958+0000 7f00296eab80 1 -- 172.21.15.115:0/1712828342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 msgr2=0x5648206dc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.958+0000 7f00296eab80 1 --2- 172.21.15.115:0/1712828342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206dc2a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f000800fba0 tx=0x7f000801e000 comp rx=0 tx=0).stop 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.958+0000 7f00296eab80 1 -- 172.21.15.115:0/1712828342 shutdown_connections 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.958+0000 7f00296eab80 1 --2- 172.21.15.115:0/1712828342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648206d8a20 0x5648206ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.958+0000 7f00296eab80 1 --2- 172.21.15.115:0/1712828342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206dc2a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.958+0000 7f00296eab80 1 --2- 172.21.15.115:0/1712828342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 0x5648206d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.958+0000 7f00296eab80 1 -- 172.21.15.115:0/1712828342 >> 172.21.15.115:0/1712828342 conn(0x56482055ef70 msgr2=0x5648206d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.958+0000 7f00296eab80 1 -- 172.21.15.115:0/1712828342 shutdown_connections 2024-05-09T05:41:04.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.958+0000 7f00296eab80 1 -- 172.21.15.115:0/1712828342 wait complete. 2024-05-09T05:41:04.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f00296eab80 1 -- start start 2024-05-09T05:41:04.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f00296eab80 1 Processor -- start 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f00296eab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56482055b4a0 0x5648206ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f00296eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 0x5648206eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f00296eab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 0x5648206eb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f00296eab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5648206ed5b0 con 0x56482055b4a0 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f00296eab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564820595b20 con 0x5648206d8a20 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f00296eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5648206ed730 con 0x5648205d5d00 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f00296eab80 10 client.0 _ll_register_callbacks cb 0x56482046c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f0018a70700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56482055b4a0 0x5648206ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f0018a70700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56482055b4a0 0x5648206ea5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50572/0 (socket says 172.21.15.115:50572) 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f0018a70700 1 -- 172.21.15.115:0/4101771119 learned_addr learned my addr 172.21.15.115:0/4101771119 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f0019271700 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 0x5648206eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.959+0000 7f0013fff700 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 0x5648206eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7f0019271700 1 -- 172.21.15.115:0/4101771119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 msgr2=0x5648206eab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7f0019271700 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 0x5648206eab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7f0019271700 1 -- 172.21.15.115:0/4101771119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56482055b4a0 msgr2=0x5648206ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7f0019271700 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56482055b4a0 0x5648206ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7f0019271700 1 -- 172.21.15.115:0/4101771119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564820476490 con 0x5648206d8a20 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7f0018a70700 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56482055b4a0 0x5648206ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7f0019271700 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 0x5648206eb050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f001419cb60 tx=0x7f000c007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4195c23c97e7875 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7efff3fff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7efff3fff700 1 -- 172.21.15.115:0/4101771119 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c01a070 con 0x5648206d8a20 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7f00296eab80 1 -- 172.21.15.115:0/4101771119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0008004cc0 con 0x5648206d8a20 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7efff3fff700 1 -- 172.21.15.115:0/4101771119 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f000c012b40 con 0x5648206d8a20 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.960+0000 7efff3fff700 1 -- 172.21.15.115:0/4101771119 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f000c026050 con 0x5648206d8a20 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7f00296eab80 10 client.25913 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7f00296eab80 1 -- 172.21.15.115:0/4101771119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5648206f82a0 con 0x5648206d8a20 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7efff3fff700 1 -- 172.21.15.115:0/4101771119 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0008004cc0 con 0x5648206d8a20 2024-05-09T05:41:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7efff3fff700 1 -- 172.21.15.115:0/4101771119 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f000c016c50 con 0x5648206d8a20 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7f00296eab80 10 client.25913 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7f00296eab80 10 client.25913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7f00296eab80 20 client.25913 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7f00296eab80 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5648206fa830 0x5648206fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7f00296eab80 4 client.25913 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7f00296eab80 1 -- 172.21.15.115:0/4101771119 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564820608a80 con 0x5648206fa830 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.961+0000 7f0018a70700 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5648206fa830 0x5648206fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.962+0000 7efff3fff700 10 client.25913 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.962+0000 7f0018a70700 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5648206fa830 0x5648206fcc50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef8720a062c557ba server_cookie=bccadd0fe1c5e578 in_seq=0 out_seq=0 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.962+0000 7efff3fff700 1 -- 172.21.15.115:0/4101771119 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564820608a80 con 0x5648206fa830 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.962+0000 7efff3fff700 10 client.25913 handle_command_reply: tid=0 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.962+0000 7f00296eab80 1 client.25913 shutdown 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 -- 172.21.15.115:0/4101771119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 msgr2=0x5648206eb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 0x5648206eb050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f001419cb60 tx=0x7f000c007dd0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 -- 172.21.15.115:0/4101771119 shutdown_connections 2024-05-09T05:41:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 0x5648206eb050 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5648206fa830 0x5648206fcc50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56482055b4a0 0x5648206ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 --2- 172.21.15.115:0/4101771119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 0x5648206eab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 -- 172.21.15.115:0/4101771119 >> 172.21.15.115:0/4101771119 conn(0x56482055ef70 msgr2=0x5648206d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7efff3fff700 0 client.25913 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:41:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 -- 172.21.15.115:0/4101771119 shutdown_connections 2024-05-09T05:41:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 1 -- 172.21.15.115:0/4101771119 wait complete. 2024-05-09T05:41:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.963+0000 7f00296eab80 20 client.25913 trim_cache size 0 max 16384 2024-05-09T05:41:04.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f00296eab80 1 Processor -- start 2024-05-09T05:41:04.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f00296eab80 1 -- start start 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f00296eab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 0x5648206cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f00296eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 0x5648206eae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f00296eab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648206d8a20 0x5648206ec390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f00296eab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f00141ae090 con 0x5648206d8a20 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f00296eab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f00141afc30 con 0x56482055b4a0 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f00296eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00141ae760 con 0x5648205d5d00 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f0019271700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648206d8a20 0x5648206ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f0019271700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648206d8a20 0x5648206ec390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50584/0 (socket says 172.21.15.115:50584) 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f0019271700 1 -- 172.21.15.115:0/1450541482 learned_addr learned my addr 172.21.15.115:0/1450541482 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f0018a70700 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 0x5648206cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.981+0000 7f0013fff700 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 0x5648206eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7f0018a70700 1 -- 172.21.15.115:0/1450541482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 msgr2=0x5648206eae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7f0018a70700 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 0x5648206eae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7f0018a70700 1 -- 172.21.15.115:0/1450541482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648206d8a20 msgr2=0x5648206ec390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7f0018a70700 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648206d8a20 0x5648206ec390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7f0018a70700 1 -- 172.21.15.115:0/1450541482 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648206cf9d0 con 0x56482055b4a0 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7f0019271700 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648206d8a20 0x5648206ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7f0018a70700 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 0x5648206cda20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f00080052a0 tx=0x7f0008013440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53b5408ff26fb616 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7efffe7fc700 1 -- 172.21.15.115:0/1450541482 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000802f030 con 0x56482055b4a0 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7efffe7fc700 1 -- 172.21.15.115:0/1450541482 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00080055d0 con 0x56482055b4a0 2024-05-09T05:41:04.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.982+0000 7efffe7fc700 1 -- 172.21.15.115:0/1450541482 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000802f1d0 con 0x56482055b4a0 2024-05-09T05:41:04.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.983+0000 7f00296eab80 1 -- 172.21.15.115:0/1450541482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 msgr2=0x5648206cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.983+0000 7f00296eab80 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 0x5648206cda20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f00080052a0 tx=0x7f0008013440 comp rx=0 tx=0).stop 2024-05-09T05:41:04.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.983+0000 7f00296eab80 1 -- 172.21.15.115:0/1450541482 shutdown_connections 2024-05-09T05:41:04.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.983+0000 7f00296eab80 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56482055b4a0 0x5648206cda20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.983+0000 7f00296eab80 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648206d8a20 0x5648206ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.983+0000 7f00296eab80 1 --2- 172.21.15.115:0/1450541482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5648205d5d00 0x5648206eae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.983+0000 7f00296eab80 1 -- 172.21.15.115:0/1450541482 >> 172.21.15.115:0/1450541482 conn(0x56482055ef70 msgr2=0x5648206ebc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.983+0000 7f00296eab80 1 -- 172.21.15.115:0/1450541482 shutdown_connections 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.983+0000 7f00296eab80 1 -- 172.21.15.115:0/1450541482 wait complete. 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f00296eab80 1 -- start start 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f00296eab80 1 Processor -- start 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f00296eab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56482055b4a0 0x5648206edc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f00296eab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f00296eab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 0x564820762fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f00296eab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f00141af500 con 0x5648205d5d00 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f00296eab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f00141aee30 con 0x5648206d8a20 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f00296eab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00141ac030 con 0x56482055b4a0 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f00296eab80 10 client.0 _ll_register_callbacks cb 0x56482046c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f0013fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f0013fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206ee1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50600/0 (socket says 172.21.15.115:50600) 2024-05-09T05:41:04.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f0013fff700 1 -- 172.21.15.115:0/136200056 learned_addr learned my addr 172.21.15.115:0/136200056 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.984+0000 7f0018a70700 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56482055b4a0 0x5648206edc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f0019271700 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 0x564820762fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f0013fff700 1 -- 172.21.15.115:0/136200056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56482055b4a0 msgr2=0x5648206edc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f0013fff700 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56482055b4a0 0x5648206edc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f0013fff700 1 -- 172.21.15.115:0/136200056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 msgr2=0x564820762fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f0013fff700 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 0x564820762fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f0013fff700 1 -- 172.21.15.115:0/136200056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648207634f0 con 0x5648205d5d00 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f0013fff700 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206ee1a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f00000e0970 tx=0x7f0000004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec21bd45859f0af0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7efff3fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7efff3fff700 1 -- 172.21.15.115:0/136200056 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00000ae8e0 con 0x5648205d5d00 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7efff3fff700 1 -- 172.21.15.115:0/136200056 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0000005700 con 0x5648205d5d00 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f00296eab80 1 -- 172.21.15.115:0/136200056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00080055d0 con 0x5648205d5d00 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7efff3fff700 1 -- 172.21.15.115:0/136200056 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f00000b3300 con 0x5648205d5d00 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f00296eab80 10 client.16434 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:04.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.985+0000 7f00296eab80 1 -- 172.21.15.115:0/136200056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564820764710 con 0x5648205d5d00 2024-05-09T05:41:04.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.986+0000 7efff3fff700 1 -- 172.21.15.115:0/136200056 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00080055d0 con 0x5648205d5d00 2024-05-09T05:41:04.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.986+0000 7efff3fff700 1 -- 172.21.15.115:0/136200056 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f00000b1c60 con 0x5648205d5d00 2024-05-09T05:41:04.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.986+0000 7f00296eab80 10 client.16434 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:04.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.986+0000 7f00296eab80 10 client.16434 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:41:04.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.986+0000 7f00296eab80 20 client.16434 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:04.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.986+0000 7f00296eab80 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x564820764df0 0x564820767210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:04.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.986+0000 7f00296eab80 4 client.16434 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:41:04.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.986+0000 7f00296eab80 1 -- 172.21.15.115:0/136200056 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564820602b20 con 0x564820764df0 2024-05-09T05:41:04.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.986+0000 7f0018a70700 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x564820764df0 0x564820767210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:04.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f0018a70700 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x564820764df0 0x564820767210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c56e7be09b7926b server_cookie=6c71e4b9a0f8c19a in_seq=0 out_seq=0 2024-05-09T05:41:04.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7efff3fff700 10 client.16434 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7efff3fff700 1 -- 172.21.15.115:0/136200056 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564820602b20 con 0x564820764df0 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7efff3fff700 10 client.16434 handle_command_reply: tid=0 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 client.16434 shutdown 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 -- 172.21.15.115:0/136200056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 msgr2=0x5648206ee1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206ee1a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f00000e0970 tx=0x7f0000004f10 comp rx=0 tx=0).stop 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 -- 172.21.15.115:0/136200056 shutdown_connections 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648206d8a20 0x564820762fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x564820764df0 0x564820767210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56482055b4a0 0x5648206edc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7efff3fff700 0 client.16434 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 --2- 172.21.15.115:0/136200056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648205d5d00 0x5648206ee1a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 -- 172.21.15.115:0/136200056 >> 172.21.15.115:0/136200056 conn(0x56482055ef70 msgr2=0x5648206cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 -- 172.21.15.115:0/136200056 shutdown_connections 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.987+0000 7f00296eab80 1 -- 172.21.15.115:0/136200056 wait complete. 2024-05-09T05:41:04.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f00296eab80 20 client.16434 trim_cache size 0 max 16384 2024-05-09T05:41:04.985 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:41:04.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 -- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00040b29f0 msgr2=0x7f00040b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00040b29f0 0x7f00040b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00080139a0 tx=0x7f0008013d60 comp rx=0 tx=0).stop 2024-05-09T05:41:04.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 -- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 msgr2=0x7f001419c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:04.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 0x7f001419c0c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f000000b580 tx=0x7f0000004790 comp rx=0 tx=0).stop 2024-05-09T05:41:04.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 -- 172.21.15.115:0/537752427 shutdown_connections 2024-05-09T05:41:04.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f00040b29f0 0x7f00040b4f00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00141034c0 0x7f001419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0014108740 0x7f001419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 --2- 172.21.15.115:0/537752427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0014107df0 0x7f001419c0c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:04.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 -- 172.21.15.115:0/537752427 >> 172.21.15.115:0/537752427 conn(0x7f00140fedc0 msgr2=0x7f00141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:04.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 -- 172.21.15.115:0/537752427 shutdown_connections 2024-05-09T05:41:04.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:04.988+0000 7f001acd4700 1 -- 172.21.15.115:0/537752427 wait complete. 2024-05-09T05:41:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:05 smithi195 ceph-mon[33844]: pgmap v89: 129 pgs: 129 active+clean; 188 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 15 MiB/s wr, 135 op/s 2024-05-09T05:41:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:05 smithi115 ceph-mon[30349]: pgmap v89: 129 pgs: 129 active+clean; 188 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 15 MiB/s wr, 135 op/s 2024-05-09T05:41:05.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:05 smithi161 ceph-mon[33805]: pgmap v89: 129 pgs: 129 active+clean; 188 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 15 MiB/s wr, 135 op/s 2024-05-09T05:41:05.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:41:06.079 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:07.538 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:07 smithi115 ceph-mon[30349]: pgmap v90: 129 pgs: 129 active+clean; 188 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 15 MiB/s wr, 56 op/s 2024-05-09T05:41:07.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:07 smithi195 ceph-mon[33844]: pgmap v90: 129 pgs: 129 active+clean; 188 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 15 MiB/s wr, 56 op/s 2024-05-09T05:41:07.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:07 smithi161 ceph-mon[33805]: pgmap v90: 129 pgs: 129 active+clean; 188 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 15 MiB/s wr, 56 op/s 2024-05-09T05:41:07.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.967+0000 7f94a87ba700 1 -- 172.21.15.115:0/1821595343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 msgr2=0x7f94a01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.967+0000 7f94a87ba700 1 --2- 172.21.15.115:0/1821595343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 0x7f94a01038d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f949c0099c0 tx=0x7f949c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.967+0000 7f94a87ba700 1 -- 172.21.15.115:0/1821595343 shutdown_connections 2024-05-09T05:41:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.967+0000 7f94a87ba700 1 --2- 172.21.15.115:0/1821595343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0108740 0x7f94a01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.967+0000 7f94a87ba700 1 --2- 172.21.15.115:0/1821595343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 0x7f94a01038d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.967+0000 7f94a87ba700 1 --2- 172.21.15.115:0/1821595343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a0107df0 0x7f94a0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.967+0000 7f94a87ba700 1 -- 172.21.15.115:0/1821595343 >> 172.21.15.115:0/1821595343 conn(0x7f94a00fedc0 msgr2=0x7f94a01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.967+0000 7f94a87ba700 1 -- 172.21.15.115:0/1821595343 shutdown_connections 2024-05-09T05:41:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.967+0000 7f94a87ba700 1 -- 172.21.15.115:0/1821595343 wait complete. 2024-05-09T05:41:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.968+0000 7f94a87ba700 1 Processor -- start 2024-05-09T05:41:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.968+0000 7f94a87ba700 1 -- start start 2024-05-09T05:41:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a87ba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 0x7f94a019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a87ba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0107df0 0x7f94a019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a87ba700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a0108740 0x7f94a019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a87ba700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0113240 con 0x7f94a01034c0 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a87ba700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01130c0 con 0x7f94a0107df0 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a87ba700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01133c0 con 0x7f94a0108740 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 0x7f94a019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 0x7f94a019ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50614/0 (socket says 172.21.15.115:50614) 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6556700 1 -- 172.21.15.115:0/3795140081 learned_addr learned my addr 172.21.15.115:0/3795140081 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a5d55700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0107df0 0x7f94a019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6d57700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a0108740 0x7f94a019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6556700 1 -- 172.21.15.115:0/3795140081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a0108740 msgr2=0x7f94a019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6556700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a0108740 0x7f94a019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6556700 1 -- 172.21.15.115:0/3795140081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0107df0 msgr2=0x7f94a019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6556700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0107df0 0x7f94a019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6556700 1 -- 172.21.15.115:0/3795140081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a019fb60 con 0x7f94a01034c0 2024-05-09T05:41:07.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.969+0000 7f94a6556700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 0x7f94a019ad40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f949c006010 tx=0x7f949c004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:07.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.970+0000 7f94977fe700 1 -- 172.21.15.115:0/3795140081 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c009e60 con 0x7f94a01034c0 2024-05-09T05:41:07.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.970+0000 7f94977fe700 1 -- 172.21.15.115:0/3795140081 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f949c003bd0 con 0x7f94a01034c0 2024-05-09T05:41:07.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.970+0000 7f94977fe700 1 -- 172.21.15.115:0/3795140081 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c01d450 con 0x7f94a01034c0 2024-05-09T05:41:07.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.970+0000 7f94a87ba700 1 -- 172.21.15.115:0/3795140081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a019fdf0 con 0x7f94a01034c0 2024-05-09T05:41:07.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.970+0000 7f94a87ba700 1 -- 172.21.15.115:0/3795140081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a01a0320 con 0x7f94a01034c0 2024-05-09T05:41:07.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.973+0000 7f94977fe700 1 -- 172.21.15.115:0/3795140081 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f949c00f460 con 0x7f94a01034c0 2024-05-09T05:41:07.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.974+0000 7f94977fe700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f948c0b2a70 0x7f948c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.974+0000 7f94a5d55700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f948c0b2a70 0x7f948c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.974+0000 7f94977fe700 1 -- 172.21.15.115:0/3795140081 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f949c0e7d20 con 0x7f94a01034c0 2024-05-09T05:41:07.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.974+0000 7f94b71d0b80 1 Processor -- start 2024-05-09T05:41:07.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.974+0000 7f94b71d0b80 1 -- start start 2024-05-09T05:41:07.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1fc2d00 0x5601e20cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a5d55700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f948c0b2a70 0x7f948c0b4f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94a006c2a0 tx=0x7f9490009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:07.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1f484a0 0x5601e20cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5601e1fe80a0 con 0x5601e1fc2d00 2024-05-09T05:41:07.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5601e1f8d9a0 con 0x5601e1f484a0 2024-05-09T05:41:07.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5601e1fe4060 con 0x5601e20c5400 2024-05-09T05:41:07.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1fc2d00 0x5601e20cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1fc2d00 0x5601e20cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50628/0 (socket says 172.21.15.115:50628) 2024-05-09T05:41:07.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6556700 1 -- 172.21.15.115:0/319805793 learned_addr learned my addr 172.21.15.115:0/319805793 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:07.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6d57700 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a5d55700 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1f484a0 0x5601e20cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6556700 1 -- 172.21.15.115:0/319805793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 msgr2=0x5601e20c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:07.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6556700 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6556700 1 -- 172.21.15.115:0/319805793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1f484a0 msgr2=0x5601e20cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:07.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6556700 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1f484a0 0x5601e20cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6556700 1 -- 172.21.15.115:0/319805793 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601e1f7e500 con 0x5601e1fc2d00 2024-05-09T05:41:07.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6d57700 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:07.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f94a6556700 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1fc2d00 0x5601e20cb240 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f949c011040 tx=0x7f949c0ae720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94314bc9900a684c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:07.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.975+0000 7f9494ff9700 1 -- 172.21.15.115:0/319805793 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0c6070 con 0x5601e1fc2d00 2024-05-09T05:41:07.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.976+0000 7f9494ff9700 1 -- 172.21.15.115:0/319805793 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f949c0af450 con 0x5601e1fc2d00 2024-05-09T05:41:07.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.976+0000 7f9494ff9700 1 -- 172.21.15.115:0/319805793 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c00f750 con 0x5601e1fc2d00 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 -- 172.21.15.115:0/319805793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1fc2d00 msgr2=0x5601e20cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1fc2d00 0x5601e20cb240 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f949c011040 tx=0x7f949c0ae720 comp rx=0 tx=0).stop 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 -- 172.21.15.115:0/319805793 shutdown_connections 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1fc2d00 0x5601e20cb240 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/319805793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1f484a0 0x5601e20cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 -- 172.21.15.115:0/319805793 >> 172.21.15.115:0/319805793 conn(0x5601e1f4bf70 msgr2=0x5601e20c0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 -- 172.21.15.115:0/319805793 shutdown_connections 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 -- 172.21.15.115:0/319805793 wait complete. 2024-05-09T05:41:07.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 -- start start 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.977+0000 7f94b71d0b80 1 Processor -- start 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1f484a0 0x5601e20dd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20dd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e20c5400 0x5601e20d7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5601e1fe80a0 con 0x5601e20c5400 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5601e1f82b20 con 0x5601e1fc2d00 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5601e20c5fb0 con 0x5601e1f484a0 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94b71d0b80 10 client.0 _ll_register_callbacks cb 0x5601e1e596c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a5d55700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a5d55700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20dd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:40210/0 (socket says 172.21.15.115:40210) 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a5d55700 1 -- 172.21.15.115:0/1973631880 learned_addr learned my addr 172.21.15.115:0/1973631880 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a6d57700 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e20c5400 0x5601e20d7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a6556700 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1f484a0 0x5601e20dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a5d55700 1 -- 172.21.15.115:0/1973631880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1f484a0 msgr2=0x5601e20dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a5d55700 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1f484a0 0x5601e20dd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a5d55700 1 -- 172.21.15.115:0/1973631880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e20c5400 msgr2=0x5601e20d7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a5d55700 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e20c5400 0x5601e20d7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.978+0000 7f94a5d55700 1 -- 172.21.15.115:0/1973631880 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601e1e63490 con 0x5601e1fc2d00 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f94a5d55700 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20dd770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94900044f0 tx=0x7f94900047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1051ee36a4c6d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f9489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f9489ffb700 1 -- 172.21.15.115:0/1973631880 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490004a30 con 0x5601e1fc2d00 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f9489ffb700 1 -- 172.21.15.115:0/1973631880 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9490004bd0 con 0x5601e1fc2d00 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f94b71d0b80 1 -- 172.21.15.115:0/1973631880 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949c0af450 con 0x5601e1fc2d00 2024-05-09T05:41:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f9489ffb700 1 -- 172.21.15.115:0/1973631880 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9490004520 con 0x5601e1fc2d00 2024-05-09T05:41:07.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f94b71d0b80 10 client.25928 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:07.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f94b71d0b80 1 -- 172.21.15.115:0/1973631880 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5601e20e4eb0 con 0x5601e1fc2d00 2024-05-09T05:41:07.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f9489ffb700 1 -- 172.21.15.115:0/1973631880 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f949c0af450 con 0x5601e1fc2d00 2024-05-09T05:41:07.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.979+0000 7f9489ffb700 1 -- 172.21.15.115:0/1973631880 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f9490026030 con 0x5601e1fc2d00 2024-05-09T05:41:07.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.980+0000 7f94b71d0b80 10 client.25928 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:07.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.980+0000 7f94b71d0b80 10 client.25928 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:41:07.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.980+0000 7f94b71d0b80 20 client.25928 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:07.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.980+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5601e20e74f0 0x5601e20e9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.980+0000 7f94b71d0b80 4 client.25928 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.980+0000 7f94b71d0b80 1 -- 172.21.15.115:0/1973631880 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601e1ff5a80 con 0x5601e20e74f0 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.980+0000 7f94a6556700 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5601e20e74f0 0x5601e20e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.980+0000 7f9489ffb700 10 client.25928 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.981+0000 7f94a6556700 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5601e20e74f0 0x5601e20e9910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1450825791f8ca48 server_cookie=8a45aa4e1e4b7dec in_seq=0 out_seq=0 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.981+0000 7f9489ffb700 1 -- 172.21.15.115:0/1973631880 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5601e1ff5a80 con 0x5601e20e74f0 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f9489ffb700 10 client.25928 handle_command_reply: tid=0 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f94b71d0b80 1 client.25928 shutdown 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f94b71d0b80 1 -- 172.21.15.115:0/1973631880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 msgr2=0x5601e20dd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20dd770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94900044f0 tx=0x7f94900047c0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f94b71d0b80 1 -- 172.21.15.115:0/1973631880 shutdown_connections 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5601e20e74f0 0x5601e20e9910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e20c5400 0x5601e20d7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f9489ffb700 0 client.25928 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1f484a0 0x5601e20dd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/1973631880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20dd770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.982+0000 7f94b71d0b80 1 -- 172.21.15.115:0/1973631880 >> 172.21.15.115:0/1973631880 conn(0x5601e1f4bf70 msgr2=0x5601e20b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.983+0000 7f94b71d0b80 1 -- 172.21.15.115:0/1973631880 shutdown_connections 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.983+0000 7f94b71d0b80 1 -- 172.21.15.115:0/1973631880 wait complete. 2024-05-09T05:41:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.983+0000 7f94b71d0b80 20 client.25928 trim_cache size 0 max 16384 2024-05-09T05:41:07.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94b71d0b80 1 Processor -- start 2024-05-09T05:41:07.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94b71d0b80 1 -- start start 2024-05-09T05:41:07.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e20d65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20ba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20ef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01ae000 con 0x5601e1f484a0 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01afba0 con 0x5601e1fc2d00 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01ae6d0 con 0x5601e20c5400 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94a6556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e20d65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94a6556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e20d65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50644/0 (socket says 172.21.15.115:50644) 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.998+0000 7f94a6556700 1 -- 172.21.15.115:0/3536508741 learned_addr learned my addr 172.21.15.115:0/3536508741 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f94a5d55700 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f94a6d57700 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f94a6556700 1 -- 172.21.15.115:0/3536508741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 msgr2=0x5601e20ef080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f94a6556700 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20ef080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f94a6556700 1 -- 172.21.15.115:0/3536508741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 msgr2=0x5601e20ba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f94a6556700 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20ba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f94a6556700 1 -- 172.21.15.115:0/3536508741 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601e20bc9d0 con 0x5601e1f484a0 2024-05-09T05:41:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f94a6d57700 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20ef080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f94a6556700 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e20d65b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f949c0e51e0 tx=0x7f949c00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f8dc7b14c677eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f948bfff700 1 -- 172.21.15.115:0/3536508741 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0ce070 con 0x5601e1f484a0 2024-05-09T05:41:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f948bfff700 1 -- 172.21.15.115:0/3536508741 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f949c01c070 con 0x5601e1f484a0 2024-05-09T05:41:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:07.999+0000 7f948bfff700 1 -- 172.21.15.115:0/3536508741 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0c6040 con 0x5601e1f484a0 2024-05-09T05:41:07.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.000+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3536508741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 msgr2=0x5601e20d65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:07.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.000+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e20d65b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f949c0e51e0 tx=0x7f949c00ff30 comp rx=0 tx=0).stop 2024-05-09T05:41:07.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.000+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3536508741 shutdown_connections 2024-05-09T05:41:07.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.000+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e20c5400 0x5601e20ef080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.000+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e20d65b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.000+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3536508741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e1fc2d00 0x5601e20ba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.000+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3536508741 >> 172.21.15.115:0/3536508741 conn(0x5601e1f4bf70 msgr2=0x5601e20b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.000+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3536508741 shutdown_connections 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.000+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3536508741 wait complete. 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94b71d0b80 1 -- start start 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94b71d0b80 1 Processor -- start 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e214a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1fc2d00 0x5601e214a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94b71d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e20c5400 0x5601e20e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01af470 con 0x5601e1f484a0 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94a6556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e214a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01aeda0 con 0x5601e20c5400 2024-05-09T05:41:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94b71d0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01abfa0 con 0x5601e1fc2d00 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94a6556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e214a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50656/0 (socket says 172.21.15.115:50656) 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94a6556700 1 -- 172.21.15.115:0/3578641969 learned_addr learned my addr 172.21.15.115:0/3578641969 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94b71d0b80 10 client.0 _ll_register_callbacks cb 0x5601e1e596c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.001+0000 7f94a6d57700 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e20c5400 0x5601e20e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94a5d55700 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1fc2d00 0x5601e214a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94a6556700 1 -- 172.21.15.115:0/3578641969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1fc2d00 msgr2=0x5601e214a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94a6556700 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1fc2d00 0x5601e214a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94a6556700 1 -- 172.21.15.115:0/3578641969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e20c5400 msgr2=0x5601e20e0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94a6556700 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e20c5400 0x5601e20e0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94a6556700 1 -- 172.21.15.115:0/3578641969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601e20ddae0 con 0x5601e1f484a0 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94a6556700 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e214a030 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f949c011040 tx=0x7f949c0ae7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac68249120d382f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f9489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f9489ffb700 1 -- 172.21.15.115:0/3578641969 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0ce070 con 0x5601e1f484a0 2024-05-09T05:41:08.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f9489ffb700 1 -- 172.21.15.115:0/3578641969 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f949c01d190 con 0x5601e1f484a0 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3578641969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949c01c070 con 0x5601e1f484a0 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f9489ffb700 1 -- 172.21.15.115:0/3578641969 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f949c0c6040 con 0x5601e1f484a0 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94b71d0b80 10 client.16452 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.002+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3578641969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5601e2151540 con 0x5601e1f484a0 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f9489ffb700 1 -- 172.21.15.115:0/3578641969 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f949c01c070 con 0x5601e1f484a0 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f9489ffb700 1 -- 172.21.15.115:0/3578641969 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f949c025070 con 0x5601e1f484a0 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f94b71d0b80 10 client.16452 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f94b71d0b80 10 client.16452 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f94b71d0b80 20 client.16452 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5601e2151940 0x5601e2153d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f94b71d0b80 4 client.16452 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3578641969 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601e1fefb20 con 0x5601e2151940 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f94a5d55700 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5601e2151940 0x5601e2153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:08.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.003+0000 7f9489ffb700 10 client.16452 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94a5d55700 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5601e2151940 0x5601e2153d60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac052a255e121cfb server_cookie=4d64f74389b859ff in_seq=0 out_seq=0 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f9489ffb700 1 -- 172.21.15.115:0/3578641969 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601e1fefb20 con 0x5601e2151940 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f9489ffb700 10 client.16452 handle_command_reply: tid=0 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 client.16452 shutdown 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3578641969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 msgr2=0x5601e214a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e214a030 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f949c011040 tx=0x7f949c0ae7a0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3578641969 shutdown_connections 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5601e2151940 0x5601e2153d60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f9489ffb700 0 client.16452 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601e20c5400 0x5601e20e0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601e1f484a0 0x5601e214a030 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 --2- 172.21.15.115:0/3578641969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601e1fc2d00 0x5601e214a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3578641969 >> 172.21.15.115:0/3578641969 conn(0x5601e1f4bf70 msgr2=0x5601e20d4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3578641969 shutdown_connections 2024-05-09T05:41:08.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.004+0000 7f94b71d0b80 1 -- 172.21.15.115:0/3578641969 wait complete. 2024-05-09T05:41:08.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94b71d0b80 20 client.16452 trim_cache size 0 max 16384 2024-05-09T05:41:08.004 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 -- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f948c0b2a70 msgr2=0x7f948c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f948c0b2a70 0x7f948c0b4f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94a006c2a0 tx=0x7f9490009260 comp rx=0 tx=0).stop 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 -- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 msgr2=0x7f94a019ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 0x7f94a019ad40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f949c006010 tx=0x7f949c004660 comp rx=0 tx=0).stop 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 -- 172.21.15.115:0/3795140081 shutdown_connections 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f948c0b2a70 0x7f948c0b4f80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94a0108740 0x7f94a019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a01034c0 0x7f94a019ad40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 --2- 172.21.15.115:0/3795140081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94a0107df0 0x7f94a019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 -- 172.21.15.115:0/3795140081 >> 172.21.15.115:0/3795140081 conn(0x7f94a00fedc0 msgr2=0x7f94a01063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:08.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 -- 172.21.15.115:0/3795140081 shutdown_connections 2024-05-09T05:41:08.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:08.005+0000 7f94a87ba700 1 -- 172.21.15.115:0/3795140081 wait complete. 2024-05-09T05:41:08.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:41:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:09.467 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:09 smithi115 ceph-mon[30349]: pgmap v91: 129 pgs: 129 active+clean; 328 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 27 MiB/s wr, 76 op/s 2024-05-09T05:41:09.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:09 smithi195 ceph-mon[33844]: pgmap v91: 129 pgs: 129 active+clean; 328 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 27 MiB/s wr, 76 op/s 2024-05-09T05:41:09.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:09 smithi161 ceph-mon[33805]: pgmap v91: 129 pgs: 129 active+clean; 328 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 27 MiB/s wr, 76 op/s 2024-05-09T05:41:11.453 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:11 smithi115 ceph-mon[30349]: pgmap v92: 129 pgs: 129 active+clean; 328 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 45 op/s 2024-05-09T05:41:11.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.459+0000 7f689daf0700 1 -- 172.21.15.115:0/2122858458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898103b30 msgr2=0x7f6898103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:11.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.459+0000 7f689daf0700 1 --2- 172.21.15.115:0/2122858458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898103b30 0x7f6898103f40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f68880099c0 tx=0x7f6888009cd0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.459+0000 7f689daf0700 1 -- 172.21.15.115:0/2122858458 shutdown_connections 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.459+0000 7f689daf0700 1 --2- 172.21.15.115:0/2122858458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68981087a0 0x7f6898110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.459+0000 7f689daf0700 1 --2- 172.21.15.115:0/2122858458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898103b30 0x7f6898103f40 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.459+0000 7f689daf0700 1 --2- 172.21.15.115:0/2122858458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6898107e50 0x7f6898108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.459+0000 7f689daf0700 1 -- 172.21.15.115:0/2122858458 >> 172.21.15.115:0/2122858458 conn(0x7f68980fe520 msgr2=0x7f6898100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.459+0000 7f689daf0700 1 -- 172.21.15.115:0/2122858458 shutdown_connections 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.459+0000 7f689daf0700 1 -- 172.21.15.115:0/2122858458 wait complete. 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.460+0000 7f689daf0700 1 Processor -- start 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.460+0000 7f689daf0700 1 -- start start 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f689daf0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6898103b30 0x7f689819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f689daf0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898107e50 0x7f689819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f689daf0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68981087a0 0x7f6898199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f689daf0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f68981132c0 con 0x7f6898107e50 2024-05-09T05:41:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f689daf0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6898113140 con 0x7f68981087a0 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f689daf0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6898113440 con 0x7f6898103b30 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6896ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898107e50 0x7f689819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6896ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898107e50 0x7f689819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50668/0 (socket says 172.21.15.115:50668) 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6896ffd700 1 -- 172.21.15.115:0/3956318060 learned_addr learned my addr 172.21.15.115:0/3956318060 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6896ffd700 1 -- 172.21.15.115:0/3956318060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6898103b30 msgr2=0x7f689819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6896ffd700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6898103b30 0x7f689819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6896ffd700 1 -- 172.21.15.115:0/3956318060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68981087a0 msgr2=0x7f6898199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6896ffd700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68981087a0 0x7f6898199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6896ffd700 1 -- 172.21.15.115:0/3956318060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689819a200 con 0x7f6898107e50 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6896ffd700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898107e50 0x7f689819fdf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f688c0098f0 tx=0x7f688c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.461+0000 7f6894ff9700 1 -- 172.21.15.115:0/3956318060 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c00d020 con 0x7f6898107e50 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.462+0000 7f689daf0700 1 -- 172.21.15.115:0/3956318060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689819a4f0 con 0x7f6898107e50 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.462+0000 7f6894ff9700 1 -- 172.21.15.115:0/3956318060 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f688c00d1c0 con 0x7f6898107e50 2024-05-09T05:41:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.462+0000 7f6894ff9700 1 -- 172.21.15.115:0/3956318060 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c00c790 con 0x7f6898107e50 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.462+0000 7f689daf0700 1 -- 172.21.15.115:0/3956318060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6898102790 con 0x7f6898107e50 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.465+0000 7f6894ff9700 1 -- 172.21.15.115:0/3956318060 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f688c00c930 con 0x7f6898107e50 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.465+0000 7f6894ff9700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68800b29f0 0x7f68800b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.465+0000 7f68977fe700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68800b29f0 0x7f68800b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f6894ff9700 1 -- 172.21.15.115:0/3956318060 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f688c0e3da0 con 0x7f6898107e50 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68ac506b80 1 Processor -- start 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68977fe700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68800b29f0 0x7f68800b4f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68880098e0 tx=0x7f6888000bc0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68ac506b80 1 -- start start 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 0x5576892aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892ae1b0 0x5576892ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68ac506b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5576891c80a0 con 0x5576891a2d00 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68ac506b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55768916d9a0 con 0x5576891284a0 2024-05-09T05:41:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68ac506b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5576891c4060 con 0x5576892ae1b0 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68977fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.466+0000 7f68977fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50674/0 (socket says 172.21.15.115:50674) 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f68977fe700 1 -- 172.21.15.115:0/1400491750 learned_addr learned my addr 172.21.15.115:0/1400491750 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f6897fff700 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892ae1b0 0x5576892ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f6896ffd700 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 0x5576892aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f68977fe700 1 -- 172.21.15.115:0/1400491750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892ae1b0 msgr2=0x5576892ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f68977fe700 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892ae1b0 0x5576892ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f68977fe700 1 -- 172.21.15.115:0/1400491750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 msgr2=0x5576892aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f68977fe700 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 0x5576892aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f68977fe700 1 -- 172.21.15.115:0/1400491750 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55768915e500 con 0x5576891a2d00 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f6897fff700 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892ae1b0 0x5576892ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f68977fe700 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892a9ab0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f68880052a0 tx=0x7f688801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a71f26a33fe5821 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f687dffb700 1 -- 172.21.15.115:0/1400491750 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68880055b0 con 0x5576891a2d00 2024-05-09T05:41:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f687dffb700 1 -- 172.21.15.115:0/1400491750 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f688800bc20 con 0x5576891a2d00 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.467+0000 7f687dffb700 1 -- 172.21.15.115:0/1400491750 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688802d620 con 0x5576891a2d00 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.468+0000 7f68ac506b80 1 -- 172.21.15.115:0/1400491750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 msgr2=0x5576892a9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.468+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892a9ab0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f68880052a0 tx=0x7f688801c000 comp rx=0 tx=0).stop 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.469+0000 7f68ac506b80 1 -- 172.21.15.115:0/1400491750 shutdown_connections 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.469+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892ae1b0 0x5576892ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.469+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892a9ab0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.469+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1400491750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 0x5576892aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.469+0000 7f68ac506b80 1 -- 172.21.15.115:0/1400491750 >> 172.21.15.115:0/1400491750 conn(0x55768912bf70 msgr2=0x5576892a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.469+0000 7f68ac506b80 1 -- 172.21.15.115:0/1400491750 shutdown_connections 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.469+0000 7f68ac506b80 1 -- 172.21.15.115:0/1400491750 wait complete. 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.469+0000 7f68ac506b80 1 -- start start 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.469+0000 7f68ac506b80 1 Processor -- start 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 0x5576892b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891a2d00 0x5576892b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f68ac506b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5576892aee70 con 0x5576892ae1b0 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f68ac506b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557689162b20 con 0x5576891a2d00 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f68ac506b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5576892aeff0 con 0x5576891284a0 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6897fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6896ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891a2d00 0x5576892b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6897fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50676/0 (socket says 172.21.15.115:50676) 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6897fff700 1 -- 172.21.15.115:0/3031734555 learned_addr learned my addr 172.21.15.115:0/3031734555 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f68ac506b80 10 client.0 _ll_register_callbacks cb 0x5576890396c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f68977fe700 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 0x5576892b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6897fff700 1 -- 172.21.15.115:0/3031734555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 msgr2=0x5576892b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6897fff700 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 0x5576892b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6897fff700 1 -- 172.21.15.115:0/3031734555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891a2d00 msgr2=0x5576892b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6897fff700 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891a2d00 0x5576892b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6897fff700 1 -- 172.21.15.115:0/3031734555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557689043490 con 0x5576892ae1b0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.470+0000 7f6896ffd700 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891a2d00 0x5576892b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f6897fff700 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b3f10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6890011cf0 tx=0x7f689000f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8811eea482767ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f6876ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f6876ffd700 1 -- 172.21.15.115:0/3031734555 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689000f730 con 0x5576892ae1b0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f6876ffd700 1 -- 172.21.15.115:0/3031734555 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f689000f8d0 con 0x5576892ae1b0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f68ac506b80 1 -- 172.21.15.115:0/3031734555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688800bc20 con 0x5576892ae1b0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f6876ffd700 1 -- 172.21.15.115:0/3031734555 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6890024050 con 0x5576892ae1b0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f68ac506b80 10 client.16464 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f68ac506b80 1 -- 172.21.15.115:0/3031734555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576892c0e10 con 0x5576892ae1b0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f6876ffd700 1 -- 172.21.15.115:0/3031734555 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f688800bc20 con 0x5576892ae1b0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.471+0000 7f6876ffd700 1 -- 172.21.15.115:0/3031734555 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f6890007c40 con 0x5576892ae1b0 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.472+0000 7f68ac506b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.472+0000 7f68ac506b80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.472+0000 7f68ac506b80 20 client.16464 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.472+0000 7f68ac506b80 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5576892c34a0 0x5576892c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.472+0000 7f68ac506b80 4 client.16464 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.472+0000 7f68ac506b80 1 -- 172.21.15.115:0/3031734555 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576891d5a80 con 0x5576892c34a0 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.472+0000 7f68977fe700 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5576892c34a0 0x5576892c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.472+0000 7f6876ffd700 10 client.16464 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.472+0000 7f68977fe700 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5576892c34a0 0x5576892c5930 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b35056fdf1d361e server_cookie=e3b44edf167ea52a in_seq=0 out_seq=0 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f6876ffd700 1 -- 172.21.15.115:0/3031734555 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5576891d5a80 con 0x5576892c34a0 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f6876ffd700 10 client.16464 handle_command_reply: tid=0 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f68ac506b80 1 client.16464 shutdown 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f68ac506b80 1 -- 172.21.15.115:0/3031734555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 msgr2=0x5576892b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f68ac506b80 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b3f10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6890011cf0 tx=0x7f689000f130 comp rx=0 tx=0).stop 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f68ac506b80 1 -- 172.21.15.115:0/3031734555 shutdown_connections 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f68ac506b80 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5576892c34a0 0x5576892c5930 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f68ac506b80 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b3f10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f68ac506b80 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 0x5576892b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f6876ffd700 0 client.16464 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f68ac506b80 1 --2- 172.21.15.115:0/3031734555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891a2d00 0x5576892b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.473+0000 7f68ac506b80 1 -- 172.21.15.115:0/3031734555 >> 172.21.15.115:0/3031734555 conn(0x55768912bf70 msgr2=0x5576892a43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.474+0000 7f68ac506b80 1 -- 172.21.15.115:0/3031734555 shutdown_connections 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.474+0000 7f68ac506b80 1 -- 172.21.15.115:0/3031734555 wait complete. 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.474+0000 7f68ac506b80 20 client.16464 trim_cache size 0 max 16384 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f68ac506b80 1 Processor -- start 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f68ac506b80 1 -- start start 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 0x5576892a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576892ae1b0 0x5576892cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f68ac506b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f68981ae1a0 con 0x5576891a2d00 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f68ac506b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68981afd40 con 0x5576892ae1b0 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f68ac506b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68981ae870 con 0x5576891284a0 2024-05-09T05:41:11.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f6896ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f6896ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50684/0 (socket says 172.21.15.115:50684) 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.495+0000 7f6896ffd700 1 -- 172.21.15.115:0/4238648945 learned_addr learned my addr 172.21.15.115:0/4238648945 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f68977fe700 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 0x5576892a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f6897fff700 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576892ae1b0 0x5576892cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f6896ffd700 1 -- 172.21.15.115:0/4238648945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 msgr2=0x5576892a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f6896ffd700 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 0x5576892a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f6896ffd700 1 -- 172.21.15.115:0/4238648945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576892ae1b0 msgr2=0x5576892cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f6896ffd700 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576892ae1b0 0x5576892cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f6896ffd700 1 -- 172.21.15.115:0/4238648945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55768929c9d0 con 0x5576891a2d00 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f68977fe700 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 0x5576892a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f6896ffd700 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892cb0f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f688c0aacd0 tx=0x7f688c00d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8601d135aa972ef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f687d7fa700 1 -- 172.21.15.115:0/4238648945 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c0af0f0 con 0x5576891a2d00 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f687d7fa700 1 -- 172.21.15.115:0/4238648945 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f688c0b3ba0 con 0x5576891a2d00 2024-05-09T05:41:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.496+0000 7f687d7fa700 1 -- 172.21.15.115:0/4238648945 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c0a6830 con 0x5576891a2d00 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.497+0000 7f68ac506b80 1 -- 172.21.15.115:0/4238648945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 msgr2=0x5576892cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.497+0000 7f68ac506b80 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892cb0f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f688c0aacd0 tx=0x7f688c00d850 comp rx=0 tx=0).stop 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.498+0000 7f68ac506b80 1 -- 172.21.15.115:0/4238648945 shutdown_connections 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.498+0000 7f68ac506b80 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576892ae1b0 0x5576892cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.498+0000 7f68ac506b80 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576891284a0 0x5576892a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.498+0000 7f68ac506b80 1 --2- 172.21.15.115:0/4238648945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576891a2d00 0x5576892cb0f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.498+0000 7f68ac506b80 1 -- 172.21.15.115:0/4238648945 >> 172.21.15.115:0/4238648945 conn(0x55768912bf70 msgr2=0x55768929a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.498+0000 7f68ac506b80 1 -- 172.21.15.115:0/4238648945 shutdown_connections 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.498+0000 7f68ac506b80 1 -- 172.21.15.115:0/4238648945 wait complete. 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.498+0000 7f68ac506b80 1 -- start start 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.498+0000 7f68ac506b80 1 Processor -- start 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 0x5576892bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f68ac506b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892b9250 0x5576892b9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f68ac506b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f68981af610 con 0x5576892ae1b0 2024-05-09T05:41:11.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f68ac506b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68981aef40 con 0x5576891284a0 2024-05-09T05:41:11.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f68ac506b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68981ac140 con 0x5576892b9250 2024-05-09T05:41:11.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f68ac506b80 10 client.0 _ll_register_callbacks cb 0x5576890396c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:11.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6896ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6896ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50700/0 (socket says 172.21.15.115:50700) 2024-05-09T05:41:11.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6896ffd700 1 -- 172.21.15.115:0/1435974040 learned_addr learned my addr 172.21.15.115:0/1435974040 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:11.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f68977fe700 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 0x5576892bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.519 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6897fff700 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892b9250 0x5576892b9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6896ffd700 1 -- 172.21.15.115:0/1435974040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892b9250 msgr2=0x5576892b9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6896ffd700 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892b9250 0x5576892b9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6896ffd700 1 -- 172.21.15.115:0/1435974040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 msgr2=0x5576892bb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6896ffd700 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 0x5576892bb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6896ffd700 1 -- 172.21.15.115:0/1435974040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55768932ccb0 con 0x5576892ae1b0 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6896ffd700 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b8d10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f688c0e1260 tx=0x7f688c0aae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e43916b12f3d556 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.499+0000 7f6876ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f6876ffd700 1 -- 172.21.15.115:0/1435974040 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c0ca060 con 0x5576892ae1b0 2024-05-09T05:41:11.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f6876ffd700 1 -- 172.21.15.115:0/1435974040 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f688c0a7330 con 0x5576892ae1b0 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f68ac506b80 1 -- 172.21.15.115:0/1435974040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688c0b3ba0 con 0x5576892ae1b0 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f6876ffd700 1 -- 172.21.15.115:0/1435974040 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f688c0af0f0 con 0x5576892ae1b0 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f68ac506b80 10 client.16470 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f68ac506b80 1 -- 172.21.15.115:0/1435974040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55768932de60 con 0x5576892ae1b0 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f6876ffd700 1 -- 172.21.15.115:0/1435974040 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f688c0b3ba0 con 0x5576892ae1b0 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f6876ffd700 1 -- 172.21.15.115:0/1435974040 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f688c0a9aa0 con 0x5576892ae1b0 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f68ac506b80 10 client.16470 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f68ac506b80 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f68ac506b80 20 client.16470 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55768932e400 0x557689330820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f68ac506b80 4 client.16470 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.500+0000 7f68ac506b80 1 -- 172.21.15.115:0/1435974040 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576891cfb20 con 0x55768932e400 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.501+0000 7f68977fe700 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55768932e400 0x557689330820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.501+0000 7f6876ffd700 10 client.16470 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:11.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.501+0000 7f68977fe700 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55768932e400 0x557689330820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952cdce79404b5fa server_cookie=6a3de01edf945b2f in_seq=0 out_seq=0 2024-05-09T05:41:11.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.501+0000 7f6876ffd700 1 -- 172.21.15.115:0/1435974040 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576891cfb20 con 0x55768932e400 2024-05-09T05:41:11.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.501+0000 7f6876ffd700 10 client.16470 handle_command_reply: tid=0 2024-05-09T05:41:11.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.501+0000 7f68ac506b80 1 client.16470 shutdown 2024-05-09T05:41:11.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 -- 172.21.15.115:0/1435974040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 msgr2=0x5576892b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:11.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b8d10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f688c0e1260 tx=0x7f688c0aae80 comp rx=0 tx=0).stop 2024-05-09T05:41:11.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 -- 172.21.15.115:0/1435974040 shutdown_connections 2024-05-09T05:41:11.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576891284a0 0x5576892bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55768932e400 0x557689330820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576892b9250 0x5576892b9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f6876ffd700 0 client.16470 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:11.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 --2- 172.21.15.115:0/1435974040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5576892ae1b0 0x5576892b8d10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 -- 172.21.15.115:0/1435974040 >> 172.21.15.115:0/1435974040 conn(0x55768912bf70 msgr2=0x55768929a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:11.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 -- 172.21.15.115:0/1435974040 shutdown_connections 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 1 -- 172.21.15.115:0/1435974040 wait complete. 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.502+0000 7f68ac506b80 20 client.16470 trim_cache size 0 max 16384 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 -- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68800b29f0 msgr2=0x7f68800b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68800b29f0 0x7f68800b4f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68880098e0 tx=0x7f6888000bc0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 -- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898107e50 msgr2=0x7f689819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898107e50 0x7f689819fdf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f688c0098f0 tx=0x7f688c009cb0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 -- 172.21.15.115:0/3956318060 shutdown_connections 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68800b29f0 0x7f68800b4f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68981087a0 0x7f6898199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6898103b30 0x7f689819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 --2- 172.21.15.115:0/3956318060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6898107e50 0x7f689819fdf0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 -- 172.21.15.115:0/3956318060 >> 172.21.15.115:0/3956318060 conn(0x7f68980fe520 msgr2=0x7f6898106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 -- 172.21.15.115:0/3956318060 shutdown_connections 2024-05-09T05:41:11.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:11.503+0000 7f689daf0700 1 -- 172.21.15.115:0/3956318060 wait complete. 2024-05-09T05:41:11.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:11 smithi195 ceph-mon[33844]: pgmap v92: 129 pgs: 129 active+clean; 328 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 45 op/s 2024-05-09T05:41:11.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:11 smithi161 ceph-mon[33805]: pgmap v92: 129 pgs: 129 active+clean; 328 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 45 op/s 2024-05-09T05:41:13.366 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:41:13.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:41:13.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:13 smithi195 ceph-mon[33844]: pgmap v93: 129 pgs: 129 active+clean; 328 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 45 op/s 2024-05-09T05:41:13.640 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:13.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:13 smithi115 ceph-mon[30349]: pgmap v93: 129 pgs: 129 active+clean; 328 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 45 op/s 2024-05-09T05:41:13.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:13 smithi161 ceph-mon[33805]: pgmap v93: 129 pgs: 129 active+clean; 328 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 45 op/s 2024-05-09T05:41:15.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:15 smithi115 ceph-mon[30349]: pgmap v94: 129 pgs: 129 active+clean; 464 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 MiB/s wr, 54 op/s 2024-05-09T05:41:15.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.557+0000 7f02a40b3700 1 -- 172.21.15.115:0/703374861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 msgr2=0x7f029c110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:15.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.557+0000 7f02a40b3700 1 --2- 172.21.15.115:0/703374861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 0x7f029c110150 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f029400b0d0 tx=0x7f029400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.557+0000 7f02a40b3700 1 -- 172.21.15.115:0/703374861 shutdown_connections 2024-05-09T05:41:15.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.557+0000 7f02a40b3700 1 --2- 172.21.15.115:0/703374861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 0x7f029c110150 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.557+0000 7f02a40b3700 1 --2- 172.21.15.115:0/703374861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029c103ca0 0x7f029c1040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.557+0000 7f02a40b3700 1 --2- 172.21.15.115:0/703374861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c107e10 0x7f029c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.557+0000 7f02a40b3700 1 -- 172.21.15.115:0/703374861 >> 172.21.15.115:0/703374861 conn(0x7f029c0fe6f0 msgr2=0x7f029c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:15.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.557+0000 7f02a40b3700 1 -- 172.21.15.115:0/703374861 shutdown_connections 2024-05-09T05:41:15.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.557+0000 7f02a40b3700 1 -- 172.21.15.115:0/703374861 wait complete. 2024-05-09T05:41:15.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.558+0000 7f02a40b3700 1 Processor -- start 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.558+0000 7f02a40b3700 1 -- start start 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.558+0000 7f02a40b3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029c103ca0 0x7f029c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.558+0000 7f02a40b3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c107e10 0x7f029c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.558+0000 7f02a40b3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 0x7f029c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.558+0000 7f02a40b3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f029c113280 con 0x7f029c108760 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.558+0000 7f02a40b3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f029c113100 con 0x7f029c107e10 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.558+0000 7f02a40b3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f029c113400 con 0x7f029c103ca0 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a2650700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 0x7f029c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a2650700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 0x7f029c19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51476/0 (socket says 172.21.15.115:51476) 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a2650700 1 -- 172.21.15.115:0/2365232490 learned_addr learned my addr 172.21.15.115:0/2365232490 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029c103ca0 0x7f029c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a164e700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c107e10 0x7f029c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a2650700 1 -- 172.21.15.115:0/2365232490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029c103ca0 msgr2=0x7f029c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a2650700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029c103ca0 0x7f029c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a2650700 1 -- 172.21.15.115:0/2365232490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c107e10 msgr2=0x7f029c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a2650700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c107e10 0x7f029c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a2650700 1 -- 172.21.15.115:0/2365232490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029c19fb60 con 0x7f029c108760 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a2650700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 0x7f029c19f570 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0294007d10 tx=0x7f0294007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f0292ffd700 1 -- 172.21.15.115:0/2365232490 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029401f070 con 0x7f029c108760 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f02a40b3700 1 -- 172.21.15.115:0/2365232490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029c19fdf0 con 0x7f029c108760 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f0292ffd700 1 -- 172.21.15.115:0/2365232490 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0294003e10 con 0x7f029c108760 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.559+0000 7f0292ffd700 1 -- 172.21.15.115:0/2365232490 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029401b720 con 0x7f029c108760 2024-05-09T05:41:15.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.560+0000 7f02a40b3700 1 -- 172.21.15.115:0/2365232490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029c1a02d0 con 0x7f029c108760 2024-05-09T05:41:15.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.563+0000 7f0292ffd700 1 -- 172.21.15.115:0/2365232490 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0294022070 con 0x7f029c108760 2024-05-09T05:41:15.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.564+0000 7f0292ffd700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f02880b29f0 0x7f02880b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.564+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f02880b29f0 0x7f02880b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.564+0000 7f0292ffd700 1 -- 172.21.15.115:0/2365232490 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f029400f020 con 0x7f029c108760 2024-05-09T05:41:15.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.564+0000 7f02b2ac9b80 1 Processor -- start 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f02880b29f0 0x7f02880b4f00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f029800ac50 tx=0x7f029800a380 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02b2ac9b80 1 -- start start 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c19a8d80 0x55c5c1ab12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1ab1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 0x55c5c1aab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c5c19ce120 con 0x55c5c19a8d80 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5c1973a20 con 0x55c5c192e520 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c5c19ca0e0 con 0x55c5c1aab480 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c19a8d80 0x55c5c1ab12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c19a8d80 0x55c5c1ab12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51492/0 (socket says 172.21.15.115:51492) 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 -- 172.21.15.115:0/1244825513 learned_addr learned my addr 172.21.15.115:0/1244825513 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:15.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a164e700 1 --2- 172.21.15.115:0/1244825513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1ab1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 -- 172.21.15.115:0/1244825513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 msgr2=0x55c5c1aab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:15.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/1244825513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 0x55c5c1aab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 -- 172.21.15.115:0/1244825513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 msgr2=0x55c5c1ab1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:15.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/1244825513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1ab1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 -- 172.21.15.115:0/1244825513 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c1964580 con 0x55c5c19a8d80 2024-05-09T05:41:15.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.565+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/1244825513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c19a8d80 0x55c5c1ab12c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f029800ff90 tx=0x7f0298012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8937076ed2b809bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:15.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.566+0000 7f0287fff700 1 -- 172.21.15.115:0/1244825513 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298016070 con 0x55c5c19a8d80 2024-05-09T05:41:15.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.566+0000 7f0287fff700 1 -- 172.21.15.115:0/1244825513 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0298014070 con 0x55c5c19a8d80 2024-05-09T05:41:15.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.566+0000 7f0287fff700 1 -- 172.21.15.115:0/1244825513 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298010e40 con 0x55c5c19a8d80 2024-05-09T05:41:15.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/1244825513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c19a8d80 msgr2=0x55c5c1ab12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/1244825513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c19a8d80 0x55c5c1ab12c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f029800ff90 tx=0x7f0298012000 comp rx=0 tx=0).stop 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/1244825513 shutdown_connections 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/1244825513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 0x55c5c1aab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/1244825513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c19a8d80 0x55c5c1ab12c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/1244825513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1ab1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/1244825513 >> 172.21.15.115:0/1244825513 conn(0x55c5c1931ff0 msgr2=0x55c5c1aa62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/1244825513 shutdown_connections 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/1244825513 wait complete. 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.567+0000 7f02b2ac9b80 1 -- start start 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02b2ac9b80 1 Processor -- start 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c192e520 0x55c5c1ab5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1ac12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c5c1aac1e0 con 0x55c5c1aab480 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02a164e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5c1968ba0 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c5c1ab80a0 con 0x55c5c192e520 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02a164e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51494/0 (socket says 172.21.15.115:51494) 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02a164e700 1 -- 172.21.15.115:0/629922656 learned_addr learned my addr 172.21.15.115:0/629922656 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02b2ac9b80 10 client.0 _ll_register_callbacks cb 0x55c5c183f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02a2650700 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1ac12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.568+0000 7f02a1e4f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c192e520 0x55c5c1ab5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02a164e700 1 -- 172.21.15.115:0/629922656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c192e520 msgr2=0x55c5c1ab5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02a164e700 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c192e520 0x55c5c1ab5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02a164e700 1 -- 172.21.15.115:0/629922656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c1ab5cf0 msgr2=0x55c5c1ac12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02a164e700 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1ac12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02a164e700 1 -- 172.21.15.115:0/629922656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c1849510 con 0x55c5c1aab480 2024-05-09T05:41:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02a164e700 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab57b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f028c016ae0 tx=0x7f028c016df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878bb193aa4225b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02857fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02857fa700 1 -- 172.21.15.115:0/629922656 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c00be10 con 0x55c5c1aab480 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/629922656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0298014070 con 0x55c5c1aab480 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.569+0000 7f02857fa700 1 -- 172.21.15.115:0/629922656 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f028c016170 con 0x55c5c1aab480 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02857fa700 1 -- 172.21.15.115:0/629922656 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f028c023050 con 0x55c5c1aab480 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02b2ac9b80 10 client.16482 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/629922656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c5c1ac31a0 con 0x55c5c1aab480 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02857fa700 1 -- 172.21.15.115:0/629922656 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0298014070 con 0x55c5c1aab480 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02857fa700 1 -- 172.21.15.115:0/629922656 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f028c011440 con 0x55c5c1aab480 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02b2ac9b80 10 client.16482 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02b2ac9b80 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02b2ac9b80 20 client.16482 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c5c1ac5bf0 0x55c5c1ac80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02b2ac9b80 4 client.16482 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.570+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/629922656 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5c19dbb00 con 0x55c5c1ac5bf0 2024-05-09T05:41:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.571+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c5c1ac5bf0 0x55c5c1ac80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.571+0000 7f02857fa700 10 client.16482 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.571+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c5c1ac5bf0 0x55c5c1ac80a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120c47cd004afc6c server_cookie=3942da6b4318e39c in_seq=0 out_seq=0 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02857fa700 1 -- 172.21.15.115:0/629922656 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c5c19dbb00 con 0x55c5c1ac5bf0 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02857fa700 10 client.16482 handle_command_reply: tid=0 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02b2ac9b80 1 client.16482 shutdown 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/629922656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 msgr2=0x55c5c1ab57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab57b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f028c016ae0 tx=0x7f028c016df0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/629922656 shutdown_connections 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1ac12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c5c1ac5bf0 0x55c5c1ac80a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c192e520 0x55c5c1ab5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/629922656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab57b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.572+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/629922656 >> 172.21.15.115:0/629922656 conn(0x55c5c1931ff0 msgr2=0x55c5c1aa0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.573+0000 7f02857fa700 0 client.16482 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.573+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/629922656 shutdown_connections 2024-05-09T05:41:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.573+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/629922656 wait complete. 2024-05-09T05:41:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.573+0000 7f02b2ac9b80 20 client.16482 trim_cache size 0 max 16384 2024-05-09T05:41:15.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02b2ac9b80 1 Processor -- start 2024-05-09T05:41:15.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02b2ac9b80 1 -- start start 2024-05-09T05:41:15.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1aaf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1aaaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f029c1adfd0 con 0x55c5c1aab480 2024-05-09T05:41:15.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f029c1afb70 con 0x55c5c192e520 2024-05-09T05:41:15.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f029c1ae6a0 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02a164e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02a164e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51500/0 (socket says 172.21.15.115:51500) 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02a164e700 1 -- 172.21.15.115:0/285985846 learned_addr learned my addr 172.21.15.115:0/285985846 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1aaf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02a2650700 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1aaaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.602+0000 7f02a164e700 1 -- 172.21.15.115:0/285985846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1ab5cf0 msgr2=0x55c5c1aaaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.603+0000 7f02a164e700 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1aaaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.603+0000 7f02a164e700 1 -- 172.21.15.115:0/285985846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 msgr2=0x55c5c1aaf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.603+0000 7f02a164e700 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1aaf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.603+0000 7f02a164e700 1 -- 172.21.15.115:0/285985846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c1aa2a50 con 0x55c5c1aab480 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.603+0000 7f02a2650700 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1aaaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.603+0000 7f02a164e700 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab2ef0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f028c005950 tx=0x7f028c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97efe89faf6585a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.603+0000 7f02877fe700 1 -- 172.21.15.115:0/285985846 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c02b030 con 0x55c5c1aab480 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.603+0000 7f02877fe700 1 -- 172.21.15.115:0/285985846 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f028c015070 con 0x55c5c1aab480 2024-05-09T05:41:15.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.603+0000 7f02877fe700 1 -- 172.21.15.115:0/285985846 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c005040 con 0x55c5c1aab480 2024-05-09T05:41:15.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.604+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/285985846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 msgr2=0x55c5c1ab2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.604+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab2ef0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f028c005950 tx=0x7f028c011440 comp rx=0 tx=0).stop 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.604+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/285985846 shutdown_connections 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.604+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1aaf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.604+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1aaaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.604+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/285985846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1aab480 0x55c5c1ab2ef0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.604+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/285985846 >> 172.21.15.115:0/285985846 conn(0x55c5c1931ff0 msgr2=0x55c5c1aa06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.604+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/285985846 shutdown_connections 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.604+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/285985846 wait complete. 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02b2ac9b80 1 -- start start 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02b2ac9b80 1 Processor -- start 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1ac7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 0x55c5c1ac78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02b2ac9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1b28340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f029c1af440 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f029c1aed70 con 0x55c5c192e520 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02b2ac9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f029c1abf70 con 0x55c5c1aab480 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02b2ac9b80 10 client.0 _ll_register_callbacks cb 0x55c5c183f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02a2650700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1b28340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02a164e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 0x55c5c1ac78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.605+0000 7f02a164e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 0x55c5c1ac78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:56786/0 (socket says 172.21.15.115:56786) 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02a164e700 1 -- 172.21.15.115:0/3862976287 learned_addr learned my addr 172.21.15.115:0/3862976287 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1ac7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02a2650700 1 -- 172.21.15.115:0/3862976287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 msgr2=0x55c5c1ac78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02a2650700 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 0x55c5c1ac78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02a2650700 1 -- 172.21.15.115:0/3862976287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 msgr2=0x55c5c1ac7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02a2650700 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1ac7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02a2650700 1 -- 172.21.15.115:0/3862976287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c1ac2fd0 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1ac7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02a2650700 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1b28340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f02940e11f0 tx=0x7f02940aaa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161757ccd4ada092 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02857fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02857fa700 1 -- 172.21.15.115:0/3862976287 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02940b3be0 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02857fa700 1 -- 172.21.15.115:0/3862976287 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f02940a7400 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/3862976287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f028c015070 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02857fa700 1 -- 172.21.15.115:0/3862976287 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f02940c5450 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02b2ac9b80 10 client.16488 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.606+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/3862976287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c5c1b30c20 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.607+0000 7f02857fa700 1 -- 172.21.15.115:0/3862976287 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f028c015070 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.607+0000 7f02857fa700 1 -- 172.21.15.115:0/3862976287 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f029408dce0 con 0x55c5c1ab5cf0 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.607+0000 7f02b2ac9b80 10 client.16488 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.607+0000 7f02b2ac9b80 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.607+0000 7f02b2ac9b80 20 client.16488 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:15.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.607+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c5c1b31360 0x55c5c1b33780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.607+0000 7f02b2ac9b80 4 client.16488 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.607+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/3862976287 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c5c19d5ba0 con 0x55c5c1b31360 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.607+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c5c1b31360 0x55c5c1b33780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.608+0000 7f02857fa700 10 client.16488 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.608+0000 7f02a1e4f700 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c5c1b31360 0x55c5c1b33780 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b81bea99813e213 server_cookie=9d5cbc4d712c83dc in_seq=0 out_seq=0 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.608+0000 7f02857fa700 1 -- 172.21.15.115:0/3862976287 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c5c19d5ba0 con 0x55c5c1b31360 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.608+0000 7f02857fa700 10 client.16488 handle_command_reply: tid=0 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.608+0000 7f02b2ac9b80 1 client.16488 shutdown 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/3862976287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1ab5cf0 msgr2=0x55c5c1b28340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1b28340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f02940e11f0 tx=0x7f02940aaa10 comp rx=0 tx=0).stop 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/3862976287 shutdown_connections 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5c192e520 0x55c5c1ac7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c5c1b31360 0x55c5c1b33780 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c5c1ab5cf0 0x55c5c1b28340 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 --2- 172.21.15.115:0/3862976287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c5c1aab480 0x55c5c1ac78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02857fa700 0 client.16488 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/3862976287 >> 172.21.15.115:0/3862976287 conn(0x55c5c1931ff0 msgr2=0x55c5c1ab4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/3862976287 shutdown_connections 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 1 -- 172.21.15.115:0/3862976287 wait complete. 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.609+0000 7f02b2ac9b80 20 client.16488 trim_cache size 0 max 16384 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:41:15.607 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "d6f13d18-3fd5-4c50-9e13-12ffd90ae9a8", 2024-05-09T05:41:15.608 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:41:15.608 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:41:15.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 -- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f02880b29f0 msgr2=0x7f02880b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:15.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f02880b29f0 0x7f02880b4f00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f029800ac50 tx=0x7f029800a380 comp rx=0 tx=0).stop 2024-05-09T05:41:15.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 -- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 msgr2=0x7f029c19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:15.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 0x7f029c19f570 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0294007d10 tx=0x7f0294007d40 comp rx=0 tx=0).stop 2024-05-09T05:41:15.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 -- 172.21.15.115:0/2365232490 shutdown_connections 2024-05-09T05:41:15.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f02880b29f0 0x7f02880b4f00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f029c108760 0x7f029c19f570 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029c103ca0 0x7f029c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 --2- 172.21.15.115:0/2365232490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c107e10 0x7f029c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 -- 172.21.15.115:0/2365232490 >> 172.21.15.115:0/2365232490 conn(0x7f029c0fe6f0 msgr2=0x7f029c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 -- 172.21.15.115:0/2365232490 shutdown_connections 2024-05-09T05:41:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:15.610+0000 7f02a40b3700 1 -- 172.21.15.115:0/2365232490 wait complete. 2024-05-09T05:41:15.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:15 smithi195 ceph-mon[33844]: pgmap v94: 129 pgs: 129 active+clean; 464 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 MiB/s wr, 54 op/s 2024-05-09T05:41:15.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:15 smithi161 ceph-mon[33805]: pgmap v94: 129 pgs: 129 active+clean; 464 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 MiB/s wr, 54 op/s 2024-05-09T05:41:16.115 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:41:15 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T05:41:15.621+0000 7f9d69520700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c7 [...5,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ auth{1=1} v3945 ap=2 f(v36 m2024-05-09T05:40:55.664108+0000 18=18+0) n(v1 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1) old_inodes=1 (inest sync->mix g=1) caps={15339=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55d6cd1f4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c7:[<0x10000000007/.deps v2123>,<0x10000000006/ffsb v3080>,<0x10000000005/tmp v44>,<0x10000000003/client.0 v40>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v56>,<0x10000000001/sv_0 v71>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c7:[<0x10000000007/.deps v3945>,<0x10000000006/ffsb v3496>,<0x10000000005/tmp v76>,<0x10000000003/client.0 v67>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v86>,<0x10000000001/sv_0 v101>,<0x10000000000/qa v86>,<0x1/volumes v80>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T05:40:55.643149+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1)","memory_value.dirstat":"f(v36 m2024-05-09T05:40:55.664108+0000 18=18+0)","memory_value.rstat":"n(v1 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T05:41:16.535 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:41:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:17.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:41:17.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:41:17.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:41:17.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:17 smithi195 ceph-mon[33844]: pgmap v95: 129 pgs: 129 active+clean; 464 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-05-09T05:41:17.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:41:17.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:41:17.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:41:17.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:17 smithi115 ceph-mon[30349]: pgmap v95: 129 pgs: 129 active+clean; 464 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:41:17.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:41:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:41:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:41:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:41:17.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:17 smithi161 ceph-mon[33805]: pgmap v95: 129 pgs: 129 active+clean; 464 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-05-09T05:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.895+0000 7fc182ad8640 1 -- 172.21.15.115:0/3140051492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c108bb0 msgr2=0x7fc17c102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.895+0000 7fc182ad8640 1 --2- 172.21.15.115:0/3140051492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c108bb0 0x7fc17c102750 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc170009900 tx=0x7fc17002a410 comp rx=0 tx=0).stop 2024-05-09T05:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.895+0000 7fc182ad8640 1 -- 172.21.15.115:0/3140051492 shutdown_connections 2024-05-09T05:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.895+0000 7fc182ad8640 1 --2- 172.21.15.115:0/3140051492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc17c103300 0x7fc17c10f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.895+0000 7fc182ad8640 1 --2- 172.21.15.115:0/3140051492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17c109050 0x7fc17c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.895+0000 7fc182ad8640 1 --2- 172.21.15.115:0/3140051492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c108bb0 0x7fc17c102750 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.895+0000 7fc182ad8640 1 -- 172.21.15.115:0/3140051492 >> 172.21.15.115:0/3140051492 conn(0x7fc17c0792a0 msgr2=0x7fc17c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.895+0000 7fc182ad8640 1 -- 172.21.15.115:0/3140051492 shutdown_connections 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.895+0000 7fc182ad8640 1 -- 172.21.15.115:0/3140051492 wait complete. 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc182ad8640 1 Processor -- start 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc182ad8640 1 -- start start 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc182ad8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17c103300 0x7fc17c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc182ad8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc17c108bb0 0x7fc17c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc182ad8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c109050 0x7fc17c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc182ad8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c112c50 con 0x7fc17c109050 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc182ad8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c112ad0 con 0x7fc17c108bb0 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc182ad8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c112dd0 con 0x7fc17c103300 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc181075640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c109050 0x7fc17c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc181075640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c109050 0x7fc17c19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51524/0 (socket says 172.21.15.115:51524) 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc181075640 1 -- 172.21.15.115:0/993559692 learned_addr learned my addr 172.21.15.115:0/993559692 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc180874640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17c103300 0x7fc17c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.896+0000 7fc17bfff640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc17c108bb0 0x7fc17c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc181075640 1 -- 172.21.15.115:0/993559692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17c103300 msgr2=0x7fc17c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc181075640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17c103300 0x7fc17c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc181075640 1 -- 172.21.15.115:0/993559692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc17c108bb0 msgr2=0x7fc17c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc181075640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc17c108bb0 0x7fc17c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc181075640 1 -- 172.21.15.115:0/993559692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c19e5b0 con 0x7fc17c109050 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc180874640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17c103300 0x7fc17c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc181075640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c109050 0x7fc17c19df70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc174009fd0 tx=0x7fc17400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc179ffb640 1 -- 172.21.15.115:0/993559692 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174017070 con 0x7fc17c109050 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc179ffb640 1 -- 172.21.15.115:0/993559692 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc174003e00 con 0x7fc17c109050 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc182ad8640 1 -- 172.21.15.115:0/993559692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17c19e8a0 con 0x7fc17c109050 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc179ffb640 1 -- 172.21.15.115:0/993559692 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174010460 con 0x7fc17c109050 2024-05-09T05:41:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.897+0000 7fc182ad8640 1 -- 172.21.15.115:0/993559692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17c1a6220 con 0x7fc17c109050 2024-05-09T05:41:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.900+0000 7fc189989740 1 Processor -- start 2024-05-09T05:41:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.900+0000 7fc179ffb640 1 -- 172.21.15.115:0/993559692 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc174003970 con 0x7fc17c109050 2024-05-09T05:41:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.900+0000 7fc189989740 1 -- start start 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc189989740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc4d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc189989740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc189989740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8894d0 0x55d6ecc4df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc189989740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d6ecbb27c0 con 0x55d6ec8894d0 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc189989740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d6ecb4e0e0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc189989740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d6ecbae780 con 0x55d6ec8812f0 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8894d0 0x55d6ecc4df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8894d0 0x55d6ecc4df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51534/0 (socket says 172.21.15.115:51534) 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 -- 172.21.15.115:0/3427500684 learned_addr learned my addr 172.21.15.115:0/3427500684 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 --2- 172.21.15.115:0/3427500684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc4d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc180874640 1 --2- 172.21.15.115:0/3427500684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 -- 172.21.15.115:0/3427500684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 msgr2=0x55d6ecc4da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 --2- 172.21.15.115:0/3427500684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc4da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 -- 172.21.15.115:0/3427500684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 msgr2=0x55d6ecc4d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 --2- 172.21.15.115:0/3427500684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc4d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 -- 172.21.15.115:0/3427500684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ecb777c0 con 0x55d6ec8894d0 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.901+0000 7fc17bfff640 1 --2- 172.21.15.115:0/3427500684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8894d0 0x55d6ecc4df50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc1680116f0 tx=0x7fc168004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00af0d3bc693d86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.902+0000 7fc179ffb640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc16c0b29d0 0x7fc16c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.902+0000 7fc180874640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc16c0b29d0 0x7fc16c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.902+0000 7fc166ffd640 1 -- 172.21.15.115:0/3427500684 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168005020 con 0x55d6ec8894d0 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.902+0000 7fc166ffd640 1 -- 172.21.15.115:0/3427500684 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1680047a0 con 0x55d6ec8894d0 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.902+0000 7fc166ffd640 1 -- 172.21.15.115:0/3427500684 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168005650 con 0x55d6ec8894d0 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.902+0000 7fc179ffb640 1 -- 172.21.15.115:0/993559692 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc1740e29c0 con 0x7fc17c109050 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 -- 172.21.15.115:0/3427500684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8894d0 msgr2=0x55d6ecc4df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc180874640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc16c0b29d0 0x7fc16c0b4ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc170004770 tx=0x7fc1700058d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 --2- 172.21.15.115:0/3427500684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8894d0 0x55d6ecc4df50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc1680116f0 tx=0x7fc168004730 comp rx=0 tx=0).stop 2024-05-09T05:41:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 -- 172.21.15.115:0/3427500684 shutdown_connections 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 --2- 172.21.15.115:0/3427500684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8894d0 0x55d6ecc4df50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 --2- 172.21.15.115:0/3427500684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc4da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 --2- 172.21.15.115:0/3427500684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc4d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 -- 172.21.15.115:0/3427500684 >> 172.21.15.115:0/3427500684 conn(0x55d6ec89e300 msgr2=0x55d6ecc49e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 -- 172.21.15.115:0/3427500684 shutdown_connections 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 -- 172.21.15.115:0/3427500684 wait complete. 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.903+0000 7fc189989740 1 -- start start 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc189989740 1 Processor -- start 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc189989740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc5b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc189989740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 0x55d6ecc5bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc189989740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc59dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc189989740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d6ecc5ca20 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc189989740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d6ecb399c0 con 0x55d6ec8894d0 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc189989740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d6ecc5cba0 con 0x55d6ec8812f0 2024-05-09T05:41:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc189989740 10 client.0 _ll_register_callbacks cb 0x55d6ecb66020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc181075640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc59dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc181075640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc59dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51550/0 (socket says 172.21.15.115:51550) 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc181075640 1 -- 172.21.15.115:0/2365374955 learned_addr learned my addr 172.21.15.115:0/2365374955 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc17bfff640 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 0x55d6ecc5bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.904+0000 7fc180874640 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc5b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc181075640 1 -- 172.21.15.115:0/2365374955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 msgr2=0x55d6ecc5b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc181075640 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc5b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc181075640 1 -- 172.21.15.115:0/2365374955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 msgr2=0x55d6ecc5bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc181075640 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 0x55d6ecc5bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc181075640 1 -- 172.21.15.115:0/2365374955 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ecb601f0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc181075640 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc59dc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc1740a99a0 tx=0x7fc1740e02d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56d28a8cd5f0324 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc15bfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc15bfff640 1 -- 172.21.15.115:0/2365374955 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174020070 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc15bfff640 1 -- 172.21.15.115:0/2365374955 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc1740cc0d0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc189989740 1 -- 172.21.15.115:0/2365374955 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1680047a0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc15bfff640 1 -- 172.21.15.115:0/2365374955 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1740a9520 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc189989740 10 client.16500 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc189989740 1 -- 172.21.15.115:0/2365374955 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d6ec915790 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.905+0000 7fc15bfff640 1 -- 172.21.15.115:0/2365374955 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1680047a0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc15bfff640 1 -- 172.21.15.115:0/2365374955 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc1740b2330 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc189989740 10 client.16500 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc189989740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc189989740 20 client.16500 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc189989740 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6ec868d80 0x55d6ecc6c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc189989740 4 client.16500 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc189989740 1 -- 172.21.15.115:0/2365374955 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6ecbc03d0 con 0x55d6ec868d80 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc180874640 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6ec868d80 0x55d6ecc6c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc15bfff640 10 client.16500 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.906+0000 7fc180874640 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6ec868d80 0x55d6ecc6c1e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7ce4c2b12d6a62 server_cookie=d70ceb414621f283 in_seq=0 out_seq=0 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc15bfff640 1 -- 172.21.15.115:0/2365374955 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6ecbc03d0 con 0x55d6ec868d80 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc15bfff640 10 client.16500 handle_command_reply: tid=0 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 client.16500 shutdown 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 -- 172.21.15.115:0/2365374955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 msgr2=0x55d6ecc59dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc59dc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc1740a99a0 tx=0x7fc1740e02d0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 -- 172.21.15.115:0/2365374955 shutdown_connections 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc59dc0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 0x55d6ecc5bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6ec868d80 0x55d6ecc6c1e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 --2- 172.21.15.115:0/2365374955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc5b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 -- 172.21.15.115:0/2365374955 >> 172.21.15.115:0/2365374955 conn(0x55d6ec89e300 msgr2=0x55d6ecc4d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc15bfff640 0 client.16500 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 -- 172.21.15.115:0/2365374955 shutdown_connections 2024-05-09T05:41:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.907+0000 7fc189989740 1 -- 172.21.15.115:0/2365374955 wait complete. 2024-05-09T05:41:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.908+0000 7fc189989740 20 client.16500 trim_cache size 0 max 16384 2024-05-09T05:41:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.929+0000 7fc189989740 1 Processor -- start 2024-05-09T05:41:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.929+0000 7fc189989740 1 -- start start 2024-05-09T05:41:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.929+0000 7fc189989740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8812f0 0x55d6ecc536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.929+0000 7fc189989740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8894d0 0x55d6ecc53c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.929+0000 7fc189989740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.929+0000 7fc189989740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1aedb0 con 0x55d6ec8812f0 2024-05-09T05:41:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.929+0000 7fc189989740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1b0950 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.929+0000 7fc189989740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1af480 con 0x55d6ec8894d0 2024-05-09T05:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc180874640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8812f0 0x55d6ecc536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc180874640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8812f0 0x55d6ecc536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51560/0 (socket says 172.21.15.115:51560) 2024-05-09T05:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc180874640 1 -- 172.21.15.115:0/1541938912 learned_addr learned my addr 172.21.15.115:0/1541938912 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc181075640 1 --2- 172.21.15.115:0/1541938912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc180874640 1 -- 172.21.15.115:0/1541938912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8894d0 msgr2=0x55d6ecc53c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc180874640 1 --2- 172.21.15.115:0/1541938912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8894d0 0x55d6ecc53c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc180874640 1 -- 172.21.15.115:0/1541938912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 msgr2=0x55d6ecc719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc180874640 1 --2- 172.21.15.115:0/1541938912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc180874640 1 -- 172.21.15.115:0/1541938912 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ec98c5f0 con 0x55d6ec8812f0 2024-05-09T05:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc181075640 1 --2- 172.21.15.115:0/1541938912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc180874640 1 --2- 172.21.15.115:0/1541938912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8812f0 0x55d6ecc536c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc17c103890 tx=0x7fc170040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6cecf034bc504d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc1667fc640 1 -- 172.21.15.115:0/1541938912 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170056070 con 0x55d6ec8812f0 2024-05-09T05:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.930+0000 7fc1667fc640 1 -- 172.21.15.115:0/1541938912 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1700042c0 con 0x55d6ec8812f0 2024-05-09T05:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.931+0000 7fc1667fc640 1 -- 172.21.15.115:0/1541938912 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170035040 con 0x55d6ec8812f0 2024-05-09T05:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 -- 172.21.15.115:0/1541938912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8812f0 msgr2=0x55d6ecc536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 --2- 172.21.15.115:0/1541938912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8812f0 0x55d6ecc536c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc17c103890 tx=0x7fc170040000 comp rx=0 tx=0).stop 2024-05-09T05:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 -- 172.21.15.115:0/1541938912 shutdown_connections 2024-05-09T05:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 --2- 172.21.15.115:0/1541938912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 --2- 172.21.15.115:0/1541938912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8894d0 0x55d6ecc53c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 --2- 172.21.15.115:0/1541938912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8812f0 0x55d6ecc536c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 -- 172.21.15.115:0/1541938912 >> 172.21.15.115:0/1541938912 conn(0x55d6ec89e300 msgr2=0x55d6ecc58690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 -- 172.21.15.115:0/1541938912 shutdown_connections 2024-05-09T05:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 -- 172.21.15.115:0/1541938912 wait complete. 2024-05-09T05:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 -- start start 2024-05-09T05:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.932+0000 7fc189989740 1 Processor -- start 2024-05-09T05:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc189989740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc62180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc189989740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 0x55d6ecc5d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc189989740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc5da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc189989740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1b0220 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc189989740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1afb50 con 0x55d6ec8894d0 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc189989740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1acd90 con 0x55d6ec8812f0 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc181075640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc5da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc189989740 10 client.0 _ll_register_callbacks cb 0x7fc174004330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc181075640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc5da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51574/0 (socket says 172.21.15.115:51574) 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.933+0000 7fc181075640 1 -- 172.21.15.115:0/1827005580 learned_addr learned my addr 172.21.15.115:0/1827005580 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc180874640 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc62180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc17bfff640 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 0x55d6ecc5d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc181075640 1 -- 172.21.15.115:0/1827005580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 msgr2=0x55d6ecc62180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc181075640 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc62180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc181075640 1 -- 172.21.15.115:0/1827005580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 msgr2=0x55d6ecc5d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc181075640 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 0x55d6ecc5d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc181075640 1 -- 172.21.15.115:0/1827005580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ecb76ed0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc17bfff640 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 0x55d6ecc5d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:41:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc181075640 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc5da10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc1740dfe80 tx=0x7fc1740a97e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7e6fd0e5c60c55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc15bfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc15bfff640 1 -- 172.21.15.115:0/1827005580 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174014030 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc15bfff640 1 -- 172.21.15.115:0/1827005580 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc1740a8c70 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.934+0000 7fc189989740 1 -- 172.21.15.115:0/1827005580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1700042c0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc15bfff640 1 -- 172.21.15.115:0/1827005580 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc174020040 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc189989740 10 client.16506 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc189989740 1 -- 172.21.15.115:0/1827005580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d6eccd40c0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc15bfff640 1 -- 172.21.15.115:0/1827005580 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1700042c0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc15bfff640 1 -- 172.21.15.115:0/1827005580 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc1740aeea0 con 0x55d6ec8aa1b0 2024-05-09T05:41:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc189989740 10 client.16506 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc189989740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc189989740 20 client.16506 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc189989740 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6ec83e1c0 0x55d6eccd6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc189989740 4 client.16506 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:41:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc189989740 1 -- 172.21.15.115:0/1827005580 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6ecbba240 con 0x55d6ec83e1c0 2024-05-09T05:41:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.935+0000 7fc180874640 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6ec83e1c0 0x55d6eccd6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.936+0000 7fc15bfff640 10 client.16506 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.936+0000 7fc180874640 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6ec83e1c0 0x55d6eccd6770 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc2d3dc78bee79e server_cookie=79ea941347968d70 in_seq=0 out_seq=0 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.948+0000 7fc15bfff640 1 -- 172.21.15.115:0/1827005580 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d6ecbba240 con 0x55d6ec83e1c0 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.948+0000 7fc15bfff640 10 client.16506 handle_command_reply: tid=0 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.948+0000 7fc189989740 1 client.16506 shutdown 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.948+0000 7fc189989740 1 -- 172.21.15.115:0/1827005580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 msgr2=0x55d6ecc5da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.948+0000 7fc189989740 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc5da10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc1740dfe80 tx=0x7fc1740a97e0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc189989740 1 -- 172.21.15.115:0/1827005580 shutdown_connections 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc189989740 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6ec8aa1b0 0x55d6ecc5da10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc189989740 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ec8894d0 0x55d6ecc5d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc189989740 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6ec83e1c0 0x55d6eccd6770 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc189989740 1 --2- 172.21.15.115:0/1827005580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6ec8812f0 0x55d6ecc62180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc15bfff640 0 client.16506 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc189989740 1 -- 172.21.15.115:0/1827005580 >> 172.21.15.115:0/1827005580 conn(0x55d6ec89e300 msgr2=0x55d6ecc4d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc189989740 1 -- 172.21.15.115:0/1827005580 shutdown_connections 2024-05-09T05:41:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc189989740 1 -- 172.21.15.115:0/1827005580 wait complete. 2024-05-09T05:41:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc189989740 20 client.16506 trim_cache size 0 max 16384 2024-05-09T05:41:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:41:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:41:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:41:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc182ad8640 1 -- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc16c0b29d0 msgr2=0x7fc16c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc182ad8640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc16c0b29d0 0x7fc16c0b4ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc170004770 tx=0x7fc1700058d0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.949+0000 7fc182ad8640 1 -- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c109050 msgr2=0x7fc17c19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.950+0000 7fc182ad8640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c109050 0x7fc17c19df70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc174009fd0 tx=0x7fc17400b260 comp rx=0 tx=0).stop 2024-05-09T05:41:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.950+0000 7fc182ad8640 1 -- 172.21.15.115:0/993559692 shutdown_connections 2024-05-09T05:41:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.950+0000 7fc182ad8640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc16c0b29d0 0x7fc16c0b4ee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.950+0000 7fc182ad8640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c109050 0x7fc17c19df70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.950+0000 7fc182ad8640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc17c108bb0 0x7fc17c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.950+0000 7fc182ad8640 1 --2- 172.21.15.115:0/993559692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17c103300 0x7fc17c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.950+0000 7fc182ad8640 1 -- 172.21.15.115:0/993559692 >> 172.21.15.115:0/993559692 conn(0x7fc17c0792a0 msgr2=0x7fc17c104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.950+0000 7fc182ad8640 1 -- 172.21.15.115:0/993559692 shutdown_connections 2024-05-09T05:41:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:17.950+0000 7fc182ad8640 1 -- 172.21.15.115:0/993559692 wait complete. 2024-05-09T05:41:18.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.576+0000 7fbf23f6d700 1 -- 172.21.15.115:0/3579641920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 msgr2=0x7fbf1c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:18.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.576+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/3579641920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 0x7fbf1c108200 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0099c0 tx=0x7fbf0c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.576+0000 7fbf23f6d700 1 -- 172.21.15.115:0/3579641920 shutdown_connections 2024-05-09T05:41:18.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.576+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/3579641920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf1c108740 0x7fbf1c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.576+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/3579641920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf1c1034c0 0x7fbf1c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.576+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/3579641920 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 0x7fbf1c108200 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.576+0000 7fbf23f6d700 1 -- 172.21.15.115:0/3579641920 >> 172.21.15.115:0/3579641920 conn(0x7fbf1c0fedc0 msgr2=0x7fbf1c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:18.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.576+0000 7fbf23f6d700 1 -- 172.21.15.115:0/3579641920 shutdown_connections 2024-05-09T05:41:18.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.576+0000 7fbf23f6d700 1 -- 172.21.15.115:0/3579641920 wait complete. 2024-05-09T05:41:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf23f6d700 1 Processor -- start 2024-05-09T05:41:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf23f6d700 1 -- start start 2024-05-09T05:41:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf23f6d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf1c1034c0 0x7fbf1c0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf23f6d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 0x7fbf1c0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf23f6d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf1c108740 0x7fbf1c0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf23f6d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c113250 con 0x7fbf1c107df0 2024-05-09T05:41:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf23f6d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c1130d0 con 0x7fbf1c108740 2024-05-09T05:41:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf23f6d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c1133d0 con 0x7fbf1c1034c0 2024-05-09T05:41:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf21508700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 0x7fbf1c0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf21d09700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf1c1034c0 0x7fbf1c0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf21d09700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf1c1034c0 0x7fbf1c0ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:56860/0 (socket says 172.21.15.115:56860) 2024-05-09T05:41:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf21508700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 0x7fbf1c0ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51590/0 (socket says 172.21.15.115:51590) 2024-05-09T05:41:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf21d09700 1 -- 172.21.15.115:0/461198316 learned_addr learned my addr 172.21.15.115:0/461198316 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.577+0000 7fbf2250a700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf1c108740 0x7fbf1c0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf21508700 1 -- 172.21.15.115:0/461198316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf1c1034c0 msgr2=0x7fbf1c0ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf21508700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf1c1034c0 0x7fbf1c0ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf21508700 1 -- 172.21.15.115:0/461198316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf1c108740 msgr2=0x7fbf1c0ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf21508700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf1c108740 0x7fbf1c0ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf21508700 1 -- 172.21.15.115:0/461198316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c1a90d0 con 0x7fbf1c107df0 2024-05-09T05:41:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf21d09700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf1c1034c0 0x7fbf1c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf21508700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 0x7fbf1c0ff7e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0098e0 tx=0x7fbf0c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf12ffd700 1 -- 172.21.15.115:0/461198316 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c009e60 con 0x7fbf1c107df0 2024-05-09T05:41:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf12ffd700 1 -- 172.21.15.115:0/461198316 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf0c005030 con 0x7fbf1c107df0 2024-05-09T05:41:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf23f6d700 1 -- 172.21.15.115:0/461198316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c1a9360 con 0x7fbf1c107df0 2024-05-09T05:41:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf12ffd700 1 -- 172.21.15.115:0/461198316 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c003e60 con 0x7fbf1c107df0 2024-05-09T05:41:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.578+0000 7fbf23f6d700 1 -- 172.21.15.115:0/461198316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c1a97b0 con 0x7fbf1c107df0 2024-05-09T05:41:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.581+0000 7fbf12ffd700 1 -- 172.21.15.115:0/461198316 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fbf0c028060 con 0x7fbf1c107df0 2024-05-09T05:41:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.582+0000 7fbf12ffd700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf080b2a70 0x7fbf080b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf21d09700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf080b2a70 0x7fbf080b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf12ffd700 1 -- 172.21.15.115:0/461198316 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbf0c0e2ec0 con 0x7fbf1c107df0 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf32983b80 1 Processor -- start 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf32983b80 1 -- start start 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf21d09700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf080b2a70 0x7fbf080b4f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf18011550 tx=0x7fbf18009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d75a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d6fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf32983b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560a04c91ff0 con 0x560a04cdc160 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf32983b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560a04c307f0 con 0x560a04bf24a0 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf32983b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560a04c8dfb0 con 0x560a04d6f6a0 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf21d09700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf21d09700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51604/0 (socket says 172.21.15.115:51604) 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.583+0000 7fbf21d09700 1 -- 172.21.15.115:0/4188370812 learned_addr learned my addr 172.21.15.115:0/4188370812 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf2250a700 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d6fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf21508700 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d75a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf21d09700 1 -- 172.21.15.115:0/4188370812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 msgr2=0x560a04d6fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf21d09700 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d6fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf21d09700 1 -- 172.21.15.115:0/4188370812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 msgr2=0x560a04d75a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf21d09700 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d75a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf21d09700 1 -- 172.21.15.115:0/4188370812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a04c28370 con 0x560a04cdc160 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf2250a700 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d6fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf21508700 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d75a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf21d09700 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d754f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009880 tx=0x7fbf18014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132fe4ca8dec175e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf07fff700 1 -- 172.21.15.115:0/4188370812 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18026630 con 0x560a04cdc160 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf07fff700 1 -- 172.21.15.115:0/4188370812 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf1800ee40 con 0x560a04cdc160 2024-05-09T05:41:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.584+0000 7fbf07fff700 1 -- 172.21.15.115:0/4188370812 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1801d770 con 0x560a04cdc160 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.585+0000 7fbf32983b80 1 -- 172.21.15.115:0/4188370812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 msgr2=0x560a04d754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.585+0000 7fbf32983b80 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d754f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009880 tx=0x7fbf18014000 comp rx=0 tx=0).stop 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.586+0000 7fbf32983b80 1 -- 172.21.15.115:0/4188370812 shutdown_connections 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.586+0000 7fbf32983b80 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d6fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.586+0000 7fbf32983b80 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d754f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.586+0000 7fbf32983b80 1 --2- 172.21.15.115:0/4188370812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d75a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.586+0000 7fbf32983b80 1 -- 172.21.15.115:0/4188370812 >> 172.21.15.115:0/4188370812 conn(0x560a04c526a0 msgr2=0x560a04d6a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.586+0000 7fbf32983b80 1 -- 172.21.15.115:0/4188370812 shutdown_connections 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.586+0000 7fbf32983b80 1 -- 172.21.15.115:0/4188370812 wait complete. 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.586+0000 7fbf32983b80 1 -- start start 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.586+0000 7fbf32983b80 1 Processor -- start 2024-05-09T05:41:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04bf24a0 0x560a04d791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d79710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04d6f6a0 0x560a04d79c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf32983b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560a04b0dba0 con 0x560a04cdc160 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf32983b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560a04c4c150 con 0x560a04d6f6a0 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf32983b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560a04d7c1d0 con 0x560a04bf24a0 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf21508700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d79710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf21508700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d79710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51612/0 (socket says 172.21.15.115:51612) 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf21508700 1 -- 172.21.15.115:0/3402249089 learned_addr learned my addr 172.21.15.115:0/3402249089 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf32983b80 10 client.0 _ll_register_callbacks cb 0x560a04b036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf21d09700 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04bf24a0 0x560a04d791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf2250a700 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04d6f6a0 0x560a04d79c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.587+0000 7fbf21508700 1 -- 172.21.15.115:0/3402249089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04bf24a0 msgr2=0x560a04d791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf21508700 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04bf24a0 0x560a04d791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf21508700 1 -- 172.21.15.115:0/3402249089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04d6f6a0 msgr2=0x560a04d79c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf21508700 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04d6f6a0 0x560a04d79c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf21508700 1 -- 172.21.15.115:0/3402249089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a04b0d490 con 0x560a04cdc160 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf2250a700 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04d6f6a0 0x560a04d79c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf21508700 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d79710 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0099c0 tx=0x7fbf0c0a96e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344b46fa23152b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf057fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf057fa700 1 -- 172.21.15.115:0/3402249089 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0a9ef0 con 0x560a04cdc160 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf32983b80 1 -- 172.21.15.115:0/3402249089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a04c9f9d0 con 0x560a04cdc160 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf057fa700 1 -- 172.21.15.115:0/3402249089 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbf0c0aa360 con 0x560a04cdc160 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf057fa700 1 -- 172.21.15.115:0/3402249089 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbf0c0ae2a0 con 0x560a04cdc160 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf32983b80 10 client.16518 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.588+0000 7fbf32983b80 1 -- 172.21.15.115:0/3402249089 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a04d86ba0 con 0x560a04cdc160 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.589+0000 7fbf057fa700 1 -- 172.21.15.115:0/3402249089 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560a04c9f9d0 con 0x560a04cdc160 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.589+0000 7fbf057fa700 1 -- 172.21.15.115:0/3402249089 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fbf0c0a91e0 con 0x560a04cdc160 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.589+0000 7fbf32983b80 10 client.16518 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.589+0000 7fbf32983b80 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.589+0000 7fbf32983b80 20 client.16518 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.589+0000 7fbf32983b80 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a04d89450 0x560a04d8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.589+0000 7fbf32983b80 4 client.16518 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.589+0000 7fbf32983b80 1 -- 172.21.15.115:0/3402249089 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a04c99a70 con 0x560a04d89450 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.589+0000 7fbf21d09700 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a04d89450 0x560a04d8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.590+0000 7fbf057fa700 10 client.16518 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.590+0000 7fbf21d09700 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a04d89450 0x560a04d8b8a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad1dccc04a890e8 server_cookie=ab2f2fb3a10f612b in_seq=0 out_seq=0 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.590+0000 7fbf057fa700 1 -- 172.21.15.115:0/3402249089 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a04c99a70 con 0x560a04d89450 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.590+0000 7fbf057fa700 10 client.16518 handle_command_reply: tid=0 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.590+0000 7fbf32983b80 1 client.16518 shutdown 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 -- 172.21.15.115:0/3402249089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 msgr2=0x560a04d79710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:18.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d79710 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0099c0 tx=0x7fbf0c0a96e0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 -- 172.21.15.115:0/3402249089 shutdown_connections 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04d6f6a0 0x560a04d79c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a04d89450 0x560a04d8b8a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04bf24a0 0x560a04d791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 --2- 172.21.15.115:0/3402249089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04cdc160 0x560a04d79710 secure :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0099c0 tx=0x7fbf0c0a96e0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 -- 172.21.15.115:0/3402249089 >> 172.21.15.115:0/3402249089 conn(0x560a04c526a0 msgr2=0x560a04d6a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf057fa700 0 client.16518 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 -- 172.21.15.115:0/3402249089 shutdown_connections 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 1 -- 172.21.15.115:0/3402249089 wait complete. 2024-05-09T05:41:18.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.591+0000 7fbf32983b80 20 client.16518 trim_cache size 0 max 16384 2024-05-09T05:41:18.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.614+0000 7fbf32983b80 1 Processor -- start 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.614+0000 7fbf32983b80 1 -- start start 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d6f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d84c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04d79b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf32983b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c1b2390 con 0x560a04d79740 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf32983b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c1b3f30 con 0x560a04bf24a0 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf32983b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c1b2a60 con 0x560a04d6f6a0 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf2250a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04d79b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf2250a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04d79b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51624/0 (socket says 172.21.15.115:51624) 2024-05-09T05:41:18.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf2250a700 1 -- 172.21.15.115:0/1080460813 learned_addr learned my addr 172.21.15.115:0/1080460813 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf21d09700 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d6f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf21508700 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d84c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf2250a700 1 -- 172.21.15.115:0/1080460813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 msgr2=0x560a04d84c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf2250a700 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d84c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf2250a700 1 -- 172.21.15.115:0/1080460813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 msgr2=0x560a04d6f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf2250a700 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d6f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf2250a700 1 -- 172.21.15.115:0/1080460813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a04d63f60 con 0x560a04d79740 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf21508700 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d84c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf21d09700 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d6f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.615+0000 7fbf2250a700 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04d79b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009f60 tx=0x7fbf14009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311b062105f35e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.616+0000 7fbf077fe700 1 -- 172.21.15.115:0/1080460813 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1401b070 con 0x560a04d79740 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.616+0000 7fbf077fe700 1 -- 172.21.15.115:0/1080460813 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf14007e20 con 0x560a04d79740 2024-05-09T05:41:18.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.616+0000 7fbf077fe700 1 -- 172.21.15.115:0/1080460813 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14016b90 con 0x560a04d79740 2024-05-09T05:41:18.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 -- 172.21.15.115:0/1080460813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 msgr2=0x560a04d79b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:18.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04d79b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009f60 tx=0x7fbf14009f90 comp rx=0 tx=0).stop 2024-05-09T05:41:18.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 -- 172.21.15.115:0/1080460813 shutdown_connections 2024-05-09T05:41:18.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d6f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04d79b50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 --2- 172.21.15.115:0/1080460813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d84c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 -- 172.21.15.115:0/1080460813 >> 172.21.15.115:0/1080460813 conn(0x560a04c526a0 msgr2=0x560a04d73470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:18.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 -- 172.21.15.115:0/1080460813 shutdown_connections 2024-05-09T05:41:18.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 -- 172.21.15.115:0/1080460813 wait complete. 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 -- start start 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.617+0000 7fbf32983b80 1 Processor -- start 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d90c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf32983b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04df2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf32983b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c1b3800 con 0x560a04d79740 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf32983b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c1b3130 con 0x560a04bf24a0 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf32983b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c1b0330 con 0x560a04d6f6a0 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf32983b80 10 client.0 _ll_register_callbacks cb 0x560a04b036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf2250a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04df2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf2250a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04df2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51630/0 (socket says 172.21.15.115:51630) 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf2250a700 1 -- 172.21.15.115:0/2464233275 learned_addr learned my addr 172.21.15.115:0/2464233275 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf21d09700 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d90c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf21508700 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf2250a700 1 -- 172.21.15.115:0/2464233275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 msgr2=0x560a04d911d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf2250a700 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d911d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf2250a700 1 -- 172.21.15.115:0/2464233275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 msgr2=0x560a04d90c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf2250a700 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d90c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.618+0000 7fbf2250a700 1 -- 172.21.15.115:0/2464233275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a04d73b80 con 0x560a04d79740 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf2250a700 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04df2c30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbf14016b80 tx=0x7fbf14016bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e2491812d1b945 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf057fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf057fa700 1 -- 172.21.15.115:0/2464233275 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1401b070 con 0x560a04d79740 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf057fa700 1 -- 172.21.15.115:0/2464233275 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbf14004ad0 con 0x560a04d79740 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf32983b80 1 -- 172.21.15.115:0/2464233275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf14007e20 con 0x560a04d79740 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf057fa700 1 -- 172.21.15.115:0/2464233275 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbf1402f050 con 0x560a04d79740 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf32983b80 10 client.16524 fetch_fsmap learned FSMap version 32 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf32983b80 1 -- 172.21.15.115:0/2464233275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a04df3f20 con 0x560a04d79740 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf057fa700 1 -- 172.21.15.115:0/2464233275 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf14007e20 con 0x560a04d79740 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.619+0000 7fbf057fa700 1 -- 172.21.15.115:0/2464233275 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fbf14005170 con 0x560a04d79740 2024-05-09T05:41:18.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf32983b80 10 client.16524 fetch_fsmap finished waiting for FSMap version 32 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf32983b80 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf32983b80 20 client.16524 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf32983b80 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a04df45f0 0x560a04df6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf32983b80 4 client.16524 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf32983b80 1 -- 172.21.15.115:0/2464233275 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a04d8be70 con 0x560a04df45f0 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf21d09700 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a04df45f0 0x560a04df6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf057fa700 10 client.16524 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf21d09700 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a04df45f0 0x560a04df6a10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73f8c7f5d47590c server_cookie=3c9444f46225cd9d in_seq=0 out_seq=0 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf057fa700 1 -- 172.21.15.115:0/2464233275 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560a04d8be70 con 0x560a04df45f0 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.620+0000 7fbf057fa700 10 client.16524 handle_command_reply: tid=0 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 client.16524 shutdown 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 -- 172.21.15.115:0/2464233275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 msgr2=0x560a04df2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:18.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04df2c30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbf14016b80 tx=0x7fbf14016bb0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 -- 172.21.15.115:0/2464233275 shutdown_connections 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a04bf24a0 0x560a04d90c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a04df45f0 0x560a04df6a10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a04d79740 0x560a04df2c30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 --2- 172.21.15.115:0/2464233275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a04d6f6a0 0x560a04d911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 -- 172.21.15.115:0/2464233275 >> 172.21.15.115:0/2464233275 conn(0x560a04c526a0 msgr2=0x560a04d73470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf057fa700 0 client.16524 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 -- 172.21.15.115:0/2464233275 shutdown_connections 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 1 -- 172.21.15.115:0/2464233275 wait complete. 2024-05-09T05:41:18.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.621+0000 7fbf32983b80 20 client.16524 trim_cache size 0 max 16384 2024-05-09T05:41:18.620 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:41:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T05:41:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": { 2024-05-09T05:41:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: "d6f13d18-3fd5-4c50-9e13-12ffd90ae9a8": { 2024-05-09T05:41:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/", 2024-05-09T05:41:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: "tag": "d6f13d18-3fd5-4c50-9e13-12ffd90ae9a8", 2024-05-09T05:41:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: "options": "recursive,force" 2024-05-09T05:41:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:41:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:41:18.621 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:41:18.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 -- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf080b2a70 msgr2=0x7fbf080b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:18.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf080b2a70 0x7fbf080b4f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf18011550 tx=0x7fbf18009260 comp rx=0 tx=0).stop 2024-05-09T05:41:18.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 -- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 msgr2=0x7fbf1c0ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:18.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 0x7fbf1c0ff7e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0098e0 tx=0x7fbf0c0045e0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 -- 172.21.15.115:0/461198316 shutdown_connections 2024-05-09T05:41:18.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf080b2a70 0x7fbf080b4f80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf1c108740 0x7fbf1c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf1c1034c0 0x7fbf1c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 --2- 172.21.15.115:0/461198316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf1c107df0 0x7fbf1c0ff7e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:18.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 -- 172.21.15.115:0/461198316 >> 172.21.15.115:0/461198316 conn(0x7fbf1c0fedc0 msgr2=0x7fbf1c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:18.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 -- 172.21.15.115:0/461198316 shutdown_connections 2024-05-09T05:41:18.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:18.622+0000 7fbf23f6d700 1 -- 172.21.15.115:0/461198316 wait complete. 2024-05-09T05:41:19.352 INFO:tasks.cephfs.filesystem:scrub status for tag:d6f13d18-3fd5-4c50-9e13-12ffd90ae9a8 - {'path': '/', 'tag': 'd6f13d18-3fd5-4c50-9e13-12ffd90ae9a8', 'options': 'recursive,force'} 2024-05-09T05:41:19.537 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:19 smithi115 ceph-mon[30349]: pgmap v96: 129 pgs: 129 active+clean; 608 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 35 MiB/s wr, 64 op/s 2024-05-09T05:41:19.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:19 smithi195 ceph-mon[33844]: pgmap v96: 129 pgs: 129 active+clean; 608 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 35 MiB/s wr, 64 op/s 2024-05-09T05:41:19.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:19 smithi161 ceph-mon[33805]: pgmap v96: 129 pgs: 129 active+clean; 608 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 35 MiB/s wr, 64 op/s 2024-05-09T05:41:21.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:21 smithi195 ceph-mon[33844]: pgmap v97: 129 pgs: 129 active+clean; 608 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:41:21.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:21 smithi115 ceph-mon[30349]: pgmap v97: 129 pgs: 129 active+clean; 608 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:41:21.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:21 smithi161 ceph-mon[33805]: pgmap v97: 129 pgs: 129 active+clean; 608 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:41:23.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:23 smithi195 ceph-mon[33844]: pgmap v98: 129 pgs: 129 active+clean; 608 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:41:23.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:23 smithi115 ceph-mon[30349]: pgmap v98: 129 pgs: 129 active+clean; 608 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:41:23.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:23 smithi161 ceph-mon[33805]: pgmap v98: 129 pgs: 129 active+clean; 608 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:41:25.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:25 smithi195 ceph-mon[33844]: pgmap v99: 129 pgs: 129 active+clean; 744 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 54 op/s 2024-05-09T05:41:25.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:25 smithi115 ceph-mon[30349]: pgmap v99: 129 pgs: 129 active+clean; 744 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 54 op/s 2024-05-09T05:41:25.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:25 smithi161 ceph-mon[33805]: pgmap v99: 129 pgs: 129 active+clean; 744 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 54 op/s 2024-05-09T05:41:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:27 smithi195 ceph-mon[33844]: pgmap v100: 129 pgs: 129 active+clean; 744 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:41:27.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:27 smithi115 ceph-mon[30349]: pgmap v100: 129 pgs: 129 active+clean; 744 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:41:27.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:27 smithi161 ceph-mon[33805]: pgmap v100: 129 pgs: 129 active+clean; 744 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:41:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:28 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:41:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:28 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:41:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:28 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:41:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:28 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:41:28.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:28 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:41:28.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:28 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:41:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:28 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:41:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:28 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:41:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:28 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:41:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:28 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:41:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:28 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:41:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:28 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:41:29.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:29 smithi195 ceph-mon[33844]: pgmap v101: 129 pgs: 129 active+clean; 884 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 55 op/s 2024-05-09T05:41:29.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:29 smithi115 ceph-mon[30349]: pgmap v101: 129 pgs: 129 active+clean; 884 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 55 op/s 2024-05-09T05:41:29.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:29 smithi161 ceph-mon[33805]: pgmap v101: 129 pgs: 129 active+clean; 884 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 55 op/s 2024-05-09T05:41:30.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:30 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:41:30.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:30 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:41:30.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:30 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:41:30.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:30 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:41:30.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:30 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:41:30.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:30 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:41:30.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:30 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:41:30.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:30 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:41:30.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:30 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:41:30.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:30 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:41:30.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:30 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:41:30.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:30 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:41:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.058+0000 7fabe7a9f640 1 -- 172.21.15.115:0/534407269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0109030 msgr2=0x7fabe0102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/534407269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0109030 0x7fabe0102c70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd002a410 comp rx=0 tx=0).stop 2024-05-09T05:41:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 -- 172.21.15.115:0/534407269 shutdown_connections 2024-05-09T05:41:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/534407269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabe01032e0 0x7fabe010f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/534407269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0109030 0x7fabe0102c70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/534407269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabe0108b90 0x7fabe0102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 -- 172.21.15.115:0/534407269 >> 172.21.15.115:0/534407269 conn(0x7fabe00fe390 msgr2=0x7fabe01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 -- 172.21.15.115:0/534407269 shutdown_connections 2024-05-09T05:41:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 -- 172.21.15.115:0/534407269 wait complete. 2024-05-09T05:41:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 Processor -- start 2024-05-09T05:41:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.059+0000 7fabe7a9f640 1 -- start start 2024-05-09T05:41:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe7a9f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabe01032e0 0x7fabe019b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe7a9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0108b90 0x7fabe019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe7a9f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabe0109030 0x7fabe01a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe7a9f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0112c30 con 0x7fabe0108b90 2024-05-09T05:41:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe7a9f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0112ab0 con 0x7fabe01032e0 2024-05-09T05:41:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe7a9f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0112db0 con 0x7fabe0109030 2024-05-09T05:41:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe503a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0108b90 0x7fabe019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe503a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0108b90 0x7fabe019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35286/0 (socket says 172.21.15.115:35286) 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe503a640 1 -- 172.21.15.115:0/3979810341 learned_addr learned my addr 172.21.15.115:0/3979810341 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe583b640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabe01032e0 0x7fabe019b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe603c640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabe0109030 0x7fabe01a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe503a640 1 -- 172.21.15.115:0/3979810341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabe0109030 msgr2=0x7fabe01a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe503a640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabe0109030 0x7fabe01a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe503a640 1 -- 172.21.15.115:0/3979810341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabe01032e0 msgr2=0x7fabe019b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe503a640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabe01032e0 0x7fabe019b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe503a640 1 -- 172.21.15.115:0/3979810341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe019c3f0 con 0x7fabe0108b90 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe583b640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabe01032e0 0x7fabe019b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.060+0000 7fabe503a640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0108b90 0x7fabe019bb70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fabd0005940 tx=0x7fabd00040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.061+0000 7fabd6ffd640 1 -- 172.21.15.115:0/3979810341 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd003f030 con 0x7fabe0108b90 2024-05-09T05:41:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.061+0000 7fabe7a9f640 1 -- 172.21.15.115:0/3979810341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe01a6c50 con 0x7fabe0108b90 2024-05-09T05:41:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.061+0000 7fabd6ffd640 1 -- 172.21.15.115:0/3979810341 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabd0003a80 con 0x7fabe0108b90 2024-05-09T05:41:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.061+0000 7fabd6ffd640 1 -- 172.21.15.115:0/3979810341 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0035070 con 0x7fabe0108b90 2024-05-09T05:41:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.061+0000 7fabe7a9f640 1 -- 172.21.15.115:0/3979810341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01a7140 con 0x7fabe0108b90 2024-05-09T05:41:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.064+0000 7fabd6ffd640 1 -- 172.21.15.115:0/3979810341 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fabd0042030 con 0x7fabe0108b90 2024-05-09T05:41:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.064+0000 7fabeea2f740 1 Processor -- start 2024-05-09T05:41:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabeea2f740 1 -- start start 2024-05-09T05:41:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc9859d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 0x55cc9859dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d94d0 0x55cc9859e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabeea2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cc985027c0 con 0x55cc981fa1b0 2024-05-09T05:41:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabeea2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cc9849e0e0 con 0x55cc981d94d0 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabeea2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cc984fe780 con 0x55cc981d12f0 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe603c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc9859d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe603c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc9859d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35296/0 (socket says 172.21.15.115:35296) 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe603c640 1 -- 172.21.15.115:0/1336648682 learned_addr learned my addr 172.21.15.115:0/1336648682 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe503a640 1 --2- 172.21.15.115:0/1336648682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d94d0 0x55cc9859e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe583b640 1 --2- 172.21.15.115:0/1336648682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 0x55cc9859dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabd6ffd640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabcc0b2950 0x7fabcc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe603c640 1 -- 172.21.15.115:0/1336648682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 msgr2=0x55cc9859dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe603c640 1 --2- 172.21.15.115:0/1336648682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 0x55cc9859dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe603c640 1 -- 172.21.15.115:0/1336648682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d94d0 msgr2=0x55cc9859e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe603c640 1 --2- 172.21.15.115:0/1336648682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d94d0 0x55cc9859e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.065+0000 7fabe603c640 1 -- 172.21.15.115:0/1336648682 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc984c77c0 con 0x55cc981fa1b0 2024-05-09T05:41:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.066+0000 7fabd6ffd640 1 -- 172.21.15.115:0/3979810341 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fabd0102f40 con 0x7fabe0108b90 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.066+0000 7fabe603c640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabcc0b2950 0x7fabcc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.066+0000 7fabe603c640 1 --2- 172.21.15.115:0/1336648682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc9859d680 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fabd80103a0 tx=0x7fabd8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c371658305bcb9d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.066+0000 7fabcbfff640 1 -- 172.21.15.115:0/1336648682 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd801c900 con 0x55cc981fa1b0 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.066+0000 7fabcbfff640 1 -- 172.21.15.115:0/1336648682 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabd80113e0 con 0x55cc981fa1b0 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.066+0000 7fabcbfff640 1 -- 172.21.15.115:0/1336648682 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd801b3d0 con 0x55cc981fa1b0 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.066+0000 7fabe603c640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabcc0b2950 0x7fabcc0b4e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fabd8020000 tx=0x7fabd801d000 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.067+0000 7fabeea2f740 1 -- 172.21.15.115:0/1336648682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 msgr2=0x55cc9859d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.067+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1336648682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc9859d680 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fabd80103a0 tx=0x7fabd8010860 comp rx=0 tx=0).stop 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.067+0000 7fabeea2f740 1 -- 172.21.15.115:0/1336648682 shutdown_connections 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.067+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1336648682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d94d0 0x55cc9859e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.067+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1336648682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 0x55cc9859dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.067+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1336648682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc9859d680 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.067+0000 7fabeea2f740 1 -- 172.21.15.115:0/1336648682 >> 172.21.15.115:0/1336648682 conn(0x55cc981ee300 msgr2=0x55cc98599e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.067+0000 7fabeea2f740 1 -- 172.21.15.115:0/1336648682 shutdown_connections 2024-05-09T05:41:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.067+0000 7fabeea2f740 1 -- 172.21.15.115:0/1336648682 wait complete. 2024-05-09T05:41:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabeea2f740 1 -- start start 2024-05-09T05:41:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabeea2f740 1 Processor -- start 2024-05-09T05:41:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d12f0 0x55cc985a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 0x55cc985a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc985a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabeea2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cc985aa7b0 con 0x55cc981fa1b0 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabeea2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cc984899c0 con 0x55cc981d12f0 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabeea2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cc985aa930 con 0x55cc981d94d0 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabeea2f740 10 client.0 _ll_register_callbacks cb 0x55cc984b6020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.068+0000 7fabe603c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc985a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe603c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc985a7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35306/0 (socket says 172.21.15.115:35306) 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe603c640 1 -- 172.21.15.115:0/1578270038 learned_addr learned my addr 172.21.15.115:0/1578270038 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe583b640 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d12f0 0x55cc985a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe503a640 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 0x55cc985a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe603c640 1 -- 172.21.15.115:0/1578270038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 msgr2=0x55cc985a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe603c640 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 0x55cc985a9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe603c640 1 -- 172.21.15.115:0/1578270038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d12f0 msgr2=0x55cc985a9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe603c640 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d12f0 0x55cc985a9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe603c640 1 -- 172.21.15.115:0/1578270038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc984b01f0 con 0x55cc981fa1b0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabe603c640 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc985a7c20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fabd8011c90 tx=0x7fabd8005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0867d3d19030c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabc97fa640 1 -- 172.21.15.115:0/1578270038 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd801b3d0 con 0x55cc981fa1b0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabc97fa640 1 -- 172.21.15.115:0/1578270038 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fabd80117a0 con 0x55cc981fa1b0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabeea2f740 1 -- 172.21.15.115:0/1578270038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd80113e0 con 0x55cc981fa1b0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabc97fa640 1 -- 172.21.15.115:0/1578270038 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fabd803d050 con 0x55cc981fa1b0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.069+0000 7fabeea2f740 10 client.16536 fetch_fsmap learned FSMap version 33 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabeea2f740 1 -- 172.21.15.115:0/1578270038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cc98265790 con 0x55cc981fa1b0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabc97fa640 1 -- 172.21.15.115:0/1578270038 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabd80113e0 con 0x55cc981fa1b0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabc97fa640 1 -- 172.21.15.115:0/1578270038 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fabd800f040 con 0x55cc981fa1b0 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabeea2f740 10 client.16536 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T05:41:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabeea2f740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabeea2f740 20 client.16536 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cc981b8d80 0x55cc985b9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabeea2f740 4 client.16536 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabeea2f740 1 -- 172.21.15.115:0/1578270038 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc985103d0 con 0x55cc981b8d80 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.070+0000 7fabe583b640 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cc981b8d80 0x55cc985b9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.071+0000 7fabc97fa640 10 client.16536 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.071+0000 7fabe583b640 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cc981b8d80 0x55cc985b9f50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec1308e7fb2bef4 server_cookie=814807963ab64383 in_seq=0 out_seq=0 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabc97fa640 1 -- 172.21.15.115:0/1578270038 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cc985103d0 con 0x55cc981b8d80 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabc97fa640 10 client.16536 handle_command_reply: tid=0 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 client.16536 shutdown 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 -- 172.21.15.115:0/1578270038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 msgr2=0x55cc985a7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc985a7c20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fabd8011c90 tx=0x7fabd8005f80 comp rx=0 tx=0).stop 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 -- 172.21.15.115:0/1578270038 shutdown_connections 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981fa1b0 0x55cc985a7c20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cc981b8d80 0x55cc985b9f50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 0x55cc985a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 --2- 172.21.15.115:0/1578270038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981d12f0 0x55cc985a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabc97fa640 0 client.16536 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 -- 172.21.15.115:0/1578270038 >> 172.21.15.115:0/1578270038 conn(0x55cc981ee300 msgr2=0x55cc98598700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 -- 172.21.15.115:0/1578270038 shutdown_connections 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 1 -- 172.21.15.115:0/1578270038 wait complete. 2024-05-09T05:41:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.072+0000 7fabeea2f740 20 client.16536 trim_cache size 0 max 16384 2024-05-09T05:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabeea2f740 1 Processor -- start 2024-05-09T05:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabeea2f740 1 -- start start 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 0x55cc98598e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 0x55cc985bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d94d0 0x55cc985bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabeea2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabe01b1140 con 0x55cc981d94d0 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabeea2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fabe01b2d10 con 0x55cc981ee300 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabeea2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabe01b1810 con 0x55cc981d12f0 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabe603c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d94d0 0x55cc985bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabe503a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 0x55cc985bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabe603c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d94d0 0x55cc985bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35308/0 (socket says 172.21.15.115:35308) 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.091+0000 7fabe603c640 1 -- 172.21.15.115:0/2758485880 learned_addr learned my addr 172.21.15.115:0/2758485880 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabe583b640 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 0x55cc98598e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabe503a640 1 -- 172.21.15.115:0/2758485880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 msgr2=0x55cc98598e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabe503a640 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 0x55cc98598e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabe503a640 1 -- 172.21.15.115:0/2758485880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d94d0 msgr2=0x55cc985bfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabe503a640 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d94d0 0x55cc985bfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabe503a640 1 -- 172.21.15.115:0/2758485880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc982dc5f0 con 0x55cc981d12f0 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabe603c640 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d94d0 0x55cc985bfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabe583b640 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 0x55cc98598e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabe503a640 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 0x55cc985bf630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fabd0100350 tx=0x7fabd002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aef6f4b0f2b5306 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabcb7fe640 1 -- 172.21.15.115:0/2758485880 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0049040 con 0x55cc981d12f0 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabcb7fe640 1 -- 172.21.15.115:0/2758485880 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabd00c99f0 con 0x55cc981d12f0 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.092+0000 7fabcb7fe640 1 -- 172.21.15.115:0/2758485880 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd00c9ee0 con 0x55cc981d12f0 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.093+0000 7fabeea2f740 1 -- 172.21.15.115:0/2758485880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 msgr2=0x55cc985bf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.093+0000 7fabeea2f740 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 0x55cc985bf630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fabd0100350 tx=0x7fabd002a410 comp rx=0 tx=0).stop 2024-05-09T05:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.093+0000 7fabeea2f740 1 -- 172.21.15.115:0/2758485880 shutdown_connections 2024-05-09T05:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.093+0000 7fabeea2f740 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d94d0 0x55cc985bfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.093+0000 7fabeea2f740 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d12f0 0x55cc985bf630 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.093+0000 7fabeea2f740 1 --2- 172.21.15.115:0/2758485880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 0x55cc98598e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.093+0000 7fabeea2f740 1 -- 172.21.15.115:0/2758485880 >> 172.21.15.115:0/2758485880 conn(0x55cc985a35a0 msgr2=0x55cc98598700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.093+0000 7fabeea2f740 1 -- 172.21.15.115:0/2758485880 shutdown_connections 2024-05-09T05:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.093+0000 7fabeea2f740 1 -- 172.21.15.115:0/2758485880 wait complete. 2024-05-09T05:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.094+0000 7fabeea2f740 1 -- start start 2024-05-09T05:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.094+0000 7fabeea2f740 1 Processor -- start 2024-05-09T05:41:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.094+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d12f0 0x55cc985ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.094+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 0x55cc985ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.094+0000 7fabeea2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 0x55cc9861a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.094+0000 7fabeea2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabe01b25e0 con 0x55cc981d12f0 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.094+0000 7fabeea2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fabe01b1ee0 con 0x55cc981ee300 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.094+0000 7fabeea2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabe01af120 con 0x55cc981d94d0 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.094+0000 7fabeea2f740 10 client.0 _ll_register_callbacks cb 0x7fabd801f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe583b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d12f0 0x55cc985ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe583b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d12f0 0x55cc985ad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35320/0 (socket says 172.21.15.115:35320) 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe583b640 1 -- 172.21.15.115:0/3852091603 learned_addr learned my addr 172.21.15.115:0/3852091603 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe603c640 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 0x55cc9861a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe503a640 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 0x55cc985ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe583b640 1 -- 172.21.15.115:0/3852091603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 msgr2=0x55cc985ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe583b640 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 0x55cc985ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe583b640 1 -- 172.21.15.115:0/3852091603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 msgr2=0x55cc9861a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe583b640 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 0x55cc9861a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe583b640 1 -- 172.21.15.115:0/3852091603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc984c6ed0 con 0x55cc981d12f0 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabe583b640 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d12f0 0x55cc985ad030 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fabe0103870 tx=0x7fabdc0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2570779c5fb810cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabc97fa640 1 -- 172.21.15.115:0/3852091603 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc021070 con 0x55cc981d12f0 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabc97fa640 1 -- 172.21.15.115:0/3852091603 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fabdc01d440 con 0x55cc981d12f0 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabeea2f740 1 -- 172.21.15.115:0/3852091603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd00c99f0 con 0x55cc981d12f0 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabc97fa640 1 -- 172.21.15.115:0/3852091603 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fabdc02d050 con 0x55cc981d12f0 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabeea2f740 10 client.16542 fetch_fsmap learned FSMap version 33 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.095+0000 7fabeea2f740 1 -- 172.21.15.115:0/3852091603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cc986218c0 con 0x55cc981d12f0 2024-05-09T05:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabc97fa640 1 -- 172.21.15.115:0/3852091603 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabd00c99f0 con 0x55cc981d12f0 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabc97fa640 1 -- 172.21.15.115:0/3852091603 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fabdc003b90 con 0x55cc981d12f0 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabeea2f740 10 client.16542 fetch_fsmap finished waiting for FSMap version 33 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabeea2f740 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabeea2f740 20 client.16542 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabeea2f740 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cc981fa1b0 0x55cc98624090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabeea2f740 4 client.16542 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabeea2f740 1 -- 172.21.15.115:0/3852091603 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc9850a240 con 0x55cc981fa1b0 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabe503a640 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cc981fa1b0 0x55cc98624090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.096+0000 7fabc97fa640 10 client.16542 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.097+0000 7fabe503a640 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cc981fa1b0 0x55cc98624090 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51428121054e907d server_cookie=1b939c5f81b0700b in_seq=0 out_seq=0 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.107+0000 7fabc97fa640 1 -- 172.21.15.115:0/3852091603 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cc9850a240 con 0x55cc981fa1b0 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.107+0000 7fabc97fa640 10 client.16542 handle_command_reply: tid=0 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.107+0000 7fabeea2f740 1 client.16542 shutdown 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.107+0000 7fabeea2f740 1 -- 172.21.15.115:0/3852091603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d12f0 msgr2=0x55cc985ad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.107+0000 7fabeea2f740 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d12f0 0x55cc985ad030 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fabe0103870 tx=0x7fabdc0119c0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabeea2f740 1 -- 172.21.15.115:0/3852091603 shutdown_connections 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabeea2f740 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc981ee300 0x55cc9861a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabeea2f740 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cc981fa1b0 0x55cc98624090 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabeea2f740 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cc981d94d0 0x55cc985ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabeea2f740 1 --2- 172.21.15.115:0/3852091603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cc981d12f0 0x55cc985ad030 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabc97fa640 0 client.16542 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabeea2f740 1 -- 172.21.15.115:0/3852091603 >> 172.21.15.115:0/3852091603 conn(0x55cc985a35a0 msgr2=0x55cc98599680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabeea2f740 1 -- 172.21.15.115:0/3852091603 shutdown_connections 2024-05-09T05:41:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabeea2f740 1 -- 172.21.15.115:0/3852091603 wait complete. 2024-05-09T05:41:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabeea2f740 20 client.16542 trim_cache size 0 max 16384 2024-05-09T05:41:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:41:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:41:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabe7a9f640 1 -- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabcc0b2950 msgr2=0x7fabcc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabcc0b2950 0x7fabcc0b4e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fabd8020000 tx=0x7fabd801d000 comp rx=0 tx=0).stop 2024-05-09T05:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabe7a9f640 1 -- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0108b90 msgr2=0x7fabe019bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.108+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0108b90 0x7fabe019bb70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fabd0005940 tx=0x7fabd00040e0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.109+0000 7fabe7a9f640 1 -- 172.21.15.115:0/3979810341 shutdown_connections 2024-05-09T05:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.109+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabcc0b2950 0x7fabcc0b4e60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.109+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabe0109030 0x7fabe01a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.109+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe0108b90 0x7fabe019bb70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.109+0000 7fabe7a9f640 1 --2- 172.21.15.115:0/3979810341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabe01032e0 0x7fabe019b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.109+0000 7fabe7a9f640 1 -- 172.21.15.115:0/3979810341 >> 172.21.15.115:0/3979810341 conn(0x7fabe00fe390 msgr2=0x7fabe010e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.109+0000 7fabe7a9f640 1 -- 172.21.15.115:0/3979810341 shutdown_connections 2024-05-09T05:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:31.109+0000 7fabe7a9f640 1 -- 172.21.15.115:0/3979810341 wait complete. 2024-05-09T05:41:31.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:31 smithi195 ceph-mon[33844]: pgmap v102: 129 pgs: 129 active+clean; 884 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:41:31.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:31 smithi115 ceph-mon[30349]: pgmap v102: 129 pgs: 129 active+clean; 884 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:41:31.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:31 smithi161 ceph-mon[33805]: pgmap v102: 129 pgs: 129 active+clean; 884 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:41:33.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:33 smithi195 ceph-mon[33844]: pgmap v103: 129 pgs: 129 active+clean; 884 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:41:33.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:33 smithi115 ceph-mon[30349]: pgmap v103: 129 pgs: 129 active+clean; 884 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:41:33.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:33 smithi161 ceph-mon[33805]: pgmap v103: 129 pgs: 129 active+clean; 884 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:41:35.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:35 smithi195 ceph-mon[33844]: pgmap v104: 129 pgs: 129 active+clean; 1.0 GiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:41:35.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:35 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:41:35.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:35 smithi115 ceph-mon[30349]: pgmap v104: 129 pgs: 129 active+clean; 1.0 GiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:41:35.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:35 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:41:35.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:35 smithi161 ceph-mon[33805]: pgmap v104: 129 pgs: 129 active+clean; 1.0 GiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:41:35.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:35 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:41:37.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:37 smithi195 ceph-mon[33844]: pgmap v105: 129 pgs: 129 active+clean; 1.0 GiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-05-09T05:41:37.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:37 smithi115 ceph-mon[30349]: pgmap v105: 129 pgs: 129 active+clean; 1.0 GiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-05-09T05:41:37.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:37 smithi161 ceph-mon[33805]: pgmap v105: 129 pgs: 129 active+clean; 1.0 GiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-05-09T05:41:39.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:39 smithi195 ceph-mon[33844]: pgmap v106: 129 pgs: 129 active+clean; 1.1 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:41:39.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:39 smithi115 ceph-mon[30349]: pgmap v106: 129 pgs: 129 active+clean; 1.1 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:41:39.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:39 smithi161 ceph-mon[33805]: pgmap v106: 129 pgs: 129 active+clean; 1.1 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:41:41.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:41 smithi195 ceph-mon[33844]: pgmap v107: 129 pgs: 129 active+clean; 1.1 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 21 op/s 2024-05-09T05:41:41.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:41 smithi115 ceph-mon[30349]: pgmap v107: 129 pgs: 129 active+clean; 1.1 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 21 op/s 2024-05-09T05:41:41.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:41 smithi161 ceph-mon[33805]: pgmap v107: 129 pgs: 129 active+clean; 1.1 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 21 op/s 2024-05-09T05:41:43.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:43 smithi195 ceph-mon[33844]: pgmap v108: 129 pgs: 129 active+clean; 1.1 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 21 op/s 2024-05-09T05:41:43.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:43 smithi115 ceph-mon[30349]: pgmap v108: 129 pgs: 129 active+clean; 1.1 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 21 op/s 2024-05-09T05:41:43.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:43 smithi161 ceph-mon[33805]: pgmap v108: 129 pgs: 129 active+clean; 1.1 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 21 op/s 2024-05-09T05:41:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.215+0000 7ff6548b8640 1 -- 172.21.15.115:0/3359574842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010e220 msgr2=0x7ff6501106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.215+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3359574842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010e220 0x7ff6501106f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff648009ca0 tx=0x7ff64802a950 comp rx=0 tx=0).stop 2024-05-09T05:41:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.215+0000 7ff6548b8640 1 -- 172.21.15.115:0/3359574842 shutdown_connections 2024-05-09T05:41:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.215+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3359574842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010e220 0x7ff6501106f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.215+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3359574842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff65010b8c0 0x7ff65010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.215+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3359574842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff650104e60 0x7ff650105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.215+0000 7ff6548b8640 1 -- 172.21.15.115:0/3359574842 >> 172.21.15.115:0/3359574842 conn(0x7ff6500782e0 msgr2=0x7ff650078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.215+0000 7ff6548b8640 1 -- 172.21.15.115:0/3359574842 shutdown_connections 2024-05-09T05:41:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.215+0000 7ff6548b8640 1 -- 172.21.15.115:0/3359574842 wait complete. 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff6548b8640 1 Processor -- start 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff6548b8640 1 -- start start 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff6548b8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650104e60 0x7ff65019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff6548b8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010b8c0 0x7ff65019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff6548b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff65010e220 0x7ff65019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff6548b8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff650113d10 con 0x7ff65010b8c0 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff6548b8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff650113b90 con 0x7ff650104e60 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff6548b8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff650113e90 con 0x7ff65010e220 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff64dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010b8c0 0x7ff65019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff64dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010b8c0 0x7ff65019acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47240/0 (socket says 172.21.15.115:47240) 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff64dd9b640 1 -- 172.21.15.115:0/3470503882 learned_addr learned my addr 172.21.15.115:0/3470503882 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.216+0000 7ff64e59c640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650104e60 0x7ff65019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff64ed9d640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff65010e220 0x7ff65019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff64dd9b640 1 -- 172.21.15.115:0/3470503882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff65010e220 msgr2=0x7ff65019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff65010e220 0x7ff65019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff64dd9b640 1 -- 172.21.15.115:0/3470503882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650104e60 msgr2=0x7ff65019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650104e60 0x7ff65019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff64dd9b640 1 -- 172.21.15.115:0/3470503882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65019f5b0 con 0x7ff65010b8c0 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010b8c0 0x7ff65019acb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff64400e9f0 tx=0x7ff64400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff63f7fe640 1 -- 172.21.15.115:0/3470503882 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64400cd80 con 0x7ff65010b8c0 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff63f7fe640 1 -- 172.21.15.115:0/3470503882 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6440108c0 con 0x7ff65010b8c0 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff6548b8640 1 -- 172.21.15.115:0/3470503882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65019f8a0 con 0x7ff65010b8c0 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff63f7fe640 1 -- 172.21.15.115:0/3470503882 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6440187c0 con 0x7ff65010b8c0 2024-05-09T05:41:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.217+0000 7ff6548b8640 1 -- 172.21.15.115:0/3470503882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6501a7220 con 0x7ff65010b8c0 2024-05-09T05:41:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.220+0000 7ff65b76c740 1 Processor -- start 2024-05-09T05:41:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff65b76c740 1 -- start start 2024-05-09T05:41:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba4805680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba4805bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4806100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff65b76c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bba476a7c0 con 0x55bba44414d0 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff65b76c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bba47060e0 con 0x55bba44621b0 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff65b76c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bba4766780 con 0x55bba44392f0 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff63f7fe640 1 -- 172.21.15.115:0/3470503882 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff644010430 con 0x7ff65010b8c0 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff64dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4806100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff64ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba4805680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff64dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4806100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47242/0 (socket says 172.21.15.115:47242) 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff64dd9b640 1 -- 172.21.15.115:0/3488507625 learned_addr learned my addr 172.21.15.115:0/3488507625 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff64ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba4805680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:40486/0 (socket says 172.21.15.115:40486) 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.221+0000 7ff64e59c640 1 --2- 172.21.15.115:0/3488507625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba4805bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff64e59c640 1 -- 172.21.15.115:0/3488507625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 msgr2=0x55bba4805680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff64e59c640 1 --2- 172.21.15.115:0/3488507625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba4805680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff64e59c640 1 -- 172.21.15.115:0/3488507625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 msgr2=0x55bba4806100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff64e59c640 1 --2- 172.21.15.115:0/3488507625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4806100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff64e59c640 1 -- 172.21.15.115:0/3488507625 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba472f7c0 con 0x55bba44392f0 2024-05-09T05:41:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff64e59c640 1 --2- 172.21.15.115:0/3488507625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba4805bc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff640004630 tx=0x7ff64000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b70f003917760470 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff63f7fe640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6380b2950 0x7ff6380b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff63cff9640 1 -- 172.21.15.115:0/3488507625 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640003bd0 con 0x55bba44392f0 2024-05-09T05:41:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff64ed9d640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6380b2950 0x7ff6380b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff63cff9640 1 -- 172.21.15.115:0/3488507625 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff640015070 con 0x55bba44392f0 2024-05-09T05:41:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff63cff9640 1 -- 172.21.15.115:0/3488507625 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640004c00 con 0x55bba44392f0 2024-05-09T05:41:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.222+0000 7ff63f7fe640 1 -- 172.21.15.115:0/3470503882 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff6440e2c60 con 0x7ff65010b8c0 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.223+0000 7ff65b76c740 1 -- 172.21.15.115:0/3488507625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 msgr2=0x55bba4805bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.223+0000 7ff65b76c740 1 --2- 172.21.15.115:0/3488507625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba4805bc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff640004630 tx=0x7ff64000bb20 comp rx=0 tx=0).stop 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.223+0000 7ff64ed9d640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6380b2950 0x7ff6380b4e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff648009a60 tx=0x7ff648005c50 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.223+0000 7ff65b76c740 1 -- 172.21.15.115:0/3488507625 shutdown_connections 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.223+0000 7ff65b76c740 1 --2- 172.21.15.115:0/3488507625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4806100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.223+0000 7ff65b76c740 1 --2- 172.21.15.115:0/3488507625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba4805bc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.223+0000 7ff65b76c740 1 --2- 172.21.15.115:0/3488507625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba4805680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.223+0000 7ff65b76c740 1 -- 172.21.15.115:0/3488507625 >> 172.21.15.115:0/3488507625 conn(0x55bba4456300 msgr2=0x55bba4801e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 -- 172.21.15.115:0/3488507625 shutdown_connections 2024-05-09T05:41:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 -- 172.21.15.115:0/3488507625 wait complete. 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 -- start start 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 Processor -- start 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba48116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4811c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba480fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bba48127b0 con 0x55bba44414d0 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bba46f19c0 con 0x55bba44621b0 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bba4812930 con 0x55bba44392f0 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff65b76c740 10 client.0 _ll_register_callbacks cb 0x55bba471e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.224+0000 7ff64dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4811c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4811c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47254/0 (socket says 172.21.15.115:47254) 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64dd9b640 1 -- 172.21.15.115:0/2141047900 learned_addr learned my addr 172.21.15.115:0/2141047900 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64e59c640 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba48116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64ed9d640 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba480fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64dd9b640 1 -- 172.21.15.115:0/2141047900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 msgr2=0x55bba48116c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba48116c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64dd9b640 1 -- 172.21.15.115:0/2141047900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 msgr2=0x55bba480fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba480fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64dd9b640 1 -- 172.21.15.115:0/2141047900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba47181f0 con 0x55bba44414d0 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64ed9d640 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba480fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64e59c640 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba48116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4811c00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff6440e0120 tx=0x7ff6440a5b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb4ce4587d4a271 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff635ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff635ffb640 1 -- 172.21.15.115:0/2141047900 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6440a9420 con 0x55bba44414d0 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff635ffb640 1 -- 172.21.15.115:0/2141047900 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff6440a62a0 con 0x55bba44414d0 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff65b76c740 1 -- 172.21.15.115:0/2141047900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff640015070 con 0x55bba44414d0 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff635ffb640 1 -- 172.21.15.115:0/2141047900 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff644010720 con 0x55bba44414d0 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff65b76c740 10 client.16554 fetch_fsmap learned FSMap version 34 2024-05-09T05:41:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.225+0000 7ff65b76c740 1 -- 172.21.15.115:0/2141047900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bba44cd790 con 0x55bba44414d0 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.226+0000 7ff635ffb640 1 -- 172.21.15.115:0/2141047900 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff640015070 con 0x55bba44414d0 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.226+0000 7ff635ffb640 1 -- 172.21.15.115:0/2141047900 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7ff6440a5620 con 0x55bba44414d0 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.226+0000 7ff65b76c740 10 client.16554 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.226+0000 7ff65b76c740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.226+0000 7ff65b76c740 20 client.16554 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.226+0000 7ff65b76c740 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bba4420d80 0x55bba4821fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.226+0000 7ff65b76c740 4 client.16554 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.226+0000 7ff65b76c740 1 -- 172.21.15.115:0/2141047900 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bba47783d0 con 0x55bba4420d80 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.226+0000 7ff64e59c640 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bba4420d80 0x55bba4821fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.227+0000 7ff635ffb640 10 client.16554 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.227+0000 7ff64e59c640 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bba4420d80 0x55bba4821fd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6f374a7f728386 server_cookie=aaa654e9718ac43b in_seq=0 out_seq=0 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.227+0000 7ff635ffb640 1 -- 172.21.15.115:0/2141047900 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bba47783d0 con 0x55bba4420d80 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.227+0000 7ff635ffb640 10 client.16554 handle_command_reply: tid=0 2024-05-09T05:41:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.227+0000 7ff65b76c740 1 client.16554 shutdown 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 -- 172.21.15.115:0/2141047900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 msgr2=0x55bba4811c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4811c00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff6440e0120 tx=0x7ff6440a5b70 comp rx=0 tx=0).stop 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 -- 172.21.15.115:0/2141047900 shutdown_connections 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44621b0 0x55bba480fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44414d0 0x55bba4811c00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bba4420d80 0x55bba4821fd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 --2- 172.21.15.115:0/2141047900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44392f0 0x55bba48116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 -- 172.21.15.115:0/2141047900 >> 172.21.15.115:0/2141047900 conn(0x55bba4456300 msgr2=0x55bba48073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff635ffb640 0 client.16554 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 -- 172.21.15.115:0/2141047900 shutdown_connections 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 1 -- 172.21.15.115:0/2141047900 wait complete. 2024-05-09T05:41:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.228+0000 7ff65b76c740 20 client.16554 trim_cache size 0 max 16384 2024-05-09T05:41:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff65b76c740 1 Processor -- start 2024-05-09T05:41:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff65b76c740 1 -- start start 2024-05-09T05:41:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48277d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 0x55bba4827d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 0x55bba4817be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff65b76c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b10e0 con 0x55bba44392f0 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff65b76c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b2d10 con 0x55bba44414d0 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff65b76c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b17b0 con 0x55bba44621b0 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff64e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff64e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48277d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47260/0 (socket says 172.21.15.115:47260) 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff64e59c640 1 -- 172.21.15.115:0/1676899876 learned_addr learned my addr 172.21.15.115:0/1676899876 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/1676899876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 0x55bba4827d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.246+0000 7ff64ed9d640 1 --2- 172.21.15.115:0/1676899876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 0x55bba4817be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.247+0000 7ff64e59c640 1 -- 172.21.15.115:0/1676899876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 msgr2=0x55bba4817be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.247+0000 7ff64e59c640 1 --2- 172.21.15.115:0/1676899876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 0x55bba4817be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.247+0000 7ff64e59c640 1 -- 172.21.15.115:0/1676899876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 msgr2=0x55bba4827d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.247+0000 7ff64e59c640 1 --2- 172.21.15.115:0/1676899876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 0x55bba4827d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.247+0000 7ff64e59c640 1 -- 172.21.15.115:0/1676899876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba45445f0 con 0x55bba44392f0 2024-05-09T05:41:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.247+0000 7ff64e59c640 1 --2- 172.21.15.115:0/1676899876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48277d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff640000c00 tx=0x7ff640004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42aac92faa52c136 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.247+0000 7ff637fff640 1 -- 172.21.15.115:0/1676899876 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64000be90 con 0x55bba44392f0 2024-05-09T05:41:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.247+0000 7ff637fff640 1 -- 172.21.15.115:0/1676899876 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff640029070 con 0x55bba44392f0 2024-05-09T05:41:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.247+0000 7ff637fff640 1 -- 172.21.15.115:0/1676899876 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6400116a0 con 0x55bba44392f0 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 -- 172.21.15.115:0/1676899876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 msgr2=0x55bba48277d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1676899876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48277d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff640000c00 tx=0x7ff640004560 comp rx=0 tx=0).stop 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 -- 172.21.15.115:0/1676899876 shutdown_connections 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1676899876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 0x55bba4817be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1676899876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 0x55bba4827d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1676899876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48277d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 -- 172.21.15.115:0/1676899876 >> 172.21.15.115:0/1676899876 conn(0x55bba4456300 msgr2=0x55bba480ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 -- 172.21.15.115:0/1676899876 shutdown_connections 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 -- 172.21.15.115:0/1676899876 wait complete. 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 -- start start 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.248+0000 7ff65b76c740 1 Processor -- start 2024-05-09T05:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48180b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 0x55bba4813400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff65b76c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 0x55bba4813940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff65b76c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b25e0 con 0x55bba44392f0 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff65b76c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501b1e80 con 0x55bba44414d0 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff65b76c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501af0c0 con 0x55bba44621b0 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff65b76c740 10 client.0 _ll_register_callbacks cb 0x7ff64400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48180b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47276/0 (socket says 172.21.15.115:47276) 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64e59c640 1 -- 172.21.15.115:0/1631467883 learned_addr learned my addr 172.21.15.115:0/1631467883 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 0x55bba4813400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64ed9d640 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 0x55bba4813940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64e59c640 1 -- 172.21.15.115:0/1631467883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 msgr2=0x55bba4813940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64e59c640 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 0x55bba4813940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64e59c640 1 -- 172.21.15.115:0/1631467883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 msgr2=0x55bba4813400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64e59c640 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 0x55bba4813400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64e59c640 1 -- 172.21.15.115:0/1631467883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba472eed0 con 0x55bba44392f0 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff64e59c640 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48180b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff64000bb60 tx=0x7ff640004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0fe6877391b2cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.249+0000 7ff635ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff635ffb640 1 -- 172.21.15.115:0/1631467883 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640038070 con 0x55bba44392f0 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff635ffb640 1 -- 172.21.15.115:0/1631467883 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff640003cd0 con 0x55bba44392f0 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff65b76c740 1 -- 172.21.15.115:0/1631467883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff640029070 con 0x55bba44392f0 2024-05-09T05:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff635ffb640 1 -- 172.21.15.115:0/1631467883 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff640044050 con 0x55bba44392f0 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff65b76c740 10 client.16560 fetch_fsmap learned FSMap version 34 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff65b76c740 1 -- 172.21.15.115:0/1631467883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bba4889ff0 con 0x55bba44392f0 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff635ffb640 1 -- 172.21.15.115:0/1631467883 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff640029070 con 0x55bba44392f0 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff635ffb640 1 -- 172.21.15.115:0/1631467883 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7ff640034490 con 0x55bba44392f0 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff65b76c740 10 client.16560 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff65b76c740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff65b76c740 20 client.16560 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bba43f61c0 0x55bba488c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff65b76c740 4 client.16560 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff65b76c740 1 -- 172.21.15.115:0/1631467883 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bba4772240 con 0x55bba43f61c0 2024-05-09T05:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.250+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bba43f61c0 0x55bba488c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.251+0000 7ff635ffb640 10 client.16560 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.251+0000 7ff64dd9b640 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bba43f61c0 0x55bba488c610 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0409c1be2c8c4d server_cookie=2ec5af41add3e0b5 in_seq=0 out_seq=0 2024-05-09T05:41:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.260+0000 7ff635ffb640 1 -- 172.21.15.115:0/1631467883 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bba4772240 con 0x55bba43f61c0 2024-05-09T05:41:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff635ffb640 10 client.16560 handle_command_reply: tid=0 2024-05-09T05:41:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 client.16560 shutdown 2024-05-09T05:41:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 -- 172.21.15.115:0/1631467883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 msgr2=0x55bba48180b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48180b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff64000bb60 tx=0x7ff640004290 comp rx=0 tx=0).stop 2024-05-09T05:41:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 -- 172.21.15.115:0/1631467883 shutdown_connections 2024-05-09T05:41:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bba43f61c0 0x55bba488c610 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bba44621b0 0x55bba4813940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bba44414d0 0x55bba4813400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 --2- 172.21.15.115:0/1631467883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bba44392f0 0x55bba48180b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 -- 172.21.15.115:0/1631467883 >> 172.21.15.115:0/1631467883 conn(0x55bba4456300 msgr2=0x55bba480ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff635ffb640 0 client.16560 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 -- 172.21.15.115:0/1631467883 shutdown_connections 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 1 -- 172.21.15.115:0/1631467883 wait complete. 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.261+0000 7ff65b76c740 20 client.16560 trim_cache size 0 max 16384 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:41:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:41:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 -- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6380b2950 msgr2=0x7ff6380b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6380b2950 0x7ff6380b4e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff648009a60 tx=0x7ff648005c50 comp rx=0 tx=0).stop 2024-05-09T05:41:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 -- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010b8c0 msgr2=0x7ff65019acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010b8c0 0x7ff65019acb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff64400e9f0 tx=0x7ff64400eeb0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 -- 172.21.15.115:0/3470503882 shutdown_connections 2024-05-09T05:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6380b2950 0x7ff6380b4e60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff65010e220 0x7ff65019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff65010b8c0 0x7ff65019acb0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 --2- 172.21.15.115:0/3470503882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff650104e60 0x7ff65019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 -- 172.21.15.115:0/3470503882 >> 172.21.15.115:0/3470503882 conn(0x7ff6500782e0 msgr2=0x7ff6501012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 -- 172.21.15.115:0/3470503882 shutdown_connections 2024-05-09T05:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:44.262+0000 7ff6548b8640 1 -- 172.21.15.115:0/3470503882 wait complete. 2024-05-09T05:41:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:45 smithi195 ceph-mon[33844]: pgmap v109: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 30 op/s 2024-05-09T05:41:45.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:45 smithi115 ceph-mon[30349]: pgmap v109: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 30 op/s 2024-05-09T05:41:45.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:45 smithi161 ceph-mon[33805]: pgmap v109: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 30 op/s 2024-05-09T05:41:47.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:47 smithi195 ceph-mon[33844]: pgmap v110: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-05-09T05:41:47.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:47 smithi115 ceph-mon[30349]: pgmap v110: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-05-09T05:41:47.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:47 smithi161 ceph-mon[33805]: pgmap v110: 129 pgs: 129 active+clean; 1.3 GiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-05-09T05:41:49.354 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:41:49.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:49 smithi195 ceph-mon[33844]: pgmap v111: 129 pgs: 129 active+clean; 1.4 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 MiB/s wr, 29 op/s 2024-05-09T05:41:49.630 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:49.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:49 smithi115 ceph-mon[30349]: pgmap v111: 129 pgs: 129 active+clean; 1.4 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 MiB/s wr, 29 op/s 2024-05-09T05:41:49.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:49 smithi161 ceph-mon[33805]: pgmap v111: 129 pgs: 129 active+clean; 1.4 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 MiB/s wr, 29 op/s 2024-05-09T05:41:51.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:51 smithi115 ceph-mon[30349]: pgmap v112: 129 pgs: 129 active+clean; 1.4 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:41:51.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:51 smithi195 ceph-mon[33844]: pgmap v112: 129 pgs: 129 active+clean; 1.4 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:41:51.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:51 smithi161 ceph-mon[33805]: pgmap v112: 129 pgs: 129 active+clean; 1.4 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:41:51.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.689+0000 7fbb01460700 1 -- 172.21.15.115:0/572361932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc109870 msgr2=0x7fbafc1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:51.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.689+0000 7fbb01460700 1 --2- 172.21.15.115:0/572361932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc109870 0x7fbafc1111d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b0d0 tx=0x7fbaf400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.689+0000 7fbb01460700 1 -- 172.21.15.115:0/572361932 shutdown_connections 2024-05-09T05:41:51.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.689+0000 7fbb01460700 1 --2- 172.21.15.115:0/572361932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc109870 0x7fbafc1111d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.689+0000 7fbb01460700 1 --2- 172.21.15.115:0/572361932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbafc1045f0 0x7fbafc104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.689+0000 7fbb01460700 1 --2- 172.21.15.115:0/572361932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbafc108f20 0x7fbafc109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.689+0000 7fbb01460700 1 -- 172.21.15.115:0/572361932 >> 172.21.15.115:0/572361932 conn(0x7fbafc100040 msgr2=0x7fbafc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:51.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.689+0000 7fbb01460700 1 -- 172.21.15.115:0/572361932 shutdown_connections 2024-05-09T05:41:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.689+0000 7fbb01460700 1 -- 172.21.15.115:0/572361932 wait complete. 2024-05-09T05:41:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.690+0000 7fbb01460700 1 Processor -- start 2024-05-09T05:41:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.690+0000 7fbb01460700 1 -- start start 2024-05-09T05:41:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.690+0000 7fbb01460700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc1045f0 0x7fbafc19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.690+0000 7fbb01460700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbafc108f20 0x7fbafc19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.690+0000 7fbb01460700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbafc109870 0x7fbafc19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.690+0000 7fbb01460700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc114350 con 0x7fbafc1045f0 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.690+0000 7fbb01460700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc1141d0 con 0x7fbafc108f20 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.690+0000 7fbb01460700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc1144d0 con 0x7fbafc109870 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.690+0000 7fbafaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc1045f0 0x7fbafc19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc1045f0 0x7fbafc19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47294/0 (socket says 172.21.15.115:47294) 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafaffd700 1 -- 172.21.15.115:0/3188966998 learned_addr learned my addr 172.21.15.115:0/3188966998 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbafc108f20 0x7fbafc19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafb7fe700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbafc109870 0x7fbafc19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafaffd700 1 -- 172.21.15.115:0/3188966998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbafc109870 msgr2=0x7fbafc19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafaffd700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbafc109870 0x7fbafc19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafaffd700 1 -- 172.21.15.115:0/3188966998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbafc108f20 msgr2=0x7fbafc19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafaffd700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbafc108f20 0x7fbafc19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafaffd700 1 -- 172.21.15.115:0/3188966998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc19fc30 con 0x7fbafc1045f0 2024-05-09T05:41:51.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbafaffd700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc1045f0 0x7fbafc19ad60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00e9a0 tx=0x7fbaec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:51.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbae3fff700 1 -- 172.21.15.115:0/3188966998 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec00cbd0 con 0x7fbafc1045f0 2024-05-09T05:41:51.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbb01460700 1 -- 172.21.15.115:0/3188966998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc19ff20 con 0x7fbafc1045f0 2024-05-09T05:41:51.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbae3fff700 1 -- 172.21.15.115:0/3188966998 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaec00cd70 con 0x7fbafc1045f0 2024-05-09T05:41:51.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbae3fff700 1 -- 172.21.15.115:0/3188966998 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec018680 con 0x7fbafc1045f0 2024-05-09T05:41:51.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.691+0000 7fbb01460700 1 -- 172.21.15.115:0/3188966998 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc1a7840 con 0x7fbafc1045f0 2024-05-09T05:41:51.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.694+0000 7fbae3fff700 1 -- 172.21.15.115:0/3188966998 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fbaec018820 con 0x7fbafc1045f0 2024-05-09T05:41:51.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.695+0000 7fbae3fff700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbae40b29f0 0x7fbae40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.695+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbae40b29f0 0x7fbae40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.695+0000 7fbae3fff700 1 -- 172.21.15.115:0/3188966998 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbaec0e2c10 con 0x7fbafc1045f0 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbae40b29f0 0x7fbae40b4f00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbafc103ae0 tx=0x7fbaf00073d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbb0fe76b80 1 Processor -- start 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbb0fe76b80 1 -- start start 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a4087f160 0x564a409184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a40918a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a409126a0 0x564a40912ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564a40834ff0 con 0x564a409126a0 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564a407d37f0 con 0x564a4087f160 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a40830fb0 con 0x564a407954a0 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbafb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a409126a0 0x564a40912ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbafb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a409126a0 0x564a40912ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47298/0 (socket says 172.21.15.115:47298) 2024-05-09T05:41:51.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbafa7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a40918a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbafb7fe700 1 -- 172.21.15.115:0/2277036296 learned_addr learned my addr 172.21.15.115:0/2277036296 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.696+0000 7fbafaffd700 1 --2- 172.21.15.115:0/2277036296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a4087f160 0x564a409184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbafa7fc700 1 -- 172.21.15.115:0/2277036296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a4087f160 msgr2=0x564a409184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/2277036296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a4087f160 0x564a409184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbafa7fc700 1 -- 172.21.15.115:0/2277036296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a409126a0 msgr2=0x564a40912ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/2277036296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a409126a0 0x564a40912ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbafa7fc700 1 -- 172.21.15.115:0/2277036296 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a407cb370 con 0x564a407954a0 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbafb7fe700 1 --2- 172.21.15.115:0/2277036296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a409126a0 0x564a40912ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/2277036296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a40918a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0010c70 tx=0x7fbaf0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b372385151ef95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbae17fa700 1 -- 172.21.15.115:0/2277036296 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf001eae0 con 0x564a407954a0 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbae17fa700 1 -- 172.21.15.115:0/2277036296 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaf001ec80 con 0x564a407954a0 2024-05-09T05:41:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.697+0000 7fbae17fa700 1 -- 172.21.15.115:0/2277036296 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf001c910 con 0x564a407954a0 2024-05-09T05:41:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.698+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/2277036296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 msgr2=0x564a40918a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.698+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/2277036296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a40918a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0010c70 tx=0x7fbaf0013000 comp rx=0 tx=0).stop 2024-05-09T05:41:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.698+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/2277036296 shutdown_connections 2024-05-09T05:41:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.698+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/2277036296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a4087f160 0x564a409184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.698+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/2277036296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a409126a0 0x564a40912ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.698+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/2277036296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a40918a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.698+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/2277036296 >> 172.21.15.115:0/2277036296 conn(0x564a407f56a0 msgr2=0x564a4090d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.698+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/2277036296 shutdown_connections 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.698+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/2277036296 wait complete. 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.699+0000 7fbb0fe76b80 1 -- start start 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.699+0000 7fbb0fe76b80 1 Processor -- start 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.699+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a4091c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.699+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4091c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.699+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 0x564a4091cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.699+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564a406b0ba0 con 0x564a4087f160 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.699+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564a407ef150 con 0x564a409126a0 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.699+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564a4091f1d0 con 0x564a407954a0 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.699+0000 7fbb0fe76b80 10 client.0 _ll_register_callbacks cb 0x564a406a66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafaffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a4091c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4091c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafaffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a4091c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:56360/0 (socket says 172.21.15.115:56360) 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafaffd700 1 -- 172.21.15.115:0/1669038270 learned_addr learned my addr 172.21.15.115:0/1669038270 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafb7fe700 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 0x564a4091cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafa7fc700 1 -- 172.21.15.115:0/1669038270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 msgr2=0x564a4091c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a4091c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafa7fc700 1 -- 172.21.15.115:0/1669038270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 msgr2=0x564a4091cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 0x564a4091cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafa7fc700 1 -- 172.21.15.115:0/1669038270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a406b0490 con 0x564a4087f160 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafb7fe700 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 0x564a4091cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4091c800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0004410 tx=0x7fbaf001eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f27b1701146a01a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbadeff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbadeff5700 1 -- 172.21.15.115:0/1669038270 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0036060 con 0x564a4087f160 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbadeff5700 1 -- 172.21.15.115:0/1669038270 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbaf001e250 con 0x564a4087f160 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.700+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/1669038270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf001ec80 con 0x564a4087f160 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbadeff5700 1 -- 172.21.15.115:0/1669038270 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbaf002c040 con 0x564a4087f160 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbb0fe76b80 10 client.16572 fetch_fsmap learned FSMap version 34 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/1669038270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564a40929c90 con 0x564a4087f160 2024-05-09T05:41:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbadeff5700 1 -- 172.21.15.115:0/1669038270 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaf001e450 con 0x564a4087f160 2024-05-09T05:41:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbadeff5700 1 -- 172.21.15.115:0/1669038270 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fbaf002f030 con 0x564a4087f160 2024-05-09T05:41:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbb0fe76b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:41:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbb0fe76b80 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbb0fe76b80 20 client.16572 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564a4092c2d0 0x564a4092e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbb0fe76b80 4 client.16572 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/1669038270 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbaf001ec80 con 0x564a4092c2d0 2024-05-09T05:41:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.701+0000 7fbafaffd700 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564a4092c2d0 0x564a4092e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.702+0000 7fbafaffd700 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564a4092c2d0 0x564a4092e6f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5998effda905bbed server_cookie=13331e99356fba38 in_seq=0 out_seq=0 2024-05-09T05:41:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.702+0000 7fbadeff5700 10 client.16572 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.702+0000 7fbadeff5700 1 -- 172.21.15.115:0/1669038270 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fbaf001ec80 con 0x564a4092c2d0 2024-05-09T05:41:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbadeff5700 10 client.16572 handle_command_reply: tid=0 2024-05-09T05:41:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 client.16572 shutdown 2024-05-09T05:41:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/1669038270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 msgr2=0x564a4091c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4091c800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0004410 tx=0x7fbaf001eae0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/1669038270 shutdown_connections 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 0x564a4091cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564a4092c2d0 0x564a4092e6f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a407954a0 0x564a4091c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/1669038270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4091c800 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/1669038270 >> 172.21.15.115:0/1669038270 conn(0x564a407f56a0 msgr2=0x564a4090d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbadeff5700 0 client.16572 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/1669038270 shutdown_connections 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/1669038270 wait complete. 2024-05-09T05:41:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.703+0000 7fbb0fe76b80 20 client.16572 trim_cache size 0 max 16384 2024-05-09T05:41:51.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.730+0000 7fbb0fe76b80 1 Processor -- start 2024-05-09T05:41:51.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.730+0000 7fbb0fe76b80 1 -- start start 2024-05-09T05:41:51.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.730+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a407954a0 0x564a4091b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.730+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a4087f160 0x564a4091cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.730+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 0x564a40928360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.730+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc1ae1a0 con 0x564a407954a0 2024-05-09T05:41:51.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.730+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc1afdd0 con 0x564a409126a0 2024-05-09T05:41:51.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.730+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc1ae870 con 0x564a4087f160 2024-05-09T05:41:51.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a407954a0 0x564a4091b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a407954a0 0x564a4091b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47316/0 (socket says 172.21.15.115:47316) 2024-05-09T05:41:51.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafaffd700 1 -- 172.21.15.115:0/436234757 learned_addr learned my addr 172.21.15.115:0/436234757 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a4087f160 0x564a4091cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafb7fe700 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 0x564a40928360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafaffd700 1 -- 172.21.15.115:0/436234757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a4087f160 msgr2=0x564a4091cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafaffd700 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a4087f160 0x564a4091cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafaffd700 1 -- 172.21.15.115:0/436234757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 msgr2=0x564a40928360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafaffd700 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 0x564a40928360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafaffd700 1 -- 172.21.15.115:0/436234757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a40917990 con 0x564a407954a0 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a4087f160 0x564a4091cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbafaffd700 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a407954a0 0x564a4091b5e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0a5c30 tx=0x7fbaec01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef79c0c886dc1b3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbae0ff9700 1 -- 172.21.15.115:0/436234757 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0cae60 con 0x564a407954a0 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbae0ff9700 1 -- 172.21.15.115:0/436234757 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaec0a9bb0 con 0x564a407954a0 2024-05-09T05:41:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.731+0000 7fbae0ff9700 1 -- 172.21.15.115:0/436234757 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0aa070 con 0x564a407954a0 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.732+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/436234757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a407954a0 msgr2=0x564a4091b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.732+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a407954a0 0x564a4091b5e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0a5c30 tx=0x7fbaec01f000 comp rx=0 tx=0).stop 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.732+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/436234757 shutdown_connections 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.732+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a409126a0 0x564a40928360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.732+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a407954a0 0x564a4091b5e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.732+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/436234757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a4087f160 0x564a4091cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.732+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/436234757 >> 172.21.15.115:0/436234757 conn(0x564a407f56a0 msgr2=0x564a40915e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.732+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/436234757 shutdown_connections 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/436234757 wait complete. 2024-05-09T05:41:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 1 -- start start 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 1 Processor -- start 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a407954a0 0x564a4092f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4092f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a409126a0 0x564a4092fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc1af6a0 con 0x564a4087f160 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc1aef40 con 0x564a407954a0 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc1ac160 con 0x564a409126a0 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.733+0000 7fbb0fe76b80 10 client.0 _ll_register_callbacks cb 0x564a406a66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4092f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4092f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47322/0 (socket says 172.21.15.115:47322) 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafa7fc700 1 -- 172.21.15.115:0/773205148 learned_addr learned my addr 172.21.15.115:0/773205148 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafaffd700 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a407954a0 0x564a4092f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafb7fe700 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a409126a0 0x564a4092fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafa7fc700 1 -- 172.21.15.115:0/773205148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a409126a0 msgr2=0x564a4092fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a409126a0 0x564a4092fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafa7fc700 1 -- 172.21.15.115:0/773205148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a407954a0 msgr2=0x564a4092f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a407954a0 0x564a4092f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafa7fc700 1 -- 172.21.15.115:0/773205148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a40995820 con 0x564a4087f160 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbafa7fc700 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4092f900 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x564a4090f260 tx=0x7fbaf0004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9a6f16e17e35cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbadeff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbadeff5700 1 -- 172.21.15.115:0/773205148 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0036040 con 0x564a4087f160 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbadeff5700 1 -- 172.21.15.115:0/773205148 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbaf001e2e0 con 0x564a4087f160 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/773205148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaec0a9bb0 con 0x564a4087f160 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.734+0000 7fbadeff5700 1 -- 172.21.15.115:0/773205148 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbaf0049050 con 0x564a4087f160 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbb0fe76b80 10 client.16578 fetch_fsmap learned FSMap version 34 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/773205148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564a40996a90 con 0x564a4087f160 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbadeff5700 1 -- 172.21.15.115:0/773205148 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaec0a9bb0 con 0x564a4087f160 2024-05-09T05:41:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbadeff5700 1 -- 172.21.15.115:0/773205148 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fbaf002f040 con 0x564a4087f160 2024-05-09T05:41:51.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbb0fe76b80 10 client.16578 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:41:51.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbb0fe76b80 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:51.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbb0fe76b80 20 client.16578 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:51.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564a409970b0 0x564a409994d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:51.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbb0fe76b80 4 client.16578 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:41:51.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/773205148 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a408429d0 con 0x564a409970b0 2024-05-09T05:41:51.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.735+0000 7fbafaffd700 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564a409970b0 0x564a409994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbadeff5700 10 client.16578 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbafaffd700 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564a409970b0 0x564a409994d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6918223dba81126 server_cookie=8d7c4c7c0113ec84 in_seq=0 out_seq=0 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbadeff5700 1 -- 172.21.15.115:0/773205148 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564a408429d0 con 0x564a409970b0 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbadeff5700 10 client.16578 handle_command_reply: tid=0 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 client.16578 shutdown 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/773205148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 msgr2=0x564a4092f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4092f900 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x564a4090f260 tx=0x7fbaf0004910 comp rx=0 tx=0).stop 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/773205148 shutdown_connections 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564a407954a0 0x564a4092f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564a409970b0 0x564a409994d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbadeff5700 0 client.16578 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564a409126a0 0x564a4092fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 --2- 172.21.15.115:0/773205148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564a4087f160 0x564a4092f900 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/773205148 >> 172.21.15.115:0/773205148 conn(0x564a407f56a0 msgr2=0x564a4091b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/773205148 shutdown_connections 2024-05-09T05:41:51.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 1 -- 172.21.15.115:0/773205148 wait complete. 2024-05-09T05:41:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.736+0000 7fbb0fe76b80 20 client.16578 trim_cache size 0 max 16384 2024-05-09T05:41:51.735 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:41:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:41:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:41:51.735 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:41:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 -- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbae40b29f0 msgr2=0x7fbae40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbae40b29f0 0x7fbae40b4f00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbafc103ae0 tx=0x7fbaf00073d0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 -- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc1045f0 msgr2=0x7fbafc19ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc1045f0 0x7fbafc19ad60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00e9a0 tx=0x7fbaec00ecb0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 -- 172.21.15.115:0/3188966998 shutdown_connections 2024-05-09T05:41:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbae40b29f0 0x7fbae40b4f00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbafc109870 0x7fbafc19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbafc1045f0 0x7fbafc19ad60 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 --2- 172.21.15.115:0/3188966998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbafc108f20 0x7fbafc19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:51.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 -- 172.21.15.115:0/3188966998 >> 172.21.15.115:0/3188966998 conn(0x7fbafc100040 msgr2=0x7fbafc1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:51.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.737+0000 7fbb01460700 1 -- 172.21.15.115:0/3188966998 shutdown_connections 2024-05-09T05:41:51.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:51.738+0000 7fbb01460700 1 -- 172.21.15.115:0/3188966998 wait complete. 2024-05-09T05:41:52.559 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:41:52.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:41:52.846 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:53.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:53 smithi195 ceph-mon[33844]: pgmap v113: 129 pgs: 129 active+clean; 1.4 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:41:53.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:53 smithi115 ceph-mon[30349]: pgmap v113: 129 pgs: 129 active+clean; 1.4 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:41:53.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:53 smithi161 ceph-mon[33805]: pgmap v113: 129 pgs: 129 active+clean; 1.4 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:41:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.647+0000 7f9666101700 1 -- 172.21.15.115:0/572256578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 msgr2=0x7f96601100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.647+0000 7f9666101700 1 --2- 172.21.15.115:0/572256578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 0x7f96601100a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f965800b0d0 tx=0x7f965800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.647+0000 7f9666101700 1 -- 172.21.15.115:0/572256578 shutdown_connections 2024-05-09T05:41:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.647+0000 7f9666101700 1 --2- 172.21.15.115:0/572256578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96601034c0 0x7f96601038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.647+0000 7f9666101700 1 --2- 172.21.15.115:0/572256578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 0x7f96601100a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.647+0000 7f9666101700 1 --2- 172.21.15.115:0/572256578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9660107df0 0x7f9660108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.647+0000 7f9666101700 1 -- 172.21.15.115:0/572256578 >> 172.21.15.115:0/572256578 conn(0x7f96600fedc0 msgr2=0x7f96601011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.647+0000 7f9666101700 1 -- 172.21.15.115:0/572256578 shutdown_connections 2024-05-09T05:41:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.647+0000 7f9666101700 1 -- 172.21.15.115:0/572256578 wait complete. 2024-05-09T05:41:54.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f9666101700 1 Processor -- start 2024-05-09T05:41:54.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f9666101700 1 -- start start 2024-05-09T05:41:54.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f9666101700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96601034c0 0x7f966019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:54.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f9666101700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9660107df0 0x7f966019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f9666101700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 0x7f966019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f9666101700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9660113220 con 0x7f9660108740 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f9666101700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f96601130a0 con 0x7f96601034c0 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f9666101700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96601133a0 con 0x7f9660107df0 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f965ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 0x7f966019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f965ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 0x7f966019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50028/0 (socket says 172.21.15.115:50028) 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f965ffff700 1 -- 172.21.15.115:0/3096379238 learned_addr learned my addr 172.21.15.115:0/3096379238 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f965f7fe700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96601034c0 0x7f966019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.648+0000 7f965effd700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9660107df0 0x7f966019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965ffff700 1 -- 172.21.15.115:0/3096379238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9660107df0 msgr2=0x7f966019c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965ffff700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9660107df0 0x7f966019c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965ffff700 1 -- 172.21.15.115:0/3096379238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96601034c0 msgr2=0x7f966019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965ffff700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96601034c0 0x7f966019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965ffff700 1 -- 172.21.15.115:0/3096379238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96601a6e70 con 0x7f9660108740 2024-05-09T05:41:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965f7fe700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96601034c0 0x7f966019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965ffff700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 0x7f966019c5c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9658009850 tx=0x7f9658007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965cff9700 1 -- 172.21.15.115:0/3096379238 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965801f070 con 0x7f9660108740 2024-05-09T05:41:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96601a7100 con 0x7f9660108740 2024-05-09T05:41:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965cff9700 1 -- 172.21.15.115:0/3096379238 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9658004c50 con 0x7f9660108740 2024-05-09T05:41:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f965cff9700 1 -- 172.21.15.115:0/3096379238 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965801b690 con 0x7f9660108740 2024-05-09T05:41:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.649+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96601a75b0 con 0x7f9660108740 2024-05-09T05:41:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.650+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96600512c0 con 0x7f9660108740 2024-05-09T05:41:54.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.653+0000 7f965cff9700 1 -- 172.21.15.115:0/3096379238 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f9658022070 con 0x7f9660108740 2024-05-09T05:41:54.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.654+0000 7f965cff9700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96480b29f0 0x7f96480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:54.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.654+0000 7f965f7fe700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96480b29f0 0x7f96480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:54.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.654+0000 7f965cff9700 1 -- 172.21.15.115:0/3096379238 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f965800f020 con 0x7f9660108740 2024-05-09T05:41:54.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.654+0000 7f965f7fe700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96480b29f0 0x7f96480b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9650011490 tx=0x7f9650009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.658+0000 7f965cff9700 1 -- 172.21.15.115:0/3096379238 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9658029020 con 0x7f9660108740 2024-05-09T05:41:54.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.975+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f966002e5f0 con 0x7f9660108740 2024-05-09T05:41:54.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.976+0000 7f965cff9700 1 -- 172.21.15.115:0/3096379238 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11961 (secure 0 0 0) 0x7f96580ab560 con 0x7f9660108740 2024-05-09T05:41:54.969 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:41:54.969 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:41:33.359460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:41:54.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96480b29f0 msgr2=0x7f96480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:54.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96480b29f0 0x7f96480b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9650011490 tx=0x7f9650009260 comp rx=0 tx=0).stop 2024-05-09T05:41:54.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 msgr2=0x7f966019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:54.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 0x7f966019c5c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9658009850 tx=0x7f9658007a20 comp rx=0 tx=0).stop 2024-05-09T05:41:54.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 shutdown_connections 2024-05-09T05:41:54.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f96480b29f0 0x7f96480b4f00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96601034c0 0x7f966019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9660108740 0x7f966019c5c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 --2- 172.21.15.115:0/3096379238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9660107df0 0x7f966019c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:54.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 >> 172.21.15.115:0/3096379238 conn(0x7f96600fedc0 msgr2=0x7f96601096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:54.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 shutdown_connections 2024-05-09T05:41:54.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:54.979+0000 7f9666101700 1 -- 172.21.15.115:0/3096379238 wait complete. 2024-05-09T05:41:54.973 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 34 2024-05-09T05:41:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:55 smithi195 ceph-mon[33844]: pgmap v114: 129 pgs: 129 active+clean; 1.6 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:41:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:55 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3096379238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:41:55.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:55 smithi115 ceph-mon[30349]: pgmap v114: 129 pgs: 129 active+clean; 1.6 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:41:55.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:55 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3096379238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:41:55.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:55 smithi161 ceph-mon[33805]: pgmap v114: 129 pgs: 129 active+clean; 1.6 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:41:55.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:55 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3096379238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:41:55.830 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:41:56.117 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.378+0000 7fa939300640 1 -- 172.21.15.115:0/253683000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 msgr2=0x7fa934102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.378+0000 7fa939300640 1 --2- 172.21.15.115:0/253683000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 0x7fa934102740 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa924009900 tx=0x7fa92402a410 comp rx=0 tx=0).stop 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.378+0000 7fa939300640 1 -- 172.21.15.115:0/253683000 shutdown_connections 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.378+0000 7fa939300640 1 --2- 172.21.15.115:0/253683000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341032f0 0x7fa93410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.378+0000 7fa939300640 1 --2- 172.21.15.115:0/253683000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa934109040 0x7fa934102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.378+0000 7fa939300640 1 --2- 172.21.15.115:0/253683000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 0x7fa934102740 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.378+0000 7fa939300640 1 -- 172.21.15.115:0/253683000 >> 172.21.15.115:0/253683000 conn(0x7fa9340782e0 msgr2=0x7fa934078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.378+0000 7fa939300640 1 -- 172.21.15.115:0/253683000 shutdown_connections 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.378+0000 7fa939300640 1 -- 172.21.15.115:0/253683000 wait complete. 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa939300640 1 Processor -- start 2024-05-09T05:41:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa939300640 1 -- start start 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa939300640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9341032f0 0x7fa93419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa939300640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 0x7fa93419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa939300640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa934109040 0x7fa9341a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa939300640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa934112c40 con 0x7fa934108ba0 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa939300640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa934112ac0 con 0x7fa9341032f0 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa939300640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa934112dc0 con 0x7fa934109040 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 0x7fa93419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 0x7fa93419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50040/0 (socket says 172.21.15.115:50040) 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa92bfff640 1 -- 172.21.15.115:0/895008235 learned_addr learned my addr 172.21.15.115:0/895008235 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa932ffd640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9341032f0 0x7fa93419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.379+0000 7fa9337fe640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa934109040 0x7fa9341a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa92bfff640 1 -- 172.21.15.115:0/895008235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa934109040 msgr2=0x7fa9341a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa92bfff640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa934109040 0x7fa9341a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa92bfff640 1 -- 172.21.15.115:0/895008235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9341032f0 msgr2=0x7fa93419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa92bfff640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9341032f0 0x7fa93419b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa92bfff640 1 -- 172.21.15.115:0/895008235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93419c3e0 con 0x7fa934108ba0 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa9337fe640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa934109040 0x7fa9341a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa92bfff640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 0x7fa93419bb60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005fd0 tx=0x7fa91c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa930ff9640 1 -- 172.21.15.115:0/895008235 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c017070 con 0x7fa934108ba0 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa930ff9640 1 -- 172.21.15.115:0/895008235 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa91c009c30 con 0x7fa934108ba0 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa930ff9640 1 -- 172.21.15.115:0/895008235 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c003c90 con 0x7fa934108ba0 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa939300640 1 -- 172.21.15.115:0/895008235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9341a6ca0 con 0x7fa934108ba0 2024-05-09T05:41:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.380+0000 7fa939300640 1 -- 172.21.15.115:0/895008235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9341a71e0 con 0x7fa934108ba0 2024-05-09T05:41:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa9401b1740 1 Processor -- start 2024-05-09T05:41:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa9401b1740 1 -- start start 2024-05-09T05:41:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa930ff9640 1 -- 172.21.15.115:0/895008235 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa91c003e30 con 0x7fa934108ba0 2024-05-09T05:41:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd806680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 0x55f4fd806bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd4424d0 0x55f4fd807100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa9401b1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f4fd76b7c0 con 0x55f4fd4631b0 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa9401b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4fd7070e0 con 0x55f4fd43a2f0 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa9401b1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f4fd767780 con 0x55f4fd4424d0 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.383+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd806680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd806680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50046/0 (socket says 172.21.15.115:50046) 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9337fe640 1 -- 172.21.15.115:0/2238863959 learned_addr learned my addr 172.21.15.115:0/2238863959 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa932ffd640 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 0x55f4fd806bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa92bfff640 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd4424d0 0x55f4fd807100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9337fe640 1 -- 172.21.15.115:0/2238863959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd4424d0 msgr2=0x55f4fd807100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9337fe640 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd4424d0 0x55f4fd807100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9337fe640 1 -- 172.21.15.115:0/2238863959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 msgr2=0x55f4fd806bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:57.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9337fe640 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 0x55f4fd806bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9337fe640 1 -- 172.21.15.115:0/2238863959 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4fd7307c0 con 0x55f4fd4631b0 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa932ffd640 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 0x55f4fd806bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9337fe640 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd806680 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00f360 tx=0x7fa92c00f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6867110aebbffc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa930ff9640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29d0 0x7fa9200b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9297fa640 1 -- 172.21.15.115:0/2238863959 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c015070 con 0x55f4fd4631b0 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa932ffd640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29d0 0x7fa9200b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9297fa640 1 -- 172.21.15.115:0/2238863959 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa92c011b90 con 0x55f4fd4631b0 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa9297fa640 1 -- 172.21.15.115:0/2238863959 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c010b60 con 0x55f4fd4631b0 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.384+0000 7fa930ff9640 1 -- 172.21.15.115:0/895008235 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa91c0e2420 con 0x7fa934108ba0 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.385+0000 7fa9401b1740 1 -- 172.21.15.115:0/2238863959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 msgr2=0x55f4fd806680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.385+0000 7fa932ffd640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29d0 0x7fa9200b4ee0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa924004770 tx=0x7fa9240058d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.385+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd806680 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00f360 tx=0x7fa92c00f820 comp rx=0 tx=0).stop 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.385+0000 7fa9401b1740 1 -- 172.21.15.115:0/2238863959 shutdown_connections 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.385+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd4424d0 0x55f4fd807100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.385+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 0x55f4fd806bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.385+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2238863959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd806680 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.385+0000 7fa9401b1740 1 -- 172.21.15.115:0/2238863959 >> 172.21.15.115:0/2238863959 conn(0x55f4fd457300 msgr2=0x55f4fd802e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.385+0000 7fa9401b1740 1 -- 172.21.15.115:0/2238863959 shutdown_connections 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 1 -- 172.21.15.115:0/2238863959 wait complete. 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 1 -- start start 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 1 Processor -- start 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd43a2f0 0x55f4fd8126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 0x55f4fd812c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd810ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f4fd8137b0 con 0x55f4fd4631b0 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4fd6f29c0 con 0x55f4fd4424d0 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f4fd813930 con 0x55f4fd43a2f0 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.386+0000 7fa9401b1740 10 client.0 _ll_register_callbacks cb 0x55f4fd71f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd810ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd810ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50056/0 (socket says 172.21.15.115:50056) 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9337fe640 1 -- 172.21.15.115:0/3392663436 learned_addr learned my addr 172.21.15.115:0/3392663436 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa932ffd640 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd43a2f0 0x55f4fd8126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa92bfff640 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 0x55f4fd812c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9337fe640 1 -- 172.21.15.115:0/3392663436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd43a2f0 msgr2=0x55f4fd8126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9337fe640 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd43a2f0 0x55f4fd8126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9337fe640 1 -- 172.21.15.115:0/3392663436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 msgr2=0x55f4fd812c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9337fe640 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 0x55f4fd812c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9337fe640 1 -- 172.21.15.115:0/3392663436 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4fd7191f0 con 0x55f4fd4631b0 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa932ffd640 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd43a2f0 0x55f4fd8126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa92bfff640 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 0x55f4fd812c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9337fe640 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd810ba0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0106e0 tx=0x7fa92c009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72440e65db433b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa91a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa91a7fc640 1 -- 172.21.15.115:0/3392663436 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c015050 con 0x55f4fd4631b0 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9401b1740 1 -- 172.21.15.115:0/3392663436 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa92c011b90 con 0x55f4fd4631b0 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa91a7fc640 1 -- 172.21.15.115:0/3392663436 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa92c004c50 con 0x55f4fd4631b0 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa91a7fc640 1 -- 172.21.15.115:0/3392663436 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa92c012e90 con 0x55f4fd4631b0 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9401b1740 10 client.16596 fetch_fsmap learned FSMap version 34 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.387+0000 7fa9401b1740 1 -- 172.21.15.115:0/3392663436 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f4fd4ce790 con 0x55f4fd4631b0 2024-05-09T05:41:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.388+0000 7fa91a7fc640 1 -- 172.21.15.115:0/3392663436 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa92c011b90 con 0x55f4fd4631b0 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.388+0000 7fa91a7fc640 1 -- 172.21.15.115:0/3392663436 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fa92c005040 con 0x55f4fd4631b0 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.388+0000 7fa9401b1740 10 client.16596 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.388+0000 7fa9401b1740 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.388+0000 7fa9401b1740 20 client.16596 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.388+0000 7fa9401b1740 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4fd421d80 0x55f4fd822f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.388+0000 7fa9401b1740 4 client.16596 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.388+0000 7fa9401b1740 1 -- 172.21.15.115:0/3392663436 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4fd7793d0 con 0x55f4fd421d80 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.388+0000 7fa932ffd640 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4fd421d80 0x55f4fd822f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.389+0000 7fa932ffd640 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4fd421d80 0x55f4fd822f30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15a6011434c20cb server_cookie=5bcf423518cbe8bd in_seq=0 out_seq=0 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.389+0000 7fa91a7fc640 10 client.16596 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.389+0000 7fa91a7fc640 1 -- 172.21.15.115:0/3392663436 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4fd7793d0 con 0x55f4fd421d80 2024-05-09T05:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.389+0000 7fa91a7fc640 10 client.16596 handle_command_reply: tid=0 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 client.16596 shutdown 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 -- 172.21.15.115:0/3392663436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 msgr2=0x55f4fd810ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd810ba0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0106e0 tx=0x7fa92c009d70 comp rx=0 tx=0).stop 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 -- 172.21.15.115:0/3392663436 shutdown_connections 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4631b0 0x55f4fd810ba0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 0x55f4fd812c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4fd421d80 0x55f4fd822f30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa91a7fc640 0 client.16596 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 --2- 172.21.15.115:0/3392663436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd43a2f0 0x55f4fd8126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 -- 172.21.15.115:0/3392663436 >> 172.21.15.115:0/3392663436 conn(0x55f4fd457300 msgr2=0x55f4fd801700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 -- 172.21.15.115:0/3392663436 shutdown_connections 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 1 -- 172.21.15.115:0/3392663436 wait complete. 2024-05-09T05:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.390+0000 7fa9401b1740 20 client.16596 trim_cache size 0 max 16384 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9401b1740 1 Processor -- start 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9401b1740 1 -- start start 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 0x55f4fd811270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 0x55f4fd8285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4424d0 0x55f4fd828b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9401b1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b1140 con 0x55f4fd4424d0 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9401b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b2d70 con 0x55f4fd43a2f0 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9401b1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b1810 con 0x55f4fd421d80 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4424d0 0x55f4fd828b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa932ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 0x55f4fd811270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4424d0 0x55f4fd828b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50072/0 (socket says 172.21.15.115:50072) 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa9337fe640 1 -- 172.21.15.115:0/2843286951 learned_addr learned my addr 172.21.15.115:0/2843286951 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.407+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 0x55f4fd8285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.408+0000 7fa9337fe640 1 -- 172.21.15.115:0/2843286951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 msgr2=0x55f4fd811270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.408+0000 7fa9337fe640 1 --2- 172.21.15.115:0/2843286951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 0x55f4fd811270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.408+0000 7fa9337fe640 1 -- 172.21.15.115:0/2843286951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 msgr2=0x55f4fd8285c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.408+0000 7fa9337fe640 1 --2- 172.21.15.115:0/2843286951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 0x55f4fd8285c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.408+0000 7fa9337fe640 1 -- 172.21.15.115:0/2843286951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4fd5455f0 con 0x55f4fd4424d0 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.408+0000 7fa9337fe640 1 --2- 172.21.15.115:0/2843286951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4424d0 0x55f4fd828b00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009f50 tx=0x7fa92c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbc91f4b7330063 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.408+0000 7fa928ff9640 1 -- 172.21.15.115:0/2843286951 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c015070 con 0x55f4fd4424d0 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.408+0000 7fa928ff9640 1 -- 172.21.15.115:0/2843286951 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa92c010450 con 0x55f4fd4424d0 2024-05-09T05:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.408+0000 7fa928ff9640 1 -- 172.21.15.115:0/2843286951 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c010910 con 0x55f4fd4424d0 2024-05-09T05:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.409+0000 7fa9401b1740 1 -- 172.21.15.115:0/2843286951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4424d0 msgr2=0x55f4fd828b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.409+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2843286951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4424d0 0x55f4fd828b00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009f50 tx=0x7fa92c005fd0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.409+0000 7fa9401b1740 1 -- 172.21.15.115:0/2843286951 shutdown_connections 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.409+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2843286951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd4424d0 0x55f4fd828b00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.409+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2843286951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd43a2f0 0x55f4fd8285c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.409+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2843286951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 0x55f4fd811270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.409+0000 7fa9401b1740 1 -- 172.21.15.115:0/2843286951 >> 172.21.15.115:0/2843286951 conn(0x55f4fd457300 msgr2=0x55f4fd810b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.409+0000 7fa9401b1740 1 -- 172.21.15.115:0/2843286951 shutdown_connections 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.409+0000 7fa9401b1740 1 -- 172.21.15.115:0/2843286951 wait complete. 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9401b1740 1 -- start start 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9401b1740 1 Processor -- start 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 0x55f4fd883670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd43a2f0 0x55f4fd883bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9401b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 0x55f4fd886310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9401b1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b2640 con 0x55f4fd43a2f0 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9401b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341b1ee0 con 0x55f4fd4424d0 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9401b1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341af120 con 0x55f4fd421d80 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9401b1740 10 client.0 _ll_register_callbacks cb 0x7fa92c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd43a2f0 0x55f4fd883bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa92bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd43a2f0 0x55f4fd883bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50086/0 (socket says 172.21.15.115:50086) 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa92bfff640 1 -- 172.21.15.115:0/2463669327 learned_addr learned my addr 172.21.15.115:0/2463669327 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa932ffd640 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 0x55f4fd883670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa9337fe640 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 0x55f4fd886310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.410+0000 7fa92bfff640 1 -- 172.21.15.115:0/2463669327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 msgr2=0x55f4fd883670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa92bfff640 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 0x55f4fd883670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa92bfff640 1 -- 172.21.15.115:0/2463669327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 msgr2=0x55f4fd886310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa92bfff640 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 0x55f4fd886310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa92bfff640 1 -- 172.21.15.115:0/2463669327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4fd72fed0 con 0x55f4fd43a2f0 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa92bfff640 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd43a2f0 0x55f4fd883bb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0058c0 tx=0x7fa91c0dfe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81c667040a3f810 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa91a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa91a7fc640 1 -- 172.21.15.115:0/2463669327 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0e5e50 con 0x55f4fd43a2f0 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa91a7fc640 1 -- 172.21.15.115:0/2463669327 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa91c0b0b90 con 0x55f4fd43a2f0 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa9401b1740 1 -- 172.21.15.115:0/2463669327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa92c010450 con 0x55f4fd43a2f0 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa91a7fc640 1 -- 172.21.15.115:0/2463669327 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa91c0035f0 con 0x55f4fd43a2f0 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa9401b1740 10 client.16602 fetch_fsmap learned FSMap version 34 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.411+0000 7fa9401b1740 1 -- 172.21.15.115:0/2463669327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f4fd816840 con 0x55f4fd43a2f0 2024-05-09T05:41:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.412+0000 7fa91a7fc640 1 -- 172.21.15.115:0/2463669327 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa92c010450 con 0x55f4fd43a2f0 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.412+0000 7fa91a7fc640 1 -- 172.21.15.115:0/2463669327 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fa91c0a3b80 con 0x55f4fd43a2f0 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.412+0000 7fa9401b1740 10 client.16602 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.412+0000 7fa9401b1740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.412+0000 7fa9401b1740 20 client.16602 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.412+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4fd4631b0 0x55f4fd891650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.412+0000 7fa9401b1740 4 client.16602 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.412+0000 7fa9401b1740 1 -- 172.21.15.115:0/2463669327 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4fd773240 con 0x55f4fd4631b0 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.412+0000 7fa932ffd640 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4fd4631b0 0x55f4fd891650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.413+0000 7fa91a7fc640 10 client.16602 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.413+0000 7fa932ffd640 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4fd4631b0 0x55f4fd891650 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eda8feb02d39eb3 server_cookie=3e381afeb2090dd5 in_seq=0 out_seq=0 2024-05-09T05:41:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.421+0000 7fa91a7fc640 1 -- 172.21.15.115:0/2463669327 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f4fd773240 con 0x55f4fd4631b0 2024-05-09T05:41:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.421+0000 7fa91a7fc640 10 client.16602 handle_command_reply: tid=0 2024-05-09T05:41:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.421+0000 7fa9401b1740 1 client.16602 shutdown 2024-05-09T05:41:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 -- 172.21.15.115:0/2463669327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd43a2f0 msgr2=0x55f4fd883bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd43a2f0 0x55f4fd883bb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0058c0 tx=0x7fa91c0dfe10 comp rx=0 tx=0).stop 2024-05-09T05:41:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 -- 172.21.15.115:0/2463669327 shutdown_connections 2024-05-09T05:41:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4fd4424d0 0x55f4fd886310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4fd43a2f0 0x55f4fd883bb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4fd4631b0 0x55f4fd891650 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 --2- 172.21.15.115:0/2463669327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4fd421d80 0x55f4fd883670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 -- 172.21.15.115:0/2463669327 >> 172.21.15.115:0/2463669327 conn(0x55f4fd457300 msgr2=0x55f4fd80f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa91a7fc640 0 client.16602 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:41:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 -- 172.21.15.115:0/2463669327 shutdown_connections 2024-05-09T05:41:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 1 -- 172.21.15.115:0/2463669327 wait complete. 2024-05-09T05:41:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.422+0000 7fa9401b1740 20 client.16602 trim_cache size 0 max 16384 2024-05-09T05:41:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:41:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:41:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:41:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 -- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29d0 msgr2=0x7fa9200b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29d0 0x7fa9200b4ee0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa924004770 tx=0x7fa9240058d0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 -- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 msgr2=0x7fa93419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 0x7fa93419bb60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005fd0 tx=0x7fa91c00dde0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 -- 172.21.15.115:0/895008235 shutdown_connections 2024-05-09T05:41:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29d0 0x7fa9200b4ee0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa934109040 0x7fa9341a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934108ba0 0x7fa93419bb60 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 --2- 172.21.15.115:0/895008235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9341032f0 0x7fa93419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 -- 172.21.15.115:0/895008235 >> 172.21.15.115:0/895008235 conn(0x7fa9340782e0 msgr2=0x7fa934100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 -- 172.21.15.115:0/895008235 shutdown_connections 2024-05-09T05:41:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:41:57.423+0000 7fa939300640 1 -- 172.21.15.115:0/895008235 wait complete. 2024-05-09T05:41:57.579 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:57 smithi115 ceph-mon[30349]: pgmap v115: 129 pgs: 129 active+clean; 1.6 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:41:57.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:57 smithi195 ceph-mon[33844]: pgmap v115: 129 pgs: 129 active+clean; 1.6 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:41:57.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:57 smithi161 ceph-mon[33805]: pgmap v115: 129 pgs: 129 active+clean; 1.6 GiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:41:58.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.355+0000 7f1a05932700 1 -- 172.21.15.115:0/567561659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00107df0 msgr2=0x7f1a00108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:58.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.355+0000 7f1a05932700 1 --2- 172.21.15.115:0/567561659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00107df0 0x7f1a00108200 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f19f40099c0 tx=0x7f19f4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.355+0000 7f1a05932700 1 -- 172.21.15.115:0/567561659 shutdown_connections 2024-05-09T05:41:58.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.355+0000 7f1a05932700 1 --2- 172.21.15.115:0/567561659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a00108740 0x7f1a001100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.355+0000 7f1a05932700 1 --2- 172.21.15.115:0/567561659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a001034c0 0x7f1a001038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.355+0000 7f1a05932700 1 --2- 172.21.15.115:0/567561659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00107df0 0x7f1a00108200 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.355+0000 7f1a05932700 1 -- 172.21.15.115:0/567561659 >> 172.21.15.115:0/567561659 conn(0x7f1a000fedc0 msgr2=0x7f1a001011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:58.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.355+0000 7f1a05932700 1 -- 172.21.15.115:0/567561659 shutdown_connections 2024-05-09T05:41:58.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.355+0000 7f1a05932700 1 -- 172.21.15.115:0/567561659 wait complete. 2024-05-09T05:41:58.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.356+0000 7f1a05932700 1 Processor -- start 2024-05-09T05:41:58.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.356+0000 7f1a05932700 1 -- start start 2024-05-09T05:41:58.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f1a05932700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a001034c0 0x7f1a0019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f1a05932700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a00107df0 0x7f1a0019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f1a05932700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00108740 0x7f1a0019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f1a05932700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00113200 con 0x7f1a00108740 2024-05-09T05:41:58.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f1a05932700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00113080 con 0x7f1a001034c0 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f1a05932700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00113380 con 0x7f1a00107df0 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00108740 0x7f1a0019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00108740 0x7f1a0019c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50102/0 (socket says 172.21.15.115:50102) 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19ff7fe700 1 -- 172.21.15.115:0/3000307327 learned_addr learned my addr 172.21.15.115:0/3000307327 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19feffd700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a001034c0 0x7f1a0019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a00107df0 0x7f1a0019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19ff7fe700 1 -- 172.21.15.115:0/3000307327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a00107df0 msgr2=0x7f1a0019c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a00107df0 0x7f1a0019c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19ff7fe700 1 -- 172.21.15.115:0/3000307327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a001034c0 msgr2=0x7f1a0019bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a001034c0 0x7f1a0019bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19ff7fe700 1 -- 172.21.15.115:0/3000307327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a001a6eb0 con 0x7f1a00108740 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.357+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00108740 0x7f1a0019c600 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007ee0 tx=0x7f19f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.358+0000 7f1a04930700 1 -- 172.21.15.115:0/3000307327 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8004bb0 con 0x7f1a00108740 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.358+0000 7f1a04930700 1 -- 172.21.15.115:0/3000307327 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f8004d50 con 0x7f1a00108740 2024-05-09T05:41:58.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.358+0000 7f1a05932700 1 -- 172.21.15.115:0/3000307327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a001a71a0 con 0x7f1a00108740 2024-05-09T05:41:58.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.358+0000 7f1a04930700 1 -- 172.21.15.115:0/3000307327 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f80058b0 con 0x7f1a00108740 2024-05-09T05:41:58.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.358+0000 7f1a05932700 1 -- 172.21.15.115:0/3000307327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a001a76b0 con 0x7f1a00108740 2024-05-09T05:41:58.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.361+0000 7f1a04930700 1 -- 172.21.15.115:0/3000307327 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f19f801a020 con 0x7f1a00108740 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a04930700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f19e80b2a70 0x7f19e80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f19feffd700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f19e80b2a70 0x7f19e80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a14348b80 1 Processor -- start 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a04930700 1 -- 172.21.15.115:0/3000307327 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f19f80e1f10 con 0x7f1a00108740 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a14348b80 1 -- start start 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 0x55edbb3e44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb3de6a0 0x55edbb3dead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a14348b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55edbb300ff0 con 0x55edbb2614a0 2024-05-09T05:41:58.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a14348b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55edbb29f7f0 con 0x55edbb3de6a0 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f1a14348b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55edbb2fcfb0 con 0x55edbb34b160 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.362+0000 7f19feffd700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f19e80b2a70 0x7f19e80b4f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005950 tx=0x7f19f000b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19fe7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19fe7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50108/0 (socket says 172.21.15.115:50108) 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19fe7fc700 1 -- 172.21.15.115:0/3622492923 learned_addr learned my addr 172.21.15.115:0/3622492923 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19feffd700 1 --2- 172.21.15.115:0/3622492923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 0x55edbb3e44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19fe7fc700 1 -- 172.21.15.115:0/3622492923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 msgr2=0x55edbb3e44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/3622492923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 0x55edbb3e44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19fe7fc700 1 -- 172.21.15.115:0/3622492923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb3de6a0 msgr2=0x55edbb3dead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/3622492923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb3de6a0 0x55edbb3dead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19fe7fc700 1 -- 172.21.15.115:0/3622492923 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edbb297370 con 0x55edbb2614a0 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/3622492923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3e4a30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1a00076ab0 tx=0x7f19f4005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7ef46b49c8c927 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19e5ffb700 1 -- 172.21.15.115:0/3622492923 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4027070 con 0x55edbb2614a0 2024-05-09T05:41:58.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19e5ffb700 1 -- 172.21.15.115:0/3622492923 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f400b790 con 0x55edbb2614a0 2024-05-09T05:41:58.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.363+0000 7f19e5ffb700 1 -- 172.21.15.115:0/3622492923 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40038e0 con 0x55edbb2614a0 2024-05-09T05:41:58.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.364+0000 7f1a14348b80 1 -- 172.21.15.115:0/3622492923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 msgr2=0x55edbb3e4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:58.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.364+0000 7f1a14348b80 1 --2- 172.21.15.115:0/3622492923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3e4a30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1a00076ab0 tx=0x7f19f4005ea0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.364+0000 7f1a14348b80 1 -- 172.21.15.115:0/3622492923 shutdown_connections 2024-05-09T05:41:58.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.364+0000 7f1a14348b80 1 --2- 172.21.15.115:0/3622492923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb3de6a0 0x55edbb3dead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.364+0000 7f1a14348b80 1 --2- 172.21.15.115:0/3622492923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 0x55edbb3e44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.364+0000 7f1a14348b80 1 --2- 172.21.15.115:0/3622492923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3e4a30 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.364+0000 7f1a14348b80 1 -- 172.21.15.115:0/3622492923 >> 172.21.15.115:0/3622492923 conn(0x55edbb2c16a0 msgr2=0x55edbb3d9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:58.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.364+0000 7f1a14348b80 1 -- 172.21.15.115:0/3622492923 shutdown_connections 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.364+0000 7f1a14348b80 1 -- 172.21.15.115:0/3622492923 wait complete. 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.365+0000 7f1a14348b80 1 -- start start 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.365+0000 7f1a14348b80 1 Processor -- start 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.365+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb2614a0 0x55edbb3e8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.365+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 0x55edbb3e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.365+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.365+0000 7f1a14348b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55edbb17cba0 con 0x55edbb3de6a0 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.365+0000 7f1a14348b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55edbb2bb150 con 0x55edbb34b160 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.365+0000 7f1a14348b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55edbb3ef5e0 con 0x55edbb2614a0 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.365+0000 7f1a14348b80 10 client.0 _ll_register_callbacks cb 0x55edbb1726c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3e8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50124/0 (socket says 172.21.15.115:50124) 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19ff7fe700 1 -- 172.21.15.115:0/1977798409 learned_addr learned my addr 172.21.15.115:0/1977798409 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 0x55edbb3e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19feffd700 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb2614a0 0x55edbb3e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19ff7fe700 1 -- 172.21.15.115:0/1977798409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb2614a0 msgr2=0x55edbb3e8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb2614a0 0x55edbb3e8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19ff7fe700 1 -- 172.21.15.115:0/1977798409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 msgr2=0x55edbb3e87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 0x55edbb3e87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19ff7fe700 1 -- 172.21.15.115:0/1977798409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edbb17c490 con 0x55edbb3de6a0 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 0x55edbb3e87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3e8d00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f19f80df3d0 tx=0x7f19f80a53f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48897ffe543f4a6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19deffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19deffd700 1 -- 172.21.15.115:0/1977798409 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f80a95b0 con 0x55edbb3de6a0 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f19deffd700 1 -- 172.21.15.115:0/1977798409 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f19f80a9320 con 0x55edbb3de6a0 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.366+0000 7f1a14348b80 1 -- 172.21.15.115:0/1977798409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f400b790 con 0x55edbb3de6a0 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f19deffd700 1 -- 172.21.15.115:0/1977798409 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f19f80a8eb0 con 0x55edbb3de6a0 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f1a14348b80 10 client.16614 fetch_fsmap learned FSMap version 34 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f1a14348b80 1 -- 172.21.15.115:0/1977798409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55edbb3fa120 con 0x55edbb3de6a0 2024-05-09T05:41:58.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f19deffd700 1 -- 172.21.15.115:0/1977798409 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f400b790 con 0x55edbb3de6a0 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f19deffd700 1 -- 172.21.15.115:0/1977798409 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f19f800db20 con 0x55edbb3de6a0 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f1a14348b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f1a14348b80 10 client.16614 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f1a14348b80 20 client.16614 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f1a14348b80 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55edbb3fc5f0 0x55edbb3fea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f1a14348b80 4 client.16614 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f1a14348b80 1 -- 172.21.15.115:0/1977798409 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edbb30e9d0 con 0x55edbb3fc5f0 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.367+0000 7f19feffd700 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55edbb3fc5f0 0x55edbb3fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.368+0000 7f19deffd700 10 client.16614 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.368+0000 7f19feffd700 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55edbb3fc5f0 0x55edbb3fea10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a00759e8052da1c server_cookie=949a4162e40ef397 in_seq=0 out_seq=0 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f19deffd700 1 -- 172.21.15.115:0/1977798409 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55edbb30e9d0 con 0x55edbb3fc5f0 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f19deffd700 10 client.16614 handle_command_reply: tid=0 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 client.16614 shutdown 2024-05-09T05:41:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 -- 172.21.15.115:0/1977798409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 msgr2=0x55edbb3e8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3e8d00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f19f80df3d0 tx=0x7f19f80a53f0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 -- 172.21.15.115:0/1977798409 shutdown_connections 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55edbb3fc5f0 0x55edbb3fea10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3e8d00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb2614a0 0x55edbb3e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 --2- 172.21.15.115:0/1977798409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 0x55edbb3e87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f19deffd700 0 client.16614 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 -- 172.21.15.115:0/1977798409 >> 172.21.15.115:0/1977798409 conn(0x55edbb2c16a0 msgr2=0x55edbb3d91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 -- 172.21.15.115:0/1977798409 shutdown_connections 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 1 -- 172.21.15.115:0/1977798409 wait complete. 2024-05-09T05:41:58.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.369+0000 7f1a14348b80 20 client.16614 trim_cache size 0 max 16384 2024-05-09T05:41:58.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f1a14348b80 1 Processor -- start 2024-05-09T05:41:58.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f1a14348b80 1 -- start start 2024-05-09T05:41:58.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb2614a0 0x55edbb404150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 0x55edbb404690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3f4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f1a14348b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ae0e0 con 0x55edbb3de6a0 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f1a14348b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001afd10 con 0x55edbb2614a0 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f1a14348b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ae7b0 con 0x55edbb34b160 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50130/0 (socket says 172.21.15.115:50130) 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f19ff7fe700 1 -- 172.21.15.115:0/133943366 learned_addr learned my addr 172.21.15.115:0/133943366 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/133943366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 0x55edbb404690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.387+0000 7f19feffd700 1 --2- 172.21.15.115:0/133943366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb2614a0 0x55edbb404150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.388+0000 7f19ff7fe700 1 -- 172.21.15.115:0/133943366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 msgr2=0x55edbb404690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.388+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/133943366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 0x55edbb404690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.388+0000 7f19ff7fe700 1 -- 172.21.15.115:0/133943366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb2614a0 msgr2=0x55edbb404150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.388+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/133943366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb2614a0 0x55edbb404150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.388+0000 7f19ff7fe700 1 -- 172.21.15.115:0/133943366 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edbb3df640 con 0x55edbb3de6a0 2024-05-09T05:41:58.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.388+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/133943366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3f4790 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f19f8019ff0 tx=0x7f19f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb284794b649c3d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:58.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.388+0000 7f19e57fa700 1 -- 172.21.15.115:0/133943366 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8087030 con 0x55edbb3de6a0 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.388+0000 7f19e57fa700 1 -- 172.21.15.115:0/133943366 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f80a95b0 con 0x55edbb3de6a0 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.388+0000 7f19e57fa700 1 -- 172.21.15.115:0/133943366 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f80b1880 con 0x55edbb3de6a0 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.389+0000 7f1a14348b80 1 -- 172.21.15.115:0/133943366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 msgr2=0x55edbb3f4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.389+0000 7f1a14348b80 1 --2- 172.21.15.115:0/133943366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3f4790 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f19f8019ff0 tx=0x7f19f8004ad0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.389+0000 7f1a14348b80 1 -- 172.21.15.115:0/133943366 shutdown_connections 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.389+0000 7f1a14348b80 1 --2- 172.21.15.115:0/133943366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb2614a0 0x55edbb404150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.389+0000 7f1a14348b80 1 --2- 172.21.15.115:0/133943366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb3de6a0 0x55edbb3f4790 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.389+0000 7f1a14348b80 1 --2- 172.21.15.115:0/133943366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb34b160 0x55edbb404690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.389+0000 7f1a14348b80 1 -- 172.21.15.115:0/133943366 >> 172.21.15.115:0/133943366 conn(0x55edbb2c16a0 msgr2=0x55edbb3e3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.389+0000 7f1a14348b80 1 -- 172.21.15.115:0/133943366 shutdown_connections 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.389+0000 7f1a14348b80 1 -- 172.21.15.115:0/133943366 wait complete. 2024-05-09T05:41:58.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f1a14348b80 1 -- start start 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f1a14348b80 1 Processor -- start 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3f1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 0x55edbb3f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f1a14348b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb3de6a0 0x55edbb464d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f1a14348b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001af5e0 con 0x55edbb2614a0 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f1a14348b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001aee80 con 0x55edbb34b160 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f1a14348b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ac080 con 0x55edbb3de6a0 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f1a14348b80 10 client.0 _ll_register_callbacks cb 0x55edbb1726c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f19feffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f19feffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50142/0 (socket says 172.21.15.115:50142) 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.390+0000 7f19feffd700 1 -- 172.21.15.115:0/954106903 learned_addr learned my addr 172.21.15.115:0/954106903 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 0x55edbb3f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19ff7fe700 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb3de6a0 0x55edbb464d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19feffd700 1 -- 172.21.15.115:0/954106903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb3de6a0 msgr2=0x55edbb464d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19feffd700 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb3de6a0 0x55edbb464d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19feffd700 1 -- 172.21.15.115:0/954106903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 msgr2=0x55edbb3f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19feffd700 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 0x55edbb3f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19feffd700 1 -- 172.21.15.115:0/954106903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edbb3dad00 con 0x55edbb2614a0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19feffd700 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3f1790 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005d90 tx=0x7f19f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2684128eb7acf37f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19deffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19deffd700 1 -- 172.21.15.115:0/954106903 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f002a6f0 con 0x55edbb2614a0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f19deffd700 1 -- 172.21.15.115:0/954106903 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f19f002a890 con 0x55edbb2614a0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.391+0000 7f1a14348b80 1 -- 172.21.15.115:0/954106903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f80a95b0 con 0x55edbb2614a0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f19deffd700 1 -- 172.21.15.115:0/954106903 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f19f0039050 con 0x55edbb2614a0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f1a14348b80 10 client.16620 fetch_fsmap learned FSMap version 34 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f1a14348b80 1 -- 172.21.15.115:0/954106903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55edbb4661a0 con 0x55edbb2614a0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f19deffd700 1 -- 172.21.15.115:0/954106903 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19f80a95b0 con 0x55edbb2614a0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f19deffd700 1 -- 172.21.15.115:0/954106903 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f19f00293a0 con 0x55edbb2614a0 2024-05-09T05:41:58.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f1a14348b80 10 client.16620 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:41:58.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f1a14348b80 10 client.16620 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:41:58.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f1a14348b80 20 client.16620 populate_metadata read hostname 'smithi115' 2024-05-09T05:41:58.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f1a14348b80 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55edbb4668a0 0x55edbb468cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:41:58.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f1a14348b80 4 client.16620 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:41:58.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f1a14348b80 1 -- 172.21.15.115:0/954106903 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edbb308a70 con 0x55edbb4668a0 2024-05-09T05:41:58.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.392+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55edbb4668a0 0x55edbb468cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:41:58.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.393+0000 7f19deffd700 10 client.16620 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:41:58.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.393+0000 7f19fe7fc700 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55edbb4668a0 0x55edbb468cc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2778a3332462f187 server_cookie=48dfa66202cb8fad in_seq=0 out_seq=0 2024-05-09T05:41:58.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.393+0000 7f19deffd700 1 -- 172.21.15.115:0/954106903 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edbb308a70 con 0x55edbb4668a0 2024-05-09T05:41:58.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.393+0000 7f19deffd700 10 client.16620 handle_command_reply: tid=0 2024-05-09T05:41:58.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.393+0000 7f1a14348b80 1 client.16620 shutdown 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.393+0000 7f1a14348b80 1 -- 172.21.15.115:0/954106903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 msgr2=0x55edbb3f1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.393+0000 7f1a14348b80 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3f1790 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005d90 tx=0x7f19f00047c0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a14348b80 1 -- 172.21.15.115:0/954106903 shutdown_connections 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a14348b80 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55edbb4668a0 0x55edbb468cc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a14348b80 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55edbb3de6a0 0x55edbb464d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a14348b80 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55edbb2614a0 0x55edbb3f1790 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f19deffd700 0 client.16620 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a14348b80 1 --2- 172.21.15.115:0/954106903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55edbb34b160 0x55edbb3f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a14348b80 1 -- 172.21.15.115:0/954106903 >> 172.21.15.115:0/954106903 conn(0x55edbb2c16a0 msgr2=0x55edbb3e1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:58.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a14348b80 1 -- 172.21.15.115:0/954106903 shutdown_connections 2024-05-09T05:41:58.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a14348b80 1 -- 172.21.15.115:0/954106903 wait complete. 2024-05-09T05:41:58.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a14348b80 20 client.16620 trim_cache size 0 max 16384 2024-05-09T05:41:58.391 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:41:58.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a05932700 1 -- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f19e80b2a70 msgr2=0x7f19e80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:58.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a05932700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f19e80b2a70 0x7f19e80b4f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005950 tx=0x7f19f000b410 comp rx=0 tx=0).stop 2024-05-09T05:41:58.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a05932700 1 -- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00108740 msgr2=0x7f1a0019c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:41:58.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.394+0000 7f1a05932700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00108740 0x7f1a0019c600 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007ee0 tx=0x7f19f800d3b0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.395+0000 7f1a05932700 1 -- 172.21.15.115:0/3000307327 shutdown_connections 2024-05-09T05:41:58.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.395+0000 7f1a05932700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f19e80b2a70 0x7f19e80b4f80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.395+0000 7f1a05932700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a001034c0 0x7f1a0019bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.395+0000 7f1a05932700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a00108740 0x7f1a0019c600 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.395+0000 7f1a05932700 1 --2- 172.21.15.115:0/3000307327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a00107df0 0x7f1a0019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:41:58.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.395+0000 7f1a05932700 1 -- 172.21.15.115:0/3000307327 >> 172.21.15.115:0/3000307327 conn(0x7f1a000fedc0 msgr2=0x7f1a001011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:41:58.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.395+0000 7f1a05932700 1 -- 172.21.15.115:0/3000307327 shutdown_connections 2024-05-09T05:41:58.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:41:58.395+0000 7f1a05932700 1 -- 172.21.15.115:0/3000307327 wait complete. 2024-05-09T05:41:59.261 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:41:59.532 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:41:59.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:41:59 smithi195 ceph-mon[33844]: pgmap v116: 129 pgs: 129 active+clean; 1.7 GiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:41:59.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:41:59 smithi115 ceph-mon[30349]: pgmap v116: 129 pgs: 129 active+clean; 1.7 GiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:41:59.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:41:59 smithi161 ceph-mon[33805]: pgmap v116: 129 pgs: 129 active+clean; 1.7 GiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:42:01.366 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:01 smithi115 ceph-mon[30349]: pgmap v117: 129 pgs: 129 active+clean; 1.7 GiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.500+0000 7f667ce67700 1 -- 172.21.15.115:0/3904928003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 msgr2=0x7f6678110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.500+0000 7f667ce67700 1 --2- 172.21.15.115:0/3904928003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 0x7f6678110140 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f667000b0d0 tx=0x7f667000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.500+0000 7f667ce67700 1 -- 172.21.15.115:0/3904928003 shutdown_connections 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.500+0000 7f667ce67700 1 --2- 172.21.15.115:0/3904928003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678103b30 0x7f6678103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.500+0000 7f667ce67700 1 --2- 172.21.15.115:0/3904928003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 0x7f6678110140 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.500+0000 7f667ce67700 1 --2- 172.21.15.115:0/3904928003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6678107e50 0x7f6678108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.500+0000 7f667ce67700 1 -- 172.21.15.115:0/3904928003 >> 172.21.15.115:0/3904928003 conn(0x7f66780fe520 msgr2=0x7f6678100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.500+0000 7f667ce67700 1 -- 172.21.15.115:0/3904928003 shutdown_connections 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.500+0000 7f667ce67700 1 -- 172.21.15.115:0/3904928003 wait complete. 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.501+0000 7f667ce67700 1 Processor -- start 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.501+0000 7f667ce67700 1 -- start start 2024-05-09T05:42:01.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.501+0000 7f667ce67700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678103b30 0x7f667819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.501+0000 7f667ce67700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6678107e50 0x7f667819c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.501+0000 7f667ce67700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 0x7f667819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.501+0000 7f667ce67700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f66781132c0 con 0x7f66781087a0 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.501+0000 7f667ce67700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6678113140 con 0x7f6678103b30 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.501+0000 7f667ce67700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6678113440 con 0x7f6678107e50 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f6676d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 0x7f667819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f667659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678103b30 0x7f667819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f6676d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 0x7f667819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50162/0 (socket says 172.21.15.115:50162) 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f6676d9d700 1 -- 172.21.15.115:0/1692128672 learned_addr learned my addr 172.21.15.115:0/1692128672 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f666ffff700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6678107e50 0x7f667819c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f6676d9d700 1 -- 172.21.15.115:0/1692128672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6678107e50 msgr2=0x7f667819c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f6676d9d700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6678107e50 0x7f667819c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f6676d9d700 1 -- 172.21.15.115:0/1692128672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678103b30 msgr2=0x7f667819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f6676d9d700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678103b30 0x7f667819bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f6676d9d700 1 -- 172.21.15.115:0/1692128672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66781a6f10 con 0x7f66781087a0 2024-05-09T05:42:01.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f6676d9d700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 0x7f667819c660 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6670009850 tx=0x7f6670007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:01.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f666f7fe700 1 -- 172.21.15.115:0/1692128672 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667001f070 con 0x7f66781087a0 2024-05-09T05:42:01.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f666f7fe700 1 -- 172.21.15.115:0/1692128672 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6670005500 con 0x7f66781087a0 2024-05-09T05:42:01.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f667ce67700 1 -- 172.21.15.115:0/1692128672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6678199990 con 0x7f66781087a0 2024-05-09T05:42:01.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.502+0000 7f666f7fe700 1 -- 172.21.15.115:0/1692128672 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667001b8f0 con 0x7f66781087a0 2024-05-09T05:42:01.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.503+0000 7f667ce67700 1 -- 172.21.15.115:0/1692128672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6678199e70 con 0x7f66781087a0 2024-05-09T05:42:01.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.506+0000 7f666f7fe700 1 -- 172.21.15.115:0/1692128672 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f6670022070 con 0x7f66781087a0 2024-05-09T05:42:01.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f668b87db80 1 Processor -- start 2024-05-09T05:42:01.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f666f7fe700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66640b29f0 0x7f66640b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f668b87db80 1 -- start start 2024-05-09T05:42:01.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f666f7fe700 1 -- 172.21.15.115:0/1692128672 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f667000f020 con 0x7f66781087a0 2024-05-09T05:42:01.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f666ffff700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66640b29f0 0x7f66640b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f668b87db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc118d00 0x5650bc221240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f668b87db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc09e4a0 0x5650bc221780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f668b87db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc21b400 0x5650bc21b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f668b87db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5650bc13e0a0 con 0x5650bc21b400 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f668b87db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5650bc0e39a0 con 0x5650bc09e4a0 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f668b87db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5650bc13a060 con 0x5650bc118d00 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f6676d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc21b400 0x5650bc21b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f6676d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc21b400 0x5650bc21b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50172/0 (socket says 172.21.15.115:50172) 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f6676d9d700 1 -- 172.21.15.115:0/3437356563 learned_addr learned my addr 172.21.15.115:0/3437356563 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.507+0000 7f667659c700 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc118d00 0x5650bc221240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f666ffff700 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc09e4a0 0x5650bc221780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f666ffff700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66640b29f0 0x7f66640b4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6678101bf0 tx=0x7f666000b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f667659c700 1 -- 172.21.15.115:0/3437356563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc09e4a0 msgr2=0x5650bc221780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f667659c700 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc09e4a0 0x5650bc221780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f667659c700 1 -- 172.21.15.115:0/3437356563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc21b400 msgr2=0x5650bc21b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f667659c700 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc21b400 0x5650bc21b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f667659c700 1 -- 172.21.15.115:0/3437356563 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bc0d4500 con 0x5650bc118d00 2024-05-09T05:42:01.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f6676d9d700 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc21b400 0x5650bc21b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:01.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f667659c700 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc118d00 0x5650bc221240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66680092f0 tx=0x7f6668009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d412e8c8faba63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:01.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.508+0000 7f666cff9700 1 -- 172.21.15.115:0/3437356563 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668004030 con 0x5650bc118d00 2024-05-09T05:42:01.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.509+0000 7f666cff9700 1 -- 172.21.15.115:0/3437356563 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6668004830 con 0x5650bc118d00 2024-05-09T05:42:01.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.509+0000 7f666cff9700 1 -- 172.21.15.115:0/3437356563 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668005280 con 0x5650bc118d00 2024-05-09T05:42:01.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.509+0000 7f668b87db80 1 -- 172.21.15.115:0/3437356563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc118d00 msgr2=0x5650bc221240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:01.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.509+0000 7f668b87db80 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc118d00 0x5650bc221240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66680092f0 tx=0x7f6668009600 comp rx=0 tx=0).stop 2024-05-09T05:42:01.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.509+0000 7f668b87db80 1 -- 172.21.15.115:0/3437356563 shutdown_connections 2024-05-09T05:42:01.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.509+0000 7f668b87db80 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc21b400 0x5650bc21b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.509+0000 7f668b87db80 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc118d00 0x5650bc221240 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.510+0000 7f668b87db80 1 --2- 172.21.15.115:0/3437356563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc09e4a0 0x5650bc221780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.510+0000 7f668b87db80 1 -- 172.21.15.115:0/3437356563 >> 172.21.15.115:0/3437356563 conn(0x5650bc0a1f70 msgr2=0x5650bc216220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.510+0000 7f668b87db80 1 -- 172.21.15.115:0/3437356563 shutdown_connections 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.510+0000 7f668b87db80 1 -- 172.21.15.115:0/3437356563 wait complete. 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.510+0000 7f668b87db80 1 -- start start 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.510+0000 7f668b87db80 1 Processor -- start 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f668b87db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc09e4a0 0x5650bc224f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f668b87db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc118d00 0x5650bc225460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f668b87db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc21b400 0x5650bc2259a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f668b87db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5650bc21c160 con 0x5650bc118d00 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f668b87db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5650bc0d8b20 con 0x5650bc21b400 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f668b87db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5650bc228020 con 0x5650bc09e4a0 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f668b87db80 10 client.0 _ll_register_callbacks cb 0x5650bbfaf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f667659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc09e4a0 0x5650bc224f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f6676d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc21b400 0x5650bc2259a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f666ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc118d00 0x5650bc225460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f667659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc09e4a0 0x5650bc224f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:51192/0 (socket says 172.21.15.115:51192) 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f6676d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc21b400 0x5650bc2259a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49214/0 (socket says 172.21.15.115:49214) 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f667659c700 1 -- 172.21.15.115:0/38678464 learned_addr learned my addr 172.21.15.115:0/38678464 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f667659c700 1 -- 172.21.15.115:0/38678464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc21b400 msgr2=0x5650bc2259a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f667659c700 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc21b400 0x5650bc2259a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f667659c700 1 -- 172.21.15.115:0/38678464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc118d00 msgr2=0x5650bc225460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f667659c700 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc118d00 0x5650bc225460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f667659c700 1 -- 172.21.15.115:0/38678464 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bbfb9490 con 0x5650bc09e4a0 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f667659c700 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc09e4a0 0x5650bc224f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f666800fad0 tx=0x7f6668009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f5ff7810d8f716 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.511+0000 7f665dffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f665dffb700 1 -- 172.21.15.115:0/38678464 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668028020 con 0x5650bc09e4a0 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f665dffb700 1 -- 172.21.15.115:0/38678464 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6668004f90 con 0x5650bc09e4a0 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f668b87db80 1 -- 172.21.15.115:0/38678464 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6668004830 con 0x5650bc09e4a0 2024-05-09T05:42:01.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f665dffb700 1 -- 172.21.15.115:0/38678464 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6668033050 con 0x5650bc09e4a0 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f668b87db80 10 client.26170 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f668b87db80 1 -- 172.21.15.115:0/38678464 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5650bc232900 con 0x5650bc09e4a0 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f665dffb700 1 -- 172.21.15.115:0/38678464 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6668004830 con 0x5650bc09e4a0 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f665dffb700 1 -- 172.21.15.115:0/38678464 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f6668003b40 con 0x5650bc09e4a0 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f668b87db80 10 client.26170 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f668b87db80 10 client.26170 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.512+0000 7f668b87db80 20 client.26170 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.513+0000 7f668b87db80 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5650bc234e80 0x5650bc2372a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.513+0000 7f668b87db80 4 client.26170 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.513+0000 7f668b87db80 1 -- 172.21.15.115:0/38678464 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650bc14ba80 con 0x5650bc234e80 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.513+0000 7f666ffff700 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5650bc234e80 0x5650bc2372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.513+0000 7f665dffb700 10 client.26170 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.513+0000 7f666ffff700 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5650bc234e80 0x5650bc2372a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97ea9471f3c7a653 server_cookie=f84371eb46ce415e in_seq=0 out_seq=0 2024-05-09T05:42:01.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f665dffb700 1 -- 172.21.15.115:0/38678464 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5650bc14ba80 con 0x5650bc234e80 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f665dffb700 10 client.26170 handle_command_reply: tid=0 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 client.26170 shutdown 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 -- 172.21.15.115:0/38678464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc09e4a0 msgr2=0x5650bc224f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc09e4a0 0x5650bc224f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f666800fad0 tx=0x7f6668009f70 comp rx=0 tx=0).stop 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 -- 172.21.15.115:0/38678464 shutdown_connections 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc21b400 0x5650bc2259a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5650bc234e80 0x5650bc2372a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc09e4a0 0x5650bc224f20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 --2- 172.21.15.115:0/38678464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc118d00 0x5650bc225460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f665dffb700 0 client.26170 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 -- 172.21.15.115:0/38678464 >> 172.21.15.115:0/38678464 conn(0x5650bc0a1f70 msgr2=0x5650bc20fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 -- 172.21.15.115:0/38678464 shutdown_connections 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 1 -- 172.21.15.115:0/38678464 wait complete. 2024-05-09T05:42:01.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.514+0000 7f668b87db80 20 client.26170 trim_cache size 0 max 16384 2024-05-09T05:42:01.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.537+0000 7f668b87db80 1 Processor -- start 2024-05-09T05:42:01.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.537+0000 7f668b87db80 1 -- start start 2024-05-09T05:42:01.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.537+0000 7f668b87db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc2311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.537+0000 7f668b87db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 0x5650bc210a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.537+0000 7f668b87db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 0x5650bc2257e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.537+0000 7f668b87db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f66781ae140 con 0x5650bc09e4a0 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f668b87db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f66781afd70 con 0x5650bc118d00 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f668b87db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66781ae810 con 0x5650bc21b400 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f667659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc2311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f667659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc2311f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50182/0 (socket says 172.21.15.115:50182) 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f667659c700 1 -- 172.21.15.115:0/1889482406 learned_addr learned my addr 172.21.15.115:0/1889482406 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f6676d9d700 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 0x5650bc2257e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f666ffff700 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 0x5650bc210a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f667659c700 1 -- 172.21.15.115:0/1889482406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 msgr2=0x5650bc2257e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f667659c700 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 0x5650bc2257e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f667659c700 1 -- 172.21.15.115:0/1889482406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 msgr2=0x5650bc210a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f667659c700 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 0x5650bc210a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f667659c700 1 -- 172.21.15.115:0/1889482406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bc2277c0 con 0x5650bc09e4a0 2024-05-09T05:42:01.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f666ffff700 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 0x5650bc210a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:01.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f667659c700 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc2311f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6668000c00 tx=0x7f6668024710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc5d63d6dd2bba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:01.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.538+0000 7f665ffff700 1 -- 172.21.15.115:0/1889482406 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666803c070 con 0x5650bc09e4a0 2024-05-09T05:42:01.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.539+0000 7f665ffff700 1 -- 172.21.15.115:0/1889482406 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66680047e0 con 0x5650bc09e4a0 2024-05-09T05:42:01.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.539+0000 7f665ffff700 1 -- 172.21.15.115:0/1889482406 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668028070 con 0x5650bc09e4a0 2024-05-09T05:42:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 -- 172.21.15.115:0/1889482406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 msgr2=0x5650bc2311f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc2311f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6668000c00 tx=0x7f6668024710 comp rx=0 tx=0).stop 2024-05-09T05:42:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 -- 172.21.15.115:0/1889482406 shutdown_connections 2024-05-09T05:42:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 0x5650bc2257e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc2311f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 --2- 172.21.15.115:0/1889482406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 0x5650bc210a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 -- 172.21.15.115:0/1889482406 >> 172.21.15.115:0/1889482406 conn(0x5650bc0a1f70 msgr2=0x5650bc2206f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 -- 172.21.15.115:0/1889482406 shutdown_connections 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 -- 172.21.15.115:0/1889482406 wait complete. 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 -- start start 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 Processor -- start 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc23a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 0x5650bc2379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.540+0000 7f668b87db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 0x5650bc237f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f668b87db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f66781af640 con 0x5650bc09e4a0 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f668b87db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f66781aeee0 con 0x5650bc118d00 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f668b87db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66781ac0e0 con 0x5650bc21b400 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f668b87db80 10 client.0 _ll_register_callbacks cb 0x5650bbfaf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f667659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc23a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f667659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc23a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50198/0 (socket says 172.21.15.115:50198) 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f667659c700 1 -- 172.21.15.115:0/1789705903 learned_addr learned my addr 172.21.15.115:0/1789705903 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:01.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f6676d9d700 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 0x5650bc237f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f666ffff700 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 0x5650bc2379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f667659c700 1 -- 172.21.15.115:0/1789705903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 msgr2=0x5650bc237f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f667659c700 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 0x5650bc237f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f667659c700 1 -- 172.21.15.115:0/1789705903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 msgr2=0x5650bc2379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f667659c700 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 0x5650bc2379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f667659c700 1 -- 172.21.15.115:0/1789705903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bc2384b0 con 0x5650bc09e4a0 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f6676d9d700 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 0x5650bc237f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f667659c700 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc23a660 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6668009950 tx=0x7f6668004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcabc5486fb7526 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f665dffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f665dffb700 1 -- 172.21.15.115:0/1789705903 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668028070 con 0x5650bc09e4a0 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f665dffb700 1 -- 172.21.15.115:0/1789705903 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6668024c20 con 0x5650bc09e4a0 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.541+0000 7f668b87db80 1 -- 172.21.15.115:0/1789705903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66680047e0 con 0x5650bc09e4a0 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f665dffb700 1 -- 172.21.15.115:0/1789705903 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f666803c040 con 0x5650bc09e4a0 2024-05-09T05:42:01.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f668b87db80 10 client.16638 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f668b87db80 1 -- 172.21.15.115:0/1789705903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5650bc29f0f0 con 0x5650bc09e4a0 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f665dffb700 1 -- 172.21.15.115:0/1789705903 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66680047e0 con 0x5650bc09e4a0 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f665dffb700 1 -- 172.21.15.115:0/1789705903 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f666803f070 con 0x5650bc09e4a0 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f668b87db80 10 client.16638 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f668b87db80 10 client.16638 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f668b87db80 20 client.16638 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f668b87db80 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5650bc29f720 0x5650bc2a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f668b87db80 4 client.16638 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.542+0000 7f668b87db80 1 -- 172.21.15.115:0/1789705903 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650bc20fd40 con 0x5650bc29f720 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f666ffff700 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5650bc29f720 0x5650bc2a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f665dffb700 10 client.16638 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f666ffff700 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5650bc29f720 0x5650bc2a1b40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27b47decd69d17e3 server_cookie=d09756653cf80ea5 in_seq=0 out_seq=0 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f665dffb700 1 -- 172.21.15.115:0/1789705903 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650bc20fd40 con 0x5650bc29f720 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f665dffb700 10 client.16638 handle_command_reply: tid=0 2024-05-09T05:42:01.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f668b87db80 1 client.16638 shutdown 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f668b87db80 1 -- 172.21.15.115:0/1789705903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 msgr2=0x5650bc23a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f668b87db80 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc23a660 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6668009950 tx=0x7f6668004580 comp rx=0 tx=0).stop 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f668b87db80 1 -- 172.21.15.115:0/1789705903 shutdown_connections 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.543+0000 7f668b87db80 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x5650bc29f720 0x5650bc2a1b40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f668b87db80 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5650bc21b400 0x5650bc237f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f668b87db80 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5650bc09e4a0 0x5650bc23a660 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f665dffb700 0 client.16638 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f668b87db80 1 --2- 172.21.15.115:0/1789705903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bc118d00 0x5650bc2379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f668b87db80 1 -- 172.21.15.115:0/1789705903 >> 172.21.15.115:0/1789705903 conn(0x5650bc0a1f70 msgr2=0x5650bc21ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f668b87db80 1 -- 172.21.15.115:0/1789705903 shutdown_connections 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f668b87db80 1 -- 172.21.15.115:0/1789705903 wait complete. 2024-05-09T05:42:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f668b87db80 20 client.16638 trim_cache size 0 max 16384 2024-05-09T05:42:01.542 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:42:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 -- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66640b29f0 msgr2=0x7f66640b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66640b29f0 0x7f66640b4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6678101bf0 tx=0x7f666000b410 comp rx=0 tx=0).stop 2024-05-09T05:42:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 -- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 msgr2=0x7f667819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 0x7f667819c660 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6670009850 tx=0x7f6670007d90 comp rx=0 tx=0).stop 2024-05-09T05:42:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 -- 172.21.15.115:0/1692128672 shutdown_connections 2024-05-09T05:42:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66640b29f0 0x7f66640b4f00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6678103b30 0x7f667819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66781087a0 0x7f667819c660 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 --2- 172.21.15.115:0/1692128672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6678107e50 0x7f667819c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.544+0000 7f667ce67700 1 -- 172.21.15.115:0/1692128672 >> 172.21.15.115:0/1692128672 conn(0x7f66780fe520 msgr2=0x7f6678105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.545+0000 7f667ce67700 1 -- 172.21.15.115:0/1692128672 shutdown_connections 2024-05-09T05:42:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:01.545+0000 7f667ce67700 1 -- 172.21.15.115:0/1692128672 wait complete. 2024-05-09T05:42:01.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:01 smithi195 ceph-mon[33844]: pgmap v117: 129 pgs: 129 active+clean; 1.7 GiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:42:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:01 smithi161 ceph-mon[33805]: pgmap v117: 129 pgs: 129 active+clean; 1.7 GiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:42:02.446 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:42:02.710 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:42:03.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:03 smithi195 ceph-mon[33844]: pgmap v118: 129 pgs: 129 active+clean; 1.7 GiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:42:03.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:03 smithi115 ceph-mon[30349]: pgmap v118: 129 pgs: 129 active+clean; 1.7 GiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:42:03.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:03 smithi161 ceph-mon[33805]: pgmap v118: 129 pgs: 129 active+clean; 1.7 GiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:42:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:05 smithi195 ceph-mon[33844]: pgmap v119: 129 pgs: 129 active+clean; 1.8 GiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 44 op/s 2024-05-09T05:42:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:05 smithi115 ceph-mon[30349]: pgmap v119: 129 pgs: 129 active+clean; 1.8 GiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 44 op/s 2024-05-09T05:42:05.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:05 smithi161 ceph-mon[33805]: pgmap v119: 129 pgs: 129 active+clean; 1.8 GiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 44 op/s 2024-05-09T05:42:07.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:07 smithi195 ceph-mon[33844]: pgmap v120: 129 pgs: 129 active+clean; 1.8 GiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 35 op/s 2024-05-09T05:42:07.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:07 smithi115 ceph-mon[30349]: pgmap v120: 129 pgs: 129 active+clean; 1.8 GiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 35 op/s 2024-05-09T05:42:07.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:07 smithi161 ceph-mon[33805]: pgmap v120: 129 pgs: 129 active+clean; 1.8 GiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 35 op/s 2024-05-09T05:42:09.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:09 smithi195 ceph-mon[33844]: pgmap v121: 129 pgs: 129 active+clean; 1.9 GiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 43 op/s 2024-05-09T05:42:09.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:09 smithi115 ceph-mon[30349]: pgmap v121: 129 pgs: 129 active+clean; 1.9 GiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 43 op/s 2024-05-09T05:42:09.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:09 smithi161 ceph-mon[33805]: pgmap v121: 129 pgs: 129 active+clean; 1.9 GiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 43 op/s 2024-05-09T05:42:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.540+0000 7f54b068b640 1 -- 172.21.15.115:0/1069117111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8108bb0 msgr2=0x7f54a8102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.540+0000 7f54b068b640 1 --2- 172.21.15.115:0/1069117111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8108bb0 0x7f54a8102750 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009900 tx=0x7f54a402a410 comp rx=0 tx=0).stop 2024-05-09T05:42:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.540+0000 7f54b068b640 1 -- 172.21.15.115:0/1069117111 shutdown_connections 2024-05-09T05:42:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.540+0000 7f54b068b640 1 --2- 172.21.15.115:0/1069117111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54a8103300 0x7f54a810f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.540+0000 7f54b068b640 1 --2- 172.21.15.115:0/1069117111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a8109050 0x7f54a8102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.540+0000 7f54b068b640 1 --2- 172.21.15.115:0/1069117111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8108bb0 0x7f54a8102750 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.540+0000 7f54b068b640 1 -- 172.21.15.115:0/1069117111 >> 172.21.15.115:0/1069117111 conn(0x7f54a8078280 msgr2=0x7f54a80786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.540+0000 7f54b068b640 1 -- 172.21.15.115:0/1069117111 shutdown_connections 2024-05-09T05:42:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.540+0000 7f54b068b640 1 -- 172.21.15.115:0/1069117111 wait complete. 2024-05-09T05:42:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54b068b640 1 Processor -- start 2024-05-09T05:42:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54b068b640 1 -- start start 2024-05-09T05:42:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54b068b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8103300 0x7f54a819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54b068b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a8108bb0 0x7f54a819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54b068b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54a8109050 0x7f54a819c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54b068b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8112c50 con 0x7f54a8103300 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54b068b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8112ad0 con 0x7f54a8108bb0 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54b068b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8112dd0 con 0x7f54a8109050 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54ae427640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8103300 0x7f54a819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54ae427640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8103300 0x7f54a819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44346/0 (socket says 172.21.15.115:44346) 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54ae427640 1 -- 172.21.15.115:0/2932974713 learned_addr learned my addr 172.21.15.115:0/2932974713 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54adc26640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a8108bb0 0x7f54a819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.541+0000 7f54aec28640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54a8109050 0x7f54a819c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f54ae427640 1 -- 172.21.15.115:0/2932974713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54a8109050 msgr2=0x7f54a819c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f54ae427640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54a8109050 0x7f54a819c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f54ae427640 1 -- 172.21.15.115:0/2932974713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a8108bb0 msgr2=0x7f54a819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f54ae427640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a8108bb0 0x7f54a819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f54ae427640 1 -- 172.21.15.115:0/2932974713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a81a6970 con 0x7f54a8103300 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f54ae427640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8103300 0x7f54a819b630 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005940 tx=0x7f54a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f549f7fe640 1 -- 172.21.15.115:0/2932974713 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a403f030 con 0x7f54a8103300 2024-05-09T05:42:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f54b068b640 1 -- 172.21.15.115:0/2932974713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a81a6c00 con 0x7f54a8103300 2024-05-09T05:42:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f549f7fe640 1 -- 172.21.15.115:0/2932974713 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a4003c90 con 0x7f54a8103300 2024-05-09T05:42:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f549f7fe640 1 -- 172.21.15.115:0/2932974713 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4035070 con 0x7f54a8103300 2024-05-09T05:42:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.542+0000 7f54b068b640 1 -- 172.21.15.115:0/2932974713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a81a70f0 con 0x7f54a8103300 2024-05-09T05:42:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.545+0000 7f54b7617740 1 Processor -- start 2024-05-09T05:42:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.545+0000 7f54b7617740 1 -- start start 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54b7617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa0496680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54b7617740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa0496bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54b7617740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa0497100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54b7617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cfa03fb7c0 con 0x55cfa00f31b0 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54b7617740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cfa03970e0 con 0x55cfa00ca2f0 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54b7617740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cfa03f7780 con 0x55cfa00d24d0 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f549f7fe640 1 -- 172.21.15.115:0/2932974713 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f54a4042030 con 0x7f54a8103300 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54aec28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa0496680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54aec28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa0496680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44348/0 (socket says 172.21.15.115:44348) 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54aec28640 1 -- 172.21.15.115:0/211607123 learned_addr learned my addr 172.21.15.115:0/211607123 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54adc26640 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa0497100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54aec28640 1 -- 172.21.15.115:0/211607123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 msgr2=0x55cfa0497100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54ae427640 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa0496bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54aec28640 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa0497100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54aec28640 1 -- 172.21.15.115:0/211607123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 msgr2=0x55cfa0496bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54aec28640 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa0496bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54aec28640 1 -- 172.21.15.115:0/211607123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa03c07c0 con 0x55cfa00f31b0 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54adc26640 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa0497100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54ae427640 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa0496bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.546+0000 7f54aec28640 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa0496680 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009fa0 tx=0x7f54a0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a3e81b2b28ec64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.547+0000 7f549f7fe640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54940b2950 0x7f54940b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.547+0000 7f549cbfe640 1 -- 172.21.15.115:0/211607123 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0016070 con 0x55cfa00f31b0 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.547+0000 7f549cbfe640 1 -- 172.21.15.115:0/211607123 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a001baf0 con 0x55cfa00f31b0 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.547+0000 7f549cbfe640 1 -- 172.21.15.115:0/211607123 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0012c00 con 0x55cfa00f31b0 2024-05-09T05:42:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.547+0000 7f54ae427640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54940b2950 0x7f54940b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.547+0000 7f549f7fe640 1 -- 172.21.15.115:0/2932974713 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f54a4102db0 con 0x7f54a8103300 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 -- 172.21.15.115:0/211607123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 msgr2=0x55cfa0496680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa0496680 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009fa0 tx=0x7f54a0010800 comp rx=0 tx=0).stop 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54ae427640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54940b2950 0x7f54940b4e60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54a4100270 tx=0x7f54a4009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 -- 172.21.15.115:0/211607123 shutdown_connections 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa0497100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa0496bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 --2- 172.21.15.115:0/211607123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa0496680 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 -- 172.21.15.115:0/211607123 >> 172.21.15.115:0/211607123 conn(0x55cfa00e7300 msgr2=0x55cfa0492e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 -- 172.21.15.115:0/211607123 shutdown_connections 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 -- 172.21.15.115:0/211607123 wait complete. 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 -- start start 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.548+0000 7f54b7617740 1 Processor -- start 2024-05-09T05:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54b7617740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54b7617740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa04a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54b7617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54b7617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cfa04a37b0 con 0x55cfa00f31b0 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54b7617740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cfa03829c0 con 0x55cfa00ca2f0 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54b7617740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cfa04a3930 con 0x55cfa00d24d0 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54b7617740 10 client.0 _ll_register_callbacks cb 0x55cfa03af020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44358/0 (socket says 172.21.15.115:44358) 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 -- 172.21.15.115:0/2284797556 learned_addr learned my addr 172.21.15.115:0/2284797556 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54adc26640 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa04a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 -- 172.21.15.115:0/2284797556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 msgr2=0x55cfa04a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa04a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 -- 172.21.15.115:0/2284797556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 msgr2=0x55cfa04a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 -- 172.21.15.115:0/2284797556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa03a91f0 con 0x55cfa00f31b0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f54aec28640 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a0c10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f54a001bf90 tx=0x7f54a0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d94c8c6afc03e85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.549+0000 7f5491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f5491ffb640 1 -- 172.21.15.115:0/2284797556 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0016070 con 0x55cfa00f31b0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f5491ffb640 1 -- 172.21.15.115:0/2284797556 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f54a001b670 con 0x55cfa00f31b0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f54b7617740 1 -- 172.21.15.115:0/2284797556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a001baf0 con 0x55cfa00f31b0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f5491ffb640 1 -- 172.21.15.115:0/2284797556 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f54a001b870 con 0x55cfa00f31b0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f54b7617740 10 client.16650 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f54b7617740 1 -- 172.21.15.115:0/2284797556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cfa015e790 con 0x55cfa00f31b0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f5491ffb640 1 -- 172.21.15.115:0/2284797556 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a001baf0 con 0x55cfa00f31b0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f5491ffb640 1 -- 172.21.15.115:0/2284797556 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f54a000f040 con 0x55cfa00f31b0 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f54b7617740 10 client.16650 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f54b7617740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f54b7617740 20 client.16650 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f54b7617740 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cfa00b1d80 0x55cfa04b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f54b7617740 4 client.16650 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.550+0000 7f54b7617740 1 -- 172.21.15.115:0/2284797556 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfa04093d0 con 0x55cfa00b1d80 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.551+0000 7f54adc26640 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cfa00b1d80 0x55cfa04b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.551+0000 7f5491ffb640 10 client.16650 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.551+0000 7f54adc26640 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cfa00b1d80 0x55cfa04b2df0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaadb87abba1c5c4 server_cookie=abb63f76652ab914 in_seq=0 out_seq=0 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f5491ffb640 1 -- 172.21.15.115:0/2284797556 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cfa04093d0 con 0x55cfa00b1d80 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f5491ffb640 10 client.16650 handle_command_reply: tid=0 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 client.16650 shutdown 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 -- 172.21.15.115:0/2284797556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 msgr2=0x55cfa04a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a0c10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f54a001bf90 tx=0x7f54a0009800 comp rx=0 tx=0).stop 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 -- 172.21.15.115:0/2284797556 shutdown_connections 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a0c10 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cfa00b1d80 0x55cfa04b2df0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa04a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 --2- 172.21.15.115:0/2284797556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 -- 172.21.15.115:0/2284797556 >> 172.21.15.115:0/2284797556 conn(0x55cfa00e7300 msgr2=0x55cfa0491700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f5491ffb640 0 client.16650 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 -- 172.21.15.115:0/2284797556 shutdown_connections 2024-05-09T05:42:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.552+0000 7f54b7617740 1 -- 172.21.15.115:0/2284797556 wait complete. 2024-05-09T05:42:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.553+0000 7f54b7617740 20 client.16650 trim_cache size 0 max 16384 2024-05-09T05:42:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.572+0000 7f54b7617740 1 Processor -- start 2024-05-09T05:42:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.572+0000 7f54b7617740 1 -- start start 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.572+0000 7f54b7617740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa049c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54b7617740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa0491e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54b7617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54b7617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81b1110 con 0x55cfa00f31b0 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54b7617740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81b2d40 con 0x55cfa00ca2f0 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54b7617740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81b17e0 con 0x55cfa00d24d0 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44370/0 (socket says 172.21.15.115:44370) 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 -- 172.21.15.115:0/4167393519 learned_addr learned my addr 172.21.15.115:0/4167393519 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 --2- 172.21.15.115:0/4167393519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa049c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54adc26640 1 --2- 172.21.15.115:0/4167393519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa0491e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 -- 172.21.15.115:0/4167393519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 msgr2=0x55cfa0491e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 --2- 172.21.15.115:0/4167393519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa0491e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 -- 172.21.15.115:0/4167393519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 msgr2=0x55cfa049c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 --2- 172.21.15.115:0/4167393519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa049c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 -- 172.21.15.115:0/4167393519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa01d55f0 con 0x55cfa00f31b0 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f54aec28640 1 --2- 172.21.15.115:0/4167393519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a8bc0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009a80 tx=0x7f54a0009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d43d6fc6ca01ca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f5493fff640 1 -- 172.21.15.115:0/4167393519 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0016040 con 0x55cfa00f31b0 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f5493fff640 1 -- 172.21.15.115:0/4167393519 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a001baa0 con 0x55cfa00f31b0 2024-05-09T05:42:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.573+0000 7f5493fff640 1 -- 172.21.15.115:0/4167393519 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0004850 con 0x55cfa00f31b0 2024-05-09T05:42:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.574+0000 7f54b7617740 1 -- 172.21.15.115:0/4167393519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 msgr2=0x55cfa04a8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.574+0000 7f54b7617740 1 --2- 172.21.15.115:0/4167393519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a8bc0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009a80 tx=0x7f54a0009600 comp rx=0 tx=0).stop 2024-05-09T05:42:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.574+0000 7f54b7617740 1 -- 172.21.15.115:0/4167393519 shutdown_connections 2024-05-09T05:42:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.574+0000 7f54b7617740 1 --2- 172.21.15.115:0/4167393519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00f31b0 0x55cfa04a8bc0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.574+0000 7f54b7617740 1 --2- 172.21.15.115:0/4167393519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa0491e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.574+0000 7f54b7617740 1 --2- 172.21.15.115:0/4167393519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00ca2f0 0x55cfa049c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.574+0000 7f54b7617740 1 -- 172.21.15.115:0/4167393519 >> 172.21.15.115:0/4167393519 conn(0x55cfa00e7300 msgr2=0x55cfa0491700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.574+0000 7f54b7617740 1 -- 172.21.15.115:0/4167393519 shutdown_connections 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.574+0000 7f54b7617740 1 -- 172.21.15.115:0/4167393519 wait complete. 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54b7617740 1 -- start start 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54b7617740 1 Processor -- start 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54b7617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54b7617740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa04a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54b7617740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00f31b0 0x55cfa04a6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54b7617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81b2610 con 0x55cfa00ca2f0 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54b7617740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81b1eb0 con 0x55cfa00f31b0 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54b7617740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81af0f0 con 0x55cfa00d24d0 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54b7617740 10 client.0 _ll_register_callbacks cb 0x7f54a0009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54aec28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54aec28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44376/0 (socket says 172.21.15.115:44376) 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54aec28640 1 -- 172.21.15.115:0/3430900802 learned_addr learned my addr 172.21.15.115:0/3430900802 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.575+0000 7f54adc26640 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa04a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54aec28640 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00f31b0 0x55cfa04a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54aec28640 1 -- 172.21.15.115:0/3430900802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 msgr2=0x55cfa04a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54aec28640 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa04a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54aec28640 1 -- 172.21.15.115:0/3430900802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00f31b0 msgr2=0x55cfa04a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54aec28640 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00f31b0 0x55cfa04a6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54aec28640 1 -- 172.21.15.115:0/3430900802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa03bfed0 con 0x55cfa00ca2f0 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54aec28640 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a59e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f54a0000f80 tx=0x7f54a0009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ed3315e1b71aa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f5491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f5491ffb640 1 -- 172.21.15.115:0/3430900802 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0016040 con 0x55cfa00ca2f0 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f5491ffb640 1 -- 172.21.15.115:0/3430900802 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f54a0004190 con 0x55cfa00ca2f0 2024-05-09T05:42:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54b7617740 1 -- 172.21.15.115:0/3430900802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a001baa0 con 0x55cfa00ca2f0 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f5491ffb640 1 -- 172.21.15.115:0/3430900802 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f54a0003970 con 0x55cfa00ca2f0 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54b7617740 10 client.16656 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f54b7617740 1 -- 172.21.15.115:0/3430900802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cfa051e600 con 0x55cfa00ca2f0 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.576+0000 7f5491ffb640 1 -- 172.21.15.115:0/3430900802 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54a001baa0 con 0x55cfa00ca2f0 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f5491ffb640 1 -- 172.21.15.115:0/3430900802 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f54a0005020 con 0x55cfa00ca2f0 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f54b7617740 10 client.16656 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f54b7617740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f54b7617740 20 client.16656 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f54b7617740 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cfa00871c0 0x55cfa0520b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f54b7617740 4 client.16656 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f54b7617740 1 -- 172.21.15.115:0/3430900802 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfa0403240 con 0x55cfa00871c0 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f54adc26640 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cfa00871c0 0x55cfa0520b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f54adc26640 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cfa00871c0 0x55cfa0520b30 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91415de56ff3f561 server_cookie=c5dad159aa64abcf in_seq=0 out_seq=0 2024-05-09T05:42:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.577+0000 7f5491ffb640 10 client.16656 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f5491ffb640 1 -- 172.21.15.115:0/3430900802 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cfa0403240 con 0x55cfa00871c0 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f5491ffb640 10 client.16656 handle_command_reply: tid=0 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f54b7617740 1 client.16656 shutdown 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f54b7617740 1 -- 172.21.15.115:0/3430900802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00ca2f0 msgr2=0x55cfa04a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f54b7617740 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a59e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f54a0000f80 tx=0x7f54a0009c50 comp rx=0 tx=0).stop 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f54b7617740 1 -- 172.21.15.115:0/3430900802 shutdown_connections 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f54b7617740 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cfa00f31b0 0x55cfa04a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f54b7617740 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cfa00871c0 0x55cfa0520b30 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f54b7617740 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfa00d24d0 0x55cfa04a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f54b7617740 1 --2- 172.21.15.115:0/3430900802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cfa00ca2f0 0x55cfa04a59e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.587+0000 7f54b7617740 1 -- 172.21.15.115:0/3430900802 >> 172.21.15.115:0/3430900802 conn(0x55cfa00e7300 msgr2=0x55cfa049a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f5491ffb640 0 client.16656 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b7617740 1 -- 172.21.15.115:0/3430900802 shutdown_connections 2024-05-09T05:42:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b7617740 1 -- 172.21.15.115:0/3430900802 wait complete. 2024-05-09T05:42:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b7617740 20 client.16656 trim_cache size 0 max 16384 2024-05-09T05:42:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:42:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:42:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:42:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 -- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54940b2950 msgr2=0x7f54940b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54940b2950 0x7f54940b4e60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54a4100270 tx=0x7f54a4009260 comp rx=0 tx=0).stop 2024-05-09T05:42:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 -- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8103300 msgr2=0x7f54a819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8103300 0x7f54a819b630 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005940 tx=0x7f54a40042f0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 -- 172.21.15.115:0/2932974713 shutdown_connections 2024-05-09T05:42:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54940b2950 0x7f54940b4e60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54a8109050 0x7f54a819c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a8108bb0 0x7f54a819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 --2- 172.21.15.115:0/2932974713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54a8103300 0x7f54a819b630 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 -- 172.21.15.115:0/2932974713 >> 172.21.15.115:0/2932974713 conn(0x7f54a8078280 msgr2=0x7f54a8104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.588+0000 7f54b068b640 1 -- 172.21.15.115:0/2932974713 shutdown_connections 2024-05-09T05:42:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:10.589+0000 7f54b068b640 1 -- 172.21.15.115:0/2932974713 wait complete. 2024-05-09T05:42:10.612 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-05-09T05:42:10.613 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:42:11.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:11 smithi195 ceph-mon[33844]: pgmap v122: 129 pgs: 129 active+clean; 1.9 GiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 32 op/s 2024-05-09T05:42:11.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:11 smithi115 ceph-mon[30349]: pgmap v122: 129 pgs: 129 active+clean; 1.9 GiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 32 op/s 2024-05-09T05:42:11.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:11 smithi161 ceph-mon[33805]: pgmap v122: 129 pgs: 129 active+clean; 1.9 GiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 32 op/s 2024-05-09T05:42:12.220 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:42:13.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.006+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3984912443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8108790 msgr2=0x7fd6c81100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:13.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.006+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3984912443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8108790 0x7fd6c81100f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000b0d0 tx=0x7fd6c000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.006+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3984912443 shutdown_connections 2024-05-09T05:42:13.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.006+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3984912443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8108790 0x7fd6c81100f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.006+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3984912443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c8103c40 0x7fd6c8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.006+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3984912443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c8107e40 0x7fd6c8108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.006+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3984912443 >> 172.21.15.115:0/3984912443 conn(0x7fd6c80fe6f0 msgr2=0x7fd6c8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:13.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.006+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3984912443 shutdown_connections 2024-05-09T05:42:13.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.006+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3984912443 wait complete. 2024-05-09T05:42:13.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.007+0000 7fd6cf21c700 1 Processor -- start 2024-05-09T05:42:13.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.007+0000 7fd6cf21c700 1 -- start start 2024-05-09T05:42:13.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.007+0000 7fd6cf21c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8103c40 0x7fd6c819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.007+0000 7fd6cf21c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c8107e40 0x7fd6c819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.007+0000 7fd6cf21c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c8108790 0x7fd6c819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6cf21c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c81132d0 con 0x7fd6c8103c40 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6cf21c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8113150 con 0x7fd6c8108790 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6cf21c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8113450 con 0x7fd6c8107e40 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6ccfb8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8103c40 0x7fd6c819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6ccfb8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8103c40 0x7fd6c819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48870/0 (socket says 172.21.15.115:48870) 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c8107e40 0x7fd6c819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/3125489142 learned_addr learned my addr 172.21.15.115:0/3125489142 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c8108790 0x7fd6c819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/3125489142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c8107e40 msgr2=0x7fd6c819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c8107e40 0x7fd6c819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/3125489142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c8108790 msgr2=0x7fd6c819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c8108790 0x7fd6c819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/3125489142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c819fbe0 con 0x7fd6c8103c40 2024-05-09T05:42:13.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c8108790 0x7fd6c819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:13.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8103c40 0x7fd6c819ad10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00e970 tx=0x7fd6bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:13.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.008+0000 7fd6c5ffb700 1 -- 172.21.15.115:0/3125489142 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc00cbd0 con 0x7fd6c8103c40 2024-05-09T05:42:13.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.009+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3125489142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c819fed0 con 0x7fd6c8103c40 2024-05-09T05:42:13.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.009+0000 7fd6c5ffb700 1 -- 172.21.15.115:0/3125489142 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6bc0044b0 con 0x7fd6c8103c40 2024-05-09T05:42:13.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.009+0000 7fd6c5ffb700 1 -- 172.21.15.115:0/3125489142 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc003c90 con 0x7fd6c8103c40 2024-05-09T05:42:13.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.009+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3125489142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c810a0c0 con 0x7fd6c8103c40 2024-05-09T05:42:13.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.013+0000 7fd6c5ffb700 1 -- 172.21.15.115:0/3125489142 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fd6bc018430 con 0x7fd6c8103c40 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.013+0000 7fd6ddc32b80 1 Processor -- start 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.013+0000 7fd6c5ffb700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6b80b2a70 0x7fd6b80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6ddc32b80 1 -- start start 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6c5ffb700 1 -- 172.21.15.115:0/3125489142 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd6bc0e2ca0 con 0x7fd6c8103c40 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7da4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6b80b2a70 0x7fd6b80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7daa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 0x5602ce7d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5602ce6f6ff0 con 0x5602ce6574a0 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5602ce6957f0 con 0x5602ce7d46a0 2024-05-09T05:42:13.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5602ce6f2fb0 con 0x5602ce741160 2024-05-09T05:42:13.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7daa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6c7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7daa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48872/0 (socket says 172.21.15.115:48872) 2024-05-09T05:42:13.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6c7fff700 1 -- 172.21.15.115:0/2603265566 learned_addr learned my addr 172.21.15.115:0/2603265566 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 0x5602ce7d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7da4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6c7fff700 1 -- 172.21.15.115:0/2603265566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 msgr2=0x5602ce7da4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6c7fff700 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7da4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6c7fff700 1 -- 172.21.15.115:0/2603265566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 msgr2=0x5602ce7d4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.014+0000 7fd6c7fff700 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 0x5602ce7d4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.015+0000 7fd6c7fff700 1 -- 172.21.15.115:0/2603265566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ce68d370 con 0x5602ce6574a0 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.015+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6b80b2a70 0x7fd6b80b4f80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000b0d0 tx=0x7fd6c0009360 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.015+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7da4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.015+0000 7fd6c7fff700 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7daa30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4010d50 tx=0x7fd6b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f105c3c39eeb50a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.015+0000 7fd6b2ffd700 1 -- 172.21.15.115:0/2603265566 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b401e020 con 0x5602ce6574a0 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.015+0000 7fd6b2ffd700 1 -- 172.21.15.115:0/2603265566 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6b40044e0 con 0x5602ce6574a0 2024-05-09T05:42:13.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.015+0000 7fd6b2ffd700 1 -- 172.21.15.115:0/2603265566 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4005280 con 0x5602ce6574a0 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.016+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/2603265566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 msgr2=0x5602ce7daa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.016+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7daa30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4010d50 tx=0x7fd6b4009260 comp rx=0 tx=0).stop 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.016+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/2603265566 shutdown_connections 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.016+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 0x5602ce7d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.016+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7da4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.016+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/2603265566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7daa30 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.016+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/2603265566 >> 172.21.15.115:0/2603265566 conn(0x5602ce6b76a0 msgr2=0x5602ce7cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.016+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/2603265566 shutdown_connections 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.016+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/2603265566 wait complete. 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.017+0000 7fd6ddc32b80 1 -- start start 2024-05-09T05:42:13.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.017+0000 7fd6ddc32b80 1 Processor -- start 2024-05-09T05:42:13.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.017+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce6574a0 0x5602ce7de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.017+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce741160 0x5602ce7de630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.017+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7deb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5602ce572ba0 con 0x5602ce7d46a0 2024-05-09T05:42:13.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5602ce6b1150 con 0x5602ce741160 2024-05-09T05:42:13.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5602ce7e11d0 con 0x5602ce6574a0 2024-05-09T05:42:13.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ccfb8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce6574a0 0x5602ce7de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ddc32b80 10 client.0 _ll_register_callbacks cb 0x5602ce5686c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ccfb8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce6574a0 0x5602ce7de0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:54252/0 (socket says 172.21.15.115:54252) 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/3054979541 learned_addr learned my addr 172.21.15.115:0/3054979541 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7deb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6c7fff700 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce741160 0x5602ce7de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/3054979541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce741160 msgr2=0x5602ce7de630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce741160 0x5602ce7de630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/3054979541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 msgr2=0x5602ce7deb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7deb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/3054979541 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ce572490 con 0x5602ce6574a0 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6c7fff700 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce741160 0x5602ce7de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce6574a0 0x5602ce7de0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0e06b0 tx=0x7fd6bc0a9460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=383d80963a94ebfe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:42:13.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.018+0000 7fd6a7fff700 1 -- 172.21.15.115:0/3054979541 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0b23e0 con 0x5602ce6574a0 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6a7fff700 1 -- 172.21.15.115:0/3054979541 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd6bc0b2820 con 0x5602ce6574a0 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/3054979541 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6b40044e0 con 0x5602ce6574a0 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6a7fff700 1 -- 172.21.15.115:0/3054979541 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd6bc0a9750 con 0x5602ce6574a0 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6ddc32b80 10 client.26206 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/3054979541 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5602ce7ebb60 con 0x5602ce6574a0 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6a7fff700 1 -- 172.21.15.115:0/3054979541 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6b40044e0 con 0x5602ce6574a0 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6a7fff700 1 -- 172.21.15.115:0/3054979541 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fd6bc0a55b0 con 0x5602ce6574a0 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6ddc32b80 10 client.26206 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6ddc32b80 10 client.26206 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6ddc32b80 20 client.26206 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5602ce7ee1a0 0x5602ce7f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6ddc32b80 4 client.26206 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/3054979541 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602ce7049d0 con 0x5602ce7ee1a0 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.019+0000 7fd6c7fff700 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5602ce7ee1a0 0x5602ce7f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.020+0000 7fd6c7fff700 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5602ce7ee1a0 0x5602ce7f05c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eee791dd8e9849b8 server_cookie=c68e3c292de9bb02 in_seq=0 out_seq=0 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.020+0000 7fd6a7fff700 10 client.26206 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6a7fff700 1 -- 172.21.15.115:0/3054979541 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5602ce7049d0 con 0x5602ce7ee1a0 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6a7fff700 10 client.26206 handle_command_reply: tid=0 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 client.26206 shutdown 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/3054979541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce6574a0 msgr2=0x5602ce7de0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce6574a0 0x5602ce7de0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0e06b0 tx=0x7fd6bc0a9460 comp rx=0 tx=0).stop 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/3054979541 shutdown_connections 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5602ce7ee1a0 0x5602ce7f05c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7deb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce6574a0 0x5602ce7de0f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/3054979541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce741160 0x5602ce7de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/3054979541 >> 172.21.15.115:0/3054979541 conn(0x5602ce6b76a0 msgr2=0x5602ce7cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6a7fff700 0 client.26206 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:42:13.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/3054979541 shutdown_connections 2024-05-09T05:42:13.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/3054979541 wait complete. 2024-05-09T05:42:13.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.021+0000 7fd6ddc32b80 20 client.26206 trim_cache size 0 max 16384 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ddc32b80 1 Processor -- start 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ddc32b80 1 -- start start 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7d8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 0x5602ce7de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c81ae150 con 0x5602ce6574a0 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c81afd80 con 0x5602ce7d46a0 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c81ae820 con 0x5602ce741160 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ccfb8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7d8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ccfb8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7d8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48888/0 (socket says 172.21.15.115:48888) 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.042+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/1810130692 learned_addr learned my addr 172.21.15.115:0/1810130692 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/1810130692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 0x5602ce7de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6c7fff700 1 --2- 172.21.15.115:0/1810130692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/1810130692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 msgr2=0x5602ce7ea0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/1810130692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7ea0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/1810130692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 msgr2=0x5602ce7de0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/1810130692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 0x5602ce7de0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6ccfb8700 1 -- 172.21.15.115:0/1810130692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ce7ddec0 con 0x5602ce6574a0 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/1810130692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7d8510 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0a9bd0 tx=0x7fd6bc005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893e5e9e9674cda3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6b27fc700 1 -- 172.21.15.115:0/1810130692 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0b23e0 con 0x5602ce6574a0 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6b27fc700 1 -- 172.21.15.115:0/1810130692 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6bc0aa340 con 0x5602ce6574a0 2024-05-09T05:42:13.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.043+0000 7fd6b27fc700 1 -- 172.21.15.115:0/1810130692 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0adff0 con 0x5602ce6574a0 2024-05-09T05:42:13.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.044+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/1810130692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 msgr2=0x5602ce7d8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:13.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.044+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/1810130692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7d8510 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0a9bd0 tx=0x7fd6bc005910 comp rx=0 tx=0).stop 2024-05-09T05:42:13.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.044+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/1810130692 shutdown_connections 2024-05-09T05:42:13.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.044+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/1810130692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce7d46a0 0x5602ce7de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.044+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/1810130692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce6574a0 0x5602ce7d8510 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.044+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/1810130692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7ea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.044+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/1810130692 >> 172.21.15.115:0/1810130692 conn(0x5602ce6b76a0 msgr2=0x5602ce7d7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.044+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/1810130692 shutdown_connections 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.044+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/1810130692 wait complete. 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6ddc32b80 1 -- start start 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6ddc32b80 1 Processor -- start 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce6574a0 0x5602ce7f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7e6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6ddc32b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c81af650 con 0x5602ce7d46a0 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c81aeef0 con 0x5602ce6574a0 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6ddc32b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c81ac0f0 con 0x5602ce741160 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6ddc32b80 10 client.0 _ll_register_callbacks cb 0x5602ce5686c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:13.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6cd7b9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6cd7b9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7e6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48900/0 (socket says 172.21.15.115:48900) 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.045+0000 7fd6cd7b9700 1 -- 172.21.15.115:0/499435576 learned_addr learned my addr 172.21.15.115:0/499435576 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6c7fff700 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce6574a0 0x5602ce7f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6cd7b9700 1 -- 172.21.15.115:0/499435576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 msgr2=0x5602ce7e6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7e6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6cd7b9700 1 -- 172.21.15.115:0/499435576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce6574a0 msgr2=0x5602ce7f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce6574a0 0x5602ce7f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6cd7b9700 1 -- 172.21.15.115:0/499435576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ce7f4e20 con 0x5602ce7d46a0 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6cd7b9700 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7e6960 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0004ea0 tx=0x7fd6c000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddd68770e75c9be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6a7fff700 1 -- 172.21.15.115:0/499435576 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0026070 con 0x5602ce7d46a0 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6a7fff700 1 -- 172.21.15.115:0/499435576 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd6c0007be0 con 0x5602ce7d46a0 2024-05-09T05:42:13.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.046+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/499435576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6bc0aa340 con 0x5602ce7d46a0 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6a7fff700 1 -- 172.21.15.115:0/499435576 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd6c003f050 con 0x5602ce7d46a0 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6ddc32b80 10 client.16674 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/499435576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5602ce857c70 con 0x5602ce7d46a0 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6a7fff700 1 -- 172.21.15.115:0/499435576 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6bc0aa340 con 0x5602ce7d46a0 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6a7fff700 1 -- 172.21.15.115:0/499435576 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fd6c0011070 con 0x5602ce7d46a0 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6ddc32b80 10 client.16674 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6ddc32b80 10 client.16674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6ddc32b80 20 client.16674 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5602ce858430 0x5602ce85a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6ddc32b80 4 client.16674 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/499435576 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602ce6fea70 con 0x5602ce858430 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.047+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5602ce858430 0x5602ce85a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6a7fff700 10 client.16674 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:42:13.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ccfb8700 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5602ce858430 0x5602ce85a850 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d4b129b423a07af server_cookie=f9e98393bbd89744 in_seq=0 out_seq=0 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6a7fff700 1 -- 172.21.15.115:0/499435576 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602ce6fea70 con 0x5602ce858430 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6a7fff700 10 client.16674 handle_command_reply: tid=0 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 client.16674 shutdown 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/499435576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 msgr2=0x5602ce7e6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7e6960 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0004ea0 tx=0x7fd6c000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/499435576 shutdown_connections 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5602ce6574a0 0x5602ce7f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x5602ce858430 0x5602ce85a850 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5602ce7d46a0 0x5602ce7e6960 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 --2- 172.21.15.115:0/499435576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602ce741160 0x5602ce7e6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6a7fff700 0 client.16674 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/499435576 >> 172.21.15.115:0/499435576 conn(0x5602ce6b76a0 msgr2=0x5602ce7dc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/499435576 shutdown_connections 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.048+0000 7fd6ddc32b80 1 -- 172.21.15.115:0/499435576 wait complete. 2024-05-09T05:42:13.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6ddc32b80 20 client.16674 trim_cache size 0 max 16384 2024-05-09T05:42:13.046 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6b80b2a70 msgr2=0x7fd6b80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6b80b2a70 0x7fd6b80b4f80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000b0d0 tx=0x7fd6c0009360 comp rx=0 tx=0).stop 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8103c40 msgr2=0x7fd6c819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8103c40 0x7fd6c819ad10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00e970 tx=0x7fd6bc00ec80 comp rx=0 tx=0).stop 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3125489142 shutdown_connections 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd6b80b2a70 0x7fd6b80b4f80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c8108790 0x7fd6c819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c8103c40 0x7fd6c819ad10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 --2- 172.21.15.115:0/3125489142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c8107e40 0x7fd6c819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3125489142 >> 172.21.15.115:0/3125489142 conn(0x7fd6c80fe6f0 msgr2=0x7fd6c8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3125489142 shutdown_connections 2024-05-09T05:42:13.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:13.049+0000 7fd6cf21c700 1 -- 172.21.15.115:0/3125489142 wait complete. 2024-05-09T05:42:13.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:13 smithi195 ceph-mon[33844]: pgmap v123: 129 pgs: 129 active+clean; 1.9 GiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 32 op/s 2024-05-09T05:42:13.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:13 smithi115 ceph-mon[30349]: pgmap v123: 129 pgs: 129 active+clean; 1.9 GiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 32 op/s 2024-05-09T05:42:13.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:13 smithi161 ceph-mon[33805]: pgmap v123: 129 pgs: 129 active+clean; 1.9 GiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 32 op/s 2024-05-09T05:42:14.873 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:42:15.443 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:15 smithi115 ceph-mon[30349]: pgmap v124: 129 pgs: 129 active+clean; 2.1 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 43 op/s 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.449+0000 7feaebab9700 1 -- 172.21.15.115:0/2444239685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4107e10 msgr2=0x7feae4108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.449+0000 7feaebab9700 1 --2- 172.21.15.115:0/2444239685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4107e10 0x7feae4108220 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fead40099c0 tx=0x7fead4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.449+0000 7feaebab9700 1 -- 172.21.15.115:0/2444239685 shutdown_connections 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.449+0000 7feaebab9700 1 --2- 172.21.15.115:0/2444239685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feae4108760 0x7feae4110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.449+0000 7feaebab9700 1 --2- 172.21.15.115:0/2444239685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae4103ca0 0x7feae41040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.449+0000 7feaebab9700 1 --2- 172.21.15.115:0/2444239685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4107e10 0x7feae4108220 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.449+0000 7feaebab9700 1 -- 172.21.15.115:0/2444239685 >> 172.21.15.115:0/2444239685 conn(0x7feae40fe6f0 msgr2=0x7feae4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.450+0000 7feaebab9700 1 -- 172.21.15.115:0/2444239685 shutdown_connections 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.450+0000 7feaebab9700 1 -- 172.21.15.115:0/2444239685 wait complete. 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.451+0000 7feaebab9700 1 Processor -- start 2024-05-09T05:42:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.451+0000 7feaebab9700 1 -- start start 2024-05-09T05:42:15.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feaebab9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4103ca0 0x7feae419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feae9855700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4103ca0 0x7feae419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feae9855700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4103ca0 0x7feae419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48920/0 (socket says 172.21.15.115:48920) 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feaebab9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae4107e10 0x7feae419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feae9855700 1 -- 172.21.15.115:0/895660172 learned_addr learned my addr 172.21.15.115:0/895660172 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feaebab9700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feae4108760 0x7feae4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feae4112190 con 0x7feae4103ca0 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feae4112010 con 0x7feae4108760 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feae4112310 con 0x7feae4107e10 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feae9054700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae4107e10 0x7feae419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feaea056700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feae4108760 0x7feae4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feae9054700 1 -- 172.21.15.115:0/895660172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feae4108760 msgr2=0x7feae4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feae9054700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feae4108760 0x7feae4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feae9054700 1 -- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4103ca0 msgr2=0x7feae419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.452+0000 7feae9054700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4103ca0 0x7feae419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.453+0000 7feae9054700 1 -- 172.21.15.115:0/895660172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae4199120 con 0x7feae4107e10 2024-05-09T05:42:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.453+0000 7feae9855700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4103ca0 0x7feae419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.453+0000 7feae9054700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae4107e10 0x7feae419ed90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fead40098e0 tx=0x7fead400dc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.453+0000 7feadaffd700 1 -- 172.21.15.115:0/895660172 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4023070 con 0x7feae4107e10 2024-05-09T05:42:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.453+0000 7feadaffd700 1 -- 172.21.15.115:0/895660172 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fead4004240 con 0x7feae4107e10 2024-05-09T05:42:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.453+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae41993b0 con 0x7feae4107e10 2024-05-09T05:42:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.454+0000 7feadaffd700 1 -- 172.21.15.115:0/895660172 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead401a070 con 0x7feae4107e10 2024-05-09T05:42:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.454+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae4199820 con 0x7feae4107e10 2024-05-09T05:42:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.455+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae40512c0 con 0x7feae4107e10 2024-05-09T05:42:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.456+0000 7feadaffd700 1 -- 172.21.15.115:0/895660172 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fead4026070 con 0x7feae4107e10 2024-05-09T05:42:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.457+0000 7feadaffd700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fead00b2a70 0x7fead00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.457+0000 7feae9855700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fead00b2a70 0x7fead00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.457+0000 7feadaffd700 1 -- 172.21.15.115:0/895660172 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fead40e2af0 con 0x7feae4107e10 2024-05-09T05:42:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.458+0000 7feae9855700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fead00b2a70 0x7fead00b4f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feae4199bb0 tx=0x7feae0009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:42:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.462+0000 7feadaffd700 1 -- 172.21.15.115:0/895660172 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fead400de00 con 0x7feae4107e10 2024-05-09T05:42:15.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:15 smithi195 ceph-mon[33844]: pgmap v124: 129 pgs: 129 active+clean; 2.1 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 43 op/s 2024-05-09T05:42:15.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:15 smithi161 ceph-mon[33805]: pgmap v124: 129 pgs: 129 active+clean; 2.1 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 43 op/s 2024-05-09T05:42:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.761+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feae402e5f0 con 0x7feae4107e10 2024-05-09T05:42:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.762+0000 7feadaffd700 1 -- 172.21.15.115:0/895660172 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11961 (secure 0 0 0) 0x7fead40aa220 con 0x7feae4107e10 2024-05-09T05:42:15.755 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:42:15.755 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:41:33.359460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:42:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fead00b2a70 msgr2=0x7fead00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fead00b2a70 0x7fead00b4f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feae4199bb0 tx=0x7feae0009260 comp rx=0 tx=0).stop 2024-05-09T05:42:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae4107e10 msgr2=0x7feae419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae4107e10 0x7feae419ed90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fead40098e0 tx=0x7fead400dc50 comp rx=0 tx=0).stop 2024-05-09T05:42:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 shutdown_connections 2024-05-09T05:42:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fead00b2a70 0x7fead00b4f80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feae4108760 0x7feae4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae4103ca0 0x7feae419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 --2- 172.21.15.115:0/895660172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feae4107e10 0x7feae419ed90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 >> 172.21.15.115:0/895660172 conn(0x7feae40fe6f0 msgr2=0x7feae4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:15.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 shutdown_connections 2024-05-09T05:42:15.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:15.765+0000 7feaebab9700 1 -- 172.21.15.115:0/895660172 wait complete. 2024-05-09T05:42:15.762 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 34 2024-05-09T05:42:16.603 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:16 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/895660172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:42:16.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:16 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/895660172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:42:16.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:16 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/895660172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:42:17.051 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-05-09T05:42:17.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-05-09T05:42:17.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:17 smithi195 ceph-mon[33844]: pgmap v125: 129 pgs: 129 active+clean; 2.1 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 19 op/s 2024-05-09T05:42:17.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:17 smithi115 ceph-mon[30349]: pgmap v125: 129 pgs: 129 active+clean; 2.1 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 19 op/s 2024-05-09T05:42:17.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:17 smithi161 ceph-mon[33805]: pgmap v125: 129 pgs: 129 active+clean; 2.1 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 19 op/s 2024-05-09T05:42:17.757 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:42:19.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.537+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3696555708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbad01034c0 msgr2=0x7fbad01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:19.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.537+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3696555708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbad01034c0 0x7fbad01038d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009920 tx=0x7fbacc009c30 comp rx=0 tx=0).stop 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.537+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3696555708 shutdown_connections 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.537+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3696555708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0108740 0x7fbad01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.537+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3696555708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbad01034c0 0x7fbad01038d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.537+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3696555708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad0107df0 0x7fbad0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.537+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3696555708 >> 172.21.15.115:0/3696555708 conn(0x7fbad00fedc0 msgr2=0x7fbad01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.537+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3696555708 shutdown_connections 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.538+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3696555708 wait complete. 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.538+0000 7fbad7bd3700 1 Processor -- start 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.538+0000 7fbad7bd3700 1 -- start start 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad7bd3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad01034c0 0x7fbad019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad7bd3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbad0107df0 0x7fbad019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad7bd3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0108740 0x7fbad019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad7bd3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01143c0 con 0x7fbad0108740 2024-05-09T05:42:19.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad7bd3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0114240 con 0x7fbad01034c0 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad7bd3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0114540 con 0x7fbad0107df0 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad6170700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0108740 0x7fbad019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad6170700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0108740 0x7fbad019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48934/0 (socket says 172.21.15.115:48934) 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad6170700 1 -- 172.21.15.115:0/3100044312 learned_addr learned my addr 172.21.15.115:0/3100044312 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad596f700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad01034c0 0x7fbad019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad516e700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbad0107df0 0x7fbad019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad6170700 1 -- 172.21.15.115:0/3100044312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbad0107df0 msgr2=0x7fbad019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad6170700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbad0107df0 0x7fbad019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad6170700 1 -- 172.21.15.115:0/3100044312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad01034c0 msgr2=0x7fbad019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad6170700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad01034c0 0x7fbad019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad6170700 1 -- 172.21.15.115:0/3100044312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad019fd30 con 0x7fbad0108740 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.539+0000 7fbad6170700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0108740 0x7fbad019f690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbac800e3c0 tx=0x7fbac800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.540+0000 7fbac6ffd700 1 -- 172.21.15.115:0/3100044312 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac80090d0 con 0x7fbad0108740 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.540+0000 7fbac6ffd700 1 -- 172.21.15.115:0/3100044312 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbac8009270 con 0x7fbad0108740 2024-05-09T05:42:19.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.540+0000 7fbac6ffd700 1 -- 172.21.15.115:0/3100044312 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8007d30 con 0x7fbad0108740 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.540+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3100044312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad01a0020 con 0x7fbad0108740 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.540+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3100044312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad01a7940 con 0x7fbad0108740 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.543+0000 7fbac6ffd700 1 -- 172.21.15.115:0/3100044312 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fbac8012050 con 0x7fbad0108740 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.544+0000 7fbac6ffd700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbabc0b2a70 0x7fbabc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.544+0000 7fbad596f700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbabc0b2a70 0x7fbabc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.544+0000 7fbac6ffd700 1 -- 172.21.15.115:0/3100044312 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbac80e2470 con 0x7fbad0108740 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.544+0000 7fbad596f700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbabc0b2a70 0x7fbabc0b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0098f0 tx=0x7fbacc019040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbae65e9b80 1 Processor -- start 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbae65e9b80 1 -- start start 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 0x55a7d899b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8995400 0x55a7d8995810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbae65e9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a7d88b80a0 con 0x55a7d8995400 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbae65e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7d885d9a0 con 0x55a7d88184a0 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbae65e9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7d88b4060 con 0x55a7d8892d00 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbad596f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbad596f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:54320/0 (socket says 172.21.15.115:54320) 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbad596f700 1 -- 172.21.15.115:0/929907066 learned_addr learned my addr 172.21.15.115:0/929907066 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.546+0000 7fbad6170700 1 --2- 172.21.15.115:0/929907066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8995400 0x55a7d8995810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.547+0000 7fbad516e700 1 --2- 172.21.15.115:0/929907066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 0x55a7d899b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.547+0000 7fbad596f700 1 -- 172.21.15.115:0/929907066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 msgr2=0x55a7d899b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.547+0000 7fbad596f700 1 --2- 172.21.15.115:0/929907066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 0x55a7d899b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.547+0000 7fbad596f700 1 -- 172.21.15.115:0/929907066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8995400 msgr2=0x55a7d8995810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.547+0000 7fbad596f700 1 --2- 172.21.15.115:0/929907066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8995400 0x55a7d8995810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.547+0000 7fbad596f700 1 -- 172.21.15.115:0/929907066 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7d884e500 con 0x55a7d8892d00 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.547+0000 7fbad596f700 1 --2- 172.21.15.115:0/929907066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899b240 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbacc016000 tx=0x7fbacc01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad3c133fd16d592 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.547+0000 7fbabbfff700 1 -- 172.21.15.115:0/929907066 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00f5a0 con 0x55a7d8892d00 2024-05-09T05:42:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.547+0000 7fbabbfff700 1 -- 172.21.15.115:0/929907066 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbacc00f740 con 0x55a7d8892d00 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.548+0000 7fbabbfff700 1 -- 172.21.15.115:0/929907066 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc003b20 con 0x55a7d8892d00 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.548+0000 7fbae65e9b80 1 -- 172.21.15.115:0/929907066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 msgr2=0x55a7d899b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.548+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/929907066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899b240 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbacc016000 tx=0x7fbacc01e000 comp rx=0 tx=0).stop 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.549+0000 7fbae65e9b80 1 -- 172.21.15.115:0/929907066 shutdown_connections 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.549+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/929907066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8995400 0x55a7d8995810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.549+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/929907066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899b240 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.549+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/929907066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 0x55a7d899b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.549+0000 7fbae65e9b80 1 -- 172.21.15.115:0/929907066 >> 172.21.15.115:0/929907066 conn(0x55a7d881bf70 msgr2=0x55a7d8990220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.549+0000 7fbae65e9b80 1 -- 172.21.15.115:0/929907066 shutdown_connections 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.549+0000 7fbae65e9b80 1 -- 172.21.15.115:0/929907066 wait complete. 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.549+0000 7fbae65e9b80 1 -- start start 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.549+0000 7fbae65e9b80 1 Processor -- start 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d899f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8995400 0x55a7d899fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad596f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d899f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbae65e9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a7d89960f0 con 0x55a7d88184a0 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad596f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d899f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48948/0 (socket says 172.21.15.115:48948) 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad596f700 1 -- 172.21.15.115:0/1418015255 learned_addr learned my addr 172.21.15.115:0/1418015255 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7d8852b20 con 0x55a7d8995400 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7d8996270 con 0x55a7d8892d00 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad6170700 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8995400 0x55a7d899fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbae65e9b80 10 client.0 _ll_register_callbacks cb 0x55a7d87296c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad516e700 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad596f700 1 -- 172.21.15.115:0/1418015255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 msgr2=0x55a7d899f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad596f700 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad596f700 1 -- 172.21.15.115:0/1418015255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8995400 msgr2=0x55a7d899fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad596f700 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8995400 0x55a7d899fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad596f700 1 -- 172.21.15.115:0/1418015255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7d8733490 con 0x55a7d88184a0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.550+0000 7fbad6170700 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8995400 0x55a7d899fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.551+0000 7fbad596f700 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d899f080 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0052a0 tx=0x7fbacc005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b8ad601c1e0dfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.551+0000 7fbab97fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.551+0000 7fbab97fa700 1 -- 172.21.15.115:0/1418015255 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00f8f0 con 0x55a7d88184a0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.551+0000 7fbab97fa700 1 -- 172.21.15.115:0/1418015255 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbacc0365d0 con 0x55a7d88184a0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.551+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbacc00f740 con 0x55a7d88184a0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.551+0000 7fbab97fa700 1 -- 172.21.15.115:0/1418015255 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbacc0367d0 con 0x55a7d88184a0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.551+0000 7fbae65e9b80 10 client.16692 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.551+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a7d89ac770 con 0x55a7d88184a0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.551+0000 7fbab97fa700 1 -- 172.21.15.115:0/1418015255 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbacc00f740 con 0x55a7d88184a0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.552+0000 7fbab97fa700 1 -- 172.21.15.115:0/1418015255 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fbacc00bbb0 con 0x55a7d88184a0 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.552+0000 7fbae65e9b80 10 client.16692 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.552+0000 7fbae65e9b80 10 client.16692 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:42:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.552+0000 7fbae65e9b80 20 client.16692 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.552+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55a7d89aec40 0x55a7d89b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.552+0000 7fbae65e9b80 4 client.16692 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.552+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7d88c5a80 con 0x55a7d89aec40 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.552+0000 7fbad516e700 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55a7d89aec40 0x55a7d89b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.552+0000 7fbab97fa700 10 client.16692 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.553+0000 7fbad516e700 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55a7d89aec40 0x55a7d89b1060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=530affb0a206964c server_cookie=7de19436d20ec644 in_seq=0 out_seq=0 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.553+0000 7fbab97fa700 1 -- 172.21.15.115:0/1418015255 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7d88c5a80 con 0x55a7d89aec40 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.553+0000 7fbab97fa700 10 client.16692 handle_command_reply: tid=0 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.553+0000 7fbae65e9b80 1 client.16692 shutdown 2024-05-09T05:42:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 msgr2=0x55a7d899f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d899f080 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0052a0 tx=0x7fbacc005f00 comp rx=0 tx=0).stop 2024-05-09T05:42:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 shutdown_connections 2024-05-09T05:42:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55a7d89aec40 0x55a7d89b1060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8995400 0x55a7d899fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d899f080 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1418015255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8892d00 0x55a7d899f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbab97fa700 0 client.16692 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:42:19.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 >> 172.21.15.115:0/1418015255 conn(0x55a7d881bf70 msgr2=0x55a7d8989ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:19.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 shutdown_connections 2024-05-09T05:42:19.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1418015255 wait complete. 2024-05-09T05:42:19.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.554+0000 7fbae65e9b80 20 client.16692 trim_cache size 0 max 16384 2024-05-09T05:42:19.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbae65e9b80 1 Processor -- start 2024-05-09T05:42:19.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbae65e9b80 1 -- start start 2024-05-09T05:42:19.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d898aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8892d00 0x55a7d899dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 0x55a7d89ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbae65e9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01ae280 con 0x55a7d88184a0 2024-05-09T05:42:19.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbae65e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01afeb0 con 0x55a7d8892d00 2024-05-09T05:42:19.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbae65e9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01ae950 con 0x55a7d8995400 2024-05-09T05:42:19.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad596f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d898aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad596f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d898aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48952/0 (socket says 172.21.15.115:48952) 2024-05-09T05:42:19.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad596f700 1 -- 172.21.15.115:0/2883018171 learned_addr learned my addr 172.21.15.115:0/2883018171 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:19.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad6170700 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 0x55a7d89ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad516e700 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8892d00 0x55a7d899dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad596f700 1 -- 172.21.15.115:0/2883018171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 msgr2=0x55a7d89ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:19.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad596f700 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 0x55a7d89ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad596f700 1 -- 172.21.15.115:0/2883018171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8892d00 msgr2=0x55a7d899dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:19.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad596f700 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8892d00 0x55a7d899dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad596f700 1 -- 172.21.15.115:0/2883018171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7d899ee50 con 0x55a7d88184a0 2024-05-09T05:42:19.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.575+0000 7fbad516e700 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8892d00 0x55a7d899dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:19.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.576+0000 7fbad596f700 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d898aa20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005950 tx=0x7fbacc005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244b74721a66a77b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.576+0000 7fbabb7fe700 1 -- 172.21.15.115:0/2883018171 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0365d0 con 0x55a7d88184a0 2024-05-09T05:42:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.576+0000 7fbabb7fe700 1 -- 172.21.15.115:0/2883018171 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbacc03a070 con 0x55a7d88184a0 2024-05-09T05:42:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.576+0000 7fbabb7fe700 1 -- 172.21.15.115:0/2883018171 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc036d70 con 0x55a7d88184a0 2024-05-09T05:42:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.577+0000 7fbae65e9b80 1 -- 172.21.15.115:0/2883018171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 msgr2=0x55a7d898aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.577+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d898aa20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005950 tx=0x7fbacc005c20 comp rx=0 tx=0).stop 2024-05-09T05:42:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.577+0000 7fbae65e9b80 1 -- 172.21.15.115:0/2883018171 shutdown_connections 2024-05-09T05:42:19.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.577+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 0x55a7d89ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.577+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d88184a0 0x55a7d898aa20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.577+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/2883018171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d8892d00 0x55a7d899dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.577+0000 7fbae65e9b80 1 -- 172.21.15.115:0/2883018171 >> 172.21.15.115:0/2883018171 conn(0x55a7d881bf70 msgr2=0x55a7d899f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:19.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.577+0000 7fbae65e9b80 1 -- 172.21.15.115:0/2883018171 shutdown_connections 2024-05-09T05:42:19.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.577+0000 7fbae65e9b80 1 -- 172.21.15.115:0/2883018171 wait complete. 2024-05-09T05:42:19.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.578+0000 7fbae65e9b80 1 -- start start 2024-05-09T05:42:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.578+0000 7fbae65e9b80 1 Processor -- start 2024-05-09T05:42:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.578+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 0x55a7d8a104e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.578+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8892d00 0x55a7d8a10a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.578+0000 7fbad516e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8892d00 0x55a7d8a10a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.578+0000 7fbad516e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8892d00 0x55a7d8a10a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48954/0 (socket says 172.21.15.115:48954) 2024-05-09T05:42:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.578+0000 7fbae65e9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 0x55a7d89b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.578+0000 7fbad516e700 1 -- 172.21.15.115:0/1938974263 learned_addr learned my addr 172.21.15.115:0/1938974263 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01af780 con 0x55a7d8892d00 2024-05-09T05:42:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01af020 con 0x55a7d88184a0 2024-05-09T05:42:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01ac240 con 0x55a7d8995400 2024-05-09T05:42:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad596f700 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 0x55a7d8a104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbae65e9b80 10 client.0 _ll_register_callbacks cb 0x55a7d87296c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad6170700 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 0x55a7d89b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad516e700 1 -- 172.21.15.115:0/1938974263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 msgr2=0x55a7d89b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad516e700 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 0x55a7d89b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad516e700 1 -- 172.21.15.115:0/1938974263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 msgr2=0x55a7d8a104e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad516e700 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 0x55a7d8a104e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad516e700 1 -- 172.21.15.115:0/1938974263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7d89b1ba0 con 0x55a7d8892d00 2024-05-09T05:42:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad596f700 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 0x55a7d8a104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad6170700 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 0x55a7d89b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbad516e700 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8892d00 0x55a7d8a10a20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbac0005950 tx=0x7fbac000bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b39ed4e634460d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbab97fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbab97fa700 1 -- 172.21.15.115:0/1938974263 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac001d420 con 0x55a7d8892d00 2024-05-09T05:42:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbab97fa700 1 -- 172.21.15.115:0/1938974263 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbac001d5c0 con 0x55a7d8892d00 2024-05-09T05:42:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.579+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbacc03a070 con 0x55a7d8892d00 2024-05-09T05:42:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbab97fa700 1 -- 172.21.15.115:0/1938974263 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbac002b050 con 0x55a7d8892d00 2024-05-09T05:42:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbae65e9b80 10 client.16698 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a7d8a18b10 con 0x55a7d8892d00 2024-05-09T05:42:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbab97fa700 1 -- 172.21.15.115:0/1938974263 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbac0022070 con 0x55a7d8892d00 2024-05-09T05:42:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbab97fa700 1 -- 172.21.15.115:0/1938974263 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fbac0011c50 con 0x55a7d8892d00 2024-05-09T05:42:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbae65e9b80 10 client.16698 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbae65e9b80 10 client.16698 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:42:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbae65e9b80 20 client.16698 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55a7d8a192e0 0x55a7d8a1b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbae65e9b80 4 client.16698 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:42:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.580+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbacc03a070 con 0x55a7d8a192e0 2024-05-09T05:42:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.581+0000 7fbad596f700 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55a7d8a192e0 0x55a7d8a1b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.581+0000 7fbab97fa700 10 client.16698 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:42:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.581+0000 7fbad596f700 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55a7d8a192e0 0x55a7d8a1b700 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fd4a5b1a44374c3 server_cookie=e80598606e9b2d87 in_seq=0 out_seq=0 2024-05-09T05:42:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbab97fa700 1 -- 172.21.15.115:0/1938974263 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbacc03a070 con 0x55a7d8a192e0 2024-05-09T05:42:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbab97fa700 10 client.16698 handle_command_reply: tid=0 2024-05-09T05:42:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 client.16698 shutdown 2024-05-09T05:42:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8892d00 msgr2=0x55a7d8a10a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8892d00 0x55a7d8a10a20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbac0005950 tx=0x7fbac000bdb0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 shutdown_connections 2024-05-09T05:42:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55a7d8a192e0 0x55a7d8a1b700 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7d88184a0 0x55a7d8a104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7d8995400 0x55a7d89b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbab97fa700 0 client.16698 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:42:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 --2- 172.21.15.115:0/1938974263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7d8892d00 0x55a7d8a10a20 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 >> 172.21.15.115:0/1938974263 conn(0x55a7d881bf70 msgr2=0x55a7d898a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 shutdown_connections 2024-05-09T05:42:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 1 -- 172.21.15.115:0/1938974263 wait complete. 2024-05-09T05:42:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.582+0000 7fbae65e9b80 20 client.16698 trim_cache size 0 max 16384 2024-05-09T05:42:19.056 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:42:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbabc0b2a70 msgr2=0x7fbabc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbabc0b2a70 0x7fbabc0b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0098f0 tx=0x7fbacc019040 comp rx=0 tx=0).stop 2024-05-09T05:42:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0108740 msgr2=0x7fbad019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0108740 0x7fbad019f690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbac800e3c0 tx=0x7fbac800e6d0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3100044312 shutdown_connections 2024-05-09T05:42:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbabc0b2a70 0x7fbabc0b4f80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad01034c0 0x7fbad019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0108740 0x7fbad019f690 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 --2- 172.21.15.115:0/3100044312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbad0107df0 0x7fbad019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3100044312 >> 172.21.15.115:0/3100044312 conn(0x7fbad00fedc0 msgr2=0x7fbad01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3100044312 shutdown_connections 2024-05-09T05:42:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:18.583+0000 7fbad7bd3700 1 -- 172.21.15.115:0/3100044312 wait complete. 2024-05-09T05:42:19.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:19 smithi195 ceph-mon[33844]: pgmap v126: 129 pgs: 129 active+clean; 2.2 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-05-09T05:42:19.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:19 smithi115 ceph-mon[30349]: pgmap v126: 129 pgs: 129 active+clean; 2.2 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-05-09T05:42:19.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:19 smithi161 ceph-mon[33805]: pgmap v126: 129 pgs: 129 active+clean; 2.2 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-05-09T05:42:20.384 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:42:20.988 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:42:20.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.006+0000 7efc4284a700 1 -- 172.21.15.115:0/11312074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c108740 msgr2=0x7efc3c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:20.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.006+0000 7efc4284a700 1 --2- 172.21.15.115:0/11312074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c108740 0x7efc3c1100a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efc3400b0d0 tx=0x7efc3400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:42:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.006+0000 7efc4284a700 1 -- 172.21.15.115:0/11312074 shutdown_connections 2024-05-09T05:42:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.006+0000 7efc4284a700 1 --2- 172.21.15.115:0/11312074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c1034c0 0x7efc3c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.006+0000 7efc4284a700 1 --2- 172.21.15.115:0/11312074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c108740 0x7efc3c1100a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.006+0000 7efc4284a700 1 --2- 172.21.15.115:0/11312074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c107df0 0x7efc3c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.006+0000 7efc4284a700 1 -- 172.21.15.115:0/11312074 >> 172.21.15.115:0/11312074 conn(0x7efc3c0fedc0 msgr2=0x7efc3c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.006+0000 7efc4284a700 1 -- 172.21.15.115:0/11312074 shutdown_connections 2024-05-09T05:42:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.006+0000 7efc4284a700 1 -- 172.21.15.115:0/11312074 wait complete. 2024-05-09T05:42:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc4284a700 1 Processor -- start 2024-05-09T05:42:21.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc4284a700 1 -- start start 2024-05-09T05:42:21.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc4284a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c1034c0 0x7efc3c19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc4284a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c107df0 0x7efc3c19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc4284a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c108740 0x7efc3c19c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc4284a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c113240 con 0x7efc3c1034c0 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc4284a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c1130c0 con 0x7efc3c108740 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc4284a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c1133c0 con 0x7efc3c107df0 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c1034c0 0x7efc3c19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c1034c0 0x7efc3c19bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48978/0 (socket says 172.21.15.115:48978) 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.007+0000 7efc3bfff700 1 -- 172.21.15.115:0/1162728070 learned_addr learned my addr 172.21.15.115:0/1162728070 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc3b7fe700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c107df0 0x7efc3c19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc40de7700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c108740 0x7efc3c19c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc3bfff700 1 -- 172.21.15.115:0/1162728070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c107df0 msgr2=0x7efc3c19c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc3bfff700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c107df0 0x7efc3c19c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc3bfff700 1 -- 172.21.15.115:0/1162728070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c108740 msgr2=0x7efc3c19c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc3bfff700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c108740 0x7efc3c19c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc3bfff700 1 -- 172.21.15.115:0/1162728070 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c199930 con 0x7efc3c1034c0 2024-05-09T05:42:21.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc3bfff700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c1034c0 0x7efc3c19bb40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00d950 tx=0x7efc2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:21.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc397fa700 1 -- 172.21.15.115:0/1162728070 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c016070 con 0x7efc3c1034c0 2024-05-09T05:42:21.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.008+0000 7efc4284a700 1 -- 172.21.15.115:0/1162728070 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c199c20 con 0x7efc3c1034c0 2024-05-09T05:42:21.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.009+0000 7efc4284a700 1 -- 172.21.15.115:0/1162728070 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c19a1b0 con 0x7efc3c1034c0 2024-05-09T05:42:21.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.009+0000 7efc397fa700 1 -- 172.21.15.115:0/1162728070 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc2c00fd20 con 0x7efc3c1034c0 2024-05-09T05:42:21.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.009+0000 7efc397fa700 1 -- 172.21.15.115:0/1162728070 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c020060 con 0x7efc3c1034c0 2024-05-09T05:42:21.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.012+0000 7efc397fa700 1 -- 172.21.15.115:0/1162728070 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7efc2c019070 con 0x7efc3c1034c0 2024-05-09T05:42:21.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.013+0000 7efc397fa700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efc240b29f0 0x7efc240b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.013+0000 7efc51260b80 1 Processor -- start 2024-05-09T05:42:21.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.013+0000 7efc51260b80 1 -- start start 2024-05-09T05:42:21.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.013+0000 7efc397fa700 1 -- 172.21.15.115:0/1162728070 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efc2c0e2430 con 0x7efc3c1034c0 2024-05-09T05:42:21.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.013+0000 7efc3b7fe700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efc240b29f0 0x7efc240b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc51260b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 0x556e084bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc51260b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084c0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc51260b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc40de7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc40de7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48990/0 (socket says 172.21.15.115:48990) 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc51260b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556e083e3230 con 0x556e084ca990 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc51260b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556e08388b30 con 0x556e08343630 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc51260b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e083df1f0 con 0x556e083bde90 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3b7fe700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efc240b29f0 0x7efc240b4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc3c102440 tx=0x7efc30009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 0x556e084bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084c0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:34602/0 (socket says 172.21.15.115:34602) 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3b7fe700 1 -- 172.21.15.115:0/2602720319 learned_addr learned my addr 172.21.15.115:0/2602720319 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:21.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3bfff700 1 -- 172.21.15.115:0/2602720319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 msgr2=0x556e084c0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:21.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3bfff700 1 --2- 172.21.15.115:0/2602720319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084c0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3bfff700 1 -- 172.21.15.115:0/2602720319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 msgr2=0x556e084ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:21.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3bfff700 1 --2- 172.21.15.115:0/2602720319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3bfff700 1 -- 172.21.15.115:0/2602720319 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e08379690 con 0x556e083bde90 2024-05-09T05:42:21.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.014+0000 7efc3b7fe700 1 --2- 172.21.15.115:0/2602720319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084c0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:21.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.015+0000 7efc40de7700 1 --2- 172.21.15.115:0/2602720319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:21.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.015+0000 7efc3bfff700 1 --2- 172.21.15.115:0/2602720319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 0x556e084bfce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0df7c0 tx=0x7efc2c0a8bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8952856b6c22ebfa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.015+0000 7efc227fc700 1 -- 172.21.15.115:0/2602720319 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0ad700 con 0x556e083bde90 2024-05-09T05:42:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.015+0000 7efc227fc700 1 -- 172.21.15.115:0/2602720319 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc2c0b21b0 con 0x556e083bde90 2024-05-09T05:42:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.015+0000 7efc227fc700 1 -- 172.21.15.115:0/2602720319 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0a85b0 con 0x556e083bde90 2024-05-09T05:42:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.016+0000 7efc51260b80 1 -- 172.21.15.115:0/2602720319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 msgr2=0x556e084bfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.016+0000 7efc51260b80 1 --2- 172.21.15.115:0/2602720319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 0x556e084bfce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0df7c0 tx=0x7efc2c0a8bf0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.016+0000 7efc51260b80 1 -- 172.21.15.115:0/2602720319 shutdown_connections 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.016+0000 7efc51260b80 1 --2- 172.21.15.115:0/2602720319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.016+0000 7efc51260b80 1 --2- 172.21.15.115:0/2602720319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 0x556e084bfce0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.016+0000 7efc51260b80 1 --2- 172.21.15.115:0/2602720319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084c0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.016+0000 7efc51260b80 1 -- 172.21.15.115:0/2602720319 >> 172.21.15.115:0/2602720319 conn(0x556e08347100 msgr2=0x556e084bbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.016+0000 7efc51260b80 1 -- 172.21.15.115:0/2602720319 shutdown_connections 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.016+0000 7efc51260b80 1 -- 172.21.15.115:0/2602720319 wait complete. 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.017+0000 7efc51260b80 1 -- start start 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.017+0000 7efc51260b80 1 Processor -- start 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.017+0000 7efc51260b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e08343630 0x556e084c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.017+0000 7efc51260b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e083bde90 0x556e084c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.017+0000 7efc51260b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e084ca990 0x556e084c8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.017+0000 7efc51260b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556e084cd460 con 0x556e08343630 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.017+0000 7efc51260b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556e0837dcb0 con 0x556e083bde90 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.017+0000 7efc51260b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e084cd5e0 con 0x556e084ca990 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.017+0000 7efc51260b80 10 client.0 _ll_register_callbacks cb 0x556e08254850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e08343630 0x556e084c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e08343630 0x556e084c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48992/0 (socket says 172.21.15.115:48992) 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc3bfff700 1 -- 172.21.15.115:0/4053321336 learned_addr learned my addr 172.21.15.115:0/4053321336 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc40de7700 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e084ca990 0x556e084c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc3b7fe700 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e083bde90 0x556e084c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc40de7700 1 -- 172.21.15.115:0/4053321336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e083bde90 msgr2=0x556e084c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc40de7700 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e083bde90 0x556e084c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc40de7700 1 -- 172.21.15.115:0/4053321336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e08343630 msgr2=0x556e084c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc40de7700 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e08343630 0x556e084c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc40de7700 1 -- 172.21.15.115:0/4053321336 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0825e620 con 0x556e084ca990 2024-05-09T05:42:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc3b7fe700 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e083bde90 0x556e084c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc3bfff700 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e08343630 0x556e084c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc40de7700 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e084ca990 0x556e084c8490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc34009fd0 tx=0x7efc34009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b13d614aa48a0207 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.018+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4053321336 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34003c10 con 0x556e084ca990 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc51260b80 1 -- 172.21.15.115:0/4053321336 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc2c0b21b0 con 0x556e084ca990 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4053321336 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efc34003db0 con 0x556e084ca990 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4053321336 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efc3402f050 con 0x556e084ca990 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc51260b80 10 client.26248 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc51260b80 1 -- 172.21.15.115:0/4053321336 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556e084e09d0 con 0x556e084ca990 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4053321336 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc2c0b21b0 con 0x556e084ca990 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4053321336 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7efc34011070 con 0x556e084ca990 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc51260b80 10 client.26248 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:21.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc51260b80 10 client.26248 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:42:21.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc51260b80 20 client.26248 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:21.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc51260b80 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e084e34c0 0x556e084e5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc51260b80 4 client.26248 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:21.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.019+0000 7efc51260b80 1 -- 172.21.15.115:0/4053321336 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e083f0c10 con 0x556e084e34c0 2024-05-09T05:42:21.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.020+0000 7efc3bfff700 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e084e34c0 0x556e084e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.020+0000 7efc3bfff700 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e084e34c0 0x556e084e5910 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c42c7c7b7946cf server_cookie=ab347e21d9ba98a2 in_seq=0 out_seq=0 2024-05-09T05:42:21.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.020+0000 7efc1b7fe700 10 client.26248 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:21.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4053321336 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e083f0c10 con 0x556e084e34c0 2024-05-09T05:42:21.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc1b7fe700 10 client.26248 handle_command_reply: tid=0 2024-05-09T05:42:21.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 client.26248 shutdown 2024-05-09T05:42:21.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 -- 172.21.15.115:0/4053321336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e084ca990 msgr2=0x556e084c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:21.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e084ca990 0x556e084c8490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc34009fd0 tx=0x7efc34009e20 comp rx=0 tx=0).stop 2024-05-09T05:42:21.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 -- 172.21.15.115:0/4053321336 shutdown_connections 2024-05-09T05:42:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e084e34c0 0x556e084e5910 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e084ca990 0x556e084c8490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e08343630 0x556e084c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 --2- 172.21.15.115:0/4053321336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e083bde90 0x556e084c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 -- 172.21.15.115:0/4053321336 >> 172.21.15.115:0/4053321336 conn(0x556e08347100 msgr2=0x556e084bae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc1b7fe700 0 client.26248 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 -- 172.21.15.115:0/4053321336 shutdown_connections 2024-05-09T05:42:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.021+0000 7efc51260b80 1 -- 172.21.15.115:0/4053321336 wait complete. 2024-05-09T05:42:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.022+0000 7efc51260b80 20 client.26248 trim_cache size 0 max 16384 2024-05-09T05:42:21.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.043+0000 7efc51260b80 1 Processor -- start 2024-05-09T05:42:21.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.043+0000 7efc51260b80 1 -- start start 2024-05-09T05:42:21.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.043+0000 7efc51260b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.043+0000 7efc51260b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 0x556e084eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.043+0000 7efc51260b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.043+0000 7efc51260b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efc2800eba0 con 0x556e084ca990 2024-05-09T05:42:21.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.043+0000 7efc51260b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efc280107d0 con 0x556e08343630 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.043+0000 7efc51260b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efc2800f270 con 0x556e083bde90 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084eb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:34626/0 (socket says 172.21.15.115:34626) 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3bfff700 1 -- 172.21.15.115:0/2154870897 learned_addr learned my addr 172.21.15.115:0/2154870897 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3b7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 0x556e084eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc40de7700 1 --2- 172.21.15.115:0/2154870897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3bfff700 1 -- 172.21.15.115:0/2154870897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 msgr2=0x556e084eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3bfff700 1 --2- 172.21.15.115:0/2154870897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 0x556e084eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3bfff700 1 -- 172.21.15.115:0/2154870897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 msgr2=0x556e084d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3bfff700 1 --2- 172.21.15.115:0/2154870897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084d2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3bfff700 1 -- 172.21.15.115:0/2154870897 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e084c1270 con 0x556e08343630 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc3bfff700 1 --2- 172.21.15.115:0/2154870897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084eb0f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0a92e0 tx=0x7efc2c005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e931eae9f3acc9a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.044+0000 7efc21ffb700 1 -- 172.21.15.115:0/2154870897 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c06d070 con 0x556e08343630 2024-05-09T05:42:21.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.045+0000 7efc21ffb700 1 -- 172.21.15.115:0/2154870897 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc2c0b21b0 con 0x556e08343630 2024-05-09T05:42:21.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.045+0000 7efc21ffb700 1 -- 172.21.15.115:0/2154870897 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0a85b0 con 0x556e08343630 2024-05-09T05:42:21.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.045+0000 7efc51260b80 1 -- 172.21.15.115:0/2154870897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 msgr2=0x556e084eb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:21.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.045+0000 7efc51260b80 1 --2- 172.21.15.115:0/2154870897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084eb0f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0a92e0 tx=0x7efc2c005c80 comp rx=0 tx=0).stop 2024-05-09T05:42:21.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.045+0000 7efc51260b80 1 -- 172.21.15.115:0/2154870897 shutdown_connections 2024-05-09T05:42:21.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.045+0000 7efc51260b80 1 --2- 172.21.15.115:0/2154870897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e08343630 0x556e084eb0f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.045+0000 7efc51260b80 1 --2- 172.21.15.115:0/2154870897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e084ca990 0x556e084d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.045+0000 7efc51260b80 1 --2- 172.21.15.115:0/2154870897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e083bde90 0x556e084eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.045+0000 7efc51260b80 1 -- 172.21.15.115:0/2154870897 >> 172.21.15.115:0/2154870897 conn(0x556e08347100 msgr2=0x556e084bae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.046+0000 7efc51260b80 1 -- 172.21.15.115:0/2154870897 shutdown_connections 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.046+0000 7efc51260b80 1 -- 172.21.15.115:0/2154870897 wait complete. 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.046+0000 7efc51260b80 1 -- start start 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.046+0000 7efc51260b80 1 Processor -- start 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.046+0000 7efc51260b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e08343630 0x556e084d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.046+0000 7efc51260b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e083bde90 0x556e084ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc51260b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e084ca990 0x556e084ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc51260b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efc280100a0 con 0x556e083bde90 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc51260b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efc2800f940 con 0x556e084ca990 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc51260b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efc2800cb40 con 0x556e08343630 2024-05-09T05:42:21.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e083bde90 0x556e084ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e083bde90 0x556e084ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49014/0 (socket says 172.21.15.115:49014) 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3b7fe700 1 -- 172.21.15.115:0/4091331341 learned_addr learned my addr 172.21.15.115:0/4091331341 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3bfff700 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e08343630 0x556e084d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc40de7700 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e084ca990 0x556e084ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3bfff700 1 -- 172.21.15.115:0/4091331341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e084ca990 msgr2=0x556e084ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3bfff700 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e084ca990 0x556e084ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3bfff700 1 -- 172.21.15.115:0/4091331341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e083bde90 msgr2=0x556e084ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3bfff700 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e083bde90 0x556e084ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3bfff700 1 -- 172.21.15.115:0/4091331341 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e084cf160 con 0x556e08343630 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3b7fe700 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e083bde90 0x556e084ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc51260b80 10 client.0 _ll_register_callbacks cb 0x556e08254850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc51260b80 1 -- 172.21.15.115:0/4091331341 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc2c0b21b0 con 0x556e08343630 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc3bfff700 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e08343630 0x556e084d2f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00b2f0 tx=0x7efc2c0a8fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=becbabd15fceaff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc1b7fe700 10 client.26254 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:42:21.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.047+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4091331341 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c088050 con 0x556e08343630 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4091331341 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efc2c0ca300 con 0x556e08343630 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4091331341 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efc2c06d070 con 0x556e08343630 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc51260b80 10 client.26254 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc51260b80 1 -- 172.21.15.115:0/4091331341 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({fsmap=34}) -- 0x556e0854daa0 con 0x556e08343630 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4091331341 <== mon.1 v2:172.21.15.195:3300/0 4 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7efc2c069ce0 con 0x556e08343630 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc51260b80 10 client.26254 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc51260b80 10 client.26254 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc51260b80 20 client.26254 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc51260b80 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e0854dd60 0x556e08550180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:21.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc51260b80 4 client.26254 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-05-09T05:42:21.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc51260b80 1 -- 172.21.15.115:0/4091331341 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x556e083eacb0 con 0x556e0854dd60 2024-05-09T05:42:21.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.048+0000 7efc3b7fe700 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e0854dd60 0x556e08550180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:21.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.049+0000 7efc3b7fe700 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e0854dd60 0x556e08550180 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afaa88e98b5dd160 server_cookie=cf519bd5b058b15c in_seq=0 out_seq=0 2024-05-09T05:42:21.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.049+0000 7efc1b7fe700 10 client.26254 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.162+0000 7efc1b7fe700 1 -- 172.21.15.115:0/4091331341 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556e083eacb0 con 0x556e0854dd60 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.162+0000 7efc1b7fe700 10 client.26254 handle_command_reply: tid=0 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.162+0000 7efc51260b80 1 client.26254 shutdown 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 -- 172.21.15.115:0/4091331341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e08343630 msgr2=0x556e084d2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e08343630 0x556e084d2f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00b2f0 tx=0x7efc2c0a8fa0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 -- 172.21.15.115:0/4091331341 shutdown_connections 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e084ca990 0x556e084ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e0854dd60 0x556e08550180 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e08343630 0x556e084d2f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 --2- 172.21.15.115:0/4091331341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e083bde90 0x556e084ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc1b7fe700 0 client.26254 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:21.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 -- 172.21.15.115:0/4091331341 >> 172.21.15.115:0/4091331341 conn(0x556e08347100 msgr2=0x556e084bb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:21.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 -- 172.21.15.115:0/4091331341 shutdown_connections 2024-05-09T05:42:21.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 1 -- 172.21.15.115:0/4091331341 wait complete. 2024-05-09T05:42:21.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc51260b80 20 client.26254 trim_cache size 0 max 16384 2024-05-09T05:42:21.157 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:42:21.157 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 34, 2024-05-09T05:42:21.157 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:42:21.157 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 10, 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "b011d006": { 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 10, 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 139.4, 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:42:21.158 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:42:21.159 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:42:21.159 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:42:21.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:21.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:21.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:21.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:21.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:21.159 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc4284a700 1 -- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efc240b29f0 msgr2=0x7efc240b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc4284a700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efc240b29f0 0x7efc240b4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc3c102440 tx=0x7efc30009260 comp rx=0 tx=0).stop 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc4284a700 1 -- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c1034c0 msgr2=0x7efc3c19bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.163+0000 7efc4284a700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c1034c0 0x7efc3c19bb40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00d950 tx=0x7efc2c00dc60 comp rx=0 tx=0).stop 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.164+0000 7efc4284a700 1 -- 172.21.15.115:0/1162728070 shutdown_connections 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.164+0000 7efc4284a700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efc240b29f0 0x7efc240b4f00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.164+0000 7efc4284a700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c108740 0x7efc3c19c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.164+0000 7efc4284a700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc3c1034c0 0x7efc3c19bb40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.164+0000 7efc4284a700 1 --2- 172.21.15.115:0/1162728070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c107df0 0x7efc3c19c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.164+0000 7efc4284a700 1 -- 172.21.15.115:0/1162728070 >> 172.21.15.115:0/1162728070 conn(0x7efc3c0fedc0 msgr2=0x7efc3c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.164+0000 7efc4284a700 1 -- 172.21.15.115:0/1162728070 shutdown_connections 2024-05-09T05:42:21.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:21.164+0000 7efc4284a700 1 -- 172.21.15.115:0/1162728070 wait complete. 2024-05-09T05:42:21.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:21 smithi195 ceph-mon[33844]: pgmap v127: 129 pgs: 129 active+clean; 2.2 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:42:21.619 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:21 smithi115 ceph-mon[30349]: pgmap v127: 129 pgs: 129 active+clean; 2.2 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:42:21.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:21 smithi161 ceph-mon[33805]: pgmap v127: 129 pgs: 129 active+clean; 2.2 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:42:22.471 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b011d006', quiesce took 11.9 seconds. Will release after: 7.499999999999998 2024-05-09T05:42:23.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.513+0000 7f6054e65700 1 -- 172.21.15.115:0/3756996182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050108760 msgr2=0x7f60501100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.513+0000 7f6054e65700 1 --2- 172.21.15.115:0/3756996182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050108760 0x7f60501100c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f604800b0d0 tx=0x7f604800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.513+0000 7f6054e65700 1 -- 172.21.15.115:0/3756996182 shutdown_connections 2024-05-09T05:42:23.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.513+0000 7f6054e65700 1 --2- 172.21.15.115:0/3756996182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050108760 0x7f60501100c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.513+0000 7f6054e65700 1 --2- 172.21.15.115:0/3756996182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60501034e0 0x7f60501038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.513+0000 7f6054e65700 1 --2- 172.21.15.115:0/3756996182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6050107e10 0x7f6050108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.513+0000 7f6054e65700 1 -- 172.21.15.115:0/3756996182 >> 172.21.15.115:0/3756996182 conn(0x7f60500fef50 msgr2=0x7f6050101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.513+0000 7f6054e65700 1 -- 172.21.15.115:0/3756996182 shutdown_connections 2024-05-09T05:42:23.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.513+0000 7f6054e65700 1 -- 172.21.15.115:0/3756996182 wait complete. 2024-05-09T05:42:23.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:23 smithi115 ceph-mon[30349]: pgmap v128: 129 pgs: 129 active+clean; 2.2 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.514+0000 7f6054e65700 1 Processor -- start 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.514+0000 7f6054e65700 1 -- start start 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f6054e65700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60501034e0 0x7f605019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f6054e65700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050107e10 0x7f605019c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f6054e65700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6050108760 0x7f60501a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f6054e65700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6050113290 con 0x7f6050107e10 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f6054e65700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6050113110 con 0x7f6050108760 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f6054e65700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6050113410 con 0x7f60501034e0 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f604dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050107e10 0x7f605019c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f604dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050107e10 0x7f605019c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44940/0 (socket says 172.21.15.115:44940) 2024-05-09T05:42:23.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f604dd9b700 1 -- 172.21.15.115:0/588327099 learned_addr learned my addr 172.21.15.115:0/588327099 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f604e59c700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60501034e0 0x7f605019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f604ed9d700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6050108760 0x7f60501a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f604e59c700 1 -- 172.21.15.115:0/588327099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6050108760 msgr2=0x7f60501a6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f604e59c700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6050108760 0x7f60501a6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f604e59c700 1 -- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050107e10 msgr2=0x7f605019c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.515+0000 7f604e59c700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050107e10 0x7f605019c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.516+0000 7f604e59c700 1 -- 172.21.15.115:0/588327099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605019c940 con 0x7f60501034e0 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.516+0000 7f604dd9b700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050107e10 0x7f605019c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.516+0000 7f604e59c700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60501034e0 0x7f605019bb50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f603800d750 tx=0x7f603800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.516+0000 7f60477fe700 1 -- 172.21.15.115:0/588327099 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038014030 con 0x7f60501034e0 2024-05-09T05:42:23.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.516+0000 7f6054e65700 1 -- 172.21.15.115:0/588327099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60501a7230 con 0x7f60501034e0 2024-05-09T05:42:23.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.516+0000 7f60477fe700 1 -- 172.21.15.115:0/588327099 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60380044b0 con 0x7f60501034e0 2024-05-09T05:42:23.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.516+0000 7f60477fe700 1 -- 172.21.15.115:0/588327099 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038005280 con 0x7f60501034e0 2024-05-09T05:42:23.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.516+0000 7f6054e65700 1 -- 172.21.15.115:0/588327099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60501a7770 con 0x7f60501034e0 2024-05-09T05:42:23.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.520+0000 7f60477fe700 1 -- 172.21.15.115:0/588327099 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f60380054e0 con 0x7f60501034e0 2024-05-09T05:42:23.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f606387bb80 1 Processor -- start 2024-05-09T05:42:23.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f60477fe700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f603c0b29f0 0x7f603c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f604ed9d700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f603c0b29f0 0x7f603c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f60477fe700 1 -- 172.21.15.115:0/588327099 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f60380e25b0 con 0x7f60501034e0 2024-05-09T05:42:23.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f606387bb80 1 -- start start 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f606387bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d0ad160 0x56146d1464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f606387bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 0x56146d146a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f606387bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f606387bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56146d062ff0 con 0x56146d1406a0 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f606387bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56146d0017f0 con 0x56146d0ad160 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.521+0000 7f606387bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56146d05efb0 con 0x56146cfc34a0 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d0ad160 0x56146d1464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44954/0 (socket says 172.21.15.115:44954) 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604ed9d700 1 -- 172.21.15.115:0/1238562929 learned_addr learned my addr 172.21.15.115:0/1238562929 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604ed9d700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f603c0b29f0 0x7f603c0b4f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6048009fd0 tx=0x7f604800b040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604dd9b700 1 --2- 172.21.15.115:0/1238562929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 0x56146d146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604e59c700 1 -- 172.21.15.115:0/1238562929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 msgr2=0x56146d146a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604e59c700 1 --2- 172.21.15.115:0/1238562929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 0x56146d146a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604e59c700 1 -- 172.21.15.115:0/1238562929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 msgr2=0x56146d140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604e59c700 1 --2- 172.21.15.115:0/1238562929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604e59c700 1 -- 172.21.15.115:0/1238562929 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56146cff9370 con 0x56146d0ad160 2024-05-09T05:42:23.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604ed9d700 1 --2- 172.21.15.115:0/1238562929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:23.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.522+0000 7f604e59c700 1 --2- 172.21.15.115:0/1238562929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d0ad160 0x56146d1464f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60380df9c0 tx=0x7f60380a90e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732145a54e4390ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.523+0000 7f6044ff9700 1 -- 172.21.15.115:0/1238562929 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603801d040 con 0x56146d0ad160 2024-05-09T05:42:23.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.523+0000 7f6044ff9700 1 -- 172.21.15.115:0/1238562929 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60380910e0 con 0x56146d0ad160 2024-05-09T05:42:23.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.523+0000 7f6044ff9700 1 -- 172.21.15.115:0/1238562929 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60380e2210 con 0x56146d0ad160 2024-05-09T05:42:23.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 -- 172.21.15.115:0/1238562929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d0ad160 msgr2=0x56146d1464f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 --2- 172.21.15.115:0/1238562929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d0ad160 0x56146d1464f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60380df9c0 tx=0x7f60380a90e0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 -- 172.21.15.115:0/1238562929 shutdown_connections 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 --2- 172.21.15.115:0/1238562929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d0ad160 0x56146d1464f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 --2- 172.21.15.115:0/1238562929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 --2- 172.21.15.115:0/1238562929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 0x56146d146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 -- 172.21.15.115:0/1238562929 >> 172.21.15.115:0/1238562929 conn(0x56146d0236a0 msgr2=0x56146d13b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 -- 172.21.15.115:0/1238562929 shutdown_connections 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 -- 172.21.15.115:0/1238562929 wait complete. 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 -- start start 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.524+0000 7f606387bb80 1 Processor -- start 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f606387bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 0x56146d14a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f606387bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d0ad160 0x56146d14a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f606387bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d14ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f606387bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56146cedeba0 con 0x56146d1406a0 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f606387bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56146d01d150 con 0x56146cfc34a0 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f606387bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56146d14d1d0 con 0x56146d0ad160 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f606387bb80 10 client.0 _ll_register_callbacks cb 0x56146ced46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d14ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d14ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44964/0 (socket says 172.21.15.115:44964) 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604ed9d700 1 -- 172.21.15.115:0/4249698816 learned_addr learned my addr 172.21.15.115:0/4249698816 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604e59c700 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 0x56146d14a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604dd9b700 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d0ad160 0x56146d14a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604ed9d700 1 -- 172.21.15.115:0/4249698816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d0ad160 msgr2=0x56146d14a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604ed9d700 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d0ad160 0x56146d14a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604ed9d700 1 -- 172.21.15.115:0/4249698816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 msgr2=0x56146d14a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604ed9d700 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 0x56146d14a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.525+0000 7f604ed9d700 1 -- 172.21.15.115:0/4249698816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56146cede490 con 0x56146d1406a0 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f604ed9d700 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d14ad20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f604800d010 tx=0x7f604800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb8c1e68ea15e37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f6035ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f6035ffb700 1 -- 172.21.15.115:0/4249698816 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048035060 con 0x56146d1406a0 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f6035ffb700 1 -- 172.21.15.115:0/4249698816 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6048027410 con 0x56146d1406a0 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f606387bb80 1 -- 172.21.15.115:0/4249698816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60380910e0 con 0x56146d1406a0 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f6035ffb700 1 -- 172.21.15.115:0/4249698816 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f604803c050 con 0x56146d1406a0 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f606387bb80 10 client.16728 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f606387bb80 1 -- 172.21.15.115:0/4249698816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56146d157cc0 con 0x56146d1406a0 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f6035ffb700 1 -- 172.21.15.115:0/4249698816 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60380910e0 con 0x56146d1406a0 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f6035ffb700 1 -- 172.21.15.115:0/4249698816 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f6048011070 con 0x56146d1406a0 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f606387bb80 10 client.16728 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f606387bb80 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f606387bb80 20 client.16728 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:23.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f606387bb80 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56146d15a350 0x56146d15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f606387bb80 4 client.16728 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.526+0000 7f606387bb80 1 -- 172.21.15.115:0/4249698816 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56146d0709d0 con 0x56146d15a350 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.527+0000 7f604e59c700 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56146d15a350 0x56146d15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.527+0000 7f6035ffb700 10 client.16728 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.527+0000 7f604e59c700 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56146d15a350 0x56146d15c7d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895508c6361433a5 server_cookie=eb20b7b1b578b120 in_seq=0 out_seq=0 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f6035ffb700 1 -- 172.21.15.115:0/4249698816 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56146d0709d0 con 0x56146d15a350 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f6035ffb700 10 client.16728 handle_command_reply: tid=0 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 client.16728 shutdown 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 -- 172.21.15.115:0/4249698816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 msgr2=0x56146d14ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d14ad20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f604800d010 tx=0x7f604800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 -- 172.21.15.115:0/4249698816 shutdown_connections 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 0x56146d14a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56146d15a350 0x56146d15c7d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d1406a0 0x56146d14ad20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 --2- 172.21.15.115:0/4249698816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d0ad160 0x56146d14a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 -- 172.21.15.115:0/4249698816 >> 172.21.15.115:0/4249698816 conn(0x56146d0236a0 msgr2=0x56146d13b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f6035ffb700 0 client.16728 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:23.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 -- 172.21.15.115:0/4249698816 shutdown_connections 2024-05-09T05:42:23.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 1 -- 172.21.15.115:0/4249698816 wait complete. 2024-05-09T05:42:23.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.528+0000 7f606387bb80 20 client.16728 trim_cache size 0 max 16384 2024-05-09T05:42:23.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.543+0000 7f606387bb80 1 Processor -- start 2024-05-09T05:42:23.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.543+0000 7f606387bb80 1 -- start start 2024-05-09T05:42:23.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f606387bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 0x56146d161f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f606387bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d1624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f606387bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d1406a0 0x56146d1452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f606387bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60501ae0b0 con 0x56146d0ad160 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f606387bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f60501afce0 con 0x56146d1406a0 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f606387bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f60501ae780 con 0x56146cfc34a0 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d1624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d1624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44972/0 (socket says 172.21.15.115:44972) 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604dd9b700 1 -- 172.21.15.115:0/3782688069 learned_addr learned my addr 172.21.15.115:0/3782688069 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604e59c700 1 --2- 172.21.15.115:0/3782688069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 0x56146d161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604ed9d700 1 --2- 172.21.15.115:0/3782688069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d1406a0 0x56146d1452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604dd9b700 1 -- 172.21.15.115:0/3782688069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 msgr2=0x56146d161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604dd9b700 1 --2- 172.21.15.115:0/3782688069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 0x56146d161f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604dd9b700 1 -- 172.21.15.115:0/3782688069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d1406a0 msgr2=0x56146d1452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604dd9b700 1 --2- 172.21.15.115:0/3782688069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d1406a0 0x56146d1452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604dd9b700 1 -- 172.21.15.115:0/3782688069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56146d13c110 con 0x56146d0ad160 2024-05-09T05:42:23.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f604dd9b700 1 --2- 172.21.15.115:0/3782688069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d1624d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f60400125e0 tx=0x7f60400128f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1321ab0dc270d1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.544+0000 7f6037fff700 1 -- 172.21.15.115:0/3782688069 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040020070 con 0x56146d0ad160 2024-05-09T05:42:23.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.545+0000 7f6037fff700 1 -- 172.21.15.115:0/3782688069 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6040010990 con 0x56146d0ad160 2024-05-09T05:42:23.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.545+0000 7f6037fff700 1 -- 172.21.15.115:0/3782688069 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604001b9b0 con 0x56146d0ad160 2024-05-09T05:42:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.545+0000 7f606387bb80 1 -- 172.21.15.115:0/3782688069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 msgr2=0x56146d1624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.545+0000 7f606387bb80 1 --2- 172.21.15.115:0/3782688069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d1624d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f60400125e0 tx=0x7f60400128f0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 -- 172.21.15.115:0/3782688069 shutdown_connections 2024-05-09T05:42:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 --2- 172.21.15.115:0/3782688069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146d1406a0 0x56146d1452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 --2- 172.21.15.115:0/3782688069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146cfc34a0 0x56146d161f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 --2- 172.21.15.115:0/3782688069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d1624d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 -- 172.21.15.115:0/3782688069 >> 172.21.15.115:0/3782688069 conn(0x56146d0236a0 msgr2=0x56146d148bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 -- 172.21.15.115:0/3782688069 shutdown_connections 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 -- 172.21.15.115:0/3782688069 wait complete. 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 -- start start 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 Processor -- start 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 0x56146d1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d161580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d1406a0 0x56146d161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60501af5b0 con 0x56146d0ad160 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f60501aee50 con 0x56146cfc34a0 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f60501ac050 con 0x56146d1406a0 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.546+0000 7f606387bb80 10 client.0 _ll_register_callbacks cb 0x56146ced46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d161580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44982/0 (socket says 172.21.15.115:44982) 2024-05-09T05:42:23.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604dd9b700 1 -- 172.21.15.115:0/1302279504 learned_addr learned my addr 172.21.15.115:0/1302279504 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604e59c700 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 0x56146d1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604ed9d700 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d1406a0 0x56146d161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604dd9b700 1 -- 172.21.15.115:0/1302279504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d1406a0 msgr2=0x56146d161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604dd9b700 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d1406a0 0x56146d161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604dd9b700 1 -- 172.21.15.115:0/1302279504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 msgr2=0x56146d1527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604dd9b700 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 0x56146d1527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604dd9b700 1 -- 172.21.15.115:0/1302279504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56146d1621b0 con 0x56146d0ad160 2024-05-09T05:42:23.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604ed9d700 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d1406a0 0x56146d161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f604dd9b700 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d161580 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f604001e040 tx=0x7f6040010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aced7271fe362986 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f6035ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f6035ffb700 1 -- 172.21.15.115:0/1302279504 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040020040 con 0x56146d0ad160 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f6035ffb700 1 -- 172.21.15.115:0/1302279504 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6040010790 con 0x56146d0ad160 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.547+0000 7f606387bb80 1 -- 172.21.15.115:0/1302279504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6040010990 con 0x56146d0ad160 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f6035ffb700 1 -- 172.21.15.115:0/1302279504 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6040033050 con 0x56146d0ad160 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f606387bb80 10 client.16734 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f606387bb80 1 -- 172.21.15.115:0/1302279504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56146d1c49d0 con 0x56146d0ad160 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f6035ffb700 1 -- 172.21.15.115:0/1302279504 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6040010990 con 0x56146d0ad160 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f6035ffb700 1 -- 172.21.15.115:0/1302279504 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f604002e030 con 0x56146d0ad160 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f606387bb80 10 client.16734 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f606387bb80 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f606387bb80 20 client.16734 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:23.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f606387bb80 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56146d1c5090 0x56146d1c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f606387bb80 4 client.16734 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f606387bb80 1 -- 172.21.15.115:0/1302279504 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56146d06aa70 con 0x56146d1c5090 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.548+0000 7f604e59c700 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56146d1c5090 0x56146d1c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f6035ffb700 10 client.16734 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f604e59c700 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56146d1c5090 0x56146d1c74b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f36cd7410fa0479e server_cookie=afa59e2d85d3c303 in_seq=0 out_seq=0 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f6035ffb700 1 -- 172.21.15.115:0/1302279504 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56146d06aa70 con 0x56146d1c5090 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f6035ffb700 10 client.16734 handle_command_reply: tid=0 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 client.16734 shutdown 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 -- 172.21.15.115:0/1302279504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 msgr2=0x56146d161580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d161580 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f604001e040 tx=0x7f6040010d70 comp rx=0 tx=0).stop 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 -- 172.21.15.115:0/1302279504 shutdown_connections 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56146cfc34a0 0x56146d1527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56146d1c5090 0x56146d1c74b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56146d1406a0 0x56146d161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 --2- 172.21.15.115:0/1302279504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56146d0ad160 0x56146d161580 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f6035ffb700 0 client.16734 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:23.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 -- 172.21.15.115:0/1302279504 >> 172.21.15.115:0/1302279504 conn(0x56146d0236a0 msgr2=0x56146d14a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 -- 172.21.15.115:0/1302279504 shutdown_connections 2024-05-09T05:42:23.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 1 -- 172.21.15.115:0/1302279504 wait complete. 2024-05-09T05:42:23.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.549+0000 7f606387bb80 20 client.16734 trim_cache size 0 max 16384 2024-05-09T05:42:23.551 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 -- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f603c0b29f0 msgr2=0x7f603c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f603c0b29f0 0x7f603c0b4f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6048009fd0 tx=0x7f604800b040 comp rx=0 tx=0).stop 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 -- 172.21.15.115:0/588327099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60501034e0 msgr2=0x7f605019bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60501034e0 0x7f605019bb50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f603800d750 tx=0x7f603800da60 comp rx=0 tx=0).stop 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 -- 172.21.15.115:0/588327099 shutdown_connections 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f603c0b29f0 0x7f603c0b4f00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6050108760 0x7f60501a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f60501034e0 0x7f605019bb50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 --2- 172.21.15.115:0/588327099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6050107e10 0x7f605019c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 -- 172.21.15.115:0/588327099 >> 172.21.15.115:0/588327099 conn(0x7f60500fef50 msgr2=0x7f6050107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 -- 172.21.15.115:0/588327099 shutdown_connections 2024-05-09T05:42:23.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:23.550+0000 7f6054e65700 1 -- 172.21.15.115:0/588327099 wait complete. 2024-05-09T05:42:23.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:23 smithi195 ceph-mon[33844]: pgmap v128: 129 pgs: 129 active+clean; 2.2 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:42:23.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:23 smithi161 ceph-mon[33805]: pgmap v128: 129 pgs: 129 active+clean; 2.2 GiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:42:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.688+0000 7f90c497e640 1 -- 172.21.15.115:0/684053092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01090b0 msgr2=0x7f90c0103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.688+0000 7f90c497e640 1 --2- 172.21.15.115:0/684053092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01090b0 0x7f90c0103350 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009900 tx=0x7f90a802a410 comp rx=0 tx=0).stop 2024-05-09T05:42:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.688+0000 7f90c497e640 1 -- 172.21.15.115:0/684053092 shutdown_connections 2024-05-09T05:42:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.688+0000 7f90c497e640 1 --2- 172.21.15.115:0/684053092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c01039c0 0x7f90c010f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.688+0000 7f90c497e640 1 --2- 172.21.15.115:0/684053092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01090b0 0x7f90c0103350 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.688+0000 7f90c497e640 1 --2- 172.21.15.115:0/684053092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c0108c10 0x7f90c0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.688+0000 7f90c497e640 1 -- 172.21.15.115:0/684053092 >> 172.21.15.115:0/684053092 conn(0x7f90c00fd9d0 msgr2=0x7f90c00ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.688+0000 7f90c497e640 1 -- 172.21.15.115:0/684053092 shutdown_connections 2024-05-09T05:42:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.689+0000 7f90c497e640 1 -- 172.21.15.115:0/684053092 wait complete. 2024-05-09T05:42:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.689+0000 7f90c497e640 1 Processor -- start 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.689+0000 7f90c497e640 1 -- start start 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90c497e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01039c0 0x7f90c01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90c497e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0108c10 0x7f90c0199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90c497e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c01090b0 0x7f90c019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90c497e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0112cb0 con 0x7f90c01039c0 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90c497e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0112b30 con 0x7f90c0108c10 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90c497e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0112e30 con 0x7f90c01090b0 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90be59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01039c0 0x7f90c01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90be59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01039c0 0x7f90c01997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45000/0 (socket says 172.21.15.115:45000) 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90be59c640 1 -- 172.21.15.115:0/1004932290 learned_addr learned my addr 172.21.15.115:0/1004932290 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90bed9d640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c01090b0 0x7f90c019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90b7fff640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0108c10 0x7f90c0199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90be59c640 1 -- 172.21.15.115:0/1004932290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c01090b0 msgr2=0x7f90c019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90be59c640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c01090b0 0x7f90c019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90be59c640 1 -- 172.21.15.115:0/1004932290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0108c10 msgr2=0x7f90c0199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90be59c640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0108c10 0x7f90c0199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90be59c640 1 -- 172.21.15.115:0/1004932290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c019e630 con 0x7f90c01039c0 2024-05-09T05:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.690+0000 7f90be59c640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01039c0 0x7f90c01997d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f90b000d740 tx=0x7f90b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.691+0000 7f90b77fe640 1 -- 172.21.15.115:0/1004932290 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0017070 con 0x7f90c01039c0 2024-05-09T05:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.691+0000 7f90c497e640 1 -- 172.21.15.115:0/1004932290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c019e920 con 0x7f90c01039c0 2024-05-09T05:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.691+0000 7f90b77fe640 1 -- 172.21.15.115:0/1004932290 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90b000de50 con 0x7f90c01039c0 2024-05-09T05:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.691+0000 7f90b77fe640 1 -- 172.21.15.115:0/1004932290 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0004c60 con 0x7f90c01039c0 2024-05-09T05:42:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.691+0000 7f90c497e640 1 -- 172.21.15.115:0/1004932290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c019edf0 con 0x7f90c01039c0 2024-05-09T05:42:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.693+0000 7f90cb830740 1 Processor -- start 2024-05-09T05:42:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90cb830740 1 -- start start 2024-05-09T05:42:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90cb830740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96fa181b0 0x55f96fdbee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90cb830740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96f9ef2f0 0x55f96fdbd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90cb830740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9f74d0 0x55f96fdbdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90cb830740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f96fd207c0 con 0x55f96f9ef2f0 2024-05-09T05:42:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90cb830740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f96fcbc0e0 con 0x55f96fa181b0 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90cb830740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f96fd1c780 con 0x55f96f9f74d0 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90b7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9f74d0 0x55f96fdbdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90bed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96fa181b0 0x55f96fdbee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90b7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9f74d0 0x55f96fdbdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:50280/0 (socket says 172.21.15.115:50280) 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90b7fff640 1 -- 172.21.15.115:0/2213814849 learned_addr learned my addr 172.21.15.115:0/2213814849 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90b7fff640 1 -- 172.21.15.115:0/2213814849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96fa181b0 msgr2=0x55f96fdbee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90b7fff640 1 --2- 172.21.15.115:0/2213814849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96fa181b0 0x55f96fdbee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90b7fff640 1 -- 172.21.15.115:0/2213814849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96f9ef2f0 msgr2=0x55f96fdbd720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90b7fff640 1 --2- 172.21.15.115:0/2213814849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96f9ef2f0 0x55f96fdbd720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.694+0000 7f90b7fff640 1 -- 172.21.15.115:0/2213814849 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96fce57c0 con 0x55f96f9f74d0 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.695+0000 7f90b77fe640 1 -- 172.21.15.115:0/1004932290 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f90b0004e00 con 0x7f90c01039c0 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.695+0000 7f90b7fff640 1 --2- 172.21.15.115:0/2213814849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9f74d0 0x55f96fdbdcc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f90a80057c0 tx=0x7f90a8031ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91334ec83a277e4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.695+0000 7f90b4ff9640 1 -- 172.21.15.115:0/2213814849 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8040070 con 0x55f96f9f74d0 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.695+0000 7f90b4ff9640 1 -- 172.21.15.115:0/2213814849 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90a8031e50 con 0x55f96f9f74d0 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.695+0000 7f90b4ff9640 1 -- 172.21.15.115:0/2213814849 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8035040 con 0x55f96f9f74d0 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90b77fe640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90ac0b2950 0x7f90ac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90bed9d640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90ac0b2950 0x7f90ac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90b77fe640 1 -- 172.21.15.115:0/1004932290 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f90b00e21e0 con 0x7f90c01039c0 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90cb830740 1 -- 172.21.15.115:0/2213814849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9f74d0 msgr2=0x55f96fdbdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90bed9d640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90ac0b2950 0x7f90ac0b4e60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f90b8005fd0 tx=0x7f90b8009600 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90cb830740 1 --2- 172.21.15.115:0/2213814849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9f74d0 0x55f96fdbdcc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f90a80057c0 tx=0x7f90a8031ba0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90cb830740 1 -- 172.21.15.115:0/2213814849 shutdown_connections 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90cb830740 1 --2- 172.21.15.115:0/2213814849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9f74d0 0x55f96fdbdcc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90cb830740 1 --2- 172.21.15.115:0/2213814849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96f9ef2f0 0x55f96fdbd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90cb830740 1 --2- 172.21.15.115:0/2213814849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96fa181b0 0x55f96fdbee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90cb830740 1 -- 172.21.15.115:0/2213814849 >> 172.21.15.115:0/2213814849 conn(0x55f96fa0c300 msgr2=0x55f96fdba040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90cb830740 1 -- 172.21.15.115:0/2213814849 shutdown_connections 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.696+0000 7f90cb830740 1 -- 172.21.15.115:0/2213814849 wait complete. 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90cb830740 1 -- start start 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90cb830740 1 Processor -- start 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90cb830740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdcbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90cb830740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fdcc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90cb830740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90cb830740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f96fdccbd0 con 0x55f96fa181b0 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90cb830740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f96fca79c0 con 0x55f96f9f74d0 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90cb830740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f96fdccd50 con 0x55f96f9ef2f0 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90cb830740 10 client.0 _ll_register_callbacks cb 0x55f96fcd4020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90bed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90bed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45008/0 (socket says 172.21.15.115:45008) 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90bed9d640 1 -- 172.21.15.115:0/541724219 learned_addr learned my addr 172.21.15.115:0/541724219 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90be59c640 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdcbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.697+0000 7f90b7fff640 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fdcc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90be59c640 1 -- 172.21.15.115:0/541724219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 msgr2=0x55f96fdcc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90be59c640 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fdcc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90be59c640 1 -- 172.21.15.115:0/541724219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 msgr2=0x55f96fdca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90be59c640 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90be59c640 1 -- 172.21.15.115:0/541724219 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96fcce1f0 con 0x55f96f9ef2f0 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90bed9d640 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90be59c640 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdcbce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90b00df5f0 tx=0x7f90b00a8c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6a6093c42e7851 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90a5ffb640 1 -- 172.21.15.115:0/541724219 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0020040 con 0x55f96f9ef2f0 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90cb830740 1 -- 172.21.15.115:0/541724219 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a8031e50 con 0x55f96f9ef2f0 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90a5ffb640 1 -- 172.21.15.115:0/541724219 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f90b00a86a0 con 0x55f96f9ef2f0 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90a5ffb640 1 -- 172.21.15.115:0/541724219 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90b00ad530 con 0x55f96f9ef2f0 2024-05-09T05:42:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90cb830740 10 client.26284 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90cb830740 1 -- 172.21.15.115:0/541724219 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f96fa83790 con 0x55f96f9ef2f0 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90a5ffb640 1 -- 172.21.15.115:0/541724219 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90a8031e50 con 0x55f96f9ef2f0 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.698+0000 7f90a5ffb640 1 -- 172.21.15.115:0/541724219 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f90b00a8090 con 0x55f96f9ef2f0 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.699+0000 7f90cb830740 10 client.26284 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.699+0000 7f90cb830740 10 client.26284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.699+0000 7f90cb830740 20 client.26284 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.699+0000 7f90cb830740 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f96f9d6d80 0x55f96fddc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.699+0000 7f90cb830740 4 client.26284 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.699+0000 7f90cb830740 1 -- 172.21.15.115:0/541724219 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f96fd2e3d0 con 0x55f96f9d6d80 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.699+0000 7f90b7fff640 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f96f9d6d80 0x55f96fddc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.699+0000 7f90b7fff640 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f96f9d6d80 0x55f96fddc440 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a491f78843f258a server_cookie=53c71c919b2e1317 in_seq=0 out_seq=0 2024-05-09T05:42:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.699+0000 7f90a5ffb640 10 client.26284 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90a5ffb640 1 -- 172.21.15.115:0/541724219 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f90a804fc70 con 0x55f96f9d6d80 2024-05-09T05:42:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90a5ffb640 10 client.26284 handle_command_reply: tid=0 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 client.26284 shutdown 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 -- 172.21.15.115:0/541724219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 msgr2=0x55f96fdcbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdcbce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90b00df5f0 tx=0x7f90b00a8c80 comp rx=0 tx=0).stop 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 -- 172.21.15.115:0/541724219 shutdown_connections 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fdcc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f96f9d6d80 0x55f96fddc440 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 --2- 172.21.15.115:0/541724219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdcbce0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90a5ffb640 0 client.26284 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 -- 172.21.15.115:0/541724219 >> 172.21.15.115:0/541724219 conn(0x55f96fa0c300 msgr2=0x55f96fdbf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 -- 172.21.15.115:0/541724219 shutdown_connections 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 1 -- 172.21.15.115:0/541724219 wait complete. 2024-05-09T05:42:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.700+0000 7f90cb830740 20 client.26284 trim_cache size 0 max 16384 2024-05-09T05:42:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.715+0000 7f90cb830740 1 Processor -- start 2024-05-09T05:42:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.715+0000 7f90cb830740 1 -- start start 2024-05-09T05:42:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.715+0000 7f90cb830740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdbdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90cb830740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fde1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90cb830740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fde2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90cb830740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01aee80 con 0x55f96fa181b0 2024-05-09T05:42:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90cb830740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01b0ab0 con 0x55f96f9f74d0 2024-05-09T05:42:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90cb830740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01af550 con 0x55f96f9ef2f0 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90bed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fde2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90bed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fde2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45010/0 (socket says 172.21.15.115:45010) 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90bed9d640 1 -- 172.21.15.115:0/1035719599 learned_addr learned my addr 172.21.15.115:0/1035719599 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90b7fff640 1 --2- 172.21.15.115:0/1035719599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fde1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90be59c640 1 --2- 172.21.15.115:0/1035719599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdbdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90bed9d640 1 -- 172.21.15.115:0/1035719599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 msgr2=0x55f96fdbdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90bed9d640 1 --2- 172.21.15.115:0/1035719599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdbdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90bed9d640 1 -- 172.21.15.115:0/1035719599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 msgr2=0x55f96fde1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90bed9d640 1 --2- 172.21.15.115:0/1035719599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fde1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90bed9d640 1 -- 172.21.15.115:0/1035719599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96fafa5f0 con 0x55f96fa181b0 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90bed9d640 1 --2- 172.21.15.115:0/1035719599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fde2130 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009fd0 tx=0x7f90b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6aafe2fadbd39e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90a7fff640 1 -- 172.21.15.115:0/1035719599 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8004b20 con 0x55f96fa181b0 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90a7fff640 1 -- 172.21.15.115:0/1035719599 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90b8004cc0 con 0x55f96fa181b0 2024-05-09T05:42:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.716+0000 7f90a7fff640 1 -- 172.21.15.115:0/1035719599 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8005440 con 0x55f96fa181b0 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.717+0000 7f90cb830740 1 -- 172.21.15.115:0/1035719599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 msgr2=0x55f96fde2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.717+0000 7f90cb830740 1 --2- 172.21.15.115:0/1035719599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fde2130 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009fd0 tx=0x7f90b8016000 comp rx=0 tx=0).stop 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.717+0000 7f90cb830740 1 -- 172.21.15.115:0/1035719599 shutdown_connections 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.717+0000 7f90cb830740 1 --2- 172.21.15.115:0/1035719599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fde2130 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.717+0000 7f90cb830740 1 --2- 172.21.15.115:0/1035719599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fde1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.717+0000 7f90cb830740 1 --2- 172.21.15.115:0/1035719599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdbdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.717+0000 7f90cb830740 1 -- 172.21.15.115:0/1035719599 >> 172.21.15.115:0/1035719599 conn(0x55f96fa0c300 msgr2=0x55f96fdbd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.717+0000 7f90cb830740 1 -- 172.21.15.115:0/1035719599 shutdown_connections 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.717+0000 7f90cb830740 1 -- 172.21.15.115:0/1035719599 wait complete. 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90cb830740 1 -- start start 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90cb830740 1 Processor -- start 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90cb830740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdcfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90cb830740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fdd0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90cb830740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdd08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90cb830740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01b0380 con 0x55f96fa181b0 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90cb830740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01afc20 con 0x55f96f9f74d0 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90cb830740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01ace60 con 0x55f96f9ef2f0 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90cb830740 10 client.0 _ll_register_callbacks cb 0x7f90b0009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90bed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdd08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90bed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdd08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45022/0 (socket says 172.21.15.115:45022) 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90bed9d640 1 -- 172.21.15.115:0/2183708186 learned_addr learned my addr 172.21.15.115:0/2183708186 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90b7fff640 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fdd0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.718+0000 7f90be59c640 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdcfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90bed9d640 1 -- 172.21.15.115:0/2183708186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 msgr2=0x55f96fdcfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90bed9d640 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdcfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90bed9d640 1 -- 172.21.15.115:0/2183708186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 msgr2=0x55f96fdd0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90bed9d640 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fdd0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90bed9d640 1 -- 172.21.15.115:0/2183708186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96fce4ed0 con 0x55f96fa181b0 2024-05-09T05:42:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90bed9d640 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdd08b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f90b8020da0 tx=0x7f90b8009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816d2a20461edd8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90a5ffb640 1 -- 172.21.15.115:0/2183708186 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8004a40 con 0x55f96fa181b0 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90a5ffb640 1 -- 172.21.15.115:0/2183708186 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f90b80051c0 con 0x55f96fa181b0 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90cb830740 1 -- 172.21.15.115:0/2183708186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b8004cc0 con 0x55f96fa181b0 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90a5ffb640 1 -- 172.21.15.115:0/2183708186 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90b8041050 con 0x55f96fa181b0 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90cb830740 10 client.16749 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90cb830740 1 -- 172.21.15.115:0/2183708186 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f96fe44a00 con 0x55f96fa181b0 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.719+0000 7f90a5ffb640 1 -- 172.21.15.115:0/2183708186 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90b8004cc0 con 0x55f96fa181b0 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90a5ffb640 1 -- 172.21.15.115:0/2183708186 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f90b800f040 con 0x55f96fa181b0 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90cb830740 10 client.16749 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90cb830740 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90cb830740 20 client.16749 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90cb830740 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f96f9ac1c0 0x55f96fe470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90cb830740 4 client.16749 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:42:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90cb830740 1 -- 172.21.15.115:0/2183708186 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f96fd28240 con 0x55f96f9ac1c0 2024-05-09T05:42:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90be59c640 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f96f9ac1c0 0x55f96fe470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90a5ffb640 10 client.16749 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.720+0000 7f90be59c640 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f96f9ac1c0 0x55f96fe470a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed66c09434488f1 server_cookie=87b8603e948e2453 in_seq=0 out_seq=0 2024-05-09T05:42:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90a5ffb640 1 -- 172.21.15.115:0/2183708186 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f96fd28240 con 0x55f96f9ac1c0 2024-05-09T05:42:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90a5ffb640 10 client.16749 handle_command_reply: tid=0 2024-05-09T05:42:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 client.16749 shutdown 2024-05-09T05:42:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 -- 172.21.15.115:0/2183708186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 msgr2=0x55f96fdd08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdd08b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f90b8020da0 tx=0x7f90b8009df0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 -- 172.21.15.115:0/2183708186 shutdown_connections 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f96fa181b0 0x55f96fdd08b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f96f9f74d0 0x55f96fdd0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f96f9ac1c0 0x55f96fe470a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 --2- 172.21.15.115:0/2183708186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f96f9ef2f0 0x55f96fdcfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90a5ffb640 0 client.16749 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 -- 172.21.15.115:0/2183708186 >> 172.21.15.115:0/2183708186 conn(0x55f96fa0c300 msgr2=0x55f96fdcb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 -- 172.21.15.115:0/2183708186 shutdown_connections 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 1 -- 172.21.15.115:0/2183708186 wait complete. 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.730+0000 7f90cb830740 20 client.16749 trim_cache size 0 max 16384 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:42:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:42:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 -- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90ac0b2950 msgr2=0x7f90ac0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90ac0b2950 0x7f90ac0b4e60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f90b8005fd0 tx=0x7f90b8009600 comp rx=0 tx=0).stop 2024-05-09T05:42:23.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 -- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01039c0 msgr2=0x7f90c01997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01039c0 0x7f90c01997d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f90b000d740 tx=0x7f90b000dc00 comp rx=0 tx=0).stop 2024-05-09T05:42:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 -- 172.21.15.115:0/1004932290 shutdown_connections 2024-05-09T05:42:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90ac0b2950 0x7f90ac0b4e60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90c01090b0 0x7f90c019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c0108c10 0x7f90c0199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 --2- 172.21.15.115:0/1004932290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c01039c0 0x7f90c01997d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 -- 172.21.15.115:0/1004932290 >> 172.21.15.115:0/1004932290 conn(0x7f90c00fd9d0 msgr2=0x7f90c010ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 -- 172.21.15.115:0/1004932290 shutdown_connections 2024-05-09T05:42:23.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:23.731+0000 7f90c497e640 1 -- 172.21.15.115:0/1004932290 wait complete. 2024-05-09T05:42:25.375 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:42:25.375 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:42:25.400 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:25 smithi115 ceph-mon[30349]: pgmap v129: 129 pgs: 129 active+clean; 2.3 GiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 MiB/s wr, 28 op/s 2024-05-09T05:42:25.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:25 smithi195 ceph-mon[33844]: pgmap v129: 129 pgs: 129 active+clean; 2.3 GiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 MiB/s wr, 28 op/s 2024-05-09T05:42:25.624 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:42:25.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:25 smithi161 ceph-mon[33805]: pgmap v129: 129 pgs: 129 active+clean; 2.3 GiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 MiB/s wr, 28 op/s 2024-05-09T05:42:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.543+0000 7f14cb50a700 1 -- 172.21.15.115:0/3304041670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4108740 msgr2=0x7f14c41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 --2- 172.21.15.115:0/3304041670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4108740 0x7f14c41100a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00b0d0 tx=0x7f14bc00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 -- 172.21.15.115:0/3304041670 shutdown_connections 2024-05-09T05:42:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 --2- 172.21.15.115:0/3304041670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14c41034c0 0x7f14c41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 --2- 172.21.15.115:0/3304041670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4108740 0x7f14c41100a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 --2- 172.21.15.115:0/3304041670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c4107df0 0x7f14c4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 -- 172.21.15.115:0/3304041670 >> 172.21.15.115:0/3304041670 conn(0x7f14c40fedc0 msgr2=0x7f14c41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 -- 172.21.15.115:0/3304041670 shutdown_connections 2024-05-09T05:42:27.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 -- 172.21.15.115:0/3304041670 wait complete. 2024-05-09T05:42:27.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 Processor -- start 2024-05-09T05:42:27.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.544+0000 7f14cb50a700 1 -- start start 2024-05-09T05:42:27.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14cb50a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c41034c0 0x7f14c419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14cb50a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4107df0 0x7f14c419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14cb50a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14c4108740 0x7f14c41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14cb50a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14c4113290 con 0x7f14c4107df0 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14cb50a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14c4113110 con 0x7f14c4108740 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14cb50a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14c4113410 con 0x7f14c41034c0 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c8aa5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4107df0 0x7f14c419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c8aa5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4107df0 0x7f14c419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45036/0 (socket says 172.21.15.115:45036) 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c8aa5700 1 -- 172.21.15.115:0/1205432127 learned_addr learned my addr 172.21.15.115:0/1205432127 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14c4108740 0x7f14c41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c92a6700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c41034c0 0x7f14c419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c8aa5700 1 -- 172.21.15.115:0/1205432127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c41034c0 msgr2=0x7f14c419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c41034c0 0x7f14c419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c8aa5700 1 -- 172.21.15.115:0/1205432127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14c4108740 msgr2=0x7f14c41a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14c4108740 0x7f14c41a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c8aa5700 1 -- 172.21.15.115:0/1205432127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c41a1dd0 con 0x7f14c4107df0 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c92a6700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c41034c0 0x7f14c419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14c4108740 0x7f14c41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4107df0 0x7f14c419d4a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f14b400b500 tx=0x7f14b400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14ba7fc700 1 -- 172.21.15.115:0/1205432127 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4015020 con 0x7f14c4107df0 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14ba7fc700 1 -- 172.21.15.115:0/1205432127 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14b4011cc0 con 0x7f14c4107df0 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14cb50a700 1 -- 172.21.15.115:0/1205432127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c41a20c0 con 0x7f14c4107df0 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.545+0000 7f14ba7fc700 1 -- 172.21.15.115:0/1205432127 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4010c80 con 0x7f14c4107df0 2024-05-09T05:42:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.546+0000 7f14cb50a700 1 -- 172.21.15.115:0/1205432127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c41a99e0 con 0x7f14c4107df0 2024-05-09T05:42:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.548+0000 7f14ba7fc700 1 -- 172.21.15.115:0/1205432127 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f14b4018020 con 0x7f14c4107df0 2024-05-09T05:42:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.549+0000 7f14ba7fc700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f14b00b29f0 0x7f14b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.549+0000 7f14c92a6700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f14b00b29f0 0x7f14b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.549+0000 7f14d9f20b80 1 Processor -- start 2024-05-09T05:42:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.549+0000 7f14ba7fc700 1 -- 172.21.15.115:0/1205432127 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f14b40e2100 con 0x7f14c4107df0 2024-05-09T05:42:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.549+0000 7f14d9f20b80 1 -- start start 2024-05-09T05:42:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 0x55e8c28072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2684520 0x55e8c2807800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2801480 0x55e8c2801890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e8c2724120 con 0x55e8c2684520 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e8c26c9a20 con 0x55e8c2801480 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8c27200e0 con 0x55e8c26fed80 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c8aa5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2684520 0x55e8c2807800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c8aa5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2684520 0x55e8c2807800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45038/0 (socket says 172.21.15.115:45038) 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c8aa5700 1 -- 172.21.15.115:0/1349473081 learned_addr learned my addr 172.21.15.115:0/1349473081 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c92a6700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f14b00b29f0 0x7f14b00b4f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14c0005fd0 tx=0x7f14c0005dc0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c92a6700 1 --2- 172.21.15.115:0/1349473081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 0x55e8c28072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/1349473081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2801480 0x55e8c2801890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c8aa5700 1 -- 172.21.15.115:0/1349473081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 msgr2=0x55e8c28072c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/1349473081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 0x55e8c28072c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c8aa5700 1 -- 172.21.15.115:0/1349473081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2801480 msgr2=0x55e8c2801890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/1349473081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2801480 0x55e8c2801890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c8aa5700 1 -- 172.21.15.115:0/1349473081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c26ba580 con 0x55e8c2684520 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/1349473081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2684520 0x55e8c2807800 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f14b40a5230 tx=0x7f14b40a8db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d7a72268e055fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.550+0000 7f14af7fe700 1 -- 172.21.15.115:0/1349473081 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b40c3070 con 0x55e8c2684520 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.551+0000 7f14af7fe700 1 -- 172.21.15.115:0/1349473081 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14b40b2010 con 0x55e8c2684520 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.551+0000 7f14af7fe700 1 -- 172.21.15.115:0/1349473081 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b40b1b80 con 0x55e8c2684520 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.551+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1349473081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2684520 msgr2=0x55e8c2807800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.551+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1349473081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2684520 0x55e8c2807800 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f14b40a5230 tx=0x7f14b40a8db0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.551+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1349473081 shutdown_connections 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.551+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1349473081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2801480 0x55e8c2801890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.551+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1349473081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 0x55e8c28072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.551+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1349473081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2684520 0x55e8c2807800 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.551+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1349473081 >> 172.21.15.115:0/1349473081 conn(0x55e8c2687ff0 msgr2=0x55e8c27fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.552+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1349473081 shutdown_connections 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.552+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1349473081 wait complete. 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.552+0000 7f14d9f20b80 1 -- start start 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.552+0000 7f14d9f20b80 1 Processor -- start 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.552+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 0x55e8c280b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.552+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 0x55e8c280b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c280bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e8c28021e0 con 0x55e8c2801480 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e8c26beba0 con 0x55e8c2684520 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e8c280e0a0 con 0x55e8c26fed80 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c9aa7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c280bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c8aa5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 0x55e8c280b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c9aa7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c280bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45040/0 (socket says 172.21.15.115:45040) 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c9aa7700 1 -- 172.21.15.115:0/3498030194 learned_addr learned my addr 172.21.15.115:0/3498030194 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14d9f20b80 10 client.0 _ll_register_callbacks cb 0x55e8c2595740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c92a6700 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 0x55e8c280b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c9aa7700 1 -- 172.21.15.115:0/3498030194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 msgr2=0x55e8c280b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 0x55e8c280b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c9aa7700 1 -- 172.21.15.115:0/3498030194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 msgr2=0x55e8c280b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 0x55e8c280b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c9aa7700 1 -- 172.21.15.115:0/3498030194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c259f510 con 0x55e8c2801480 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c280bbe0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f14bc004cd0 tx=0x7f14bc0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602ba3ba21672ca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14acff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14acff9700 1 -- 172.21.15.115:0/3498030194 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc020070 con 0x55e8c2801480 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14acff9700 1 -- 172.21.15.115:0/3498030194 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f14bc0077c0 con 0x55e8c2801480 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.553+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3498030194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b40b2010 con 0x55e8c2801480 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14acff9700 1 -- 172.21.15.115:0/3498030194 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f14bc0079c0 con 0x55e8c2801480 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14d9f20b80 10 client.16761 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3498030194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e8c2818b10 con 0x55e8c2801480 2024-05-09T05:42:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14acff9700 1 -- 172.21.15.115:0/3498030194 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14b40b2010 con 0x55e8c2801480 2024-05-09T05:42:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14acff9700 1 -- 172.21.15.115:0/3498030194 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f14bc011070 con 0x55e8c2801480 2024-05-09T05:42:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14d9f20b80 10 client.16761 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14d9f20b80 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14d9f20b80 20 client.16761 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e8c281b150 0x55e8c281d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14d9f20b80 4 client.16761 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3498030194 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8c2731b00 con 0x55e8c281b150 2024-05-09T05:42:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.554+0000 7f14c92a6700 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e8c281b150 0x55e8c281d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.555+0000 7f14acff9700 10 client.16761 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.555+0000 7f14c92a6700 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e8c281b150 0x55e8c281d570 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c749be3657a9fa84 server_cookie=f560b9c86d0b1b4a in_seq=0 out_seq=0 2024-05-09T05:42:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.555+0000 7f14acff9700 1 -- 172.21.15.115:0/3498030194 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8c2731b00 con 0x55e8c281b150 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14acff9700 10 client.16761 handle_command_reply: tid=0 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 client.16761 shutdown 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3498030194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 msgr2=0x55e8c280bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c280bbe0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f14bc004cd0 tx=0x7f14bc0095d0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3498030194 shutdown_connections 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 0x55e8c280b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e8c281b150 0x55e8c281d570 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c280bbe0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3498030194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c26fed80 0x55e8c280b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3498030194 >> 172.21.15.115:0/3498030194 conn(0x55e8c2687ff0 msgr2=0x55e8c27f6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14acff9700 0 client.16761 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3498030194 shutdown_connections 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3498030194 wait complete. 2024-05-09T05:42:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.556+0000 7f14d9f20b80 20 client.16761 trim_cache size 0 max 16384 2024-05-09T05:42:27.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.576+0000 7f14d9f20b80 1 Processor -- start 2024-05-09T05:42:27.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.576+0000 7f14d9f20b80 1 -- start start 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 0x55e8c27f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c26fed80 0x55e8c2808d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2801480 0x55e8c280a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14c41b25d0 con 0x55e8c26fed80 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14c41b4200 con 0x55e8c2684520 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14c41b2ca0 con 0x55e8c2801480 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c8aa5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c26fed80 0x55e8c2808d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c8aa5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c26fed80 0x55e8c2808d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45042/0 (socket says 172.21.15.115:45042) 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c8aa5700 1 -- 172.21.15.115:0/3164673740 learned_addr learned my addr 172.21.15.115:0/3164673740 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c92a6700 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 0x55e8c27f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2801480 0x55e8c280a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c8aa5700 1 -- 172.21.15.115:0/3164673740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2801480 msgr2=0x55e8c280a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2801480 0x55e8c280a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c8aa5700 1 -- 172.21.15.115:0/3164673740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 msgr2=0x55e8c27f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 0x55e8c27f6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c8aa5700 1 -- 172.21.15.115:0/3164673740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c280da00 con 0x55e8c26fed80 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c92a6700 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 0x55e8c27f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c26fed80 0x55e8c2808d70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f14b4010660 tx=0x7f14b4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aee50cffc3e8bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14aeffd700 1 -- 172.21.15.115:0/3164673740 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b40c3040 con 0x55e8c26fed80 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14aeffd700 1 -- 172.21.15.115:0/3164673740 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14b40ad170 con 0x55e8c26fed80 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.577+0000 7f14aeffd700 1 -- 172.21.15.115:0/3164673740 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b40a96a0 con 0x55e8c26fed80 2024-05-09T05:42:27.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.578+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3164673740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c26fed80 msgr2=0x55e8c2808d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.578+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c26fed80 0x55e8c2808d70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f14b4010660 tx=0x7f14b4004860 comp rx=0 tx=0).stop 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.578+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3164673740 shutdown_connections 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.578+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c2684520 0x55e8c27f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.578+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2801480 0x55e8c280a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.578+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/3164673740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c26fed80 0x55e8c2808d70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.578+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3164673740 >> 172.21.15.115:0/3164673740 conn(0x55e8c2687ff0 msgr2=0x55e8c280b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.578+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3164673740 shutdown_connections 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.578+0000 7f14d9f20b80 1 -- 172.21.15.115:0/3164673740 wait complete. 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14d9f20b80 1 -- start start 2024-05-09T05:42:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14d9f20b80 1 Processor -- start 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2684520 0x55e8c28101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c26fed80 0x55e8c2810710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14d9f20b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c2810c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14c41b3ad0 con 0x55e8c2801480 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14c41b3370 con 0x55e8c26fed80 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14d9f20b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14c41b0570 con 0x55e8c2684520 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14d9f20b80 10 client.0 _ll_register_callbacks cb 0x55e8c2595740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14c9aa7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c2810c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14c9aa7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c2810c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45052/0 (socket says 172.21.15.115:45052) 2024-05-09T05:42:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14c9aa7700 1 -- 172.21.15.115:0/1333117016 learned_addr learned my addr 172.21.15.115:0/1333117016 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.579+0000 7f14c92a6700 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2684520 0x55e8c28101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c26fed80 0x55e8c2810710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14c9aa7700 1 -- 172.21.15.115:0/1333117016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2684520 msgr2=0x55e8c28101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2684520 0x55e8c28101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14c9aa7700 1 -- 172.21.15.115:0/1333117016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c26fed80 msgr2=0x55e8c2810710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c26fed80 0x55e8c2810710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14c9aa7700 1 -- 172.21.15.115:0/1333117016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c2885930 con 0x55e8c2801480 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14c8aa5700 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c26fed80 0x55e8c2810710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14c92a6700 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2684520 0x55e8c28101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14c9aa7700 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c2810c50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f14bc025c90 tx=0x7f14bc005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9500227d23b25383 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14acff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14acff9700 1 -- 172.21.15.115:0/1333117016 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc037060 con 0x55e8c2801480 2024-05-09T05:42:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14acff9700 1 -- 172.21.15.115:0/1333117016 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f14bc025670 con 0x55e8c2801480 2024-05-09T05:42:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1333117016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b40ad170 con 0x55e8c2801480 2024-05-09T05:42:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14acff9700 1 -- 172.21.15.115:0/1333117016 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f14bc020040 con 0x55e8c2801480 2024-05-09T05:42:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14d9f20b80 10 client.16767 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1333117016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e8c2886af0 con 0x55e8c2801480 2024-05-09T05:42:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.580+0000 7f14acff9700 1 -- 172.21.15.115:0/1333117016 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14b40ad170 con 0x55e8c2801480 2024-05-09T05:42:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14acff9700 1 -- 172.21.15.115:0/1333117016 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f14bc011070 con 0x55e8c2801480 2024-05-09T05:42:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14d9f20b80 10 client.16767 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14d9f20b80 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14d9f20b80 20 client.16767 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e8c28871d0 0x55e8c28895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14d9f20b80 4 client.16767 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1333117016 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e8c272bba0 con 0x55e8c28871d0 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14c92a6700 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e8c28871d0 0x55e8c28895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14acff9700 10 client.16767 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.581+0000 7f14c92a6700 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e8c28871d0 0x55e8c28895f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac142b360ff3dd4f server_cookie=b194e22a84c8a82 in_seq=0 out_seq=0 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.582+0000 7f14acff9700 1 -- 172.21.15.115:0/1333117016 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e8c272bba0 con 0x55e8c28871d0 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.582+0000 7f14acff9700 10 client.16767 handle_command_reply: tid=0 2024-05-09T05:42:27.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.582+0000 7f14d9f20b80 1 client.16767 shutdown 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1333117016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 msgr2=0x55e8c2810c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c2810c50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f14bc025c90 tx=0x7f14bc005c60 comp rx=0 tx=0).stop 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1333117016 shutdown_connections 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e8c28871d0 0x55e8c28895f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8c2801480 0x55e8c2810c50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c2684520 0x55e8c28101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 --2- 172.21.15.115:0/1333117016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8c26fed80 0x55e8c2810710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14acff9700 0 client.16767 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1333117016 >> 172.21.15.115:0/1333117016 conn(0x55e8c2687ff0 msgr2=0x55e8c27f6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1333117016 shutdown_connections 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 1 -- 172.21.15.115:0/1333117016 wait complete. 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.583+0000 7f14d9f20b80 20 client.16767 trim_cache size 0 max 16384 2024-05-09T05:42:27.583 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:42:27.584 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:42:27.584 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "022d2f7d-dec7-4d53-896d-6c34f539c9ef", 2024-05-09T05:42:27.584 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:42:27.584 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 -- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f14b00b29f0 msgr2=0x7f14b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f14b00b29f0 0x7f14b00b4f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14c0005fd0 tx=0x7f14c0005dc0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 -- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4107df0 msgr2=0x7f14c419d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4107df0 0x7f14c419d4a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f14b400b500 tx=0x7f14b400b8c0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 -- 172.21.15.115:0/1205432127 shutdown_connections 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f14b00b29f0 0x7f14b00b4f00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14c4108740 0x7f14c41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14c41034c0 0x7f14c419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 --2- 172.21.15.115:0/1205432127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14c4107df0 0x7f14c419d4a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 -- 172.21.15.115:0/1205432127 >> 172.21.15.115:0/1205432127 conn(0x7f14c40fedc0 msgr2=0x7f14c410ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 -- 172.21.15.115:0/1205432127 shutdown_connections 2024-05-09T05:42:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:27.584+0000 7f14cb50a700 1 -- 172.21.15.115:0/1205432127 wait complete. 2024-05-09T05:42:27.588 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:27 smithi195 ceph-mon[33844]: pgmap v130: 129 pgs: 129 active+clean; 2.3 GiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 17 op/s 2024-05-09T05:42:27.589 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:27 smithi115 ceph-mon[30349]: pgmap v130: 129 pgs: 129 active+clean; 2.3 GiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 17 op/s 2024-05-09T05:42:27.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:27 smithi161 ceph-mon[33805]: pgmap v130: 129 pgs: 129 active+clean; 2.3 GiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 17 op/s 2024-05-09T05:42:27.865 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:42:27 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T05:42:27.596+0000 7f9d69520700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c7 [...5,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ auth{1=1} v3945 ap=4 f(v36 m2024-05-09T05:40:55.664108+0000 18=18+0) n(v1 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:140 nref=3)) caps={15339=pAsLsXs/-@178} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55d6cd1f4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c7:[<0x10000000007/.deps v2123>,<0x10000000006/ffsb v3080>,<0x10000000005/tmp v44>,<0x10000000003/client.0 v40>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v56>,<0x10000000001/sv_0 v71>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c7:[<0x10000000007/.deps v3945>,<0x10000000006/ffsb v3517>,<0x10000000005/tmp v95>,<0x10000000003/client.0 v88>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v108>,<0x10000000001/sv_0 v126>,<0x10000000000/qa v116>,<0x1/volumes v107>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T05:40:55.643149+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1)","memory_value.dirstat":"f(v36 m2024-05-09T05:40:55.664108+0000 18=18+0)","memory_value.rstat":"n(v1 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T05:42:28.482 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:42:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:28 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:42:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:28 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:28 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:42:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:28 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:42:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:28 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:28 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:42:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:28 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:28 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:28 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:42:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:28 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:42:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:28 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:28 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:42:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:28 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:42:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:28 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:28 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:42:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:28 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:28 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:28 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:42:28.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:28 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:42:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:28 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:28 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:42:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:28 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:42:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:28 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:28 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:42:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:28 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:28 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:28.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:28 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:42:28.776 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:42:29.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:42:29.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:42:29.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:42:29.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: pgmap v131: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 46 op/s 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:42:29.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:29 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:42:29.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:42:29.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:42:29.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:42:29.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:42:29.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:42:29.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:42:29.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: pgmap v131: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 46 op/s 2024-05-09T05:42:29.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:42:29.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:42:29.629 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:29 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:42:29.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:42:29.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:42:29.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:42:29.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: pgmap v131: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 19 MiB/s wr, 46 op/s 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:42:29.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:29 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:42:29.973 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:42:30.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.646+0000 7f439414f700 1 -- 172.21.15.115:0/2706915087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c1034e0 msgr2=0x7f438c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:30.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.646+0000 7f439414f700 1 --2- 172.21.15.115:0/2706915087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c1034e0 0x7f438c1038f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f43880099c0 tx=0x7f4388009cd0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.646+0000 7f439414f700 1 -- 172.21.15.115:0/2706915087 shutdown_connections 2024-05-09T05:42:30.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.646+0000 7f439414f700 1 --2- 172.21.15.115:0/2706915087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f438c108760 0x7f438c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.646+0000 7f439414f700 1 --2- 172.21.15.115:0/2706915087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c1034e0 0x7f438c1038f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.646+0000 7f439414f700 1 --2- 172.21.15.115:0/2706915087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f438c107e10 0x7f438c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.646+0000 7f439414f700 1 -- 172.21.15.115:0/2706915087 >> 172.21.15.115:0/2706915087 conn(0x7f438c0fef50 msgr2=0x7f438c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:30.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.646+0000 7f439414f700 1 -- 172.21.15.115:0/2706915087 shutdown_connections 2024-05-09T05:42:30.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.647+0000 7f439414f700 1 -- 172.21.15.115:0/2706915087 wait complete. 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f439414f700 1 Processor -- start 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f439414f700 1 -- start start 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f439414f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f438c1034e0 0x7f438c19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f439414f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f438c107e10 0x7f438c19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f439414f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c108760 0x7f438c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f439414f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f438c113240 con 0x7f438c108760 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f439414f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1130c0 con 0x7f438c107e10 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f439414f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1133c0 con 0x7f438c1034e0 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f43926ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c108760 0x7f438c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f43926ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c108760 0x7f438c19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45066/0 (socket says 172.21.15.115:45066) 2024-05-09T05:42:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f43926ec700 1 -- 172.21.15.115:0/1036460075 learned_addr learned my addr 172.21.15.115:0/1036460075 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f4391eeb700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f438c1034e0 0x7f438c19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.648+0000 7f43916ea700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f438c107e10 0x7f438c19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f43926ec700 1 -- 172.21.15.115:0/1036460075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f438c1034e0 msgr2=0x7f438c19bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f43926ec700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f438c1034e0 0x7f438c19bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f43926ec700 1 -- 172.21.15.115:0/1036460075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f438c107e10 msgr2=0x7f438c19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f43926ec700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f438c107e10 0x7f438c19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f43926ec700 1 -- 172.21.15.115:0/1036460075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438c1a6da0 con 0x7f438c108760 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f43916ea700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f438c107e10 0x7f438c19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f43926ec700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c108760 0x7f438c19c580 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4384007d70 tx=0x7f438400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f4382ffd700 1 -- 172.21.15.115:0/1036460075 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384016020 con 0x7f438c108760 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f4382ffd700 1 -- 172.21.15.115:0/1036460075 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f438400ad40 con 0x7f438c108760 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f4382ffd700 1 -- 172.21.15.115:0/1036460075 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43840115e0 con 0x7f438c108760 2024-05-09T05:42:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f439414f700 1 -- 172.21.15.115:0/1036460075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438c1a7090 con 0x7f438c108760 2024-05-09T05:42:30.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.649+0000 7f439414f700 1 -- 172.21.15.115:0/1036460075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438c1a75d0 con 0x7f438c108760 2024-05-09T05:42:30.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.653+0000 7f4382ffd700 1 -- 172.21.15.115:0/1036460075 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4384019020 con 0x7f438c108760 2024-05-09T05:42:30.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.654+0000 7f43a2b65b80 1 Processor -- start 2024-05-09T05:42:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.654+0000 7f43a2b65b80 1 -- start start 2024-05-09T05:42:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.654+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.654+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c466494a0 0x556c467d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.654+0000 7f4382ffd700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43780b2a70 0x7f43780b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.654+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 0x556c467caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556c466e8ff0 con 0x556c466494a0 2024-05-09T05:42:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556c466877f0 con 0x556c467caab0 2024-05-09T05:42:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556c466e4fb0 con 0x556c46733160 2024-05-09T05:42:30.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4391eeb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c466494a0 0x556c467d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4391eeb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c466494a0 0x556c467d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45068/0 (socket says 172.21.15.115:45068) 2024-05-09T05:42:30.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4391eeb700 1 -- 172.21.15.115:0/3317131808 learned_addr learned my addr 172.21.15.115:0/3317131808 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f43916ea700 1 --2- 172.21.15.115:0/3317131808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f43916ea700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43780b2a70 0x7f43780b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4382ffd700 1 -- 172.21.15.115:0/1036460075 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f43840e2700 con 0x7f438c108760 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f43926ec700 1 --2- 172.21.15.115:0/3317131808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 0x556c467caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4391eeb700 1 -- 172.21.15.115:0/3317131808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 msgr2=0x556c467d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4391eeb700 1 --2- 172.21.15.115:0/3317131808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4391eeb700 1 -- 172.21.15.115:0/3317131808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 msgr2=0x556c467caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4391eeb700 1 --2- 172.21.15.115:0/3317131808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 0x556c467caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4391eeb700 1 -- 172.21.15.115:0/3317131808 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4667f370 con 0x556c466494a0 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4391eeb700 1 --2- 172.21.15.115:0/3317131808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c466494a0 0x556c467d0df0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f438800b580 tx=0x7f4388005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37bbbc592de1311 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f43916ea700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43780b2a70 0x7f43780b4f80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f437c005ea0 tx=0x7f437c005e30 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.655+0000 7f4377fff700 1 -- 172.21.15.115:0/3317131808 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388027070 con 0x556c466494a0 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.656+0000 7f4377fff700 1 -- 172.21.15.115:0/3317131808 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4388005680 con 0x556c466494a0 2024-05-09T05:42:30.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.656+0000 7f4377fff700 1 -- 172.21.15.115:0/3317131808 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388003ba0 con 0x556c466494a0 2024-05-09T05:42:30.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.656+0000 7f43a2b65b80 1 -- 172.21.15.115:0/3317131808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c466494a0 msgr2=0x556c467d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:30.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.656+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/3317131808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c466494a0 0x556c467d0df0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f438800b580 tx=0x7f4388005290 comp rx=0 tx=0).stop 2024-05-09T05:42:30.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.656+0000 7f43a2b65b80 1 -- 172.21.15.115:0/3317131808 shutdown_connections 2024-05-09T05:42:30.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.656+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/3317131808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 0x556c467caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.656+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/3317131808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.656+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/3317131808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c466494a0 0x556c467d0df0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.656+0000 7f43a2b65b80 1 -- 172.21.15.115:0/3317131808 >> 172.21.15.115:0/3317131808 conn(0x556c466a96a0 msgr2=0x556c467c1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.657+0000 7f43a2b65b80 1 -- 172.21.15.115:0/3317131808 shutdown_connections 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.657+0000 7f43a2b65b80 1 -- 172.21.15.115:0/3317131808 wait complete. 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.657+0000 7f43a2b65b80 1 -- start start 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.657+0000 7f43a2b65b80 1 Processor -- start 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 0x556c467d4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467d4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556c46564ba0 con 0x556c467caab0 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556c466a3150 con 0x556c466494a0 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556c467cb890 con 0x556c46733160 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43a2b65b80 10 client.0 _ll_register_callbacks cb 0x556c4655a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43926ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f4391eeb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 0x556c467d4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43926ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45072/0 (socket says 172.21.15.115:45072) 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43926ec700 1 -- 172.21.15.115:0/1097175529 learned_addr learned my addr 172.21.15.115:0/1097175529 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43916ea700 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467d4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43926ec700 1 -- 172.21.15.115:0/1097175529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 msgr2=0x556c467d4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43926ec700 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467d4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43926ec700 1 -- 172.21.15.115:0/1097175529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 msgr2=0x556c467d4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43926ec700 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 0x556c467d4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43926ec700 1 -- 172.21.15.115:0/1097175529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c46564490 con 0x556c467caab0 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43926ec700 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d50a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4384007d40 tx=0x7f43840a9cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff13eedeef03013e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:30.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.658+0000 7f43757fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43757fa700 1 -- 172.21.15.115:0/1097175529 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43840a9100 con 0x556c467caab0 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43757fa700 1 -- 172.21.15.115:0/1097175529 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f43840910d0 con 0x556c467caab0 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43a2b65b80 1 -- 172.21.15.115:0/1097175529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4388005680 con 0x556c467caab0 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43757fa700 1 -- 172.21.15.115:0/1097175529 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f43840c40a0 con 0x556c467caab0 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43a2b65b80 10 client.16779 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43a2b65b80 1 -- 172.21.15.115:0/1097175529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556c467e1f60 con 0x556c467caab0 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43757fa700 1 -- 172.21.15.115:0/1097175529 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4388005680 con 0x556c467caab0 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43757fa700 1 -- 172.21.15.115:0/1097175529 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f43840aaae0 con 0x556c467caab0 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43a2b65b80 10 client.16779 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43a2b65b80 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43a2b65b80 20 client.16779 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c467e4430 0x556c467e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43a2b65b80 4 client.16779 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:30.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.659+0000 7f43a2b65b80 1 -- 172.21.15.115:0/1097175529 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c466f69d0 con 0x556c467e4430 2024-05-09T05:42:30.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.660+0000 7f4391eeb700 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c467e4430 0x556c467e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.660+0000 7f43757fa700 10 client.16779 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:30.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.660+0000 7f4391eeb700 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c467e4430 0x556c467e68e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1763a5b6c09ae58b server_cookie=ec5aa768b5e2a248 in_seq=0 out_seq=0 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.661+0000 7f43757fa700 1 -- 172.21.15.115:0/1097175529 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c466f69d0 con 0x556c467e4430 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.661+0000 7f43757fa700 10 client.16779 handle_command_reply: tid=0 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.661+0000 7f43a2b65b80 1 client.16779 shutdown 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 -- 172.21.15.115:0/1097175529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 msgr2=0x556c467d50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d50a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4384007d40 tx=0x7f43840a9cf0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 -- 172.21.15.115:0/1097175529 shutdown_connections 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 0x556c467d4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c467e4430 0x556c467e68e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d50a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/1097175529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467d4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 -- 172.21.15.115:0/1097175529 >> 172.21.15.115:0/1097175529 conn(0x556c466a96a0 msgr2=0x556c467c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43757fa700 0 client.16779 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 -- 172.21.15.115:0/1097175529 shutdown_connections 2024-05-09T05:42:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 1 -- 172.21.15.115:0/1097175529 wait complete. 2024-05-09T05:42:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.662+0000 7f43a2b65b80 20 client.16779 trim_cache size 0 max 16384 2024-05-09T05:42:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.685+0000 7f43a2b65b80 1 Processor -- start 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43a2b65b80 1 -- start start 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 0x556c467ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1adf50 con 0x556c467caab0 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1afb80 con 0x556c466494a0 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1ae620 con 0x556c46733160 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43926ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43926ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45084/0 (socket says 172.21.15.115:45084) 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43926ec700 1 -- 172.21.15.115:0/2779065300 learned_addr learned my addr 172.21.15.115:0/2779065300 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f4391eeb700 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 0x556c467ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.686+0000 7f43916ea700 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43926ec700 1 -- 172.21.15.115:0/2779065300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 msgr2=0x556c467c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43926ec700 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43926ec700 1 -- 172.21.15.115:0/2779065300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 msgr2=0x556c467ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43926ec700 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 0x556c467ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43926ec700 1 -- 172.21.15.115:0/2779065300 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c467cfe50 con 0x556c467caab0 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43916ea700 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43926ec700 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d4ee0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f43840a94b0 tx=0x7f4384006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbd61498d887281 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43777fe700 1 -- 172.21.15.115:0/2779065300 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43840c4280 con 0x556c467caab0 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43777fe700 1 -- 172.21.15.115:0/2779065300 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43840cc0f0 con 0x556c467caab0 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.687+0000 7f43777fe700 1 -- 172.21.15.115:0/2779065300 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43840a8f20 con 0x556c467caab0 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2779065300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 msgr2=0x556c467d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d4ee0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f43840a94b0 tx=0x7f4384006040 comp rx=0 tx=0).stop 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2779065300 shutdown_connections 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c466494a0 0x556c467ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c467caab0 0x556c467d4ee0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2779065300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c46733160 0x556c467c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2779065300 >> 172.21.15.115:0/2779065300 conn(0x556c466a96a0 msgr2=0x556c467d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2779065300 shutdown_connections 2024-05-09T05:42:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2779065300 wait complete. 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.688+0000 7f43a2b65b80 1 -- start start 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43a2b65b80 1 Processor -- start 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c466494a0 0x556c467dca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c46733160 0x556c467eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43a2b65b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 0x556c467eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1af450 con 0x556c46733160 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1aecf0 con 0x556c467caab0 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43a2b65b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f438c1abef0 con 0x556c466494a0 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43a2b65b80 10 client.0 _ll_register_callbacks cb 0x556c467d03a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43916ea700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c46733160 0x556c467eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43916ea700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c46733160 0x556c467eb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45100/0 (socket says 172.21.15.115:45100) 2024-05-09T05:42:30.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43916ea700 1 -- 172.21.15.115:0/2401529725 learned_addr learned my addr 172.21.15.115:0/2401529725 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f4391eeb700 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c466494a0 0x556c467dca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.689+0000 7f43926ec700 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 0x556c467eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43916ea700 1 -- 172.21.15.115:0/2401529725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c466494a0 msgr2=0x556c467dca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43916ea700 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c466494a0 0x556c467dca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43916ea700 1 -- 172.21.15.115:0/2401529725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 msgr2=0x556c467eba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43916ea700 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 0x556c467eba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43916ea700 1 -- 172.21.15.115:0/2401529725 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c467ebf60 con 0x556c46733160 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43916ea700 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c46733160 0x556c467eb4e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f438c106630 tx=0x7f437c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b35d244c85b6c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43757fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43757fa700 1 -- 172.21.15.115:0/2401529725 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c010ac0 con 0x556c46733160 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43757fa700 1 -- 172.21.15.115:0/2401529725 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f437c010c60 con 0x556c46733160 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2401529725 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c466f0a70 con 0x556c46733160 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43757fa700 1 -- 172.21.15.115:0/2401529725 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f437c035050 con 0x556c46733160 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43a2b65b80 10 client.16785 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.690+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2401529725 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556c467dc420 con 0x556c46733160 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.691+0000 7f43757fa700 1 -- 172.21.15.115:0/2401529725 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556c466f0a70 con 0x556c46733160 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.691+0000 7f43757fa700 1 -- 172.21.15.115:0/2401529725 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f437c021410 con 0x556c46733160 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.691+0000 7f43a2b65b80 10 client.16785 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.691+0000 7f43a2b65b80 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.691+0000 7f43a2b65b80 20 client.16785 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.691+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c4684f130 0x556c46851550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.691+0000 7f43a2b65b80 4 client.16785 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:42:30.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.691+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2401529725 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c46564970 con 0x556c4684f130 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.691+0000 7f4391eeb700 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c4684f130 0x556c46851550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.692+0000 7f4391eeb700 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c4684f130 0x556c46851550 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec92f2a614a8dac9 server_cookie=238efa96f1901ac6 in_seq=0 out_seq=0 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.692+0000 7f43757fa700 10 client.16785 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.692+0000 7f43757fa700 1 -- 172.21.15.115:0/2401529725 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556c46564970 con 0x556c4684f130 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.692+0000 7f43757fa700 10 client.16785 handle_command_reply: tid=0 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.692+0000 7f43a2b65b80 1 client.16785 shutdown 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.692+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2401529725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c46733160 msgr2=0x556c467eb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.692+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c46733160 0x556c467eb4e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f438c106630 tx=0x7f437c0047c0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2401529725 shutdown_connections 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c467caab0 0x556c467eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c4684f130 0x556c46851550 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c466494a0 0x556c467dca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43a2b65b80 1 --2- 172.21.15.115:0/2401529725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c46733160 0x556c467eb4e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43757fa700 0 client.16785 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2401529725 >> 172.21.15.115:0/2401529725 conn(0x556c466a96a0 msgr2=0x556c467c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2401529725 shutdown_connections 2024-05-09T05:42:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43a2b65b80 1 -- 172.21.15.115:0/2401529725 wait complete. 2024-05-09T05:42:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.693+0000 7f43a2b65b80 20 client.16785 trim_cache size 0 max 16384 2024-05-09T05:42:30.692 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:42:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T05:42:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": { 2024-05-09T05:42:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "022d2f7d-dec7-4d53-896d-6c34f539c9ef": { 2024-05-09T05:42:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/", 2024-05-09T05:42:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "tag": "022d2f7d-dec7-4d53-896d-6c34f539c9ef", 2024-05-09T05:42:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "options": "recursive,force" 2024-05-09T05:42:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:30.693 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 -- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43780b2a70 msgr2=0x7f43780b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43780b2a70 0x7f43780b4f80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f437c005ea0 tx=0x7f437c005e30 comp rx=0 tx=0).stop 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 -- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c108760 msgr2=0x7f438c19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c108760 0x7f438c19c580 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4384007d70 tx=0x7f438400a340 comp rx=0 tx=0).stop 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 -- 172.21.15.115:0/1036460075 shutdown_connections 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f43780b2a70 0x7f43780b4f80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f438c108760 0x7f438c19c580 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f438c1034e0 0x7f438c19bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 --2- 172.21.15.115:0/1036460075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f438c107e10 0x7f438c19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 -- 172.21.15.115:0/1036460075 >> 172.21.15.115:0/1036460075 conn(0x7f438c0fef50 msgr2=0x7f438c107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 -- 172.21.15.115:0/1036460075 shutdown_connections 2024-05-09T05:42:30.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:30.694+0000 7f439414f700 1 -- 172.21.15.115:0/1036460075 wait complete. 2024-05-09T05:42:30.695 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:42:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:31 smithi195 ceph-mon[33844]: pgmap v132: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 37 op/s 2024-05-09T05:42:31.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:31 smithi115 ceph-mon[30349]: pgmap v132: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 37 op/s 2024-05-09T05:42:31.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:31 smithi161 ceph-mon[33805]: pgmap v132: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 37 op/s 2024-05-09T05:42:31.753 INFO:tasks.cephfs.filesystem:scrub status for tag:022d2f7d-dec7-4d53-896d-6c34f539c9ef - {'path': '/', 'tag': '022d2f7d-dec7-4d53-896d-6c34f539c9ef', 'options': 'recursive,force'} 2024-05-09T05:42:33.528 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:33 smithi115 ceph-mon[30349]: pgmap v133: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 37 op/s 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.534+0000 7f69f519f700 1 -- 172.21.15.115:0/1178621034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0108740 msgr2=0x7f69f01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.534+0000 7f69f519f700 1 --2- 172.21.15.115:0/1178621034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0108740 0x7f69f01100a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f69e800b0d0 tx=0x7f69e800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.534+0000 7f69f519f700 1 -- 172.21.15.115:0/1178621034 shutdown_connections 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.534+0000 7f69f519f700 1 --2- 172.21.15.115:0/1178621034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0108740 0x7f69f01100a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.534+0000 7f69f519f700 1 --2- 172.21.15.115:0/1178621034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f01034c0 0x7f69f01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.534+0000 7f69f519f700 1 --2- 172.21.15.115:0/1178621034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f0107df0 0x7f69f0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.534+0000 7f69f519f700 1 -- 172.21.15.115:0/1178621034 >> 172.21.15.115:0/1178621034 conn(0x7f69f00fedf0 msgr2=0x7f69f0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.534+0000 7f69f519f700 1 -- 172.21.15.115:0/1178621034 shutdown_connections 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.534+0000 7f69f519f700 1 -- 172.21.15.115:0/1178621034 wait complete. 2024-05-09T05:42:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.535+0000 7f69f519f700 1 Processor -- start 2024-05-09T05:42:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.535+0000 7f69f519f700 1 -- start start 2024-05-09T05:42:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69f519f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f01034c0 0x7f69f019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69f519f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0107df0 0x7f69f019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69f519f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f0108740 0x7f69f0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69f519f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01132a0 con 0x7f69f0107df0 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69f519f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0113120 con 0x7f69f01034c0 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69f519f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0113420 con 0x7f69f0108740 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ee59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0107df0 0x7f69f019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ee59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0107df0 0x7f69f019fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42414/0 (socket says 172.21.15.115:42414) 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ee59c700 1 -- 172.21.15.115:0/2831551738 learned_addr learned my addr 172.21.15.115:0/2831551738 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69eed9d700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f01034c0 0x7f69f019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ef59e700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f0108740 0x7f69f0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ee59c700 1 -- 172.21.15.115:0/2831551738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f0108740 msgr2=0x7f69f0199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ee59c700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f0108740 0x7f69f0199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ee59c700 1 -- 172.21.15.115:0/2831551738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f01034c0 msgr2=0x7f69f019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ee59c700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f01034c0 0x7f69f019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ee59c700 1 -- 172.21.15.115:0/2831551738 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f019a150 con 0x7f69f0107df0 2024-05-09T05:42:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69ee59c700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0107df0 0x7f69f019fdc0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f69e40098f0 tx=0x7f69e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.536+0000 7f69d7fff700 1 -- 172.21.15.115:0/2831551738 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e400d020 con 0x7f69f0107df0 2024-05-09T05:42:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.537+0000 7f69d7fff700 1 -- 172.21.15.115:0/2831551738 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69e400d1c0 con 0x7f69f0107df0 2024-05-09T05:42:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.537+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f019a440 con 0x7f69f0107df0 2024-05-09T05:42:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.537+0000 7f69d7fff700 1 -- 172.21.15.115:0/2831551738 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e400c7e0 con 0x7f69f0107df0 2024-05-09T05:42:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.537+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f00759f0 con 0x7f69f0107df0 2024-05-09T05:42:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.538+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f0104970 con 0x7f69f0107df0 2024-05-09T05:42:33.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.539+0000 7f69d7fff700 1 -- 172.21.15.115:0/2831551738 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f69e40074f0 con 0x7f69f0107df0 2024-05-09T05:42:33.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.540+0000 7f69d7fff700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69d80b29f0 0x7f69d80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:33.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.540+0000 7f69eed9d700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69d80b29f0 0x7f69d80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:33.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.540+0000 7f69d7fff700 1 -- 172.21.15.115:0/2831551738 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f69e40e4170 con 0x7f69f0107df0 2024-05-09T05:42:33.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.540+0000 7f69eed9d700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69d80b29f0 0x7f69d80b4f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69f019abe0 tx=0x7f69e0009190 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.543+0000 7f69d7fff700 1 -- 172.21.15.115:0/2831551738 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69e40ab810 con 0x7f69f0107df0 2024-05-09T05:42:33.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:33 smithi195 ceph-mon[33844]: pgmap v133: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 37 op/s 2024-05-09T05:42:33.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:33 smithi161 ceph-mon[33805]: pgmap v133: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 37 op/s 2024-05-09T05:42:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.794+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69f00512c0 con 0x7f69f0107df0 2024-05-09T05:42:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.795+0000 7f69d7fff700 1 -- 172.21.15.115:0/2831551738 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11961 (secure 0 0 0) 0x7f69e40ab110 con 0x7f69f0107df0 2024-05-09T05:42:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:42:33.789 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:41:33.359460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:42:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69d80b29f0 msgr2=0x7f69d80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69d80b29f0 0x7f69d80b4f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69f019abe0 tx=0x7f69e0009190 comp rx=0 tx=0).stop 2024-05-09T05:42:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0107df0 msgr2=0x7f69f019fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0107df0 0x7f69f019fdc0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f69e40098f0 tx=0x7f69e4009c00 comp rx=0 tx=0).stop 2024-05-09T05:42:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 shutdown_connections 2024-05-09T05:42:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69d80b29f0 0x7f69d80b4f00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f01034c0 0x7f69f019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f0108740 0x7f69f0199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 --2- 172.21.15.115:0/2831551738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69f0107df0 0x7f69f019fdc0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.797+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 >> 172.21.15.115:0/2831551738 conn(0x7f69f00fedf0 msgr2=0x7f69f0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.798+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 shutdown_connections 2024-05-09T05:42:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:33.798+0000 7f69f519f700 1 -- 172.21.15.115:0/2831551738 wait complete. 2024-05-09T05:42:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 34 2024-05-09T05:42:34.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:34 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2831551738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:42:34.579 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id b011d006 --release --await' 2024-05-09T05:42:34.579 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id b011d006 --release --await 2024-05-09T05:42:34.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:34 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2831551738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:42:34.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:34 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2831551738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:42:34.823 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:42:35.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:35 smithi195 ceph-mon[33844]: pgmap v134: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.7 MiB/s wr, 42 op/s 2024-05-09T05:42:35.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:35 smithi115 ceph-mon[30349]: pgmap v134: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.7 MiB/s wr, 42 op/s 2024-05-09T05:42:35.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:35 smithi161 ceph-mon[33805]: pgmap v134: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.7 MiB/s wr, 42 op/s 2024-05-09T05:42:36.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.600+0000 7f1c024fa700 1 -- 172.21.15.115:0/368849210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bfc1030f0 msgr2=0x7f1bfc111180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.600+0000 7f1c024fa700 1 --2- 172.21.15.115:0/368849210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bfc1030f0 0x7f1bfc111180 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400b0a0 tx=0x7f1bf400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.600+0000 7f1c024fa700 1 -- 172.21.15.115:0/368849210 shutdown_connections 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.600+0000 7f1c024fa700 1 --2- 172.21.15.115:0/368849210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bfc1030f0 0x7f1bfc111180 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.600+0000 7f1c024fa700 1 --2- 172.21.15.115:0/368849210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bfc109310 0x7f1bfc109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.600+0000 7f1c024fa700 1 --2- 172.21.15.115:0/368849210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc1027a0 0x7f1bfc102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.600+0000 7f1c024fa700 1 -- 172.21.15.115:0/368849210 >> 172.21.15.115:0/368849210 conn(0x7f1bfc0fe4d0 msgr2=0x7f1bfc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.600+0000 7f1c024fa700 1 -- 172.21.15.115:0/368849210 shutdown_connections 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.600+0000 7f1c024fa700 1 -- 172.21.15.115:0/368849210 wait complete. 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1c024fa700 1 Processor -- start 2024-05-09T05:42:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1c024fa700 1 -- start start 2024-05-09T05:42:36.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1c024fa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bfc1027a0 0x7f1bfc103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1c024fa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc1030f0 0x7f1bfc103ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1c024fa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bfc109310 0x7f1bfc104400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1c024fa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc1144d0 con 0x7f1bfc109310 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1c024fa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc114350 con 0x7f1bfc1030f0 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1c024fa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc114650 con 0x7f1bfc1027a0 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1bfbfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bfc1027a0 0x7f1bfc103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.601+0000 7f1c00a97700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bfc109310 0x7f1bfc104400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1bfbfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bfc1027a0 0x7f1bfc103980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:49764/0 (socket says 172.21.15.115:49764) 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1bfbfff700 1 -- 172.21.15.115:0/3023287931 learned_addr learned my addr 172.21.15.115:0/3023287931 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc1030f0 0x7f1bfc103ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1c00a97700 1 -- 172.21.15.115:0/3023287931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bfc1027a0 msgr2=0x7f1bfc103980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1c00a97700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bfc1027a0 0x7f1bfc103980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1c00a97700 1 -- 172.21.15.115:0/3023287931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc1030f0 msgr2=0x7f1bfc103ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1c00a97700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc1030f0 0x7f1bfc103ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1c00a97700 1 -- 172.21.15.115:0/3023287931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bfc1a91a0 con 0x7f1bfc109310 2024-05-09T05:42:36.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1c00a97700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bfc109310 0x7f1bfc104400 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400b0a0 tx=0x7f1bf4004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1bf97fa700 1 -- 172.21.15.115:0/3023287931 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4021070 con 0x7f1bfc109310 2024-05-09T05:42:36.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1bf97fa700 1 -- 172.21.15.115:0/3023287931 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bf4007b80 con 0x7f1bfc109310 2024-05-09T05:42:36.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1bf97fa700 1 -- 172.21.15.115:0/3023287931 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4005040 con 0x7f1bfc109310 2024-05-09T05:42:36.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1c024fa700 1 -- 172.21.15.115:0/3023287931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bfc1a9430 con 0x7f1bfc109310 2024-05-09T05:42:36.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.602+0000 7f1c024fa700 1 -- 172.21.15.115:0/3023287931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bfc1a98e0 con 0x7f1bfc109310 2024-05-09T05:42:36.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.605+0000 7f1bf97fa700 1 -- 172.21.15.115:0/3023287931 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1bf4024070 con 0x7f1bfc109310 2024-05-09T05:42:36.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.605+0000 7f1bf97fa700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1be40b2a70 0x7f1be40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1be40b2a70 0x7f1be40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1bf97fa700 1 -- 172.21.15.115:0/3023287931 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1bf400f020 con 0x7f1bfc109310 2024-05-09T05:42:36.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1c10f10b80 1 Processor -- start 2024-05-09T05:42:36.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1c10f10b80 1 -- start start 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1be40b2a70 0x7f1be40b4f80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1bec005d90 tx=0x7f1bec00b480 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d171a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 0x55816d171fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d16bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55816d08e0f0 con 0x55816d0d8260 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55816d02c8f0 con 0x55816d16bb50 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55816d08a0b0 con 0x55816cfee5a0 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1bfbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d171a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1bfbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d171a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42440/0 (socket says 172.21.15.115:42440) 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.606+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2616129040 learned_addr learned my addr 172.21.15.115:0/2616129040 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1c00a97700 1 --2- 172.21.15.115:0/2616129040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d16bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/2616129040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 0x55816d171fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2616129040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 msgr2=0x55816d171fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/2616129040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 0x55816d171fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2616129040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 msgr2=0x55816d16bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/2616129040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d16bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2616129040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816d024470 con 0x55816d0d8260 2024-05-09T05:42:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/2616129040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d171a90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1bec010f20 tx=0x7f1bec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1302ba3be416de7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1be27fc700 1 -- 172.21.15.115:0/2616129040 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec011db0 con 0x55816d0d8260 2024-05-09T05:42:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1be27fc700 1 -- 172.21.15.115:0/2616129040 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bec01e8b0 con 0x55816d0d8260 2024-05-09T05:42:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.607+0000 7f1be27fc700 1 -- 172.21.15.115:0/2616129040 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec01c730 con 0x55816d0d8260 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.608+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2616129040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 msgr2=0x55816d171a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.608+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2616129040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d171a90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1bec010f20 tx=0x7f1bec013000 comp rx=0 tx=0).stop 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.608+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2616129040 shutdown_connections 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.608+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2616129040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d16bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.608+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2616129040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d171a90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.608+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2616129040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 0x55816d171fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.608+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2616129040 >> 172.21.15.115:0/2616129040 conn(0x55816d04e7a0 msgr2=0x55816d166a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.608+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2616129040 shutdown_connections 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.608+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2616129040 wait complete. 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c10f10b80 1 -- start start 2024-05-09T05:42:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c10f10b80 1 Processor -- start 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 0x55816d175750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d175c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d1761d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55816cf09ca0 con 0x55816d0d8260 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55816d048250 con 0x55816d16bb50 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55816d181930 con 0x55816cfee5a0 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c10f10b80 10 client.0 _ll_register_callbacks cb 0x55816ceff7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1bfb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d175c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1bfb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d175c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42452/0 (socket says 172.21.15.115:42452) 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1bfb7fe700 1 -- 172.21.15.115:0/1897829476 learned_addr learned my addr 172.21.15.115:0/1897829476 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 0x55816d175750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.609+0000 7f1c00a97700 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d1761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bfb7fe700 1 -- 172.21.15.115:0/1897829476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 msgr2=0x55816d175750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 0x55816d175750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bfb7fe700 1 -- 172.21.15.115:0/1897829476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 msgr2=0x55816d1761d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d1761d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bfb7fe700 1 -- 172.21.15.115:0/1897829476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55816cf09590 con 0x55816d0d8260 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d175c90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000bee0 tx=0x7f1bf000f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85089d705d102660 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bdb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/1897829476 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0018020 con 0x55816d0d8260 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/1897829476 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1bf0014460 con 0x55816d0d8260 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1c10f10b80 1 -- 172.21.15.115:0/1897829476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bec01e8b0 con 0x55816d0d8260 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/1897829476 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1bf0023050 con 0x55816d0d8260 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1c10f10b80 10 client.16803 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1c10f10b80 1 -- 172.21.15.115:0/1897829476 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55816d1832c0 con 0x55816d0d8260 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/1897829476 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bec01e8b0 con 0x55816d0d8260 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.610+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/1897829476 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f1bf00136f0 con 0x55816d0d8260 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.611+0000 7f1c10f10b80 10 client.16803 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.611+0000 7f1c10f10b80 10 client.16803 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:42:36.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.611+0000 7f1c10f10b80 20 client.16803 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.611+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55816d1859a0 0x55816d187df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.611+0000 7f1c10f10b80 4 client.16803 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.611+0000 7f1c10f10b80 1 -- 172.21.15.115:0/1897829476 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55816d09bad0 con 0x55816d1859a0 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.611+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55816d1859a0 0x55816d187df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.611+0000 7f1bdb7fe700 10 client.16803 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.611+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55816d1859a0 0x55816d187df0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc39276ee7b6a578 server_cookie=41baf6c9ac55bcc4 in_seq=0 out_seq=0 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.612+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/1897829476 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55816d09bad0 con 0x55816d1859a0 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.612+0000 7f1bdb7fe700 10 client.16803 handle_command_reply: tid=0 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.612+0000 7f1c10f10b80 1 client.16803 shutdown 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.612+0000 7f1c10f10b80 1 -- 172.21.15.115:0/1897829476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 msgr2=0x55816d175c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.612+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d175c90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000bee0 tx=0x7f1bf000f4c0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1c10f10b80 1 -- 172.21.15.115:0/1897829476 shutdown_connections 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d1761d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55816d1859a0 0x55816d187df0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1bdb7fe700 0 client.16803 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:36.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816cfee5a0 0x55816d175750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/1897829476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816d0d8260 0x55816d175c90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1c10f10b80 1 -- 172.21.15.115:0/1897829476 >> 172.21.15.115:0/1897829476 conn(0x55816d04e7a0 msgr2=0x55816d166750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1c10f10b80 1 -- 172.21.15.115:0/1897829476 shutdown_connections 2024-05-09T05:42:36.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1c10f10b80 1 -- 172.21.15.115:0/1897829476 wait complete. 2024-05-09T05:42:36.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.613+0000 7f1c10f10b80 20 client.16803 trim_cache size 0 max 16384 2024-05-09T05:42:36.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1c10f10b80 1 Processor -- start 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1c10f10b80 1 -- start start 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d16b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d0d8260 0x55816d175cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d16bb50 0x55816d18d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc1b25a0 con 0x55816cfee5a0 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc1b41d0 con 0x55816d0d8260 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc1b2c70 con 0x55816d16bb50 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d16b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d16b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42466/0 (socket says 172.21.15.115:42466) 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2745630958 learned_addr learned my addr 172.21.15.115:0/2745630958 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1c00a97700 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d16bb50 0x55816d18d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d0d8260 0x55816d175cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2745630958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d16bb50 msgr2=0x55816d18d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d16bb50 0x55816d18d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2745630958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d0d8260 msgr2=0x55816d175cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d0d8260 0x55816d175cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2745630958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816d166c60 con 0x55816cfee5a0 2024-05-09T05:42:36.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.633+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d0d8260 0x55816d175cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:36.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.634+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d16b250 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1bec010f20 tx=0x7f1bec010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fe65882b8260e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.634+0000 7f1be1ffb700 1 -- 172.21.15.115:0/2745630958 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec036030 con 0x55816cfee5a0 2024-05-09T05:42:36.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.634+0000 7f1be1ffb700 1 -- 172.21.15.115:0/2745630958 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bec0044b0 con 0x55816cfee5a0 2024-05-09T05:42:36.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.634+0000 7f1be1ffb700 1 -- 172.21.15.115:0/2745630958 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec032680 con 0x55816cfee5a0 2024-05-09T05:42:36.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2745630958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 msgr2=0x55816d16b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d16b250 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1bec010f20 tx=0x7f1bec010f50 comp rx=0 tx=0).stop 2024-05-09T05:42:36.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2745630958 shutdown_connections 2024-05-09T05:42:36.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d16bb50 0x55816d18d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d16b250 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2745630958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d0d8260 0x55816d175cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2745630958 >> 172.21.15.115:0/2745630958 conn(0x55816d04e7a0 msgr2=0x55816d16ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2745630958 shutdown_connections 2024-05-09T05:42:36.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2745630958 wait complete. 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 -- start start 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.635+0000 7f1c10f10b80 1 Processor -- start 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d17a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d0d8260 0x55816d17ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1c10f10b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d17b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc1b3aa0 con 0x55816cfee5a0 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc1b3340 con 0x55816d16bb50 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1c10f10b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc1b0540 con 0x55816d0d8260 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1c10f10b80 10 client.0 _ll_register_callbacks cb 0x55816ceff7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d17a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d17a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42470/0 (socket says 172.21.15.115:42470) 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2441768581 learned_addr learned my addr 172.21.15.115:0/2441768581 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d0d8260 0x55816d17ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1c00a97700 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d17b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2441768581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d0d8260 msgr2=0x55816d17ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d0d8260 0x55816d17ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2441768581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 msgr2=0x55816d17b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d17b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfbfff700 1 -- 172.21.15.115:0/2441768581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55816d1f0340 con 0x55816cfee5a0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bfbfff700 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d17a860 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1bec011010 tx=0x7f1bec00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacc810225245f06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.636+0000 7f1bdb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/2441768581 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec005300 con 0x55816cfee5a0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/2441768581 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1bec032420 con 0x55816cfee5a0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2441768581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bec0044b0 con 0x55816cfee5a0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/2441768581 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1bec032620 con 0x55816cfee5a0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1c10f10b80 10 client.16809 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2441768581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55816d1f1530 con 0x55816cfee5a0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/2441768581 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bec0044b0 con 0x55816cfee5a0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/2441768581 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f1bec045470 con 0x55816cfee5a0 2024-05-09T05:42:36.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1c10f10b80 10 client.16809 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:36.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1c10f10b80 10 client.16809 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:42:36.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1c10f10b80 20 client.16809 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:36.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55816d1859a0 0x55816d1f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1c10f10b80 4 client.16809 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b011d006", "release": true, "await": true}] 2024-05-09T05:42:36.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.637+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2441768581 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b011d006", "release": true, "await": true}) -- 0x55816d095b70 con 0x55816d1859a0 2024-05-09T05:42:36.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.638+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55816d1859a0 0x55816d1f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.638+0000 7f1bfb7fe700 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55816d1859a0 0x55816d1f3a20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a55f73d7a7b36d server_cookie=26ef7a77cce9111 in_seq=0 out_seq=0 2024-05-09T05:42:36.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.638+0000 7f1bdb7fe700 10 client.16809 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.648+0000 7f1bdb7fe700 1 -- 172.21.15.115:0/2441768581 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55816d095b70 con 0x55816d1859a0 2024-05-09T05:42:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.648+0000 7f1bdb7fe700 10 client.16809 handle_command_reply: tid=0 2024-05-09T05:42:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.648+0000 7f1c10f10b80 1 client.16809 shutdown 2024-05-09T05:42:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2441768581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 msgr2=0x55816d17a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d17a860 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1bec011010 tx=0x7f1bec00be60 comp rx=0 tx=0).stop 2024-05-09T05:42:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2441768581 shutdown_connections 2024-05-09T05:42:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55816d16bb50 0x55816d17b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55816d1859a0 0x55816d1f3a20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55816cfee5a0 0x55816d17a860 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 --2- 172.21.15.115:0/2441768581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55816d0d8260 0x55816d17ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1bdb7fe700 0 client.16809 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2441768581 >> 172.21.15.115:0/2441768581 conn(0x55816d04e7a0 msgr2=0x55816d16fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2441768581 shutdown_connections 2024-05-09T05:42:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 1 -- 172.21.15.115:0/2441768581 wait complete. 2024-05-09T05:42:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.649+0000 7f1c10f10b80 20 client.16809 trim_cache size 0 max 16384 2024-05-09T05:42:36.644 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 34, 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 12, 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "b011d006": { 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 12, 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:42:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 139.4, 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:42:36.646 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:42:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 -- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1be40b2a70 msgr2=0x7f1be40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1be40b2a70 0x7f1be40b4f80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1bec005d90 tx=0x7f1bec00b480 comp rx=0 tx=0).stop 2024-05-09T05:42:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 -- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bfc109310 msgr2=0x7f1bfc104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bfc109310 0x7f1bfc104400 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400b0a0 tx=0x7f1bf4004ce0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 -- 172.21.15.115:0/3023287931 shutdown_connections 2024-05-09T05:42:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1be40b2a70 0x7f1be40b4f80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bfc109310 0x7f1bfc104400 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bfc1027a0 0x7f1bfc103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 --2- 172.21.15.115:0/3023287931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc1030f0 0x7f1bfc103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 -- 172.21.15.115:0/3023287931 >> 172.21.15.115:0/3023287931 conn(0x7f1bfc0fe4d0 msgr2=0x7f1bfc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 -- 172.21.15.115:0/3023287931 shutdown_connections 2024-05-09T05:42:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:42:36.650+0000 7f1c024fa700 1 -- 172.21.15.115:0/3023287931 wait complete. 2024-05-09T05:42:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.835+0000 7ff79efa8640 1 -- 172.21.15.115:0/2640666690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 msgr2=0x7ff798102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.835+0000 7ff79efa8640 1 --2- 172.21.15.115:0/2640666690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 0x7ff798102750 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c02a410 comp rx=0 tx=0).stop 2024-05-09T05:42:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.835+0000 7ff79efa8640 1 -- 172.21.15.115:0/2640666690 shutdown_connections 2024-05-09T05:42:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.835+0000 7ff79efa8640 1 --2- 172.21.15.115:0/2640666690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff798103300 0x7ff79810f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.835+0000 7ff79efa8640 1 --2- 172.21.15.115:0/2640666690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff798109050 0x7ff798102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.835+0000 7ff79efa8640 1 --2- 172.21.15.115:0/2640666690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 0x7ff798102750 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.835+0000 7ff79efa8640 1 -- 172.21.15.115:0/2640666690 >> 172.21.15.115:0/2640666690 conn(0x7ff798078280 msgr2=0x7ff7980786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.835+0000 7ff79efa8640 1 -- 172.21.15.115:0/2640666690 shutdown_connections 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.835+0000 7ff79efa8640 1 -- 172.21.15.115:0/2640666690 wait complete. 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff79efa8640 1 Processor -- start 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff79efa8640 1 -- start start 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff79efa8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff798103300 0x7ff7981a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff79efa8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 0x7ff7981a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff79efa8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff798109050 0x7ff79819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff79efa8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff798112ca0 con 0x7ff798108bb0 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff79efa8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff798112b20 con 0x7ff798103300 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff79efa8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff798112e20 con 0x7ff798109050 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff797fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 0x7ff7981a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff797fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 0x7ff7981a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42496/0 (socket says 172.21.15.115:42496) 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff797fff640 1 -- 172.21.15.115:0/1708287562 learned_addr learned my addr 172.21.15.115:0/1708287562 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.836+0000 7ff79cd44640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff798103300 0x7ff7981a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff79d545640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff798109050 0x7ff79819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff797fff640 1 -- 172.21.15.115:0/1708287562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff798109050 msgr2=0x7ff79819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff797fff640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff798109050 0x7ff79819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff797fff640 1 -- 172.21.15.115:0/1708287562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff798103300 msgr2=0x7ff7981a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff797fff640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff798103300 0x7ff7981a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff797fff640 1 -- 172.21.15.115:0/1708287562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79819ac90 con 0x7ff798108bb0 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff79d545640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff798109050 0x7ff79819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff797fff640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 0x7ff7981a0a10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff784005fd0 tx=0x7ff78400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff795ffb640 1 -- 172.21.15.115:0/1708287562 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784017070 con 0x7ff798108bb0 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff795ffb640 1 -- 172.21.15.115:0/1708287562 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff784009c30 con 0x7ff798108bb0 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff79efa8640 1 -- 172.21.15.115:0/1708287562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79819af80 con 0x7ff798108bb0 2024-05-09T05:42:36.833 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff795ffb640 1 -- 172.21.15.115:0/1708287562 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784003c90 con 0x7ff798108bb0 2024-05-09T05:42:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.837+0000 7ff79efa8640 1 -- 172.21.15.115:0/1708287562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7980ff360 con 0x7ff798108bb0 2024-05-09T05:42:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.840+0000 7ff7a5e59740 1 Processor -- start 2024-05-09T05:42:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.840+0000 7ff795ffb640 1 -- 172.21.15.115:0/1708287562 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff784003e30 con 0x7ff798108bb0 2024-05-09T05:42:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.840+0000 7ff7a5e59740 1 -- start start 2024-05-09T05:42:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9991b0 0x564c4bd3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd3cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd3d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c4bca17c0 con 0x564c4b9991b0 2024-05-09T05:42:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564c4bc3d0e0 con 0x564c4b9702f0 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564c4bc9d780 con 0x564c4b9784d0 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79d545640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9991b0 0x564c4bd3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79d545640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9991b0 0x564c4bd3c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42506/0 (socket says 172.21.15.115:42506) 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79d545640 1 -- 172.21.15.115:0/44507163 learned_addr learned my addr 172.21.15.115:0/44507163 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff797fff640 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd3d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79cd44640 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd3cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff795ffb640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff7880b2950 0x7ff7880b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79cd44640 1 -- 172.21.15.115:0/44507163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 msgr2=0x564c4bd3d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79cd44640 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd3d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79cd44640 1 -- 172.21.15.115:0/44507163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9991b0 msgr2=0x564c4bd3c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79cd44640 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9991b0 0x564c4bd3c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79cd44640 1 -- 172.21.15.115:0/44507163 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4bc667c0 con 0x564c4b9702f0 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79d545640 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9991b0 0x564c4bd3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff79d545640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff7880b2950 0x7ff7880b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.841+0000 7ff795ffb640 1 -- 172.21.15.115:0/1708287562 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff7840e2570 con 0x7ff798108bb0 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.842+0000 7ff79cd44640 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd3cbc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009230 tx=0x7ff78c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6acfded6bf1fe34f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.842+0000 7ff79d545640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff7880b2950 0x7ff7880b4e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff79000efd0 tx=0x7ff790010040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.842+0000 7ff782ffd640 1 -- 172.21.15.115:0/44507163 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c040070 con 0x564c4b9702f0 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.842+0000 7ff782ffd640 1 -- 172.21.15.115:0/44507163 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff78c004490 con 0x564c4b9702f0 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.842+0000 7ff782ffd640 1 -- 172.21.15.115:0/44507163 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c035040 con 0x564c4b9702f0 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 -- 172.21.15.115:0/44507163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 msgr2=0x564c4bd3cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd3cbc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009230 tx=0x7ff78c005fd0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 -- 172.21.15.115:0/44507163 shutdown_connections 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd3d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd3cbc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/44507163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9991b0 0x564c4bd3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 -- 172.21.15.115:0/44507163 >> 172.21.15.115:0/44507163 conn(0x564c4b98d300 msgr2=0x564c4bd38e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 -- 172.21.15.115:0/44507163 shutdown_connections 2024-05-09T05:42:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 -- 172.21.15.115:0/44507163 wait complete. 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 -- start start 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.843+0000 7ff7a5e59740 1 Processor -- start 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9784d0 0x564c4bd48c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9991b0 0x564c4bd46bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c4bd49750 con 0x564c4b9784d0 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564c4bc289c0 con 0x564c4b9702f0 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564c4bd498d0 con 0x564c4b9991b0 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff7a5e59740 10 client.0 _ll_register_callbacks cb 0x564c4bc55020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff797fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9784d0 0x564c4bd48c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff797fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9784d0 0x564c4bd48c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42510/0 (socket says 172.21.15.115:42510) 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff797fff640 1 -- 172.21.15.115:0/3088412129 learned_addr learned my addr 172.21.15.115:0/3088412129 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff79cd44640 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff79d545640 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9991b0 0x564c4bd46bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff797fff640 1 -- 172.21.15.115:0/3088412129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9991b0 msgr2=0x564c4bd46bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff797fff640 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9991b0 0x564c4bd46bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff797fff640 1 -- 172.21.15.115:0/3088412129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 msgr2=0x564c4bd486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.844+0000 7ff797fff640 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff797fff640 1 -- 172.21.15.115:0/3088412129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4bc4f1f0 con 0x564c4b9784d0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff797fff640 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9784d0 0x564c4bd48c20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff784000c00 tx=0x7ff784005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9ae629bc8f7e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff777fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff777fff640 1 -- 172.21.15.115:0/3088412129 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7840a8c80 con 0x564c4b9784d0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff777fff640 1 -- 172.21.15.115:0/3088412129 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff7840a8980 con 0x564c4b9784d0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3088412129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78c004490 con 0x564c4b9784d0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff777fff640 1 -- 172.21.15.115:0/3088412129 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff7840035f0 con 0x564c4b9784d0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff7a5e59740 10 client.16821 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3088412129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564c4ba04790 con 0x564c4b9784d0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff777fff640 1 -- 172.21.15.115:0/3088412129 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff78c004490 con 0x564c4b9784d0 2024-05-09T05:42:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.845+0000 7ff777fff640 1 -- 172.21.15.115:0/3088412129 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7ff7840e21b0 con 0x564c4b9784d0 2024-05-09T05:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.846+0000 7ff7a5e59740 10 client.16821 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.846+0000 7ff7a5e59740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.846+0000 7ff7a5e59740 20 client.16821 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.846+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c4b957d80 0x564c4bd59000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.846+0000 7ff7a5e59740 4 client.16821 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.846+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3088412129 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c4bcaf3d0 con 0x564c4b957d80 2024-05-09T05:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.846+0000 7ff79cd44640 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c4b957d80 0x564c4bd59000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.846+0000 7ff777fff640 10 client.16821 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.846+0000 7ff79cd44640 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c4b957d80 0x564c4bd59000 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a41492b9f73b65 server_cookie=17d6c0d848c2e230 in_seq=0 out_seq=0 2024-05-09T05:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff777fff640 1 -- 172.21.15.115:0/3088412129 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c4bcaf3d0 con 0x564c4b957d80 2024-05-09T05:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff777fff640 10 client.16821 handle_command_reply: tid=0 2024-05-09T05:42:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 client.16821 shutdown 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3088412129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9784d0 msgr2=0x564c4bd48c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9784d0 0x564c4bd48c20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff784000c00 tx=0x7ff784005bf0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3088412129 shutdown_connections 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c4b957d80 0x564c4bd59000 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff777fff640 0 client.16821 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9991b0 0x564c4bd46bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9784d0 0x564c4bd48c20 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3088412129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3088412129 >> 172.21.15.115:0/3088412129 conn(0x564c4b98d300 msgr2=0x564c4bd37700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3088412129 shutdown_connections 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.847+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3088412129 wait complete. 2024-05-09T05:42:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.848+0000 7ff7a5e59740 20 client.16821 trim_cache size 0 max 16384 2024-05-09T05:42:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.864+0000 7ff7a5e59740 1 Processor -- start 2024-05-09T05:42:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.864+0000 7ff7a5e59740 1 -- start start 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b957d80 0x564c4bd37e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9702f0 0x564c4bd403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd40920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981aee30 con 0x564c4b9702f0 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b0a60 con 0x564c4b957d80 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981af500 con 0x564c4b9784d0 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff797fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9702f0 0x564c4bd403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff797fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9702f0 0x564c4bd403e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42512/0 (socket says 172.21.15.115:42512) 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff797fff640 1 -- 172.21.15.115:0/1422373752 learned_addr learned my addr 172.21.15.115:0/1422373752 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff79cd44640 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b957d80 0x564c4bd37e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff79d545640 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd40920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff797fff640 1 -- 172.21.15.115:0/1422373752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 msgr2=0x564c4bd40920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff797fff640 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd40920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff797fff640 1 -- 172.21.15.115:0/1422373752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b957d80 msgr2=0x564c4bd37e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff797fff640 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b957d80 0x564c4bd37e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff797fff640 1 -- 172.21.15.115:0/1422373752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4ba7b5f0 con 0x564c4b9702f0 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff79d545640 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd40920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff797fff640 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9702f0 0x564c4bd403e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff784005c60 tx=0x7ff7840dfa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4519041bc8f6b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff7827fc640 1 -- 172.21.15.115:0/1422373752 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7840c2070 con 0x564c4b9702f0 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff7827fc640 1 -- 172.21.15.115:0/1422373752 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7840910d0 con 0x564c4b9702f0 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.865+0000 7ff7827fc640 1 -- 172.21.15.115:0/1422373752 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7840b2370 con 0x564c4b9702f0 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.866+0000 7ff7a5e59740 1 -- 172.21.15.115:0/1422373752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9702f0 msgr2=0x564c4bd403e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.866+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9702f0 0x564c4bd403e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff784005c60 tx=0x7ff7840dfa30 comp rx=0 tx=0).stop 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.866+0000 7ff7a5e59740 1 -- 172.21.15.115:0/1422373752 shutdown_connections 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.866+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd40920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.866+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b9702f0 0x564c4bd403e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.866+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/1422373752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b957d80 0x564c4bd37e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.866+0000 7ff7a5e59740 1 -- 172.21.15.115:0/1422373752 >> 172.21.15.115:0/1422373752 conn(0x564c4b98d300 msgr2=0x564c4bd37700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 -- 172.21.15.115:0/1422373752 shutdown_connections 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 -- 172.21.15.115:0/1422373752 wait complete. 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 -- start start 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 Processor -- start 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b957d80 0x564c4bd4eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd4f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd4a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981b0330 con 0x564c4b957d80 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981afbd0 con 0x564c4b9702f0 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981ace10 con 0x564c4b9784d0 2024-05-09T05:42:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.867+0000 7ff7a5e59740 10 client.0 _ll_register_callbacks cb 0x7ff78400df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79cd44640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b957d80 0x564c4bd4eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79cd44640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b957d80 0x564c4bd4eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42528/0 (socket says 172.21.15.115:42528) 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79cd44640 1 -- 172.21.15.115:0/3496288835 learned_addr learned my addr 172.21.15.115:0/3496288835 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff797fff640 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd4f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79d545640 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd4a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79cd44640 1 -- 172.21.15.115:0/3496288835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 msgr2=0x564c4bd4a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79cd44640 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd4a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79cd44640 1 -- 172.21.15.115:0/3496288835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 msgr2=0x564c4bd4f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79cd44640 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd4f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79cd44640 1 -- 172.21.15.115:0/3496288835 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4bc65ed0 con 0x564c4b957d80 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79d545640 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd4a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff79cd44640 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b957d80 0x564c4bd4eb10 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d95ed2a05b7a76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff777fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff777fff640 1 -- 172.21.15.115:0/3496288835 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c054060 con 0x564c4b957d80 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff777fff640 1 -- 172.21.15.115:0/3496288835 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff78c04f7c0 con 0x564c4b957d80 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3496288835 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7840910d0 con 0x564c4b957d80 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff777fff640 1 -- 172.21.15.115:0/3496288835 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff78c05b050 con 0x564c4b957d80 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff7a5e59740 10 client.16827 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.868+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3496288835 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564c4bdc0b60 con 0x564c4b957d80 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.869+0000 7ff777fff640 1 -- 172.21.15.115:0/3496288835 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7840910d0 con 0x564c4b957d80 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.869+0000 7ff777fff640 1 -- 172.21.15.115:0/3496288835 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7ff78c0035f0 con 0x564c4b957d80 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.869+0000 7ff7a5e59740 10 client.16827 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.869+0000 7ff7a5e59740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.869+0000 7ff7a5e59740 20 client.16827 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.869+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c4b9991b0 0x564c4bdc2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.869+0000 7ff7a5e59740 4 client.16827 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:42:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.869+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3496288835 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c4bca9240 con 0x564c4b9991b0 2024-05-09T05:42:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.869+0000 7ff797fff640 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c4b9991b0 0x564c4bdc2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.870+0000 7ff777fff640 10 client.16827 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.870+0000 7ff797fff640 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c4b9991b0 0x564c4bdc2f20 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1623f57b4d038090 server_cookie=8bbd89c7e24d86b2 in_seq=0 out_seq=0 2024-05-09T05:42:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff777fff640 1 -- 172.21.15.115:0/3496288835 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564c4bca9240 con 0x564c4b9991b0 2024-05-09T05:42:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff777fff640 10 client.16827 handle_command_reply: tid=0 2024-05-09T05:42:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 client.16827 shutdown 2024-05-09T05:42:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3496288835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b957d80 msgr2=0x564c4bd4eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b957d80 0x564c4bd4eb10 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c004890 comp rx=0 tx=0).stop 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3496288835 shutdown_connections 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c4b9991b0 0x564c4bdc2f20 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c4b9784d0 0x564c4bd4a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff777fff640 0 client.16827 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c4b9702f0 0x564c4bd4f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 --2- 172.21.15.115:0/3496288835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c4b957d80 0x564c4bd4eb10 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3496288835 >> 172.21.15.115:0/3496288835 conn(0x564c4b98d300 msgr2=0x564c4bd450a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3496288835 shutdown_connections 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.879+0000 7ff7a5e59740 1 -- 172.21.15.115:0/3496288835 wait complete. 2024-05-09T05:42:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff7a5e59740 20 client.16827 trim_cache size 0 max 16384 2024-05-09T05:42:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:42:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:42:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 -- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff7880b2950 msgr2=0x7ff7880b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff7880b2950 0x7ff7880b4e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff79000efd0 tx=0x7ff790010040 comp rx=0 tx=0).stop 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 -- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 msgr2=0x7ff7981a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 0x7ff7981a0a10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff784005fd0 tx=0x7ff78400dde0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 -- 172.21.15.115:0/1708287562 shutdown_connections 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff7880b2950 0x7ff7880b4e60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff798109050 0x7ff79819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff798108bb0 0x7ff7981a0a10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 --2- 172.21.15.115:0/1708287562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff798103300 0x7ff7981a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 -- 172.21.15.115:0/1708287562 >> 172.21.15.115:0/1708287562 conn(0x7ff798078280 msgr2=0x7ff79806b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 -- 172.21.15.115:0/1708287562 shutdown_connections 2024-05-09T05:42:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:36.880+0000 7ff79efa8640 1 -- 172.21.15.115:0/1708287562 wait complete. 2024-05-09T05:42:37.450 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b011d006', total seconds elapsed: 26.8 2024-05-09T05:42:37.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:37 smithi195 ceph-mon[33844]: pgmap v135: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 379 KiB/s wr, 33 op/s 2024-05-09T05:42:37.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:37 smithi115 ceph-mon[30349]: pgmap v135: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 379 KiB/s wr, 33 op/s 2024-05-09T05:42:37.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:37 smithi161 ceph-mon[33805]: pgmap v135: 129 pgs: 129 active+clean; 2.3 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 379 KiB/s wr, 33 op/s 2024-05-09T05:42:39.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:39 smithi115 ceph-mon[30349]: pgmap v136: 129 pgs: 129 active+clean; 2.3 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.0 MiB/s wr, 41 op/s 2024-05-09T05:42:39.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:39 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:42:39.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:39 smithi161 ceph-mon[33805]: pgmap v136: 129 pgs: 129 active+clean; 2.3 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.0 MiB/s wr, 41 op/s 2024-05-09T05:42:39.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:39 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:42:39.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:39 smithi195 ceph-mon[33844]: pgmap v136: 129 pgs: 129 active+clean; 2.3 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.0 MiB/s wr, 41 op/s 2024-05-09T05:42:39.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:39 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:42:40.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:40 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:42:40.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:40 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:42:40.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:40 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:42:40.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:40 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:42:40.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:40 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:42:40.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:40 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:42:40.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:40 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:42:40.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:40 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:42:40.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:40 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:42:41.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:41 smithi115 ceph-mon[30349]: pgmap v137: 129 pgs: 129 active+clean; 2.3 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.6 MiB/s wr, 12 op/s 2024-05-09T05:42:41.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:41 smithi161 ceph-mon[33805]: pgmap v137: 129 pgs: 129 active+clean; 2.3 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.6 MiB/s wr, 12 op/s 2024-05-09T05:42:41.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:41 smithi195 ceph-mon[33844]: pgmap v137: 129 pgs: 129 active+clean; 2.3 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.6 MiB/s wr, 12 op/s 2024-05-09T05:42:43.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:43 smithi115 ceph-mon[30349]: pgmap v138: 129 pgs: 129 active+clean; 2.3 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.6 MiB/s wr, 12 op/s 2024-05-09T05:42:43.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:43 smithi161 ceph-mon[33805]: pgmap v138: 129 pgs: 129 active+clean; 2.3 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.6 MiB/s wr, 12 op/s 2024-05-09T05:42:43.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:43 smithi195 ceph-mon[33844]: pgmap v138: 129 pgs: 129 active+clean; 2.3 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.6 MiB/s wr, 12 op/s 2024-05-09T05:42:45.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:45 smithi115 ceph-mon[30349]: pgmap v139: 129 pgs: 129 active+clean; 2.5 GiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 20 op/s 2024-05-09T05:42:45.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:45 smithi161 ceph-mon[33805]: pgmap v139: 129 pgs: 129 active+clean; 2.5 GiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 20 op/s 2024-05-09T05:42:45.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:45 smithi195 ceph-mon[33844]: pgmap v139: 129 pgs: 129 active+clean; 2.5 GiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 20 op/s 2024-05-09T05:42:47.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:47 smithi115 ceph-mon[30349]: pgmap v140: 129 pgs: 129 active+clean; 2.5 GiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-05-09T05:42:47.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:47 smithi161 ceph-mon[33805]: pgmap v140: 129 pgs: 129 active+clean; 2.5 GiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-05-09T05:42:47.667 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:47 smithi195 ceph-mon[33844]: pgmap v140: 129 pgs: 129 active+clean; 2.5 GiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-05-09T05:42:49.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:49 smithi115 ceph-mon[30349]: pgmap v141: 129 pgs: 129 active+clean; 2.6 GiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 25 op/s 2024-05-09T05:42:49.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:49 smithi161 ceph-mon[33805]: pgmap v141: 129 pgs: 129 active+clean; 2.6 GiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 25 op/s 2024-05-09T05:42:49.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:49 smithi195 ceph-mon[33844]: pgmap v141: 129 pgs: 129 active+clean; 2.6 GiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 25 op/s 2024-05-09T05:42:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.993+0000 7f4bf57c7640 1 -- 172.21.15.115:0/370973614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 msgr2=0x7f4bf0102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.993+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/370973614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 0x7f4bf0102740 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009900 tx=0x7f4be002a410 comp rx=0 tx=0).stop 2024-05-09T05:42:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.993+0000 7f4bf57c7640 1 -- 172.21.15.115:0/370973614 shutdown_connections 2024-05-09T05:42:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.993+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/370973614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf01032f0 0x7f4bf010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.993+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/370973614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bf0109040 0x7f4bf0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.993+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/370973614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 0x7f4bf0102740 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.993+0000 7f4bf57c7640 1 -- 172.21.15.115:0/370973614 >> 172.21.15.115:0/370973614 conn(0x7f4bf00782e0 msgr2=0x7f4bf0078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.993+0000 7f4bf57c7640 1 -- 172.21.15.115:0/370973614 shutdown_connections 2024-05-09T05:42:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.993+0000 7f4bf57c7640 1 -- 172.21.15.115:0/370973614 wait complete. 2024-05-09T05:42:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bf57c7640 1 Processor -- start 2024-05-09T05:42:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bf57c7640 1 -- start start 2024-05-09T05:42:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bf57c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bf01032f0 0x7f4bf019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bf57c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 0x7f4bf019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bf57c7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf0109040 0x7f4bf019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bf57c7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0112c40 con 0x7f4bf0108ba0 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bf57c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0112ac0 con 0x7f4bf01032f0 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bf57c7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0112dc0 con 0x7f4bf0109040 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 0x7f4bf019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 0x7f4bf019bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44106/0 (socket says 172.21.15.115:44106) 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4bee7fc640 1 -- 172.21.15.115:0/1333834833 learned_addr learned my addr 172.21.15.115:0/1333834833 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.994+0000 7f4beeffd640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bf01032f0 0x7f4bf019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bef7fe640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf0109040 0x7f4bf019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bee7fc640 1 -- 172.21.15.115:0/1333834833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf0109040 msgr2=0x7f4bf019c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf0109040 0x7f4bf019c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bee7fc640 1 -- 172.21.15.115:0/1333834833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bf01032f0 msgr2=0x7f4bf019b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bf01032f0 0x7f4bf019b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bee7fc640 1 -- 172.21.15.115:0/1333834833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf01a6980 con 0x7f4bf0108ba0 2024-05-09T05:42:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 0x7f4bf019bb80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d740 tx=0x7f4be400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bd7fff640 1 -- 172.21.15.115:0/1333834833 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4017070 con 0x7f4bf0108ba0 2024-05-09T05:42:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bd7fff640 1 -- 172.21.15.115:0/1333834833 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4be400de50 con 0x7f4bf0108ba0 2024-05-09T05:42:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bf57c7640 1 -- 172.21.15.115:0/1333834833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf01a6c70 con 0x7f4bf0108ba0 2024-05-09T05:42:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bd7fff640 1 -- 172.21.15.115:0/1333834833 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4003bb0 con 0x7f4bf0108ba0 2024-05-09T05:42:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.995+0000 7f4bf57c7640 1 -- 172.21.15.115:0/1333834833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf01a71c0 con 0x7f4bf0108ba0 2024-05-09T05:42:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bfc675740 1 Processor -- start 2024-05-09T05:42:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bfc675740 1 -- start start 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd47a1b0 0x5565bd81d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd4512f0 0x5565bd81dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd81e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bd7fff640 1 -- 172.21.15.115:0/1333834833 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4be4003d50 con 0x7f4bf0108ba0 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bfc675740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5565bd7827c0 con 0x5565bd4594d0 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bfc675740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5565bd71e0e0 con 0x5565bd4512f0 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bfc675740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5565bd77e780 con 0x5565bd47a1b0 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd81e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd47a1b0 0x5565bd81d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.998+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd81e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44114/0 (socket says 172.21.15.115:44114) 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4beeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd4512f0 0x5565bd81dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd47a1b0 0x5565bd81d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:50772/0 (socket says 172.21.15.115:50772) 2024-05-09T05:42:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bee7fc640 1 -- 172.21.15.115:0/3716502734 learned_addr learned my addr 172.21.15.115:0/3716502734 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bef7fe640 1 -- 172.21.15.115:0/3716502734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd4512f0 msgr2=0x5565bd81dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bef7fe640 1 --2- 172.21.15.115:0/3716502734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd4512f0 0x5565bd81dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bef7fe640 1 -- 172.21.15.115:0/3716502734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 msgr2=0x5565bd81e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bef7fe640 1 --2- 172.21.15.115:0/3716502734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd81e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bef7fe640 1 -- 172.21.15.115:0/3716502734 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565bd7477c0 con 0x5565bd47a1b0 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/3716502734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd81e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bef7fe640 1 --2- 172.21.15.115:0/3716502734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd47a1b0 0x5565bd81d680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4bf019c620 tx=0x7f4be8010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54bc7f74f3bbb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bd57fa640 1 -- 172.21.15.115:0/3716502734 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be8016070 con 0x5565bd47a1b0 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bd57fa640 1 -- 172.21.15.115:0/3716502734 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4be8004d20 con 0x5565bd47a1b0 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bd57fa640 1 -- 172.21.15.115:0/3716502734 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be80126b0 con 0x5565bd47a1b0 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:49.999+0000 7f4bd7fff640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bd80b29d0 0x7f4bd80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4beeffd640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bd80b29d0 0x7f4bd80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4bd7fff640 1 -- 172.21.15.115:0/1333834833 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4be40e21a0 con 0x7f4bf0108ba0 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4bfc675740 1 -- 172.21.15.115:0/3716502734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd47a1b0 msgr2=0x5565bd81d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4beeffd640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bd80b29d0 0x7f4bd80b4ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4be0004770 tx=0x7f4be00058d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4bfc675740 1 --2- 172.21.15.115:0/3716502734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd47a1b0 0x5565bd81d680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4bf019c620 tx=0x7f4be8010740 comp rx=0 tx=0).stop 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4bfc675740 1 -- 172.21.15.115:0/3716502734 shutdown_connections 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4bfc675740 1 --2- 172.21.15.115:0/3716502734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd81e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4bfc675740 1 --2- 172.21.15.115:0/3716502734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd4512f0 0x5565bd81dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4bfc675740 1 --2- 172.21.15.115:0/3716502734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd47a1b0 0x5565bd81d680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4bfc675740 1 -- 172.21.15.115:0/3716502734 >> 172.21.15.115:0/3716502734 conn(0x5565bd46e300 msgr2=0x5565bd819e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.000+0000 7f4bfc675740 1 -- 172.21.15.115:0/3716502734 shutdown_connections 2024-05-09T05:42:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 1 -- 172.21.15.115:0/3716502734 wait complete. 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 1 -- start start 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 1 Processor -- start 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 0x5565bd829720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd829c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd47a1b0 0x5565bd827bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5565bd82a7b0 con 0x5565bd4594d0 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5565bd7099c0 con 0x5565bd47a1b0 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5565bd82a930 con 0x5565bd4512f0 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.001+0000 7f4bfc675740 10 client.0 _ll_register_callbacks cb 0x5565bd736020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd829c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd829c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44126/0 (socket says 172.21.15.115:44126) 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bee7fc640 1 -- 172.21.15.115:0/540266582 learned_addr learned my addr 172.21.15.115:0/540266582 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4beeffd640 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 0x5565bd829720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bef7fe640 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd47a1b0 0x5565bd827bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bee7fc640 1 -- 172.21.15.115:0/540266582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 msgr2=0x5565bd829720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 0x5565bd829720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bee7fc640 1 -- 172.21.15.115:0/540266582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd47a1b0 msgr2=0x5565bd827bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd47a1b0 0x5565bd827bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bee7fc640 1 -- 172.21.15.115:0/540266582 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565bd7301f0 con 0x5565bd4594d0 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd829c80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4be40a6460 tx=0x7f4be40dfab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11222254eb6fbecc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/540266582 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4020070 con 0x5565bd4594d0 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/540266582 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4be40a8450 con 0x5565bd4594d0 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bfc675740 1 -- 172.21.15.115:0/540266582 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be8004d20 con 0x5565bd4594d0 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/540266582 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4be40a8fc0 con 0x5565bd4594d0 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bfc675740 10 client.16839 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.002+0000 7f4bfc675740 1 -- 172.21.15.115:0/540266582 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5565bd4e5790 con 0x5565bd4594d0 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.003+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/540266582 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4be8004d20 con 0x5565bd4594d0 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.003+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/540266582 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f4be40ad140 con 0x5565bd4594d0 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.003+0000 7f4bfc675740 10 client.16839 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.003+0000 7f4bfc675740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.003+0000 7f4bfc675740 20 client.16839 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.003+0000 7f4bfc675740 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565bd438d80 0x5565bd83a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.003+0000 7f4bfc675740 4 client.16839 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.003+0000 7f4bfc675740 1 -- 172.21.15.115:0/540266582 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565bd7903d0 con 0x5565bd438d80 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.003+0000 7f4beeffd640 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565bd438d80 0x5565bd83a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.004+0000 7f4beeffd640 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565bd438d80 0x5565bd83a020 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6769a690f4fb96a server_cookie=6c26e9ee2cf61ffa in_seq=0 out_seq=0 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.004+0000 7f4bd2ff5640 10 client.16839 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.004+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/540266582 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5565bd7903d0 con 0x5565bd438d80 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.004+0000 7f4bd2ff5640 10 client.16839 handle_command_reply: tid=0 2024-05-09T05:42:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.004+0000 7f4bfc675740 1 client.16839 shutdown 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 -- 172.21.15.115:0/540266582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 msgr2=0x5565bd829c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd829c80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4be40a6460 tx=0x7f4be40dfab0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 -- 172.21.15.115:0/540266582 shutdown_connections 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd47a1b0 0x5565bd827bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd829c80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565bd438d80 0x5565bd83a020 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 --2- 172.21.15.115:0/540266582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 0x5565bd829720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 -- 172.21.15.115:0/540266582 >> 172.21.15.115:0/540266582 conn(0x5565bd46e300 msgr2=0x5565bd818700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bd2ff5640 0 client.16839 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 -- 172.21.15.115:0/540266582 shutdown_connections 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 1 -- 172.21.15.115:0/540266582 wait complete. 2024-05-09T05:42:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.005+0000 7f4bfc675740 20 client.16839 trim_cache size 0 max 16384 2024-05-09T05:42:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.021+0000 7f4bfc675740 1 Processor -- start 2024-05-09T05:42:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.021+0000 7f4bfc675740 1 -- start start 2024-05-09T05:42:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.021+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 0x5565bd8219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.021+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4512f0 0x5565bd83f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.021+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4594d0 0x5565bd83fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.021+0000 7f4bfc675740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01b1170 con 0x5565bd4512f0 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.021+0000 7f4bfc675740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01b2da0 con 0x5565bd46e300 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.021+0000 7f4bfc675740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01b1840 con 0x5565bd4594d0 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4512f0 0x5565bd83f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4512f0 0x5565bd83f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44128/0 (socket says 172.21.15.115:44128) 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bee7fc640 1 -- 172.21.15.115:0/2544434903 learned_addr learned my addr 172.21.15.115:0/2544434903 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bef7fe640 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4594d0 0x5565bd83fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4beeffd640 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 0x5565bd8219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bee7fc640 1 -- 172.21.15.115:0/2544434903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4594d0 msgr2=0x5565bd83fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4594d0 0x5565bd83fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bee7fc640 1 -- 172.21.15.115:0/2544434903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 msgr2=0x5565bd8219e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 0x5565bd8219e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bee7fc640 1 -- 172.21.15.115:0/2544434903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565bd55c5f0 con 0x5565bd4512f0 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bef7fe640 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4594d0 0x5565bd83fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4beeffd640 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 0x5565bd8219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:42:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4512f0 0x5565bd83f790 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4be40b2160 tx=0x7f4be4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa9042a240069b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bd4ff9640 1 -- 172.21.15.115:0/2544434903 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4020070 con 0x5565bd4512f0 2024-05-09T05:42:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bd4ff9640 1 -- 172.21.15.115:0/2544434903 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4be40e59e0 con 0x5565bd4512f0 2024-05-09T05:42:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.022+0000 7f4bd4ff9640 1 -- 172.21.15.115:0/2544434903 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be40a8960 con 0x5565bd4512f0 2024-05-09T05:42:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.023+0000 7f4bfc675740 1 -- 172.21.15.115:0/2544434903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4512f0 msgr2=0x5565bd83f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.023+0000 7f4bfc675740 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4512f0 0x5565bd83f790 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4be40b2160 tx=0x7f4be4009c30 comp rx=0 tx=0).stop 2024-05-09T05:42:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.023+0000 7f4bfc675740 1 -- 172.21.15.115:0/2544434903 shutdown_connections 2024-05-09T05:42:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.023+0000 7f4bfc675740 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4594d0 0x5565bd83fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.023+0000 7f4bfc675740 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4512f0 0x5565bd83f790 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.023+0000 7f4bfc675740 1 --2- 172.21.15.115:0/2544434903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 0x5565bd8219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.023+0000 7f4bfc675740 1 -- 172.21.15.115:0/2544434903 >> 172.21.15.115:0/2544434903 conn(0x5565bd823540 msgr2=0x5565bd8212d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.023+0000 7f4bfc675740 1 -- 172.21.15.115:0/2544434903 shutdown_connections 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.023+0000 7f4bfc675740 1 -- 172.21.15.115:0/2544434903 wait complete. 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bfc675740 1 -- start start 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bfc675740 1 Processor -- start 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 0x5565bd830260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd82d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bfc675740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 0x5565bd82dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bfc675740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01b2670 con 0x5565bd4594d0 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bfc675740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01b1f10 con 0x5565bd46e300 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bfc675740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01af150 con 0x5565bd4512f0 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bfc675740 10 client.0 _ll_register_callbacks cb 0x7f4be4005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd82d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bee7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd82d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44138/0 (socket says 172.21.15.115:44138) 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bee7fc640 1 -- 172.21.15.115:0/278206285 learned_addr learned my addr 172.21.15.115:0/278206285 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4beeffd640 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 0x5565bd830260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.024+0000 7f4bef7fe640 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 0x5565bd82dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bee7fc640 1 -- 172.21.15.115:0/278206285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 msgr2=0x5565bd830260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 0x5565bd830260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bee7fc640 1 -- 172.21.15.115:0/278206285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 msgr2=0x5565bd82dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 0x5565bd82dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bee7fc640 1 -- 172.21.15.115:0/278206285 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565bd746ed0 con 0x5565bd4594d0 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bee7fc640 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd82d620 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4be40ad6b0 tx=0x7f4be40a8680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b1ab3b4ffb5a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/278206285 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4020070 con 0x5565bd4594d0 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bfc675740 1 -- 172.21.15.115:0/278206285 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be40e59e0 con 0x5565bd4594d0 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/278206285 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4be40a4d70 con 0x5565bd4594d0 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/278206285 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4be40a8b40 con 0x5565bd4594d0 2024-05-09T05:42:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bfc675740 10 client.16845 fetch_fsmap learned FSMap version 34 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bfc675740 1 -- 172.21.15.115:0/278206285 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5565bd8a2100 con 0x5565bd4594d0 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/278206285 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4be40e59e0 con 0x5565bd4594d0 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.025+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/278206285 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f4be40ae680 con 0x5565bd4594d0 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.026+0000 7f4bfc675740 10 client.16845 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.026+0000 7f4bfc675740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.026+0000 7f4bfc675740 20 client.16845 populate_metadata read hostname 'smithi115' 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.026+0000 7f4bfc675740 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565bd47a1b0 0x5565bd8a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.026+0000 7f4bfc675740 4 client.16845 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.026+0000 7f4bfc675740 1 -- 172.21.15.115:0/278206285 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565bd78a240 con 0x5565bd47a1b0 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.026+0000 7f4beeffd640 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565bd47a1b0 0x5565bd8a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.026+0000 7f4bd2ff5640 10 client.16845 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.026+0000 7f4beeffd640 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565bd47a1b0 0x5565bd8a4570 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e32e47ec5a36a57 server_cookie=ecdc3ff12c6758c in_seq=0 out_seq=0 2024-05-09T05:42:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bd2ff5640 1 -- 172.21.15.115:0/278206285 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5565bd78a240 con 0x5565bd47a1b0 2024-05-09T05:42:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bd2ff5640 10 client.16845 handle_command_reply: tid=0 2024-05-09T05:42:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 client.16845 shutdown 2024-05-09T05:42:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 -- 172.21.15.115:0/278206285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 msgr2=0x5565bd82d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd82d620 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4be40ad6b0 tx=0x7f4be40a8680 comp rx=0 tx=0).stop 2024-05-09T05:42:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 -- 172.21.15.115:0/278206285 shutdown_connections 2024-05-09T05:42:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565bd46e300 0x5565bd82dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565bd4594d0 0x5565bd82d620 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565bd47a1b0 0x5565bd8a4570 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 --2- 172.21.15.115:0/278206285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565bd4512f0 0x5565bd830260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bd2ff5640 0 client.16845 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:42:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 -- 172.21.15.115:0/278206285 >> 172.21.15.115:0/278206285 conn(0x5565bd823540 msgr2=0x5565bd8212d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 -- 172.21.15.115:0/278206285 shutdown_connections 2024-05-09T05:42:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 1 -- 172.21.15.115:0/278206285 wait complete. 2024-05-09T05:42:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.035+0000 7f4bfc675740 20 client.16845 trim_cache size 0 max 16384 2024-05-09T05:42:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:42:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:42:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:42:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 -- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bd80b29d0 msgr2=0x7f4bd80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bd80b29d0 0x7f4bd80b4ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4be0004770 tx=0x7f4be00058d0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 -- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 msgr2=0x7f4bf019bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:42:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 0x7f4bf019bb80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d740 tx=0x7f4be400dc00 comp rx=0 tx=0).stop 2024-05-09T05:42:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 -- 172.21.15.115:0/1333834833 shutdown_connections 2024-05-09T05:42:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bd80b29d0 0x7f4bd80b4ee0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bf0109040 0x7f4bf019c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf0108ba0 0x7f4bf019bb80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 --2- 172.21.15.115:0/1333834833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bf01032f0 0x7f4bf019b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:42:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 -- 172.21.15.115:0/1333834833 >> 172.21.15.115:0/1333834833 conn(0x7f4bf00782e0 msgr2=0x7f4bf0100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:42:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 -- 172.21.15.115:0/1333834833 shutdown_connections 2024-05-09T05:42:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:42:50.036+0000 7f4bf57c7640 1 -- 172.21.15.115:0/1333834833 wait complete. 2024-05-09T05:42:51.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:51 smithi115 ceph-mon[30349]: pgmap v142: 129 pgs: 129 active+clean; 2.6 GiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-05-09T05:42:51.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:51 smithi161 ceph-mon[33805]: pgmap v142: 129 pgs: 129 active+clean; 2.6 GiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-05-09T05:42:51.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:51 smithi195 ceph-mon[33844]: pgmap v142: 129 pgs: 129 active+clean; 2.6 GiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-05-09T05:42:53.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:53 smithi115 ceph-mon[30349]: pgmap v143: 129 pgs: 129 active+clean; 2.6 GiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-05-09T05:42:53.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:53 smithi161 ceph-mon[33805]: pgmap v143: 129 pgs: 129 active+clean; 2.6 GiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-05-09T05:42:53.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:53 smithi195 ceph-mon[33844]: pgmap v143: 129 pgs: 129 active+clean; 2.6 GiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 MiB/s wr, 17 op/s 2024-05-09T05:42:55.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:55 smithi115 ceph-mon[30349]: pgmap v144: 129 pgs: 129 active+clean; 2.7 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 MiB/s wr, 28 op/s 2024-05-09T05:42:55.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:55 smithi161 ceph-mon[33805]: pgmap v144: 129 pgs: 129 active+clean; 2.7 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 MiB/s wr, 28 op/s 2024-05-09T05:42:55.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:55 smithi195 ceph-mon[33844]: pgmap v144: 129 pgs: 129 active+clean; 2.7 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 MiB/s wr, 28 op/s 2024-05-09T05:42:57.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:57 smithi115 ceph-mon[30349]: pgmap v145: 129 pgs: 129 active+clean; 2.7 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:42:57.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:57 smithi161 ceph-mon[33805]: pgmap v145: 129 pgs: 129 active+clean; 2.7 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:42:57.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:57 smithi195 ceph-mon[33844]: pgmap v145: 129 pgs: 129 active+clean; 2.7 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:42:59.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:42:59 smithi161 ceph-mon[33805]: pgmap v146: 129 pgs: 129 active+clean; 2.9 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:42:59.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:42:59 smithi195 ceph-mon[33844]: pgmap v146: 129 pgs: 129 active+clean; 2.9 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:42:59.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:42:59 smithi115 ceph-mon[30349]: pgmap v146: 129 pgs: 129 active+clean; 2.9 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:43:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:01 smithi161 ceph-mon[33805]: pgmap v147: 129 pgs: 129 active+clean; 2.9 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:43:01.755 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:43:01.780 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:01 smithi115 ceph-mon[30349]: pgmap v147: 129 pgs: 129 active+clean; 2.9 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:43:01.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:01 smithi195 ceph-mon[33844]: pgmap v147: 129 pgs: 129 active+clean; 2.9 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:43:02.054 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.148+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/2895994743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 msgr2=0x7fd7f0102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.148+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/2895994743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 0x7fd7f0102740 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009900 tx=0x7fd7ec02a410 comp rx=0 tx=0).stop 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.148+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/2895994743 shutdown_connections 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.148+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/2895994743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f01032f0 0x7fd7f010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.148+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/2895994743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0109040 0x7fd7f0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.148+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/2895994743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 0x7fd7f0102740 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.148+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/2895994743 >> 172.21.15.115:0/2895994743 conn(0x7fd7f00782e0 msgr2=0x7fd7f0078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.148+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/2895994743 shutdown_connections 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.148+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/2895994743 wait complete. 2024-05-09T05:43:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f7dd5640 1 Processor -- start 2024-05-09T05:43:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f7dd5640 1 -- start start 2024-05-09T05:43:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f7dd5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f01032f0 0x7fd7f019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f7dd5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 0x7fd7f019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f7dd5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0109040 0x7fd7f019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f7dd5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0112c40 con 0x7fd7f0108ba0 2024-05-09T05:43:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f7dd5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0112ac0 con 0x7fd7f0109040 2024-05-09T05:43:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f7dd5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0112dc0 con 0x7fd7f01032f0 2024-05-09T05:43:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f5370640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 0x7fd7f019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f5370640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 0x7fd7f019bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49534/0 (socket says 172.21.15.115:49534) 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f5370640 1 -- 172.21.15.115:0/1176775742 learned_addr learned my addr 172.21.15.115:0/1176775742 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.149+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f01032f0 0x7fd7f019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7f6372640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0109040 0x7fd7f019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7f5370640 1 -- 172.21.15.115:0/1176775742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f01032f0 msgr2=0x7fd7f019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7f5370640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f01032f0 0x7fd7f019b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7f5370640 1 -- 172.21.15.115:0/1176775742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0109040 msgr2=0x7fd7f019c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7f5370640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0109040 0x7fd7f019c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7f5370640 1 -- 172.21.15.115:0/1176775742 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f01a68e0 con 0x7fd7f0108ba0 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7f5370640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 0x7fd7f019bba0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c970 tx=0x7fd7e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7e6ffd640 1 -- 172.21.15.115:0/1176775742 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0007c70 con 0x7fd7f0108ba0 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7e6ffd640 1 -- 172.21.15.115:0/1176775742 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7e0007e10 con 0x7fd7f0108ba0 2024-05-09T05:43:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7e6ffd640 1 -- 172.21.15.115:0/1176775742 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e00186f0 con 0x7fd7f0108ba0 2024-05-09T05:43:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/1176775742 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f01a6bd0 con 0x7fd7f0108ba0 2024-05-09T05:43:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.150+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/1176775742 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f01a70d0 con 0x7fd7f0108ba0 2024-05-09T05:43:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.153+0000 7fd7fed63740 1 Processor -- start 2024-05-09T05:43:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.153+0000 7fd7fed63740 1 -- start start 2024-05-09T05:43:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7e6ffd640 1 -- 172.21.15.115:0/1176775742 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fd7e000fa90 con 0x7fd7f0108ba0 2024-05-09T05:43:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1daf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dafbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 0x5616d1db0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7fed63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5616d1d147c0 con 0x5616d1a0c1b0 2024-05-09T05:43:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7fed63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5616d1cb00e0 con 0x5616d19e32f0 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7fed63740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5616d1d10780 con 0x5616d19eb4d0 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f6372640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1daf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f6372640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1daf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49550/0 (socket says 172.21.15.115:49550) 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f6372640 1 -- 172.21.15.115:0/3550923777 learned_addr learned my addr 172.21.15.115:0/3550923777 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dafbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f5370640 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 0x5616d1db0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f6372640 1 -- 172.21.15.115:0/3550923777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 msgr2=0x5616d1db0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f6372640 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 0x5616d1db0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f6372640 1 -- 172.21.15.115:0/3550923777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 msgr2=0x5616d1dafbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f6372640 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dafbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f6372640 1 -- 172.21.15.115:0/3550923777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d1cd97c0 con 0x5616d1a0c1b0 2024-05-09T05:43:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.154+0000 7fd7f5370640 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 0x5616d1db0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.155+0000 7fd7f6372640 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1daf680 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009eb0 tx=0x7fd7e8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3163a4b4c416f148 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.155+0000 7fd7e6ffd640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd7dc0b2950 0x7fd7dc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.155+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd7dc0b2950 0x7fd7dc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.155+0000 7fd7dbfff640 1 -- 172.21.15.115:0/3550923777 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8016070 con 0x5616d1a0c1b0 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.155+0000 7fd7dbfff640 1 -- 172.21.15.115:0/3550923777 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7e8004d20 con 0x5616d1a0c1b0 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.155+0000 7fd7e6ffd640 1 -- 172.21.15.115:0/1176775742 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd7e00e29a0 con 0x7fd7f0108ba0 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.155+0000 7fd7dbfff640 1 -- 172.21.15.115:0/3550923777 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80126b0 con 0x5616d1a0c1b0 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 -- 172.21.15.115:0/3550923777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 msgr2=0x5616d1daf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd7dc0b2950 0x7fd7dc0b4e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec004770 tx=0x7fd7ec0058d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1daf680 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009eb0 tx=0x7fd7e8010740 comp rx=0 tx=0).stop 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 -- 172.21.15.115:0/3550923777 shutdown_connections 2024-05-09T05:43:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 0x5616d1db0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dafbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 --2- 172.21.15.115:0/3550923777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1daf680 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 -- 172.21.15.115:0/3550923777 >> 172.21.15.115:0/3550923777 conn(0x5616d1a00300 msgr2=0x5616d1dabe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 -- 172.21.15.115:0/3550923777 shutdown_connections 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 -- 172.21.15.115:0/3550923777 wait complete. 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 -- start start 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.156+0000 7fd7fed63740 1 Processor -- start 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19e32f0 0x5616d1dbb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19eb4d0 0x5616d1dbbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1db9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7fed63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5616d1dbc7b0 con 0x5616d1a0c1b0 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7fed63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5616d1c9b9c0 con 0x5616d19eb4d0 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7fed63740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5616d1dbc930 con 0x5616d19e32f0 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7fed63740 10 client.0 _ll_register_callbacks cb 0x5616d1cc8020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7f6372640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1db9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7f6372640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1db9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49556/0 (socket says 172.21.15.115:49556) 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7f6372640 1 -- 172.21.15.115:0/2311674636 learned_addr learned my addr 172.21.15.115:0/2311674636 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7f5370640 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19eb4d0 0x5616d1dbbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.157+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19e32f0 0x5616d1dbb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7f6372640 1 -- 172.21.15.115:0/2311674636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19e32f0 msgr2=0x5616d1dbb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7f6372640 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19e32f0 0x5616d1dbb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7f6372640 1 -- 172.21.15.115:0/2311674636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19eb4d0 msgr2=0x5616d1dbbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7f6372640 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19eb4d0 0x5616d1dbbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7f6372640 1 -- 172.21.15.115:0/2311674636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d1cc21f0 con 0x5616d1a0c1b0 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7f6372640 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1db9ba0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005a30 tx=0x7fd7e8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f301b4ce1e10f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7d97fa640 1 -- 172.21.15.115:0/2311674636 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8028060 con 0x5616d1a0c1b0 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7d97fa640 1 -- 172.21.15.115:0/2311674636 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd7e80052e0 con 0x5616d1a0c1b0 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7fed63740 1 -- 172.21.15.115:0/2311674636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7e8004d20 con 0x5616d1a0c1b0 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7d97fa640 1 -- 172.21.15.115:0/2311674636 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd7e802f050 con 0x5616d1a0c1b0 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7fed63740 10 client.16857 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7fed63740 1 -- 172.21.15.115:0/2311674636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5616d1a77790 con 0x5616d1a0c1b0 2024-05-09T05:43:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.158+0000 7fd7d97fa640 1 -- 172.21.15.115:0/2311674636 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7e8004d20 con 0x5616d1a0c1b0 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7d97fa640 1 -- 172.21.15.115:0/2311674636 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fd7e8022030 con 0x5616d1a0c1b0 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7fed63740 10 client.16857 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7fed63740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7fed63740 20 client.16857 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7fed63740 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5616d19cad80 0x5616d1dcbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7fed63740 4 client.16857 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7fed63740 1 -- 172.21.15.115:0/2311674636 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616d1d223d0 con 0x5616d19cad80 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5616d19cad80 0x5616d1dcbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7d97fa640 10 client.16857 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.159+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5616d19cad80 0x5616d1dcbe40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6465e6caaa900543 server_cookie=82eedeba95082342 in_seq=0 out_seq=0 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.160+0000 7fd7d97fa640 1 -- 172.21.15.115:0/2311674636 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5616d1d223d0 con 0x5616d19cad80 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.160+0000 7fd7d97fa640 10 client.16857 handle_command_reply: tid=0 2024-05-09T05:43:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 client.16857 shutdown 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 -- 172.21.15.115:0/2311674636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 msgr2=0x5616d1db9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1db9ba0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005a30 tx=0x7fd7e8009bf0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 -- 172.21.15.115:0/2311674636 shutdown_connections 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1db9ba0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19eb4d0 0x5616d1dbbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5616d19cad80 0x5616d1dcbe40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 --2- 172.21.15.115:0/2311674636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19e32f0 0x5616d1dbb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7d97fa640 0 client.16857 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 -- 172.21.15.115:0/2311674636 >> 172.21.15.115:0/2311674636 conn(0x5616d1a00300 msgr2=0x5616d1daa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 -- 172.21.15.115:0/2311674636 shutdown_connections 2024-05-09T05:43:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 1 -- 172.21.15.115:0/2311674636 wait complete. 2024-05-09T05:43:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.161+0000 7fd7fed63740 20 client.16857 trim_cache size 0 max 16384 2024-05-09T05:43:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.181+0000 7fd7fed63740 1 Processor -- start 2024-05-09T05:43:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.181+0000 7fd7fed63740 1 -- start start 2024-05-09T05:43:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dabf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 0x5616d1dcee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1dcf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7fed63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01b1120 con 0x5616d1a0c1b0 2024-05-09T05:43:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7fed63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01b2d50 con 0x5616d19e32f0 2024-05-09T05:43:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7fed63740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01b17f0 con 0x5616d19eb4d0 2024-05-09T05:43:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f6372640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1dcf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f6372640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1dcf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49572/0 (socket says 172.21.15.115:49572) 2024-05-09T05:43:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f6372640 1 -- 172.21.15.115:0/3460099161 learned_addr learned my addr 172.21.15.115:0/3460099161 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/3460099161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dabf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f5370640 1 --2- 172.21.15.115:0/3460099161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 0x5616d1dcee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f6372640 1 -- 172.21.15.115:0/3460099161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 msgr2=0x5616d1dcee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f6372640 1 --2- 172.21.15.115:0/3460099161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 0x5616d1dcee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f6372640 1 -- 172.21.15.115:0/3460099161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 msgr2=0x5616d1dabf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f6372640 1 --2- 172.21.15.115:0/3460099161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dabf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f6372640 1 -- 172.21.15.115:0/3460099161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d1aee5f0 con 0x5616d1a0c1b0 2024-05-09T05:43:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.182+0000 7fd7f6372640 1 --2- 172.21.15.115:0/3460099161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1dcf3b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005d70 tx=0x7fd7e8005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7fa9288477f653 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.183+0000 7fd7db7fe640 1 -- 172.21.15.115:0/3460099161 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8038070 con 0x5616d1a0c1b0 2024-05-09T05:43:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.183+0000 7fd7db7fe640 1 -- 172.21.15.115:0/3460099161 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7e801adc0 con 0x5616d1a0c1b0 2024-05-09T05:43:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.183+0000 7fd7db7fe640 1 -- 172.21.15.115:0/3460099161 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8028040 con 0x5616d1a0c1b0 2024-05-09T05:43:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 -- 172.21.15.115:0/3460099161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 msgr2=0x5616d1dcf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 --2- 172.21.15.115:0/3460099161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1dcf3b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005d70 tx=0x7fd7e8005a60 comp rx=0 tx=0).stop 2024-05-09T05:43:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 -- 172.21.15.115:0/3460099161 shutdown_connections 2024-05-09T05:43:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 --2- 172.21.15.115:0/3460099161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d1a0c1b0 0x5616d1dcf3b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 --2- 172.21.15.115:0/3460099161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d19eb4d0 0x5616d1dcee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 --2- 172.21.15.115:0/3460099161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dabf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 -- 172.21.15.115:0/3460099161 >> 172.21.15.115:0/3460099161 conn(0x5616d1a00300 msgr2=0x5616d1dab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 -- 172.21.15.115:0/3460099161 shutdown_connections 2024-05-09T05:43:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 -- 172.21.15.115:0/3460099161 wait complete. 2024-05-09T05:43:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 -- start start 2024-05-09T05:43:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.184+0000 7fd7fed63740 1 Processor -- start 2024-05-09T05:43:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d19eb4d0 0x5616d1dce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7fed63740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d1a0c1b0 0x5616d1e31f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7fed63740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01b2620 con 0x5616d19eb4d0 2024-05-09T05:43:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7fed63740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01b1ec0 con 0x5616d19e32f0 2024-05-09T05:43:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7fed63740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01af100 con 0x5616d1a0c1b0 2024-05-09T05:43:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7fed63740 10 client.0 _ll_register_callbacks cb 0x7fd7ec005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5370640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d19eb4d0 0x5616d1dce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5370640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d19eb4d0 0x5616d1dce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49588/0 (socket says 172.21.15.115:49588) 2024-05-09T05:43:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5370640 1 -- 172.21.15.115:0/86091150 learned_addr learned my addr 172.21.15.115:0/86091150 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f6372640 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d1a0c1b0 0x5616d1e31f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5370640 1 -- 172.21.15.115:0/86091150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d1a0c1b0 msgr2=0x5616d1e31f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5370640 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d1a0c1b0 0x5616d1e31f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5370640 1 -- 172.21.15.115:0/86091150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 msgr2=0x5616d1dce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5370640 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5370640 1 -- 172.21.15.115:0/86091150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d1cd8ed0 con 0x5616d19eb4d0 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f6372640 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d1a0c1b0 0x5616d1e31f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.185+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7f5370640 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d19eb4d0 0x5616d1dce6f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000cfd0 tx=0x7fd7e00cc330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabaf112e369d958 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7d97fa640 1 -- 172.21.15.115:0/86091150 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e00c3060 con 0x5616d19eb4d0 2024-05-09T05:43:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7d97fa640 1 -- 172.21.15.115:0/86091150 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd7e00b2520 con 0x5616d19eb4d0 2024-05-09T05:43:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7fed63740 1 -- 172.21.15.115:0/86091150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7e801adc0 con 0x5616d19eb4d0 2024-05-09T05:43:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7d97fa640 1 -- 172.21.15.115:0/86091150 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd7e008fe60 con 0x5616d19eb4d0 2024-05-09T05:43:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7fed63740 10 client.16863 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7fed63740 1 -- 172.21.15.115:0/86091150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5616d1e335a0 con 0x5616d19eb4d0 2024-05-09T05:43:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7d97fa640 1 -- 172.21.15.115:0/86091150 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7e801adc0 con 0x5616d19eb4d0 2024-05-09T05:43:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7d97fa640 1 -- 172.21.15.115:0/86091150 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7fd7e006f1a0 con 0x5616d19eb4d0 2024-05-09T05:43:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7fed63740 10 client.16863 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7fed63740 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7fed63740 20 client.16863 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7fed63740 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5616d19a01c0 0x5616d1e35d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.186+0000 7fd7fed63740 4 client.16863 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:43:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.187+0000 7fd7fed63740 1 -- 172.21.15.115:0/86091150 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616d1d1c240 con 0x5616d19a01c0 2024-05-09T05:43:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.187+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5616d19a01c0 0x5616d1e35d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.187+0000 7fd7d97fa640 10 client.16863 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.187+0000 7fd7f5b71640 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5616d19a01c0 0x5616d1e35d90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f08cff1db2f794 server_cookie=16e3806e087bc2ed in_seq=0 out_seq=0 2024-05-09T05:43:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7d97fa640 1 -- 172.21.15.115:0/86091150 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5616d1d1c240 con 0x5616d19a01c0 2024-05-09T05:43:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7d97fa640 10 client.16863 handle_command_reply: tid=0 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 client.16863 shutdown 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 -- 172.21.15.115:0/86091150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d19eb4d0 msgr2=0x5616d1dce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d19eb4d0 0x5616d1dce6f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000cfd0 tx=0x7fd7e00cc330 comp rx=0 tx=0).stop 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 -- 172.21.15.115:0/86091150 shutdown_connections 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5616d19a01c0 0x5616d1e35d90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5616d1a0c1b0 0x5616d1e31f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5616d19eb4d0 0x5616d1dce6f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 --2- 172.21.15.115:0/86091150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5616d19e32f0 0x5616d1dce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 -- 172.21.15.115:0/86091150 >> 172.21.15.115:0/86091150 conn(0x5616d1a00300 msgr2=0x5616d1db9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7d97fa640 0 client.16863 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 -- 172.21.15.115:0/86091150 shutdown_connections 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 1 -- 172.21.15.115:0/86091150 wait complete. 2024-05-09T05:43:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.368+0000 7fd7fed63740 20 client.16863 trim_cache size 0 max 16384 2024-05-09T05:43:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:43:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:43:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd7dc0b2950 msgr2=0x7fd7dc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd7dc0b2950 0x7fd7dc0b4e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec004770 tx=0x7fd7ec0058d0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 msgr2=0x7fd7f019bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 0x7fd7f019bba0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c970 tx=0x7fd7e000ce30 comp rx=0 tx=0).stop 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/1176775742 shutdown_connections 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd7dc0b2950 0x7fd7dc0b4e60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7f0109040 0x7fd7f019c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7f0108ba0 0x7fd7f019bba0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 --2- 172.21.15.115:0/1176775742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7f01032f0 0x7fd7f019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/1176775742 >> 172.21.15.115:0/1176775742 conn(0x7fd7f00782e0 msgr2=0x7fd7f00ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/1176775742 shutdown_connections 2024-05-09T05:43:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:03.369+0000 7fd7f7dd5640 1 -- 172.21.15.115:0/1176775742 wait complete. 2024-05-09T05:43:03.521 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:03 smithi115 ceph-mon[30349]: pgmap v148: 129 pgs: 129 active+clean; 2.9 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:43:03.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:03 smithi161 ceph-mon[33805]: pgmap v148: 129 pgs: 129 active+clean; 2.9 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:43:03.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:03 smithi195 ceph-mon[33844]: pgmap v148: 129 pgs: 129 active+clean; 2.9 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:43:03.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 -- 172.21.15.115:0/1417792207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 msgr2=0x7f779c105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/1417792207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 0x7f779c105910 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7798009990 tx=0x7f7798009ca0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 -- 172.21.15.115:0/1417792207 shutdown_connections 2024-05-09T05:43:03.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/1417792207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c10e930 0x7f779c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/1417792207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 0x7f779c105910 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/1417792207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f779c105e50 0x7f779c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 -- 172.21.15.115:0/1417792207 >> 172.21.15.115:0/1417792207 conn(0x7f779c0fedf0 msgr2=0x7f779c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 -- 172.21.15.115:0/1417792207 shutdown_connections 2024-05-09T05:43:03.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 -- 172.21.15.115:0/1417792207 wait complete. 2024-05-09T05:43:03.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.921+0000 7f77a3eb7700 1 Processor -- start 2024-05-09T05:43:03.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a3eb7700 1 -- start start 2024-05-09T05:43:03.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a3eb7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 0x7f779c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a3eb7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f779c105e50 0x7f779c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a3eb7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c10e930 0x7f779c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a3eb7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1142f0 con 0x7f779c105500 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a3eb7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f779c114170 con 0x7f779c105e50 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a3eb7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f779c114470 con 0x7f779c10e930 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a1c53700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 0x7f779c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a1c53700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 0x7f779c19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49614/0 (socket says 172.21.15.115:49614) 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a1c53700 1 -- 172.21.15.115:0/185982806 learned_addr learned my addr 172.21.15.115:0/185982806 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a1452700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f779c105e50 0x7f779c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a2454700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c10e930 0x7f779c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a1c53700 1 -- 172.21.15.115:0/185982806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c10e930 msgr2=0x7f779c19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a1c53700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c10e930 0x7f779c19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a1c53700 1 -- 172.21.15.115:0/185982806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f779c105e50 msgr2=0x7f779c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a1c53700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f779c105e50 0x7f779c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a1c53700 1 -- 172.21.15.115:0/185982806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c19fb60 con 0x7f779c105500 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.922+0000 7f77a2454700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c10e930 0x7f779c19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.923+0000 7f77a1c53700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 0x7f779c19ad40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7798000c00 tx=0x7f77980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.923+0000 7f7792ffd700 1 -- 172.21.15.115:0/185982806 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798027070 con 0x7f779c105500 2024-05-09T05:43:03.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.923+0000 7f7792ffd700 1 -- 172.21.15.115:0/185982806 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f779800fd00 con 0x7f779c105500 2024-05-09T05:43:03.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.923+0000 7f77a3eb7700 1 -- 172.21.15.115:0/185982806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c19fdf0 con 0x7f779c105500 2024-05-09T05:43:03.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.923+0000 7f7792ffd700 1 -- 172.21.15.115:0/185982806 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798022e40 con 0x7f779c105500 2024-05-09T05:43:03.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.923+0000 7f77a3eb7700 1 -- 172.21.15.115:0/185982806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c1a0320 con 0x7f779c105500 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.926+0000 7f7792ffd700 1 -- 172.21.15.115:0/185982806 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f779802a070 con 0x7f779c105500 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.927+0000 7f7792ffd700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f77880b2a70 0x7f77880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.927+0000 7f77a1452700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f77880b2a70 0x7f77880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.927+0000 7f7792ffd700 1 -- 172.21.15.115:0/185982806 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f77980e36c0 con 0x7f779c105500 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.927+0000 7f77b28cdb80 1 Processor -- start 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77a1452700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f77880b2a70 0x7f77880b4f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f779c102360 tx=0x7f778c00a380 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77b28cdb80 1 -- start start 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 0x55f9513b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 0x55f9513b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f9512c5ff0 con 0x55f9513abeb0 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f9512647f0 con 0x55f9512264a0 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f9512c1fb0 con 0x55f951310160 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77a2454700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77a2454700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513ac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49622/0 (socket says 172.21.15.115:49622) 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77a2454700 1 -- 172.21.15.115:0/5184608 learned_addr learned my addr 172.21.15.115:0/5184608 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77a1c53700 1 --2- 172.21.15.115:0/5184608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 0x55f9513b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.928+0000 7f77a1452700 1 --2- 172.21.15.115:0/5184608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 0x55f9513b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.929+0000 7f77a2454700 1 -- 172.21.15.115:0/5184608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 msgr2=0x55f9513b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.929+0000 7f77a2454700 1 --2- 172.21.15.115:0/5184608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 0x55f9513b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.929+0000 7f77a2454700 1 -- 172.21.15.115:0/5184608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 msgr2=0x55f9513b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.929+0000 7f77a2454700 1 --2- 172.21.15.115:0/5184608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 0x55f9513b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.929+0000 7f77a2454700 1 -- 172.21.15.115:0/5184608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95125c370 con 0x55f9513abeb0 2024-05-09T05:43:03.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.929+0000 7f77a2454700 1 --2- 172.21.15.115:0/5184608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513ac2e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f77940085a0 tx=0x7f77940088b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986a5ac2fb326d40 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.929+0000 7f7787fff700 1 -- 172.21.15.115:0/5184608 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794003c00 con 0x55f9513abeb0 2024-05-09T05:43:03.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.929+0000 7f7787fff700 1 -- 172.21.15.115:0/5184608 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7794003da0 con 0x55f9513abeb0 2024-05-09T05:43:03.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.929+0000 7f7787fff700 1 -- 172.21.15.115:0/5184608 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794012920 con 0x55f9513abeb0 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.930+0000 7f77b28cdb80 1 -- 172.21.15.115:0/5184608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 msgr2=0x55f9513ac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.930+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/5184608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513ac2e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f77940085a0 tx=0x7f77940088b0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.930+0000 7f77b28cdb80 1 -- 172.21.15.115:0/5184608 shutdown_connections 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.931+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/5184608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513ac2e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.931+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/5184608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 0x55f9513b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.931+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/5184608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 0x55f9513b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.931+0000 7f77b28cdb80 1 -- 172.21.15.115:0/5184608 >> 172.21.15.115:0/5184608 conn(0x55f9512866a0 msgr2=0x55f95139e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.931+0000 7f77b28cdb80 1 -- 172.21.15.115:0/5184608 shutdown_connections 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.931+0000 7f77b28cdb80 1 -- 172.21.15.115:0/5184608 wait complete. 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.931+0000 7f77b28cdb80 1 -- start start 2024-05-09T05:43:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.931+0000 7f77b28cdb80 1 Processor -- start 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 0x55f9513b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f951310160 0x55f9513b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f951141ba0 con 0x55f9513abeb0 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f951280150 con 0x55f951310160 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f9513b89e0 con 0x55f9512264a0 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77b28cdb80 10 client.0 _ll_register_callbacks cb 0x55f9511376c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77a2454700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77a1c53700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 0x55f9513b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77a2454700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513b6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49624/0 (socket says 172.21.15.115:49624) 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77a1c53700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 0x55f9513b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46172/0 (socket says 172.21.15.115:46172) 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77a2454700 1 -- 172.21.15.115:0/1283866760 learned_addr learned my addr 172.21.15.115:0/1283866760 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.932+0000 7f77a1452700 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f951310160 0x55f9513b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77a1452700 1 -- 172.21.15.115:0/1283866760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 msgr2=0x55f9513b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77a1452700 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 0x55f9513b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77a1452700 1 -- 172.21.15.115:0/1283866760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 msgr2=0x55f9513b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77a1452700 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77a1452700 1 -- 172.21.15.115:0/1283866760 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f951141490 con 0x55f951310160 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77a1452700 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f951310160 0x55f9513b5ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f95139fed0 tx=0x7f778c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec774fb9685367b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77857fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77857fa700 1 -- 172.21.15.115:0/1283866760 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c016070 con 0x55f951310160 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77b28cdb80 1 -- 172.21.15.115:0/1283866760 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7794003da0 con 0x55f951310160 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77857fa700 1 -- 172.21.15.115:0/1283866760 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f778c027670 con 0x55f951310160 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77857fa700 1 -- 172.21.15.115:0/1283866760 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f778c030050 con 0x55f951310160 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77b28cdb80 10 client.26300 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.933+0000 7f77b28cdb80 1 -- 172.21.15.115:0/1283866760 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f9513c3530 con 0x55f951310160 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.934+0000 7f77857fa700 1 -- 172.21.15.115:0/1283866760 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7794003da0 con 0x55f951310160 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.934+0000 7f77857fa700 1 -- 172.21.15.115:0/1283866760 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f778c010910 con 0x55f951310160 2024-05-09T05:43:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.934+0000 7f77b28cdb80 10 client.26300 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:03.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.934+0000 7f77b28cdb80 10 client.26300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:03.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.934+0000 7f77b28cdb80 20 client.26300 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:03.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.934+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f9513c5ac0 0x55f9513c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.934+0000 7f77b28cdb80 4 client.26300 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:03.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.934+0000 7f77b28cdb80 1 -- 172.21.15.115:0/1283866760 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9512d39d0 con 0x55f9513c5ac0 2024-05-09T05:43:03.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.934+0000 7f77a1c53700 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f9513c5ac0 0x55f9513c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.935+0000 7f77857fa700 10 client.26300 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:03.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.935+0000 7f77a1c53700 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f9513c5ac0 0x55f9513c7f70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=775a7fff4faa8107 server_cookie=916b1422d4441bd3 in_seq=0 out_seq=0 2024-05-09T05:43:03.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.935+0000 7f77857fa700 1 -- 172.21.15.115:0/1283866760 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9512d39d0 con 0x55f9513c5ac0 2024-05-09T05:43:03.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.935+0000 7f77857fa700 10 client.26300 handle_command_reply: tid=0 2024-05-09T05:43:03.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.935+0000 7f77b28cdb80 1 client.26300 shutdown 2024-05-09T05:43:03.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 -- 172.21.15.115:0/1283866760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f951310160 msgr2=0x55f9513b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f951310160 0x55f9513b5ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f95139fed0 tx=0x7f778c0047c0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 -- 172.21.15.115:0/1283866760 shutdown_connections 2024-05-09T05:43:03.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f9513c5ac0 0x55f9513c7f70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f9513b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 0x55f9513b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/1283866760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f951310160 0x55f9513b5ff0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77857fa700 0 client.26300 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:03.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 -- 172.21.15.115:0/1283866760 >> 172.21.15.115:0/1283866760 conn(0x55f9512866a0 msgr2=0x55f95139e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 -- 172.21.15.115:0/1283866760 shutdown_connections 2024-05-09T05:43:03.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 1 -- 172.21.15.115:0/1283866760 wait complete. 2024-05-09T05:43:03.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.936+0000 7f77b28cdb80 20 client.26300 trim_cache size 0 max 16384 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77b28cdb80 1 Processor -- start 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77b28cdb80 1 -- start start 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 0x55f9513b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f951310160 0x55f9513b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9513abeb0 0x55f9513b7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1ae030 con 0x55f951310160 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1afc60 con 0x55f9513abeb0 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1ae700 con 0x55f9512264a0 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77a1452700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f951310160 0x55f9513b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77a1452700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f951310160 0x55f9513b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49636/0 (socket says 172.21.15.115:49636) 2024-05-09T05:43:03.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77a1452700 1 -- 172.21.15.115:0/713417205 learned_addr learned my addr 172.21.15.115:0/713417205 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77a2454700 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9513abeb0 0x55f9513b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.963+0000 7f77a1c53700 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 0x55f9513b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77a1452700 1 -- 172.21.15.115:0/713417205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 msgr2=0x55f9513b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77a1452700 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 0x55f9513b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77a1452700 1 -- 172.21.15.115:0/713417205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9513abeb0 msgr2=0x55f9513b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77a1452700 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9513abeb0 0x55f9513b7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77a1452700 1 -- 172.21.15.115:0/713417205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95139fd00 con 0x55f951310160 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77a2454700 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9513abeb0 0x55f9513b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77a1452700 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f951310160 0x55f9513b0040 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f778c00fb80 tx=0x7f778c00f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9b8d24f4331671 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77877fe700 1 -- 172.21.15.115:0/713417205 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c038070 con 0x55f951310160 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77877fe700 1 -- 172.21.15.115:0/713417205 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f778c022070 con 0x55f951310160 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.964+0000 7f77877fe700 1 -- 172.21.15.115:0/713417205 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c027a40 con 0x55f951310160 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.965+0000 7f77b28cdb80 1 -- 172.21.15.115:0/713417205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f951310160 msgr2=0x55f9513b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.965+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f951310160 0x55f9513b0040 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f778c00fb80 tx=0x7f778c00f800 comp rx=0 tx=0).stop 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.965+0000 7f77b28cdb80 1 -- 172.21.15.115:0/713417205 shutdown_connections 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.965+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9513abeb0 0x55f9513b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.965+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9512264a0 0x55f9513b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.965+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/713417205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f951310160 0x55f9513b0040 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.965+0000 7f77b28cdb80 1 -- 172.21.15.115:0/713417205 >> 172.21.15.115:0/713417205 conn(0x55f9512866a0 msgr2=0x55f9513b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.965+0000 7f77b28cdb80 1 -- 172.21.15.115:0/713417205 shutdown_connections 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.965+0000 7f77b28cdb80 1 -- 172.21.15.115:0/713417205 wait complete. 2024-05-09T05:43:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77b28cdb80 1 -- start start 2024-05-09T05:43:03.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77b28cdb80 1 Processor -- start 2024-05-09T05:43:03.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 0x55f9513b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 0x55f9513ba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77b28cdb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f95142f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1af530 con 0x55f9513abeb0 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1aedd0 con 0x55f9512264a0 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77b28cdb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1abff0 con 0x55f951310160 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77b28cdb80 10 client.0 _ll_register_callbacks cb 0x55f9511376c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.966+0000 7f77a2454700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f95142f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a2454700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f95142f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49650/0 (socket says 172.21.15.115:49650) 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a1c53700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 0x55f9513b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a2454700 1 -- 172.21.15.115:0/3194278526 learned_addr learned my addr 172.21.15.115:0/3194278526 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a1452700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 0x55f9513ba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a2454700 1 -- 172.21.15.115:0/3194278526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 msgr2=0x55f9513ba420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a2454700 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 0x55f9513ba420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a2454700 1 -- 172.21.15.115:0/3194278526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 msgr2=0x55f9513b9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a2454700 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 0x55f9513b9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a2454700 1 -- 172.21.15.115:0/3194278526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9513ac510 con 0x55f9513abeb0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77a2454700 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f95142f330 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7794000f80 tx=0x7f77940127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce76f8f090041e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77857fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77857fa700 1 -- 172.21.15.115:0/3194278526 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794012d20 con 0x55f9513abeb0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77857fa700 1 -- 172.21.15.115:0/3194278526 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7794003a00 con 0x55f9513abeb0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.967+0000 7f77b28cdb80 1 -- 172.21.15.115:0/3194278526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778c022070 con 0x55f9513abeb0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77857fa700 1 -- 172.21.15.115:0/3194278526 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f779402f050 con 0x55f9513abeb0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77b28cdb80 10 client.16881 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77b28cdb80 1 -- 172.21.15.115:0/3194278526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f951430940 con 0x55f9513abeb0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77857fa700 1 -- 172.21.15.115:0/3194278526 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f778c022070 con 0x55f9513abeb0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77857fa700 1 -- 172.21.15.115:0/3194278526 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f7794013070 con 0x55f9513abeb0 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77b28cdb80 10 client.16881 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77b28cdb80 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77b28cdb80 20 client.16881 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f951430dd0 0x55f9514331f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77b28cdb80 4 client.16881 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77b28cdb80 1 -- 172.21.15.115:0/3194278526 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9512cda70 con 0x55f951430dd0 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.968+0000 7f77a1c53700 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f951430dd0 0x55f9514331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77857fa700 10 client.16881 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77a1c53700 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f951430dd0 0x55f9514331f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59296e65f9797d8a server_cookie=869f9f36189efd3e in_seq=0 out_seq=0 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77857fa700 1 -- 172.21.15.115:0/3194278526 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f9512cda70 con 0x55f951430dd0 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77857fa700 10 client.16881 handle_command_reply: tid=0 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77b28cdb80 1 client.16881 shutdown 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77b28cdb80 1 -- 172.21.15.115:0/3194278526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 msgr2=0x55f95142f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f95142f330 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7794000f80 tx=0x7f77940127a0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77b28cdb80 1 -- 172.21.15.115:0/3194278526 shutdown_connections 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9512264a0 0x55f9513b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f951430dd0 0x55f9514331f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f9513abeb0 0x55f95142f330 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77857fa700 0 client.16881 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77b28cdb80 1 --2- 172.21.15.115:0/3194278526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f951310160 0x55f9513ba420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.969+0000 7f77b28cdb80 1 -- 172.21.15.115:0/3194278526 >> 172.21.15.115:0/3194278526 conn(0x55f9512866a0 msgr2=0x55f9513afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77b28cdb80 1 -- 172.21.15.115:0/3194278526 shutdown_connections 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77b28cdb80 1 -- 172.21.15.115:0/3194278526 wait complete. 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77b28cdb80 20 client.16881 trim_cache size 0 max 16384 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:43:03.966 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:43:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 -- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f77880b2a70 msgr2=0x7f77880b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f77880b2a70 0x7f77880b4f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f779c102360 tx=0x7f778c00a380 comp rx=0 tx=0).stop 2024-05-09T05:43:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 -- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 msgr2=0x7f779c19ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 0x7f779c19ad40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7798000c00 tx=0x7f77980047b0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 -- 172.21.15.115:0/185982806 shutdown_connections 2024-05-09T05:43:03.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f77880b2a70 0x7f77880b4f80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f779c10e930 0x7f779c19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f779c105500 0x7f779c19ad40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 --2- 172.21.15.115:0/185982806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f779c105e50 0x7f779c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:03.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.970+0000 7f77a3eb7700 1 -- 172.21.15.115:0/185982806 >> 172.21.15.115:0/185982806 conn(0x7f779c0fedf0 msgr2=0x7f779c0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:03.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.971+0000 7f77a3eb7700 1 -- 172.21.15.115:0/185982806 shutdown_connections 2024-05-09T05:43:03.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:03.971+0000 7f77a3eb7700 1 -- 172.21.15.115:0/185982806 wait complete. 2024-05-09T05:43:04.709 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:43:04.710 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:43:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:05.432 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:05 smithi115 ceph-mon[30349]: pgmap v149: 129 pgs: 129 active+clean; 3.0 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 43 op/s 2024-05-09T05:43:05.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:05 smithi161 ceph-mon[33805]: pgmap v149: 129 pgs: 129 active+clean; 3.0 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 43 op/s 2024-05-09T05:43:05.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:05 smithi195 ceph-mon[33844]: pgmap v149: 129 pgs: 129 active+clean; 3.0 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 43 op/s 2024-05-09T05:43:07.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.082+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/3580326181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf341034c0 msgr2=0x7fbf341038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:07.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.082+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/3580326181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf341034c0 0x7fbf341038d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fbf300099c0 tx=0x7fbf30009cd0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.082+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/3580326181 shutdown_connections 2024-05-09T05:43:07.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.082+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/3580326181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf34108740 0x7fbf341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.082+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/3580326181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf341034c0 0x7fbf341038d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.082+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/3580326181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf34107df0 0x7fbf34108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.082+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/3580326181 >> 172.21.15.115:0/3580326181 conn(0x7fbf340fedc0 msgr2=0x7fbf341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:07.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.082+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/3580326181 shutdown_connections 2024-05-09T05:43:07.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.082+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/3580326181 wait complete. 2024-05-09T05:43:07.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.083+0000 7fbf3b8f3700 1 Processor -- start 2024-05-09T05:43:07.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.083+0000 7fbf3b8f3700 1 -- start start 2024-05-09T05:43:07.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.083+0000 7fbf3b8f3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf341034c0 0x7fbf3419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:07.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf3b8f3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf34107df0 0x7fbf3419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:07.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf3b8f3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf34108740 0x7fbf34199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:07.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf3b8f3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34113310 con 0x7fbf34107df0 2024-05-09T05:43:07.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf3b8f3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34113190 con 0x7fbf341034c0 2024-05-09T05:43:07.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf3b8f3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34113490 con 0x7fbf34108740 2024-05-09T05:43:07.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf38e8e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf34107df0 0x7fbf3419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:07.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf38e8e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf34107df0 0x7fbf3419ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49670/0 (socket says 172.21.15.115:49670) 2024-05-09T05:43:07.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf38e8e700 1 -- 172.21.15.115:0/576772451 learned_addr learned my addr 172.21.15.115:0/576772451 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:07.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf3968f700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf341034c0 0x7fbf3419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:07.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf38e8e700 1 -- 172.21.15.115:0/576772451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf34108740 msgr2=0x7fbf34199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:07.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf38e8e700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf34108740 0x7fbf34199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf38e8e700 1 -- 172.21.15.115:0/576772451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf341034c0 msgr2=0x7fbf3419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:07.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf38e8e700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf341034c0 0x7fbf3419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf38e8e700 1 -- 172.21.15.115:0/576772451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3419a280 con 0x7fbf34107df0 2024-05-09T05:43:07.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.084+0000 7fbf38e8e700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf34107df0 0x7fbf3419ff00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400d9b0 tx=0x7fbf2400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:07.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.085+0000 7fbf2a7fc700 1 -- 172.21.15.115:0/576772451 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2401b070 con 0x7fbf34107df0 2024-05-09T05:43:07.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.085+0000 7fbf2a7fc700 1 -- 172.21.15.115:0/576772451 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf2400ecd0 con 0x7fbf34107df0 2024-05-09T05:43:07.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.085+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3419a570 con 0x7fbf34107df0 2024-05-09T05:43:07.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.085+0000 7fbf2a7fc700 1 -- 172.21.15.115:0/576772451 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf240145a0 con 0x7fbf34107df0 2024-05-09T05:43:07.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.085+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3406b610 con 0x7fbf34107df0 2024-05-09T05:43:07.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.086+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf34104970 con 0x7fbf34107df0 2024-05-09T05:43:07.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.088+0000 7fbf2a7fc700 1 -- 172.21.15.115:0/576772451 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fbf2401e070 con 0x7fbf34107df0 2024-05-09T05:43:07.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.089+0000 7fbf2a7fc700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf200b2a70 0x7fbf200b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:07.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.089+0000 7fbf3968f700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf200b2a70 0x7fbf200b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:07.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.089+0000 7fbf2a7fc700 1 -- 172.21.15.115:0/576772451 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbf240e42a0 con 0x7fbf34107df0 2024-05-09T05:43:07.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.090+0000 7fbf3968f700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf200b2a70 0x7fbf200b4f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf30000c00 tx=0x7fbf30005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:07.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.093+0000 7fbf2a7fc700 1 -- 172.21.15.115:0/576772451 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf240aba50 con 0x7fbf34107df0 2024-05-09T05:43:07.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.379+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf340512c0 con 0x7fbf34107df0 2024-05-09T05:43:07.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.380+0000 7fbf2a7fc700 1 -- 172.21.15.115:0/576772451 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11961 (secure 0 0 0) 0x7fbf240ab350 con 0x7fbf34107df0 2024-05-09T05:43:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:43:07.374 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:41:33.359460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:43:07.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf200b2a70 msgr2=0x7fbf200b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:07.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf200b2a70 0x7fbf200b4f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf30000c00 tx=0x7fbf30005fb0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf34107df0 msgr2=0x7fbf3419ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:07.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf34107df0 0x7fbf3419ff00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400d9b0 tx=0x7fbf2400dd70 comp rx=0 tx=0).stop 2024-05-09T05:43:07.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 shutdown_connections 2024-05-09T05:43:07.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbf200b2a70 0x7fbf200b4f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf341034c0 0x7fbf3419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf34108740 0x7fbf34199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 --2- 172.21.15.115:0/576772451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf34107df0 0x7fbf3419ff00 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:07.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 >> 172.21.15.115:0/576772451 conn(0x7fbf340fedc0 msgr2=0x7fbf34101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:07.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 shutdown_connections 2024-05-09T05:43:07.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:07.383+0000 7fbf3b8f3700 1 -- 172.21.15.115:0/576772451 wait complete. 2024-05-09T05:43:07.378 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 34 2024-05-09T05:43:07.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:07 smithi115 ceph-mon[30349]: pgmap v150: 129 pgs: 129 active+clean; 3.0 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 32 op/s 2024-05-09T05:43:07.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:07 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/576772451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:43:07.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:07 smithi161 ceph-mon[33805]: pgmap v150: 129 pgs: 129 active+clean; 3.0 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 32 op/s 2024-05-09T05:43:07.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:07 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/576772451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:43:07.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:07 smithi195 ceph-mon[33844]: pgmap v150: 129 pgs: 129 active+clean; 3.0 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 32 op/s 2024-05-09T05:43:07.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:07 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/576772451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:43:08.048 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:43:08.324 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:09.674 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:09 smithi115 ceph-mon[30349]: pgmap v151: 129 pgs: 129 active+clean; 3.1 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:43:09.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:09 smithi195 ceph-mon[33844]: pgmap v151: 129 pgs: 129 active+clean; 3.1 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:43:09.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:09 smithi161 ceph-mon[33805]: pgmap v151: 129 pgs: 129 active+clean; 3.1 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:43:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.094+0000 7ff6ce285700 1 -- 172.21.15.115:0/1569184021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8108f20 msgr2=0x7ff6c8109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.094+0000 7ff6ce285700 1 --2- 172.21.15.115:0/1569184021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8108f20 0x7ff6c8109330 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0099c0 tx=0x7ff6bc009cd0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.094+0000 7ff6ce285700 1 -- 172.21.15.115:0/1569184021 shutdown_connections 2024-05-09T05:43:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.094+0000 7ff6ce285700 1 --2- 172.21.15.115:0/1569184021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c8109870 0x7ff6c81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.094+0000 7ff6ce285700 1 --2- 172.21.15.115:0/1569184021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c81045f0 0x7ff6c8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.094+0000 7ff6ce285700 1 --2- 172.21.15.115:0/1569184021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8108f20 0x7ff6c8109330 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.094+0000 7ff6ce285700 1 -- 172.21.15.115:0/1569184021 >> 172.21.15.115:0/1569184021 conn(0x7ff6c8100040 msgr2=0x7ff6c8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.094+0000 7ff6ce285700 1 -- 172.21.15.115:0/1569184021 shutdown_connections 2024-05-09T05:43:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.094+0000 7ff6ce285700 1 -- 172.21.15.115:0/1569184021 wait complete. 2024-05-09T05:43:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.095+0000 7ff6ce285700 1 Processor -- start 2024-05-09T05:43:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.095+0000 7ff6ce285700 1 -- start start 2024-05-09T05:43:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6ce285700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c81045f0 0x7ff6c819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6ce285700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c8108f20 0x7ff6c819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6ce285700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8109870 0x7ff6c819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6ce285700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c8114380 con 0x7ff6c8109870 2024-05-09T05:43:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6ce285700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c8114200 con 0x7ff6c8108f20 2024-05-09T05:43:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6ce285700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c8114500 con 0x7ff6c81045f0 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6cc822700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8109870 0x7ff6c819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6cc822700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8109870 0x7ff6c819f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49694/0 (socket says 172.21.15.115:49694) 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6cc822700 1 -- 172.21.15.115:0/2705383024 learned_addr learned my addr 172.21.15.115:0/2705383024 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c81045f0 0x7ff6c819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c8108f20 0x7ff6c819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6cc822700 1 -- 172.21.15.115:0/2705383024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c81045f0 msgr2=0x7ff6c819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6cc822700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c81045f0 0x7ff6c819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6cc822700 1 -- 172.21.15.115:0/2705383024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c8108f20 msgr2=0x7ff6c819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6cc822700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c8108f20 0x7ff6c819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6cc822700 1 -- 172.21.15.115:0/2705383024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c819fcc0 con 0x7ff6c8109870 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.096+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c81045f0 0x7ff6c819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.097+0000 7ff6cc822700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8109870 0x7ff6c819f620 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0007ee0 tx=0x7ff6c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.097+0000 7ff6c57fa700 1 -- 172.21.15.115:0/2705383024 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000dd20 con 0x7ff6c8109870 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.097+0000 7ff6c57fa700 1 -- 172.21.15.115:0/2705383024 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6c0004020 con 0x7ff6c8109870 2024-05-09T05:43:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.097+0000 7ff6ce285700 1 -- 172.21.15.115:0/2705383024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c819ffb0 con 0x7ff6c8109870 2024-05-09T05:43:10.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.097+0000 7ff6c57fa700 1 -- 172.21.15.115:0/2705383024 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0015660 con 0x7ff6c8109870 2024-05-09T05:43:10.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.097+0000 7ff6ce285700 1 -- 172.21.15.115:0/2705383024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c81a78d0 con 0x7ff6c8109870 2024-05-09T05:43:10.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.100+0000 7ff6c57fa700 1 -- 172.21.15.115:0/2705383024 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff6c0015800 con 0x7ff6c8109870 2024-05-09T05:43:10.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.101+0000 7ff6c57fa700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6b00b2a70 0x7ff6b00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.101+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6b00b2a70 0x7ff6b00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.101+0000 7ff6c57fa700 1 -- 172.21.15.115:0/2705383024 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff6c00e2250 con 0x7ff6c8109870 2024-05-09T05:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.101+0000 7ff6dcc9bb80 1 Processor -- start 2024-05-09T05:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.101+0000 7ff6dcc9bb80 1 -- start start 2024-05-09T05:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.101+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6b00b2a70 0x7ff6b00b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8011490 tx=0x7ff6b8009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dc9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dc9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dc3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ac84ce60a0 con 0x55ac84cc0d00 2024-05-09T05:43:10.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ac84c8b9a0 con 0x55ac84c464a0 2024-05-09T05:43:10.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac84ce2060 con 0x55ac84dc3400 2024-05-09T05:43:10.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dc9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dc9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49702/0 (socket says 172.21.15.115:49702) 2024-05-09T05:43:10.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c7fff700 1 -- 172.21.15.115:0/1434031809 learned_addr learned my addr 172.21.15.115:0/1434031809 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:10.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dc9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6cc822700 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dc3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c7fff700 1 -- 172.21.15.115:0/1434031809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 msgr2=0x55ac84dc3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:10.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dc3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c7fff700 1 -- 172.21.15.115:0/1434031809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 msgr2=0x55ac84dc9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:10.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dc9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c7fff700 1 -- 172.21.15.115:0/1434031809 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac84c7c500 con 0x55ac84cc0d00 2024-05-09T05:43:10.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dc9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:10.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.102+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dc9240 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80043c0 tx=0x7ff6b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9447038ab3598ad2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:10.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.103+0000 7ff6ae7fc700 1 -- 172.21.15.115:0/1434031809 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8003d10 con 0x55ac84cc0d00 2024-05-09T05:43:10.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.103+0000 7ff6ae7fc700 1 -- 172.21.15.115:0/1434031809 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b8011e50 con 0x55ac84cc0d00 2024-05-09T05:43:10.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.103+0000 7ff6ae7fc700 1 -- 172.21.15.115:0/1434031809 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b801e650 con 0x55ac84cc0d00 2024-05-09T05:43:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1434031809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 msgr2=0x55ac84dc9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dc9240 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80043c0 tx=0x7ff6b8014000 comp rx=0 tx=0).stop 2024-05-09T05:43:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1434031809 shutdown_connections 2024-05-09T05:43:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dc3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dc9240 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1434031809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dc9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1434031809 >> 172.21.15.115:0/1434031809 conn(0x55ac84c49f70 msgr2=0x55ac84dbe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1434031809 shutdown_connections 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1434031809 wait complete. 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.104+0000 7ff6dcc9bb80 1 -- start start 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6dcc9bb80 1 Processor -- start 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ac84dc4160 con 0x55ac84cc0d00 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ac84c80b20 con 0x55ac84c464a0 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac84dd4430 con 0x55ac84dc3400 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6dcc9bb80 10 client.0 _ll_register_callbacks cb 0x55ac84b576c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6c77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6c77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49704/0 (socket says 172.21.15.115:49704) 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6c77fe700 1 -- 172.21.15.115:0/1119522156 learned_addr learned my addr 172.21.15.115:0/1119522156 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6cc822700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.105+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6c77fe700 1 -- 172.21.15.115:0/1119522156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 msgr2=0x55ac84dcda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6c77fe700 1 -- 172.21.15.115:0/1119522156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 msgr2=0x55ac84dccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6c77fe700 1 -- 172.21.15.115:0/1119522156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac84b61490 con 0x55ac84cc0d00 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6cc822700 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcd500 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff6c806c680 tx=0x7ff6bc003720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4999bb93716e053b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6a77fe700 1 -- 172.21.15.115:0/1119522156 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc003bd0 con 0x55ac84cc0d00 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1119522156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b8011e50 con 0x55ac84cc0d00 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.106+0000 7ff6a77fe700 1 -- 172.21.15.115:0/1119522156 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff6bc003d70 con 0x55ac84cc0d00 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6a77fe700 1 -- 172.21.15.115:0/1119522156 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff6bc032050 con 0x55ac84cc0d00 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6dcc9bb80 10 client.16899 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1119522156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ac84ddedf0 con 0x55ac84cc0d00 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6a77fe700 1 -- 172.21.15.115:0/1119522156 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b8011e50 con 0x55ac84cc0d00 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6a77fe700 1 -- 172.21.15.115:0/1119522156 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7ff6bc024670 con 0x55ac84cc0d00 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6dcc9bb80 10 client.16899 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6dcc9bb80 10 client.16899 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6dcc9bb80 20 client.16899 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55ac84de1800 0x55ac84de3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6dcc9bb80 4 client.16899 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.107+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1119522156 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac84cf3a80 con 0x55ac84de1800 2024-05-09T05:43:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.108+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55ac84de1800 0x55ac84de3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.108+0000 7ff6a77fe700 10 client.16899 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.110+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55ac84de1800 0x55ac84de3c50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dc5099d96e0a37d9 server_cookie=f1eb89aff58ea729 in_seq=0 out_seq=0 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.110+0000 7ff6a77fe700 1 -- 172.21.15.115:0/1119522156 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac84cf3a80 con 0x55ac84de1800 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.110+0000 7ff6a77fe700 10 client.16899 handle_command_reply: tid=0 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.110+0000 7ff6dcc9bb80 1 client.16899 shutdown 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1119522156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 msgr2=0x55ac84dcd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcd500 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff6c806c680 tx=0x7ff6bc003720 comp rx=0 tx=0).stop 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1119522156 shutdown_connections 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55ac84de1800 0x55ac84de3c50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/1119522156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcd500 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6a77fe700 0 client.16899 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:43:10.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1119522156 >> 172.21.15.115:0/1119522156 conn(0x55ac84c49f70 msgr2=0x55ac84db7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:10.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1119522156 shutdown_connections 2024-05-09T05:43:10.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/1119522156 wait complete. 2024-05-09T05:43:10.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.111+0000 7ff6dcc9bb80 20 client.16899 trim_cache size 0 max 16384 2024-05-09T05:43:10.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.130+0000 7ff6dcc9bb80 1 Processor -- start 2024-05-09T05:43:10.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.130+0000 7ff6dcc9bb80 1 -- start start 2024-05-09T05:43:10.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dcbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c81ae150 con 0x55ac84cc0d00 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c81afd80 con 0x55ac84c464a0 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c81ae820 con 0x55ac84dc3400 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6c77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dcbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6c77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49716/0 (socket says 172.21.15.115:49716) 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6c77fe700 1 -- 172.21.15.115:0/2594806798 learned_addr learned my addr 172.21.15.115:0/2594806798 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6cc822700 1 --2- 172.21.15.115:0/2594806798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6c77fe700 1 -- 172.21.15.115:0/2594806798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 msgr2=0x55ac84dcd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/2594806798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6c77fe700 1 -- 172.21.15.115:0/2594806798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 msgr2=0x55ac84dcbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/2594806798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dcbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6c77fe700 1 -- 172.21.15.115:0/2594806798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac84dc45d0 con 0x55ac84cc0d00 2024-05-09T05:43:10.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.131+0000 7ff6cc822700 1 --2- 172.21.15.115:0/2594806798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:10.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.132+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/2594806798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcf460 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc000c00 tx=0x7ff6bc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39766dd4b6707cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:10.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.132+0000 7ff6adffb700 1 -- 172.21.15.115:0/2594806798 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc03a020 con 0x55ac84cc0d00 2024-05-09T05:43:10.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.132+0000 7ff6adffb700 1 -- 172.21.15.115:0/2594806798 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6bc029070 con 0x55ac84cc0d00 2024-05-09T05:43:10.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.132+0000 7ff6adffb700 1 -- 172.21.15.115:0/2594806798 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00b990 con 0x55ac84cc0d00 2024-05-09T05:43:10.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2594806798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 msgr2=0x55ac84dcf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:10.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2594806798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcf460 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc000c00 tx=0x7ff6bc009cd0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2594806798 shutdown_connections 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2594806798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84c464a0 0x55ac84dcbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2594806798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84dcd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2594806798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84cc0d00 0x55ac84dcf460 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2594806798 >> 172.21.15.115:0/2594806798 conn(0x55ac84c49f70 msgr2=0x55ac84db7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2594806798 shutdown_connections 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2594806798 wait complete. 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.133+0000 7ff6dcc9bb80 1 -- start start 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6dcc9bb80 1 Processor -- start 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84c464a0 0x55ac84dd9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84cc0d00 0x55ac84dd99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6dcc9bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84ddc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c81af650 con 0x55ac84c464a0 2024-05-09T05:43:10.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c81aeef0 con 0x55ac84cc0d00 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6dcc9bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c81ac0f0 con 0x55ac84dc3400 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84c464a0 0x55ac84dd9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84c464a0 0x55ac84dd9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49726/0 (socket says 172.21.15.115:49726) 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c7fff700 1 -- 172.21.15.115:0/2368718631 learned_addr learned my addr 172.21.15.115:0/2368718631 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6dcc9bb80 10 client.0 _ll_register_callbacks cb 0x55ac84b576c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84cc0d00 0x55ac84dd99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6cc822700 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84ddc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c7fff700 1 -- 172.21.15.115:0/2368718631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 msgr2=0x55ac84ddc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84ddc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c7fff700 1 -- 172.21.15.115:0/2368718631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84cc0d00 msgr2=0x55ac84dd99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84cc0d00 0x55ac84dd99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c7fff700 1 -- 172.21.15.115:0/2368718631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac84ddc620 con 0x55ac84c464a0 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.134+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84cc0d00 0x55ac84dd99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:43:10.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.135+0000 7ff6c7fff700 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84c464a0 0x55ac84dd9470 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8004360 tx=0x7ff6b8003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9834587648689d6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.135+0000 7ff6a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.135+0000 7ff6a77fe700 1 -- 172.21.15.115:0/2368718631 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8037070 con 0x55ac84c464a0 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.135+0000 7ff6a77fe700 1 -- 172.21.15.115:0/2368718631 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7ff6b8005450 con 0x55ac84c464a0 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.135+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2368718631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6bc029070 con 0x55ac84c464a0 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.135+0000 7ff6a77fe700 1 -- 172.21.15.115:0/2368718631 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff6b8043050 con 0x55ac84c464a0 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.135+0000 7ff6dcc9bb80 10 client.16905 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.135+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2368718631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ac84e4b5f0 con 0x55ac84c464a0 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.136+0000 7ff6a77fe700 1 -- 172.21.15.115:0/2368718631 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6bc029070 con 0x55ac84c464a0 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.136+0000 7ff6a77fe700 1 -- 172.21.15.115:0/2368718631 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7ff6b80265a0 con 0x55ac84c464a0 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.136+0000 7ff6dcc9bb80 10 client.16905 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.136+0000 7ff6dcc9bb80 10 client.16905 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.136+0000 7ff6dcc9bb80 20 client.16905 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.136+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55ac84de1800 0x55ac84e4da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:10.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.136+0000 7ff6dcc9bb80 4 client.16905 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:43:10.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.136+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2368718631 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac84db7d40 con 0x55ac84de1800 2024-05-09T05:43:10.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.136+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55ac84de1800 0x55ac84e4da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:10.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6a77fe700 10 client.16905 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:43:10.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6c77fe700 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55ac84de1800 0x55ac84e4da30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b69429ab182954e6 server_cookie=c8b7f98a29e31e44 in_seq=0 out_seq=0 2024-05-09T05:43:10.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6a77fe700 1 -- 172.21.15.115:0/2368718631 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac84db7d40 con 0x55ac84de1800 2024-05-09T05:43:10.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6a77fe700 10 client.16905 handle_command_reply: tid=0 2024-05-09T05:43:10.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6dcc9bb80 1 client.16905 shutdown 2024-05-09T05:43:10.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2368718631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84c464a0 msgr2=0x55ac84dd9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:10.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84c464a0 0x55ac84dd9470 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8004360 tx=0x7ff6b8003bc0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2368718631 shutdown_connections 2024-05-09T05:43:10.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55ac84de1800 0x55ac84e4da30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac84dc3400 0x55ac84ddc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac84c464a0 0x55ac84dd9470 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6dcc9bb80 1 --2- 172.21.15.115:0/2368718631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac84cc0d00 0x55ac84dd99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6a77fe700 0 client.16905 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:43:10.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.137+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2368718631 >> 172.21.15.115:0/2368718631 conn(0x55ac84c49f70 msgr2=0x55ac84dc6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:10.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.138+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2368718631 shutdown_connections 2024-05-09T05:43:10.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.138+0000 7ff6dcc9bb80 1 -- 172.21.15.115:0/2368718631 wait complete. 2024-05-09T05:43:10.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.138+0000 7ff6dcc9bb80 20 client.16905 trim_cache size 0 max 16384 2024-05-09T05:43:10.135 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:43:10.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.138+0000 7ff6ce285700 1 -- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6b00b2a70 msgr2=0x7ff6b00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:10.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.138+0000 7ff6ce285700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6b00b2a70 0x7ff6b00b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8011490 tx=0x7ff6b8009260 comp rx=0 tx=0).stop 2024-05-09T05:43:10.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.138+0000 7ff6ce285700 1 -- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8109870 msgr2=0x7ff6c819f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:10.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.138+0000 7ff6ce285700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8109870 0x7ff6c819f620 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0007ee0 tx=0x7ff6c000d3b0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.139+0000 7ff6ce285700 1 -- 172.21.15.115:0/2705383024 shutdown_connections 2024-05-09T05:43:10.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.139+0000 7ff6ce285700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff6b00b2a70 0x7ff6b00b4f80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.139+0000 7ff6ce285700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6c8109870 0x7ff6c819f620 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.139+0000 7ff6ce285700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6c81045f0 0x7ff6c819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.139+0000 7ff6ce285700 1 --2- 172.21.15.115:0/2705383024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6c8108f20 0x7ff6c819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:10.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.139+0000 7ff6ce285700 1 -- 172.21.15.115:0/2705383024 >> 172.21.15.115:0/2705383024 conn(0x7ff6c8100040 msgr2=0x7ff6c810a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:10.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.139+0000 7ff6ce285700 1 -- 172.21.15.115:0/2705383024 shutdown_connections 2024-05-09T05:43:10.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:10.139+0000 7ff6ce285700 1 -- 172.21.15.115:0/2705383024 wait complete. 2024-05-09T05:43:10.875 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:43:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:11.575 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:11 smithi115 ceph-mon[30349]: pgmap v152: 129 pgs: 129 active+clean; 3.1 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 34 op/s 2024-05-09T05:43:11.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:11 smithi195 ceph-mon[33844]: pgmap v152: 129 pgs: 129 active+clean; 3.1 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 34 op/s 2024-05-09T05:43:11.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:11 smithi161 ceph-mon[33805]: pgmap v152: 129 pgs: 129 active+clean; 3.1 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 34 op/s 2024-05-09T05:43:13.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.163+0000 7f33eb286700 1 -- 172.21.15.115:0/600103416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 msgr2=0x7f33e4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:13.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.163+0000 7f33eb286700 1 --2- 172.21.15.115:0/600103416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 0x7f33e4108200 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f33d40099c0 tx=0x7f33d4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.163+0000 7f33eb286700 1 -- 172.21.15.115:0/600103416 shutdown_connections 2024-05-09T05:43:13.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.163+0000 7f33eb286700 1 --2- 172.21.15.115:0/600103416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e4108740 0x7f33e41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.163+0000 7f33eb286700 1 --2- 172.21.15.115:0/600103416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33e41034c0 0x7f33e41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.163+0000 7f33eb286700 1 --2- 172.21.15.115:0/600103416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 0x7f33e4108200 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.163+0000 7f33eb286700 1 -- 172.21.15.115:0/600103416 >> 172.21.15.115:0/600103416 conn(0x7f33e40fedc0 msgr2=0x7f33e41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:13.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.163+0000 7f33eb286700 1 -- 172.21.15.115:0/600103416 shutdown_connections 2024-05-09T05:43:13.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.163+0000 7f33eb286700 1 -- 172.21.15.115:0/600103416 wait complete. 2024-05-09T05:43:13.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33eb286700 1 Processor -- start 2024-05-09T05:43:13.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33eb286700 1 -- start start 2024-05-09T05:43:13.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33eb286700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e41034c0 0x7f33e40ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33eb286700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 0x7f33e40ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33eb286700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33e4108740 0x7f33e40ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33eb286700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113200 con 0x7f33e4107df0 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33eb286700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113080 con 0x7f33e41034c0 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33eb286700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113380 con 0x7f33e4108740 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33e8821700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 0x7f33e40ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33e8821700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 0x7f33e40ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51066/0 (socket says 172.21.15.115:51066) 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33e8821700 1 -- 172.21.15.115:0/3142602077 learned_addr learned my addr 172.21.15.115:0/3142602077 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.164+0000 7f33e9823700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33e4108740 0x7f33e40ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33e9022700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e41034c0 0x7f33e40ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33e8821700 1 -- 172.21.15.115:0/3142602077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33e4108740 msgr2=0x7f33e40ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33e8821700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33e4108740 0x7f33e40ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33e8821700 1 -- 172.21.15.115:0/3142602077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e41034c0 msgr2=0x7f33e40ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33e8821700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e41034c0 0x7f33e40ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33e8821700 1 -- 172.21.15.115:0/3142602077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e41a9080 con 0x7f33e4107df0 2024-05-09T05:43:13.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33e8821700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 0x7f33e40ff7e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f33d400b580 tx=0x7f33d40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:13.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33da7fc700 1 -- 172.21.15.115:0/3142602077 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4027070 con 0x7f33e4107df0 2024-05-09T05:43:13.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33da7fc700 1 -- 172.21.15.115:0/3142602077 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33d400fd30 con 0x7f33e4107df0 2024-05-09T05:43:13.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33eb286700 1 -- 172.21.15.115:0/3142602077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e41a9310 con 0x7f33e4107df0 2024-05-09T05:43:13.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33da7fc700 1 -- 172.21.15.115:0/3142602077 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d40173f0 con 0x7f33e4107df0 2024-05-09T05:43:13.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.165+0000 7f33eb286700 1 -- 172.21.15.115:0/3142602077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41a9760 con 0x7f33e4107df0 2024-05-09T05:43:13.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.169+0000 7f33da7fc700 1 -- 172.21.15.115:0/3142602077 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f33d402a070 con 0x7f33e4107df0 2024-05-09T05:43:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.169+0000 7f33f9c9cb80 1 Processor -- start 2024-05-09T05:43:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33f9c9cb80 1 -- start start 2024-05-09T05:43:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33da7fc700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33d00b29f0 0x7f33d00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb262c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb24a94a0 0x55abb262ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb26266a0 0x55abb2626ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55abb2548ff0 con 0x55abb24a94a0 2024-05-09T05:43:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55abb24e77f0 con 0x55abb2593160 2024-05-09T05:43:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55abb2544fb0 con 0x55abb26266a0 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33e9823700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33d00b29f0 0x7f33d00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33da7fc700 1 -- 172.21.15.115:0/3142602077 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f33d40e37b0 con 0x7f33e4107df0 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33e8821700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb24a94a0 0x55abb262ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33e8821700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb24a94a0 0x55abb262ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51082/0 (socket says 172.21.15.115:51082) 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33e8821700 1 -- 172.21.15.115:0/2982312429 learned_addr learned my addr 172.21.15.115:0/2982312429 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.170+0000 7f33e9022700 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb262c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33e9823700 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb26266a0 0x55abb2626ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33e8821700 1 -- 172.21.15.115:0/2982312429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb26266a0 msgr2=0x55abb2626ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33e8821700 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb26266a0 0x55abb2626ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33e8821700 1 -- 172.21.15.115:0/2982312429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 msgr2=0x55abb262c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33e8821700 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb262c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33e8821700 1 -- 172.21.15.115:0/2982312429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb24df370 con 0x55abb24a94a0 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33e9022700 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb262c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:43:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33e9823700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33d00b29f0 0x7f33d00b4f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f33dc003ee0 tx=0x7f33dc009100 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:13.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33e8821700 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb24a94a0 0x55abb262ca30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f33d40a6860 tx=0x7f33d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7b097a3b6d3c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:13.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33cf7fe700 1 -- 172.21.15.115:0/2982312429 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d40aa2f0 con 0x55abb24a94a0 2024-05-09T05:43:13.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33cf7fe700 1 -- 172.21.15.115:0/2982312429 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33d40aaee0 con 0x55abb24a94a0 2024-05-09T05:43:13.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.171+0000 7f33cf7fe700 1 -- 172.21.15.115:0/2982312429 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d40aa7e0 con 0x55abb24a94a0 2024-05-09T05:43:13.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.172+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/2982312429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb24a94a0 msgr2=0x55abb262ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:13.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.172+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb24a94a0 0x55abb262ca30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f33d40a6860 tx=0x7f33d400cf90 comp rx=0 tx=0).stop 2024-05-09T05:43:13.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.172+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/2982312429 shutdown_connections 2024-05-09T05:43:13.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.172+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb262c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.172+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb26266a0 0x55abb2626ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.172+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/2982312429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb24a94a0 0x55abb262ca30 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.172+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/2982312429 >> 172.21.15.115:0/2982312429 conn(0x55abb25096a0 msgr2=0x55abb2621400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:13.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.172+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/2982312429 shutdown_connections 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.172+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/2982312429 wait complete. 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.173+0000 7f33f9c9cb80 1 -- start start 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.173+0000 7f33f9c9cb80 1 Processor -- start 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.173+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb24a94a0 0x55abb2635af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.173+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb2593160 0x55abb2636030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.173+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb262fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.173+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55abb23c4ba0 con 0x55abb26266a0 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.173+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55abb2503150 con 0x55abb24a94a0 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.173+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55abb2548ff0 con 0x55abb2593160 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33f9c9cb80 10 client.0 _ll_register_callbacks cb 0x55abb23ba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9823700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb262fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9823700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb262fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51086/0 (socket says 172.21.15.115:51086) 2024-05-09T05:43:13.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9823700 1 -- 172.21.15.115:0/1002267823 learned_addr learned my addr 172.21.15.115:0/1002267823 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e8821700 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb2593160 0x55abb2636030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9022700 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb24a94a0 0x55abb2635af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9823700 1 -- 172.21.15.115:0/1002267823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb2593160 msgr2=0x55abb2636030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9823700 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb2593160 0x55abb2636030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9823700 1 -- 172.21.15.115:0/1002267823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb24a94a0 msgr2=0x55abb2635af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9823700 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb24a94a0 0x55abb2635af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9823700 1 -- 172.21.15.115:0/1002267823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb23c4490 con 0x55abb26266a0 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33e9823700 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb262fef0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f33dc008790 tx=0x7f33dc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2370abe2ea6d90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33ccff9700 1 -- 172.21.15.115:0/1002267823 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc025030 con 0x55abb26266a0 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33ccff9700 1 -- 172.21.15.115:0/1002267823 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f33dc01b070 con 0x55abb26266a0 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.174+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1002267823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33d40aaee0 con 0x55abb26266a0 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33ccff9700 1 -- 172.21.15.115:0/1002267823 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f33dc02f050 con 0x55abb26266a0 2024-05-09T05:43:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33f9c9cb80 10 client.16917 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1002267823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55abb263d6e0 con 0x55abb26266a0 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33ccff9700 1 -- 172.21.15.115:0/1002267823 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33dc004bb0 con 0x55abb26266a0 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33ccff9700 1 -- 172.21.15.115:0/1002267823 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f33dc020070 con 0x55abb26266a0 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33f9c9cb80 10 client.16917 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33f9c9cb80 10 client.16917 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33f9c9cb80 20 client.16917 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55abb263fd70 0x55abb2642190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33f9c9cb80 4 client.16917 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.175+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1002267823 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f33d40aaee0 con 0x55abb263fd70 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.176+0000 7f33e9022700 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55abb263fd70 0x55abb2642190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.176+0000 7f33ccff9700 10 client.16917 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:43:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.176+0000 7f33e9022700 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55abb263fd70 0x55abb2642190 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3f6326f00fd2d77 server_cookie=53a658764d5b509 in_seq=0 out_seq=0 2024-05-09T05:43:13.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33ccff9700 1 -- 172.21.15.115:0/1002267823 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f33d40aaee0 con 0x55abb263fd70 2024-05-09T05:43:13.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33ccff9700 10 client.16917 handle_command_reply: tid=0 2024-05-09T05:43:13.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 client.16917 shutdown 2024-05-09T05:43:13.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1002267823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 msgr2=0x55abb262fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb262fef0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f33dc008790 tx=0x7f33dc004b40 comp rx=0 tx=0).stop 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1002267823 shutdown_connections 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb24a94a0 0x55abb2635af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55abb263fd70 0x55abb2642190 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb262fef0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1002267823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb2593160 0x55abb2636030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1002267823 >> 172.21.15.115:0/1002267823 conn(0x55abb25096a0 msgr2=0x55abb26211c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33ccff9700 0 client.16917 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.177+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1002267823 shutdown_connections 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.178+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1002267823 wait complete. 2024-05-09T05:43:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.178+0000 7f33f9c9cb80 20 client.16917 trim_cache size 0 max 16384 2024-05-09T05:43:13.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.196+0000 7f33f9c9cb80 1 Processor -- start 2024-05-09T05:43:13.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.196+0000 7f33f9c9cb80 1 -- start start 2024-05-09T05:43:13.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb2647950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb2647e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb2629e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b2450 con 0x55abb26266a0 2024-05-09T05:43:13.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b4080 con 0x55abb2593160 2024-05-09T05:43:13.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b2b20 con 0x55abb24a94a0 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9022700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb2647950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9022700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb2647950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:38250/0 (socket says 172.21.15.115:38250) 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9022700 1 -- 172.21.15.115:0/1171673361 learned_addr learned my addr 172.21.15.115:0/1171673361 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9823700 1 --2- 172.21.15.115:0/1171673361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb2629e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e8821700 1 --2- 172.21.15.115:0/1171673361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb2647e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9022700 1 -- 172.21.15.115:0/1171673361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 msgr2=0x55abb2647e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9022700 1 --2- 172.21.15.115:0/1171673361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb2647e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9022700 1 -- 172.21.15.115:0/1171673361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 msgr2=0x55abb2629e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9022700 1 --2- 172.21.15.115:0/1171673361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb2629e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9022700 1 -- 172.21.15.115:0/1171673361 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb2627640 con 0x55abb24a94a0 2024-05-09T05:43:13.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.197+0000 7f33e9022700 1 --2- 172.21.15.115:0/1171673361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb2647950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f33e000cfd0 tx=0x7f33e000c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0e09b79707f94c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:13.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.198+0000 7f33ceffd700 1 -- 172.21.15.115:0/1171673361 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0005020 con 0x55abb24a94a0 2024-05-09T05:43:13.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.198+0000 7f33ceffd700 1 -- 172.21.15.115:0/1171673361 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33e0005650 con 0x55abb24a94a0 2024-05-09T05:43:13.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.198+0000 7f33ceffd700 1 -- 172.21.15.115:0/1171673361 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0003b20 con 0x55abb24a94a0 2024-05-09T05:43:13.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1171673361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 msgr2=0x55abb2647950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:13.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1171673361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb2647950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f33e000cfd0 tx=0x7f33e000c690 comp rx=0 tx=0).stop 2024-05-09T05:43:13.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1171673361 shutdown_connections 2024-05-09T05:43:13.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1171673361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb2629e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1171673361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb2647950 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/1171673361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb2647e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1171673361 >> 172.21.15.115:0/1171673361 conn(0x55abb25096a0 msgr2=0x55abb262e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1171673361 shutdown_connections 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/1171673361 wait complete. 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 -- start start 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.199+0000 7f33f9c9cb80 1 Processor -- start 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb26a2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb26a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33f9c9cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb2639470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b3950 con 0x55abb26266a0 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b31f0 con 0x55abb2593160 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33f9c9cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b03f0 con 0x55abb24a94a0 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33f9c9cb80 10 client.0 _ll_register_callbacks cb 0x55abb23ba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33e9022700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb26a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33e9022700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb26a2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:38254/0 (socket says 172.21.15.115:38254) 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33e9022700 1 -- 172.21.15.115:0/4106226132 learned_addr learned my addr 172.21.15.115:0/4106226132 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33e9823700 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb2639470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.200+0000 7f33e8821700 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb26a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33e9022700 1 -- 172.21.15.115:0/4106226132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 msgr2=0x55abb26a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33e9022700 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb26a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33e9022700 1 -- 172.21.15.115:0/4106226132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 msgr2=0x55abb2639470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33e9022700 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb2639470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33e9022700 1 -- 172.21.15.115:0/4106226132 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb261af60 con 0x55abb24a94a0 2024-05-09T05:43:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33e9022700 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb26a2900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f33e000cd70 tx=0x7f33e000cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601760ba3d42ce93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:43:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33ccff9700 1 -- 172.21.15.115:0/4106226132 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0029020 con 0x55abb24a94a0 2024-05-09T05:43:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/4106226132 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55abb25569d0 con 0x55abb24a94a0 2024-05-09T05:43:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33ccff9700 1 -- 172.21.15.115:0/4106226132 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f33e00047c0 con 0x55abb24a94a0 2024-05-09T05:43:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33ccff9700 1 -- 172.21.15.115:0/4106226132 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f33e0034050 con 0x55abb24a94a0 2024-05-09T05:43:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33f9c9cb80 10 client.26425 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.201+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/4106226132 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55abb26a9df0 con 0x55abb24a94a0 2024-05-09T05:43:13.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33ccff9700 1 -- 172.21.15.115:0/4106226132 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55abb25569d0 con 0x55abb24a94a0 2024-05-09T05:43:13.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33ccff9700 1 -- 172.21.15.115:0/4106226132 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f33e000f040 con 0x55abb24a94a0 2024-05-09T05:43:13.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33f9c9cb80 10 client.26425 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:13.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33f9c9cb80 10 client.26425 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:43:13.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33f9c9cb80 20 client.26425 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:13.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55abb26aa4a0 0x55abb26ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:13.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33f9c9cb80 4 client.26425 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:43:13.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/4106226132 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abb2550a70 con 0x55abb26aa4a0 2024-05-09T05:43:13.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33e8821700 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55abb26aa4a0 0x55abb26ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:13.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.202+0000 7f33ccff9700 10 client.26425 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33e8821700 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55abb26aa4a0 0x55abb26ac8c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=775c40e1b5e02a02 server_cookie=55f3f915b0fda120 in_seq=0 out_seq=0 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33ccff9700 1 -- 172.21.15.115:0/4106226132 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abb2550a70 con 0x55abb26aa4a0 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33ccff9700 10 client.26425 handle_command_reply: tid=0 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 client.26425 shutdown 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/4106226132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 msgr2=0x55abb26a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb26a2900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f33e000cd70 tx=0x7f33e000cda0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/4106226132 shutdown_connections 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55abb26aa4a0 0x55abb26ac8c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55abb26266a0 0x55abb2639470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55abb24a94a0 0x55abb26a2900 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 --2- 172.21.15.115:0/4106226132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55abb2593160 0x55abb26a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/4106226132 >> 172.21.15.115:0/4106226132 conn(0x55abb25096a0 msgr2=0x55abb262aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33ccff9700 0 client.26425 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/4106226132 shutdown_connections 2024-05-09T05:43:13.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 1 -- 172.21.15.115:0/4106226132 wait complete. 2024-05-09T05:43:13.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.203+0000 7f33f9c9cb80 20 client.26425 trim_cache size 0 max 16384 2024-05-09T05:43:13.201 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:43:13.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 -- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33d00b29f0 msgr2=0x7f33d00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:13.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33d00b29f0 0x7f33d00b4f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f33dc003ee0 tx=0x7f33dc009100 comp rx=0 tx=0).stop 2024-05-09T05:43:13.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 -- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 msgr2=0x7f33e40ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:13.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 0x7f33e40ff7e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f33d400b580 tx=0x7f33d40057c0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 -- 172.21.15.115:0/3142602077 shutdown_connections 2024-05-09T05:43:13.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33d00b29f0 0x7f33d00b4f00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e41034c0 0x7f33e40ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33e4108740 0x7f33e40ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 --2- 172.21.15.115:0/3142602077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e4107df0 0x7f33e40ff7e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:13.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 -- 172.21.15.115:0/3142602077 >> 172.21.15.115:0/3142602077 conn(0x7f33e40fedc0 msgr2=0x7f33e406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:13.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 -- 172.21.15.115:0/3142602077 shutdown_connections 2024-05-09T05:43:13.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:13.204+0000 7f33eb286700 1 -- 172.21.15.115:0/3142602077 wait complete. 2024-05-09T05:43:13.480 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.5 2024-05-09T05:43:13.480 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:43:13.760 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:13 smithi115 ceph-mon[30349]: pgmap v153: 129 pgs: 129 active+clean; 3.1 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 34 op/s 2024-05-09T05:43:13.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:13 smithi195 ceph-mon[33844]: pgmap v153: 129 pgs: 129 active+clean; 3.1 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 34 op/s 2024-05-09T05:43:13.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:13 smithi161 ceph-mon[33805]: pgmap v153: 129 pgs: 129 active+clean; 3.1 GiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 34 op/s 2024-05-09T05:43:13.959 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:43:14.089 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:15.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:15 smithi115 ceph-mon[30349]: pgmap v154: 129 pgs: 129 active+clean; 3.3 GiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 43 op/s 2024-05-09T05:43:15.716 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:15.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:15 smithi195 ceph-mon[33844]: pgmap v154: 129 pgs: 129 active+clean; 3.3 GiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 43 op/s 2024-05-09T05:43:15.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:15 smithi161 ceph-mon[33805]: pgmap v154: 129 pgs: 129 active+clean; 3.3 GiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 43 op/s 2024-05-09T05:43:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.236+0000 7fcea2c4c700 1 -- 172.21.15.115:0/907632901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 msgr2=0x7fce9c102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.236+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/907632901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 0x7fce9c102dd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fce88009990 tx=0x7fce88009ca0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.236+0000 7fcea2c4c700 1 -- 172.21.15.115:0/907632901 shutdown_connections 2024-05-09T05:43:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.236+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/907632901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce9c1089f0 0x7fce9c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.236+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/907632901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce9c103310 0x7fce9c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.236+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/907632901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 0x7fce9c102dd0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.236+0000 7fcea2c4c700 1 -- 172.21.15.115:0/907632901 >> 172.21.15.115:0/907632901 conn(0x7fce9c0fe6f0 msgr2=0x7fce9c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.236+0000 7fcea2c4c700 1 -- 172.21.15.115:0/907632901 shutdown_connections 2024-05-09T05:43:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.236+0000 7fcea2c4c700 1 -- 172.21.15.115:0/907632901 wait complete. 2024-05-09T05:43:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea2c4c700 1 Processor -- start 2024-05-09T05:43:16.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea2c4c700 1 -- start start 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea2c4c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 0x7fce9c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea2c4c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce9c103310 0x7fce9c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea2c4c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce9c1089f0 0x7fce9c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea2c4c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c112190 con 0x7fce9c1029c0 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea2c4c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c112010 con 0x7fce9c103310 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea2c4c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c112310 con 0x7fce9c1089f0 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea09e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 0x7fce9c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea09e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 0x7fce9c19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51122/0 (socket says 172.21.15.115:51122) 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fcea09e8700 1 -- 172.21.15.115:0/1802409606 learned_addr learned my addr 172.21.15.115:0/1802409606 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.237+0000 7fce9bfff700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce9c103310 0x7fce9c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fcea11e9700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce9c1089f0 0x7fce9c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fcea09e8700 1 -- 172.21.15.115:0/1802409606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce9c1089f0 msgr2=0x7fce9c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fcea09e8700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce9c1089f0 0x7fce9c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fcea09e8700 1 -- 172.21.15.115:0/1802409606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce9c103310 msgr2=0x7fce9c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fcea09e8700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce9c103310 0x7fce9c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fcea09e8700 1 -- 172.21.15.115:0/1802409606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c199100 con 0x7fce9c1029c0 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fcea09e8700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 0x7fce9c19e830 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fce9000fdf0 tx=0x7fce9000d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fce99ffb700 1 -- 172.21.15.115:0/1802409606 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90004a30 con 0x7fce9c1029c0 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fce99ffb700 1 -- 172.21.15.115:0/1802409606 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce9000de30 con 0x7fce9c1029c0 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.238+0000 7fce99ffb700 1 -- 172.21.15.115:0/1802409606 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90018640 con 0x7fce9c1029c0 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.239+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c1993f0 con 0x7fce9c1029c0 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.239+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c1a56c0 con 0x7fce9c1029c0 2024-05-09T05:43:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.240+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9c0512c0 con 0x7fce9c1029c0 2024-05-09T05:43:16.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.242+0000 7fce99ffb700 1 -- 172.21.15.115:0/1802409606 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fce90005000 con 0x7fce9c1029c0 2024-05-09T05:43:16.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.243+0000 7fce99ffb700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fce8c0b2a70 0x7fce8c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.243+0000 7fce9bfff700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fce8c0b2a70 0x7fce8c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.243+0000 7fce99ffb700 1 -- 172.21.15.115:0/1802409606 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fce900e33b0 con 0x7fce9c1029c0 2024-05-09T05:43:16.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.243+0000 7fce9bfff700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fce8c0b2a70 0x7fce8c0b4f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce880059d0 tx=0x7fce88018040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:16.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.247+0000 7fce99ffb700 1 -- 172.21.15.115:0/1802409606 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce90014070 con 0x7fce9c1029c0 2024-05-09T05:43:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.481+0000 7f0387e70640 1 -- 172.21.15.115:0/1615159272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380108ba0 msgr2=0x7f0380102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.481+0000 7f0387e70640 1 --2- 172.21.15.115:0/1615159272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380108ba0 0x7f0380102740 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f037c009900 tx=0x7f037c02a410 comp rx=0 tx=0).stop 2024-05-09T05:43:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.482+0000 7f0387e70640 1 -- 172.21.15.115:0/1615159272 shutdown_connections 2024-05-09T05:43:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.482+0000 7f0387e70640 1 --2- 172.21.15.115:0/1615159272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03801032f0 0x7f038010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.482+0000 7f0387e70640 1 --2- 172.21.15.115:0/1615159272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0380109040 0x7f0380102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.482+0000 7f0387e70640 1 --2- 172.21.15.115:0/1615159272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380108ba0 0x7f0380102740 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.482+0000 7f0387e70640 1 -- 172.21.15.115:0/1615159272 >> 172.21.15.115:0/1615159272 conn(0x7f03800782e0 msgr2=0x7f0380078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.482+0000 7f0387e70640 1 -- 172.21.15.115:0/1615159272 shutdown_connections 2024-05-09T05:43:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.482+0000 7f0387e70640 1 -- 172.21.15.115:0/1615159272 wait complete. 2024-05-09T05:43:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.482+0000 7f0387e70640 1 Processor -- start 2024-05-09T05:43:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f0387e70640 1 -- start start 2024-05-09T05:43:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f0387e70640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03801032f0 0x7f0380199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f0387e70640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0380108ba0 0x7f0380199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f0387e70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380109040 0x7f038019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f0387e70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0380112c40 con 0x7f0380109040 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f0387e70640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0380112ac0 con 0x7f0380108ba0 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f0387e70640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0380112dc0 con 0x7f03801032f0 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038640d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380109040 0x7f038019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038540b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0380108ba0 0x7f0380199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038640d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380109040 0x7f038019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51138/0 (socket says 172.21.15.115:51138) 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038640d640 1 -- 172.21.15.115:0/1250315193 learned_addr learned my addr 172.21.15.115:0/1250315193 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f0385c0c640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03801032f0 0x7f0380199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038640d640 1 -- 172.21.15.115:0/1250315193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03801032f0 msgr2=0x7f0380199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038640d640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03801032f0 0x7f0380199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038640d640 1 -- 172.21.15.115:0/1250315193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0380108ba0 msgr2=0x7f0380199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038640d640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0380108ba0 0x7f0380199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038640d640 1 -- 172.21.15.115:0/1250315193 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038019e560 con 0x7f0380109040 2024-05-09T05:43:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.483+0000 7f038540b640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0380108ba0 0x7f0380199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.484+0000 7f038640d640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380109040 0x7f038019df20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f037800eb10 tx=0x7f037800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.484+0000 7f0376ffd640 1 -- 172.21.15.115:0/1250315193 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378004b20 con 0x7f0380109040 2024-05-09T05:43:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.484+0000 7f0387e70640 1 -- 172.21.15.115:0/1250315193 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038019e850 con 0x7f0380109040 2024-05-09T05:43:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.484+0000 7f0376ffd640 1 -- 172.21.15.115:0/1250315193 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0378004cc0 con 0x7f0380109040 2024-05-09T05:43:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.484+0000 7f0376ffd640 1 -- 172.21.15.115:0/1250315193 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378010690 con 0x7f0380109040 2024-05-09T05:43:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.484+0000 7f0387e70640 1 -- 172.21.15.115:0/1250315193 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038006a4f0 con 0x7f0380109040 2024-05-09T05:43:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.488+0000 7f0376ffd640 1 -- 172.21.15.115:0/1250315193 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0378005020 con 0x7f0380109040 2024-05-09T05:43:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.488+0000 7f038ee00740 1 Processor -- start 2024-05-09T05:43:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.488+0000 7f038ee00740 1 -- start start 2024-05-09T05:43:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.488+0000 7f038ee00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182cf1b0 0x55fd186724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.488+0000 7f038ee00740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd18672a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.488+0000 7f038ee00740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182ae4d0 0x55fd18672f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.488+0000 7f038ee00740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fd185d77c0 con 0x55fd182a62f0 2024-05-09T05:43:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.488+0000 7f038ee00740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fd185730e0 con 0x55fd182ae4d0 2024-05-09T05:43:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.488+0000 7f038ee00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fd185d3780 con 0x55fd182cf1b0 2024-05-09T05:43:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f0385c0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd18672a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f0385c0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd18672a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51140/0 (socket says 172.21.15.115:51140) 2024-05-09T05:43:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f0385c0c640 1 -- 172.21.15.115:0/764450112 learned_addr learned my addr 172.21.15.115:0/764450112 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f038540b640 1 --2- 172.21.15.115:0/764450112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182cf1b0 0x55fd186724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f038540b640 1 --2- 172.21.15.115:0/764450112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182ae4d0 0x55fd18672f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f0385c0c640 1 -- 172.21.15.115:0/764450112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182cf1b0 msgr2=0x55fd186724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f0385c0c640 1 --2- 172.21.15.115:0/764450112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182cf1b0 0x55fd186724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f0385c0c640 1 -- 172.21.15.115:0/764450112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182ae4d0 msgr2=0x55fd18672f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f0385c0c640 1 --2- 172.21.15.115:0/764450112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182ae4d0 0x55fd18672f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f0385c0c640 1 -- 172.21.15.115:0/764450112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd1859c7c0 con 0x55fd182a62f0 2024-05-09T05:43:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.489+0000 7f0385c0c640 1 --2- 172.21.15.115:0/764450112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd18672a30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f037c0057c0 tx=0x7f037c031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b43f7c2696ee5d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.490+0000 7f036bfff640 1 -- 172.21.15.115:0/764450112 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c035030 con 0x55fd182a62f0 2024-05-09T05:43:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.490+0000 7f036bfff640 1 -- 172.21.15.115:0/764450112 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f037c040070 con 0x55fd182a62f0 2024-05-09T05:43:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.490+0000 7f036bfff640 1 -- 172.21.15.115:0/764450112 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c003850 con 0x55fd182a62f0 2024-05-09T05:43:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.490+0000 7f0376ffd640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f036c0b29e0 0x7f036c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.490+0000 7f038540b640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f036c0b29e0 0x7f036c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.490+0000 7f0376ffd640 1 -- 172.21.15.115:0/1250315193 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f03780e2e80 con 0x7f0380109040 2024-05-09T05:43:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 -- 172.21.15.115:0/764450112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 msgr2=0x55fd18672a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 --2- 172.21.15.115:0/764450112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd18672a30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f037c0057c0 tx=0x7f037c031af0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 -- 172.21.15.115:0/764450112 shutdown_connections 2024-05-09T05:43:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 --2- 172.21.15.115:0/764450112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182ae4d0 0x55fd18672f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 --2- 172.21.15.115:0/764450112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd18672a30 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 --2- 172.21.15.115:0/764450112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182cf1b0 0x55fd186724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 -- 172.21.15.115:0/764450112 >> 172.21.15.115:0/764450112 conn(0x55fd182c3300 msgr2=0x55fd1866ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 -- 172.21.15.115:0/764450112 shutdown_connections 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 -- 172.21.15.115:0/764450112 wait complete. 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038540b640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f036c0b29e0 0x7f036c0b4ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55fd1866fd50 tx=0x7f037000bf90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.491+0000 7f038ee00740 1 -- start start 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038ee00740 1 Processor -- start 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038ee00740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd186810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038ee00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182ae4d0 0x55fd18681620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038ee00740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd18687070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038ee00740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fd18683bc0 con 0x55fd182a62f0 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038ee00740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fd1855e9c0 con 0x55fd182cf1b0 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038ee00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fd18683d40 con 0x55fd182ae4d0 2024-05-09T05:43:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038ee00740 10 client.0 _ll_register_callbacks cb 0x55fd1858b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f0385c0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd186810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f0385c0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd186810e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51144/0 (socket says 172.21.15.115:51144) 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f0385c0c640 1 -- 172.21.15.115:0/3058411672 learned_addr learned my addr 172.21.15.115:0/3058411672 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038640d640 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd18687070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.492+0000 7f038540b640 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182ae4d0 0x55fd18681620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f0385c0c640 1 -- 172.21.15.115:0/3058411672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182ae4d0 msgr2=0x55fd18681620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f0385c0c640 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182ae4d0 0x55fd18681620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f0385c0c640 1 -- 172.21.15.115:0/3058411672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 msgr2=0x55fd18687070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f0385c0c640 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd18687070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f0385c0c640 1 -- 172.21.15.115:0/3058411672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd185851f0 con 0x55fd182a62f0 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f038640d640 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd18687070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f0385c0c640 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd186810e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f037c0095a0 tx=0x7f037c0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba516639831271c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f03697fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f03697fa640 1 -- 172.21.15.115:0/3058411672 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c04c070 con 0x55fd182a62f0 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f03697fa640 1 -- 172.21.15.115:0/3058411672 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f037c03c440 con 0x55fd182a62f0 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f038ee00740 1 -- 172.21.15.115:0/3058411672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f037c040070 con 0x55fd182a62f0 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f03697fa640 1 -- 172.21.15.115:0/3058411672 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f037c058050 con 0x55fd182a62f0 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f038ee00740 10 client.16941 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.493+0000 7f038ee00740 1 -- 172.21.15.115:0/3058411672 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fd1833a790 con 0x55fd182a62f0 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.494+0000 7f03697fa640 1 -- 172.21.15.115:0/3058411672 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f037c040070 con 0x55fd182a62f0 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.494+0000 7f03697fa640 1 -- 172.21.15.115:0/3058411672 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f037c044740 con 0x55fd182a62f0 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.494+0000 7f038ee00740 10 client.16941 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.494+0000 7f038ee00740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.494+0000 7f038ee00740 20 client.16941 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.494+0000 7f038ee00740 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fd1828dd80 0x55fd18693130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.494+0000 7f038ee00740 4 client.16941 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.494+0000 7f038ee00740 1 -- 172.21.15.115:0/3058411672 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd185e53d0 con 0x55fd1828dd80 2024-05-09T05:43:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.494+0000 7f038540b640 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fd1828dd80 0x55fd18693130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.495+0000 7f03697fa640 10 client.16941 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.495+0000 7f038540b640 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fd1828dd80 0x55fd18693130 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a33de96be1ca8c server_cookie=e41bff22c3aa2397 in_seq=0 out_seq=0 2024-05-09T05:43:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.495+0000 7f03697fa640 1 -- 172.21.15.115:0/3058411672 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fd185e53d0 con 0x55fd1828dd80 2024-05-09T05:43:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.495+0000 7f03697fa640 10 client.16941 handle_command_reply: tid=0 2024-05-09T05:43:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.495+0000 7f038ee00740 1 client.16941 shutdown 2024-05-09T05:43:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 -- 172.21.15.115:0/3058411672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 msgr2=0x55fd186810e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd186810e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f037c0095a0 tx=0x7f037c0095d0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 -- 172.21.15.115:0/3058411672 shutdown_connections 2024-05-09T05:43:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd18687070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fd1828dd80 0x55fd18693130 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182ae4d0 0x55fd18681620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 --2- 172.21.15.115:0/3058411672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182a62f0 0x55fd186810e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f03697fa640 0 client.16941 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 -- 172.21.15.115:0/3058411672 >> 172.21.15.115:0/3058411672 conn(0x55fd182c3300 msgr2=0x55fd18674290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 -- 172.21.15.115:0/3058411672 shutdown_connections 2024-05-09T05:43:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 1 -- 172.21.15.115:0/3058411672 wait complete. 2024-05-09T05:43:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.496+0000 7f038ee00740 20 client.16941 trim_cache size 0 max 16384 2024-05-09T05:43:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038ee00740 1 Processor -- start 2024-05-09T05:43:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038ee00740 1 -- start start 2024-05-09T05:43:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038ee00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 0x55fd18672e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038ee00740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd18682d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038ee00740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd18683240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038ee00740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f03801aef80 con 0x55fd182ae4d0 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038ee00740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f03801b0bb0 con 0x55fd182cf1b0 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038ee00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03801af650 con 0x55fd182a62f0 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038540b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd18682d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038540b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd18682d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51148/0 (socket says 172.21.15.115:51148) 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038540b640 1 -- 172.21.15.115:0/629166227 learned_addr learned my addr 172.21.15.115:0/629166227 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f038640d640 1 --2- 172.21.15.115:0/629166227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd18683240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.513+0000 7f0385c0c640 1 --2- 172.21.15.115:0/629166227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 0x55fd18672e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.514+0000 7f038540b640 1 -- 172.21.15.115:0/629166227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 msgr2=0x55fd18672e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.514+0000 7f038540b640 1 --2- 172.21.15.115:0/629166227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 0x55fd18672e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.514+0000 7f038540b640 1 -- 172.21.15.115:0/629166227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 msgr2=0x55fd18683240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.514+0000 7f038540b640 1 --2- 172.21.15.115:0/629166227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd18683240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.514+0000 7f038540b640 1 -- 172.21.15.115:0/629166227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd183b15f0 con 0x55fd182ae4d0 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.514+0000 7f038540b640 1 --2- 172.21.15.115:0/629166227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd18682d00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0380109470 tx=0x7f037000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fb788c1161cef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.514+0000 7f036b7fe640 1 -- 172.21.15.115:0/629166227 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037002c470 con 0x55fd182ae4d0 2024-05-09T05:43:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.514+0000 7f036b7fe640 1 -- 172.21.15.115:0/629166227 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0370031070 con 0x55fd182ae4d0 2024-05-09T05:43:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.514+0000 7f036b7fe640 1 -- 172.21.15.115:0/629166227 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370010b80 con 0x55fd182ae4d0 2024-05-09T05:43:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.515+0000 7f038ee00740 1 -- 172.21.15.115:0/629166227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 msgr2=0x55fd18682d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.515+0000 7f038ee00740 1 --2- 172.21.15.115:0/629166227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd18682d00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0380109470 tx=0x7f037000b670 comp rx=0 tx=0).stop 2024-05-09T05:43:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.515+0000 7f038ee00740 1 -- 172.21.15.115:0/629166227 shutdown_connections 2024-05-09T05:43:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.515+0000 7f038ee00740 1 --2- 172.21.15.115:0/629166227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd18683240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.515+0000 7f038ee00740 1 --2- 172.21.15.115:0/629166227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd18682d00 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.515+0000 7f038ee00740 1 --2- 172.21.15.115:0/629166227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 0x55fd18672e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.515+0000 7f038ee00740 1 -- 172.21.15.115:0/629166227 >> 172.21.15.115:0/629166227 conn(0x55fd182c3300 msgr2=0x55fd186825f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.515+0000 7f038ee00740 1 -- 172.21.15.115:0/629166227 shutdown_connections 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.515+0000 7f038ee00740 1 -- 172.21.15.115:0/629166227 wait complete. 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.516+0000 7f038ee00740 1 -- start start 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.516+0000 7f038ee00740 1 Processor -- start 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.516+0000 7f038ee00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 0x55fd1868a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.516+0000 7f038ee00740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd1868ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.516+0000 7f038ee00740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd186f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.516+0000 7f038ee00740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f03801b0480 con 0x55fd182ae4d0 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.516+0000 7f038ee00740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f03801afd20 con 0x55fd182cf1b0 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.516+0000 7f038ee00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03801acf60 con 0x55fd182a62f0 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.516+0000 7f038ee00740 10 client.0 _ll_register_callbacks cb 0x7f03700125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f038540b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd1868ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f0385c0c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 0x55fd1868a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f038540b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd1868ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51152/0 (socket says 172.21.15.115:51152) 2024-05-09T05:43:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f038540b640 1 -- 172.21.15.115:0/1515809206 learned_addr learned my addr 172.21.15.115:0/1515809206 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f038640d640 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd186f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f0385c0c640 1 -- 172.21.15.115:0/1515809206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 msgr2=0x55fd186f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f0385c0c640 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd186f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f0385c0c640 1 -- 172.21.15.115:0/1515809206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 msgr2=0x55fd1868ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f0385c0c640 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd1868ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f0385c0c640 1 -- 172.21.15.115:0/1515809206 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd1859bed0 con 0x55fd182a62f0 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f038540b640 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd1868ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f0385c0c640 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 0x55fd1868a840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f037c02a930 tx=0x7f037c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=670fc1f25ace97e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f03697fa640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f03697fa640 1 -- 172.21.15.115:0/1515809206 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c061060 con 0x55fd182a62f0 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f03697fa640 1 -- 172.21.15.115:0/1515809206 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f037c004240 con 0x55fd182a62f0 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.517+0000 7f038ee00740 1 -- 172.21.15.115:0/1515809206 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0370031070 con 0x55fd182a62f0 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f03697fa640 1 -- 172.21.15.115:0/1515809206 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f037c04c040 con 0x55fd182a62f0 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f038ee00740 10 client.26449 fetch_fsmap learned FSMap version 34 2024-05-09T05:43:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f038ee00740 1 -- 172.21.15.115:0/1515809206 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fd186fa8d0 con 0x55fd182a62f0 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f03697fa640 1 -- 172.21.15.115:0/1515809206 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0370031070 con 0x55fd182a62f0 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f03697fa640 1 -- 172.21.15.115:0/1515809206 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f037c044ce0 con 0x55fd182a62f0 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f038ee00740 10 client.26449 fetch_fsmap finished waiting for FSMap version 34 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f038ee00740 10 client.26449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f038ee00740 20 client.26449 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f038ee00740 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fd182631c0 0x55fd186fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f038ee00740 4 client.26449 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f038ee00740 1 -- 172.21.15.115:0/1515809206 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd185df240 con 0x55fd182631c0 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.518+0000 7f038540b640 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fd182631c0 0x55fd186fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.519+0000 7f03697fa640 10 client.26449 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.519+0000 7f038540b640 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fd182631c0 0x55fd186fd060 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca6138bc402a982 server_cookie=89cca3990ce0ee1e in_seq=0 out_seq=0 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.527+0000 7f03697fa640 1 -- 172.21.15.115:0/1515809206 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fd185df240 con 0x55fd182631c0 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.527+0000 7f03697fa640 10 client.26449 handle_command_reply: tid=0 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.527+0000 7f038ee00740 1 client.26449 shutdown 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.527+0000 7f038ee00740 1 -- 172.21.15.115:0/1515809206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 msgr2=0x55fd1868a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.527+0000 7f038ee00740 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 0x55fd1868a840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f037c02a930 tx=0x7f037c02a410 comp rx=0 tx=0).stop 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f038ee00740 1 -- 172.21.15.115:0/1515809206 shutdown_connections 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f038ee00740 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fd182cf1b0 0x55fd186f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f038ee00740 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fd182ae4d0 0x55fd1868ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f038ee00740 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fd182631c0 0x55fd186fd060 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f038ee00740 1 --2- 172.21.15.115:0/1515809206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd182a62f0 0x55fd1868a840 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f03697fa640 0 client.26449 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f038ee00740 1 -- 172.21.15.115:0/1515809206 >> 172.21.15.115:0/1515809206 conn(0x55fd182c3300 msgr2=0x55fd18682fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f038ee00740 1 -- 172.21.15.115:0/1515809206 shutdown_connections 2024-05-09T05:43:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f038ee00740 1 -- 172.21.15.115:0/1515809206 wait complete. 2024-05-09T05:43:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f038ee00740 20 client.26449 trim_cache size 0 max 16384 2024-05-09T05:43:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:43:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:43:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:43:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f0387e70640 1 -- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f036c0b29e0 msgr2=0x7f036c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f0387e70640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f036c0b29e0 0x7f036c0b4ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55fd1866fd50 tx=0x7f037000bf90 comp rx=0 tx=0).stop 2024-05-09T05:43:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f0387e70640 1 -- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380109040 msgr2=0x7f038019df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f0387e70640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380109040 0x7f038019df20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f037800eb10 tx=0x7f037800c2d0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f0387e70640 1 -- 172.21.15.115:0/1250315193 shutdown_connections 2024-05-09T05:43:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.528+0000 7f0387e70640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f036c0b29e0 0x7f036c0b4ef0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.529+0000 7f0387e70640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0380109040 0x7f038019df20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.529+0000 7f0387e70640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0380108ba0 0x7f0380199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.529+0000 7f0387e70640 1 --2- 172.21.15.115:0/1250315193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03801032f0 0x7f0380199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.529+0000 7f0387e70640 1 -- 172.21.15.115:0/1250315193 >> 172.21.15.115:0/1250315193 conn(0x7f03800782e0 msgr2=0x7f0380100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.529+0000 7f0387e70640 1 -- 172.21.15.115:0/1250315193 shutdown_connections 2024-05-09T05:43:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:16.529+0000 7f0387e70640 1 -- 172.21.15.115:0/1250315193 wait complete. 2024-05-09T05:43:16.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.557+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce9c199f80 con 0x7fce9c1029c0 2024-05-09T05:43:16.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.558+0000 7fce99ffb700 1 -- 172.21.15.115:0/1802409606 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11961 (secure 0 0 0) 0x7fce9000fe20 con 0x7fce9c1029c0 2024-05-09T05:43:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:43:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:41:33.359460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:43:16.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.560+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fce8c0b2a70 msgr2=0x7fce8c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.560+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fce8c0b2a70 0x7fce8c0b4f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce880059d0 tx=0x7fce88018040 comp rx=0 tx=0).stop 2024-05-09T05:43:16.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 msgr2=0x7fce9c19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:16.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 0x7fce9c19e830 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fce9000fdf0 tx=0x7fce9000d6c0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 shutdown_connections 2024-05-09T05:43:16.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fce8c0b2a70 0x7fce8c0b4f80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce9c1089f0 0x7fce9c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce9c1029c0 0x7fce9c19e830 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 --2- 172.21.15.115:0/1802409606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce9c103310 0x7fce9c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:16.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 >> 172.21.15.115:0/1802409606 conn(0x7fce9c0fe6f0 msgr2=0x7fce9c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:16.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 shutdown_connections 2024-05-09T05:43:16.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:16.561+0000 7fcea2c4c700 1 -- 172.21.15.115:0/1802409606 wait complete. 2024-05-09T05:43:16.559 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 34 2024-05-09T05:43:17.712 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:17 smithi161 ceph-mon[33805]: pgmap v155: 129 pgs: 129 active+clean; 3.3 GiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:43:17.712 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:17 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1802409606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:43:17.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:17 smithi195 ceph-mon[33844]: pgmap v155: 129 pgs: 129 active+clean; 3.3 GiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:43:17.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:17 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1802409606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:43:17.879 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:17 smithi115 ceph-mon[30349]: pgmap v155: 129 pgs: 129 active+clean; 3.3 GiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:43:17.879 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:17 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1802409606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:43:19.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:19 smithi195 ceph-mon[33844]: pgmap v156: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 31 op/s 2024-05-09T05:43:19.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:19 smithi115 ceph-mon[30349]: pgmap v156: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 31 op/s 2024-05-09T05:43:19.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:19 smithi161 ceph-mon[33805]: pgmap v156: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 31 op/s 2024-05-09T05:43:21.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:21 smithi195 ceph-mon[33844]: pgmap v157: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 19 op/s 2024-05-09T05:43:21.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:21 smithi115 ceph-mon[30349]: pgmap v157: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 19 op/s 2024-05-09T05:43:21.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:21 smithi161 ceph-mon[33805]: pgmap v157: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 19 op/s 2024-05-09T05:43:22.004 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 135.5 --await-for 45.0' 2024-05-09T05:43:22.005 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 135.5 --await-for 45.0 2024-05-09T05:43:23.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:23 smithi195 ceph-mon[33844]: pgmap v158: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 19 op/s 2024-05-09T05:43:23.899 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:23 smithi115 ceph-mon[30349]: pgmap v158: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 19 op/s 2024-05-09T05:43:23.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:23 smithi161 ceph-mon[33805]: pgmap v158: 129 pgs: 129 active+clean; 3.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 19 op/s 2024-05-09T05:43:24.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:24 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:43:24.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:24 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:43:24.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:24 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:43:25.796 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:25 smithi115 ceph-mon[30349]: pgmap v159: 129 pgs: 129 active+clean; 3.5 GiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-05-09T05:43:25.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:25 smithi195 ceph-mon[33844]: pgmap v159: 129 pgs: 129 active+clean; 3.5 GiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-05-09T05:43:25.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:25 smithi161 ceph-mon[33805]: pgmap v159: 129 pgs: 129 active+clean; 3.5 GiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-05-09T05:43:26.617 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:26 smithi115 ceph-mon[30349]: pgmap v160: 129 pgs: 129 active+clean; 3.5 GiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-05-09T05:43:26.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:26 smithi195 ceph-mon[33844]: pgmap v160: 129 pgs: 129 active+clean; 3.5 GiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-05-09T05:43:26.868 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:26.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:26 smithi161 ceph-mon[33805]: pgmap v160: 129 pgs: 129 active+clean; 3.5 GiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-05-09T05:43:26.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.920+0000 7fa7d392e700 1 -- 172.21.15.115:0/1502961834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc111bd0 msgr2=0x7fa7cc111fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:26.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.920+0000 7fa7d392e700 1 --2- 172.21.15.115:0/1502961834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc111bd0 0x7fa7cc111fe0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009990 tx=0x7fa7c8009ca0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.920+0000 7fa7d392e700 1 -- 172.21.15.115:0/1502961834 shutdown_connections 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.920+0000 7fa7d392e700 1 --2- 172.21.15.115:0/1502961834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc10b280 0x7fa7cc1198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.920+0000 7fa7d392e700 1 --2- 172.21.15.115:0/1502961834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc111bd0 0x7fa7cc111fe0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.920+0000 7fa7d392e700 1 --2- 172.21.15.115:0/1502961834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7cc10a930 0x7fa7cc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.920+0000 7fa7d392e700 1 -- 172.21.15.115:0/1502961834 >> 172.21.15.115:0/1502961834 conn(0x7fa7cc06f4d0 msgr2=0x7fa7cc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.920+0000 7fa7d392e700 1 -- 172.21.15.115:0/1502961834 shutdown_connections 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.920+0000 7fa7d392e700 1 -- 172.21.15.115:0/1502961834 wait complete. 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d392e700 1 Processor -- start 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d392e700 1 -- start start 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d392e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc10a930 0x7fa7cc1aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d392e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc10b280 0x7fa7cc1aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d392e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7cc111bd0 0x7fa7cc1b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d392e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc11cb00 con 0x7fa7cc10a930 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d392e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc11c980 con 0x7fa7cc111bd0 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d392e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc11cc80 con 0x7fa7cc10b280 2024-05-09T05:43:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d16ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc10a930 0x7fa7cc1aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d16ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc10a930 0x7fa7cc1aa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37738/0 (socket says 172.21.15.115:37738) 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d16ca700 1 -- 172.21.15.115:0/532433220 learned_addr learned my addr 172.21.15.115:0/532433220 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d1ecb700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7cc111bd0 0x7fa7cc1b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc10b280 0x7fa7cc1aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d16ca700 1 -- 172.21.15.115:0/532433220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc10b280 msgr2=0x7fa7cc1aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.921+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc10b280 0x7fa7cc1aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.922+0000 7fa7d16ca700 1 -- 172.21.15.115:0/532433220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7cc111bd0 msgr2=0x7fa7cc1b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.922+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7cc111bd0 0x7fa7cc1b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.922+0000 7fa7d16ca700 1 -- 172.21.15.115:0/532433220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc1b5da0 con 0x7fa7cc10a930 2024-05-09T05:43:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.922+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc10a930 0x7fa7cc1aa7e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8000c00 tx=0x7fa7c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:26.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.922+0000 7fa7c27fc700 1 -- 172.21.15.115:0/532433220 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8027070 con 0x7fa7cc10a930 2024-05-09T05:43:26.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.922+0000 7fa7d392e700 1 -- 172.21.15.115:0/532433220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc1b5f70 con 0x7fa7cc10a930 2024-05-09T05:43:26.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.922+0000 7fa7c27fc700 1 -- 172.21.15.115:0/532433220 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7c800fb80 con 0x7fa7cc10a930 2024-05-09T05:43:26.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.922+0000 7fa7c27fc700 1 -- 172.21.15.115:0/532433220 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8022d00 con 0x7fa7cc10a930 2024-05-09T05:43:26.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.922+0000 7fa7d392e700 1 -- 172.21.15.115:0/532433220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc1b6310 con 0x7fa7cc10a930 2024-05-09T05:43:26.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.926+0000 7fa7c27fc700 1 -- 172.21.15.115:0/532433220 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa7c802a070 con 0x7fa7cc10a930 2024-05-09T05:43:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.927+0000 7fa7c27fc700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa7b80b29f0 0x7fa7b80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.927+0000 7fa7c27fc700 1 -- 172.21.15.115:0/532433220 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa7c80e37f0 con 0x7fa7cc10a930 2024-05-09T05:43:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.927+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa7b80b29f0 0x7fa7b80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.928+0000 7fa7e2344b80 1 Processor -- start 2024-05-09T05:43:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.928+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa7b80b29f0 0x7fa7b80b4f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc005fd0 tx=0x7fa7bc005f60 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.928+0000 7fa7e2344b80 1 -- start start 2024-05-09T05:43:26.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.928+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c9cd160 0x558f9ca664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.929+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca66a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.929+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca60ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.929+0000 7fa7d16ca700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c9cd160 0x558f9ca664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.929+0000 7fa7d16ca700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c9cd160 0x558f9ca664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51454/0 (socket says 172.21.15.115:51454) 2024-05-09T05:43:26.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.929+0000 7fa7d0ec9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca66a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.929+0000 7fa7d0ec9700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca66a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48452/0 (socket says 172.21.15.115:48452) 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.929+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558f9c982ff0 con 0x558f9ca606a0 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.929+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f9c9217f0 con 0x558f9c9cd160 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.929+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558f9c97efb0 con 0x558f9c8e34a0 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7d16ca700 1 -- 172.21.15.115:0/2981399304 learned_addr learned my addr 172.21.15.115:0/2981399304 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7d1ecb700 1 --2- 172.21.15.115:0/2981399304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca60ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7d16ca700 1 -- 172.21.15.115:0/2981399304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 msgr2=0x558f9ca66a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/2981399304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca66a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7d16ca700 1 -- 172.21.15.115:0/2981399304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 msgr2=0x558f9ca60ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/2981399304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca60ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7d16ca700 1 -- 172.21.15.115:0/2981399304 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f9c919370 con 0x558f9c9cd160 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/2981399304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c9cd160 0x558f9ca664f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80a68a0 tx=0x7fa7c80aa2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77d2442e3bad4418 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7b77fe700 1 -- 172.21.15.115:0/2981399304 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80cd0d0 con 0x558f9c9cd160 2024-05-09T05:43:26.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7b77fe700 1 -- 172.21.15.115:0/2981399304 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7c80b10f0 con 0x558f9c9cd160 2024-05-09T05:43:26.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.930+0000 7fa7b77fe700 1 -- 172.21.15.115:0/2981399304 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80aaa80 con 0x558f9c9cd160 2024-05-09T05:43:26.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.931+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2981399304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c9cd160 msgr2=0x558f9ca664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:26.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.931+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2981399304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c9cd160 0x558f9ca664f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80a68a0 tx=0x7fa7c80aa2a0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.932+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2981399304 shutdown_connections 2024-05-09T05:43:26.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.932+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2981399304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c9cd160 0x558f9ca664f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.932+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2981399304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca60ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.932+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2981399304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca66a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.932+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2981399304 >> 172.21.15.115:0/2981399304 conn(0x558f9c9436a0 msgr2=0x558f9ca5b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:26.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.932+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2981399304 shutdown_connections 2024-05-09T05:43:26.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.932+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2981399304 wait complete. 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.932+0000 7fa7e2344b80 1 -- start start 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.932+0000 7fa7e2344b80 1 Processor -- start 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca80d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 0x558f9ca7b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558f9c7feba0 con 0x558f9c9cd160 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f9c93d150 con 0x558f9ca606a0 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558f9ca61360 con 0x558f9c8e34a0 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7e2344b80 10 client.0 _ll_register_callbacks cb 0x558f9c7f46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d0ec9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca81270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d0ec9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca81270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37748/0 (socket says 172.21.15.115:37748) 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d0ec9700 1 -- 172.21.15.115:0/2485359889 learned_addr learned my addr 172.21.15.115:0/2485359889 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:26.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca80d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d1ecb700 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 0x558f9ca7b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d0ec9700 1 -- 172.21.15.115:0/2485359889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 msgr2=0x558f9ca80d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:26.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca80d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d0ec9700 1 -- 172.21.15.115:0/2485359889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 msgr2=0x558f9ca7b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 0x558f9ca7b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.933+0000 7fa7d0ec9700 1 -- 172.21.15.115:0/2485359889 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9c7fe490 con 0x558f9c9cd160 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.934+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca81270 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00feb0 tx=0x7fa7bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45393b6fcbf68a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.934+0000 7fa7b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.934+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/2485359889 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc028670 con 0x558f9c9cd160 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.934+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2485359889 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c80b10f0 con 0x558f9c9cd160 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.934+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/2485359889 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa7bc01f430 con 0x558f9c9cd160 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.934+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/2485359889 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa7bc028810 con 0x558f9c9cd160 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.934+0000 7fa7e2344b80 10 client.16959 fetch_fsmap learned FSMap version 35 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.934+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2485359889 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558f9ca88a00 con 0x558f9c9cd160 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.934+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/2485359889 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7c80b10f0 con 0x558f9c9cd160 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.935+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/2485359889 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa7bc01f9a0 con 0x558f9c9cd160 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.935+0000 7fa7e2344b80 10 client.16959 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T05:43:26.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.935+0000 7fa7e2344b80 10 client.16959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.935+0000 7fa7e2344b80 20 client.16959 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.935+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558f9ca8af90 0x558f9ca8d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.935+0000 7fa7e2344b80 4 client.16959 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.935+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2485359889 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f9c9909d0 con 0x558f9ca8af90 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.935+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558f9ca8af90 0x558f9ca8d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.936+0000 7fa7b4ff9700 10 client.16959 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.936+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558f9ca8af90 0x558f9ca8d3b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8c4d793233a9c73 server_cookie=daebef1f03936d02 in_seq=0 out_seq=0 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.936+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/2485359889 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f9c9909d0 con 0x558f9ca8af90 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.936+0000 7fa7b4ff9700 10 client.16959 handle_command_reply: tid=0 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.936+0000 7fa7e2344b80 1 client.16959 shutdown 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2485359889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 msgr2=0x558f9ca81270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca81270 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00feb0 tx=0x7fa7bc004750 comp rx=0 tx=0).stop 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2485359889 shutdown_connections 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 0x558f9ca7b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558f9ca8af90 0x558f9ca8d3b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca80d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/2485359889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca81270 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7b4ff9700 0 client.16959 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:43:26.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2485359889 >> 172.21.15.115:0/2485359889 conn(0x558f9c9436a0 msgr2=0x558f9ca5b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:26.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2485359889 shutdown_connections 2024-05-09T05:43:26.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 1 -- 172.21.15.115:0/2485359889 wait complete. 2024-05-09T05:43:26.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.937+0000 7fa7e2344b80 20 client.16959 trim_cache size 0 max 16384 2024-05-09T05:43:26.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.953+0000 7fa7e2344b80 1 Processor -- start 2024-05-09T05:43:26.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.953+0000 7fa7e2344b80 1 -- start start 2024-05-09T05:43:26.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca62e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 0x558f9ca664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc1b68c0 con 0x558f9c9cd160 2024-05-09T05:43:26.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc1b8490 con 0x558f9ca606a0 2024-05-09T05:43:26.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7d16ca700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc1b6f90 con 0x558f9c8e34a0 2024-05-09T05:43:26.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7d16ca700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48484/0 (socket says 172.21.15.115:48484) 2024-05-09T05:43:26.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7d16ca700 1 -- 172.21.15.115:0/3026276401 learned_addr learned my addr 172.21.15.115:0/3026276401 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:26.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca62e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.954+0000 7fa7d1ecb700 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 0x558f9ca664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7d16ca700 1 -- 172.21.15.115:0/3026276401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 msgr2=0x558f9ca664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:26.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 0x558f9ca664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7d16ca700 1 -- 172.21.15.115:0/3026276401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 msgr2=0x558f9ca62e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca62e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7d16ca700 1 -- 172.21.15.115:0/3026276401 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f9ca54f60 con 0x558f9c8e34a0 2024-05-09T05:43:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca62e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca646c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80aa7b0 tx=0x7fa7c80aa200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a0af3842200ff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7b6ffd700 1 -- 172.21.15.115:0/3026276401 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80cd0d0 con 0x558f9c8e34a0 2024-05-09T05:43:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7b6ffd700 1 -- 172.21.15.115:0/3026276401 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7c80aa820 con 0x558f9c8e34a0 2024-05-09T05:43:26.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.955+0000 7fa7b6ffd700 1 -- 172.21.15.115:0/3026276401 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80b3160 con 0x558f9c8e34a0 2024-05-09T05:43:26.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.956+0000 7fa7e2344b80 1 -- 172.21.15.115:0/3026276401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 msgr2=0x558f9ca646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:26.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.956+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca646c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80aa7b0 tx=0x7fa7c80aa200 comp rx=0 tx=0).stop 2024-05-09T05:43:26.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.956+0000 7fa7e2344b80 1 -- 172.21.15.115:0/3026276401 shutdown_connections 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.956+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9ca606a0 0x558f9ca664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.956+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c8e34a0 0x558f9ca646c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.956+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/3026276401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9c9cd160 0x558f9ca62e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.956+0000 7fa7e2344b80 1 -- 172.21.15.115:0/3026276401 >> 172.21.15.115:0/3026276401 conn(0x558f9c9436a0 msgr2=0x558f9ca8cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.956+0000 7fa7e2344b80 1 -- 172.21.15.115:0/3026276401 shutdown_connections 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.956+0000 7fa7e2344b80 1 -- 172.21.15.115:0/3026276401 wait complete. 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.957+0000 7fa7e2344b80 1 -- start start 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.957+0000 7fa7e2344b80 1 Processor -- start 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.957+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c8e34a0 0x558f9ca872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.957+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c9cd160 0x558f9ca82620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.957+0000 7fa7e2344b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca82b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.957+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc1b7d60 con 0x558f9ca606a0 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.957+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc1b7660 con 0x558f9c8e34a0 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.957+0000 7fa7e2344b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc1a8b50 con 0x558f9c9cd160 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7e2344b80 10 client.0 _ll_register_callbacks cb 0x558f9c7f46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d1ecb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca82b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d1ecb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca82b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37754/0 (socket says 172.21.15.115:37754) 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d1ecb700 1 -- 172.21.15.115:0/1063431000 learned_addr learned my addr 172.21.15.115:0/1063431000 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c8e34a0 0x558f9ca872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c9cd160 0x558f9ca82620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d1ecb700 1 -- 172.21.15.115:0/1063431000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c9cd160 msgr2=0x558f9ca82620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d1ecb700 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c9cd160 0x558f9ca82620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d1ecb700 1 -- 172.21.15.115:0/1063431000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c8e34a0 msgr2=0x558f9ca872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d1ecb700 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c8e34a0 0x558f9ca872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d1ecb700 1 -- 172.21.15.115:0/1063431000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9ca830a0 con 0x558f9ca606a0 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c8e34a0 0x558f9ca872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d0ec9700 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c9cd160 0x558f9ca82620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7d1ecb700 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca82b60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4007b20 tx=0x7fa7c4007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3207feb88da8c173 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.958+0000 7fa7b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.959+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/1063431000 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4004db0 con 0x558f9ca606a0 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.959+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/1063431000 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa7c40175b0 con 0x558f9ca606a0 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.959+0000 7fa7e2344b80 1 -- 172.21.15.115:0/1063431000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f9c98aa70 con 0x558f9ca606a0 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.959+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/1063431000 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa7c402a050 con 0x558f9ca606a0 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.959+0000 7fa7e2344b80 10 client.16965 fetch_fsmap learned FSMap version 35 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.959+0000 7fa7e2344b80 1 -- 172.21.15.115:0/1063431000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558f9caf53f0 con 0x558f9ca606a0 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.959+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/1063431000 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x558f9c98aa70 con 0x558f9ca606a0 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.959+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/1063431000 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa7c40219b0 con 0x558f9ca606a0 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.960+0000 7fa7e2344b80 10 client.16965 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.960+0000 7fa7e2344b80 10 client.16965 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:43:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.960+0000 7fa7e2344b80 20 client.16965 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.960+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558f9caf5b00 0x558f9caf7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.960+0000 7fa7e2344b80 4 client.16965 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.960+0000 7fa7e2344b80 1 -- 172.21.15.115:0/1063431000 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f9c7fe970 con 0x558f9caf5b00 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.960+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558f9caf5b00 0x558f9caf7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.960+0000 7fa7b4ff9700 10 client.16965 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.960+0000 7fa7d16ca700 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558f9caf5b00 0x558f9caf7f20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a35bf037c4828b4c server_cookie=6ed0fdccfa4afe2c in_seq=0 out_seq=0 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7b4ff9700 1 -- 172.21.15.115:0/1063431000 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f9c7fe970 con 0x558f9caf5b00 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7b4ff9700 10 client.16965 handle_command_reply: tid=0 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 client.16965 shutdown 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 -- 172.21.15.115:0/1063431000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 msgr2=0x558f9ca82b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca82b60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4007b20 tx=0x7fa7c4007b50 comp rx=0 tx=0).stop 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 -- 172.21.15.115:0/1063431000 shutdown_connections 2024-05-09T05:43:26.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f9c8e34a0 0x558f9ca872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558f9caf5b00 0x558f9caf7f20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f9ca606a0 0x558f9ca82b60 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 --2- 172.21.15.115:0/1063431000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f9c9cd160 0x558f9ca82620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 -- 172.21.15.115:0/1063431000 >> 172.21.15.115:0/1063431000 conn(0x558f9c9436a0 msgr2=0x558f9ca8cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7b4ff9700 0 client.16965 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:43:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.961+0000 7fa7e2344b80 1 -- 172.21.15.115:0/1063431000 shutdown_connections 2024-05-09T05:43:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.962+0000 7fa7e2344b80 1 -- 172.21.15.115:0/1063431000 wait complete. 2024-05-09T05:43:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.962+0000 7fa7e2344b80 20 client.16965 trim_cache size 0 max 16384 2024-05-09T05:43:26.964 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:43:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.962+0000 7fa7d392e700 1 -- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa7b80b29f0 msgr2=0x7fa7b80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.962+0000 7fa7d392e700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa7b80b29f0 0x7fa7b80b4f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc005fd0 tx=0x7fa7bc005f60 comp rx=0 tx=0).stop 2024-05-09T05:43:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.962+0000 7fa7d392e700 1 -- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc10a930 msgr2=0x7fa7cc1aa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.962+0000 7fa7d392e700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc10a930 0x7fa7cc1aa7e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8000c00 tx=0x7fa7c80047b0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.963+0000 7fa7d392e700 1 -- 172.21.15.115:0/532433220 shutdown_connections 2024-05-09T05:43:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.963+0000 7fa7d392e700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa7b80b29f0 0x7fa7b80b4f00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.963+0000 7fa7d392e700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7cc111bd0 0x7fa7cc1b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.963+0000 7fa7d392e700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7cc10a930 0x7fa7cc1aa7e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.963+0000 7fa7d392e700 1 --2- 172.21.15.115:0/532433220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc10b280 0x7fa7cc1aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.963+0000 7fa7d392e700 1 -- 172.21.15.115:0/532433220 >> 172.21.15.115:0/532433220 conn(0x7fa7cc06f4d0 msgr2=0x7fa7cc1104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.963+0000 7fa7d392e700 1 -- 172.21.15.115:0/532433220 shutdown_connections 2024-05-09T05:43:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:26.963+0000 7fa7d392e700 1 -- 172.21.15.115:0/532433220 wait complete. 2024-05-09T05:43:29.005 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:43:29.221 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:29 smithi115 ceph-mon[30349]: pgmap v161: 129 pgs: 129 active+clean; 3.6 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 27 op/s 2024-05-09T05:43:29.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:29 smithi195 ceph-mon[33844]: pgmap v161: 129 pgs: 129 active+clean; 3.6 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 27 op/s 2024-05-09T05:43:29.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:29 smithi161 ceph-mon[33805]: pgmap v161: 129 pgs: 129 active+clean; 3.6 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 27 op/s 2024-05-09T05:43:29.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.512+0000 7f0281f9e700 1 -- 172.21.15.115:0/873832094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f027c108d30 msgr2=0x7f027c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.512+0000 7f0281f9e700 1 --2- 172.21.15.115:0/873832094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f027c108d30 0x7f027c109140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0268009990 tx=0x7f0268009ca0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.512+0000 7f0281f9e700 1 -- 172.21.15.115:0/873832094 shutdown_connections 2024-05-09T05:43:29.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.512+0000 7f0281f9e700 1 --2- 172.21.15.115:0/873832094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f027c109680 0x7f027c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.512+0000 7f0281f9e700 1 --2- 172.21.15.115:0/873832094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c104400 0x7f027c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.512+0000 7f0281f9e700 1 --2- 172.21.15.115:0/873832094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f027c108d30 0x7f027c109140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.512+0000 7f0281f9e700 1 -- 172.21.15.115:0/873832094 >> 172.21.15.115:0/873832094 conn(0x7f027c079d10 msgr2=0x7f027c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.512+0000 7f0281f9e700 1 -- 172.21.15.115:0/873832094 shutdown_connections 2024-05-09T05:43:29.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.513+0000 7f0281f9e700 1 -- 172.21.15.115:0/873832094 wait complete. 2024-05-09T05:43:29.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.514+0000 7f0281f9e700 1 Processor -- start 2024-05-09T05:43:29.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.514+0000 7f0281f9e700 1 -- start start 2024-05-09T05:43:29.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.514+0000 7f0281f9e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f027c104400 0x7f027c1a2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.514+0000 7f0281f9e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c108d30 0x7f027c1a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.514+0000 7f0281f9e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f027c109680 0x7f027c1acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.514+0000 7f0281f9e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f027c114190 con 0x7f027c108d30 2024-05-09T05:43:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.514+0000 7f0281f9e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f027c114010 con 0x7f027c104400 2024-05-09T05:43:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f028179d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f027c109680 0x7f027c1acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f0280f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f027c104400 0x7f027c1a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f028179d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f027c109680 0x7f027c1acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48522/0 (socket says 172.21.15.115:48522) 2024-05-09T05:43:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f028179d700 1 -- 172.21.15.115:0/853836558 learned_addr learned my addr 172.21.15.115:0/853836558 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f0281f9e700 1 -- 172.21.15.115:0/853836558 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f027c114310 con 0x7f027c109680 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f0280f9c700 1 -- 172.21.15.115:0/853836558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f027c109680 msgr2=0x7f027c1acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f0280f9c700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f027c109680 0x7f027c1acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f0280f9c700 1 -- 172.21.15.115:0/853836558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c108d30 msgr2=0x7f027c1a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f0280f9c700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c108d30 0x7f027c1a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f0280f9c700 1 -- 172.21.15.115:0/853836558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c1ad430 con 0x7f027c104400 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f0280f9c700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f027c104400 0x7f027c1a2040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f027000e9f0 tx=0x7f027000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.515+0000 7f0279ffb700 1 -- 172.21.15.115:0/853836558 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027000cbd0 con 0x7f027c104400 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.516+0000 7f0279ffb700 1 -- 172.21.15.115:0/853836558 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f027000cd70 con 0x7f027c104400 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.516+0000 7f0279ffb700 1 -- 172.21.15.115:0/853836558 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270018680 con 0x7f027c104400 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.516+0000 7f0281f9e700 1 -- 172.21.15.115:0/853836558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027c1ad720 con 0x7f027c104400 2024-05-09T05:43:29.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.516+0000 7f0281f9e700 1 -- 172.21.15.115:0/853836558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027c1adbe0 con 0x7f027c104400 2024-05-09T05:43:29.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.520+0000 7f0279ffb700 1 -- 172.21.15.115:0/853836558 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0270010430 con 0x7f027c104400 2024-05-09T05:43:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.521+0000 7f0279ffb700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f026c0b2a70 0x7f026c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.521+0000 7f0291c16b80 1 Processor -- start 2024-05-09T05:43:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.521+0000 7f0291c16b80 1 -- start start 2024-05-09T05:43:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.521+0000 7f027bfff700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f026c0b2a70 0x7f026c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf637bce90 0x55cf638bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f0279ffb700 1 -- 172.21.15.115:0/853836558 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f02700e2a50 con 0x7f027c104400 2024-05-09T05:43:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 0x55cf638bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf638bf8b0 0x55cf638cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f0291c16b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cf637e2230 con 0x55cf637bce90 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f0291c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cf63787b30 con 0x55cf63742630 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f0291c16b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cf637de1f0 con 0x55cf638bf8b0 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f028179d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf638bf8b0 0x55cf638cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f028179d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf638bf8b0 0x55cf638cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48528/0 (socket says 172.21.15.115:48528) 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f028179d700 1 -- 172.21.15.115:0/4206589098 learned_addr learned my addr 172.21.15.115:0/4206589098 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f027bfff700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f026c0b2a70 0x7f026c0b4f80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f02680098e0 tx=0x7f026800b610 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f028179d700 1 -- 172.21.15.115:0/4206589098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 msgr2=0x55cf638bf2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f028179d700 1 --2- 172.21.15.115:0/4206589098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 0x55cf638bf2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f028179d700 1 -- 172.21.15.115:0/4206589098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf637bce90 msgr2=0x55cf638bed70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f028179d700 1 --2- 172.21.15.115:0/4206589098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf637bce90 0x55cf638bed70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f028179d700 1 -- 172.21.15.115:0/4206589098 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf63778690 con 0x55cf638bf8b0 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.522+0000 7f028179d700 1 --2- 172.21.15.115:0/4206589098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf638bf8b0 0x55cf638cbb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f027400e680 tx=0x7f0274009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e21709bf24b13b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.523+0000 7f0266ffd700 1 -- 172.21.15.115:0/4206589098 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274019030 con 0x55cf638bf8b0 2024-05-09T05:43:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.523+0000 7f0266ffd700 1 -- 172.21.15.115:0/4206589098 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02740093a0 con 0x55cf638bf8b0 2024-05-09T05:43:29.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.523+0000 7f0266ffd700 1 -- 172.21.15.115:0/4206589098 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740048a0 con 0x55cf638bf8b0 2024-05-09T05:43:29.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.524+0000 7f0291c16b80 1 -- 172.21.15.115:0/4206589098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf638bf8b0 msgr2=0x55cf638cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.524+0000 7f0291c16b80 1 --2- 172.21.15.115:0/4206589098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf638bf8b0 0x55cf638cbb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f027400e680 tx=0x7f0274009920 comp rx=0 tx=0).stop 2024-05-09T05:43:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.524+0000 7f0291c16b80 1 -- 172.21.15.115:0/4206589098 shutdown_connections 2024-05-09T05:43:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.524+0000 7f0291c16b80 1 --2- 172.21.15.115:0/4206589098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf638bf8b0 0x55cf638cbb70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.524+0000 7f0291c16b80 1 --2- 172.21.15.115:0/4206589098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf637bce90 0x55cf638bed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.524+0000 7f0291c16b80 1 --2- 172.21.15.115:0/4206589098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 0x55cf638bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.524+0000 7f0291c16b80 1 -- 172.21.15.115:0/4206589098 >> 172.21.15.115:0/4206589098 conn(0x55cf63746100 msgr2=0x55cf638badc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.524+0000 7f0291c16b80 1 -- 172.21.15.115:0/4206589098 shutdown_connections 2024-05-09T05:43:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.525+0000 7f0291c16b80 1 -- 172.21.15.115:0/4206589098 wait complete. 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.525+0000 7f0291c16b80 1 -- start start 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.525+0000 7f0291c16b80 1 Processor -- start 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f0280f9c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f0280f9c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48544/0 (socket says 172.21.15.115:48544) 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f028179d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f028179d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37784/0 (socket says 172.21.15.115:37784) 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f027bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f027bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51530/0 (socket says 172.21.15.115:51530) 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cf638cc550 con 0x55cf638bf8b0 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f0280f9c700 1 -- 172.21.15.115:0/1388818586 learned_addr learned my addr 172.21.15.115:0/1388818586 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cf6377ccb0 con 0x55cf637bce90 2024-05-09T05:43:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.526+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cf638cc6d0 con 0x55cf63742630 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f0280f9c700 1 -- 172.21.15.115:0/1388818586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 msgr2=0x55cf638c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f0280f9c700 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f0280f9c700 1 -- 172.21.15.115:0/1388818586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 msgr2=0x55cf638c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f0280f9c700 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f0280f9c700 1 -- 172.21.15.115:0/1388818586 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf6365d620 con 0x55cf63742630 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f028179d700 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f027bfff700 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f0291c16b80 10 client.0 _ll_register_callbacks cb 0x55cf63653850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f0280f9c700 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c90d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02700dfec0 tx=0x7f02700a9f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d94c3061cccb91b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cf637efc10 con 0x55cf63742630 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f025bfff700 10 client.26479 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f025bfff700 1 -- 172.21.15.115:0/1388818586 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700a5180 con 0x55cf63742630 2024-05-09T05:43:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.527+0000 7f025bfff700 1 -- 172.21.15.115:0/1388818586 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f02700cc2c0 con 0x55cf63742630 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f025bfff700 1 -- 172.21.15.115:0/1388818586 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f02700a9500 con 0x55cf63742630 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f0291c16b80 10 client.26479 fetch_fsmap learned FSMap version 35 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({fsmap=35}) -- 0x55cf638d7050 con 0x55cf63742630 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f025bfff700 1 -- 172.21.15.115:0/1388818586 <== mon.1 v2:172.21.15.195:3300/0 4 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f0270010720 con 0x55cf63742630 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f0291c16b80 10 client.26479 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f0291c16b80 10 client.26479 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f0291c16b80 20 client.26479 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f0291c16b80 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf638d9520 0x55cf638dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f0291c16b80 4 client.26479 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf637e9cb0 con 0x55cf638d9520 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.528+0000 7f027bfff700 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf638d9520 0x55cf638dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.529+0000 7f025bfff700 10 client.26479 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.529+0000 7f027bfff700 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf638d9520 0x55cf638dba10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc487cebc4e452aa server_cookie=6f3eb1c6bd19b13b in_seq=0 out_seq=0 2024-05-09T05:43:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.530+0000 7f025bfff700 1 -- 172.21.15.115:0/1388818586 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cf637e9cb0 con 0x55cf638d9520 2024-05-09T05:43:29.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.530+0000 7f025bfff700 10 client.26479 handle_command_reply: tid=0 2024-05-09T05:43:29.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.530+0000 7f0291c16b80 1 client.26479 shutdown 2024-05-09T05:43:29.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.530+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 msgr2=0x55cf638c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.530+0000 7f0291c16b80 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c90d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02700dfec0 tx=0x7f02700a9f30 comp rx=0 tx=0).stop 2024-05-09T05:43:29.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.530+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 shutdown_connections 2024-05-09T05:43:29.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.530+0000 7f0291c16b80 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf638d9520 0x55cf638dba10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.531+0000 7f0291c16b80 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.531+0000 7f0291c16b80 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c90d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.531+0000 7f0291c16b80 1 --2- 172.21.15.115:0/1388818586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.531+0000 7f025bfff700 0 client.26479 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.531+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 >> 172.21.15.115:0/1388818586 conn(0x55cf63746100 msgr2=0x55cf638ba910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.531+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 shutdown_connections 2024-05-09T05:43:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.531+0000 7f0291c16b80 1 -- 172.21.15.115:0/1388818586 wait complete. 2024-05-09T05:43:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.531+0000 7f0291c16b80 20 client.26479 trim_cache size 0 max 16384 2024-05-09T05:43:29.542 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:29.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.555+0000 7f0291c16b80 1 Processor -- start 2024-05-09T05:43:29.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.555+0000 7f0291c16b80 1 -- start start 2024-05-09T05:43:29.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638e11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f027bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f027bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51542/0 (socket says 172.21.15.115:51542) 2024-05-09T05:43:29.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f028179d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638e11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f0280f9c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f028179d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638e11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37796/0 (socket says 172.21.15.115:37796) 2024-05-09T05:43:29.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f0280f9c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48550/0 (socket says 172.21.15.115:48550) 2024-05-09T05:43:29.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f027bfff700 1 -- 172.21.15.115:0/3460344632 learned_addr learned my addr 172.21.15.115:0/3460344632 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f0291c16b80 1 -- 172.21.15.115:0/3460344632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1bf0b0 con 0x55cf638bf8b0 2024-05-09T05:43:29.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f0291c16b80 1 -- 172.21.15.115:0/3460344632 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1c0c80 con 0x55cf637bce90 2024-05-09T05:43:29.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.556+0000 7f0291c16b80 1 -- 172.21.15.115:0/3460344632 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1bf780 con 0x55cf63742630 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.557+0000 7f0280f9c700 1 -- 172.21.15.115:0/3460344632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 msgr2=0x55cf638c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.557+0000 7f0280f9c700 1 --2- 172.21.15.115:0/3460344632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.557+0000 7f0280f9c700 1 -- 172.21.15.115:0/3460344632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 msgr2=0x55cf638e11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.557+0000 7f0280f9c700 1 --2- 172.21.15.115:0/3460344632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638e11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.557+0000 7f0280f9c700 1 -- 172.21.15.115:0/3460344632 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf638bba10 con 0x55cf63742630 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.557+0000 7f028179d700 1 --2- 172.21.15.115:0/3460344632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638e11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.557+0000 7f0280f9c700 1 --2- 172.21.15.115:0/3460344632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c83f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0270000c00 tx=0x7f0270019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=babbd40497cca52e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.557+0000 7f02667fc700 1 -- 172.21.15.115:0/3460344632 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700adda0 con 0x55cf63742630 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.558+0000 7f02667fc700 1 -- 172.21.15.115:0/3460344632 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02700c2030 con 0x55cf63742630 2024-05-09T05:43:29.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.558+0000 7f02667fc700 1 -- 172.21.15.115:0/3460344632 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700adda0 con 0x55cf63742630 2024-05-09T05:43:29.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.559+0000 7f0291c16b80 1 -- 172.21.15.115:0/3460344632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 msgr2=0x55cf638c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.559+0000 7f0291c16b80 1 --2- 172.21.15.115:0/3460344632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c83f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0270000c00 tx=0x7f0270019000 comp rx=0 tx=0).stop 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.559+0000 7f0291c16b80 1 -- 172.21.15.115:0/3460344632 shutdown_connections 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.559+0000 7f0291c16b80 1 --2- 172.21.15.115:0/3460344632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf638e11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.559+0000 7f0291c16b80 1 --2- 172.21.15.115:0/3460344632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf63742630 0x55cf638c83f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.559+0000 7f0291c16b80 1 --2- 172.21.15.115:0/3460344632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf637bce90 0x55cf638c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.559+0000 7f0291c16b80 1 -- 172.21.15.115:0/3460344632 >> 172.21.15.115:0/3460344632 conn(0x55cf63746100 msgr2=0x55cf638c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.559+0000 7f0291c16b80 1 -- 172.21.15.115:0/3460344632 shutdown_connections 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.559+0000 7f0291c16b80 1 -- 172.21.15.115:0/3460344632 wait complete. 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.560+0000 7f0291c16b80 1 -- start start 2024-05-09T05:43:29.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.560+0000 7f0291c16b80 1 Processor -- start 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 0x55cf638cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf637bce90 0x55cf638d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f0291c16b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf63942d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f0280f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 0x55cf638cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f027bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf637bce90 0x55cf638d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f0280f9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 0x55cf638cfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51556/0 (socket says 172.21.15.115:51556) 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f027bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf637bce90 0x55cf638d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48562/0 (socket says 172.21.15.115:48562) 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f028179d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf63942d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f028179d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf63942d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37804/0 (socket says 172.21.15.115:37804) 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f0291c16b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1c0550 con 0x55cf638bf8b0 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f0291c16b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1bfe50 con 0x55cf63742630 2024-05-09T05:43:29.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f0291c16b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1a0070 con 0x55cf637bce90 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.561+0000 7f0291c16b80 10 client.0 _ll_register_callbacks cb 0x55cf63653850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.562+0000 7f0280f9c700 1 -- 172.21.15.115:0/502798631 learned_addr learned my addr 172.21.15.115:0/502798631 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.562+0000 7f0280f9c700 1 -- 172.21.15.115:0/502798631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf637bce90 msgr2=0x55cf638d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.562+0000 7f0280f9c700 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf637bce90 0x55cf638d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.562+0000 7f0280f9c700 1 -- 172.21.15.115:0/502798631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 msgr2=0x55cf63942d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.562+0000 7f0280f9c700 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf63942d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.562+0000 7f0280f9c700 1 -- 172.21.15.115:0/502798631 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf639432d0 con 0x55cf63742630 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.562+0000 7f028179d700 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf63942d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.563+0000 7f0280f9c700 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 0x55cf638cfba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f02700a96a0 tx=0x7f02700a9890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a206fd826c1eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.563+0000 7f025bfff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:43:29.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.563+0000 7f025bfff700 1 -- 172.21.15.115:0/502798631 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027008e030 con 0x55cf63742630 2024-05-09T05:43:29.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.563+0000 7f025bfff700 1 -- 172.21.15.115:0/502798631 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f02700a5fe0 con 0x55cf63742630 2024-05-09T05:43:29.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.563+0000 7f0291c16b80 1 -- 172.21.15.115:0/502798631 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02700c2030 con 0x55cf63742630 2024-05-09T05:43:29.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.563+0000 7f025bfff700 1 -- 172.21.15.115:0/502798631 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f02700adda0 con 0x55cf63742630 2024-05-09T05:43:29.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.563+0000 7f0291c16b80 10 client.26393 fetch_fsmap learned FSMap version 35 2024-05-09T05:43:29.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.563+0000 7f0291c16b80 1 -- 172.21.15.115:0/502798631 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cf63944390 con 0x55cf63742630 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.564+0000 7f025bfff700 1 -- 172.21.15.115:0/502798631 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02700c2030 con 0x55cf63742630 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.564+0000 7f025bfff700 1 -- 172.21.15.115:0/502798631 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f02700b4b70 con 0x55cf63742630 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.564+0000 7f0291c16b80 10 client.26393 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.564+0000 7f0291c16b80 10 client.26393 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.564+0000 7f0291c16b80 20 client.26393 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.564+0000 7f0291c16b80 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf638d9520 0x55cf639468e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.564+0000 7f0291c16b80 4 client.26393 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.5, "await_for": 45.0}] 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.564+0000 7f0291c16b80 1 -- 172.21.15.115:0/502798631 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.5, "await_for": 45.0}) -- 0x55cf6365db00 con 0x55cf638d9520 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.564+0000 7f027bfff700 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf638d9520 0x55cf639468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.565+0000 7f025bfff700 10 client.26393 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:29.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.565+0000 7f027bfff700 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf638d9520 0x55cf639468e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4f28f21cd95e05 server_cookie=f655668be82d1748 in_seq=0 out_seq=0 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.640+0000 7fa542396640 1 -- 172.21.15.115:0/663156803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c109040 msgr2=0x7fa53c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.640+0000 7fa542396640 1 --2- 172.21.15.115:0/663156803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c109040 0x7fa53c102c80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa524009900 tx=0x7fa52402a410 comp rx=0 tx=0).stop 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.640+0000 7fa542396640 1 -- 172.21.15.115:0/663156803 shutdown_connections 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.640+0000 7fa542396640 1 --2- 172.21.15.115:0/663156803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa53c1032f0 0x7fa53c10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.640+0000 7fa542396640 1 --2- 172.21.15.115:0/663156803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c109040 0x7fa53c102c80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.640+0000 7fa542396640 1 --2- 172.21.15.115:0/663156803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53c108ba0 0x7fa53c102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.640+0000 7fa542396640 1 -- 172.21.15.115:0/663156803 >> 172.21.15.115:0/663156803 conn(0x7fa53c0782e0 msgr2=0x7fa53c078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.641+0000 7fa542396640 1 -- 172.21.15.115:0/663156803 shutdown_connections 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.641+0000 7fa542396640 1 -- 172.21.15.115:0/663156803 wait complete. 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.641+0000 7fa542396640 1 Processor -- start 2024-05-09T05:43:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.641+0000 7fa542396640 1 -- start start 2024-05-09T05:43:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.641+0000 7fa542396640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c1032f0 0x7fa53c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.641+0000 7fa542396640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa53c108ba0 0x7fa53c199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa542396640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53c109040 0x7fa53c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa542396640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c112c90 con 0x7fa53c1032f0 2024-05-09T05:43:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa542396640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c112b10 con 0x7fa53c108ba0 2024-05-09T05:43:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa542396640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c112e10 con 0x7fa53c109040 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c1032f0 0x7fa53c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa533fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa53c108ba0 0x7fa53c199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c1032f0 0x7fa53c199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37824/0 (socket says 172.21.15.115:37824) 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa533fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa53c108ba0 0x7fa53c199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51580/0 (socket says 172.21.15.115:51580) 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa53bfff640 1 -- 172.21.15.115:0/2712296003 learned_addr learned my addr 172.21.15.115:0/2712296003 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa540933640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53c109040 0x7fa53c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa53bfff640 1 -- 172.21.15.115:0/2712296003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53c109040 msgr2=0x7fa53c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa53bfff640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53c109040 0x7fa53c19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa53bfff640 1 -- 172.21.15.115:0/2712296003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa53c108ba0 msgr2=0x7fa53c199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa53bfff640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa53c108ba0 0x7fa53c199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa53bfff640 1 -- 172.21.15.115:0/2712296003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c19e5d0 con 0x7fa53c1032f0 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa53bfff640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c1032f0 0x7fa53c199770 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e9f0 tx=0x7fa52c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa539ffb640 1 -- 172.21.15.115:0/2712296003 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00dad0 con 0x7fa53c1032f0 2024-05-09T05:43:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa539ffb640 1 -- 172.21.15.115:0/2712296003 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa52c0054b0 con 0x7fa53c1032f0 2024-05-09T05:43:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa542396640 1 -- 172.21.15.115:0/2712296003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53c19e8c0 con 0x7fa53c1032f0 2024-05-09T05:43:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa539ffb640 1 -- 172.21.15.115:0/2712296003 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c004ce0 con 0x7fa53c1032f0 2024-05-09T05:43:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.642+0000 7fa542396640 1 -- 172.21.15.115:0/2712296003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53c19ed90 con 0x7fa53c1032f0 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa54924c740 1 Processor -- start 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa54924c740 1 -- start start 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa539ffb640 1 -- 172.21.15.115:0/2712296003 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa52c004ea0 con 0x7fa53c1032f0 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa54924c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c6e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa54924c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa54924c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa54924c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561020bd37c0 con 0x5610208a22f0 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa54924c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561020b6f0e0 con 0x5610208aa4d0 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa54924c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561020bcf780 con 0x5610208cb1b0 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa533fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa540933640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c6e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37832/0 (socket says 172.21.15.115:37832) 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.645+0000 7fa53bfff640 1 -- 172.21.15.115:0/4287617473 learned_addr learned my addr 172.21.15.115:0/4287617473 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa53bfff640 1 -- 172.21.15.115:0/4287617473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 msgr2=0x561020c6e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa53bfff640 1 --2- 172.21.15.115:0/4287617473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c6e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa53bfff640 1 -- 172.21.15.115:0/4287617473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 msgr2=0x561020c6f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa53bfff640 1 --2- 172.21.15.115:0/4287617473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa53bfff640 1 -- 172.21.15.115:0/4287617473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561020b987c0 con 0x5610208a22f0 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa540933640 1 --2- 172.21.15.115:0/4287617473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c6e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa533fff640 1 --2- 172.21.15.115:0/4287617473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa539ffb640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5280b2950 0x7fa5280b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa53bfff640 1 --2- 172.21.15.115:0/4287617473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6ebc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0df5d0 tx=0x7fa52c0ea950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e60f13ee37b6736 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa540933640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5280b2950 0x7fa5280b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa539ffb640 1 -- 172.21.15.115:0/2712296003 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa52c0e2250 con 0x7fa53c1032f0 2024-05-09T05:43:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa5327fc640 1 -- 172.21.15.115:0/4287617473 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0a5160 con 0x5610208a22f0 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa5327fc640 1 -- 172.21.15.115:0/4287617473 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa52c0a8db0 con 0x5610208a22f0 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.646+0000 7fa5327fc640 1 -- 172.21.15.115:0/4287617473 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0a8a10 con 0x5610208a22f0 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa54924c740 1 -- 172.21.15.115:0/4287617473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 msgr2=0x561020c6ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa540933640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5280b2950 0x7fa5280b4e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x561020c6b7f0 tx=0x7fa5340096a0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa54924c740 1 --2- 172.21.15.115:0/4287617473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6ebc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0df5d0 tx=0x7fa52c0ea950 comp rx=0 tx=0).stop 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa54924c740 1 -- 172.21.15.115:0/4287617473 shutdown_connections 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa54924c740 1 --2- 172.21.15.115:0/4287617473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa54924c740 1 --2- 172.21.15.115:0/4287617473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6ebc0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa54924c740 1 --2- 172.21.15.115:0/4287617473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c6e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa54924c740 1 -- 172.21.15.115:0/4287617473 >> 172.21.15.115:0/4287617473 conn(0x5610208bf300 msgr2=0x561020c6ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa54924c740 1 -- 172.21.15.115:0/4287617473 shutdown_connections 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.647+0000 7fa54924c740 1 -- 172.21.15.115:0/4287617473 wait complete. 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa54924c740 1 -- start start 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa54924c740 1 Processor -- start 2024-05-09T05:43:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa54924c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c7a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa54924c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208aa4d0 0x561020c7ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa54924c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208cb1b0 0x561020c78ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa54924c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561020c7b630 con 0x5610208a22f0 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa54924c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561020b5a9c0 con 0x5610208cb1b0 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa54924c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561020c7b7b0 con 0x5610208aa4d0 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c7a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa54924c740 10 client.0 _ll_register_callbacks cb 0x561020b87020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa533fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208aa4d0 0x561020c7ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c7a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37840/0 (socket says 172.21.15.115:37840) 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa53bfff640 1 -- 172.21.15.115:0/194082288 learned_addr learned my addr 172.21.15.115:0/194082288 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.648+0000 7fa540933640 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208cb1b0 0x561020c78ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa53bfff640 1 -- 172.21.15.115:0/194082288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208aa4d0 msgr2=0x561020c7ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa53bfff640 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208aa4d0 0x561020c7ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa53bfff640 1 -- 172.21.15.115:0/194082288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208cb1b0 msgr2=0x561020c78ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa53bfff640 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208cb1b0 0x561020c78ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa53bfff640 1 -- 172.21.15.115:0/194082288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561020b811f0 con 0x5610208a22f0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa533fff640 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208aa4d0 0x561020c7ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa53bfff640 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c7a6e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0a9230 tx=0x7fa52c00dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3310acd34b42e751 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa51b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa51b7fe640 1 -- 172.21.15.115:0/194082288 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0a8bf0 con 0x5610208a22f0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa51b7fe640 1 -- 172.21.15.115:0/194082288 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa52c0df8e0 con 0x5610208a22f0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa54924c740 1 -- 172.21.15.115:0/194082288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa52c0a8db0 con 0x5610208a22f0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa51b7fe640 1 -- 172.21.15.115:0/194082288 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa52c0a9070 con 0x5610208a22f0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa54924c740 10 client.16989 fetch_fsmap learned FSMap version 35 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa54924c740 1 -- 172.21.15.115:0/194082288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561020936790 con 0x5610208a22f0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.649+0000 7fa51b7fe640 1 -- 172.21.15.115:0/194082288 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa52c0a8db0 con 0x5610208a22f0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa51b7fe640 1 -- 172.21.15.115:0/194082288 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa52c0ad160 con 0x5610208a22f0 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa54924c740 10 client.16989 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T05:43:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa54924c740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa54924c740 20 client.16989 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa54924c740 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561020889d80 0x561020c8aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa54924c740 4 client.16989 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa54924c740 1 -- 172.21.15.115:0/194082288 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561020be13d0 con 0x561020889d80 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa533fff640 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561020889d80 0x561020c8aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa51b7fe640 10 client.16989 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.650+0000 7fa533fff640 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561020889d80 0x561020c8aca0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ba75fe0a476942 server_cookie=6ec6a4b7abb73d6a in_seq=0 out_seq=0 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa51b7fe640 1 -- 172.21.15.115:0/194082288 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561020be13d0 con 0x561020889d80 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa51b7fe640 10 client.16989 handle_command_reply: tid=0 2024-05-09T05:43:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 client.16989 shutdown 2024-05-09T05:43:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 -- 172.21.15.115:0/194082288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 msgr2=0x561020c7a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c7a6e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0a9230 tx=0x7fa52c00dea0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 -- 172.21.15.115:0/194082288 shutdown_connections 2024-05-09T05:43:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208cb1b0 0x561020c78ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561020889d80 0x561020c8aca0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208aa4d0 0x561020c7ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 --2- 172.21.15.115:0/194082288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c7a6e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa51b7fe640 0 client.16989 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 -- 172.21.15.115:0/194082288 >> 172.21.15.115:0/194082288 conn(0x5610208bf300 msgr2=0x561020c703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 -- 172.21.15.115:0/194082288 shutdown_connections 2024-05-09T05:43:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 1 -- 172.21.15.115:0/194082288 wait complete. 2024-05-09T05:43:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.651+0000 7fa54924c740 20 client.16989 trim_cache size 0 max 16384 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.665+0000 7f025bfff700 1 -- 172.21.15.115:0/502798631 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cf6365db00 con 0x55cf638d9520 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.665+0000 7f025bfff700 10 client.26393 handle_command_reply: tid=0 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 client.26393 shutdown 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 -- 172.21.15.115:0/502798631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 msgr2=0x55cf638cfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 0x55cf638cfba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f02700a96a0 tx=0x7f02700a9890 comp rx=0 tx=0).stop 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 -- 172.21.15.115:0/502798631 shutdown_connections 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf63742630 0x55cf638cfba0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf638d9520 0x55cf639468e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf638bf8b0 0x55cf63942d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f025bfff700 0 client.26393 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 --2- 172.21.15.115:0/502798631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf637bce90 0x55cf638d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 -- 172.21.15.115:0/502798631 >> 172.21.15.115:0/502798631 conn(0x55cf63746100 msgr2=0x55cf638c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 -- 172.21.15.115:0/502798631 shutdown_connections 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 1 -- 172.21.15.115:0/502798631 wait complete. 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.666+0000 7f0291c16b80 20 client.26393 trim_cache size 0 max 16384 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 35, 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 14, 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "2c61de64": { 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 14, 2024-05-09T05:43:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 135.5, 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:29.663 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:43:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa54924c740 1 Processor -- start 2024-05-09T05:43:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa54924c740 1 -- start start 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 -- 172.21.15.115:0/853836558 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f026c0b2a70 msgr2=0x7f026c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f026c0b2a70 0x7f026c0b4f80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f02680098e0 tx=0x7f026800b610 comp rx=0 tx=0).stop 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 -- 172.21.15.115:0/853836558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f027c104400 msgr2=0x7f027c1a2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f027c104400 0x7f027c1a2040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f027000e9f0 tx=0x7f027000ed00 comp rx=0 tx=0).stop 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 -- 172.21.15.115:0/853836558 shutdown_connections 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f026c0b2a70 0x7f026c0b4f80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f027c104400 0x7f027c1a2040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f027c109680 0x7f027c1acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 --2- 172.21.15.115:0/853836558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c108d30 0x7f027c1a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 -- 172.21.15.115:0/853836558 >> 172.21.15.115:0/853836558 conn(0x7f027c079d10 msgr2=0x7f027c101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa54924c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 -- 172.21.15.115:0/853836558 shutdown_connections 2024-05-09T05:43:29.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:29.667+0000 7f0281f9e700 1 -- 172.21.15.115:0/853836558 wait complete. 2024-05-09T05:43:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa54924c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa54924c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c75de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa54924c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1aee60 con 0x5610208a22f0 2024-05-09T05:43:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa54924c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1b0a30 con 0x5610208aa4d0 2024-05-09T05:43:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa54924c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1af530 con 0x5610208cb1b0 2024-05-09T05:43:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa533fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37850/0 (socket says 172.21.15.115:37850) 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa533fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51608/0 (socket says 172.21.15.115:51608) 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa53bfff640 1 -- 172.21.15.115:0/895252520 learned_addr learned my addr 172.21.15.115:0/895252520 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.667+0000 7fa540933640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c75de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa540933640 1 -- 172.21.15.115:0/895252520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 msgr2=0x561020c6ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa540933640 1 --2- 172.21.15.115:0/895252520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa540933640 1 -- 172.21.15.115:0/895252520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 msgr2=0x561020c6a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa540933640 1 --2- 172.21.15.115:0/895252520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa540933640 1 -- 172.21.15.115:0/895252520 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610209ad5f0 con 0x5610208cb1b0 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa533fff640 1 --2- 172.21.15.115:0/895252520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa540933640 1 --2- 172.21.15.115:0/895252520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c75de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa534005fd0 tx=0x7fa534016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8788b84523d2e983 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa531ffb640 1 -- 172.21.15.115:0/895252520 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534020b10 con 0x5610208cb1b0 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa531ffb640 1 -- 172.21.15.115:0/895252520 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa534020cb0 con 0x5610208cb1b0 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.668+0000 7fa531ffb640 1 -- 172.21.15.115:0/895252520 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534011440 con 0x5610208cb1b0 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.669+0000 7fa54924c740 1 -- 172.21.15.115:0/895252520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 msgr2=0x561020c75de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.669+0000 7fa54924c740 1 --2- 172.21.15.115:0/895252520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c75de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa534005fd0 tx=0x7fa534016000 comp rx=0 tx=0).stop 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.670+0000 7fa54924c740 1 -- 172.21.15.115:0/895252520 shutdown_connections 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.670+0000 7fa54924c740 1 --2- 172.21.15.115:0/895252520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c75de0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.670+0000 7fa54924c740 1 --2- 172.21.15.115:0/895252520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208aa4d0 0x561020c6ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.670+0000 7fa54924c740 1 --2- 172.21.15.115:0/895252520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208a22f0 0x561020c6a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.670+0000 7fa54924c740 1 -- 172.21.15.115:0/895252520 >> 172.21.15.115:0/895252520 conn(0x5610208bf300 msgr2=0x561020c77470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.670+0000 7fa54924c740 1 -- 172.21.15.115:0/895252520 shutdown_connections 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.670+0000 7fa54924c740 1 -- 172.21.15.115:0/895252520 wait complete. 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.670+0000 7fa54924c740 1 -- start start 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.670+0000 7fa54924c740 1 Processor -- start 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa54924c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208a22f0 0x561020c8de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa54924c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208aa4d0 0x561020c8e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa54924c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c8b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa54924c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1b0300 con 0x5610208aa4d0 2024-05-09T05:43:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa54924c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1afc00 con 0x5610208a22f0 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa54924c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1ace40 con 0x5610208cb1b0 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa54924c740 10 client.0 _ll_register_callbacks cb 0x7fa524009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa533fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208aa4d0 0x561020c8e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208a22f0 0x561020c8de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa540933640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c8b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa533fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208aa4d0 0x561020c8e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37856/0 (socket says 172.21.15.115:37856) 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa533fff640 1 -- 172.21.15.115:0/664226967 learned_addr learned my addr 172.21.15.115:0/664226967 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208a22f0 0x561020c8de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51614/0 (socket says 172.21.15.115:51614) 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa53bfff640 1 -- 172.21.15.115:0/664226967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 msgr2=0x561020c8b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa53bfff640 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c8b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa53bfff640 1 -- 172.21.15.115:0/664226967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208aa4d0 msgr2=0x561020c8e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa53bfff640 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208aa4d0 0x561020c8e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa53bfff640 1 -- 172.21.15.115:0/664226967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561020b97ed0 con 0x5610208a22f0 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa533fff640 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208aa4d0 0x561020c8e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.671+0000 7fa540933640 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c8b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa53bfff640 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208a22f0 0x561020c8de30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0e5980 tx=0x7fa52c0e21e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67fb35899b048926 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa51b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa51b7fe640 1 -- 172.21.15.115:0/664226967 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0a8bf0 con 0x5610208a22f0 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa51b7fe640 1 -- 172.21.15.115:0/664226967 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa52c0b1bc0 con 0x5610208a22f0 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa54924c740 1 -- 172.21.15.115:0/664226967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa534020cb0 con 0x5610208a22f0 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa51b7fe640 1 -- 172.21.15.115:0/664226967 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa52c0a8db0 con 0x5610208a22f0 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa54924c740 10 client.26411 fetch_fsmap learned FSMap version 35 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa54924c740 1 -- 172.21.15.115:0/664226967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561020cf26e0 con 0x5610208a22f0 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa51b7fe640 1 -- 172.21.15.115:0/664226967 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa534020cb0 con 0x5610208a22f0 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.672+0000 7fa51b7fe640 1 -- 172.21.15.115:0/664226967 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa52c0ca0f0 con 0x5610208a22f0 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.673+0000 7fa54924c740 10 client.26411 fetch_fsmap finished waiting for FSMap version 35 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.673+0000 7fa54924c740 10 client.26411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.673+0000 7fa54924c740 20 client.26411 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.673+0000 7fa54924c740 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56102085f1c0 0x561020cf4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.673+0000 7fa54924c740 4 client.26411 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:43:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.673+0000 7fa54924c740 1 -- 172.21.15.115:0/664226967 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561020bdb240 con 0x56102085f1c0 2024-05-09T05:43:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.673+0000 7fa533fff640 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56102085f1c0 0x561020cf4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.673+0000 7fa51b7fe640 10 client.26411 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.673+0000 7fa533fff640 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56102085f1c0 0x561020cf4c80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b7439eb693eab93 server_cookie=f1b88ac4ce5eb607 in_seq=0 out_seq=0 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa51b7fe640 1 -- 172.21.15.115:0/664226967 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561020bdb240 con 0x56102085f1c0 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa51b7fe640 10 client.26411 handle_command_reply: tid=0 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 client.26411 shutdown 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 -- 172.21.15.115:0/664226967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208a22f0 msgr2=0x561020c8de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208a22f0 0x561020c8de30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0e5980 tx=0x7fa52c0e21e0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 -- 172.21.15.115:0/664226967 shutdown_connections 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56102085f1c0 0x561020cf4c80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5610208cb1b0 0x561020c8b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5610208aa4d0 0x561020c8e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 --2- 172.21.15.115:0/664226967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5610208a22f0 0x561020c8de30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa51b7fe640 0 client.26411 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 -- 172.21.15.115:0/664226967 >> 172.21.15.115:0/664226967 conn(0x5610208bf300 msgr2=0x561020c70130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 -- 172.21.15.115:0/664226967 shutdown_connections 2024-05-09T05:43:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.683+0000 7fa54924c740 1 -- 172.21.15.115:0/664226967 wait complete. 2024-05-09T05:43:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa54924c740 20 client.26411 trim_cache size 0 max 16384 2024-05-09T05:43:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:43:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:43:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:43:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 -- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5280b2950 msgr2=0x7fa5280b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5280b2950 0x7fa5280b4e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x561020c6b7f0 tx=0x7fa5340096a0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 -- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c1032f0 msgr2=0x7fa53c199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c1032f0 0x7fa53c199770 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e9f0 tx=0x7fa52c00eeb0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 -- 172.21.15.115:0/2712296003 shutdown_connections 2024-05-09T05:43:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5280b2950 0x7fa5280b4e60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa53c109040 0x7fa53c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa53c108ba0 0x7fa53c199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 --2- 172.21.15.115:0/2712296003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa53c1032f0 0x7fa53c199770 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 -- 172.21.15.115:0/2712296003 >> 172.21.15.115:0/2712296003 conn(0x7fa53c0782e0 msgr2=0x7fa53c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 -- 172.21.15.115:0/2712296003 shutdown_connections 2024-05-09T05:43:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:29.684+0000 7fa542396640 1 -- 172.21.15.115:0/2712296003 wait complete. 2024-05-09T05:43:30.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:30 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:43:30.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:30 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:43:30.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:30 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:43:30.554 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2c61de64', quiesce took 17.1 seconds. Will release after: -1.6000000000000014 2024-05-09T05:43:30.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:43:31.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:31 smithi195 ceph-mon[33844]: pgmap v162: 129 pgs: 129 active+clean; 3.6 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:43:31.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:31 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:43:31.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:31 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:43:31.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:31 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:43:31.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:31 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:43:31.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:31 smithi115 ceph-mon[30349]: pgmap v162: 129 pgs: 129 active+clean; 3.6 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:43:31.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:31 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:43:31.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:31 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:43:31.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:31 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:43:31.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:31 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:43:31.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:31 smithi161 ceph-mon[33805]: pgmap v162: 129 pgs: 129 active+clean; 3.6 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:43:31.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:31 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:43:31.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:31 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:43:31.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:31 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:43:31.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:31 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:43:31.638 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.396+0000 7fa674002700 1 -- 172.21.15.115:0/1851200191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c103c40 msgr2=0x7fa66c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.396+0000 7fa674002700 1 --2- 172.21.15.115:0/1851200191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c103c40 0x7fa66c104050 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa66800b0d0 tx=0x7fa66800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.396+0000 7fa674002700 1 -- 172.21.15.115:0/1851200191 shutdown_connections 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.396+0000 7fa674002700 1 --2- 172.21.15.115:0/1851200191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa66c108790 0x7fa66c1100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.396+0000 7fa674002700 1 --2- 172.21.15.115:0/1851200191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c103c40 0x7fa66c104050 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.396+0000 7fa674002700 1 --2- 172.21.15.115:0/1851200191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa66c107e40 0x7fa66c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.396+0000 7fa674002700 1 -- 172.21.15.115:0/1851200191 >> 172.21.15.115:0/1851200191 conn(0x7fa66c0fe6f0 msgr2=0x7fa66c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.396+0000 7fa674002700 1 -- 172.21.15.115:0/1851200191 shutdown_connections 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.396+0000 7fa674002700 1 -- 172.21.15.115:0/1851200191 wait complete. 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.397+0000 7fa674002700 1 Processor -- start 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.397+0000 7fa674002700 1 -- start start 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.397+0000 7fa674002700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa66c103c40 0x7fa66c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.397+0000 7fa674002700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c107e40 0x7fa66c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa674002700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa66c108790 0x7fa66c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa674002700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1132c0 con 0x7fa66c107e40 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa674002700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c113140 con 0x7fa66c103c40 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa674002700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c113440 con 0x7fa66c108790 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67159d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c107e40 0x7fa66c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67159d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c107e40 0x7fa66c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37864/0 (socket says 172.21.15.115:37864) 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67159d700 1 -- 172.21.15.115:0/455285499 learned_addr learned my addr 172.21.15.115:0/455285499 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa671d9e700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa66c103c40 0x7fa66c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67259f700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa66c108790 0x7fa66c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67159d700 1 -- 172.21.15.115:0/455285499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa66c108790 msgr2=0x7fa66c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67159d700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa66c108790 0x7fa66c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67159d700 1 -- 172.21.15.115:0/455285499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa66c103c40 msgr2=0x7fa66c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67159d700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa66c103c40 0x7fa66c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67159d700 1 -- 172.21.15.115:0/455285499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c19ec20 con 0x7fa66c107e40 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa67159d700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c107e40 0x7fa66c19a2f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0098f0 tx=0x7fa65c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.398+0000 7fa662ffd700 1 -- 172.21.15.115:0/455285499 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00d020 con 0x7fa66c107e40 2024-05-09T05:43:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.399+0000 7fa674002700 1 -- 172.21.15.115:0/455285499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c19ef10 con 0x7fa66c107e40 2024-05-09T05:43:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.399+0000 7fa662ffd700 1 -- 172.21.15.115:0/455285499 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa65c00d1c0 con 0x7fa66c107e40 2024-05-09T05:43:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.399+0000 7fa662ffd700 1 -- 172.21.15.115:0/455285499 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c007730 con 0x7fa66c107e40 2024-05-09T05:43:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.399+0000 7fa674002700 1 -- 172.21.15.115:0/455285499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c1028a0 con 0x7fa66c107e40 2024-05-09T05:43:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.401+0000 7fa662ffd700 1 -- 172.21.15.115:0/455285499 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa65c00c460 con 0x7fa66c107e40 2024-05-09T05:43:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.402+0000 7fa662ffd700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6580b29f0 0x7fa6580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.402+0000 7fa671d9e700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6580b29f0 0x7fa6580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.402+0000 7fa662ffd700 1 -- 172.21.15.115:0/455285499 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa65c0e3b50 con 0x7fa66c107e40 2024-05-09T05:43:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.402+0000 7fa682a18b80 1 Processor -- start 2024-05-09T05:43:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa682a18b80 1 -- start start 2024-05-09T05:43:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa671d9e700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6580b29f0 0x7fa6580b4f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa668009fd0 tx=0x7fa66800b040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 0x5586d48eb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa682a18b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5586d48090a0 con 0x5586d48ef1e0 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa682a18b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5586d47ae9a0 con 0x5586d47e3d00 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa682a18b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5586d4805060 con 0x5586d47694a0 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa67259f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa67259f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48ef5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37872/0 (socket says 172.21.15.115:37872) 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa67259f700 1 -- 172.21.15.115:0/39275334 learned_addr learned my addr 172.21.15.115:0/39275334 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa671d9e700 1 --2- 172.21.15.115:0/39275334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa67159d700 1 --2- 172.21.15.115:0/39275334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 0x5586d48eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa67259f700 1 -- 172.21.15.115:0/39275334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 msgr2=0x5586d48eb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa67259f700 1 --2- 172.21.15.115:0/39275334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 0x5586d48eb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa67259f700 1 -- 172.21.15.115:0/39275334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 msgr2=0x5586d48eaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa67259f700 1 --2- 172.21.15.115:0/39275334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48eaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.403+0000 7fa67259f700 1 -- 172.21.15.115:0/39275334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d479f500 con 0x5586d48ef1e0 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.404+0000 7fa67259f700 1 --2- 172.21.15.115:0/39275334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48ef5f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa6640046a0 tx=0x7fa66400b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d743609ab3ecab1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.404+0000 7fa657fff700 1 -- 172.21.15.115:0/39275334 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664004a30 con 0x5586d48ef1e0 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.404+0000 7fa657fff700 1 -- 172.21.15.115:0/39275334 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa664015070 con 0x5586d48ef1e0 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.404+0000 7fa657fff700 1 -- 172.21.15.115:0/39275334 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6640039a0 con 0x5586d48ef1e0 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 -- 172.21.15.115:0/39275334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 msgr2=0x5586d48ef5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 --2- 172.21.15.115:0/39275334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48ef5f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa6640046a0 tx=0x7fa66400b970 comp rx=0 tx=0).stop 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 -- 172.21.15.115:0/39275334 shutdown_connections 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 --2- 172.21.15.115:0/39275334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48eaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 --2- 172.21.15.115:0/39275334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48ef5f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 --2- 172.21.15.115:0/39275334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 0x5586d48eb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 -- 172.21.15.115:0/39275334 >> 172.21.15.115:0/39275334 conn(0x5586d476cf70 msgr2=0x5586d48e1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 -- 172.21.15.115:0/39275334 shutdown_connections 2024-05-09T05:43:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 -- 172.21.15.115:0/39275334 wait complete. 2024-05-09T05:43:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 -- start start 2024-05-09T05:43:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.405+0000 7fa682a18b80 1 Processor -- start 2024-05-09T05:43:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47694a0 0x5586d48f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 0x5586d48f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa682a18b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5586d48eff00 con 0x5586d47694a0 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa682a18b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5586d47a3b20 con 0x5586d47e3d00 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa682a18b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5586d48f7470 con 0x5586d48ef1e0 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa682a18b80 10 client.0 _ll_register_callbacks cb 0x5586d467a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa671d9e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47694a0 0x5586d48f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa671d9e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47694a0 0x5586d48f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37878/0 (socket says 172.21.15.115:37878) 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa671d9e700 1 -- 172.21.15.115:0/3013232555 learned_addr learned my addr 172.21.15.115:0/3013232555 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa67259f700 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 0x5586d48f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa671d9e700 1 -- 172.21.15.115:0/3013232555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 msgr2=0x5586d48f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa671d9e700 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 0x5586d48f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa67159d700 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa671d9e700 1 -- 172.21.15.115:0/3013232555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 msgr2=0x5586d48f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa671d9e700 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa671d9e700 1 -- 172.21.15.115:0/3013232555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d4684490 con 0x5586d47694a0 2024-05-09T05:43:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.406+0000 7fa67159d700 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa671d9e700 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47694a0 0x5586d48f43c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa66c06afd0 tx=0x7fa66800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78788d2d8318db78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa6557fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa6557fa700 1 -- 172.21.15.115:0/3013232555 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668022070 con 0x5586d47694a0 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa6557fa700 1 -- 172.21.15.115:0/3013232555 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa668029840 con 0x5586d47694a0 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa682a18b80 1 -- 172.21.15.115:0/3013232555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa664015070 con 0x5586d47694a0 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa6557fa700 1 -- 172.21.15.115:0/3013232555 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa668009450 con 0x5586d47694a0 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa682a18b80 10 client.17007 fetch_fsmap learned FSMap version 36 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa682a18b80 1 -- 172.21.15.115:0/3013232555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5586d4901d50 con 0x5586d47694a0 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa6557fa700 1 -- 172.21.15.115:0/3013232555 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa664015070 con 0x5586d47694a0 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa6557fa700 1 -- 172.21.15.115:0/3013232555 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa668011070 con 0x5586d47694a0 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa682a18b80 10 client.17007 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa682a18b80 10 client.17007 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa682a18b80 20 client.17007 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5586d49043e0 0x5586d4906880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa682a18b80 4 client.17007 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.407+0000 7fa682a18b80 1 -- 172.21.15.115:0/3013232555 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d4816a80 con 0x5586d49043e0 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.408+0000 7fa67159d700 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5586d49043e0 0x5586d4906880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.408+0000 7fa67159d700 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5586d49043e0 0x5586d4906880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4194dae3a10afa21 server_cookie=4f5dda11f7b99736 in_seq=0 out_seq=0 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.408+0000 7fa6557fa700 10 client.17007 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa6557fa700 1 -- 172.21.15.115:0/3013232555 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586d4816a80 con 0x5586d49043e0 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa6557fa700 10 client.17007 handle_command_reply: tid=0 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa682a18b80 1 client.17007 shutdown 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa682a18b80 1 -- 172.21.15.115:0/3013232555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47694a0 msgr2=0x5586d48f43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47694a0 0x5586d48f43c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa66c06afd0 tx=0x7fa66800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa682a18b80 1 -- 172.21.15.115:0/3013232555 shutdown_connections 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5586d49043e0 0x5586d4906880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 0x5586d48f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47694a0 0x5586d48f43c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3013232555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa6557fa700 0 client.17007 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:43:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.409+0000 7fa682a18b80 1 -- 172.21.15.115:0/3013232555 >> 172.21.15.115:0/3013232555 conn(0x5586d476cf70 msgr2=0x5586d48daff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.410+0000 7fa682a18b80 1 -- 172.21.15.115:0/3013232555 shutdown_connections 2024-05-09T05:43:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.410+0000 7fa682a18b80 1 -- 172.21.15.115:0/3013232555 wait complete. 2024-05-09T05:43:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.410+0000 7fa682a18b80 20 client.17007 trim_cache size 0 max 16384 2024-05-09T05:43:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.426+0000 7fa682a18b80 1 Processor -- start 2024-05-09T05:43:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.426+0000 7fa682a18b80 1 -- start start 2024-05-09T05:43:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.426+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 0x5586d48f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.426+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.426+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48eeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.426+0000 7fa682a18b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1ae2c0 con 0x5586d48ef1e0 2024-05-09T05:43:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.426+0000 7fa682a18b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1afe60 con 0x5586d47e3d00 2024-05-09T05:43:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.426+0000 7fa682a18b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1ae990 con 0x5586d47694a0 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67259f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48eeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67259f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48eeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37880/0 (socket says 172.21.15.115:37880) 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67259f700 1 -- 172.21.15.115:0/3273829378 learned_addr learned my addr 172.21.15.115:0/3273829378 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa671d9e700 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 0x5586d48f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67159d700 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67259f700 1 -- 172.21.15.115:0/3273829378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 msgr2=0x5586d48f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67259f700 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 0x5586d48f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67259f700 1 -- 172.21.15.115:0/3273829378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 msgr2=0x5586d48f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67259f700 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67259f700 1 -- 172.21.15.115:0/3273829378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d48f2950 con 0x5586d48ef1e0 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67159d700 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa67259f700 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48eeae0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa6640051d0 tx=0x7fa664005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffce01335c2d8bea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:32.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa6577fe700 1 -- 172.21.15.115:0/3273829378 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6640053a0 con 0x5586d48ef1e0 2024-05-09T05:43:32.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa6577fe700 1 -- 172.21.15.115:0/3273829378 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa664015070 con 0x5586d48ef1e0 2024-05-09T05:43:32.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.427+0000 7fa6577fe700 1 -- 172.21.15.115:0/3273829378 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6640049c0 con 0x5586d48ef1e0 2024-05-09T05:43:32.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.428+0000 7fa682a18b80 1 -- 172.21.15.115:0/3273829378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 msgr2=0x5586d48eeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:32.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.428+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48eeae0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa6640051d0 tx=0x7fa664005200 comp rx=0 tx=0).stop 2024-05-09T05:43:32.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.428+0000 7fa682a18b80 1 -- 172.21.15.115:0/3273829378 shutdown_connections 2024-05-09T05:43:32.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.428+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d48ef1e0 0x5586d48eeae0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.428+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d47694a0 0x5586d48f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.428+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3273829378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47e3d00 0x5586d48f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.428+0000 7fa682a18b80 1 -- 172.21.15.115:0/3273829378 >> 172.21.15.115:0/3273829378 conn(0x5586d476cf70 msgr2=0x5586d48ee3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.428+0000 7fa682a18b80 1 -- 172.21.15.115:0/3273829378 shutdown_connections 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.428+0000 7fa682a18b80 1 -- 172.21.15.115:0/3273829378 wait complete. 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa682a18b80 1 -- start start 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa682a18b80 1 Processor -- start 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47694a0 0x5586d48fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47e3d00 0x5586d48fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa682a18b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 0x5586d48ed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa682a18b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1af760 con 0x5586d47e3d00 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa682a18b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1af060 con 0x5586d47694a0 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa682a18b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1ac260 con 0x5586d48ef1e0 2024-05-09T05:43:32.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa682a18b80 10 client.0 _ll_register_callbacks cb 0x5586d467a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa67159d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47e3d00 0x5586d48fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa67159d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47e3d00 0x5586d48fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37884/0 (socket says 172.21.15.115:37884) 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa67159d700 1 -- 172.21.15.115:0/3668845384 learned_addr learned my addr 172.21.15.115:0/3668845384 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa67259f700 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 0x5586d48ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.429+0000 7fa671d9e700 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47694a0 0x5586d48fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa67159d700 1 -- 172.21.15.115:0/3668845384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 msgr2=0x5586d48ed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa67159d700 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 0x5586d48ed010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa67159d700 1 -- 172.21.15.115:0/3668845384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47694a0 msgr2=0x5586d48fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa67159d700 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47694a0 0x5586d48fc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa67159d700 1 -- 172.21.15.115:0/3668845384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d48ed550 con 0x5586d47e3d00 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa67159d700 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47e3d00 0x5586d48fcae0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0e1560 tx=0x7fa65c0a6ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62341db28e449a13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa6557fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa6557fa700 1 -- 172.21.15.115:0/3668845384 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c090060 con 0x5586d47e3d00 2024-05-09T05:43:32.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa6557fa700 1 -- 172.21.15.115:0/3668845384 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa65c0aad50 con 0x5586d47e3d00 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa682a18b80 1 -- 172.21.15.115:0/3668845384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa664015070 con 0x5586d47e3d00 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa6557fa700 1 -- 172.21.15.115:0/3668845384 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa65c0aaaf0 con 0x5586d47e3d00 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa682a18b80 10 client.17013 fetch_fsmap learned FSMap version 36 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa682a18b80 1 -- 172.21.15.115:0/3668845384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5586d490b8c0 con 0x5586d47e3d00 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa6557fa700 1 -- 172.21.15.115:0/3668845384 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa664015070 con 0x5586d47e3d00 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa6557fa700 1 -- 172.21.15.115:0/3668845384 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa65c0b34c0 con 0x5586d47e3d00 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa682a18b80 10 client.17013 fetch_fsmap finished waiting for FSMap version 36 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa682a18b80 10 client.17013 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa682a18b80 20 client.17013 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5586d496ec30 0x5586d4971050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa682a18b80 4 client.17013 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.430+0000 7fa682a18b80 1 -- 172.21.15.115:0/3668845384 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586d4810b20 con 0x5586d496ec30 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.431+0000 7fa671d9e700 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5586d496ec30 0x5586d4971050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.431+0000 7fa6557fa700 10 client.17013 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:43:32.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.431+0000 7fa671d9e700 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5586d496ec30 0x5586d4971050 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fdea3c32428c35b server_cookie=755791da3af4b945 in_seq=0 out_seq=0 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.431+0000 7fa6557fa700 1 -- 172.21.15.115:0/3668845384 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586d4810b20 con 0x5586d496ec30 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.431+0000 7fa6557fa700 10 client.17013 handle_command_reply: tid=0 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.431+0000 7fa682a18b80 1 client.17013 shutdown 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.431+0000 7fa682a18b80 1 -- 172.21.15.115:0/3668845384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47e3d00 msgr2=0x5586d48fcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47e3d00 0x5586d48fcae0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0e1560 tx=0x7fa65c0a6ae0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 1 -- 172.21.15.115:0/3668845384 shutdown_connections 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5586d496ec30 0x5586d4971050 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586d47694a0 0x5586d48fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586d48ef1e0 0x5586d48ed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 1 --2- 172.21.15.115:0/3668845384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5586d47e3d00 0x5586d48fcae0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa6557fa700 0 client.17013 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 1 -- 172.21.15.115:0/3668845384 >> 172.21.15.115:0/3668845384 conn(0x5586d476cf70 msgr2=0x5586d48daff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 1 -- 172.21.15.115:0/3668845384 shutdown_connections 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 1 -- 172.21.15.115:0/3668845384 wait complete. 2024-05-09T05:43:32.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa682a18b80 20 client.17013 trim_cache size 0 max 16384 2024-05-09T05:43:32.431 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:43:32.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 -- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6580b29f0 msgr2=0x7fa6580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6580b29f0 0x7fa6580b4f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa668009fd0 tx=0x7fa66800b040 comp rx=0 tx=0).stop 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 -- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c107e40 msgr2=0x7fa66c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c107e40 0x7fa66c19a2f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0098f0 tx=0x7fa65c009c00 comp rx=0 tx=0).stop 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 -- 172.21.15.115:0/455285499 shutdown_connections 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6580b29f0 0x7fa6580b4f00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa66c103c40 0x7fa66c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa66c108790 0x7fa66c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 --2- 172.21.15.115:0/455285499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa66c107e40 0x7fa66c19a2f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.432+0000 7fa674002700 1 -- 172.21.15.115:0/455285499 >> 172.21.15.115:0/455285499 conn(0x7fa66c0fe6f0 msgr2=0x7fa66c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.433+0000 7fa674002700 1 -- 172.21.15.115:0/455285499 shutdown_connections 2024-05-09T05:43:32.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:32.433+0000 7fa674002700 1 -- 172.21.15.115:0/455285499 wait complete. 2024-05-09T05:43:33.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:33 smithi195 ceph-mon[33844]: pgmap v163: 129 pgs: 129 active+clean; 3.6 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:43:33.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:33 smithi115 ceph-mon[30349]: pgmap v163: 129 pgs: 129 active+clean; 3.6 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:43:33.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:33 smithi161 ceph-mon[33805]: pgmap v163: 129 pgs: 129 active+clean; 3.6 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:43:33.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:43:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.854+0000 7fe298df0700 1 -- 172.21.15.115:0/2609241807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294108200 msgr2=0x7fe294108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.854+0000 7fe298df0700 1 --2- 172.21.15.115:0/2609241807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294108200 0x7fe294108610 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe284009990 tx=0x7fe284009ca0 comp rx=0 tx=0).stop 2024-05-09T05:43:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.854+0000 7fe298df0700 1 -- 172.21.15.115:0/2609241807 shutdown_connections 2024-05-09T05:43:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.854+0000 7fe298df0700 1 --2- 172.21.15.115:0/2609241807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2941042b0 0x7fe294110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.854+0000 7fe298df0700 1 --2- 172.21.15.115:0/2609241807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294108200 0x7fe294108610 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.854+0000 7fe298df0700 1 --2- 172.21.15.115:0/2609241807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe294103960 0x7fe294103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.854+0000 7fe298df0700 1 -- 172.21.15.115:0/2609241807 >> 172.21.15.115:0/2609241807 conn(0x7fe2940fe520 msgr2=0x7fe294100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.854+0000 7fe298df0700 1 -- 172.21.15.115:0/2609241807 shutdown_connections 2024-05-09T05:43:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.854+0000 7fe298df0700 1 -- 172.21.15.115:0/2609241807 wait complete. 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.855+0000 7fe298df0700 1 Processor -- start 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.855+0000 7fe298df0700 1 -- start start 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe298df0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294103960 0x7fe29419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe298df0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2941042b0 0x7fe29419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe298df0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe294108200 0x7fe29419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe298df0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe294113220 con 0x7fe294103960 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe298df0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941130a0 con 0x7fe294108200 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe298df0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941133a0 con 0x7fe2941042b0 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe29259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294103960 0x7fe29419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe29259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294103960 0x7fe29419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58838/0 (socket says 172.21.15.115:58838) 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe29259c700 1 -- 172.21.15.115:0/3665609604 learned_addr learned my addr 172.21.15.115:0/3665609604 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe292d9d700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe294108200 0x7fe29419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe291d9b700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2941042b0 0x7fe29419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe29259c700 1 -- 172.21.15.115:0/3665609604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2941042b0 msgr2=0x7fe29419c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe29259c700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2941042b0 0x7fe29419c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe29259c700 1 -- 172.21.15.115:0/3665609604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe294108200 msgr2=0x7fe29419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe29259c700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe294108200 0x7fe29419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe29259c700 1 -- 172.21.15.115:0/3665609604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2941a6eb0 con 0x7fe294103960 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe291d9b700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2941042b0 0x7fe29419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.856+0000 7fe29259c700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294103960 0x7fe29419bb60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe284006010 tx=0x7fe2840057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.857+0000 7fe2837fe700 1 -- 172.21.15.115:0/3665609604 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284027070 con 0x7fe294103960 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.857+0000 7fe2837fe700 1 -- 172.21.15.115:0/3665609604 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe28400fbc0 con 0x7fe294103960 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.857+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2941a7140 con 0x7fe294103960 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.857+0000 7fe2837fe700 1 -- 172.21.15.115:0/3665609604 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284022a40 con 0x7fe294103960 2024-05-09T05:43:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.857+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2941a7590 con 0x7fe294103960 2024-05-09T05:43:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.858+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29410b7a0 con 0x7fe294103960 2024-05-09T05:43:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.859+0000 7fe2837fe700 1 -- 172.21.15.115:0/3665609604 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fe28402a070 con 0x7fe294103960 2024-05-09T05:43:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.860+0000 7fe2837fe700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe27c0b29f0 0x7fe27c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.860+0000 7fe2837fe700 1 -- 172.21.15.115:0/3665609604 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fe2840e3d90 con 0x7fe294103960 2024-05-09T05:43:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.860+0000 7fe291d9b700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe27c0b29f0 0x7fe27c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.861+0000 7fe291d9b700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe27c0b29f0 0x7fe27c0b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe288005950 tx=0x7fe2880058e0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:34.864+0000 7fe2837fe700 1 -- 172.21.15.115:0/3665609604 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe28405fdf0 con 0x7fe294103960 2024-05-09T05:43:35.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.130+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2940512c0 con 0x7fe294103960 2024-05-09T05:43:35.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.131+0000 7fe2837fe700 1 -- 172.21.15.115:0/3665609604 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11954 (secure 0 0 0) 0x7fe284013070 con 0x7fe294103960 2024-05-09T05:43:35.125 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:43:35.125 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:43:34.278608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:43:35.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe27c0b29f0 msgr2=0x7fe27c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:35.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe27c0b29f0 0x7fe27c0b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe288005950 tx=0x7fe2880058e0 comp rx=0 tx=0).stop 2024-05-09T05:43:35.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294103960 msgr2=0x7fe29419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:35.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294103960 0x7fe29419bb60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe284006010 tx=0x7fe2840057a0 comp rx=0 tx=0).stop 2024-05-09T05:43:35.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 shutdown_connections 2024-05-09T05:43:35.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe27c0b29f0 0x7fe27c0b4f00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:35.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe294108200 0x7fe29419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:35.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe294103960 0x7fe29419bb60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:35.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 --2- 172.21.15.115:0/3665609604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2941042b0 0x7fe29419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:35.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 >> 172.21.15.115:0/3665609604 conn(0x7fe2940fe520 msgr2=0x7fe294100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:35.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.133+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 shutdown_connections 2024-05-09T05:43:35.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:35.134+0000 7fe298df0700 1 -- 172.21.15.115:0/3665609604 wait complete. 2024-05-09T05:43:35.129 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 37 2024-05-09T05:43:35.222 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:35 smithi115 ceph-mon[30349]: pgmap v164: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 25 op/s 2024-05-09T05:43:35.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:35 smithi195 ceph-mon[33844]: pgmap v164: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 25 op/s 2024-05-09T05:43:35.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:35 smithi161 ceph-mon[33805]: pgmap v164: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 25 op/s 2024-05-09T05:43:36.145 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:36 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:43:36.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:36 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3665609604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:43:36.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:36 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:43:36.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:36 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3665609604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:43:36.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:36 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:43:36.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:36 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3665609604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:43:36.838 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 2c61de64 --release --await' 2024-05-09T05:43:36.838 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id 2c61de64 --release --await 2024-05-09T05:43:37.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.366+0000 7fcaf4ead700 1 -- 172.21.15.115:0/1532661793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf010e970 msgr2=0x7fcaf0110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:37.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.366+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/1532661793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf010e970 0x7fcaf0110e10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcae800b0a0 tx=0x7fcae800b3b0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.366+0000 7fcaf4ead700 1 -- 172.21.15.115:0/1532661793 shutdown_connections 2024-05-09T05:43:37.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.366+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/1532661793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf0105590 0x7fcaf01059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.366+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/1532661793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf010e970 0x7fcaf0110e10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.366+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/1532661793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf0105ee0 0x7fcaf010e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.366+0000 7fcaf4ead700 1 -- 172.21.15.115:0/1532661793 >> 172.21.15.115:0/1532661793 conn(0x7fcaf0078e00 msgr2=0x7fcaf0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:37.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.366+0000 7fcaf4ead700 1 -- 172.21.15.115:0/1532661793 shutdown_connections 2024-05-09T05:43:37.361 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:37 smithi115 ceph-mon[30349]: pgmap v165: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-05-09T05:43:37.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.366+0000 7fcaf4ead700 1 -- 172.21.15.115:0/1532661793 wait complete. 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.367+0000 7fcaf4ead700 1 Processor -- start 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.367+0000 7fcaf4ead700 1 -- start start 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.367+0000 7fcaf4ead700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf0105590 0x7fcaf0103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.367+0000 7fcaf4ead700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf0105ee0 0x7fcaf01037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.367+0000 7fcaf4ead700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf010e970 0x7fcaf0103d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.367+0000 7fcaf4ead700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0114340 con 0x7fcaf0105590 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.367+0000 7fcaf4ead700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01141c0 con 0x7fcaf010e970 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.367+0000 7fcaf4ead700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01144c0 con 0x7fcaf0105ee0 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaee59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf0105590 0x7fcaf0103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaee59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf0105590 0x7fcaf0103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58846/0 (socket says 172.21.15.115:58846) 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaee59c700 1 -- 172.21.15.115:0/2871846658 learned_addr learned my addr 172.21.15.115:0/2871846658 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcae7fff700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf0105ee0 0x7fcaf01037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaeed9d700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf010e970 0x7fcaf0103d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaee59c700 1 -- 172.21.15.115:0/2871846658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf0105ee0 msgr2=0x7fcaf01037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:37.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaee59c700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf0105ee0 0x7fcaf01037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaee59c700 1 -- 172.21.15.115:0/2871846658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf010e970 msgr2=0x7fcaf0103d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:37.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaee59c700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf010e970 0x7fcaf0103d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaee59c700 1 -- 172.21.15.115:0/2871846658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf01a9050 con 0x7fcaf0105590 2024-05-09T05:43:37.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaee59c700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf0105590 0x7fcaf0103280 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcae000e9a0 tx=0x7fcae000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:37.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcae77fe700 1 -- 172.21.15.115:0/2871846658 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae000cbd0 con 0x7fcaf0105590 2024-05-09T05:43:37.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaf4ead700 1 -- 172.21.15.115:0/2871846658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf01a9340 con 0x7fcaf0105590 2024-05-09T05:43:37.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcae77fe700 1 -- 172.21.15.115:0/2871846658 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcae000cd70 con 0x7fcaf0105590 2024-05-09T05:43:37.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcae77fe700 1 -- 172.21.15.115:0/2871846658 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0018680 con 0x7fcaf0105590 2024-05-09T05:43:37.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.368+0000 7fcaf4ead700 1 -- 172.21.15.115:0/2871846658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf01a98d0 con 0x7fcaf0105590 2024-05-09T05:43:37.364 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:37 smithi195 ceph-mon[33844]: pgmap v165: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-05-09T05:43:37.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.372+0000 7fcae77fe700 1 -- 172.21.15.115:0/2871846658 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcae0010430 con 0x7fcaf0105590 2024-05-09T05:43:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.372+0000 7fcb038c3b80 1 Processor -- start 2024-05-09T05:43:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcb038c3b80 1 -- start start 2024-05-09T05:43:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcae77fe700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcadc0b2a70 0x7fcadc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcaeed9d700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcadc0b2a70 0x7fcadc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcae77fe700 1 -- 172.21.15.115:0/2871846658 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fcae00e2b50 con 0x7fcaf0105590 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 0x55bff5f484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f48a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5f426a0 0x55bff5f42ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bff5e64ff0 con 0x55bff5dc54a0 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bff5e037f0 con 0x55bff5f426a0 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bff5e60fb0 con 0x55bff5eaf160 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcae7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f48a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcae7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f48a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58862/0 (socket says 172.21.15.115:58862) 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcae7fff700 1 -- 172.21.15.115:0/3047577083 learned_addr learned my addr 172.21.15.115:0/3047577083 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.373+0000 7fcaee59c700 1 --2- 172.21.15.115:0/3047577083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 0x55bff5f484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcae7fff700 1 -- 172.21.15.115:0/3047577083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 msgr2=0x55bff5f484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcae7fff700 1 --2- 172.21.15.115:0/3047577083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 0x55bff5f484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcae7fff700 1 -- 172.21.15.115:0/3047577083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5f426a0 msgr2=0x55bff5f42ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcae7fff700 1 --2- 172.21.15.115:0/3047577083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5f426a0 0x55bff5f42ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcae7fff700 1 -- 172.21.15.115:0/3047577083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff5dfb370 con 0x55bff5dc54a0 2024-05-09T05:43:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcae7fff700 1 --2- 172.21.15.115:0/3047577083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f48a30 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009ca0 tx=0x7fcad8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c838e6caff6344 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcae4ff9700 1 -- 172.21.15.115:0/3047577083 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8011d80 con 0x55bff5dc54a0 2024-05-09T05:43:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcaeed9d700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcadc0b2a70 0x7fcadc0b4f80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55bff5f490c0 tx=0x7fcae80098c0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcae4ff9700 1 -- 172.21.15.115:0/3047577083 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcad8004cc0 con 0x55bff5dc54a0 2024-05-09T05:43:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.374+0000 7fcae4ff9700 1 -- 172.21.15.115:0/3047577083 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8019430 con 0x55bff5dc54a0 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.375+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3047577083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 msgr2=0x55bff5f48a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.375+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3047577083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f48a30 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009ca0 tx=0x7fcad8011440 comp rx=0 tx=0).stop 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.375+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3047577083 shutdown_connections 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.375+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3047577083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5f426a0 0x55bff5f42ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.375+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3047577083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 0x55bff5f484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.375+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3047577083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f48a30 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.375+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3047577083 >> 172.21.15.115:0/3047577083 conn(0x55bff5e256a0 msgr2=0x55bff5f3d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.375+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3047577083 shutdown_connections 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.375+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3047577083 wait complete. 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcb038c3b80 1 -- start start 2024-05-09T05:43:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcb038c3b80 1 Processor -- start 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5dc54a0 0x55bff5f4c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5eaf160 0x55bff5f4c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5f426a0 0x55bff5f4cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bff5ce0ba0 con 0x55bff5eaf160 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bff5e1f150 con 0x55bff5f426a0 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bff5f4f1d0 con 0x55bff5dc54a0 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcb038c3b80 10 client.0 _ll_register_callbacks cb 0x55bff5cd66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcaee59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5dc54a0 0x55bff5f4c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcae7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5eaf160 0x55bff5f4c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcaee59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5dc54a0 0x55bff5f4c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:58796/0 (socket says 172.21.15.115:58796) 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcaee59c700 1 -- 172.21.15.115:0/2583860656 learned_addr learned my addr 172.21.15.115:0/2583860656 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.376+0000 7fcae7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5eaf160 0x55bff5f4c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58864/0 (socket says 172.21.15.115:58864) 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcaeed9d700 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5f426a0 0x55bff5f4cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcaee59c700 1 -- 172.21.15.115:0/2583860656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5f426a0 msgr2=0x55bff5f4cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcaee59c700 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5f426a0 0x55bff5f4cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcaee59c700 1 -- 172.21.15.115:0/2583860656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5eaf160 msgr2=0x55bff5f4c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcaee59c700 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5eaf160 0x55bff5f4c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcaee59c700 1 -- 172.21.15.115:0/2583860656 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff5ce0490 con 0x55bff5dc54a0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcae7fff700 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5eaf160 0x55bff5f4c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcaee59c700 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5dc54a0 0x55bff5f4c1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcae00a9a80 tx=0x7fcae00b27a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42f32de84aca4855 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcad5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcad5ffb700 1 -- 172.21.15.115:0/2583860656 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae00a5280 con 0x55bff5dc54a0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcb038c3b80 1 -- 172.21.15.115:0/2583860656 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad8004cc0 con 0x55bff5dc54a0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcad5ffb700 1 -- 172.21.15.115:0/2583860656 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fcae00a8f60 con 0x55bff5dc54a0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcad5ffb700 1 -- 172.21.15.115:0/2583860656 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcae00b2290 con 0x55bff5dc54a0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.377+0000 7fcb038c3b80 10 client.26536 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcb038c3b80 1 -- 172.21.15.115:0/2583860656 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bff5f59c50 con 0x55bff5dc54a0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcad5ffb700 1 -- 172.21.15.115:0/2583860656 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcad8004cc0 con 0x55bff5dc54a0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcad5ffb700 1 -- 172.21.15.115:0/2583860656 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fcae00a5460 con 0x55bff5dc54a0 2024-05-09T05:43:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcb038c3b80 10 client.26536 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcb038c3b80 10 client.26536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcb038c3b80 20 client.26536 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bff5f5c290 0x55bff5f5e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcb038c3b80 4 client.26536 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcb038c3b80 1 -- 172.21.15.115:0/2583860656 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bff5e729d0 con 0x55bff5f5c290 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.378+0000 7fcae7fff700 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bff5f5c290 0x55bff5f5e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.379+0000 7fcad5ffb700 10 client.26536 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.379+0000 7fcae7fff700 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bff5f5c290 0x55bff5f5e6b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e50c0b575f5ebe5 server_cookie=c01477f9ff52b022 in_seq=0 out_seq=0 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcad5ffb700 1 -- 172.21.15.115:0/2583860656 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bff5e729d0 con 0x55bff5f5c290 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcad5ffb700 10 client.26536 handle_command_reply: tid=0 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcb038c3b80 1 client.26536 shutdown 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcb038c3b80 1 -- 172.21.15.115:0/2583860656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5dc54a0 msgr2=0x55bff5f4c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5dc54a0 0x55bff5f4c1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcae00a9a80 tx=0x7fcae00b27a0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcb038c3b80 1 -- 172.21.15.115:0/2583860656 shutdown_connections 2024-05-09T05:43:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5f426a0 0x55bff5f4cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bff5f5c290 0x55bff5f5e6b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5dc54a0 0x55bff5f4c1d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/2583860656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5eaf160 0x55bff5f4c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.380+0000 7fcb038c3b80 1 -- 172.21.15.115:0/2583860656 >> 172.21.15.115:0/2583860656 conn(0x55bff5e256a0 msgr2=0x55bff5f3d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.381+0000 7fcad5ffb700 0 client.26536 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.381+0000 7fcb038c3b80 1 -- 172.21.15.115:0/2583860656 shutdown_connections 2024-05-09T05:43:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.381+0000 7fcb038c3b80 1 -- 172.21.15.115:0/2583860656 wait complete. 2024-05-09T05:43:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.381+0000 7fcb038c3b80 20 client.26536 trim_cache size 0 max 16384 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.405+0000 7fcb038c3b80 1 Processor -- start 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.405+0000 7fcb038c3b80 1 -- start start 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f57ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5eaf160 0x55bff5f4aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5f426a0 0x55bff5f4b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01b25c0 con 0x55bff5dc54a0 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01b4130 con 0x55bff5eaf160 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01b2c90 con 0x55bff5f426a0 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcaee59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f57ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcaee59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f57ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58868/0 (socket says 172.21.15.115:58868) 2024-05-09T05:43:37.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcaee59c700 1 -- 172.21.15.115:0/1992642975 learned_addr learned my addr 172.21.15.115:0/1992642975 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:37.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcae7fff700 1 --2- 172.21.15.115:0/1992642975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5eaf160 0x55bff5f4aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcaee59c700 1 -- 172.21.15.115:0/1992642975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5f426a0 msgr2=0x55bff5f4b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:37.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcaee59c700 1 --2- 172.21.15.115:0/1992642975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5f426a0 0x55bff5f4b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcaee59c700 1 -- 172.21.15.115:0/1992642975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5eaf160 msgr2=0x55bff5f4aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:37.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcaee59c700 1 --2- 172.21.15.115:0/1992642975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5eaf160 0x55bff5f4aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcaee59c700 1 -- 172.21.15.115:0/1992642975 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff5f3ece0 con 0x55bff5dc54a0 2024-05-09T05:43:37.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.406+0000 7fcae7fff700 1 --2- 172.21.15.115:0/1992642975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5eaf160 0x55bff5f4aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.407+0000 7fcaee59c700 1 --2- 172.21.15.115:0/1992642975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f57ee0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcae00a5b70 tx=0x7fcae00a9390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd39eaa915d604e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.407+0000 7fcad7fff700 1 -- 172.21.15.115:0/1992642975 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae00ca070 con 0x55bff5dc54a0 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.407+0000 7fcad7fff700 1 -- 172.21.15.115:0/1992642975 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcae00a9600 con 0x55bff5dc54a0 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.407+0000 7fcad7fff700 1 -- 172.21.15.115:0/1992642975 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae00aa1f0 con 0x55bff5dc54a0 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 -- 172.21.15.115:0/1992642975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 msgr2=0x55bff5f57ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/1992642975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f57ee0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcae00a5b70 tx=0x7fcae00a9390 comp rx=0 tx=0).stop 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 -- 172.21.15.115:0/1992642975 shutdown_connections 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/1992642975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5f426a0 0x55bff5f4b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/1992642975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5dc54a0 0x55bff5f57ee0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/1992642975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5eaf160 0x55bff5f4aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 -- 172.21.15.115:0/1992642975 >> 172.21.15.115:0/1992642975 conn(0x55bff5e256a0 msgr2=0x55bff5f5e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 -- 172.21.15.115:0/1992642975 shutdown_connections 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 -- 172.21.15.115:0/1992642975 wait complete. 2024-05-09T05:43:37.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 -- start start 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.408+0000 7fcb038c3b80 1 Processor -- start 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5dc54a0 0x55bff5f547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 0x55bff5f4dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcb038c3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5f426a0 0x55bff5f4e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01b3a60 con 0x55bff5f426a0 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01b3360 con 0x55bff5dc54a0 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcb038c3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01b0560 con 0x55bff5eaf160 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcb038c3b80 10 client.0 _ll_register_callbacks cb 0x55bff5cd66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcaeed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5f426a0 0x55bff5f4e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcaeed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5f426a0 0x55bff5f4e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58874/0 (socket says 172.21.15.115:58874) 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcaeed9d700 1 -- 172.21.15.115:0/3507557839 learned_addr learned my addr 172.21.15.115:0/3507557839 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcaee59c700 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5dc54a0 0x55bff5f547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcae7fff700 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 0x55bff5f4dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcaeed9d700 1 -- 172.21.15.115:0/3507557839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 msgr2=0x55bff5f4dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcaeed9d700 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 0x55bff5f4dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcaeed9d700 1 -- 172.21.15.115:0/3507557839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5dc54a0 msgr2=0x55bff5f547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcaeed9d700 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5dc54a0 0x55bff5f547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.409+0000 7fcaeed9d700 1 -- 172.21.15.115:0/3507557839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff5f56ed0 con 0x55bff5f426a0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcaeed9d700 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5f426a0 0x55bff5f4e250 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fcae8015000 tx=0x7fcae800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427e61fb64ad0e5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcad5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcad5ffb700 1 -- 172.21.15.115:0/3507557839 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae802e020 con 0x55bff5f426a0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcad5ffb700 1 -- 172.21.15.115:0/3507557839 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fcae80079d0 con 0x55bff5f426a0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3507557839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae00a9600 con 0x55bff5f426a0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcad5ffb700 1 -- 172.21.15.115:0/3507557839 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcae800be30 con 0x55bff5f426a0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcb038c3b80 10 client.17037 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3507557839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bff5fce200 con 0x55bff5f426a0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcad5ffb700 1 -- 172.21.15.115:0/3507557839 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcae00a9600 con 0x55bff5f426a0 2024-05-09T05:43:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcad5ffb700 1 -- 172.21.15.115:0/3507557839 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fcae8029070 con 0x55bff5f426a0 2024-05-09T05:43:37.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcb038c3b80 10 client.17037 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:37.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcb038c3b80 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:37.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcb038c3b80 20 client.17037 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:37.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bff5fce990 0x55bff5fd0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:37.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcb038c3b80 4 client.17037 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:43:37.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.410+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3507557839 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bff5e6ca70 con 0x55bff5fce990 2024-05-09T05:43:37.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.411+0000 7fcaee59c700 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bff5fce990 0x55bff5fd0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:37.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.411+0000 7fcad5ffb700 10 client.17037 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:37.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.412+0000 7fcaee59c700 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bff5fce990 0x55bff5fd0db0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3627a24e6f83d10a server_cookie=1a092f5e39e5692d in_seq=0 out_seq=0 2024-05-09T05:43:37.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.412+0000 7fcad5ffb700 1 -- 172.21.15.115:0/3507557839 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bff5e6ca70 con 0x55bff5fce990 2024-05-09T05:43:37.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.412+0000 7fcad5ffb700 10 client.17037 handle_command_reply: tid=0 2024-05-09T05:43:37.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.412+0000 7fcb038c3b80 1 client.17037 shutdown 2024-05-09T05:43:37.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3507557839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5f426a0 msgr2=0x55bff5f4e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5f426a0 0x55bff5f4e250 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fcae8015000 tx=0x7fcae800bd10 comp rx=0 tx=0).stop 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3507557839 shutdown_connections 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bff5dc54a0 0x55bff5f547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bff5fce990 0x55bff5fd0db0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bff5f426a0 0x55bff5f4e250 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 --2- 172.21.15.115:0/3507557839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bff5eaf160 0x55bff5f4dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcad5ffb700 0 client.17037 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3507557839 >> 172.21.15.115:0/3507557839 conn(0x55bff5e256a0 msgr2=0x55bff5f471d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3507557839 shutdown_connections 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 1 -- 172.21.15.115:0/3507557839 wait complete. 2024-05-09T05:43:37.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.413+0000 7fcb038c3b80 20 client.17037 trim_cache size 0 max 16384 2024-05-09T05:43:37.411 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:43:37.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 -- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcadc0b2a70 msgr2=0x7fcadc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:37.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcadc0b2a70 0x7fcadc0b4f80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55bff5f490c0 tx=0x7fcae80098c0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 -- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf0105590 msgr2=0x7fcaf0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:37.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf0105590 0x7fcaf0103280 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcae000e9a0 tx=0x7fcae000ecb0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 -- 172.21.15.115:0/2871846658 shutdown_connections 2024-05-09T05:43:37.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcadc0b2a70 0x7fcadc0b4f80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf010e970 0x7fcaf0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf0105590 0x7fcaf0103280 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 --2- 172.21.15.115:0/2871846658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf0105ee0 0x7fcaf01037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:37.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 -- 172.21.15.115:0/2871846658 >> 172.21.15.115:0/2871846658 conn(0x7fcaf0078e00 msgr2=0x7fcaf0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:37.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 -- 172.21.15.115:0/2871846658 shutdown_connections 2024-05-09T05:43:37.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:37.414+0000 7fcaf4ead700 1 -- 172.21.15.115:0/2871846658 wait complete. 2024-05-09T05:43:37.412 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:37.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:37 smithi161 ceph-mon[33805]: pgmap v165: 129 pgs: 129 active+clean; 3.7 GiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-05-09T05:43:39.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:39 smithi115 ceph-mon[30349]: pgmap v166: 129 pgs: 129 active+clean; 3.7 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 22 op/s 2024-05-09T05:43:39.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:39 smithi195 ceph-mon[33844]: pgmap v166: 129 pgs: 129 active+clean; 3.7 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 22 op/s 2024-05-09T05:43:39.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:39 smithi161 ceph-mon[33805]: pgmap v166: 129 pgs: 129 active+clean; 3.7 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 22 op/s 2024-05-09T05:43:40.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 -- 172.21.15.115:0/1939760052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd474107e10 msgr2=0x7fd474108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:40.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/1939760052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd474107e10 0x7fd474108220 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd4640099c0 tx=0x7fd464009cd0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 -- 172.21.15.115:0/1939760052 shutdown_connections 2024-05-09T05:43:40.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/1939760052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4741034e0 0x7fd4741038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/1939760052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474108760 0x7fd4741100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/1939760052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd474107e10 0x7fd474108220 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 -- 172.21.15.115:0/1939760052 >> 172.21.15.115:0/1939760052 conn(0x7fd4740fef50 msgr2=0x7fd474101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:40.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 -- 172.21.15.115:0/1939760052 shutdown_connections 2024-05-09T05:43:40.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 -- 172.21.15.115:0/1939760052 wait complete. 2024-05-09T05:43:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.102+0000 7fd47b5f7700 1 Processor -- start 2024-05-09T05:43:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd47b5f7700 1 -- start start 2024-05-09T05:43:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd47b5f7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4741034e0 0x7fd47419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd47b5f7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd474107e10 0x7fd47419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd47b5f7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474108760 0x7fd47419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd47b5f7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd474113220 con 0x7fd4741034e0 2024-05-09T05:43:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd47b5f7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741130a0 con 0x7fd474107e10 2024-05-09T05:43:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd47b5f7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741133a0 con 0x7fd474108760 2024-05-09T05:43:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd479393700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4741034e0 0x7fd47419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd479393700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4741034e0 0x7fd47419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58886/0 (socket says 172.21.15.115:58886) 2024-05-09T05:43:40.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd479393700 1 -- 172.21.15.115:0/3352874313 learned_addr learned my addr 172.21.15.115:0/3352874313 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:40.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd478b92700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd474107e10 0x7fd47419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd479b94700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474108760 0x7fd47419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd479393700 1 -- 172.21.15.115:0/3352874313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474108760 msgr2=0x7fd47419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:40.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd479393700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474108760 0x7fd47419c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd479393700 1 -- 172.21.15.115:0/3352874313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd474107e10 msgr2=0x7fd47419c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:40.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd479393700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd474107e10 0x7fd47419c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.103+0000 7fd479393700 1 -- 172.21.15.115:0/3352874313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4741a6ec0 con 0x7fd4741034e0 2024-05-09T05:43:40.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.104+0000 7fd479393700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4741034e0 0x7fd47419bbe0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd47000d950 tx=0x7fd47000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:40.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.104+0000 7fd46a7fc700 1 -- 172.21.15.115:0/3352874313 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470016070 con 0x7fd4741034e0 2024-05-09T05:43:40.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.104+0000 7fd46a7fc700 1 -- 172.21.15.115:0/3352874313 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd470009dd0 con 0x7fd4741034e0 2024-05-09T05:43:40.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.104+0000 7fd47b5f7700 1 -- 172.21.15.115:0/3352874313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4741a71b0 con 0x7fd4741034e0 2024-05-09T05:43:40.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.104+0000 7fd46a7fc700 1 -- 172.21.15.115:0/3352874313 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47000f3c0 con 0x7fd4741034e0 2024-05-09T05:43:40.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.104+0000 7fd47b5f7700 1 -- 172.21.15.115:0/3352874313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4741a76c0 con 0x7fd4741034e0 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.107+0000 7fd46a7fc700 1 -- 172.21.15.115:0/3352874313 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fd470019070 con 0x7fd4741034e0 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.108+0000 7fd46a7fc700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4600b29f0 0x7fd4600b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.108+0000 7fd478b92700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4600b29f0 0x7fd4600b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.108+0000 7fd46a7fc700 1 -- 172.21.15.115:0/3352874313 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd4700e2920 con 0x7fd4741034e0 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.108+0000 7fd478b92700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4600b29f0 0x7fd4600b4f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd464009990 tx=0x7fd464005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.108+0000 7fd48a00db80 1 Processor -- start 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.108+0000 7fd48a00db80 1 -- start start 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 0x55e7fc685c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc686190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc692d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd48a00db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e7fc5a91a0 con 0x55e7fc690900 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd48a00db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e7fc54eaa0 con 0x55e7fc583e00 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd48a00db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e7fc5a5160 con 0x55e7fc5095a0 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479b94700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc692d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479b94700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc692d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58890/0 (socket says 172.21.15.115:58890) 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479b94700 1 -- 172.21.15.115:0/3574765930 learned_addr learned my addr 172.21.15.115:0/3574765930 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479393700 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 0x55e7fc685c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd478b92700 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc686190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479b94700 1 -- 172.21.15.115:0/3574765930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 msgr2=0x55e7fc686190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479b94700 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc686190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479b94700 1 -- 172.21.15.115:0/3574765930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 msgr2=0x55e7fc685c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479b94700 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 0x55e7fc685c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479b94700 1 -- 172.21.15.115:0/3574765930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7fc53f600 con 0x55e7fc690900 2024-05-09T05:43:40.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.109+0000 7fd479393700 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 0x55e7fc685c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:40.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.110+0000 7fd479b94700 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc692d20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd46c007f40 tx=0x7fd46c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ee6b2c97c68ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:40.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.111+0000 7fd45f7fe700 1 -- 172.21.15.115:0/3574765930 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00ec70 con 0x55e7fc690900 2024-05-09T05:43:40.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.111+0000 7fd45f7fe700 1 -- 172.21.15.115:0/3574765930 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd46c00ee10 con 0x55e7fc690900 2024-05-09T05:43:40.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.111+0000 7fd45f7fe700 1 -- 172.21.15.115:0/3574765930 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c014c80 con 0x55e7fc690900 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 -- 172.21.15.115:0/3574765930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 msgr2=0x55e7fc692d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc692d20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd46c007f40 tx=0x7fd46c00e440 comp rx=0 tx=0).stop 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 -- 172.21.15.115:0/3574765930 shutdown_connections 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 0x55e7fc685c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc692d20 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 --2- 172.21.15.115:0/3574765930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc686190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 -- 172.21.15.115:0/3574765930 >> 172.21.15.115:0/3574765930 conn(0x55e7fc50d070 msgr2=0x55e7fc681b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 -- 172.21.15.115:0/3574765930 shutdown_connections 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 -- 172.21.15.115:0/3574765930 wait complete. 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.112+0000 7fd48a00db80 1 -- start start 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd48a00db80 1 Processor -- start 2024-05-09T05:43:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc68d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc583e00 0x55e7fc68ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 0x55e7fc68e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd48a00db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e7fc693320 con 0x55e7fc583e00 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd48a00db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e7fc543c20 con 0x55e7fc690900 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd48a00db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e7fc6934a0 con 0x55e7fc5095a0 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd48a00db80 10 client.0 _ll_register_callbacks cb 0x55e7fc41a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd478b92700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc583e00 0x55e7fc68ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd478b92700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc583e00 0x55e7fc68ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58902/0 (socket says 172.21.15.115:58902) 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd478b92700 1 -- 172.21.15.115:0/91615164 learned_addr learned my addr 172.21.15.115:0/91615164 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd479393700 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc68d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.113+0000 7fd479b94700 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 0x55e7fc68e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd478b92700 1 -- 172.21.15.115:0/91615164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 msgr2=0x55e7fc68d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd478b92700 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc68d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd478b92700 1 -- 172.21.15.115:0/91615164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 msgr2=0x55e7fc68e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd478b92700 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 0x55e7fc68e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd478b92700 1 -- 172.21.15.115:0/91615164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7fc424590 con 0x55e7fc583e00 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd478b92700 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc583e00 0x55e7fc68ddd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd474076b40 tx=0x7fd46401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c8cc3e9892c15a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd45cff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd45cff9700 1 -- 172.21.15.115:0/91615164 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46400f460 con 0x55e7fc583e00 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd48a00db80 1 -- 172.21.15.115:0/91615164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd46c00ee10 con 0x55e7fc583e00 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd45cff9700 1 -- 172.21.15.115:0/91615164 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd46400f600 con 0x55e7fc583e00 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd45cff9700 1 -- 172.21.15.115:0/91615164 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd46403d050 con 0x55e7fc583e00 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd48a00db80 10 client.17049 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd48a00db80 1 -- 172.21.15.115:0/91615164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e7fc69df20 con 0x55e7fc583e00 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd45cff9700 1 -- 172.21.15.115:0/91615164 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd46c00ee10 con 0x55e7fc583e00 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.114+0000 7fd45cff9700 1 -- 172.21.15.115:0/91615164 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fd464022070 con 0x55e7fc583e00 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.115+0000 7fd48a00db80 10 client.17049 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.115+0000 7fd48a00db80 10 client.17049 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.115+0000 7fd48a00db80 20 client.17049 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.115+0000 7fd48a00db80 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7fc6a0560 0x55e7fc6a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.115+0000 7fd48a00db80 4 client.17049 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.115+0000 7fd48a00db80 1 -- 172.21.15.115:0/91615164 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7fc5b6b80 con 0x55e7fc6a0560 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.115+0000 7fd479393700 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7fc6a0560 0x55e7fc6a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.115+0000 7fd479393700 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7fc6a0560 0x55e7fc6a29b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2723a4770d771ad8 server_cookie=8e323b53cee37faf in_seq=0 out_seq=0 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.115+0000 7fd45cff9700 10 client.17049 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.116+0000 7fd45cff9700 1 -- 172.21.15.115:0/91615164 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e7fc5b6b80 con 0x55e7fc6a0560 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.116+0000 7fd45cff9700 10 client.17049 handle_command_reply: tid=0 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.116+0000 7fd48a00db80 1 client.17049 shutdown 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.116+0000 7fd48a00db80 1 -- 172.21.15.115:0/91615164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc583e00 msgr2=0x55e7fc68ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.116+0000 7fd48a00db80 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc583e00 0x55e7fc68ddd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd474076b40 tx=0x7fd46401a050 comp rx=0 tx=0).stop 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.116+0000 7fd48a00db80 1 -- 172.21.15.115:0/91615164 shutdown_connections 2024-05-09T05:43:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.116+0000 7fd48a00db80 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 0x55e7fc68e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.116+0000 7fd48a00db80 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7fc6a0560 0x55e7fc6a29b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.116+0000 7fd48a00db80 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc68d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.117+0000 7fd48a00db80 1 --2- 172.21.15.115:0/91615164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc583e00 0x55e7fc68ddd0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.117+0000 7fd48a00db80 1 -- 172.21.15.115:0/91615164 >> 172.21.15.115:0/91615164 conn(0x55e7fc50d070 msgr2=0x55e7fc680de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.117+0000 7fd45cff9700 0 client.17049 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.117+0000 7fd48a00db80 1 -- 172.21.15.115:0/91615164 shutdown_connections 2024-05-09T05:43:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.117+0000 7fd48a00db80 1 -- 172.21.15.115:0/91615164 wait complete. 2024-05-09T05:43:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.117+0000 7fd48a00db80 20 client.17049 trim_cache size 0 max 16384 2024-05-09T05:43:40.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd48a00db80 1 Processor -- start 2024-05-09T05:43:40.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd48a00db80 1 -- start start 2024-05-09T05:43:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc69c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 0x55e7fc687cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc688210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd48a00db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741b0350 con 0x55e7fc690900 2024-05-09T05:43:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd48a00db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741b1ec0 con 0x55e7fc583e00 2024-05-09T05:43:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd48a00db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741b0a20 con 0x55e7fc5095a0 2024-05-09T05:43:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd479b94700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc688210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd479b94700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc688210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58908/0 (socket says 172.21.15.115:58908) 2024-05-09T05:43:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd479b94700 1 -- 172.21.15.115:0/4087453230 learned_addr learned my addr 172.21.15.115:0/4087453230 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd479393700 1 --2- 172.21.15.115:0/4087453230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc69c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd478b92700 1 --2- 172.21.15.115:0/4087453230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 0x55e7fc687cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd479b94700 1 -- 172.21.15.115:0/4087453230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 msgr2=0x55e7fc69c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd479b94700 1 --2- 172.21.15.115:0/4087453230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc69c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd479b94700 1 -- 172.21.15.115:0/4087453230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 msgr2=0x55e7fc687cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd479b94700 1 --2- 172.21.15.115:0/4087453230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 0x55e7fc687cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.137+0000 7fd479b94700 1 -- 172.21.15.115:0/4087453230 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7fc67af20 con 0x55e7fc690900 2024-05-09T05:43:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.138+0000 7fd479b94700 1 --2- 172.21.15.115:0/4087453230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc688210 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd46c007e90 tx=0x7fd46c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd204e1c335cffd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.138+0000 7fd45effd700 1 -- 172.21.15.115:0/4087453230 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c014a90 con 0x55e7fc690900 2024-05-09T05:43:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.138+0000 7fd45effd700 1 -- 172.21.15.115:0/4087453230 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd46c023030 con 0x55e7fc690900 2024-05-09T05:43:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.138+0000 7fd45effd700 1 -- 172.21.15.115:0/4087453230 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c003b70 con 0x55e7fc690900 2024-05-09T05:43:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 -- 172.21.15.115:0/4087453230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 msgr2=0x55e7fc688210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 --2- 172.21.15.115:0/4087453230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc688210 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd46c007e90 tx=0x7fd46c00ee00 comp rx=0 tx=0).stop 2024-05-09T05:43:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 -- 172.21.15.115:0/4087453230 shutdown_connections 2024-05-09T05:43:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 --2- 172.21.15.115:0/4087453230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc690900 0x55e7fc688210 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 --2- 172.21.15.115:0/4087453230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc5095a0 0x55e7fc69c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 --2- 172.21.15.115:0/4087453230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc583e00 0x55e7fc687cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 -- 172.21.15.115:0/4087453230 >> 172.21.15.115:0/4087453230 conn(0x55e7fc50d070 msgr2=0x55e7fc6875c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 -- 172.21.15.115:0/4087453230 shutdown_connections 2024-05-09T05:43:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 -- 172.21.15.115:0/4087453230 wait complete. 2024-05-09T05:43:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 -- start start 2024-05-09T05:43:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.139+0000 7fd48a00db80 1 Processor -- start 2024-05-09T05:43:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc5095a0 0x55e7fc707d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc583e00 0x55e7fc708240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd48a00db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 0x55e7fc704cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd48a00db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741b17f0 con 0x55e7fc5095a0 2024-05-09T05:43:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd48a00db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741b10f0 con 0x55e7fc690900 2024-05-09T05:43:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd48a00db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741ae2f0 con 0x55e7fc583e00 2024-05-09T05:43:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd48a00db80 10 client.0 _ll_register_callbacks cb 0x55e7fc41a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479393700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc5095a0 0x55e7fc707d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479393700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc5095a0 0x55e7fc707d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58918/0 (socket says 172.21.15.115:58918) 2024-05-09T05:43:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479393700 1 -- 172.21.15.115:0/1169226013 learned_addr learned my addr 172.21.15.115:0/1169226013 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479b94700 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 0x55e7fc704cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd478b92700 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc583e00 0x55e7fc708240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479393700 1 -- 172.21.15.115:0/1169226013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc583e00 msgr2=0x55e7fc708240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:40.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479393700 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc583e00 0x55e7fc708240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479393700 1 -- 172.21.15.115:0/1169226013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 msgr2=0x55e7fc704cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:40.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479393700 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 0x55e7fc704cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479393700 1 -- 172.21.15.115:0/1169226013 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7fc708af0 con 0x55e7fc5095a0 2024-05-09T05:43:40.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd478b92700 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc583e00 0x55e7fc708240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:43:40.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd479393700 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc5095a0 0x55e7fc707d00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd4700a6b30 tx=0x7fd470091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92046201f36dbbe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:40.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd45cff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:40.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd45cff9700 1 -- 172.21.15.115:0/1169226013 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470020040 con 0x55e7fc5095a0 2024-05-09T05:43:40.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd45cff9700 1 -- 172.21.15.115:0/1169226013 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd4700dffb0 con 0x55e7fc5095a0 2024-05-09T05:43:40.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.140+0000 7fd48a00db80 1 -- 172.21.15.115:0/1169226013 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd46c023030 con 0x55e7fc5095a0 2024-05-09T05:43:40.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd45cff9700 1 -- 172.21.15.115:0/1169226013 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd4700b2720 con 0x55e7fc5095a0 2024-05-09T05:43:40.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd48a00db80 10 client.17055 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:40.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd48a00db80 1 -- 172.21.15.115:0/1169226013 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e7fc697cf0 con 0x55e7fc5095a0 2024-05-09T05:43:40.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd45cff9700 1 -- 172.21.15.115:0/1169226013 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd46c023030 con 0x55e7fc5095a0 2024-05-09T05:43:40.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd45cff9700 1 -- 172.21.15.115:0/1169226013 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fd4700a9b20 con 0x55e7fc5095a0 2024-05-09T05:43:40.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd48a00db80 10 client.17055 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:40.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd48a00db80 10 client.17055 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:43:40.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd48a00db80 20 client.17055 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:40.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd48a00db80 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7fc698380 0x55e7fc698790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:40.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd48a00db80 4 client.17055 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2c61de64", "release": true, "await": true}] 2024-05-09T05:43:40.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd48a00db80 1 -- 172.21.15.115:0/1169226013 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2c61de64", "release": true, "await": true}) -- 0x55e7fc5b0c20 con 0x55e7fc698380 2024-05-09T05:43:40.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.141+0000 7fd478b92700 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7fc698380 0x55e7fc698790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:40.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.142+0000 7fd45cff9700 10 client.17055 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:40.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.142+0000 7fd478b92700 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7fc698380 0x55e7fc698790 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f2c4b80cd729092 server_cookie=af714f7e32713592 in_seq=0 out_seq=0 2024-05-09T05:43:40.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd45cff9700 1 -- 172.21.15.115:0/1169226013 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e7fc5b0c20 con 0x55e7fc698380 2024-05-09T05:43:40.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd45cff9700 10 client.17055 handle_command_reply: tid=0 2024-05-09T05:43:40.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 client.17055 shutdown 2024-05-09T05:43:40.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 -- 172.21.15.115:0/1169226013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc5095a0 msgr2=0x55e7fc707d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:40.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc5095a0 0x55e7fc707d00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd4700a6b30 tx=0x7fd470091040 comp rx=0 tx=0).stop 2024-05-09T05:43:40.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 -- 172.21.15.115:0/1169226013 shutdown_connections 2024-05-09T05:43:40.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7fc690900 0x55e7fc704cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7fc698380 0x55e7fc698790 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7fc5095a0 0x55e7fc707d00 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 --2- 172.21.15.115:0/1169226013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7fc583e00 0x55e7fc708240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd45cff9700 0 client.17055 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:40.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 -- 172.21.15.115:0/1169226013 >> 172.21.15.115:0/1169226013 conn(0x55e7fc50d070 msgr2=0x55e7fc687ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:40.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 -- 172.21.15.115:0/1169226013 shutdown_connections 2024-05-09T05:43:40.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 1 -- 172.21.15.115:0/1169226013 wait complete. 2024-05-09T05:43:40.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.155+0000 7fd48a00db80 20 client.17055 trim_cache size 0 max 16384 2024-05-09T05:43:40.154 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:43:40.155 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 37, 2024-05-09T05:43:40.155 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:43:40.155 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 16, 2024-05-09T05:43:40.155 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:43:40.155 INFO:teuthology.orchestra.run.smithi115.stdout: "2c61de64": { 2024-05-09T05:43:40.155 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 16, 2024-05-09T05:43:40.155 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:43:40.156 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:43:40.156 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:43:40.156 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:43:40.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:43:40.156 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:43:40.156 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 135.5, 2024-05-09T05:43:40.157 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:43:40.157 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:43:40.157 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:43:40.157 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:43:40.157 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:43:40.157 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:43:40.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:40.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:40.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:40.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:40.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:40.158 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:43:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 -- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4600b29f0 msgr2=0x7fd4600b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4600b29f0 0x7fd4600b4f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd464009990 tx=0x7fd464005fb0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 -- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4741034e0 msgr2=0x7fd47419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4741034e0 0x7fd47419bbe0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd47000d950 tx=0x7fd47000dc60 comp rx=0 tx=0).stop 2024-05-09T05:43:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 -- 172.21.15.115:0/3352874313 shutdown_connections 2024-05-09T05:43:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd4600b29f0 0x7fd4600b4f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd474108760 0x7fd47419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4741034e0 0x7fd47419bbe0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 --2- 172.21.15.115:0/3352874313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd474107e10 0x7fd47419c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 -- 172.21.15.115:0/3352874313 >> 172.21.15.115:0/3352874313 conn(0x7fd4740fef50 msgr2=0x7fd4741096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 -- 172.21.15.115:0/3352874313 shutdown_connections 2024-05-09T05:43:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:40.156+0000 7fd47b5f7700 1 -- 172.21.15.115:0/3352874313 wait complete. 2024-05-09T05:43:40.497 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:43:40.497 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:43:41.021 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2c61de64', total seconds elapsed: 27.5 2024-05-09T05:43:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:41.241 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:41 smithi115 ceph-mon[30349]: pgmap v167: 129 pgs: 129 active+clean; 3.7 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 13 op/s 2024-05-09T05:43:41.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:41 smithi195 ceph-mon[33844]: pgmap v167: 129 pgs: 129 active+clean; 3.7 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 13 op/s 2024-05-09T05:43:41.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:41 smithi161 ceph-mon[33805]: pgmap v167: 129 pgs: 129 active+clean; 3.7 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 13 op/s 2024-05-09T05:43:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.795+0000 7f7912f1f640 1 -- 172.21.15.115:0/4063578900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c103300 msgr2=0x7f790c10f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.795+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4063578900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c103300 0x7f790c10f6d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7904009ca0 tx=0x7f790402a8c0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.795+0000 7f7912f1f640 1 -- 172.21.15.115:0/4063578900 shutdown_connections 2024-05-09T05:43:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.795+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4063578900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c103300 0x7f790c10f6d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.795+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4063578900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790c109050 0x7f790c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.795+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4063578900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c108bb0 0x7f790c102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.795+0000 7f7912f1f640 1 -- 172.21.15.115:0/4063578900 >> 172.21.15.115:0/4063578900 conn(0x7f790c078280 msgr2=0x7f790c0786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.795+0000 7f7912f1f640 1 -- 172.21.15.115:0/4063578900 shutdown_connections 2024-05-09T05:43:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.795+0000 7f7912f1f640 1 -- 172.21.15.115:0/4063578900 wait complete. 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.796+0000 7f7912f1f640 1 Processor -- start 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.796+0000 7f7912f1f640 1 -- start start 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.796+0000 7f7912f1f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c103300 0x7f790c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.796+0000 7f7912f1f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790c108bb0 0x7f790c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.796+0000 7f7912f1f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c109050 0x7f790c19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.796+0000 7f7912f1f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f790c112ca0 con 0x7f790c109050 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.796+0000 7f7912f1f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f790c112b20 con 0x7f790c108bb0 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.796+0000 7f7912f1f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f790c112e20 con 0x7f790c103300 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.796+0000 7f79114bc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c109050 0x7f790c19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f79114bc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c109050 0x7f790c19df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41992/0 (socket says 172.21.15.115:41992) 2024-05-09T05:43:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f79114bc640 1 -- 172.21.15.115:0/4245612590 learned_addr learned my addr 172.21.15.115:0/4245612590 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f790bfff640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790c108bb0 0x7f790c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f79114bc640 1 -- 172.21.15.115:0/4245612590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c103300 msgr2=0x7f790c199700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T05:43:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f79114bc640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c103300 0x7f790c199700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f79114bc640 1 -- 172.21.15.115:0/4245612590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790c108bb0 msgr2=0x7f790c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f79114bc640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790c108bb0 0x7f790c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f79114bc640 1 -- 172.21.15.115:0/4245612590 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c19e4e0 con 0x7f790c109050 2024-05-09T05:43:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f790bfff640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790c108bb0 0x7f790c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:43:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f79114bc640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c109050 0x7f790c19df20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f790402aeb0 tx=0x7f7904032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f7909ffb640 1 -- 172.21.15.115:0/4245612590 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904032e00 con 0x7f790c109050 2024-05-09T05:43:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f7912f1f640 1 -- 172.21.15.115:0/4245612590 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c19e770 con 0x7f790c109050 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f7909ffb640 1 -- 172.21.15.115:0/4245612590 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79040044d0 con 0x7f790c109050 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.797+0000 7f7909ffb640 1 -- 172.21.15.115:0/4245612590 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904030b70 con 0x7f790c109050 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.798+0000 7f7912f1f640 1 -- 172.21.15.115:0/4245612590 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c19ed10 con 0x7f790c109050 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.800+0000 7f7909ffb640 1 -- 172.21.15.115:0/4245612590 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f790404e440 con 0x7f790c109050 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7919dd3740 1 Processor -- start 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7919dd3740 1 -- start start 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e241b0 0x5655361c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535dfb2f0 0x5655361c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e034d0 0x5655361c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7919dd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56553612c7c0 con 0x565535e034d0 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7919dd3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5655360c80e0 con 0x565535e241b0 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7919dd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x565536128780 con 0x565535dfb2f0 2024-05-09T05:43:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f790bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e034d0 0x5655361c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f790bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e034d0 0x5655361c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42000/0 (socket says 172.21.15.115:42000) 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f790bfff640 1 -- 172.21.15.115:0/3584988373 learned_addr learned my addr 172.21.15.115:0/3584988373 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7910cbb640 1 --2- 172.21.15.115:0/3584988373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535dfb2f0 0x5655361c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f790bfff640 1 --2- 172.21.15.115:0/3584988373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e241b0 0x5655361c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.801+0000 7f7909ffb640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f78fc0b2950 0x7f78fc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f790bfff640 1 -- 172.21.15.115:0/3584988373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535dfb2f0 msgr2=0x5655361c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f790bfff640 1 --2- 172.21.15.115:0/3584988373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535dfb2f0 0x5655361c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f790bfff640 1 -- 172.21.15.115:0/3584988373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e241b0 msgr2=0x5655361c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f790bfff640 1 --2- 172.21.15.115:0/3584988373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e241b0 0x5655361c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f790bfff640 1 -- 172.21.15.115:0/3584988373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655360f17c0 con 0x565535e034d0 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f79114bc640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f78fc0b2950 0x7f78fc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f7909ffb640 1 -- 172.21.15.115:0/4245612590 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f7904103df0 con 0x7f790c109050 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f790bfff640 1 --2- 172.21.15.115:0/3584988373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e034d0 0x5655361c8100 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f78f80058c0 tx=0x7f78f800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb459e7b6242492 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f78f6ffd640 1 -- 172.21.15.115:0/3584988373 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8004020 con 0x565535e034d0 2024-05-09T05:43:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f78f6ffd640 1 -- 172.21.15.115:0/3584988373 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78f800fbf0 con 0x565535e034d0 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f79114bc640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f78fc0b2950 0x7f78fc0b4e60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f79040caba0 tx=0x7f79040095c0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.802+0000 7f78f6ffd640 1 -- 172.21.15.115:0/3584988373 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80037f0 con 0x565535e034d0 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.803+0000 7f7919dd3740 1 -- 172.21.15.115:0/3584988373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e034d0 msgr2=0x5655361c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.803+0000 7f7919dd3740 1 --2- 172.21.15.115:0/3584988373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e034d0 0x5655361c8100 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f78f80058c0 tx=0x7f78f800c4b0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.803+0000 7f7919dd3740 1 -- 172.21.15.115:0/3584988373 shutdown_connections 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.803+0000 7f7919dd3740 1 --2- 172.21.15.115:0/3584988373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e034d0 0x5655361c8100 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.803+0000 7f7919dd3740 1 --2- 172.21.15.115:0/3584988373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535dfb2f0 0x5655361c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.803+0000 7f7919dd3740 1 --2- 172.21.15.115:0/3584988373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e241b0 0x5655361c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.803+0000 7f7919dd3740 1 -- 172.21.15.115:0/3584988373 >> 172.21.15.115:0/3584988373 conn(0x565535e18300 msgr2=0x5655361c3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.803+0000 7f7919dd3740 1 -- 172.21.15.115:0/3584988373 shutdown_connections 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.803+0000 7f7919dd3740 1 -- 172.21.15.115:0/3584988373 wait complete. 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.804+0000 7f7919dd3740 1 -- start start 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.804+0000 7f7919dd3740 1 Processor -- start 2024-05-09T05:43:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.804+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.804+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.804+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e241b0 0x5655361d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.804+0000 7f7919dd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5655361d47b0 con 0x565535dfb2f0 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.804+0000 7f7919dd3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5655360b39c0 con 0x565535e034d0 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.804+0000 7f7919dd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5655361d4930 con 0x565535e241b0 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f790bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f7919dd3740 10 client.0 _ll_register_callbacks cb 0x5655360e0020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f7910cbb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f7910cbb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:42900/0 (socket says 172.21.15.115:42900) 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f7910cbb640 1 -- 172.21.15.115:0/947800164 learned_addr learned my addr 172.21.15.115:0/947800164 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f790bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361d3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42012/0 (socket says 172.21.15.115:42012) 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f790bfff640 1 -- 172.21.15.115:0/947800164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e241b0 msgr2=0x5655361d1bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f790bfff640 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e241b0 0x5655361d1bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f790bfff640 1 -- 172.21.15.115:0/947800164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 msgr2=0x5655361d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f790bfff640 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361d3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f790bfff640 1 -- 172.21.15.115:0/947800164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655360da1f0 con 0x565535dfb2f0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f7910cbb640 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f790bfff640 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361d3770 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f78f8004310 tx=0x7f78f8013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b313259e932401 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f78ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f78ebfff640 1 -- 172.21.15.115:0/947800164 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8005250 con 0x565535dfb2f0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f78ebfff640 1 -- 172.21.15.115:0/947800164 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f78f80053f0 con 0x565535dfb2f0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.805+0000 7f7919dd3740 1 -- 172.21.15.115:0/947800164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f800fbf0 con 0x565535dfb2f0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f78ebfff640 1 -- 172.21.15.115:0/947800164 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f78f8032050 con 0x565535dfb2f0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f7919dd3740 10 client.17067 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f7919dd3740 1 -- 172.21.15.115:0/947800164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565535e8f790 con 0x565535dfb2f0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f78ebfff640 1 -- 172.21.15.115:0/947800164 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78f800fbf0 con 0x565535dfb2f0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f78ebfff640 1 -- 172.21.15.115:0/947800164 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f78f8003680 con 0x565535dfb2f0 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f7919dd3740 10 client.17067 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f7919dd3740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f7919dd3740 20 client.17067 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f7919dd3740 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x565535de2d80 0x5655361e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f7919dd3740 4 client.17067 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.806+0000 7f7919dd3740 1 -- 172.21.15.115:0/947800164 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56553613a3d0 con 0x565535de2d80 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.807+0000 7f7910cbb640 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x565535de2d80 0x5655361e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.808+0000 7f7910cbb640 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x565535de2d80 0x5655361e40e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69de765a70e03a80 server_cookie=32f679bc5b8b5d90 in_seq=0 out_seq=0 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.808+0000 7f78ebfff640 10 client.17067 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f78ebfff640 1 -- 172.21.15.115:0/947800164 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56553613a3d0 con 0x565535de2d80 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f78ebfff640 10 client.17067 handle_command_reply: tid=0 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 client.17067 shutdown 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 -- 172.21.15.115:0/947800164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 msgr2=0x5655361d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361d3770 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f78f8004310 tx=0x7f78f8013dd0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 -- 172.21.15.115:0/947800164 shutdown_connections 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x565535de2d80 0x5655361e40e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e241b0 0x5655361d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 --2- 172.21.15.115:0/947800164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361d3770 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f78ebfff640 0 client.17067 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 -- 172.21.15.115:0/947800164 >> 172.21.15.115:0/947800164 conn(0x565535e18300 msgr2=0x5655361c93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 -- 172.21.15.115:0/947800164 shutdown_connections 2024-05-09T05:43:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.809+0000 7f7919dd3740 1 -- 172.21.15.115:0/947800164 wait complete. 2024-05-09T05:43:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.810+0000 7f7919dd3740 20 client.17067 trim_cache size 0 max 16384 2024-05-09T05:43:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.826+0000 7f7919dd3740 1 Processor -- start 2024-05-09T05:43:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.826+0000 7f7919dd3740 1 -- start start 2024-05-09T05:43:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.826+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535dfb2f0 0x5655361d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.826+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e034d0 0x5655361c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.826+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e241b0 0x5655361c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.826+0000 7f7919dd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1aee00 con 0x565535e241b0 2024-05-09T05:43:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.826+0000 7f7919dd3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1b0970 con 0x565535dfb2f0 2024-05-09T05:43:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.826+0000 7f7919dd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1af4d0 con 0x565535e034d0 2024-05-09T05:43:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e241b0 0x5655361c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e241b0 0x5655361c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42018/0 (socket says 172.21.15.115:42018) 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 -- 172.21.15.115:0/3735917421 learned_addr learned my addr 172.21.15.115:0/3735917421 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f7910cbb640 1 --2- 172.21.15.115:0/3735917421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e034d0 0x5655361c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 --2- 172.21.15.115:0/3735917421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535dfb2f0 0x5655361d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 -- 172.21.15.115:0/3735917421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e034d0 msgr2=0x5655361c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 --2- 172.21.15.115:0/3735917421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e034d0 0x5655361c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 -- 172.21.15.115:0/3735917421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535dfb2f0 msgr2=0x5655361d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 --2- 172.21.15.115:0/3735917421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535dfb2f0 0x5655361d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 -- 172.21.15.115:0/3735917421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565535f065f0 con 0x565535e241b0 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f790bfff640 1 --2- 172.21.15.115:0/3735917421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e241b0 0x5655361c3710 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f78f80059f0 tx=0x7f78f8013e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1caba50a1f4d6ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f78f67fc640 1 -- 172.21.15.115:0/3735917421 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8003bf0 con 0x565535e241b0 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.827+0000 7f78f67fc640 1 -- 172.21.15.115:0/3735917421 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78f8017040 con 0x565535e241b0 2024-05-09T05:43:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.828+0000 7f78f67fc640 1 -- 172.21.15.115:0/3735917421 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8004660 con 0x565535e241b0 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.828+0000 7f7919dd3740 1 -- 172.21.15.115:0/3735917421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e241b0 msgr2=0x5655361c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.828+0000 7f7919dd3740 1 --2- 172.21.15.115:0/3735917421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e241b0 0x5655361c3710 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f78f80059f0 tx=0x7f78f8013e40 comp rx=0 tx=0).stop 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.829+0000 7f7919dd3740 1 -- 172.21.15.115:0/3735917421 shutdown_connections 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.829+0000 7f7919dd3740 1 --2- 172.21.15.115:0/3735917421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535e241b0 0x5655361c3710 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.829+0000 7f7919dd3740 1 --2- 172.21.15.115:0/3735917421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e034d0 0x5655361c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.829+0000 7f7919dd3740 1 --2- 172.21.15.115:0/3735917421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535dfb2f0 0x5655361d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.829+0000 7f7919dd3740 1 -- 172.21.15.115:0/3735917421 >> 172.21.15.115:0/3735917421 conn(0x565535e18300 msgr2=0x5655361c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.829+0000 7f7919dd3740 1 -- 172.21.15.115:0/3735917421 shutdown_connections 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.829+0000 7f7919dd3740 1 -- 172.21.15.115:0/3735917421 wait complete. 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.829+0000 7f7919dd3740 1 -- start start 2024-05-09T05:43:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.829+0000 7f7919dd3740 1 Processor -- start 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361e3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f7919dd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e241b0 0x5655361e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f7919dd3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1b02a0 con 0x565535dfb2f0 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f7919dd3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1afba0 con 0x565535e034d0 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f7919dd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1acde0 con 0x565535e241b0 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f7919dd3740 10 client.0 _ll_register_callbacks cb 0x7f78f8013ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361e7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42024/0 (socket says 172.21.15.115:42024) 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 -- 172.21.15.115:0/1119867041 learned_addr learned my addr 172.21.15.115:0/1119867041 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f7910cbb640 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e241b0 0x5655361e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 -- 172.21.15.115:0/1119867041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e241b0 msgr2=0x5655361e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e241b0 0x5655361e3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 -- 172.21.15.115:0/1119867041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 msgr2=0x5655361e3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361e3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 -- 172.21.15.115:0/1119867041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655360f0ed0 con 0x565535dfb2f0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f790bfff640 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361e7430 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f78f8003a00 tx=0x7f78f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec4c410d5393761 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f78ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f78ebfff640 1 -- 172.21.15.115:0/1119867041 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8005220 con 0x565535dfb2f0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f78ebfff640 1 -- 172.21.15.115:0/1119867041 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f78f80053c0 con 0x565535dfb2f0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.830+0000 7f7919dd3740 1 -- 172.21.15.115:0/1119867041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f8017040 con 0x565535dfb2f0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f78ebfff640 1 -- 172.21.15.115:0/1119867041 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f78f80055c0 con 0x565535dfb2f0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f7919dd3740 10 client.17073 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f7919dd3740 1 -- 172.21.15.115:0/1119867041 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56553624adb0 con 0x565535dfb2f0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f78ebfff640 1 -- 172.21.15.115:0/1119867041 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78f8017040 con 0x565535dfb2f0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f78ebfff640 1 -- 172.21.15.115:0/1119867041 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f78f8035040 con 0x565535dfb2f0 2024-05-09T05:43:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f7919dd3740 10 client.17073 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f7919dd3740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f7919dd3740 20 client.17073 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f7919dd3740 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x565535db81c0 0x56553624d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f7919dd3740 4 client.17073 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:43:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.831+0000 7f7919dd3740 1 -- 172.21.15.115:0/1119867041 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5655361cd7d0 con 0x565535db81c0 2024-05-09T05:43:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.832+0000 7f7910cbb640 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x565535db81c0 0x56553624d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.832+0000 7f78ebfff640 10 client.17073 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.832+0000 7f7910cbb640 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x565535db81c0 0x56553624d3d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d25a84852007011 server_cookie=109c84d1e5362d65 in_seq=0 out_seq=0 2024-05-09T05:43:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f78ebfff640 1 -- 172.21.15.115:0/1119867041 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5655361cd7d0 con 0x565535db81c0 2024-05-09T05:43:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f78ebfff640 10 client.17073 handle_command_reply: tid=0 2024-05-09T05:43:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 client.17073 shutdown 2024-05-09T05:43:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 -- 172.21.15.115:0/1119867041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 msgr2=0x5655361e7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361e7430 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f78f8003a00 tx=0x7f78f800c860 comp rx=0 tx=0).stop 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 -- 172.21.15.115:0/1119867041 shutdown_connections 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x565535db81c0 0x56553624d3d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565535e241b0 0x5655361e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565535e034d0 0x5655361e3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 --2- 172.21.15.115:0/1119867041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x565535dfb2f0 0x5655361e7430 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 -- 172.21.15.115:0/1119867041 >> 172.21.15.115:0/1119867041 conn(0x565535e18300 msgr2=0x5655361d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f78ebfff640 0 client.17073 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 -- 172.21.15.115:0/1119867041 shutdown_connections 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 1 -- 172.21.15.115:0/1119867041 wait complete. 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.840+0000 7f7919dd3740 20 client.17073 trim_cache size 0 max 16384 2024-05-09T05:43:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:43:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:43:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 -- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f78fc0b2950 msgr2=0x7f78fc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f78fc0b2950 0x7f78fc0b4e60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f79040caba0 tx=0x7f79040095c0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 -- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c109050 msgr2=0x7f790c19df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c109050 0x7f790c19df20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f790402aeb0 tx=0x7f7904032a10 comp rx=0 tx=0).stop 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 -- 172.21.15.115:0/4245612590 shutdown_connections 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f78fc0b2950 0x7f78fc0b4e60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f790c109050 0x7f790c19df20 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f790c108bb0 0x7f790c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 --2- 172.21.15.115:0/4245612590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c103300 0x7f790c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 -- 172.21.15.115:0/4245612590 >> 172.21.15.115:0/4245612590 conn(0x7f790c078280 msgr2=0x7f790c0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 -- 172.21.15.115:0/4245612590 shutdown_connections 2024-05-09T05:43:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:42.841+0000 7f7912f1f640 1 -- 172.21.15.115:0/4245612590 wait complete. 2024-05-09T05:43:43.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.007+0000 7fc4aed00700 1 -- 172.21.15.115:0/2416038692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a81045e0 msgr2=0x7fc4a81049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:43.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.007+0000 7fc4aed00700 1 --2- 172.21.15.115:0/2416038692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a81045e0 0x7fc4a81049f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0099c0 tx=0x7fc49c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.007+0000 7fc4aed00700 1 -- 172.21.15.115:0/2416038692 shutdown_connections 2024-05-09T05:43:43.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.007+0000 7fc4aed00700 1 --2- 172.21.15.115:0/2416038692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8109860 0x7fc4a81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.007+0000 7fc4aed00700 1 --2- 172.21.15.115:0/2416038692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a81045e0 0x7fc4a81049f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.007+0000 7fc4aed00700 1 --2- 172.21.15.115:0/2416038692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a8108f10 0x7fc4a8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.007+0000 7fc4aed00700 1 -- 172.21.15.115:0/2416038692 >> 172.21.15.115:0/2416038692 conn(0x7fc4a80ffef0 msgr2=0x7fc4a8102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:43.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.008+0000 7fc4aed00700 1 -- 172.21.15.115:0/2416038692 shutdown_connections 2024-05-09T05:43:43.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.008+0000 7fc4aed00700 1 -- 172.21.15.115:0/2416038692 wait complete. 2024-05-09T05:43:43.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.008+0000 7fc4aed00700 1 Processor -- start 2024-05-09T05:43:43.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.008+0000 7fc4aed00700 1 -- start start 2024-05-09T05:43:43.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aed00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a81045e0 0x7fc4a819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aed00700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a8108f10 0x7fc4a819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aed00700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8109860 0x7fc4a819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aed00700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8114340 con 0x7fc4a8108f10 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aed00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81141c0 con 0x7fc4a81045e0 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aed00700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81144c0 con 0x7fc4a8109860 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aca9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a81045e0 0x7fc4a819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a8108f10 0x7fc4a819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aca9c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a81045e0 0x7fc4a819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:42928/0 (socket says 172.21.15.115:42928) 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aca9c700 1 -- 172.21.15.115:0/1558755603 learned_addr learned my addr 172.21.15.115:0/1558755603 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4ad29d700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8109860 0x7fc4a819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4a7fff700 1 -- 172.21.15.115:0/1558755603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8109860 msgr2=0x7fc4a819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8109860 0x7fc4a819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4a7fff700 1 -- 172.21.15.115:0/1558755603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a81045e0 msgr2=0x7fc4a819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a81045e0 0x7fc4a819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4a7fff700 1 -- 172.21.15.115:0/1558755603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a819fc20 con 0x7fc4a8108f10 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4ad29d700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8109860 0x7fc4a819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:43.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.009+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a81045e0 0x7fc4a819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:43.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.010+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a8108f10 0x7fc4a819b2f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc4a806c1b0 tx=0x7fc49400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:43.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.010+0000 7fc4a5ffb700 1 -- 172.21.15.115:0/1558755603 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494016070 con 0x7fc4a8108f10 2024-05-09T05:43:43.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.010+0000 7fc4a5ffb700 1 -- 172.21.15.115:0/1558755603 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc49400fd20 con 0x7fc4a8108f10 2024-05-09T05:43:43.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.010+0000 7fc4a5ffb700 1 -- 172.21.15.115:0/1558755603 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49400e3b0 con 0x7fc4a8108f10 2024-05-09T05:43:43.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.010+0000 7fc4aed00700 1 -- 172.21.15.115:0/1558755603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a819ff10 con 0x7fc4a8108f10 2024-05-09T05:43:43.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.011+0000 7fc4aed00700 1 -- 172.21.15.115:0/1558755603 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a81a7830 con 0x7fc4a8108f10 2024-05-09T05:43:43.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.013+0000 7fc4a5ffb700 1 -- 172.21.15.115:0/1558755603 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc494019070 con 0x7fc4a8108f10 2024-05-09T05:43:43.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.014+0000 7fc4a5ffb700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc4980b2a70 0x7fc4980b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.014+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc4980b2a70 0x7fc4980b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.014+0000 7fc4a5ffb700 1 -- 172.21.15.115:0/1558755603 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc4940e2280 con 0x7fc4a8108f10 2024-05-09T05:43:43.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.014+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc4980b2a70 0x7fc4980b4f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009990 tx=0x7fc49c01a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:43.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4bd716b80 1 Processor -- start 2024-05-09T05:43:43.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4bd716b80 1 -- start start 2024-05-09T05:43:43.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6461e0 0x56529c6dddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c55c520 0x56529c6de370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6e2480 0x56529c6e2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56529c5fc070 con 0x56529c6e2480 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56529c59a870 con 0x56529c55c520 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56529c5f8030 con 0x56529c6461e0 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4ad29d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6e2480 0x56529c6e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c55c520 0x56529c6de370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c55c520 0x56529c6de370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:42938/0 (socket says 172.21.15.115:42938) 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4a7fff700 1 -- 172.21.15.115:0/798447096 learned_addr learned my addr 172.21.15.115:0/798447096 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4aca9c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6461e0 0x56529c6dddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.015+0000 7fc4ad29d700 1 -- 172.21.15.115:0/798447096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6461e0 msgr2=0x56529c6dddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc4ad29d700 1 --2- 172.21.15.115:0/798447096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6461e0 0x56529c6dddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc4ad29d700 1 -- 172.21.15.115:0/798447096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c55c520 msgr2=0x56529c6de370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:43.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc4ad29d700 1 --2- 172.21.15.115:0/798447096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c55c520 0x56529c6de370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc4ad29d700 1 -- 172.21.15.115:0/798447096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56529c5923f0 con 0x56529c6e2480 2024-05-09T05:43:43.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/798447096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6461e0 0x56529c6dddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:43.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/798447096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c55c520 0x56529c6de370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:43.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc4ad29d700 1 --2- 172.21.15.115:0/798447096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6e2480 0x56529c6e2890 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0003db0 tx=0x7fc4a0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d282443d82996ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:43.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc492ffd700 1 -- 172.21.15.115:0/798447096 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0019030 con 0x56529c6e2480 2024-05-09T05:43:43.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc492ffd700 1 -- 172.21.15.115:0/798447096 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4a0004bb0 con 0x56529c6e2480 2024-05-09T05:43:43.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.016+0000 7fc492ffd700 1 -- 172.21.15.115:0/798447096 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0007c80 con 0x56529c6e2480 2024-05-09T05:43:43.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.017+0000 7fc4bd716b80 1 -- 172.21.15.115:0/798447096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6e2480 msgr2=0x56529c6e2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:43.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.017+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/798447096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6e2480 0x56529c6e2890 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0003db0 tx=0x7fc4a0009920 comp rx=0 tx=0).stop 2024-05-09T05:43:43.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.017+0000 7fc4bd716b80 1 -- 172.21.15.115:0/798447096 shutdown_connections 2024-05-09T05:43:43.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.017+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/798447096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6e2480 0x56529c6e2890 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.017+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/798447096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6461e0 0x56529c6dddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.017+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/798447096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c55c520 0x56529c6de370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.017+0000 7fc4bd716b80 1 -- 172.21.15.115:0/798447096 >> 172.21.15.115:0/798447096 conn(0x56529c5bc720 msgr2=0x56529c6d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.017+0000 7fc4bd716b80 1 -- 172.21.15.115:0/798447096 shutdown_connections 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.017+0000 7fc4bd716b80 1 -- 172.21.15.115:0/798447096 wait complete. 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.018+0000 7fc4bd716b80 1 -- start start 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.018+0000 7fc4bd716b80 1 Processor -- start 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.018+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 0x56529c6e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56529c477c20 con 0x56529c6461e0 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56529c5b61d0 con 0x56529c6e2480 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56529c6e3230 con 0x56529c55c520 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4bd716b80 10 client.0 _ll_register_callbacks cb 0x56529c46d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4aca9c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4aca9c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:35712/0 (socket says 172.21.15.115:35712) 2024-05-09T05:43:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4aca9c700 1 -- 172.21.15.115:0/3688427689 learned_addr learned my addr 172.21.15.115:0/3688427689 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4ad29d700 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 0x56529c6e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4aca9c700 1 -- 172.21.15.115:0/3688427689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 msgr2=0x56529c6e8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 0x56529c6e8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4aca9c700 1 -- 172.21.15.115:0/3688427689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 msgr2=0x56529c6e7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4aca9c700 1 -- 172.21.15.115:0/3688427689 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56529c477510 con 0x56529c55c520 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.019+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e76a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc49c000c00 tx=0x7fc49c009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b13d7cb5b299aff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc487fff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc487fff700 1 -- 172.21.15.115:0/3688427689 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c035060 con 0x56529c55c520 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc4bd716b80 1 -- 172.21.15.115:0/3688427689 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a0004bb0 con 0x56529c55c520 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc487fff700 1 -- 172.21.15.115:0/3688427689 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc49c00f660 con 0x56529c55c520 2024-05-09T05:43:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc487fff700 1 -- 172.21.15.115:0/3688427689 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc49c00f860 con 0x56529c55c520 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc4bd716b80 10 client.26572 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc4bd716b80 1 -- 172.21.15.115:0/3688427689 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56529c6f5010 con 0x56529c55c520 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc487fff700 1 -- 172.21.15.115:0/3688427689 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4a0004bb0 con 0x56529c55c520 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc487fff700 1 -- 172.21.15.115:0/3688427689 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fc49c0269b0 con 0x56529c55c520 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc4bd716b80 10 client.26572 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc4bd716b80 10 client.26572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.020+0000 7fc4bd716b80 20 client.26572 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.021+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56529c6f75a0 0x56529c6f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.021+0000 7fc4bd716b80 4 client.26572 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.021+0000 7fc4bd716b80 1 -- 172.21.15.115:0/3688427689 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56529c609a50 con 0x56529c6f75a0 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.021+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56529c6f75a0 0x56529c6f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.021+0000 7fc487fff700 10 client.26572 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.021+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56529c6f75a0 0x56529c6f99c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5eb0cd51d86207c server_cookie=2cf0be3faef22e40 in_seq=0 out_seq=0 2024-05-09T05:43:43.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc487fff700 1 -- 172.21.15.115:0/3688427689 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56529c609a50 con 0x56529c6f75a0 2024-05-09T05:43:43.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc487fff700 10 client.26572 handle_command_reply: tid=0 2024-05-09T05:43:43.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 client.26572 shutdown 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 -- 172.21.15.115:0/3688427689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 msgr2=0x56529c6e76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e76a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc49c000c00 tx=0x7fc49c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 -- 172.21.15.115:0/3688427689 shutdown_connections 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 0x56529c6e8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56529c6f75a0 0x56529c6f99c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e76a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc487fff700 0 client.26572 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/3688427689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 -- 172.21.15.115:0/3688427689 >> 172.21.15.115:0/3688427689 conn(0x56529c5bc720 msgr2=0x56529c6d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 -- 172.21.15.115:0/3688427689 shutdown_connections 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 1 -- 172.21.15.115:0/3688427689 wait complete. 2024-05-09T05:43:43.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.022+0000 7fc4bd716b80 20 client.26572 trim_cache size 0 max 16384 2024-05-09T05:43:43.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.046+0000 7fc4bd716b80 1 Processor -- start 2024-05-09T05:43:43.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.046+0000 7fc4bd716b80 1 -- start start 2024-05-09T05:43:43.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.046+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.046+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.046+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 0x56529c6f3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.046+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ae200 con 0x56529c6461e0 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.046+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81afd70 con 0x56529c6e2480 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.046+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ae8d0 con 0x56529c55c520 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42080/0 (socket says 172.21.15.115:42080) 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4a7fff700 1 -- 172.21.15.115:0/2208189308 learned_addr learned my addr 172.21.15.115:0/2208189308 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4ad29d700 1 --2- 172.21.15.115:0/2208189308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 0x56529c6f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/2208189308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4a7fff700 1 -- 172.21.15.115:0/2208189308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 msgr2=0x56529c6e1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/2208189308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4a7fff700 1 -- 172.21.15.115:0/2208189308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 msgr2=0x56529c6f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/2208189308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 0x56529c6f3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4a7fff700 1 -- 172.21.15.115:0/2208189308 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56529c6de7e0 con 0x56529c6461e0 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/2208189308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7f60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc4940df740 tx=0x7fc494000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee51e270ca8fd34 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4927fc700 1 -- 172.21.15.115:0/2208189308 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494020040 con 0x56529c6461e0 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4927fc700 1 -- 172.21.15.115:0/2208189308 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4940a9220 con 0x56529c6461e0 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.047+0000 7fc4927fc700 1 -- 172.21.15.115:0/2208189308 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940ad160 con 0x56529c6461e0 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.048+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2208189308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 msgr2=0x56529c6e7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.048+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2208189308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7f60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc4940df740 tx=0x7fc494000bc0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.049+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2208189308 shutdown_connections 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.049+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2208189308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6e2480 0x56529c6f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.049+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2208189308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c55c520 0x56529c6e1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.049+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2208189308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c6461e0 0x56529c6e7f60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.049+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2208189308 >> 172.21.15.115:0/2208189308 conn(0x56529c5bc720 msgr2=0x56529c6e8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:43.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.049+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2208189308 shutdown_connections 2024-05-09T05:43:43.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.049+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2208189308 wait complete. 2024-05-09T05:43:43.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.049+0000 7fc4bd716b80 1 -- start start 2024-05-09T05:43:43.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.049+0000 7fc4bd716b80 1 Processor -- start 2024-05-09T05:43:43.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c55c520 0x56529c6e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6461e0 0x56529c6ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4bd716b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6e2480 0x56529c6efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81af6a0 con 0x56529c55c520 2024-05-09T05:43:43.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81aefa0 con 0x56529c6461e0 2024-05-09T05:43:43.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4bd716b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ac1a0 con 0x56529c6e2480 2024-05-09T05:43:43.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4bd716b80 10 client.0 _ll_register_callbacks cb 0x56529c46d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4aca9c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c55c520 0x56529c6e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6461e0 0x56529c6ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6461e0 0x56529c6ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:42962/0 (socket says 172.21.15.115:42962) 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4aca9c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c55c520 0x56529c6e0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42096/0 (socket says 172.21.15.115:42096) 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4a7fff700 1 -- 172.21.15.115:0/2823385025 learned_addr learned my addr 172.21.15.115:0/2823385025 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4ad29d700 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6e2480 0x56529c6efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4a7fff700 1 -- 172.21.15.115:0/2823385025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6e2480 msgr2=0x56529c6efdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6e2480 0x56529c6efdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4a7fff700 1 -- 172.21.15.115:0/2823385025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c55c520 msgr2=0x56529c6e0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c55c520 0x56529c6e0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4a7fff700 1 -- 172.21.15.115:0/2823385025 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56529c6febc0 con 0x56529c6461e0 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c55c520 0x56529c6e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc4a7fff700 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6461e0 0x56529c6ef800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc4940a52a0 tx=0x7fc49400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88dd1ca33594080c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.050+0000 7fc487fff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc487fff700 1 -- 172.21.15.115:0/2823385025 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494020070 con 0x56529c6461e0 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc487fff700 1 -- 172.21.15.115:0/2823385025 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fc4940a5810 con 0x56529c6461e0 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2823385025 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4940a9220 con 0x56529c6461e0 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc487fff700 1 -- 172.21.15.115:0/2823385025 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc4940b2080 con 0x56529c6461e0 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc4bd716b80 10 client.26498 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2823385025 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56529c7620f0 con 0x56529c6461e0 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc487fff700 1 -- 172.21.15.115:0/2823385025 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4940a9220 con 0x56529c6461e0 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc487fff700 1 -- 172.21.15.115:0/2823385025 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fc4940c5bd0 con 0x56529c6461e0 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc4bd716b80 10 client.26498 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc4bd716b80 10 client.26498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc4bd716b80 20 client.26498 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:43.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56529c6f75a0 0x56529c7646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:43.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc4bd716b80 4 client.26498 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:43:43.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.051+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2823385025 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56529c603af0 con 0x56529c6f75a0 2024-05-09T05:43:43.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.052+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56529c6f75a0 0x56529c7646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:43.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.052+0000 7fc487fff700 10 client.26498 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:43.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.052+0000 7fc4aca9c700 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56529c6f75a0 0x56529c7646e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2108c127078b8b0 server_cookie=e9f52635936e2e15 in_seq=0 out_seq=0 2024-05-09T05:43:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.053+0000 7fc487fff700 1 -- 172.21.15.115:0/2823385025 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56529c603af0 con 0x56529c6f75a0 2024-05-09T05:43:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.053+0000 7fc487fff700 10 client.26498 handle_command_reply: tid=0 2024-05-09T05:43:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.053+0000 7fc4bd716b80 1 client.26498 shutdown 2024-05-09T05:43:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2823385025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6461e0 msgr2=0x56529c6ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6461e0 0x56529c6ef800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc4940a52a0 tx=0x7fc49400bf90 comp rx=0 tx=0).stop 2024-05-09T05:43:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2823385025 shutdown_connections 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56529c6f75a0 0x56529c7646e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56529c6e2480 0x56529c6efdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56529c55c520 0x56529c6e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 --2- 172.21.15.115:0/2823385025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56529c6461e0 0x56529c6ef800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2823385025 >> 172.21.15.115:0/2823385025 conn(0x56529c5bc720 msgr2=0x56529c6d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc487fff700 0 client.26498 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2823385025 shutdown_connections 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 1 -- 172.21.15.115:0/2823385025 wait complete. 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.054+0000 7fc4bd716b80 20 client.26498 trim_cache size 0 max 16384 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:43:43.051 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:43:43.052 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "0eab15dd-3683-4dec-bf35-6c7317623d7a", 2024-05-09T05:43:43.052 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:43:43.052 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:43:43.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 -- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc4980b2a70 msgr2=0x7fc4980b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:43.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc4980b2a70 0x7fc4980b4f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009990 tx=0x7fc49c01a040 comp rx=0 tx=0).stop 2024-05-09T05:43:43.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 -- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a8108f10 msgr2=0x7fc4a819b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a8108f10 0x7fc4a819b2f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc4a806c1b0 tx=0x7fc49400dc30 comp rx=0 tx=0).stop 2024-05-09T05:43:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 -- 172.21.15.115:0/1558755603 shutdown_connections 2024-05-09T05:43:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc4980b2a70 0x7fc4980b4f80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4a81045e0 0x7fc4a819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4a8109860 0x7fc4a819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 --2- 172.21.15.115:0/1558755603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4a8108f10 0x7fc4a819b2f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 -- 172.21.15.115:0/1558755603 >> 172.21.15.115:0/1558755603 conn(0x7fc4a80ffef0 msgr2=0x7fc4a8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 -- 172.21.15.115:0/1558755603 shutdown_connections 2024-05-09T05:43:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:43.055+0000 7fc4aed00700 1 -- 172.21.15.115:0/1558755603 wait complete. 2024-05-09T05:43:43.145 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:43 smithi115 ceph-mon[30349]: pgmap v168: 129 pgs: 129 active+clean; 3.7 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 13 op/s 2024-05-09T05:43:43.365 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:43:43 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T05:43:43.066+0000 7f9d69520700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c7 [...5,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ auth{1=1} v3945 ap=2 f(v36 m2024-05-09T05:40:55.664108+0000 18=18+0) n(v1 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1) old_inodes=1 (inest sync->mix g=1) caps={15339=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55d6cd1f4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c7:[<0x10000000007/.deps v2123>,<0x10000000006/ffsb v3080>,<0x10000000005/tmp v44>,<0x10000000003/client.0 v40>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v56>,<0x10000000001/sv_0 v71>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c7:[<0x10000000007/.deps v3945>,<0x10000000006/ffsb v3523>,<0x10000000005/tmp v111>,<0x10000000003/client.0 v104>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v124>,<0x10000000001/sv_0 v142>,<0x10000000000/qa v132>,<0x1/volumes v123>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T05:40:55.643149+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1)","memory_value.dirstat":"f(v36 m2024-05-09T05:40:55.664108+0000 18=18+0)","memory_value.rstat":"n(v1 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T05:43:43.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:43 smithi195 ceph-mon[33844]: pgmap v168: 129 pgs: 129 active+clean; 3.7 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 13 op/s 2024-05-09T05:43:43.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:43 smithi161 ceph-mon[33805]: pgmap v168: 129 pgs: 129 active+clean; 3.7 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 13 op/s 2024-05-09T05:43:43.915 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.146 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:43:44.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:43:44.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:43:44.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:43:44.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.147 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:44 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.233 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:43:44.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:43:44.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:43:44.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:43:44.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:43:44.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.366 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:44 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:43:44.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:43:44.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:43:44.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:43:44.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:44.411 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:44 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:43:45.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:45 smithi195 ceph-mon[33844]: pgmap v169: 129 pgs: 129 active+clean; 3.7 GiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 22 op/s 2024-05-09T05:43:45.384 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:45 smithi115 ceph-mon[30349]: pgmap v169: 129 pgs: 129 active+clean; 3.7 GiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 22 op/s 2024-05-09T05:43:45.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:45 smithi161 ceph-mon[33805]: pgmap v169: 129 pgs: 129 active+clean; 3.7 GiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 22 op/s 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.124+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3485070610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcd0104610 msgr2=0x7efcd0104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.124+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3485070610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcd0104610 0x7efcd0104a20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7efcc00099c0 tx=0x7efcc0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.124+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3485070610 shutdown_connections 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.124+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3485070610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcd0109890 0x7efcd01111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.124+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3485070610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcd0104610 0x7efcd0104a20 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.124+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3485070610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcd0108f40 0x7efcd0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.124+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3485070610 >> 172.21.15.115:0/3485070610 conn(0x7efcd007af70 msgr2=0x7efcd007b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.124+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3485070610 shutdown_connections 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.124+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3485070610 wait complete. 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.125+0000 7efcd5b6c700 1 Processor -- start 2024-05-09T05:43:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.125+0000 7efcd5b6c700 1 -- start start 2024-05-09T05:43:46.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efcd5b6c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcd0104610 0x7efcd019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efcd5b6c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcd0108f40 0x7efcd019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efcd5b6c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcd0109890 0x7efcd019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efcd5b6c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01143c0 con 0x7efcd0104610 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efcd5b6c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0114240 con 0x7efcd0109890 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efcd5b6c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0114540 con 0x7efcd0108f40 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcd0104610 0x7efcd019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcd0104610 0x7efcd019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42118/0 (socket says 172.21.15.115:42118) 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efccf7fe700 1 -- 172.21.15.115:0/3452851034 learned_addr learned my addr 172.21.15.115:0/3452851034 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efccffff700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcd0109890 0x7efcd019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efcceffd700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcd0108f40 0x7efcd019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efccffff700 1 -- 172.21.15.115:0/3452851034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcd0108f40 msgr2=0x7efcd019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efccffff700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcd0108f40 0x7efcd019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efccffff700 1 -- 172.21.15.115:0/3452851034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcd0104610 msgr2=0x7efcd019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efccffff700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcd0104610 0x7efcd019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.126+0000 7efccffff700 1 -- 172.21.15.115:0/3452851034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd019fc50 con 0x7efcd0109890 2024-05-09T05:43:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.127+0000 7efccffff700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcd0109890 0x7efcd019f5b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efcc800efc0 tx=0x7efcc800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.127+0000 7efcccff9700 1 -- 172.21.15.115:0/3452851034 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc80092e0 con 0x7efcd0109890 2024-05-09T05:43:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.127+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3452851034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd019fee0 con 0x7efcd0109890 2024-05-09T05:43:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.127+0000 7efcccff9700 1 -- 172.21.15.115:0/3452851034 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc8004bb0 con 0x7efcd0109890 2024-05-09T05:43:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.127+0000 7efcccff9700 1 -- 172.21.15.115:0/3452851034 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8004020 con 0x7efcd0109890 2024-05-09T05:43:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.127+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3452851034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd01a03d0 con 0x7efcd0109890 2024-05-09T05:43:46.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.129+0000 7efcccff9700 1 -- 172.21.15.115:0/3452851034 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7efcc801b030 con 0x7efcd0109890 2024-05-09T05:43:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.130+0000 7efcccff9700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efcb80b29f0 0x7efcb80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.130+0000 7efccf7fe700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efcb80b29f0 0x7efcb80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.131+0000 7efcccff9700 1 -- 172.21.15.115:0/3452851034 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efcc80e2420 con 0x7efcd0109890 2024-05-09T05:43:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.131+0000 7efccf7fe700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efcb80b29f0 0x7efcb80b4f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efcc0000c00 tx=0x7efcc001a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.131+0000 7efce4582b80 1 Processor -- start 2024-05-09T05:43:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.131+0000 7efce4582b80 1 -- start start 2024-05-09T05:43:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efce4582b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x56018765c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efce4582b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x56018765ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efce4582b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187656ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efce4582b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560187578ff0 con 0x5601876566a0 2024-05-09T05:43:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efce4582b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5601875177f0 con 0x5601875c3160 2024-05-09T05:43:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efce4582b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560187574fb0 con 0x5601874d94a0 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187656ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187656ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42124/0 (socket says 172.21.15.115:42124) 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x56018765c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccffff700 1 -- 172.21.15.115:0/91607864 learned_addr learned my addr 172.21.15.115:0/91607864 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efcceffd700 1 --2- 172.21.15.115:0/91607864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x56018765ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccffff700 1 -- 172.21.15.115:0/91607864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 msgr2=0x56018765ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccffff700 1 --2- 172.21.15.115:0/91607864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x56018765ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccffff700 1 -- 172.21.15.115:0/91607864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 msgr2=0x56018765c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccffff700 1 --2- 172.21.15.115:0/91607864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x56018765c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccffff700 1 -- 172.21.15.115:0/91607864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018750f370 con 0x5601876566a0 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.132+0000 7efccf7fe700 1 --2- 172.21.15.115:0/91607864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x56018765c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.133+0000 7efccffff700 1 --2- 172.21.15.115:0/91607864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187656ad0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efcc80a6250 tx=0x7efcc8003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab47ce8541ae801 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.133+0000 7efcb5ffb700 1 -- 172.21.15.115:0/91607864 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc80af020 con 0x5601876566a0 2024-05-09T05:43:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.133+0000 7efcb5ffb700 1 -- 172.21.15.115:0/91607864 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc80a9ce0 con 0x5601876566a0 2024-05-09T05:43:46.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.133+0000 7efcb5ffb700 1 -- 172.21.15.115:0/91607864 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc80aa430 con 0x5601876566a0 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 -- 172.21.15.115:0/91607864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 msgr2=0x560187656ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 --2- 172.21.15.115:0/91607864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187656ad0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efcc80a6250 tx=0x7efcc8003a00 comp rx=0 tx=0).stop 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 -- 172.21.15.115:0/91607864 shutdown_connections 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 --2- 172.21.15.115:0/91607864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x56018765c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 --2- 172.21.15.115:0/91607864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187656ad0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 --2- 172.21.15.115:0/91607864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x56018765ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 -- 172.21.15.115:0/91607864 >> 172.21.15.115:0/91607864 conn(0x5601875396a0 msgr2=0x560187651400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 -- 172.21.15.115:0/91607864 shutdown_connections 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 -- 172.21.15.115:0/91607864 wait complete. 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.134+0000 7efce4582b80 1 -- start start 2024-05-09T05:43:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efce4582b80 1 Processor -- start 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efce4582b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x560187660280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efce4582b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601875c3160 0x5601876607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efce4582b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601876566a0 0x560187660d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efce4582b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5601873f4ba0 con 0x5601875c3160 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efce4582b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560187533150 con 0x5601876566a0 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efce4582b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5601876631d0 con 0x5601874d94a0 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efce4582b80 10 client.0 _ll_register_callbacks cb 0x5601873ea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x560187660280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x560187660280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:35766/0 (socket says 172.21.15.115:35766) 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efccf7fe700 1 -- 172.21.15.115:0/2323809547 learned_addr learned my addr 172.21.15.115:0/2323809547 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efccffff700 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601876566a0 0x560187660d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efccf7fe700 1 -- 172.21.15.115:0/2323809547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601876566a0 msgr2=0x560187660d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.135+0000 7efccf7fe700 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601876566a0 0x560187660d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efccf7fe700 1 -- 172.21.15.115:0/2323809547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601875c3160 msgr2=0x5601876607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efccf7fe700 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601875c3160 0x5601876607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efccf7fe700 1 -- 172.21.15.115:0/2323809547 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601873f4490 con 0x5601874d94a0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efccf7fe700 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x560187660280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efcc0003b40 tx=0x7efcc0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66983f95908d9332 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efcaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efcaeffd700 1 -- 172.21.15.115:0/2323809547 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0035060 con 0x5601874d94a0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efce4582b80 1 -- 172.21.15.115:0/2323809547 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc80a9ce0 con 0x5601874d94a0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efcaeffd700 1 -- 172.21.15.115:0/2323809547 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efcc000bce0 con 0x5601874d94a0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efcaeffd700 1 -- 172.21.15.115:0/2323809547 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efcc003c050 con 0x5601874d94a0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efce4582b80 10 client.26590 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.136+0000 7efce4582b80 1 -- 172.21.15.115:0/2323809547 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56018766dcb0 con 0x5601874d94a0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.137+0000 7efcaeffd700 1 -- 172.21.15.115:0/2323809547 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc80a9ce0 con 0x5601874d94a0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.137+0000 7efcaeffd700 1 -- 172.21.15.115:0/2323809547 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7efcc0030420 con 0x5601874d94a0 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.137+0000 7efce4582b80 10 client.26590 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.137+0000 7efce4582b80 10 client.26590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.137+0000 7efce4582b80 20 client.26590 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:46.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.137+0000 7efce4582b80 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560187670670 0x560187672b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.137+0000 7efce4582b80 4 client.26590 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:46.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.137+0000 7efce4582b80 1 -- 172.21.15.115:0/2323809547 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601875869d0 con 0x560187670670 2024-05-09T05:43:46.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.137+0000 7efcceffd700 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560187670670 0x560187672b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.138+0000 7efcaeffd700 10 client.26590 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:46.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.138+0000 7efcceffd700 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560187670670 0x560187672b40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce31fe6cafa874db server_cookie=6b71a11834498f64 in_seq=0 out_seq=0 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.138+0000 7efcaeffd700 1 -- 172.21.15.115:0/2323809547 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5601875869d0 con 0x560187670670 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.138+0000 7efcaeffd700 10 client.26590 handle_command_reply: tid=0 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.138+0000 7efce4582b80 1 client.26590 shutdown 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 -- 172.21.15.115:0/2323809547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 msgr2=0x560187660280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x560187660280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efcc0003b40 tx=0x7efcc0009310 comp rx=0 tx=0).stop 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 -- 172.21.15.115:0/2323809547 shutdown_connections 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601876566a0 0x560187660d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560187670670 0x560187672b40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x560187660280 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 --2- 172.21.15.115:0/2323809547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601875c3160 0x5601876607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 -- 172.21.15.115:0/2323809547 >> 172.21.15.115:0/2323809547 conn(0x5601875396a0 msgr2=0x5601876511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efcaeffd700 0 client.26590 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:46.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 -- 172.21.15.115:0/2323809547 shutdown_connections 2024-05-09T05:43:46.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 1 -- 172.21.15.115:0/2323809547 wait complete. 2024-05-09T05:43:46.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.139+0000 7efce4582b80 20 client.26590 trim_cache size 0 max 16384 2024-05-09T05:43:46.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efce4582b80 1 Processor -- start 2024-05-09T05:43:46.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efce4582b80 1 -- start start 2024-05-09T05:43:46.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efce4582b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x5601876729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efce4582b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x56018765f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efce4582b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187660c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efce4582b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01b0380 con 0x5601876566a0 2024-05-09T05:43:46.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efce4582b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01b1ef0 con 0x5601875c3160 2024-05-09T05:43:46.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efce4582b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01b0a50 con 0x5601874d94a0 2024-05-09T05:43:46.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x5601876729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x5601876729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:35782/0 (socket says 172.21.15.115:35782) 2024-05-09T05:43:46.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.162+0000 7efccf7fe700 1 -- 172.21.15.115:0/4133424969 learned_addr learned my addr 172.21.15.115:0/4133424969 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:46.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efccffff700 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187660c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efcceffd700 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x56018765f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efccf7fe700 1 -- 172.21.15.115:0/4133424969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 msgr2=0x56018765f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efccf7fe700 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x56018765f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efccf7fe700 1 -- 172.21.15.115:0/4133424969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 msgr2=0x560187660c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efccf7fe700 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187660c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efccf7fe700 1 -- 172.21.15.115:0/4133424969 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560187660050 con 0x5601874d94a0 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efccffff700 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187660c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efccf7fe700 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x5601876729f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efcc0003760 tx=0x7efcc0009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c11feb21a88f72c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efcb57fa700 1 -- 172.21.15.115:0/4133424969 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0035050 con 0x5601874d94a0 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efcb57fa700 1 -- 172.21.15.115:0/4133424969 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc0030a60 con 0x5601874d94a0 2024-05-09T05:43:46.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.163+0000 7efcb57fa700 1 -- 172.21.15.115:0/4133424969 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc000fb10 con 0x5601874d94a0 2024-05-09T05:43:46.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.164+0000 7efce4582b80 1 -- 172.21.15.115:0/4133424969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 msgr2=0x5601876729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:46.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.164+0000 7efce4582b80 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x5601876729f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efcc0003760 tx=0x7efcc0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.164+0000 7efce4582b80 1 -- 172.21.15.115:0/4133424969 shutdown_connections 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.164+0000 7efce4582b80 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601876566a0 0x560187660c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.164+0000 7efce4582b80 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601874d94a0 0x5601876729f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.164+0000 7efce4582b80 1 --2- 172.21.15.115:0/4133424969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x56018765f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.164+0000 7efce4582b80 1 -- 172.21.15.115:0/4133424969 >> 172.21.15.115:0/4133424969 conn(0x5601875396a0 msgr2=0x560187661930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 -- 172.21.15.115:0/4133424969 shutdown_connections 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 -- 172.21.15.115:0/4133424969 wait complete. 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 -- start start 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 Processor -- start 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601874d94a0 0x5601876685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x560187677910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601876566a0 0x560187677e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01b1820 con 0x5601874d94a0 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01b1120 con 0x5601875c3160 2024-05-09T05:43:46.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.165+0000 7efce4582b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01ae320 con 0x5601876566a0 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efce4582b80 10 client.0 _ll_register_callbacks cb 0x5601873ea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601874d94a0 0x5601876685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccf7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601874d94a0 0x5601876685c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42156/0 (socket says 172.21.15.115:42156) 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccf7fe700 1 -- 172.21.15.115:0/3012868799 learned_addr learned my addr 172.21.15.115:0/3012868799 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccffff700 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601876566a0 0x560187677e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efcceffd700 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x560187677910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccf7fe700 1 -- 172.21.15.115:0/3012868799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601876566a0 msgr2=0x560187677e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccf7fe700 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601876566a0 0x560187677e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccf7fe700 1 -- 172.21.15.115:0/3012868799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 msgr2=0x560187677910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccf7fe700 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x560187677910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccf7fe700 1 -- 172.21.15.115:0/3012868799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601876785d0 con 0x5601874d94a0 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efccf7fe700 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601874d94a0 0x5601876685c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7efcc00307f0 tx=0x7efcc0030420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325ed21a94ed0220 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efcaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efcaeffd700 1 -- 172.21.15.115:0/3012868799 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0044070 con 0x5601874d94a0 2024-05-09T05:43:46.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efcaeffd700 1 -- 172.21.15.115:0/3012868799 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7efcc000f460 con 0x5601874d94a0 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.166+0000 7efce4582b80 1 -- 172.21.15.115:0/3012868799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc0030a60 con 0x5601874d94a0 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efcaeffd700 1 -- 172.21.15.115:0/3012868799 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efcc0053050 con 0x5601874d94a0 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efce4582b80 10 client.17106 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efce4582b80 1 -- 172.21.15.115:0/3012868799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5601876dad90 con 0x5601874d94a0 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efcaeffd700 1 -- 172.21.15.115:0/3012868799 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcc0030a60 con 0x5601874d94a0 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efcaeffd700 1 -- 172.21.15.115:0/3012868799 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7efcc0040550 con 0x5601874d94a0 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efce4582b80 10 client.17106 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efce4582b80 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efce4582b80 20 client.17106 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efce4582b80 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5601876db1d0 0x5601876dd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:46.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efce4582b80 4 client.17106 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:43:46.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.167+0000 7efce4582b80 1 -- 172.21.15.115:0/3012868799 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560187580a70 con 0x5601876db1d0 2024-05-09T05:43:46.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.168+0000 7efcceffd700 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5601876db1d0 0x5601876dd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:46.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.168+0000 7efcaeffd700 10 client.17106 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:46.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.168+0000 7efcceffd700 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5601876db1d0 0x5601876dd5f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00e7e972efa8e04 server_cookie=16b043d541aa3524 in_seq=0 out_seq=0 2024-05-09T05:43:46.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.168+0000 7efcaeffd700 1 -- 172.21.15.115:0/3012868799 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560187580a70 con 0x5601876db1d0 2024-05-09T05:43:46.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.168+0000 7efcaeffd700 10 client.17106 handle_command_reply: tid=0 2024-05-09T05:43:46.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.168+0000 7efce4582b80 1 client.17106 shutdown 2024-05-09T05:43:46.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 -- 172.21.15.115:0/3012868799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601874d94a0 msgr2=0x5601876685c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:46.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601874d94a0 0x5601876685c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7efcc00307f0 tx=0x7efcc0030420 comp rx=0 tx=0).stop 2024-05-09T05:43:46.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 -- 172.21.15.115:0/3012868799 shutdown_connections 2024-05-09T05:43:46.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5601876db1d0 0x5601876dd5f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efcaeffd700 0 client.17106 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:46.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5601876566a0 0x560187677e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5601874d94a0 0x5601876685c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 --2- 172.21.15.115:0/3012868799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601875c3160 0x560187677910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 -- 172.21.15.115:0/3012868799 >> 172.21.15.115:0/3012868799 conn(0x5601875396a0 msgr2=0x560187655e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:46.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 -- 172.21.15.115:0/3012868799 shutdown_connections 2024-05-09T05:43:46.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 1 -- 172.21.15.115:0/3012868799 wait complete. 2024-05-09T05:43:46.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.169+0000 7efce4582b80 20 client.17106 trim_cache size 0 max 16384 2024-05-09T05:43:46.170 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:43:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T05:43:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": { 2024-05-09T05:43:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "0eab15dd-3683-4dec-bf35-6c7317623d7a": { 2024-05-09T05:43:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/", 2024-05-09T05:43:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "tag": "0eab15dd-3683-4dec-bf35-6c7317623d7a", 2024-05-09T05:43:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "options": "recursive,force" 2024-05-09T05:43:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:43:46.171 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:43:46.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3452851034 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efcb80b29f0 msgr2=0x7efcb80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:46.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efcb80b29f0 0x7efcb80b4f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efcc0000c00 tx=0x7efcc001a040 comp rx=0 tx=0).stop 2024-05-09T05:43:46.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3452851034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcd0109890 msgr2=0x7efcd019f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:46.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcd0109890 0x7efcd019f5b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efcc800efc0 tx=0x7efcc800c3a0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3452851034 shutdown_connections 2024-05-09T05:43:46.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7efcb80b29f0 0x7efcb80b4f00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcd0109890 0x7efcd019f5b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcd0104610 0x7efcd019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 --2- 172.21.15.115:0/3452851034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efcd0108f40 0x7efcd019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:46.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3452851034 >> 172.21.15.115:0/3452851034 conn(0x7efcd007af70 msgr2=0x7efcd0101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:46.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3452851034 shutdown_connections 2024-05-09T05:43:46.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:43:46.170+0000 7efcd5b6c700 1 -- 172.21.15.115:0/3452851034 wait complete. 2024-05-09T05:43:47.154 INFO:tasks.cephfs.filesystem:scrub status for tag:0eab15dd-3683-4dec-bf35-6c7317623d7a - {'path': '/', 'tag': '0eab15dd-3683-4dec-bf35-6c7317623d7a', 'options': 'recursive,force'} 2024-05-09T05:43:47.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:47 smithi115 ceph-mon[30349]: pgmap v170: 129 pgs: 129 active+clean; 3.7 GiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-09T05:43:47.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:47 smithi161 ceph-mon[33805]: pgmap v170: 129 pgs: 129 active+clean; 3.7 GiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-09T05:43:47.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:47 smithi195 ceph-mon[33844]: pgmap v170: 129 pgs: 129 active+clean; 3.7 GiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-09T05:43:49.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:49 smithi115 ceph-mon[30349]: pgmap v171: 129 pgs: 129 active+clean; 3.9 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 45 op/s 2024-05-09T05:43:49.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:49 smithi161 ceph-mon[33805]: pgmap v171: 129 pgs: 129 active+clean; 3.9 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 45 op/s 2024-05-09T05:43:49.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:49 smithi195 ceph-mon[33844]: pgmap v171: 129 pgs: 129 active+clean; 3.9 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 45 op/s 2024-05-09T05:43:51.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:51 smithi115 ceph-mon[30349]: pgmap v172: 129 pgs: 129 active+clean; 3.9 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 40 op/s 2024-05-09T05:43:51.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:51 smithi161 ceph-mon[33805]: pgmap v172: 129 pgs: 129 active+clean; 3.9 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 40 op/s 2024-05-09T05:43:51.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:51 smithi195 ceph-mon[33844]: pgmap v172: 129 pgs: 129 active+clean; 3.9 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 40 op/s 2024-05-09T05:43:53.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:53 smithi115 ceph-mon[30349]: pgmap v173: 129 pgs: 129 active+clean; 3.9 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 40 op/s 2024-05-09T05:43:53.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:53 smithi161 ceph-mon[33805]: pgmap v173: 129 pgs: 129 active+clean; 3.9 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 40 op/s 2024-05-09T05:43:53.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:53 smithi195 ceph-mon[33844]: pgmap v173: 129 pgs: 129 active+clean; 3.9 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 40 op/s 2024-05-09T05:43:55.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:55 smithi115 ceph-mon[30349]: pgmap v174: 129 pgs: 129 active+clean; 4.0 GiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 49 op/s 2024-05-09T05:43:55.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:55 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:43:55.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:55 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:43:55.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:55 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:43:55.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:55 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:43:55.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:55 smithi161 ceph-mon[33805]: pgmap v174: 129 pgs: 129 active+clean; 4.0 GiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 49 op/s 2024-05-09T05:43:55.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:55 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:43:55.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:55 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:43:55.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:55 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:43:55.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:55 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:43:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:55 smithi195 ceph-mon[33844]: pgmap v174: 129 pgs: 129 active+clean; 4.0 GiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 49 op/s 2024-05-09T05:43:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:55 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:43:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:55 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:43:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:55 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:43:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:55 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:43:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.952+0000 7f0c1c538640 1 -- 172.21.15.115:0/2653535280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14108b90 msgr2=0x7f0c14102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.952+0000 7f0c1c538640 1 --2- 172.21.15.115:0/2653535280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14108b90 0x7f0c14102730 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009900 tx=0x7f0c1002a410 comp rx=0 tx=0).stop 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.952+0000 7f0c1c538640 1 -- 172.21.15.115:0/2653535280 shutdown_connections 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.952+0000 7f0c1c538640 1 --2- 172.21.15.115:0/2653535280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141032e0 0x7f0c1410f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.952+0000 7f0c1c538640 1 --2- 172.21.15.115:0/2653535280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c14109030 0x7f0c14102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.952+0000 7f0c1c538640 1 --2- 172.21.15.115:0/2653535280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14108b90 0x7f0c14102730 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.952+0000 7f0c1c538640 1 -- 172.21.15.115:0/2653535280 >> 172.21.15.115:0/2653535280 conn(0x7f0c140fe390 msgr2=0x7f0c141007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.952+0000 7f0c1c538640 1 -- 172.21.15.115:0/2653535280 shutdown_connections 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.952+0000 7f0c1c538640 1 -- 172.21.15.115:0/2653535280 wait complete. 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.953+0000 7f0c1c538640 1 Processor -- start 2024-05-09T05:43:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.953+0000 7f0c1c538640 1 -- start start 2024-05-09T05:43:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1c538640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141032e0 0x7f0c141a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1c538640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c14108b90 0x7f0c141a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1c538640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14109030 0x7f0c1419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1c538640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14112c30 con 0x7f0c14109030 2024-05-09T05:43:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1c538640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14112ab0 con 0x7f0c14108b90 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1c538640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14112db0 con 0x7f0c141032e0 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1aad5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14109030 0x7f0c1419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1aad5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14109030 0x7f0c1419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54050/0 (socket says 172.21.15.115:54050) 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1aad5640 1 -- 172.21.15.115:0/1657763472 learned_addr learned my addr 172.21.15.115:0/1657763472 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c14108b90 0x7f0c141a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1a2d4640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141032e0 0x7f0c141a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1aad5640 1 -- 172.21.15.115:0/1657763472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141032e0 msgr2=0x7f0c141a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1aad5640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141032e0 0x7f0c141a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1aad5640 1 -- 172.21.15.115:0/1657763472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c14108b90 msgr2=0x7f0c141a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1aad5640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c14108b90 0x7f0c141a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1aad5640 1 -- 172.21.15.115:0/1657763472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1419ad90 con 0x7f0c14109030 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c1aad5640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14109030 0x7f0c1419a660 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00eaa0 tx=0x7f0c0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.954+0000 7f0c0b7fe640 1 -- 172.21.15.115:0/1657763472 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00cdb0 con 0x7f0c14109030 2024-05-09T05:43:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.955+0000 7f0c0b7fe640 1 -- 172.21.15.115:0/1657763472 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c0c005020 con 0x7f0c14109030 2024-05-09T05:43:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.955+0000 7f0c1c538640 1 -- 172.21.15.115:0/1657763472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1419b020 con 0x7f0c14109030 2024-05-09T05:43:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.955+0000 7f0c0b7fe640 1 -- 172.21.15.115:0/1657763472 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0116d0 con 0x7f0c14109030 2024-05-09T05:43:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.955+0000 7f0c1c538640 1 -- 172.21.15.115:0/1657763472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c141a7280 con 0x7f0c14109030 2024-05-09T05:43:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.957+0000 7f0c233ee740 1 Processor -- start 2024-05-09T05:43:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.958+0000 7f0c0b7fe640 1 -- 172.21.15.115:0/1657763472 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0c0c0051d0 con 0x7f0c14109030 2024-05-09T05:43:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c233ee740 1 -- start start 2024-05-09T05:43:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269db1b0 0x559c26d7e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269b22f0 0x559c26d7ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 0x559c26d7f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c233ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c26ce37c0 con 0x559c269db1b0 2024-05-09T05:43:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c233ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c26c7f0e0 con 0x559c269b22f0 2024-05-09T05:43:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c233ee740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559c26cdf780 con 0x559c269ba4d0 2024-05-09T05:43:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c19ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269db1b0 0x559c26d7e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c19ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269db1b0 0x559c26d7e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54058/0 (socket says 172.21.15.115:54058) 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c19ad3640 1 -- 172.21.15.115:0/848051374 learned_addr learned my addr 172.21.15.115:0/848051374 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c1a2d4640 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269b22f0 0x559c26d7ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 0x559c26d7f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.959+0000 7f0c0b7fe640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0c000b2950 0x7f0c000b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c1aad5640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0c000b2950 0x7f0c000b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c19ad3640 1 -- 172.21.15.115:0/848051374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 msgr2=0x559c26d7f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 0x559c26d7f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c19ad3640 1 -- 172.21.15.115:0/848051374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269b22f0 msgr2=0x559c26d7ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269b22f0 0x559c26d7ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c19ad3640 1 -- 172.21.15.115:0/848051374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c26ca87c0 con 0x559c269db1b0 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c0b7fe640 1 -- 172.21.15.115:0/1657763472 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0c0c0e2a50 con 0x7f0c14109030 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c1a2d4640 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269b22f0 0x559c26d7ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269db1b0 0x559c26d7e680 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0c04012680 tx=0x7f0c0400b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb587bb26ee2e675 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c08bfe640 1 -- 172.21.15.115:0/848051374 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040120d0 con 0x559c269db1b0 2024-05-09T05:43:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c08bfe640 1 -- 172.21.15.115:0/848051374 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c040044c0 con 0x559c269db1b0 2024-05-09T05:43:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.960+0000 7f0c08bfe640 1 -- 172.21.15.115:0/848051374 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040047a0 con 0x559c269db1b0 2024-05-09T05:43:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c233ee740 1 -- 172.21.15.115:0/848051374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269db1b0 msgr2=0x559c26d7e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c233ee740 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269db1b0 0x559c26d7e680 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0c04012680 tx=0x7f0c0400b740 comp rx=0 tx=0).stop 2024-05-09T05:43:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c1aad5640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0c000b2950 0x7f0c000b4e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0e0460 tx=0x7f0c0c091040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c233ee740 1 -- 172.21.15.115:0/848051374 shutdown_connections 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c233ee740 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 0x559c26d7f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c233ee740 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269b22f0 0x559c26d7ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c233ee740 1 --2- 172.21.15.115:0/848051374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269db1b0 0x559c26d7e680 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c233ee740 1 -- 172.21.15.115:0/848051374 >> 172.21.15.115:0/848051374 conn(0x559c269cf300 msgr2=0x559c26d7ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c233ee740 1 -- 172.21.15.115:0/848051374 shutdown_connections 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.961+0000 7f0c233ee740 1 -- 172.21.15.115:0/848051374 wait complete. 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c233ee740 1 -- start start 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c233ee740 1 Processor -- start 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269ba4d0 0x559c26d8ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269db1b0 0x559c26d88c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c233ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c26d8b750 con 0x559c269b22f0 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c233ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c26c6a9c0 con 0x559c269ba4d0 2024-05-09T05:43:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c233ee740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559c26d8b8d0 con 0x559c269db1b0 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c233ee740 10 client.0 _ll_register_callbacks cb 0x559c26c97020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c19ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c19ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54072/0 (socket says 172.21.15.115:54072) 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c19ad3640 1 -- 172.21.15.115:0/2481586349 learned_addr learned my addr 172.21.15.115:0/2481586349 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c1a2d4640 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269ba4d0 0x559c26d8ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.962+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269db1b0 0x559c26d88c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0c19ad3640 1 -- 172.21.15.115:0/2481586349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269db1b0 msgr2=0x559c26d88c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269db1b0 0x559c26d88c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0c19ad3640 1 -- 172.21.15.115:0/2481586349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269ba4d0 msgr2=0x559c26d8ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269ba4d0 0x559c26d8ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0c19ad3640 1 -- 172.21.15.115:0/2481586349 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c26c911f0 con 0x559c269b22f0 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8a820 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0c040059f0 tx=0x7f0c04005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc672f9d436fea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0bfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2481586349 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040046c0 con 0x559c269b22f0 2024-05-09T05:43:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2481586349 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0c04004860 con 0x559c269b22f0 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0c233ee740 1 -- 172.21.15.115:0/2481586349 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c26cf13d0 con 0x559c269b22f0 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2481586349 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0c04035050 con 0x559c269b22f0 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0c233ee740 10 client.17118 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0c233ee740 1 -- 172.21.15.115:0/2481586349 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559c26a46790 con 0x559c269b22f0 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.963+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2481586349 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559c26cf13d0 con 0x559c269b22f0 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.964+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2481586349 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f0c04010920 con 0x559c269b22f0 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.964+0000 7f0c233ee740 10 client.17118 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.964+0000 7f0c233ee740 10 client.17118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.964+0000 7f0c233ee740 20 client.17118 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.964+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c26999d80 0x559c26d9ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.964+0000 7f0c233ee740 4 client.17118 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.964+0000 7f0c233ee740 1 -- 172.21.15.115:0/2481586349 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c26ceb240 con 0x559c26999d80 2024-05-09T05:43:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.964+0000 7f0c1a2d4640 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c26999d80 0x559c26d9ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.965+0000 7f0bfdffb640 10 client.17118 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.965+0000 7f0c1a2d4640 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c26999d80 0x559c26d9ae00 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c0af05b342ab88e server_cookie=b81c09e5912e16e1 in_seq=0 out_seq=0 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.965+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2481586349 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559c26ceb240 con 0x559c26999d80 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.965+0000 7f0bfdffb640 10 client.17118 handle_command_reply: tid=0 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.965+0000 7f0c233ee740 1 client.17118 shutdown 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 -- 172.21.15.115:0/2481586349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 msgr2=0x559c26d8a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8a820 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0c040059f0 tx=0x7f0c04005bd0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 -- 172.21.15.115:0/2481586349 shutdown_connections 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c26999d80 0x559c26d9ae00 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269db1b0 0x559c26d88c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0bfdffb640 0 client.17118 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269ba4d0 0x559c26d8ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2481586349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8a820 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 -- 172.21.15.115:0/2481586349 >> 172.21.15.115:0/2481586349 conn(0x559c269cf300 msgr2=0x559c26d803a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 -- 172.21.15.115:0/2481586349 shutdown_connections 2024-05-09T05:43:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 1 -- 172.21.15.115:0/2481586349 wait complete. 2024-05-09T05:43:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.966+0000 7f0c233ee740 20 client.17118 trim_cache size 0 max 16384 2024-05-09T05:43:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.996+0000 7f0c233ee740 1 Processor -- start 2024-05-09T05:43:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c233ee740 1 -- start start 2024-05-09T05:43:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d87c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 0x559c26d90b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269db1b0 0x559c26d91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c233ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141b11d0 con 0x559c269b22f0 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c233ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141b2d40 con 0x559c269db1b0 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c233ee740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141b18a0 con 0x559c269ba4d0 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c19ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d87c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c19ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d87c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54076/0 (socket says 172.21.15.115:54076) 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.997+0000 7f0c19ad3640 1 -- 172.21.15.115:0/1077400962 learned_addr learned my addr 172.21.15.115:0/1077400962 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0c1a2d4640 1 --2- 172.21.15.115:0/1077400962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 0x559c26d90b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0c19ad3640 1 -- 172.21.15.115:0/1077400962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 msgr2=0x559c26d90b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/1077400962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 0x559c26d90b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0c19ad3640 1 -- 172.21.15.115:0/1077400962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269db1b0 msgr2=0x559c26d91090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/1077400962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269db1b0 0x559c26d91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0c19ad3640 1 -- 172.21.15.115:0/1077400962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c26abd5f0 con 0x559c269b22f0 2024-05-09T05:43:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/1077400962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d87c10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400bd20 tx=0x7f0c0400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276a6ef65b8ca0d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0bfffff640 1 -- 172.21.15.115:0/1077400962 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04005760 con 0x559c269b22f0 2024-05-09T05:43:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0bfffff640 1 -- 172.21.15.115:0/1077400962 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c040052b0 con 0x559c269b22f0 2024-05-09T05:43:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.998+0000 7f0bfffff640 1 -- 172.21.15.115:0/1077400962 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04010460 con 0x559c269b22f0 2024-05-09T05:43:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.999+0000 7f0c233ee740 1 -- 172.21.15.115:0/1077400962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 msgr2=0x559c26d87c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.999+0000 7f0c233ee740 1 --2- 172.21.15.115:0/1077400962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d87c10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400bd20 tx=0x7f0c0400be50 comp rx=0 tx=0).stop 2024-05-09T05:43:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.999+0000 7f0c233ee740 1 -- 172.21.15.115:0/1077400962 shutdown_connections 2024-05-09T05:43:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.999+0000 7f0c233ee740 1 --2- 172.21.15.115:0/1077400962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269db1b0 0x559c26d91090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.999+0000 7f0c233ee740 1 --2- 172.21.15.115:0/1077400962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269ba4d0 0x559c26d90b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.999+0000 7f0c233ee740 1 --2- 172.21.15.115:0/1077400962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d87c10 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:55.999+0000 7f0c233ee740 1 -- 172.21.15.115:0/1077400962 >> 172.21.15.115:0/1077400962 conn(0x559c269cf300 msgr2=0x559c26d87500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 -- 172.21.15.115:0/1077400962 shutdown_connections 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 -- 172.21.15.115:0/1077400962 wait complete. 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 -- start start 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 Processor -- start 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269ba4d0 0x559c26d8c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269db1b0 0x559c26d905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141b2670 con 0x559c269b22f0 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141b1f70 con 0x559c269ba4d0 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141af1b0 con 0x559c269db1b0 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.000+0000 7f0c233ee740 10 client.0 _ll_register_callbacks cb 0x7f0c10005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c19ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c19ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54080/0 (socket says 172.21.15.115:54080) 2024-05-09T05:43:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c19ad3640 1 -- 172.21.15.115:0/2678389164 learned_addr learned my addr 172.21.15.115:0/2678389164 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c19ad3640 1 -- 172.21.15.115:0/2678389164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269db1b0 msgr2=0x559c26d905b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269db1b0 0x559c26d905b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c19ad3640 1 -- 172.21.15.115:0/2678389164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269ba4d0 msgr2=0x559c26d8c6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269ba4d0 0x559c26d8c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c19ad3640 1 -- 172.21.15.115:0/2678389164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c26ca7ed0 con 0x559c269b22f0 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c19ad3640 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8c180 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400b5c0 tx=0x7f0c04005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14106b9933cefd1b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0bfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2678389164 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0402fe60 con 0x559c269b22f0 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2678389164 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f0c0402f9b0 con 0x559c269b22f0 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.001+0000 7f0c233ee740 1 -- 172.21.15.115:0/2678389164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c040052b0 con 0x559c269b22f0 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2678389164 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0c04005760 con 0x559c269b22f0 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0c233ee740 10 client.17124 fetch_fsmap learned FSMap version 37 2024-05-09T05:43:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0c233ee740 1 -- 172.21.15.115:0/2678389164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559c26e028f0 con 0x559c269b22f0 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2678389164 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c04010460 con 0x559c269b22f0 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2678389164 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f0c04004680 con 0x559c269b22f0 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0c233ee740 10 client.17124 fetch_fsmap finished waiting for FSMap version 37 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0c233ee740 10 client.17124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0c233ee740 20 client.17124 populate_metadata read hostname 'smithi115' 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c2696f1c0 0x559c26e04ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0c233ee740 4 client.17124 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.002+0000 7f0c233ee740 1 -- 172.21.15.115:0/2678389164 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0c040052b0 con 0x559c2696f1c0 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.003+0000 7f0c1a2d4640 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c2696f1c0 0x559c26e04ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:43:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.004+0000 7f0bfdffb640 10 client.17124 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.004+0000 7f0c1a2d4640 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c2696f1c0 0x559c26e04ee0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1a17001d4fdd7ea server_cookie=9775010ca75e5ba9 in_seq=0 out_seq=0 2024-05-09T05:43:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0bfdffb640 1 -- 172.21.15.115:0/2678389164 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f0c040052b0 con 0x559c2696f1c0 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0bfdffb640 10 client.17124 handle_command_reply: tid=0 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 client.17124 shutdown 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 -- 172.21.15.115:0/2678389164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 msgr2=0x559c26d8c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8c180 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400b5c0 tx=0x7f0c04005d00 comp rx=0 tx=0).stop 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 -- 172.21.15.115:0/2678389164 shutdown_connections 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c2696f1c0 0x559c26e04ee0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c269db1b0 0x559c26d905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0bfdffb640 0 client.17124 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c269ba4d0 0x559c26d8c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 --2- 172.21.15.115:0/2678389164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c269b22f0 0x559c26d8c180 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 -- 172.21.15.115:0/2678389164 >> 172.21.15.115:0/2678389164 conn(0x559c269cf300 msgr2=0x559c26d8a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 -- 172.21.15.115:0/2678389164 shutdown_connections 2024-05-09T05:43:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 1 -- 172.21.15.115:0/2678389164 wait complete. 2024-05-09T05:43:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.013+0000 7f0c233ee740 20 client.17124 trim_cache size 0 max 16384 2024-05-09T05:43:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:43:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:43:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:43:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 -- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0c000b2950 msgr2=0x7f0c000b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0c000b2950 0x7f0c000b4e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0e0460 tx=0x7f0c0c091040 comp rx=0 tx=0).stop 2024-05-09T05:43:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 -- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14109030 msgr2=0x7f0c1419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:43:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14109030 0x7f0c1419a660 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00eaa0 tx=0x7f0c0c00ef60 comp rx=0 tx=0).stop 2024-05-09T05:43:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 -- 172.21.15.115:0/1657763472 shutdown_connections 2024-05-09T05:43:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0c000b2950 0x7f0c000b4e60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c14109030 0x7f0c1419a660 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c14108b90 0x7f0c141a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 --2- 172.21.15.115:0/1657763472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c141032e0 0x7f0c141a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:43:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 -- 172.21.15.115:0/1657763472 >> 172.21.15.115:0/1657763472 conn(0x7f0c140fe390 msgr2=0x7f0c140fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:43:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 -- 172.21.15.115:0/1657763472 shutdown_connections 2024-05-09T05:43:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:43:56.014+0000 7f0c1c538640 1 -- 172.21.15.115:0/1657763472 wait complete. 2024-05-09T05:43:57.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:57 smithi115 ceph-mon[30349]: pgmap v175: 129 pgs: 129 active+clean; 4.0 GiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 39 op/s 2024-05-09T05:43:57.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:57 smithi161 ceph-mon[33805]: pgmap v175: 129 pgs: 129 active+clean; 4.0 GiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 39 op/s 2024-05-09T05:43:57.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:57 smithi195 ceph-mon[33844]: pgmap v175: 129 pgs: 129 active+clean; 4.0 GiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 39 op/s 2024-05-09T05:43:59.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:43:59 smithi115 ceph-mon[30349]: pgmap v176: 129 pgs: 129 active+clean; 4.2 GiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 51 op/s 2024-05-09T05:43:59.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:43:59 smithi161 ceph-mon[33805]: pgmap v176: 129 pgs: 129 active+clean; 4.2 GiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 51 op/s 2024-05-09T05:43:59.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:43:59 smithi195 ceph-mon[33844]: pgmap v176: 129 pgs: 129 active+clean; 4.2 GiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 51 op/s 2024-05-09T05:44:00.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:00 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:44:00.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:00 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:44:00.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:00 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:44:01.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:01 smithi115 ceph-mon[30349]: pgmap v177: 129 pgs: 129 active+clean; 4.2 GiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:44:01.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:01 smithi161 ceph-mon[33805]: pgmap v177: 129 pgs: 129 active+clean; 4.2 GiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:44:01.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:01 smithi195 ceph-mon[33844]: pgmap v177: 129 pgs: 129 active+clean; 4.2 GiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:44:03.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:03 smithi161 ceph-mon[33805]: pgmap v178: 129 pgs: 129 active+clean; 4.2 GiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:44:03.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:03 smithi195 ceph-mon[33844]: pgmap v178: 129 pgs: 129 active+clean; 4.2 GiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:44:03.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:03 smithi115 ceph-mon[30349]: pgmap v178: 129 pgs: 129 active+clean; 4.2 GiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:44:05.115 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:04 smithi195 ceph-mon[33844]: pgmap v179: 129 pgs: 129 active+clean; 4.3 GiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:44:05.145 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:04 smithi115 ceph-mon[30349]: pgmap v179: 129 pgs: 129 active+clean; 4.3 GiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:44:05.159 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:04 smithi161 ceph-mon[33805]: pgmap v179: 129 pgs: 129 active+clean; 4.3 GiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:44:07.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:07 smithi195 ceph-mon[33844]: pgmap v180: 129 pgs: 129 active+clean; 4.3 GiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 35 op/s 2024-05-09T05:44:07.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:07 smithi115 ceph-mon[30349]: pgmap v180: 129 pgs: 129 active+clean; 4.3 GiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 35 op/s 2024-05-09T05:44:07.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:07 smithi161 ceph-mon[33805]: pgmap v180: 129 pgs: 129 active+clean; 4.3 GiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 35 op/s 2024-05-09T05:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.130+0000 7feb95c13640 1 -- 172.21.15.115:0/3026804849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 msgr2=0x7feb9010f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.130+0000 7feb95c13640 1 --2- 172.21.15.115:0/3026804849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 0x7feb9010f740 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7feb88009ca0 tx=0x7feb8802a8c0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.130+0000 7feb95c13640 1 -- 172.21.15.115:0/3026804849 shutdown_connections 2024-05-09T05:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.130+0000 7feb95c13640 1 --2- 172.21.15.115:0/3026804849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 0x7feb9010f740 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.130+0000 7feb95c13640 1 --2- 172.21.15.115:0/3026804849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb901090c0 0x7feb90103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.130+0000 7feb95c13640 1 --2- 172.21.15.115:0/3026804849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb90108c20 0x7feb90102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.130+0000 7feb95c13640 1 -- 172.21.15.115:0/3026804849 >> 172.21.15.115:0/3026804849 conn(0x7feb900779e0 msgr2=0x7feb90077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.130+0000 7feb95c13640 1 -- 172.21.15.115:0/3026804849 shutdown_connections 2024-05-09T05:44:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.130+0000 7feb95c13640 1 -- 172.21.15.115:0/3026804849 wait complete. 2024-05-09T05:44:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.131+0000 7feb95c13640 1 Processor -- start 2024-05-09T05:44:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.131+0000 7feb95c13640 1 -- start start 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb95c13640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 0x7feb9019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb95c13640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb90108c20 0x7feb9019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb95c13640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb901090c0 0x7feb9019c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb95c13640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb90112d10 con 0x7feb901039d0 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb95c13640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb90112b90 con 0x7feb901090c0 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb95c13640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb90112e90 con 0x7feb90108c20 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 0x7feb9019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 0x7feb9019b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44450/0 (socket says 172.21.15.115:44450) 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8f7fe640 1 -- 172.21.15.115:0/3427339775 learned_addr learned my addr 172.21.15.115:0/3427339775 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8effd640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb90108c20 0x7feb9019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8ffff640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb901090c0 0x7feb9019c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8f7fe640 1 -- 172.21.15.115:0/3427339775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb90108c20 msgr2=0x7feb9019bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb90108c20 0x7feb9019bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8f7fe640 1 -- 172.21.15.115:0/3427339775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb901090c0 msgr2=0x7feb9019c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb901090c0 0x7feb9019c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8f7fe640 1 -- 172.21.15.115:0/3427339775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb901a6880 con 0x7feb901039d0 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.132+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 0x7feb9019b650 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7feb8000c970 tx=0x7feb8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.133+0000 7feb8cff9640 1 -- 172.21.15.115:0/3427339775 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80014070 con 0x7feb901039d0 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.133+0000 7feb8cff9640 1 -- 172.21.15.115:0/3427339775 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb80004440 con 0x7feb901039d0 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.133+0000 7feb95c13640 1 -- 172.21.15.115:0/3427339775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb901a6b70 con 0x7feb901039d0 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.133+0000 7feb8cff9640 1 -- 172.21.15.115:0/3427339775 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80003d40 con 0x7feb901039d0 2024-05-09T05:44:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.133+0000 7feb95c13640 1 -- 172.21.15.115:0/3427339775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb901a7110 con 0x7feb901039d0 2024-05-09T05:44:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.135+0000 7feb8cff9640 1 -- 172.21.15.115:0/3427339775 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7feb80017070 con 0x7feb901039d0 2024-05-09T05:44:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb9cb9f740 1 Processor -- start 2024-05-09T05:44:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb9cb9f740 1 -- start start 2024-05-09T05:44:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc771b0 0x558d0001a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0001abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 0x558d0001b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558cfff7f7c0 con 0x558cffc4e2f0 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558cfff1b0e0 con 0x558cffc564d0 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558cfff7b780 con 0x558cffc771b0 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb8f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0001abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb8cff9640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feb780b29d0 0x7feb780b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc771b0 0x558d0001a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb8f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0001abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44454/0 (socket says 172.21.15.115:44454) 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb8f7fe640 1 -- 172.21.15.115:0/3232660958 learned_addr learned my addr 172.21.15.115:0/3232660958 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.136+0000 7feb8effd640 1 --2- 172.21.15.115:0/3232660958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 0x558d0001b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8ffff640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feb780b29d0 0x7feb780b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8cff9640 1 -- 172.21.15.115:0/3427339775 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7feb800e27d0 con 0x7feb901039d0 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8f7fe640 1 -- 172.21.15.115:0/3232660958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc771b0 msgr2=0x558d0001a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/3232660958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc771b0 0x558d0001a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8f7fe640 1 -- 172.21.15.115:0/3232660958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 msgr2=0x558d0001b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/3232660958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 0x558d0001b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8f7fe640 1 -- 172.21.15.115:0/3232660958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfff447c0 con 0x558cffc4e2f0 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8ffff640 1 --2- 172.21.15.115:0/3232660958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc771b0 0x558d0001a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/3232660958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0001abc0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7feb800a57f0 tx=0x7feb800045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02fc48e5f667e9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb75ffb640 1 -- 172.21.15.115:0/3232660958 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb800a9770 con 0x558cffc4e2f0 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb8ffff640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feb780b29d0 0x7feb780b4ee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feb88005cc0 tx=0x7feb8802a8c0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb75ffb640 1 -- 172.21.15.115:0/3232660958 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb800a9280 con 0x558cffc4e2f0 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.137+0000 7feb75ffb640 1 -- 172.21.15.115:0/3232660958 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb800a5a00 con 0x558cffc4e2f0 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.138+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3232660958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 msgr2=0x558d0001abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.138+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/3232660958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0001abc0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7feb800a57f0 tx=0x7feb800045f0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.138+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3232660958 shutdown_connections 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.138+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/3232660958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 0x558d0001b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.138+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/3232660958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0001abc0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.138+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/3232660958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc771b0 0x558d0001a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.138+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3232660958 >> 172.21.15.115:0/3232660958 conn(0x558cffc6b300 msgr2=0x558d00016e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.138+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3232660958 shutdown_connections 2024-05-09T05:44:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.138+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3232660958 wait complete. 2024-05-09T05:44:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb9cb9f740 1 -- start start 2024-05-09T05:44:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb9cb9f740 1 Processor -- start 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00024d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00025280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc771b0 0x558d0002aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558d000277b0 con 0x558cffc564d0 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558cfff069c0 con 0x558cffc771b0 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558d00027930 con 0x558cffc4e2f0 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb9cb9f740 10 client.0 _ll_register_callbacks cb 0x558cfff33020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb8effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00025280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb8effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00025280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44470/0 (socket says 172.21.15.115:44470) 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb8effd640 1 -- 172.21.15.115:0/345404619 learned_addr learned my addr 172.21.15.115:0/345404619 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.139+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00024d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb8ffff640 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc771b0 0x558d0002aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb8effd640 1 -- 172.21.15.115:0/345404619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 msgr2=0x558d00024d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb8effd640 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00024d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb8effd640 1 -- 172.21.15.115:0/345404619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc771b0 msgr2=0x558d0002aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb8effd640 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc771b0 0x558d0002aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb8effd640 1 -- 172.21.15.115:0/345404619 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfff2d1f0 con 0x558cffc564d0 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00024d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb8ffff640 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc771b0 0x558d0002aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb8effd640 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00025280 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7feb8400ba00 tx=0x7feb8400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45952c4df2cc0685 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb6effd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb6effd640 1 -- 172.21.15.115:0/345404619 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84011440 con 0x558cffc564d0 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb6effd640 1 -- 172.21.15.115:0/345404619 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7feb840115e0 con 0x558cffc564d0 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb9cb9f740 1 -- 172.21.15.115:0/345404619 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb800a9280 con 0x558cffc564d0 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb6effd640 1 -- 172.21.15.115:0/345404619 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7feb840117e0 con 0x558cffc564d0 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb9cb9f740 10 client.17136 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb9cb9f740 1 -- 172.21.15.115:0/345404619 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558cffce2790 con 0x558cffc564d0 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.140+0000 7feb6effd640 1 -- 172.21.15.115:0/345404619 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb800a9280 con 0x558cffc564d0 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.141+0000 7feb6effd640 1 -- 172.21.15.115:0/345404619 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7feb8401b870 con 0x558cffc564d0 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.141+0000 7feb9cb9f740 10 client.17136 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.141+0000 7feb9cb9f740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.141+0000 7feb9cb9f740 20 client.17136 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.141+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558cffc35d80 0x558d00036fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.141+0000 7feb9cb9f740 4 client.17136 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.141+0000 7feb9cb9f740 1 -- 172.21.15.115:0/345404619 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cfff8d3d0 con 0x558cffc35d80 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.141+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558cffc35d80 0x558d00036fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.142+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558cffc35d80 0x558d00036fe0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab935d580162d47 server_cookie=64219394203b13bc in_seq=0 out_seq=0 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.142+0000 7feb6effd640 10 client.17136 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.142+0000 7feb6effd640 1 -- 172.21.15.115:0/345404619 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558cfff8d3d0 con 0x558cffc35d80 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.142+0000 7feb6effd640 10 client.17136 handle_command_reply: tid=0 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.142+0000 7feb9cb9f740 1 client.17136 shutdown 2024-05-09T05:44:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.142+0000 7feb9cb9f740 1 -- 172.21.15.115:0/345404619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 msgr2=0x558d00025280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.142+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00025280 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7feb8400ba00 tx=0x7feb8400bec0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.142+0000 7feb9cb9f740 1 -- 172.21.15.115:0/345404619 shutdown_connections 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.142+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc771b0 0x558d0002aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.143+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00025280 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.143+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558cffc35d80 0x558d00036fe0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.143+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/345404619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00024d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.143+0000 7feb9cb9f740 1 -- 172.21.15.115:0/345404619 >> 172.21.15.115:0/345404619 conn(0x558cffc6b300 msgr2=0x558d00015700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.143+0000 7feb6effd640 0 client.17136 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.143+0000 7feb9cb9f740 1 -- 172.21.15.115:0/345404619 shutdown_connections 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.143+0000 7feb9cb9f740 1 -- 172.21.15.115:0/345404619 wait complete. 2024-05-09T05:44:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.143+0000 7feb9cb9f740 20 client.17136 trim_cache size 0 max 16384 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb9cb9f740 1 Processor -- start 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb9cb9f740 1 -- start start 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc6b300 0x558d0003c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0003cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 0x558d0002aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b10e0 con 0x558cffc4e2f0 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b2c20 con 0x558cffc564d0 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b17b0 con 0x558cffc6b300 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb8effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0003cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb8effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0003cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44480/0 (socket says 172.21.15.115:44480) 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb8effd640 1 -- 172.21.15.115:0/3969995440 learned_addr learned my addr 172.21.15.115:0/3969995440 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc6b300 0x558d0003c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.161+0000 7feb8ffff640 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 0x558d0002aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb8effd640 1 -- 172.21.15.115:0/3969995440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc6b300 msgr2=0x558d0003c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb8effd640 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc6b300 0x558d0003c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb8effd640 1 -- 172.21.15.115:0/3969995440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 msgr2=0x558d0002aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb8effd640 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 0x558d0002aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb8effd640 1 -- 172.21.15.115:0/3969995440 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cffd595f0 con 0x558cffc4e2f0 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb8ffff640 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 0x558d0002aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb8effd640 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0003cd00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7feb84006010 tx=0x7feb84003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d631a1af50325aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb757fa640 1 -- 172.21.15.115:0/3969995440 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84011440 con 0x558cffc4e2f0 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb757fa640 1 -- 172.21.15.115:0/3969995440 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb8402e070 con 0x558cffc4e2f0 2024-05-09T05:44:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.162+0000 7feb757fa640 1 -- 172.21.15.115:0/3969995440 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8401b9c0 con 0x558cffc4e2f0 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3969995440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 msgr2=0x558d0003cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0003cd00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7feb84006010 tx=0x7feb84003ee0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3969995440 shutdown_connections 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc564d0 0x558d0002aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc4e2f0 0x558d0003cd00 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/3969995440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc6b300 0x558d0003c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3969995440 >> 172.21.15.115:0/3969995440 conn(0x558d00037350 msgr2=0x558d00023400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3969995440 shutdown_connections 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 -- 172.21.15.115:0/3969995440 wait complete. 2024-05-09T05:44:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.163+0000 7feb9cb9f740 1 -- start start 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb9cb9f740 1 Processor -- start 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00030b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00031090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb9cb9f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc6b300 0x558d0002bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b2550 con 0x558cffc564d0 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb901b1e80 con 0x558cffc6b300 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb9cb9f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb901af0c0 con 0x558cffc4e2f0 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb9cb9f740 10 client.0 _ll_register_callbacks cb 0x7feb840041d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00031090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00030b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00031090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44494/0 (socket says 172.21.15.115:44494) 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8effd640 1 -- 172.21.15.115:0/568600957 learned_addr learned my addr 172.21.15.115:0/568600957 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8ffff640 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc6b300 0x558d0002bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8f7fe640 1 -- 172.21.15.115:0/568600957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc6b300 msgr2=0x558d0002bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc6b300 0x558d0002bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8f7fe640 1 -- 172.21.15.115:0/568600957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 msgr2=0x558d00031090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00031090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8f7fe640 1 -- 172.21.15.115:0/568600957 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfff43ed0 con 0x558cffc4e2f0 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8ffff640 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc6b300 0x558d0002bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.164+0000 7feb8effd640 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00031090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb8f7fe640 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00030b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb800a9470 tx=0x7feb800b2060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fa496a3beab94a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb6effd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb6effd640 1 -- 172.21.15.115:0/568600957 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb800a5630 con 0x558cffc4e2f0 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb6effd640 1 -- 172.21.15.115:0/568600957 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7feb800e2420 con 0x558cffc4e2f0 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb9cb9f740 1 -- 172.21.15.115:0/568600957 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8402e070 con 0x558cffc4e2f0 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb6effd640 1 -- 172.21.15.115:0/568600957 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7feb800e2620 con 0x558cffc4e2f0 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb9cb9f740 10 client.26629 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb9cb9f740 1 -- 172.21.15.115:0/568600957 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558d0009ecc0 con 0x558cffc4e2f0 2024-05-09T05:44:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb6effd640 1 -- 172.21.15.115:0/568600957 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb8402e070 con 0x558cffc4e2f0 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb6effd640 1 -- 172.21.15.115:0/568600957 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7feb80091140 con 0x558cffc4e2f0 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb9cb9f740 10 client.26629 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb9cb9f740 10 client.26629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb9cb9f740 20 client.26629 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558cffc771b0 0x558d000a1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb9cb9f740 4 client.26629 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.165+0000 7feb9cb9f740 1 -- 172.21.15.115:0/568600957 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558cfff87240 con 0x558cffc771b0 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.166+0000 7feb8effd640 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558cffc771b0 0x558d000a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.166+0000 7feb6effd640 10 client.26629 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.166+0000 7feb8effd640 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558cffc771b0 0x558d000a1030 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d01e5695afb6b1 server_cookie=88b4c2b903e74c3f in_seq=0 out_seq=0 2024-05-09T05:44:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.175+0000 7feb6effd640 1 -- 172.21.15.115:0/568600957 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558cfff87240 con 0x558cffc771b0 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.175+0000 7feb6effd640 10 client.26629 handle_command_reply: tid=0 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.175+0000 7feb9cb9f740 1 client.26629 shutdown 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.175+0000 7feb9cb9f740 1 -- 172.21.15.115:0/568600957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 msgr2=0x558d00030b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.175+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00030b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb800a9470 tx=0x7feb800b2060 comp rx=0 tx=0).stop 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb9cb9f740 1 -- 172.21.15.115:0/568600957 shutdown_connections 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cffc6b300 0x558d0002bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558cffc564d0 0x558d00031090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558cffc771b0 0x558d000a1030 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb9cb9f740 1 --2- 172.21.15.115:0/568600957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558cffc4e2f0 0x558d00030b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb6effd640 0 client.26629 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb9cb9f740 1 -- 172.21.15.115:0/568600957 >> 172.21.15.115:0/568600957 conn(0x558d00037350 msgr2=0x558d000233a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb9cb9f740 1 -- 172.21.15.115:0/568600957 shutdown_connections 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb9cb9f740 1 -- 172.21.15.115:0/568600957 wait complete. 2024-05-09T05:44:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb9cb9f740 20 client.26629 trim_cache size 0 max 16384 2024-05-09T05:44:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:44:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:44:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb95c13640 1 -- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feb780b29d0 msgr2=0x7feb780b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb95c13640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feb780b29d0 0x7feb780b4ee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feb88005cc0 tx=0x7feb8802a8c0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb95c13640 1 -- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 msgr2=0x7feb9019b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb95c13640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 0x7feb9019b650 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7feb8000c970 tx=0x7feb8000ce30 comp rx=0 tx=0).stop 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb95c13640 1 -- 172.21.15.115:0/3427339775 shutdown_connections 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb95c13640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feb780b29d0 0x7feb780b4ee0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.176+0000 7feb95c13640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb901090c0 0x7feb9019c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.177+0000 7feb95c13640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb90108c20 0x7feb9019bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.177+0000 7feb95c13640 1 --2- 172.21.15.115:0/3427339775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb901039d0 0x7feb9019b650 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.177+0000 7feb95c13640 1 -- 172.21.15.115:0/3427339775 >> 172.21.15.115:0/3427339775 conn(0x7feb900779e0 msgr2=0x7feb900ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.177+0000 7feb95c13640 1 -- 172.21.15.115:0/3427339775 shutdown_connections 2024-05-09T05:44:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:09.177+0000 7feb95c13640 1 -- 172.21.15.115:0/3427339775 wait complete. 2024-05-09T05:44:09.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:09 smithi195 ceph-mon[33844]: pgmap v181: 129 pgs: 129 active+clean; 4.4 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:44:09.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:09 smithi115 ceph-mon[30349]: pgmap v181: 129 pgs: 129 active+clean; 4.4 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:44:09.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:09 smithi161 ceph-mon[33805]: pgmap v181: 129 pgs: 129 active+clean; 4.4 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:44:09.808 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-09T05:44:09.808 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-09T05:44:09.809 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:44:10.103 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:11.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:11 smithi195 ceph-mon[33844]: pgmap v182: 129 pgs: 129 active+clean; 4.4 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 33 op/s 2024-05-09T05:44:11.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:11 smithi115 ceph-mon[30349]: pgmap v182: 129 pgs: 129 active+clean; 4.4 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 33 op/s 2024-05-09T05:44:11.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:11 smithi161 ceph-mon[33805]: pgmap v182: 129 pgs: 129 active+clean; 4.4 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 33 op/s 2024-05-09T05:44:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.019+0000 7f57a1e38700 1 -- 172.21.15.115:0/4289933840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1089f0 msgr2=0x7f579c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.019+0000 7f57a1e38700 1 --2- 172.21.15.115:0/4289933840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1089f0 0x7f579c108e00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f578c009990 tx=0x7f578c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.019+0000 7f57a1e38700 1 -- 172.21.15.115:0/4289933840 shutdown_connections 2024-05-09T05:44:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.020+0000 7f57a1e38700 1 --2- 172.21.15.115:0/4289933840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f579c103310 0x7f579c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.020+0000 7f57a1e38700 1 --2- 172.21.15.115:0/4289933840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1089f0 0x7f579c108e00 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.020+0000 7f57a1e38700 1 --2- 172.21.15.115:0/4289933840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f579c1029c0 0x7f579c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.020+0000 7f57a1e38700 1 -- 172.21.15.115:0/4289933840 >> 172.21.15.115:0/4289933840 conn(0x7f579c0fe520 msgr2=0x7f579c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.020+0000 7f57a1e38700 1 -- 172.21.15.115:0/4289933840 shutdown_connections 2024-05-09T05:44:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.020+0000 7f57a1e38700 1 -- 172.21.15.115:0/4289933840 wait complete. 2024-05-09T05:44:12.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.020+0000 7f57a1e38700 1 Processor -- start 2024-05-09T05:44:12.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f57a1e38700 1 -- start start 2024-05-09T05:44:12.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f57a1e38700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1029c0 0x7f579c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f57a1e38700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f579c103310 0x7f579c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f57a1e38700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f579c1089f0 0x7f579c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f57a1e38700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f579c112190 con 0x7f579c1029c0 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f579b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1029c0 0x7f579c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f57a1e38700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f579c112010 con 0x7f579c103310 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f57a1e38700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f579c112310 con 0x7f579c1089f0 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f579b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1029c0 0x7f579c19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44512/0 (socket says 172.21.15.115:44512) 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f579b7fe700 1 -- 172.21.15.115:0/3773857106 learned_addr learned my addr 172.21.15.115:0/3773857106 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.021+0000 7f579bfff700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f579c1089f0 0x7f579c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.022+0000 7f579affd700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f579c103310 0x7f579c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.022+0000 7f579b7fe700 1 -- 172.21.15.115:0/3773857106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f579c1089f0 msgr2=0x7f579c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.022+0000 7f579b7fe700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f579c1089f0 0x7f579c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.022+0000 7f579b7fe700 1 -- 172.21.15.115:0/3773857106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f579c103310 msgr2=0x7f579c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.022+0000 7f579b7fe700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f579c103310 0x7f579c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.022+0000 7f579b7fe700 1 -- 172.21.15.115:0/3773857106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c198fb0 con 0x7f579c1029c0 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.022+0000 7f579affd700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f579c103310 0x7f579c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.022+0000 7f579b7fe700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1029c0 0x7f579c19e750 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f578c000c00 tx=0x7f578c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.023+0000 7f5798ff9700 1 -- 172.21.15.115:0/3773857106 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c024070 con 0x7f579c1029c0 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.023+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c199240 con 0x7f579c1029c0 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.023+0000 7f5798ff9700 1 -- 172.21.15.115:0/3773857106 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f578c020440 con 0x7f579c1029c0 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.023+0000 7f5798ff9700 1 -- 172.21.15.115:0/3773857106 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c017650 con 0x7f579c1029c0 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.023+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c199770 con 0x7f579c1029c0 2024-05-09T05:44:12.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.025+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579c0512c0 con 0x7f579c1029c0 2024-05-09T05:44:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.026+0000 7f5798ff9700 1 -- 172.21.15.115:0/3773857106 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f578c027070 con 0x7f579c1029c0 2024-05-09T05:44:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.027+0000 7f5798ff9700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f57840b2a70 0x7f57840b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.027+0000 7f579affd700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f57840b2a70 0x7f57840b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.027+0000 7f5798ff9700 1 -- 172.21.15.115:0/3773857106 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f578c0e6200 con 0x7f579c1029c0 2024-05-09T05:44:12.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.027+0000 7f579affd700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f57840b2a70 0x7f57840b4f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5790005fd0 tx=0x7f579000a2b0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:12.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.032+0000 7f5798ff9700 1 -- 172.21.15.115:0/3773857106 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f578c013070 con 0x7f579c1029c0 2024-05-09T05:44:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:44:12.445 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:43:58.362294+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:44:12.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.451+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f579c199ea0 con 0x7f579c1029c0 2024-05-09T05:44:12.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.452+0000 7f5798ff9700 1 -- 172.21.15.115:0/3773857106 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11953 (secure 0 0 0) 0x7f578c0ad930 con 0x7f579c1029c0 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f57840b2a70 msgr2=0x7f57840b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f57840b2a70 0x7f57840b4f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5790005fd0 tx=0x7f579000a2b0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1029c0 msgr2=0x7f579c19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1029c0 0x7f579c19e750 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f578c000c00 tx=0x7f578c003680 comp rx=0 tx=0).stop 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 shutdown_connections 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f57840b2a70 0x7f57840b4f80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f579c1089f0 0x7f579c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c1029c0 0x7f579c19e750 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 --2- 172.21.15.115:0/3773857106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f579c103310 0x7f579c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 >> 172.21.15.115:0/3773857106 conn(0x7f579c0fe520 msgr2=0x7f579c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:12.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 shutdown_connections 2024-05-09T05:44:12.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:12.455+0000 7f57a1e38700 1 -- 172.21.15.115:0/3773857106 wait complete. 2024-05-09T05:44:12.449 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 38 2024-05-09T05:44:13.205 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-09T05:44:13.205 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-09T05:44:13.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:13 smithi195 ceph-mon[33844]: pgmap v183: 129 pgs: 129 active+clean; 4.4 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 33 op/s 2024-05-09T05:44:13.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:13 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3773857106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:13.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:13 smithi115 ceph-mon[30349]: pgmap v183: 129 pgs: 129 active+clean; 4.4 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 33 op/s 2024-05-09T05:44:13.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:13 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3773857106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:13.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:13 smithi161 ceph-mon[33805]: pgmap v183: 129 pgs: 129 active+clean; 4.4 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 33 op/s 2024-05-09T05:44:13.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:13 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3773857106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:13.496 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:15.125 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:15 smithi115 ceph-mon[30349]: pgmap v184: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:44:15.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:15 smithi195 ceph-mon[33844]: pgmap v184: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:44:15.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:15 smithi161 ceph-mon[33805]: pgmap v184: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 44 op/s 2024-05-09T05:44:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.528+0000 7fd220e9d700 1 -- 172.21.15.115:0/3003639369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 msgr2=0x7fd21c1102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.528+0000 7fd220e9d700 1 --2- 172.21.15.115:0/3003639369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 0x7fd21c1102d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd21400b0d0 tx=0x7fd21400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.528+0000 7fd220e9d700 1 -- 172.21.15.115:0/3003639369 shutdown_connections 2024-05-09T05:44:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.528+0000 7fd220e9d700 1 --2- 172.21.15.115:0/3003639369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 0x7fd21c1102d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.528+0000 7fd220e9d700 1 --2- 172.21.15.115:0/3003639369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21c103c10 0x7fd21c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.528+0000 7fd220e9d700 1 --2- 172.21.15.115:0/3003639369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd21c108020 0x7fd21c108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.528+0000 7fd220e9d700 1 -- 172.21.15.115:0/3003639369 >> 172.21.15.115:0/3003639369 conn(0x7fd21c0fe520 msgr2=0x7fd21c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.528+0000 7fd220e9d700 1 -- 172.21.15.115:0/3003639369 shutdown_connections 2024-05-09T05:44:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.528+0000 7fd220e9d700 1 -- 172.21.15.115:0/3003639369 wait complete. 2024-05-09T05:44:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.529+0000 7fd220e9d700 1 Processor -- start 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.529+0000 7fd220e9d700 1 -- start start 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.529+0000 7fd220e9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd21c103c10 0x7fd21c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd220e9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21c108020 0x7fd21c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd220e9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 0x7fd21c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd220e9d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c113450 con 0x7fd21c108970 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd220e9d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1132d0 con 0x7fd21c103c10 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd220e9d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1135d0 con 0x7fd21c108020 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 0x7fd21c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 0x7fd21c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47594/0 (socket says 172.21.15.115:47594) 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21ad9d700 1 -- 172.21.15.115:0/4089056542 learned_addr learned my addr 172.21.15.115:0/4089056542 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd21c103c10 0x7fd21c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd219d9b700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21c108020 0x7fd21c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21ad9d700 1 -- 172.21.15.115:0/4089056542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21c108020 msgr2=0x7fd21c19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21ad9d700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21c108020 0x7fd21c19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21ad9d700 1 -- 172.21.15.115:0/4089056542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd21c103c10 msgr2=0x7fd21c19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21ad9d700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd21c103c10 0x7fd21c19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21ad9d700 1 -- 172.21.15.115:0/4089056542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c19fd80 con 0x7fd21c108970 2024-05-09T05:44:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.530+0000 7fd21ad9d700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 0x7fd21c19f6e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd214009850 tx=0x7fd214007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.531+0000 7fd20b7fe700 1 -- 172.21.15.115:0/4089056542 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21401f070 con 0x7fd21c108970 2024-05-09T05:44:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.531+0000 7fd20b7fe700 1 -- 172.21.15.115:0/4089056542 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd214004c50 con 0x7fd21c108970 2024-05-09T05:44:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.531+0000 7fd220e9d700 1 -- 172.21.15.115:0/4089056542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c1a0010 con 0x7fd21c108970 2024-05-09T05:44:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.531+0000 7fd20b7fe700 1 -- 172.21.15.115:0/4089056542 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21401b690 con 0x7fd21c108970 2024-05-09T05:44:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.531+0000 7fd220e9d700 1 -- 172.21.15.115:0/4089056542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c1a04f0 con 0x7fd21c108970 2024-05-09T05:44:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.534+0000 7fd20b7fe700 1 -- 172.21.15.115:0/4089056542 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fd214022070 con 0x7fd21c108970 2024-05-09T05:44:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.535+0000 7fd20b7fe700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd2040b29f0 0x7fd2040b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.535+0000 7fd22f8b3b80 1 Processor -- start 2024-05-09T05:44:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.535+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd2040b29f0 0x7fd2040b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.535+0000 7fd20b7fe700 1 -- 172.21.15.115:0/4089056542 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd21400f020 con 0x7fd21c108970 2024-05-09T05:44:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.535+0000 7fd22f8b3b80 1 -- start start 2024-05-09T05:44:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd2040b29f0 0x7fd2040b4f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009c60 tx=0x7fd20c009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8cebe90 0x555ce8dedce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8dee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8df8990 0x555ce8dfadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555ce8d11230 con 0x555ce8c71630 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555ce8cb6b30 con 0x555ce8df8990 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555ce8d0d1f0 con 0x555ce8cebe90 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd219d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8dee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd219d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8dee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47600/0 (socket says 172.21.15.115:47600) 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd219d9b700 1 -- 172.21.15.115:0/388880082 learned_addr learned my addr 172.21.15.115:0/388880082 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd21a59c700 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8cebe90 0x555ce8dedce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd21ad9d700 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8df8990 0x555ce8dfadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd219d9b700 1 -- 172.21.15.115:0/388880082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8cebe90 msgr2=0x555ce8dedce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd219d9b700 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8cebe90 0x555ce8dedce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd219d9b700 1 -- 172.21.15.115:0/388880082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8df8990 msgr2=0x555ce8dfadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd219d9b700 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8df8990 0x555ce8dfadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd219d9b700 1 -- 172.21.15.115:0/388880082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce8ca7690 con 0x555ce8c71630 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.536+0000 7fd21ad9d700 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8df8990 0x555ce8dfadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.537+0000 7fd219d9b700 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8dee220 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd21000bc70 tx=0x7fd21000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99dfa447ebe30e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.537+0000 7fd208ff9700 1 -- 172.21.15.115:0/388880082 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210004230 con 0x555ce8c71630 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.537+0000 7fd208ff9700 1 -- 172.21.15.115:0/388880082 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd210010db0 con 0x555ce8c71630 2024-05-09T05:44:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.537+0000 7fd208ff9700 1 -- 172.21.15.115:0/388880082 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210005020 con 0x555ce8c71630 2024-05-09T05:44:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/388880082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 msgr2=0x555ce8dee220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8dee220 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd21000bc70 tx=0x7fd21000bf80 comp rx=0 tx=0).stop 2024-05-09T05:44:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/388880082 shutdown_connections 2024-05-09T05:44:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8df8990 0x555ce8dfadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8cebe90 0x555ce8dedce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/388880082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8dee220 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/388880082 >> 172.21.15.115:0/388880082 conn(0x555ce8c75100 msgr2=0x555ce8de9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/388880082 shutdown_connections 2024-05-09T05:44:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/388880082 wait complete. 2024-05-09T05:44:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.538+0000 7fd22f8b3b80 1 -- start start 2024-05-09T05:44:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd22f8b3b80 1 Processor -- start 2024-05-09T05:44:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8df5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 0x555ce8df5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 0x555ce8df63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555ce8dfb3b0 con 0x555ce8c71630 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555ce8cabcb0 con 0x555ce8cebe90 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555ce8dfb530 con 0x555ce8df8990 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd22f8b3b80 10 client.0 _ll_register_callbacks cb 0x555ce8b82850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd21a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8df5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd21a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8df5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47616/0 (socket says 172.21.15.115:47616) 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd21a59c700 1 -- 172.21.15.115:0/4078577438 learned_addr learned my addr 172.21.15.115:0/4078577438 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd219d9b700 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 0x555ce8df5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.539+0000 7fd21ad9d700 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 0x555ce8df63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd21a59c700 1 -- 172.21.15.115:0/4078577438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 msgr2=0x555ce8df63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 0x555ce8df63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd21a59c700 1 -- 172.21.15.115:0/4078577438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 msgr2=0x555ce8df5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 0x555ce8df5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd21a59c700 1 -- 172.21.15.115:0/4078577438 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce8b8c620 con 0x555ce8c71630 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd21ad9d700 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 0x555ce8df63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8df5940 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd20c003900 tx=0x7fd20c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1d8b7beaeb84c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd201ffb700 1 -- 172.21.15.115:0/4078577438 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c01d620 con 0x555ce8c71630 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4078577438 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd210010db0 con 0x555ce8c71630 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd201ffb700 1 -- 172.21.15.115:0/4078577438 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd20c01d7c0 con 0x555ce8c71630 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd201ffb700 1 -- 172.21.15.115:0/4078577438 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd20c01d9c0 con 0x555ce8c71630 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd22f8b3b80 10 client.17160 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.540+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4078577438 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555ce8e06060 con 0x555ce8c71630 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.541+0000 7fd201ffb700 1 -- 172.21.15.115:0/4078577438 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd210010db0 con 0x555ce8c71630 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.541+0000 7fd201ffb700 1 -- 172.21.15.115:0/4078577438 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fd20c027c80 con 0x555ce8c71630 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.541+0000 7fd22f8b3b80 10 client.17160 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.541+0000 7fd22f8b3b80 10 client.17160 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:44:15.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.541+0000 7fd22f8b3b80 20 client.17160 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.541+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555ce8e08860 0x555ce8e0ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.541+0000 7fd22f8b3b80 4 client.17160 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.541+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4078577438 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ce8d1ec10 con 0x555ce8e08860 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.541+0000 7fd219d9b700 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555ce8e08860 0x555ce8e0ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.542+0000 7fd201ffb700 10 client.17160 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.542+0000 7fd219d9b700 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555ce8e08860 0x555ce8e0ad30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b467a7b230c99757 server_cookie=201af4de9319480c in_seq=0 out_seq=0 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd201ffb700 1 -- 172.21.15.115:0/4078577438 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ce8d1ec10 con 0x555ce8e08860 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd201ffb700 10 client.17160 handle_command_reply: tid=0 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 client.17160 shutdown 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4078577438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 msgr2=0x555ce8df5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8df5940 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd20c003900 tx=0x7fd20c0047c0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4078577438 shutdown_connections 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555ce8e08860 0x555ce8e0ad30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 0x555ce8df63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8c71630 0x555ce8df5940 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4078577438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 0x555ce8df5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4078577438 >> 172.21.15.115:0/4078577438 conn(0x555ce8c75100 msgr2=0x555ce8de8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd201ffb700 0 client.17160 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4078577438 shutdown_connections 2024-05-09T05:44:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4078577438 wait complete. 2024-05-09T05:44:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.543+0000 7fd22f8b3b80 20 client.17160 trim_cache size 0 max 16384 2024-05-09T05:44:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd22f8b3b80 1 Processor -- start 2024-05-09T05:44:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd22f8b3b80 1 -- start start 2024-05-09T05:44:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8c71630 0x555ce8df5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 0x555ce8df7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8df8990 0x555ce8e04470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1b0260 con 0x555ce8df8990 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1b1da0 con 0x555ce8cebe90 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1b0930 con 0x555ce8c71630 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd21ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8df8990 0x555ce8e04470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd21ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8df8990 0x555ce8e04470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47624/0 (socket says 172.21.15.115:47624) 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd21ad9d700 1 -- 172.21.15.115:0/2394362388 learned_addr learned my addr 172.21.15.115:0/2394362388 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd21a59c700 1 --2- 172.21.15.115:0/2394362388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8c71630 0x555ce8df5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.573+0000 7fd219d9b700 1 --2- 172.21.15.115:0/2394362388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 0x555ce8df7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.574+0000 7fd21a59c700 1 -- 172.21.15.115:0/2394362388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 msgr2=0x555ce8df7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.574+0000 7fd21a59c700 1 --2- 172.21.15.115:0/2394362388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 0x555ce8df7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.574+0000 7fd21a59c700 1 -- 172.21.15.115:0/2394362388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8df8990 msgr2=0x555ce8e04470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.574+0000 7fd21a59c700 1 --2- 172.21.15.115:0/2394362388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8df8990 0x555ce8e04470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.574+0000 7fd21a59c700 1 -- 172.21.15.115:0/2394362388 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce8df3eb0 con 0x555ce8c71630 2024-05-09T05:44:15.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.574+0000 7fd21a59c700 1 --2- 172.21.15.115:0/2394362388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8c71630 0x555ce8df5f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd20c000c00 tx=0x7fd20c003fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd84fedc76dfe48a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.574+0000 7fd203fff700 1 -- 172.21.15.115:0/2394362388 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c03b070 con 0x555ce8c71630 2024-05-09T05:44:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.574+0000 7fd203fff700 1 -- 172.21.15.115:0/2394362388 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd20c01d6e0 con 0x555ce8c71630 2024-05-09T05:44:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.574+0000 7fd203fff700 1 -- 172.21.15.115:0/2394362388 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c037750 con 0x555ce8c71630 2024-05-09T05:44:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.575+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/2394362388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8c71630 msgr2=0x555ce8df5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.575+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/2394362388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8c71630 0x555ce8df5f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd20c000c00 tx=0x7fd20c003fa0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.575+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/2394362388 shutdown_connections 2024-05-09T05:44:15.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.575+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/2394362388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8df8990 0x555ce8e04470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.575+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/2394362388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8c71630 0x555ce8df5f70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.575+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/2394362388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8cebe90 0x555ce8df7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.575+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/2394362388 >> 172.21.15.115:0/2394362388 conn(0x555ce8c75100 msgr2=0x555ce8de8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:15.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.575+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/2394362388 shutdown_connections 2024-05-09T05:44:15.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.575+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/2394362388 wait complete. 2024-05-09T05:44:15.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.576+0000 7fd22f8b3b80 1 -- start start 2024-05-09T05:44:15.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd22f8b3b80 1 Processor -- start 2024-05-09T05:44:15.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8c71630 0x555ce8dfd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8cebe90 0x555ce8dfdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd22f8b3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 0x555ce8e72f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1b16d0 con 0x555ce8cebe90 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1b1000 con 0x555ce8c71630 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd22f8b3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1ae200 con 0x555ce8df8990 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd22f8b3b80 10 client.0 _ll_register_callbacks cb 0x555ce8b82850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd219d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8cebe90 0x555ce8dfdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd219d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8cebe90 0x555ce8dfdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47638/0 (socket says 172.21.15.115:47638) 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.577+0000 7fd219d9b700 1 -- 172.21.15.115:0/4204611556 learned_addr learned my addr 172.21.15.115:0/4204611556 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd219d9b700 1 -- 172.21.15.115:0/4204611556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 msgr2=0x555ce8e72f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd219d9b700 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 0x555ce8e72f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd219d9b700 1 -- 172.21.15.115:0/4204611556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8c71630 msgr2=0x555ce8dfd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8c71630 0x555ce8dfd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd219d9b700 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8c71630 0x555ce8dfd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd219d9b700 1 -- 172.21.15.115:0/4204611556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce8dfe420 con 0x555ce8cebe90 2024-05-09T05:44:15.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8c71630 0x555ce8dfd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd219d9b700 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8cebe90 0x555ce8dfdc00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd21000bc70 tx=0x7fd210010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467e777017ae2548 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd201ffb700 1 -- 172.21.15.115:0/4204611556 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21002d020 con 0x555ce8cebe90 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd201ffb700 1 -- 172.21.15.115:0/4204611556 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fd210003880 con 0x555ce8cebe90 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.578+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4204611556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd20c01d6e0 con 0x555ce8cebe90 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd201ffb700 1 -- 172.21.15.115:0/4204611556 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd210038050 con 0x555ce8cebe90 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd22f8b3b80 10 client.17166 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4204611556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555ce8e743f0 con 0x555ce8cebe90 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd201ffb700 1 -- 172.21.15.115:0/4204611556 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd20c01d6e0 con 0x555ce8cebe90 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd201ffb700 1 -- 172.21.15.115:0/4204611556 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fd2100046c0 con 0x555ce8cebe90 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd22f8b3b80 10 client.17166 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd22f8b3b80 10 client.17166 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd22f8b3b80 20 client.17166 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:15.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555ce8e74890 0x555ce8e76cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:15.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd22f8b3b80 4 client.17166 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-09T05:44:15.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4204611556 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x555ce8d18cb0 con 0x555ce8e74890 2024-05-09T05:44:15.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.579+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555ce8e74890 0x555ce8e76cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:15.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.580+0000 7fd201ffb700 10 client.17166 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:15.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.580+0000 7fd21a59c700 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555ce8e74890 0x555ce8e76cb0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af78b7f5ff0ad04a server_cookie=76684ff99b17999 in_seq=0 out_seq=0 2024-05-09T05:44:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd201ffb700 1 -- 172.21.15.115:0/4204611556 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555ce8d18cb0 con 0x555ce8e74890 2024-05-09T05:44:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd201ffb700 10 client.17166 handle_command_reply: tid=0 2024-05-09T05:44:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 client.17166 shutdown 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4204611556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8cebe90 msgr2=0x555ce8dfdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8cebe90 0x555ce8dfdc00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd21000bc70 tx=0x7fd210010b90 comp rx=0 tx=0).stop 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4204611556 shutdown_connections 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ce8c71630 0x555ce8dfd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555ce8e74890 0x555ce8e76cb0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ce8df8990 0x555ce8e72f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 --2- 172.21.15.115:0/4204611556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555ce8cebe90 0x555ce8dfdc00 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4204611556 >> 172.21.15.115:0/4204611556 conn(0x555ce8c75100 msgr2=0x555ce8de8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd201ffb700 0 client.17166 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4204611556 shutdown_connections 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.658+0000 7fd22f8b3b80 1 -- 172.21.15.115:0/4204611556 wait complete. 2024-05-09T05:44:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd22f8b3b80 20 client.17166 trim_cache size 0 max 16384 2024-05-09T05:44:15.654 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:44:15.654 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 38, 2024-05-09T05:44:15.654 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:44:15.654 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 18, 2024-05-09T05:44:15.654 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:44:15.654 INFO:teuthology.orchestra.run.smithi115.stdout: "c2030c2b": { 2024-05-09T05:44:15.654 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 18, 2024-05-09T05:44:15.654 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:44:15.654 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:15.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:15.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:15.656 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:44:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 -- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd2040b29f0 msgr2=0x7fd2040b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd2040b29f0 0x7fd2040b4f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009c60 tx=0x7fd20c009260 comp rx=0 tx=0).stop 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 -- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 msgr2=0x7fd21c19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 0x7fd21c19f6e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd214009850 tx=0x7fd214007a20 comp rx=0 tx=0).stop 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 -- 172.21.15.115:0/4089056542 shutdown_connections 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd2040b29f0 0x7fd2040b4f00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd21c103c10 0x7fd21c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd21c108970 0x7fd21c19f6e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 --2- 172.21.15.115:0/4089056542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21c108020 0x7fd21c19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 -- 172.21.15.115:0/4089056542 >> 172.21.15.115:0/4089056542 conn(0x7fd21c0fe520 msgr2=0x7fd21c102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 -- 172.21.15.115:0/4089056542 shutdown_connections 2024-05-09T05:44:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:15.659+0000 7fd220e9d700 1 -- 172.21.15.115:0/4089056542 wait complete. 2024-05-09T05:44:16.420 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c2030c2b', quiesce took 6.6 seconds. Will release after: 23.4 2024-05-09T05:44:17.155 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:44:17.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:17 smithi195 ceph-mon[33844]: pgmap v185: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:44:17.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:17 smithi115 ceph-mon[30349]: pgmap v185: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:44:17.408 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:17.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:17 smithi161 ceph-mon[33805]: pgmap v185: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:44:19.273 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:19 smithi115 ceph-mon[30349]: pgmap v186: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 28 op/s 2024-05-09T05:44:19.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:19 smithi195 ceph-mon[33844]: pgmap v186: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 28 op/s 2024-05-09T05:44:19.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.398+0000 7f2d43337700 1 -- 172.21.15.115:0/1177688171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c102b40 msgr2=0x7f2d3c102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:19.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.398+0000 7f2d43337700 1 --2- 172.21.15.115:0/1177688171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c102b40 0x7f2d3c102f50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009990 tx=0x7f2d30009ca0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.398+0000 7f2d43337700 1 -- 172.21.15.115:0/1177688171 shutdown_connections 2024-05-09T05:44:19.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.398+0000 7f2d43337700 1 --2- 172.21.15.115:0/1177688171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d3c1098b0 0x7f2d3c1111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.398+0000 7f2d43337700 1 --2- 172.21.15.115:0/1177688171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c102b40 0x7f2d3c102f50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.398+0000 7f2d43337700 1 --2- 172.21.15.115:0/1177688171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d3c108f60 0x7f2d3c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.398+0000 7f2d43337700 1 -- 172.21.15.115:0/1177688171 >> 172.21.15.115:0/1177688171 conn(0x7f2d3c0fe520 msgr2=0x7f2d3c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:19.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.398+0000 7f2d43337700 1 -- 172.21.15.115:0/1177688171 shutdown_connections 2024-05-09T05:44:19.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.398+0000 7f2d43337700 1 -- 172.21.15.115:0/1177688171 wait complete. 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d43337700 1 Processor -- start 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d43337700 1 -- start start 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d43337700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d3c102b40 0x7f2d3c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d43337700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d3c108f60 0x7f2d3c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d43337700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c1098b0 0x7f2d3c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d43337700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c114350 con 0x7f2d3c1098b0 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d43337700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1141d0 con 0x7f2d3c108f60 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d43337700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1144d0 con 0x7f2d3c102b40 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d418d4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c1098b0 0x7f2d3c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d418d4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c1098b0 0x7f2d3c1a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47660/0 (socket says 172.21.15.115:47660) 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d418d4700 1 -- 172.21.15.115:0/821463494 learned_addr learned my addr 172.21.15.115:0/821463494 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d410d3700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d3c102b40 0x7f2d3c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d408d2700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d3c108f60 0x7f2d3c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d418d4700 1 -- 172.21.15.115:0/821463494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d3c102b40 msgr2=0x7f2d3c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d418d4700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d3c102b40 0x7f2d3c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d418d4700 1 -- 172.21.15.115:0/821463494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d3c108f60 msgr2=0x7f2d3c19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d418d4700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d3c108f60 0x7f2d3c19d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.399+0000 7f2d418d4700 1 -- 172.21.15.115:0/821463494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c1a1df0 con 0x7f2d3c1098b0 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.400+0000 7f2d408d2700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d3c108f60 0x7f2d3c19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.400+0000 7f2d418d4700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c1098b0 0x7f2d3c1a1750 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400ef30 tx=0x7f2d3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.400+0000 7f2d2e7fc700 1 -- 172.21.15.115:0/821463494 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d340092e0 con 0x7f2d3c1098b0 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.400+0000 7f2d43337700 1 -- 172.21.15.115:0/821463494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3c1a2080 con 0x7f2d3c1098b0 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.400+0000 7f2d2e7fc700 1 -- 172.21.15.115:0/821463494 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d34004bb0 con 0x7f2d3c1098b0 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.400+0000 7f2d2e7fc700 1 -- 172.21.15.115:0/821463494 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34004020 con 0x7f2d3c1098b0 2024-05-09T05:44:19.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.400+0000 7f2d43337700 1 -- 172.21.15.115:0/821463494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3c1a2570 con 0x7f2d3c1098b0 2024-05-09T05:44:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.403+0000 7f2d2e7fc700 1 -- 172.21.15.115:0/821463494 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f2d3401b030 con 0x7f2d3c1098b0 2024-05-09T05:44:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.403+0000 7f2d2e7fc700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d280b29f0 0x7f2d280b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d410d3700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d280b29f0 0x7f2d280b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d2e7fc700 1 -- 172.21.15.115:0/821463494 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2d340e2500 con 0x7f2d3c1098b0 2024-05-09T05:44:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d51d4db80 1 Processor -- start 2024-05-09T05:44:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d51d4db80 1 -- start start 2024-05-09T05:44:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d410d3700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d280b29f0 0x7f2d280b4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b2e0 tx=0x7f2d30005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:19.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d7ac6160 0x5596d7b67cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d79dc4a0 0x5596d7b681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5596d7a7bff0 con 0x5596d7ac6160 2024-05-09T05:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5596d7a1a7f0 con 0x5596d7b61eb0 2024-05-09T05:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5596d7a77fb0 con 0x5596d79dc4a0 2024-05-09T05:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d410d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d7ac6160 0x5596d7b67cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d410d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d7ac6160 0x5596d7b67cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47668/0 (socket says 172.21.15.115:47668) 2024-05-09T05:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.404+0000 7f2d410d3700 1 -- 172.21.15.115:0/536018919 learned_addr learned my addr 172.21.15.115:0/536018919 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d408d2700 1 --2- 172.21.15.115:0/536018919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d79dc4a0 0x5596d7b681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d418d4700 1 --2- 172.21.15.115:0/536018919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d410d3700 1 -- 172.21.15.115:0/536018919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d79dc4a0 msgr2=0x5596d7b681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d410d3700 1 --2- 172.21.15.115:0/536018919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d79dc4a0 0x5596d7b681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d410d3700 1 -- 172.21.15.115:0/536018919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 msgr2=0x5596d7b622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d410d3700 1 --2- 172.21.15.115:0/536018919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d410d3700 1 -- 172.21.15.115:0/536018919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d7a12370 con 0x5596d7ac6160 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d410d3700 1 --2- 172.21.15.115:0/536018919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d7ac6160 0x5596d7b67cb0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2d30015000 tx=0x7f2d3001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29975e8a7b22e17f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d277fe700 1 -- 172.21.15.115:0/536018919 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30003680 con 0x5596d7ac6160 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d277fe700 1 -- 172.21.15.115:0/536018919 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d30003cb0 con 0x5596d7ac6160 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.405+0000 7f2d277fe700 1 -- 172.21.15.115:0/536018919 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000fab0 con 0x5596d7ac6160 2024-05-09T05:44:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.406+0000 7f2d51d4db80 1 -- 172.21.15.115:0/536018919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d7ac6160 msgr2=0x5596d7b67cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.406+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/536018919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d7ac6160 0x5596d7b67cb0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2d30015000 tx=0x7f2d3001e000 comp rx=0 tx=0).stop 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.406+0000 7f2d51d4db80 1 -- 172.21.15.115:0/536018919 shutdown_connections 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.406+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/536018919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.406+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/536018919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d7ac6160 0x5596d7b67cb0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.406+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/536018919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d79dc4a0 0x5596d7b681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.406+0000 7f2d51d4db80 1 -- 172.21.15.115:0/536018919 >> 172.21.15.115:0/536018919 conn(0x5596d7a3c6a0 msgr2=0x5596d7b54400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.406+0000 7f2d51d4db80 1 -- 172.21.15.115:0/536018919 shutdown_connections 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.406+0000 7f2d51d4db80 1 -- 172.21.15.115:0/536018919 wait complete. 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d51d4db80 1 -- start start 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d51d4db80 1 Processor -- start 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 0x5596d7b6bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b6c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5596d78f7ba0 con 0x5596d79dc4a0 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5596d7a36150 con 0x5596d7b61eb0 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5596d7b62c30 con 0x5596d7ac6160 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d51d4db80 10 client.0 _ll_register_callbacks cb 0x5596d78ed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d410d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d410d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47672/0 (socket says 172.21.15.115:47672) 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d410d3700 1 -- 172.21.15.115:0/414852009 learned_addr learned my addr 172.21.15.115:0/414852009 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.407+0000 7f2d408d2700 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 0x5596d7b6bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d418d4700 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b6c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d408d2700 1 -- 172.21.15.115:0/414852009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 msgr2=0x5596d7b6c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d408d2700 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b6c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d408d2700 1 -- 172.21.15.115:0/414852009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 msgr2=0x5596d7b6b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d408d2700 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d408d2700 1 -- 172.21.15.115:0/414852009 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d78f7490 con 0x5596d7ac6160 2024-05-09T05:44:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d418d4700 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b6c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d410d3700 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d408d2700 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 0x5596d7b6bdb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c06ca00 tx=0x7f2d3800bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c0c2d7e56e454e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d24ff9700 1 -- 172.21.15.115:0/414852009 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3801b440 con 0x5596d7ac6160 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d51d4db80 1 -- 172.21.15.115:0/414852009 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d30003cb0 con 0x5596d7ac6160 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d24ff9700 1 -- 172.21.15.115:0/414852009 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2d3801b5e0 con 0x5596d7ac6160 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d24ff9700 1 -- 172.21.15.115:0/414852009 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2d38024050 con 0x5596d7ac6160 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d51d4db80 10 client.26659 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.408+0000 7f2d51d4db80 1 -- 172.21.15.115:0/414852009 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5596d7b79240 con 0x5596d7ac6160 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.409+0000 7f2d24ff9700 1 -- 172.21.15.115:0/414852009 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d30003cb0 con 0x5596d7ac6160 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.409+0000 7f2d24ff9700 1 -- 172.21.15.115:0/414852009 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f2d38019020 con 0x5596d7ac6160 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.409+0000 7f2d51d4db80 10 client.26659 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.409+0000 7f2d51d4db80 10 client.26659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:19.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.409+0000 7f2d51d4db80 20 client.26659 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:19.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.409+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5596d7b7b880 0x5596d7b7dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.409+0000 7f2d51d4db80 4 client.26659 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:19.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.409+0000 7f2d51d4db80 1 -- 172.21.15.115:0/414852009 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596d7a899d0 con 0x5596d7b7b880 2024-05-09T05:44:19.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.409+0000 7f2d410d3700 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5596d7b7b880 0x5596d7b7dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.410+0000 7f2d410d3700 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5596d7b7b880 0x5596d7b7dca0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28855763f27d09c server_cookie=bfda3f4aa37c7eaa in_seq=0 out_seq=0 2024-05-09T05:44:19.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.410+0000 7f2d24ff9700 10 client.26659 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:19.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.410+0000 7f2d24ff9700 1 -- 172.21.15.115:0/414852009 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596d7a899d0 con 0x5596d7b7b880 2024-05-09T05:44:19.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d24ff9700 10 client.26659 handle_command_reply: tid=0 2024-05-09T05:44:19.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 client.26659 shutdown 2024-05-09T05:44:19.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 -- 172.21.15.115:0/414852009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 msgr2=0x5596d7b6bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:19.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 0x5596d7b6bdb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c06ca00 tx=0x7f2d3800bd80 comp rx=0 tx=0).stop 2024-05-09T05:44:19.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 -- 172.21.15.115:0/414852009 shutdown_connections 2024-05-09T05:44:19.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b6c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5596d7b7b880 0x5596d7b7dca0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/414852009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 0x5596d7b6bdb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d24ff9700 0 client.26659 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:19.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 -- 172.21.15.115:0/414852009 >> 172.21.15.115:0/414852009 conn(0x5596d7a3c6a0 msgr2=0x5596d7b541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:19.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 -- 172.21.15.115:0/414852009 shutdown_connections 2024-05-09T05:44:19.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 1 -- 172.21.15.115:0/414852009 wait complete. 2024-05-09T05:44:19.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.411+0000 7f2d51d4db80 20 client.26659 trim_cache size 0 max 16384 2024-05-09T05:44:19.412 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:19 smithi161 ceph-mon[33805]: pgmap v186: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 28 op/s 2024-05-09T05:44:19.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d51d4db80 1 Processor -- start 2024-05-09T05:44:19.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d51d4db80 1 -- start start 2024-05-09T05:44:19.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 0x5596d7b6c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b77770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1ae000 con 0x5596d79dc4a0 2024-05-09T05:44:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1afb40 con 0x5596d7b61eb0 2024-05-09T05:44:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1ae6d0 con 0x5596d7ac6160 2024-05-09T05:44:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d410d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d410d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47688/0 (socket says 172.21.15.115:47688) 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d410d3700 1 -- 172.21.15.115:0/837668718 learned_addr learned my addr 172.21.15.115:0/837668718 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d418d4700 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b77770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.431+0000 7f2d408d2700 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 0x5596d7b6c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d410d3700 1 -- 172.21.15.115:0/837668718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 msgr2=0x5596d7b6c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d410d3700 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 0x5596d7b6c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d410d3700 1 -- 172.21.15.115:0/837668718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 msgr2=0x5596d7b77770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d410d3700 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b77770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d410d3700 1 -- 172.21.15.115:0/837668718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d7b553d0 con 0x5596d79dc4a0 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d418d4700 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b77770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d410d3700 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6d950 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2d300050a0 tx=0x7f2d3001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cce7c11bd517a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d26ffd700 1 -- 172.21.15.115:0/837668718 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000fa40 con 0x5596d79dc4a0 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d26ffd700 1 -- 172.21.15.115:0/837668718 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d30003b10 con 0x5596d79dc4a0 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.432+0000 7f2d26ffd700 1 -- 172.21.15.115:0/837668718 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3003c520 con 0x5596d79dc4a0 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 -- 172.21.15.115:0/837668718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 msgr2=0x5596d7b6d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6d950 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2d300050a0 tx=0x7f2d3001a040 comp rx=0 tx=0).stop 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 -- 172.21.15.115:0/837668718 shutdown_connections 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7b61eb0 0x5596d7b77770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b6d950 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/837668718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7ac6160 0x5596d7b6c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 -- 172.21.15.115:0/837668718 >> 172.21.15.115:0/837668718 conn(0x5596d7a3c6a0 msgr2=0x5596d7b58d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 -- 172.21.15.115:0/837668718 shutdown_connections 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 -- 172.21.15.115:0/837668718 wait complete. 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 -- start start 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.433+0000 7f2d51d4db80 1 Processor -- start 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b68640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7ac6160 0x5596d7b68b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d51d4db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7b61eb0 0x5596d7b690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1af470 con 0x5596d79dc4a0 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1aeda0 con 0x5596d7ac6160 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d51d4db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1abfc0 con 0x5596d7b61eb0 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d51d4db80 10 client.0 _ll_register_callbacks cb 0x5596d78ed6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d410d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b68640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d410d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b68640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47694/0 (socket says 172.21.15.115:47694) 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d410d3700 1 -- 172.21.15.115:0/2197860387 learned_addr learned my addr 172.21.15.115:0/2197860387 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d418d4700 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7b61eb0 0x5596d7b690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d408d2700 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7ac6160 0x5596d7b68b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d410d3700 1 -- 172.21.15.115:0/2197860387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7b61eb0 msgr2=0x5596d7b690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d410d3700 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7b61eb0 0x5596d7b690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d410d3700 1 -- 172.21.15.115:0/2197860387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7ac6160 msgr2=0x5596d7b68b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d410d3700 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7ac6160 0x5596d7b68b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d410d3700 1 -- 172.21.15.115:0/2197860387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d7b71d70 con 0x5596d79dc4a0 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d410d3700 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b68640 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b6d0 tx=0x7f2d30003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc97eee4c331eb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.434+0000 7f2d24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d24ff9700 1 -- 172.21.15.115:0/2197860387 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000fb40 con 0x5596d79dc4a0 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d24ff9700 1 -- 172.21.15.115:0/2197860387 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2d30003890 con 0x5596d79dc4a0 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d51d4db80 1 -- 172.21.15.115:0/2197860387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d30003b10 con 0x5596d79dc4a0 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d24ff9700 1 -- 172.21.15.115:0/2197860387 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2d3004a650 con 0x5596d79dc4a0 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d51d4db80 10 client.17184 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d51d4db80 1 -- 172.21.15.115:0/2197860387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5596d7be6110 con 0x5596d79dc4a0 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d24ff9700 1 -- 172.21.15.115:0/2197860387 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d30003b10 con 0x5596d79dc4a0 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d24ff9700 1 -- 172.21.15.115:0/2197860387 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f2d30022070 con 0x5596d79dc4a0 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d51d4db80 10 client.17184 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d51d4db80 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d51d4db80 20 client.17184 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5596d7be6390 0x5596d7be87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d51d4db80 4 client.17184 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d51d4db80 1 -- 172.21.15.115:0/2197860387 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596d7a83a70 con 0x5596d7be6390 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.435+0000 7f2d408d2700 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5596d7be6390 0x5596d7be87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d408d2700 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5596d7be6390 0x5596d7be87b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33027a840369adc7 server_cookie=34f8ba8b175bc6d0 in_seq=0 out_seq=0 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d24ff9700 10 client.17184 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d24ff9700 1 -- 172.21.15.115:0/2197860387 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5596d7a83a70 con 0x5596d7be6390 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d24ff9700 10 client.17184 handle_command_reply: tid=0 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 client.17184 shutdown 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 -- 172.21.15.115:0/2197860387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 msgr2=0x5596d7b68640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b68640 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b6d0 tx=0x7f2d30003d70 comp rx=0 tx=0).stop 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 -- 172.21.15.115:0/2197860387 shutdown_connections 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5596d7be6390 0x5596d7be87b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5596d7b61eb0 0x5596d7b690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5596d79dc4a0 0x5596d7b68640 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d24ff9700 0 client.17184 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 --2- 172.21.15.115:0/2197860387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596d7ac6160 0x5596d7b68b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 -- 172.21.15.115:0/2197860387 >> 172.21.15.115:0/2197860387 conn(0x5596d7a3c6a0 msgr2=0x5596d7b65600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 -- 172.21.15.115:0/2197860387 shutdown_connections 2024-05-09T05:44:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.436+0000 7f2d51d4db80 1 -- 172.21.15.115:0/2197860387 wait complete. 2024-05-09T05:44:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d51d4db80 20 client.17184 trim_cache size 0 max 16384 2024-05-09T05:44:19.443 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:44:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:44:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:44:19.444 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:44:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 -- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d280b29f0 msgr2=0x7f2d280b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d280b29f0 0x7f2d280b4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b2e0 tx=0x7f2d30005fb0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 -- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c1098b0 msgr2=0x7f2d3c1a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c1098b0 0x7f2d3c1a1750 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400ef30 tx=0x7f2d3400c3a0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 -- 172.21.15.115:0/821463494 shutdown_connections 2024-05-09T05:44:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2d280b29f0 0x7f2d280b4f00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d3c1098b0 0x7f2d3c1a1750 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d3c102b40 0x7f2d3c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 --2- 172.21.15.115:0/821463494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d3c108f60 0x7f2d3c19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 -- 172.21.15.115:0/821463494 >> 172.21.15.115:0/821463494 conn(0x7f2d3c0fe520 msgr2=0x7f2d3c110500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 -- 172.21.15.115:0/821463494 shutdown_connections 2024-05-09T05:44:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:19.437+0000 7f2d43337700 1 -- 172.21.15.115:0/821463494 wait complete. 2024-05-09T05:44:20.325 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:44:20.325 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:44:20.568 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:21.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:21 smithi195 ceph-mon[33844]: pgmap v187: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 18 op/s 2024-05-09T05:44:21.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:21 smithi115 ceph-mon[30349]: pgmap v187: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 18 op/s 2024-05-09T05:44:21.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:21 smithi161 ceph-mon[33805]: pgmap v187: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 18 op/s 2024-05-09T05:44:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.277+0000 7fa82d3b7640 1 -- 172.21.15.115:0/70342661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 msgr2=0x7fa828103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.277+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/70342661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 0x7fa828103d90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa810009900 tx=0x7fa81002a410 comp rx=0 tx=0).stop 2024-05-09T05:44:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.277+0000 7fa82d3b7640 1 -- 172.21.15.115:0/70342661 shutdown_connections 2024-05-09T05:44:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.277+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/70342661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa828104400 0x7fa8281107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.277+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/70342661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 0x7fa828103d90 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.277+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/70342661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa828109cb0 0x7fa828103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.277+0000 7fa82d3b7640 1 -- 172.21.15.115:0/70342661 >> 172.21.15.115:0/70342661 conn(0x7fa8280793a0 msgr2=0x7fa8280797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.277+0000 7fa82d3b7640 1 -- 172.21.15.115:0/70342661 shutdown_connections 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.277+0000 7fa82d3b7640 1 -- 172.21.15.115:0/70342661 wait complete. 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.278+0000 7fa82d3b7640 1 Processor -- start 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.278+0000 7fa82d3b7640 1 -- start start 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa82d3b7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa828104400 0x7fa8281a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa82d3b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa828109cb0 0x7fa8281a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa82d3b7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 0x7fa82819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa82d3b7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa828113d50 con 0x7fa82810a150 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa82d3b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa828113bd0 con 0x7fa828109cb0 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa82d3b7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa828113ed0 con 0x7fa828104400 2024-05-09T05:44:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 0x7fa82819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 0x7fa82819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47708/0 (socket says 172.21.15.115:47708) 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa8277fe640 1 -- 172.21.15.115:0/490728144 learned_addr learned my addr 172.21.15.115:0/490728144 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa828109cb0 0x7fa8281a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa826ffd640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa828104400 0x7fa8281a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa8277fe640 1 -- 172.21.15.115:0/490728144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa828104400 msgr2=0x7fa8281a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa8277fe640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa828104400 0x7fa8281a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa8277fe640 1 -- 172.21.15.115:0/490728144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa828109cb0 msgr2=0x7fa8281a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa8277fe640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa828109cb0 0x7fa8281a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa8277fe640 1 -- 172.21.15.115:0/490728144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82819ad30 con 0x7fa82810a150 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa826ffd640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa828104400 0x7fa8281a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa828109cb0 0x7fa8281a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa8277fe640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 0x7fa82819a6f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa82000cce0 tx=0x7fa82000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa824ff9640 1 -- 172.21.15.115:0/490728144 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82000f450 con 0x7fa82810a150 2024-05-09T05:44:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa82d3b7640 1 -- 172.21.15.115:0/490728144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82819b020 con 0x7fa82810a150 2024-05-09T05:44:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa824ff9640 1 -- 172.21.15.115:0/490728144 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa82000fa80 con 0x7fa82810a150 2024-05-09T05:44:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa824ff9640 1 -- 172.21.15.115:0/490728144 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820018ad0 con 0x7fa82810a150 2024-05-09T05:44:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.279+0000 7fa82d3b7640 1 -- 172.21.15.115:0/490728144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82819b520 con 0x7fa82810a150 2024-05-09T05:44:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.282+0000 7fa83426d740 1 Processor -- start 2024-05-09T05:44:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa824ff9640 1 -- 172.21.15.115:0/490728144 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa8200213f0 con 0x7fa82810a150 2024-05-09T05:44:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa83426d740 1 -- start start 2024-05-09T05:44:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa83426d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f63a071b0 0x563f63daa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa83426d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 0x563f63daabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa83426d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639e64d0 0x563f63dab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa83426d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f63d0f7c0 con 0x563f639e64d0 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa83426d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563f63cab0e0 con 0x563f63a071b0 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa83426d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563f63d0b780 con 0x563f639de2f0 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa81e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639e64d0 0x563f63dab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa81e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639e64d0 0x563f63dab100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47710/0 (socket says 172.21.15.115:47710) 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa81e7fc640 1 -- 172.21.15.115:0/1618710823 learned_addr learned my addr 172.21.15.115:0/1618710823 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.283+0000 7fa826ffd640 1 --2- 172.21.15.115:0/1618710823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 0x563f63daabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/1618710823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f63a071b0 0x563f63daa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa824ff9640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa8140b2950 0x7fa8140b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81e7fc640 1 -- 172.21.15.115:0/1618710823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 msgr2=0x563f63daabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/1618710823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 0x563f63daabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81e7fc640 1 -- 172.21.15.115:0/1618710823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f63a071b0 msgr2=0x563f63daa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/1618710823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f63a071b0 0x563f63daa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81e7fc640 1 -- 172.21.15.115:0/1618710823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f63cd47c0 con 0x563f639e64d0 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa8277fe640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa8140b2950 0x7fa8140b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/1618710823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639e64d0 0x563f63dab100 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa810005ab0 tx=0x7fa810009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a33b2fe8791422 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa824ff9640 1 -- 172.21.15.115:0/490728144 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa8200e3220 con 0x7fa82810a150 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81d7fa640 1 -- 172.21.15.115:0/1618710823 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810045070 con 0x563f639e64d0 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81d7fa640 1 -- 172.21.15.115:0/1618710823 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8100044c0 con 0x563f639e64d0 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.284+0000 7fa81d7fa640 1 -- 172.21.15.115:0/1618710823 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810035050 con 0x563f639e64d0 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa83426d740 1 -- 172.21.15.115:0/1618710823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639e64d0 msgr2=0x563f63dab100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa8277fe640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa8140b2950 0x7fa8140b4e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa8200e0c30 tx=0x7fa820013000 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa83426d740 1 --2- 172.21.15.115:0/1618710823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639e64d0 0x563f63dab100 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa810005ab0 tx=0x7fa810009560 comp rx=0 tx=0).stop 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa83426d740 1 -- 172.21.15.115:0/1618710823 shutdown_connections 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa83426d740 1 --2- 172.21.15.115:0/1618710823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639e64d0 0x563f63dab100 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa83426d740 1 --2- 172.21.15.115:0/1618710823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 0x563f63daabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa83426d740 1 --2- 172.21.15.115:0/1618710823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f63a071b0 0x563f63daa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa83426d740 1 -- 172.21.15.115:0/1618710823 >> 172.21.15.115:0/1618710823 conn(0x563f639fb300 msgr2=0x563f63da6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa83426d740 1 -- 172.21.15.115:0/1618710823 shutdown_connections 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.285+0000 7fa83426d740 1 -- 172.21.15.115:0/1618710823 wait complete. 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa83426d740 1 -- start start 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa83426d740 1 Processor -- start 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa83426d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 0x563f63db6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa83426d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63db6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa83426d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63db4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa83426d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563f63db7750 con 0x563f63a071b0 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa83426d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563f63c969c0 con 0x563f639e64d0 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa83426d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563f63db78d0 con 0x563f639de2f0 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa83426d740 10 client.0 _ll_register_callbacks cb 0x563f63cc3020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa81e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63db4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa81e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63db4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47722/0 (socket says 172.21.15.115:47722) 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa81e7fc640 1 -- 172.21.15.115:0/2952849921 learned_addr learned my addr 172.21.15.115:0/2952849921 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 0x563f63db6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa826ffd640 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63db6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa81e7fc640 1 -- 172.21.15.115:0/2952849921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 msgr2=0x563f63db6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 0x563f63db6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa81e7fc640 1 -- 172.21.15.115:0/2952849921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 msgr2=0x563f63db6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63db6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa81e7fc640 1 -- 172.21.15.115:0/2952849921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f63cbd1f0 con 0x563f63a071b0 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.286+0000 7fa826ffd640 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63db6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63db4c80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa810043040 tx=0x7fa810009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614e43cf2d5432e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa80e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa80e7fc640 1 -- 172.21.15.115:0/2952849921 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810035070 con 0x563f63a071b0 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa80e7fc640 1 -- 172.21.15.115:0/2952849921 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa8100048c0 con 0x563f63a071b0 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa83426d740 1 -- 172.21.15.115:0/2952849921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f63d1d3d0 con 0x563f63a071b0 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa80e7fc640 1 -- 172.21.15.115:0/2952849921 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa810045040 con 0x563f63a071b0 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa83426d740 10 client.17196 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa83426d740 1 -- 172.21.15.115:0/2952849921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563f63a72790 con 0x563f63a071b0 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa80e7fc640 1 -- 172.21.15.115:0/2952849921 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x563f63d1d3d0 con 0x563f63a071b0 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa80e7fc640 1 -- 172.21.15.115:0/2952849921 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fa81004b040 con 0x563f63a071b0 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa83426d740 10 client.17196 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa83426d740 10 client.17196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa83426d740 20 client.17196 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa83426d740 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563f639c5d80 0x563f63dc6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa83426d740 4 client.17196 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.287+0000 7fa83426d740 1 -- 172.21.15.115:0/2952849921 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f63d17240 con 0x563f639c5d80 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.288+0000 7fa826ffd640 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563f639c5d80 0x563f63dc6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.288+0000 7fa80e7fc640 10 client.17196 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.288+0000 7fa826ffd640 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563f639c5d80 0x563f63dc6e10 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1bb6c06a878470f server_cookie=c646bfb1848e99ae in_seq=0 out_seq=0 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa80e7fc640 1 -- 172.21.15.115:0/2952849921 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f63d17240 con 0x563f639c5d80 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa80e7fc640 10 client.17196 handle_command_reply: tid=0 2024-05-09T05:44:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 client.17196 shutdown 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 -- 172.21.15.115:0/2952849921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 msgr2=0x563f63db4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63db4c80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa810043040 tx=0x7fa810009260 comp rx=0 tx=0).stop 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 -- 172.21.15.115:0/2952849921 shutdown_connections 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63db4c80 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63db6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563f639c5d80 0x563f63dc6e10 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 --2- 172.21.15.115:0/2952849921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639de2f0 0x563f63db6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa80e7fc640 0 client.17196 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 -- 172.21.15.115:0/2952849921 >> 172.21.15.115:0/2952849921 conn(0x563f639fb300 msgr2=0x563f63da5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 -- 172.21.15.115:0/2952849921 shutdown_connections 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 1 -- 172.21.15.115:0/2952849921 wait complete. 2024-05-09T05:44:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.289+0000 7fa83426d740 20 client.17196 trim_cache size 0 max 16384 2024-05-09T05:44:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa83426d740 1 Processor -- start 2024-05-09T05:44:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa83426d740 1 -- start start 2024-05-09T05:44:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa83426d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639de2f0 0x563f63dab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa83426d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63daca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa83426d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f63a071b0 0x563f63dae400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa83426d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281b0fa0 con 0x563f639de2f0 2024-05-09T05:44:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa83426d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281b2ae0 con 0x563f639e64d0 2024-05-09T05:44:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa83426d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281b1670 con 0x563f63a071b0 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa81e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639de2f0 0x563f63dab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa81e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639de2f0 0x563f63dab420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47726/0 (socket says 172.21.15.115:47726) 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.304+0000 7fa81e7fc640 1 -- 172.21.15.115:0/3519993860 learned_addr learned my addr 172.21.15.115:0/3519993860 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa81e7fc640 1 -- 172.21.15.115:0/3519993860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f63a071b0 msgr2=0x563f63dae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/3519993860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f63a071b0 0x563f63dae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa826ffd640 1 --2- 172.21.15.115:0/3519993860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63daca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa81e7fc640 1 -- 172.21.15.115:0/3519993860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 msgr2=0x563f63daca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/3519993860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63daca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa81e7fc640 1 -- 172.21.15.115:0/3519993860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f63ae95f0 con 0x563f639de2f0 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa826ffd640 1 --2- 172.21.15.115:0/3519993860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63daca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/3519993860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639de2f0 0x563f63dab420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa810000c00 tx=0x7fa810009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11cb7780432c743 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa81cff9640 1 -- 172.21.15.115:0/3519993860 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810035040 con 0x563f639de2f0 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa81cff9640 1 -- 172.21.15.115:0/3519993860 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa810005310 con 0x563f639de2f0 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.305+0000 7fa81cff9640 1 -- 172.21.15.115:0/3519993860 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8100046c0 con 0x563f639de2f0 2024-05-09T05:44:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 -- 172.21.15.115:0/3519993860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639de2f0 msgr2=0x563f63dab420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 --2- 172.21.15.115:0/3519993860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639de2f0 0x563f63dab420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa810000c00 tx=0x7fa810009520 comp rx=0 tx=0).stop 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 -- 172.21.15.115:0/3519993860 shutdown_connections 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 --2- 172.21.15.115:0/3519993860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f63a071b0 0x563f63dae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 --2- 172.21.15.115:0/3519993860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639e64d0 0x563f63daca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 --2- 172.21.15.115:0/3519993860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f639de2f0 0x563f63dab420 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 -- 172.21.15.115:0/3519993860 >> 172.21.15.115:0/3519993860 conn(0x563f639fb300 msgr2=0x563f63dac330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 -- 172.21.15.115:0/3519993860 shutdown_connections 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 -- 172.21.15.115:0/3519993860 wait complete. 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 -- start start 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.306+0000 7fa83426d740 1 Processor -- start 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa83426d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639de2f0 0x563f63dc7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa83426d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639e64d0 0x563f63dc7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa83426d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63e26f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa83426d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281b2410 con 0x563f63a071b0 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa83426d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281b1d40 con 0x563f639de2f0 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa83426d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281aef80 con 0x563f639e64d0 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa83426d740 10 client.0 _ll_register_callbacks cb 0x7fa810043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63e26f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63e26f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47728/0 (socket says 172.21.15.115:47728) 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 -- 172.21.15.115:0/3527402559 learned_addr learned my addr 172.21.15.115:0/3527402559 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639de2f0 0x563f63dc7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa826ffd640 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639e64d0 0x563f63dc7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 -- 172.21.15.115:0/3527402559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639e64d0 msgr2=0x563f63dc7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639e64d0 0x563f63dc7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 -- 172.21.15.115:0/3527402559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639de2f0 msgr2=0x563f63dc7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639de2f0 0x563f63dc7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 -- 172.21.15.115:0/3527402559 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f63cd3ed0 con 0x563f63a071b0 2024-05-09T05:44:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa826ffd640 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639e64d0 0x563f63dc7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa81e7fc640 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63e26f70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa8100315f0 tx=0x7fa810009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0927d05405f5ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa80e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa80e7fc640 1 -- 172.21.15.115:0/3527402559 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810035040 con 0x563f63a071b0 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa80e7fc640 1 -- 172.21.15.115:0/3527402559 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fa810031bb0 con 0x563f63a071b0 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.307+0000 7fa83426d740 1 -- 172.21.15.115:0/3527402559 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa810005310 con 0x563f63a071b0 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa80e7fc640 1 -- 172.21.15.115:0/3527402559 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa8100046c0 con 0x563f63a071b0 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa83426d740 10 client.17202 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa83426d740 1 -- 172.21.15.115:0/3527402559 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563f63e2e560 con 0x563f63a071b0 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa80e7fc640 1 -- 172.21.15.115:0/3527402559 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa810005310 con 0x563f63a071b0 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa80e7fc640 1 -- 172.21.15.115:0/3527402559 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fa810059030 con 0x563f63a071b0 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa83426d740 10 client.17202 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa83426d740 10 client.17202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa83426d740 20 client.17202 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa83426d740 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563f6399b1c0 0x563f63e30a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa83426d740 4 client.17202 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:44:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.308+0000 7fa83426d740 1 -- 172.21.15.115:0/3527402559 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f63c92350 con 0x563f6399b1c0 2024-05-09T05:44:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.309+0000 7fa826ffd640 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563f6399b1c0 0x563f63e30a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.309+0000 7fa80e7fc640 10 client.17202 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.309+0000 7fa826ffd640 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563f6399b1c0 0x563f63e30a50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9b21f5fd3c76eb server_cookie=5889e95baa48ea03 in_seq=0 out_seq=0 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.318+0000 7fa80e7fc640 1 -- 172.21.15.115:0/3527402559 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563f63c92350 con 0x563f6399b1c0 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.318+0000 7fa80e7fc640 10 client.17202 handle_command_reply: tid=0 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.318+0000 7fa83426d740 1 client.17202 shutdown 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.318+0000 7fa83426d740 1 -- 172.21.15.115:0/3527402559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 msgr2=0x563f63e26f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.318+0000 7fa83426d740 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63e26f70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa8100315f0 tx=0x7fa810009260 comp rx=0 tx=0).stop 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa83426d740 1 -- 172.21.15.115:0/3527402559 shutdown_connections 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa83426d740 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563f63a071b0 0x563f63e26f70 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa83426d740 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563f6399b1c0 0x563f63e30a50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa83426d740 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f639e64d0 0x563f63dc7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa80e7fc640 0 client.17202 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa83426d740 1 --2- 172.21.15.115:0/3527402559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f639de2f0 0x563f63dc7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa83426d740 1 -- 172.21.15.115:0/3527402559 >> 172.21.15.115:0/3527402559 conn(0x563f639fb300 msgr2=0x563f63daa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa83426d740 1 -- 172.21.15.115:0/3527402559 shutdown_connections 2024-05-09T05:44:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa83426d740 1 -- 172.21.15.115:0/3527402559 wait complete. 2024-05-09T05:44:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa83426d740 20 client.17202 trim_cache size 0 max 16384 2024-05-09T05:44:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:44:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:44:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:44:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa82d3b7640 1 -- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa8140b2950 msgr2=0x7fa8140b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.319+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa8140b2950 0x7fa8140b4e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa8200e0c30 tx=0x7fa820013000 comp rx=0 tx=0).stop 2024-05-09T05:44:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 -- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 msgr2=0x7fa82819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 0x7fa82819a6f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa82000cce0 tx=0x7fa82000a620 comp rx=0 tx=0).stop 2024-05-09T05:44:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 -- 172.21.15.115:0/490728144 shutdown_connections 2024-05-09T05:44:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa8140b2950 0x7fa8140b4e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa82810a150 0x7fa82819a6f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa828109cb0 0x7fa8281a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 --2- 172.21.15.115:0/490728144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa828104400 0x7fa8281a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 -- 172.21.15.115:0/490728144 >> 172.21.15.115:0/490728144 conn(0x7fa8280793a0 msgr2=0x7fa8281023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 -- 172.21.15.115:0/490728144 shutdown_connections 2024-05-09T05:44:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:22.320+0000 7fa82d3b7640 1 -- 172.21.15.115:0/490728144 wait complete. 2024-05-09T05:44:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.428+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2952395493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec108740 msgr2=0x7fbcec1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.428+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2952395493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec108740 0x7fbcec1100a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbce400b0d0 tx=0x7fbce400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.428+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2952395493 shutdown_connections 2024-05-09T05:44:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.428+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2952395493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcec1034c0 0x7fbcec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.428+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2952395493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec108740 0x7fbcec1100a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.428+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2952395493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcec107df0 0x7fbcec108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.428+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2952395493 >> 172.21.15.115:0/2952395493 conn(0x7fbcec0fedc0 msgr2=0x7fbcec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.428+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2952395493 shutdown_connections 2024-05-09T05:44:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.428+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2952395493 wait complete. 2024-05-09T05:44:22.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.429+0000 7fbcf1b99700 1 Processor -- start 2024-05-09T05:44:22.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.429+0000 7fbcf1b99700 1 -- start start 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbcf1b99700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec1034c0 0x7fbcec19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbcf1b99700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcec107df0 0x7fbcec19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbcf1b99700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcec108740 0x7fbcec1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbcf1b99700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec113220 con 0x7fbcec1034c0 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbcf1b99700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1130a0 con 0x7fbcec107df0 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbcf1b99700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1133a0 con 0x7fbcec108740 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec1034c0 0x7fbcec19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceb7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec1034c0 0x7fbcec19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47738/0 (socket says 172.21.15.115:47738) 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceb7fe700 1 -- 172.21.15.115:0/2929324006 learned_addr learned my addr 172.21.15.115:0/2929324006 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceaffd700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcec107df0 0x7fbcec19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbcebfff700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcec108740 0x7fbcec1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceb7fe700 1 -- 172.21.15.115:0/2929324006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcec108740 msgr2=0x7fbcec1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceb7fe700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcec108740 0x7fbcec1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceb7fe700 1 -- 172.21.15.115:0/2929324006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcec107df0 msgr2=0x7fbcec19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceb7fe700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcec107df0 0x7fbcec19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceb7fe700 1 -- 172.21.15.115:0/2929324006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec19a020 con 0x7fbcec1034c0 2024-05-09T05:44:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbcebfff700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcec108740 0x7fbcec1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbceb7fe700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec1034c0 0x7fbcec19f7c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c9b0 tx=0x7fbcdc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbce8ff9700 1 -- 172.21.15.115:0/2929324006 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc007b00 con 0x7fbcec1034c0 2024-05-09T05:44:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbce8ff9700 1 -- 172.21.15.115:0/2929324006 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcdc007ca0 con 0x7fbcec1034c0 2024-05-09T05:44:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec19a310 con 0x7fbcec1034c0 2024-05-09T05:44:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.430+0000 7fbce8ff9700 1 -- 172.21.15.115:0/2929324006 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc018660 con 0x7fbcec1034c0 2024-05-09T05:44:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.431+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec06b610 con 0x7fbcec1034c0 2024-05-09T05:44:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.431+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcec104970 con 0x7fbcec1034c0 2024-05-09T05:44:22.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.433+0000 7fbce8ff9700 1 -- 172.21.15.115:0/2929324006 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fbcdc00f450 con 0x7fbcec1034c0 2024-05-09T05:44:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.434+0000 7fbce8ff9700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbcd40b29f0 0x7fbcd40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.434+0000 7fbce8ff9700 1 -- 172.21.15.115:0/2929324006 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbcdc0e2c00 con 0x7fbcec1034c0 2024-05-09T05:44:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.434+0000 7fbceaffd700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbcd40b29f0 0x7fbcd40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.435+0000 7fbceaffd700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbcd40b29f0 0x7fbcd40b4f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbcec102360 tx=0x7fbce0009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:22.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.437+0000 7fbce8ff9700 1 -- 172.21.15.115:0/2929324006 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcdc014030 con 0x7fbcec1034c0 2024-05-09T05:44:22.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.685+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcec0512c0 con 0x7fbcec1034c0 2024-05-09T05:44:22.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.686+0000 7fbce8ff9700 1 -- 172.21.15.115:0/2929324006 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11953 (secure 0 0 0) 0x7fbcdc0aa330 con 0x7fbcec1034c0 2024-05-09T05:44:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:44:22.679 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:43:58.362294+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:44:22.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbcd40b29f0 msgr2=0x7fbcd40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbcd40b29f0 0x7fbcd40b4f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbcec102360 tx=0x7fbce0009260 comp rx=0 tx=0).stop 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec1034c0 msgr2=0x7fbcec19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec1034c0 0x7fbcec19f7c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c9b0 tx=0x7fbcdc00ccc0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 shutdown_connections 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbcd40b29f0 0x7fbcd40b4f00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbcec108740 0x7fbcec1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcec1034c0 0x7fbcec19f7c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 --2- 172.21.15.115:0/2929324006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcec107df0 0x7fbcec19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 >> 172.21.15.115:0/2929324006 conn(0x7fbcec0fedc0 msgr2=0x7fbcec107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 shutdown_connections 2024-05-09T05:44:22.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:22.688+0000 7fbcf1b99700 1 -- 172.21.15.115:0/2929324006 wait complete. 2024-05-09T05:44:22.683 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 38 2024-05-09T05:44:23.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:23 smithi195 ceph-mon[33844]: pgmap v188: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 18 op/s 2024-05-09T05:44:23.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:23 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2929324006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:23.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:23 smithi115 ceph-mon[30349]: pgmap v188: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 18 op/s 2024-05-09T05:44:23.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:23 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2929324006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:23.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:23 smithi161 ceph-mon[33805]: pgmap v188: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 18 op/s 2024-05-09T05:44:23.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:23 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2929324006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:23.487 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:44:23.737 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:25.221 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:25 smithi115 ceph-mon[30349]: pgmap v189: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 25 op/s 2024-05-09T05:44:25.364 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:25 smithi195 ceph-mon[33844]: pgmap v189: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 25 op/s 2024-05-09T05:44:25.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:25 smithi161 ceph-mon[33805]: pgmap v189: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 25 op/s 2024-05-09T05:44:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.493+0000 7f3e54ec8700 1 -- 172.21.15.115:0/629984645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50103550 msgr2=0x7f3e50103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.493+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/629984645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50103550 0x7f3e50103960 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3e440099c0 tx=0x7f3e44009cd0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.493+0000 7f3e54ec8700 1 -- 172.21.15.115:0/629984645 shutdown_connections 2024-05-09T05:44:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.493+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/629984645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e50108780 0x7f3e501100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.493+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/629984645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50103550 0x7f3e50103960 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.493+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/629984645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50107e30 0x7f3e50108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.493+0000 7f3e54ec8700 1 -- 172.21.15.115:0/629984645 >> 172.21.15.115:0/629984645 conn(0x7f3e50078e00 msgr2=0x7f3e50079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:25.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.493+0000 7f3e54ec8700 1 -- 172.21.15.115:0/629984645 shutdown_connections 2024-05-09T05:44:25.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.493+0000 7f3e54ec8700 1 -- 172.21.15.115:0/629984645 wait complete. 2024-05-09T05:44:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e54ec8700 1 Processor -- start 2024-05-09T05:44:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e54ec8700 1 -- start start 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e54ec8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50103550 0x7f3e5019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e54ec8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50107e30 0x7f3e5019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e54ec8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e50108780 0x7f3e5019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e54ec8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50113260 con 0x7f3e50107e30 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e54ec8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501130e0 con 0x7f3e50103550 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e54ec8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501133e0 con 0x7f3e50108780 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e4dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50107e30 0x7f3e5019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e4dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50107e30 0x7f3e5019b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57364/0 (socket says 172.21.15.115:57364) 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/1511636158 learned_addr learned my addr 172.21.15.115:0/1511636158 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50103550 0x7f3e5019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.494+0000 7f3e4ed9d700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e50108780 0x7f3e5019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/1511636158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e50108780 msgr2=0x7f3e5019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e50108780 0x7f3e5019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/1511636158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50103550 msgr2=0x7f3e5019ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50103550 0x7f3e5019ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/1511636158 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5019fbf0 con 0x7f3e50107e30 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50107e30 0x7f3e5019b2c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000d950 tx=0x7f3e4000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e3f7fe700 1 -- 172.21.15.115:0/1511636158 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40016070 con 0x7f3e50107e30 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e54ec8700 1 -- 172.21.15.115:0/1511636158 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5019fee0 con 0x7f3e50107e30 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e3f7fe700 1 -- 172.21.15.115:0/1511636158 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e40004830 con 0x7f3e50107e30 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e3f7fe700 1 -- 172.21.15.115:0/1511636158 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40005420 con 0x7f3e50107e30 2024-05-09T05:44:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.495+0000 7f3e54ec8700 1 -- 172.21.15.115:0/1511636158 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e501a7800 con 0x7f3e50107e30 2024-05-09T05:44:25.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.498+0000 7f3e3f7fe700 1 -- 172.21.15.115:0/1511636158 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f3e4000f690 con 0x7f3e50107e30 2024-05-09T05:44:25.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.498+0000 7f3e3f7fe700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3e380b29f0 0x7f3e380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.498+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3e380b29f0 0x7f3e380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.498+0000 7f3e3f7fe700 1 -- 172.21.15.115:0/1511636158 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f3e400e2690 con 0x7f3e50107e30 2024-05-09T05:44:25.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e638deb80 1 Processor -- start 2024-05-09T05:44:25.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3e380b29f0 0x7f3e380b4f00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e44000c00 tx=0x7f3e44005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:25.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e638deb80 1 -- start start 2024-05-09T05:44:25.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 0x564819769240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819769780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819763400 0x564819763810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e638deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5648196860a0 con 0x5648195e64a0 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e638deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56481962b9a0 con 0x564819763400 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e638deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564819682060 con 0x564819660d00 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e4dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819769780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e4dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819769780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57366/0 (socket says 172.21.15.115:57366) 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.499+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/177291638 learned_addr learned my addr 172.21.15.115:0/177291638 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/177291638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 0x564819769240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e4ed9d700 1 --2- 172.21.15.115:0/177291638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819763400 0x564819763810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/177291638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 msgr2=0x564819769240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:25.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/177291638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 0x564819769240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/177291638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819763400 msgr2=0x564819763810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:25.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/177291638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819763400 0x564819763810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/177291638 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481961c500 con 0x5648195e64a0 2024-05-09T05:44:25.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/177291638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819769780 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3e400a95c0 tx=0x7f3e400a8eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936835efefbbca1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:25.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e3cff9700 1 -- 172.21.15.115:0/177291638 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e400b2490 con 0x5648195e64a0 2024-05-09T05:44:25.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e3cff9700 1 -- 172.21.15.115:0/177291638 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e400a58c0 con 0x5648195e64a0 2024-05-09T05:44:25.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.500+0000 7f3e3cff9700 1 -- 172.21.15.115:0/177291638 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e400a9890 con 0x5648195e64a0 2024-05-09T05:44:25.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.501+0000 7f3e638deb80 1 -- 172.21.15.115:0/177291638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 msgr2=0x564819769780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:25.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.501+0000 7f3e638deb80 1 --2- 172.21.15.115:0/177291638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819769780 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3e400a95c0 tx=0x7f3e400a8eb0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.501+0000 7f3e638deb80 1 -- 172.21.15.115:0/177291638 shutdown_connections 2024-05-09T05:44:25.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.501+0000 7f3e638deb80 1 --2- 172.21.15.115:0/177291638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819763400 0x564819763810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.501+0000 7f3e638deb80 1 --2- 172.21.15.115:0/177291638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 0x564819769240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.501+0000 7f3e638deb80 1 --2- 172.21.15.115:0/177291638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819769780 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.501+0000 7f3e638deb80 1 -- 172.21.15.115:0/177291638 >> 172.21.15.115:0/177291638 conn(0x5648195e9f70 msgr2=0x56481975e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:25.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.501+0000 7f3e638deb80 1 -- 172.21.15.115:0/177291638 shutdown_connections 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.501+0000 7f3e638deb80 1 -- 172.21.15.115:0/177291638 wait complete. 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e638deb80 1 -- start start 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e638deb80 1 Processor -- start 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 0x56481976d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 0x56481976d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819763400 0x56481976dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e638deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564819764160 con 0x564819763400 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e638deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564819620b20 con 0x5648195e64a0 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e638deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564819770020 con 0x564819660d00 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e638deb80 10 client.0 _ll_register_callbacks cb 0x5648194f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e4ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819763400 0x56481976dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e4ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819763400 0x56481976dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57370/0 (socket says 172.21.15.115:57370) 2024-05-09T05:44:25.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e4ed9d700 1 -- 172.21.15.115:0/3693335286 learned_addr learned my addr 172.21.15.115:0/3693335286 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 0x56481976d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.502+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 0x56481976d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e4ed9d700 1 -- 172.21.15.115:0/3693335286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 msgr2=0x56481976d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e4ed9d700 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 0x56481976d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e4ed9d700 1 -- 172.21.15.115:0/3693335286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 msgr2=0x56481976d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e4ed9d700 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 0x56481976d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e4ed9d700 1 -- 172.21.15.115:0/3693335286 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564819501490 con 0x564819763400 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e4ed9d700 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819763400 0x56481976dac0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3e480042a0 tx=0x7f3e48004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eaa556f26f53240 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3693335286 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4801a070 con 0x564819763400 2024-05-09T05:44:25.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3693335286 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f3e48004db0 con 0x564819763400 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e638deb80 1 -- 172.21.15.115:0/3693335286 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e400a58c0 con 0x564819763400 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3693335286 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3e48025050 con 0x564819763400 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e638deb80 10 client.17220 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e638deb80 1 -- 172.21.15.115:0/3693335286 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56481977aa70 con 0x564819763400 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3693335286 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e400a58c0 con 0x564819763400 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3693335286 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3e480166d0 con 0x564819763400 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.503+0000 7f3e638deb80 10 client.17220 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.504+0000 7f3e638deb80 10 client.17220 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.504+0000 7f3e638deb80 20 client.17220 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.504+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x56481977d050 0x56481977f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.504+0000 7f3e638deb80 4 client.17220 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:25.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.504+0000 7f3e638deb80 1 -- 172.21.15.115:0/3693335286 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564819693a80 con 0x56481977d050 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.504+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x56481977d050 0x56481977f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.504+0000 7f3e35ffb700 10 client.17220 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.504+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x56481977d050 0x56481977f4d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3240bc8ba770ea4 server_cookie=1c819f74921155b7 in_seq=0 out_seq=0 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3693335286 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564819693a80 con 0x56481977d050 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e35ffb700 10 client.17220 handle_command_reply: tid=0 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 client.17220 shutdown 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 -- 172.21.15.115:0/3693335286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819763400 msgr2=0x56481976dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819763400 0x56481976dac0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3e480042a0 tx=0x7f3e48004380 comp rx=0 tx=0).stop 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 -- 172.21.15.115:0/3693335286 shutdown_connections 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 0x56481976d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x56481977d050 0x56481977f4d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819763400 0x56481976dac0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3693335286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819660d00 0x56481976d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 -- 172.21.15.115:0/3693335286 >> 172.21.15.115:0/3693335286 conn(0x5648195e9f70 msgr2=0x564819757ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:25.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e35ffb700 0 client.17220 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:44:25.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.505+0000 7f3e638deb80 1 -- 172.21.15.115:0/3693335286 shutdown_connections 2024-05-09T05:44:25.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.506+0000 7f3e638deb80 1 -- 172.21.15.115:0/3693335286 wait complete. 2024-05-09T05:44:25.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.506+0000 7f3e638deb80 20 client.17220 trim_cache size 0 max 16384 2024-05-09T05:44:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.520+0000 7f3e638deb80 1 Processor -- start 2024-05-09T05:44:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.520+0000 7f3e638deb80 1 -- start start 2024-05-09T05:44:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.520+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 0x56481976acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.520+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819660d00 0x56481976c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.520+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 0x564819758700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.520+0000 7f3e638deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b03f0 con 0x564819660d00 2024-05-09T05:44:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.520+0000 7f3e638deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b1f30 con 0x5648195e64a0 2024-05-09T05:44:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.520+0000 7f3e638deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b0ac0 con 0x564819763400 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819660d00 0x56481976c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4dd9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819660d00 0x56481976c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57386/0 (socket says 172.21.15.115:57386) 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/1042601648 learned_addr learned my addr 172.21.15.115:0/1042601648 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 0x56481976acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4ed9d700 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 0x564819758700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/1042601648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 msgr2=0x564819758700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 0x564819758700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/1042601648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 msgr2=0x56481976acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 0x56481976acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4dd9b700 1 -- 172.21.15.115:0/1042601648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481975a9d0 con 0x564819660d00 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 0x56481976acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4ed9d700 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 0x564819758700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819660d00 0x56481976c940 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3e400dfb50 tx=0x7f3e400a90b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5731d8b9b9f7ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e37fff700 1 -- 172.21.15.115:0/1042601648 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40021300 con 0x564819660d00 2024-05-09T05:44:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e37fff700 1 -- 172.21.15.115:0/1042601648 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e400214a0 con 0x564819660d00 2024-05-09T05:44:25.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.521+0000 7f3e37fff700 1 -- 172.21.15.115:0/1042601648 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e400b2000 con 0x564819660d00 2024-05-09T05:44:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 -- 172.21.15.115:0/1042601648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819660d00 msgr2=0x56481976c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819660d00 0x56481976c940 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3e400dfb50 tx=0x7f3e400a90b0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 -- 172.21.15.115:0/1042601648 shutdown_connections 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648195e64a0 0x56481976acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 0x564819758700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 --2- 172.21.15.115:0/1042601648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564819660d00 0x56481976c940 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 -- 172.21.15.115:0/1042601648 >> 172.21.15.115:0/1042601648 conn(0x5648195e9f70 msgr2=0x564819757ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 -- 172.21.15.115:0/1042601648 shutdown_connections 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 -- 172.21.15.115:0/1042601648 wait complete. 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.522+0000 7f3e638deb80 1 -- start start 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e638deb80 1 Processor -- start 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819785050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819660d00 0x564819775110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e638deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 0x564819775650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e638deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b1860 con 0x5648195e64a0 2024-05-09T05:44:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e638deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501b1190 con 0x564819660d00 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e638deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3e501ae390 con 0x564819763400 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e638deb80 10 client.0 _ll_register_callbacks cb 0x5648194f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819785050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819785050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57394/0 (socket says 172.21.15.115:57394) 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4e59c700 1 -- 172.21.15.115:0/3011861490 learned_addr learned my addr 172.21.15.115:0/3011861490 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819660d00 0x564819775110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4ed9d700 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 0x564819775650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4e59c700 1 -- 172.21.15.115:0/3011861490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 msgr2=0x564819775650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 0x564819775650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4e59c700 1 -- 172.21.15.115:0/3011861490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819660d00 msgr2=0x564819775110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819660d00 0x564819775110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4e59c700 1 -- 172.21.15.115:0/3011861490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564819769af0 con 0x5648195e64a0 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819660d00 0x564819775110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e4e59c700 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819785050 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3e440099c0 tx=0x7f3e44009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f5c2ca9bb84636 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.523+0000 7f3e35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3011861490 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4403b030 con 0x5648195e64a0 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3011861490 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f3e44003950 con 0x5648195e64a0 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e638deb80 1 -- 172.21.15.115:0/3011861490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e400214a0 con 0x5648195e64a0 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3011861490 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3e44045050 con 0x5648195e64a0 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e638deb80 10 client.17226 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e638deb80 1 -- 172.21.15.115:0/3011861490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5648197eac50 con 0x5648195e64a0 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3011861490 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e400214a0 con 0x5648195e64a0 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3011861490 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3e44022070 con 0x5648195e64a0 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e638deb80 10 client.17226 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e638deb80 10 client.17226 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e638deb80 20 client.17226 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5648197eb1f0 0x5648197ed610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e638deb80 4 client.17226 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:44:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.524+0000 7f3e638deb80 1 -- 172.21.15.115:0/3011861490 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56481968db20 con 0x5648197eb1f0 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5648197eb1f0 0x5648197ed610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e35ffb700 10 client.17226 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e4dd9b700 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5648197eb1f0 0x5648197ed610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d7265b98d2202d0 server_cookie=a38bbbfd47998314 in_seq=0 out_seq=0 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e35ffb700 1 -- 172.21.15.115:0/3011861490 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56481968db20 con 0x5648197eb1f0 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e35ffb700 10 client.17226 handle_command_reply: tid=0 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 client.17226 shutdown 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 -- 172.21.15.115:0/3011861490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 msgr2=0x564819785050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819785050 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3e440099c0 tx=0x7f3e44009cd0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 -- 172.21.15.115:0/3011861490 shutdown_connections 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x5648197eb1f0 0x5648197ed610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564819763400 0x564819775650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e35ffb700 0 client.17226 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5648195e64a0 0x564819785050 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 --2- 172.21.15.115:0/3011861490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564819660d00 0x564819775110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 -- 172.21.15.115:0/3011861490 >> 172.21.15.115:0/3011861490 conn(0x5648195e9f70 msgr2=0x56481976cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:25.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 -- 172.21.15.115:0/3011861490 shutdown_connections 2024-05-09T05:44:25.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 1 -- 172.21.15.115:0/3011861490 wait complete. 2024-05-09T05:44:25.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.525+0000 7f3e638deb80 20 client.17226 trim_cache size 0 max 16384 2024-05-09T05:44:25.525 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 -- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3e380b29f0 msgr2=0x7f3e380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3e380b29f0 0x7f3e380b4f00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e44000c00 tx=0x7f3e44005fb0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 -- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50107e30 msgr2=0x7f3e5019b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50107e30 0x7f3e5019b2c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000d950 tx=0x7f3e4000dc60 comp rx=0 tx=0).stop 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 -- 172.21.15.115:0/1511636158 shutdown_connections 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3e380b29f0 0x7f3e380b4f00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50103550 0x7f3e5019ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e50108780 0x7f3e5019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 --2- 172.21.15.115:0/1511636158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e50107e30 0x7f3e5019b2c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 -- 172.21.15.115:0/1511636158 >> 172.21.15.115:0/1511636158 conn(0x7f3e50078e00 msgr2=0x7f3e50075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:25.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 -- 172.21.15.115:0/1511636158 shutdown_connections 2024-05-09T05:44:25.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:25.526+0000 7f3e54ec8700 1 -- 172.21.15.115:0/1511636158 wait complete. 2024-05-09T05:44:26.445 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:44:26.703 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:27.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:27 smithi195 ceph-mon[33844]: pgmap v190: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-09T05:44:27.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:27 smithi115 ceph-mon[30349]: pgmap v190: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-09T05:44:27.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:27 smithi161 ceph-mon[33805]: pgmap v190: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.429+0000 7f2cc5976700 1 -- 172.21.15.115:0/2738793439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0109850 msgr2=0x7f2cc01111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.429+0000 7f2cc5976700 1 --2- 172.21.15.115:0/2738793439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0109850 0x7f2cc01111b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800b0d0 tx=0x7f2cb800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.429+0000 7f2cc5976700 1 -- 172.21.15.115:0/2738793439 shutdown_connections 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.429+0000 7f2cc5976700 1 --2- 172.21.15.115:0/2738793439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0109850 0x7f2cc01111b0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.429+0000 7f2cc5976700 1 --2- 172.21.15.115:0/2738793439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc0102ae0 0x7f2cc0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.429+0000 7f2cc5976700 1 --2- 172.21.15.115:0/2738793439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc0108f00 0x7f2cc0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.430+0000 7f2cc5976700 1 -- 172.21.15.115:0/2738793439 >> 172.21.15.115:0/2738793439 conn(0x7f2cc00fe500 msgr2=0x7f2cc0100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.430+0000 7f2cc5976700 1 -- 172.21.15.115:0/2738793439 shutdown_connections 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.430+0000 7f2cc5976700 1 -- 172.21.15.115:0/2738793439 wait complete. 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.430+0000 7f2cc5976700 1 Processor -- start 2024-05-09T05:44:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.430+0000 7f2cc5976700 1 -- start start 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cc5976700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0102ae0 0x7f2cc019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cc5976700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc0108f00 0x7f2cc019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cc5976700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc0109850 0x7f2cc019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cc5976700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01143b0 con 0x7f2cc0102ae0 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cc5976700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0114230 con 0x7f2cc0108f00 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cc5976700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0114530 con 0x7f2cc0109850 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbeffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0102ae0 0x7f2cc019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbeffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0102ae0 0x7f2cc019ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57414/0 (socket says 172.21.15.115:57414) 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbeffd700 1 -- 172.21.15.115:0/837577555 learned_addr learned my addr 172.21.15.115:0/837577555 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc0108f00 0x7f2cc019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbf7fe700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc0109850 0x7f2cc019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbeffd700 1 -- 172.21.15.115:0/837577555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc0109850 msgr2=0x7f2cc019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc0109850 0x7f2cc019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbeffd700 1 -- 172.21.15.115:0/837577555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc0108f00 msgr2=0x7f2cc019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc0108f00 0x7f2cc019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbeffd700 1 -- 172.21.15.115:0/837577555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc019fc10 con 0x7f2cc0102ae0 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0102ae0 0x7f2cc019ad40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000b530 tx=0x7f2cb000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.431+0000 7f2cc4974700 1 -- 172.21.15.115:0/837577555 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0015020 con 0x7f2cc0102ae0 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.432+0000 7f2cc5976700 1 -- 172.21.15.115:0/837577555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc019ff00 con 0x7f2cc0102ae0 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.432+0000 7f2cc4974700 1 -- 172.21.15.115:0/837577555 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb0011cc0 con 0x7f2cc0102ae0 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.432+0000 7f2cc4974700 1 -- 172.21.15.115:0/837577555 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0010dc0 con 0x7f2cc0102ae0 2024-05-09T05:44:28.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.432+0000 7f2cc5976700 1 -- 172.21.15.115:0/837577555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc01a7820 con 0x7f2cc0102ae0 2024-05-09T05:44:28.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.434+0000 7f2cc4974700 1 -- 172.21.15.115:0/837577555 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f2cb0018020 con 0x7f2cc0102ae0 2024-05-09T05:44:28.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.435+0000 7f2cc4974700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2ca80b29f0 0x7f2ca80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.435+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2ca80b29f0 0x7f2ca80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.435+0000 7f2cc4974700 1 -- 172.21.15.115:0/837577555 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2cb00e20a0 con 0x7f2cc0102ae0 2024-05-09T05:44:28.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.435+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2ca80b29f0 0x7f2ca80b4f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0107d70 tx=0x7f2cb400b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:28.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.435+0000 7f2cd438cb80 1 Processor -- start 2024-05-09T05:44:28.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cd438cb80 1 -- start start 2024-05-09T05:44:28.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b1199160 0x55d1b12324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1232a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 0x55d1b122cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d1b114eff0 con 0x55d1b10af4a0 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d1b10ed7f0 con 0x55d1b122c6a0 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d1b114afb0 con 0x55d1b1199160 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbe7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1232a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbe7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1232a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57430/0 (socket says 172.21.15.115:57430) 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/1612640419 learned_addr learned my addr 172.21.15.115:0/1612640419 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbf7fe700 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 0x55d1b122cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b1199160 0x55d1b12324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/1612640419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b1199160 msgr2=0x55d1b12324f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b1199160 0x55d1b12324f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/1612640419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 msgr2=0x55d1b122cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 0x55d1b122cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/1612640419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b10e5370 con 0x55d1b10af4a0 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbf7fe700 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 0x55d1b122cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b1199160 0x55d1b12324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.436+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1232a30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4010f40 tx=0x7f2cb4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970acd153013af31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.437+0000 7f2ca5ffb700 1 -- 172.21.15.115:0/1612640419 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb400be60 con 0x55d1b10af4a0 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.437+0000 7f2ca5ffb700 1 -- 172.21.15.115:0/1612640419 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb401eab0 con 0x55d1b10af4a0 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.437+0000 7f2ca5ffb700 1 -- 172.21.15.115:0/1612640419 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb401ca10 con 0x55d1b10af4a0 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.437+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1612640419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 msgr2=0x55d1b1232a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.437+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1232a30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4010f40 tx=0x7f2cb4013000 comp rx=0 tx=0).stop 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.437+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1612640419 shutdown_connections 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.437+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 0x55d1b122cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b1199160 0x55d1b12324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/1612640419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1232a30 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1612640419 >> 172.21.15.115:0/1612640419 conn(0x55d1b110f6a0 msgr2=0x55d1b1227400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1612640419 shutdown_connections 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1612640419 wait complete. 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 -- start start 2024-05-09T05:44:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 Processor -- start 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b10af4a0 0x55d1b1236140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1236680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b122c6a0 0x55d1b1236bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d1b0fcaba0 con 0x55d1b1199160 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d1b1109150 con 0x55d1b10af4a0 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.438+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d1b12391d0 con 0x55d1b122c6a0 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cd438cb80 10 client.0 _ll_register_callbacks cb 0x55d1b0fc06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbe7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1236680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbe7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1236680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57442/0 (socket says 172.21.15.115:57442) 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/153695065 learned_addr learned my addr 172.21.15.115:0/153695065 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbf7fe700 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b122c6a0 0x55d1b1236bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b10af4a0 0x55d1b1236140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/153695065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b122c6a0 msgr2=0x55d1b1236bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b122c6a0 0x55d1b1236bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/153695065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b10af4a0 msgr2=0x55d1b1236140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b10af4a0 0x55d1b1236140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/153695065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b0fca490 con 0x55d1b1199160 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1236680 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400b510 tx=0x7f2cb401e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1610cadcfc1f90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2c9effd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2c9effd700 1 -- 172.21.15.115:0/153695065 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4035060 con 0x55d1b1199160 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2c9effd700 1 -- 172.21.15.115:0/153695065 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2cb4005320 con 0x55d1b1199160 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cd438cb80 1 -- 172.21.15.115:0/153695065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cb401eab0 con 0x55d1b1199160 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2c9effd700 1 -- 172.21.15.115:0/153695065 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2cb403c050 con 0x55d1b1199160 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cd438cb80 10 client.17238 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.439+0000 7f2cd438cb80 1 -- 172.21.15.115:0/153695065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d1b1243ac0 con 0x55d1b1199160 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.440+0000 7f2c9effd700 1 -- 172.21.15.115:0/153695065 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb401eab0 con 0x55d1b1199160 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.440+0000 7f2c9effd700 1 -- 172.21.15.115:0/153695065 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f2cb401c6f0 con 0x55d1b1199160 2024-05-09T05:44:28.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.440+0000 7f2cd438cb80 10 client.17238 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.440+0000 7f2cd438cb80 10 client.17238 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:44:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.440+0000 7f2cd438cb80 20 client.17238 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.440+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55d1b1246040 0x55d1b1248460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.440+0000 7f2cd438cb80 4 client.17238 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.440+0000 7f2cd438cb80 1 -- 172.21.15.115:0/153695065 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b115c9d0 con 0x55d1b1246040 2024-05-09T05:44:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.440+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55d1b1246040 0x55d1b1248460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.441+0000 7f2c9effd700 10 client.17238 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:44:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.441+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55d1b1246040 0x55d1b1248460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=812c1dfe6a1bf576 server_cookie=cb88a75a4326cf47 in_seq=0 out_seq=0 2024-05-09T05:44:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.441+0000 7f2c9effd700 1 -- 172.21.15.115:0/153695065 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1b115c9d0 con 0x55d1b1246040 2024-05-09T05:44:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.441+0000 7f2c9effd700 10 client.17238 handle_command_reply: tid=0 2024-05-09T05:44:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.441+0000 7f2cd438cb80 1 client.17238 shutdown 2024-05-09T05:44:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 -- 172.21.15.115:0/153695065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 msgr2=0x55d1b1236680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1236680 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400b510 tx=0x7f2cb401e700 comp rx=0 tx=0).stop 2024-05-09T05:44:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 -- 172.21.15.115:0/153695065 shutdown_connections 2024-05-09T05:44:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b10af4a0 0x55d1b1236140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55d1b1246040 0x55d1b1248460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2c9effd700 0 client.17238 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:44:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b122c6a0 0x55d1b1236bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/153695065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1236680 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 -- 172.21.15.115:0/153695065 >> 172.21.15.115:0/153695065 conn(0x55d1b110f6a0 msgr2=0x55d1b12271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 -- 172.21.15.115:0/153695065 shutdown_connections 2024-05-09T05:44:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 1 -- 172.21.15.115:0/153695065 wait complete. 2024-05-09T05:44:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.442+0000 7f2cd438cb80 20 client.17238 trim_cache size 0 max 16384 2024-05-09T05:44:28.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cd438cb80 1 Processor -- start 2024-05-09T05:44:28.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cd438cb80 1 -- start start 2024-05-09T05:44:28.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1242430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b1199160 0x55d1b124b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b122c6a0 0x55d1b124b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01b03d0 con 0x55d1b10af4a0 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01b1f10 con 0x55d1b1199160 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01b0aa0 con 0x55d1b122c6a0 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbeffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1242430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbeffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1242430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57456/0 (socket says 172.21.15.115:57456) 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbeffd700 1 -- 172.21.15.115:0/1235833129 learned_addr learned my addr 172.21.15.115:0/1235833129 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/1235833129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b1199160 0x55d1b124b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbeffd700 1 -- 172.21.15.115:0/1235833129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b122c6a0 msgr2=0x55d1b124b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/1235833129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b122c6a0 0x55d1b124b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbeffd700 1 -- 172.21.15.115:0/1235833129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b1199160 msgr2=0x55d1b124b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/1235833129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b1199160 0x55d1b124b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbeffd700 1 -- 172.21.15.115:0/1235833129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b1235f10 con 0x55d1b10af4a0 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/1235833129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b1199160 0x55d1b124b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.457+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/1235833129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1242430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000bf00 tx=0x7f2cb000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365649955c9980c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:28.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2ca57fa700 1 -- 172.21.15.115:0/1235833129 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb006e070 con 0x55d1b10af4a0 2024-05-09T05:44:28.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2ca57fa700 1 -- 172.21.15.115:0/1235833129 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb00a85e0 con 0x55d1b10af4a0 2024-05-09T05:44:28.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2ca57fa700 1 -- 172.21.15.115:0/1235833129 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0005cb0 con 0x55d1b10af4a0 2024-05-09T05:44:28.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1235833129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 msgr2=0x55d1b1242430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:28.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/1235833129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1242430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000bf00 tx=0x7f2cb000bdf0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1235833129 shutdown_connections 2024-05-09T05:44:28.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/1235833129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b122c6a0 0x55d1b124b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/1235833129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b10af4a0 0x55d1b1242430 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/1235833129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b1199160 0x55d1b124b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1235833129 >> 172.21.15.115:0/1235833129 conn(0x55d1b110f6a0 msgr2=0x55d1b1234a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1235833129 shutdown_connections 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.458+0000 7f2cd438cb80 1 -- 172.21.15.115:0/1235833129 wait complete. 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cd438cb80 1 -- start start 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cd438cb80 1 Processor -- start 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b10af4a0 0x55d1b1248760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1248ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cd438cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 0x55d1b12ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01b1840 con 0x55d1b1199160 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01b1170 con 0x55d1b122c6a0 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cd438cb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01ae370 con 0x55d1b10af4a0 2024-05-09T05:44:28.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cd438cb80 10 client.0 _ll_register_callbacks cb 0x55d1b0fc06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cbe7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1248ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cbe7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1248ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57460/0 (socket says 172.21.15.115:57460) 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.459+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/131614643 learned_addr learned my addr 172.21.15.115:0/131614643 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b10af4a0 0x55d1b1248760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cbf7fe700 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 0x55d1b12ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/131614643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b10af4a0 msgr2=0x55d1b1248760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b10af4a0 0x55d1b1248760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/131614643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 msgr2=0x55d1b12ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 0x55d1b12ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cbe7fc700 1 -- 172.21.15.115:0/131614643 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b12aeac0 con 0x55d1b1199160 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b10af4a0 0x55d1b1248760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cbe7fc700 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1248ca0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4011ab0 tx=0x7f2cb400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8e1f73452e9873 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2c9effd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:28.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2c9effd700 1 -- 172.21.15.115:0/131614643 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4035050 con 0x55d1b1199160 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2c9effd700 1 -- 172.21.15.115:0/131614643 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f2cb4005530 con 0x55d1b1199160 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cd438cb80 1 -- 172.21.15.115:0/131614643 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cb00a85e0 con 0x55d1b1199160 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2c9effd700 1 -- 172.21.15.115:0/131614643 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2cb401c3b0 con 0x55d1b1199160 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cd438cb80 10 client.17244 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2cd438cb80 1 -- 172.21.15.115:0/131614643 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d1b12afbe0 con 0x55d1b1199160 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.460+0000 7f2c9effd700 1 -- 172.21.15.115:0/131614643 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb00a85e0 con 0x55d1b1199160 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.461+0000 7f2c9effd700 1 -- 172.21.15.115:0/131614643 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f2cb4004bc0 con 0x55d1b1199160 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.461+0000 7f2cd438cb80 10 client.17244 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.461+0000 7f2cd438cb80 10 client.17244 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.461+0000 7f2cd438cb80 20 client.17244 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.461+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55d1b1246040 0x55d1b12b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.461+0000 7f2cd438cb80 4 client.17244 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.461+0000 7f2cd438cb80 1 -- 172.21.15.115:0/131614643 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1b1156a70 con 0x55d1b1246040 2024-05-09T05:44:28.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.461+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55d1b1246040 0x55d1b12b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2c9effd700 10 client.17244 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cbeffd700 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55d1b1246040 0x55d1b12b23b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc39ae796cf8707a server_cookie=35fd4145b39c6aea in_seq=0 out_seq=0 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2c9effd700 1 -- 172.21.15.115:0/131614643 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1b1156a70 con 0x55d1b1246040 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2c9effd700 10 client.17244 handle_command_reply: tid=0 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 client.17244 shutdown 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 -- 172.21.15.115:0/131614643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 msgr2=0x55d1b1248ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1248ca0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4011ab0 tx=0x7f2cb400bfa0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 -- 172.21.15.115:0/131614643 shutdown_connections 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1b122c6a0 0x55d1b12ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55d1b1246040 0x55d1b12b23b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1b10af4a0 0x55d1b1248760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 --2- 172.21.15.115:0/131614643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d1b1199160 0x55d1b1248ca0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 -- 172.21.15.115:0/131614643 >> 172.21.15.115:0/131614643 conn(0x55d1b110f6a0 msgr2=0x55d1b1231140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2c9effd700 0 client.17244 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:44:28.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 -- 172.21.15.115:0/131614643 shutdown_connections 2024-05-09T05:44:28.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 1 -- 172.21.15.115:0/131614643 wait complete. 2024-05-09T05:44:28.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.462+0000 7f2cd438cb80 20 client.17244 trim_cache size 0 max 16384 2024-05-09T05:44:28.461 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 -- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2ca80b29f0 msgr2=0x7f2ca80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2ca80b29f0 0x7f2ca80b4f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0107d70 tx=0x7f2cb400b410 comp rx=0 tx=0).stop 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 -- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0102ae0 msgr2=0x7f2cc019ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0102ae0 0x7f2cc019ad40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000b530 tx=0x7f2cb000b840 comp rx=0 tx=0).stop 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 -- 172.21.15.115:0/837577555 shutdown_connections 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2ca80b29f0 0x7f2ca80b4f00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc0109850 0x7f2cc019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc0102ae0 0x7f2cc019ad40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 --2- 172.21.15.115:0/837577555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc0108f00 0x7f2cc019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 -- 172.21.15.115:0/837577555 >> 172.21.15.115:0/837577555 conn(0x7f2cc00fe500 msgr2=0x7f2cc01086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 -- 172.21.15.115:0/837577555 shutdown_connections 2024-05-09T05:44:28.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:28.463+0000 7f2cc5976700 1 -- 172.21.15.115:0/837577555 wait complete. 2024-05-09T05:44:29.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:44:29.365 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:29 smithi195 ceph-mon[33844]: pgmap v191: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 18 op/s 2024-05-09T05:44:29.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:29 smithi115 ceph-mon[30349]: pgmap v191: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 18 op/s 2024-05-09T05:44:29.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:29 smithi161 ceph-mon[33805]: pgmap v191: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 18 op/s 2024-05-09T05:44:29.578 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:31.288 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:31 smithi115 ceph-mon[30349]: pgmap v192: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 409 KiB/s wr, 10 op/s 2024-05-09T05:44:31.288 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:31 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:44:31.288 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:31 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:44:31.288 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:31 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:44:31.288 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:31 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:44:31.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.407+0000 7f9c9bef4700 1 -- 172.21.15.115:0/2218631435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 msgr2=0x7f9c9410f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.407+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/2218631435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 0x7f9c9410f890 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00b0a0 tx=0x7f9c8c00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.407+0000 7f9c9bef4700 1 -- 172.21.15.115:0/2218631435 shutdown_connections 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.407+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/2218631435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 0x7f9c9410f890 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.407+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/2218631435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c941089f0 0x7f9c94108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.407+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/2218631435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c941029c0 0x7f9c94102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.407+0000 7f9c9bef4700 1 -- 172.21.15.115:0/2218631435 >> 172.21.15.115:0/2218631435 conn(0x7f9c940fe4e0 msgr2=0x7f9c94100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.408+0000 7f9c9bef4700 1 -- 172.21.15.115:0/2218631435 shutdown_connections 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.408+0000 7f9c9bef4700 1 -- 172.21.15.115:0/2218631435 wait complete. 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.408+0000 7f9c9bef4700 1 Processor -- start 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.408+0000 7f9c9bef4700 1 -- start start 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.408+0000 7f9c9bef4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c941029c0 0x7f9c9419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9bef4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 0x7f9c9419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9bef4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c941089f0 0x7f9c9419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9bef4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941132a0 con 0x7f9c94103310 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9bef4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94113120 con 0x7f9c941089f0 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9bef4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94113420 con 0x7f9c941029c0 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9948f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 0x7f9c9419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9948f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 0x7f9c9419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57482/0 (socket says 172.21.15.115:57482) 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9948f700 1 -- 172.21.15.115:0/3739997361 learned_addr learned my addr 172.21.15.115:0/3739997361 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c99c90700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c941029c0 0x7f9c9419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9a491700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c941089f0 0x7f9c9419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9948f700 1 -- 172.21.15.115:0/3739997361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c941029c0 msgr2=0x7f9c9419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9948f700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c941029c0 0x7f9c9419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9948f700 1 -- 172.21.15.115:0/3739997361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c941089f0 msgr2=0x7f9c9419c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:31.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9948f700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c941089f0 0x7f9c9419c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9948f700 1 -- 172.21.15.115:0/3739997361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c941a6e90 con 0x7f9c94103310 2024-05-09T05:44:31.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9948f700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 0x7f9c9419c080 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400d980 tx=0x7f9c8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:31.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c8affd700 1 -- 172.21.15.115:0/3739997361 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84017070 con 0x7f9c94103310 2024-05-09T05:44:31.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9bef4700 1 -- 172.21.15.115:0/3739997361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941a7120 con 0x7f9c94103310 2024-05-09T05:44:31.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c8affd700 1 -- 172.21.15.115:0/3739997361 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c8400fd20 con 0x7f9c94103310 2024-05-09T05:44:31.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c8affd700 1 -- 172.21.15.115:0/3739997361 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8400e3b0 con 0x7f9c94103310 2024-05-09T05:44:31.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.409+0000 7f9c9bef4700 1 -- 172.21.15.115:0/3739997361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941a76b0 con 0x7f9c94103310 2024-05-09T05:44:31.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:31 smithi161 ceph-mon[33805]: pgmap v192: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 409 KiB/s wr, 10 op/s 2024-05-09T05:44:31.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:31 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:44:31.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:31 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:44:31.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:31 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:44:31.414 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:31 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:44:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.412+0000 7f9c8affd700 1 -- 172.21.15.115:0/3739997361 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f9c8400f890 con 0x7f9c94103310 2024-05-09T05:44:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.413+0000 7f9c8affd700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c800b29f0 0x7f9c800b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.413+0000 7f9c99c90700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c800b29f0 0x7f9c800b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.413+0000 7f9c8affd700 1 -- 172.21.15.115:0/3739997361 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9c840e23e0 con 0x7f9c94103310 2024-05-09T05:44:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.413+0000 7f9c99c90700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c800b29f0 0x7f9c800b4f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c94103b00 tx=0x7f9c90009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.413+0000 7f9caa90ab80 1 Processor -- start 2024-05-09T05:44:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9caa90ab80 1 -- start start 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65822240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e65822780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e6581c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558e6573f0a0 con 0x558e6581c400 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558e656e49a0 con 0x558e65719d00 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558e6573b060 con 0x558e6569f4a0 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9a491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e6581c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9a491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e6581c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57494/0 (socket says 172.21.15.115:57494) 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9a491700 1 -- 172.21.15.115:0/3544666905 learned_addr learned my addr 172.21.15.115:0/3544666905 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c99c90700 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65822240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9948f700 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e65822780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9a491700 1 -- 172.21.15.115:0/3544666905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 msgr2=0x558e65822780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9a491700 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e65822780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9a491700 1 -- 172.21.15.115:0/3544666905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 msgr2=0x558e65822240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9a491700 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65822240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9a491700 1 -- 172.21.15.115:0/3544666905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e656d5500 con 0x558e6581c400 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9948f700 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e65822780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.414+0000 7f9c9a491700 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e6581c810 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00b0a0 tx=0x7f9c8c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a5e435c3bf6acc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.415+0000 7f9c7ffff700 1 -- 172.21.15.115:0/3544666905 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c020070 con 0x558e6581c400 2024-05-09T05:44:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.415+0000 7f9c7ffff700 1 -- 172.21.15.115:0/3544666905 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c8c006000 con 0x558e6581c400 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.415+0000 7f9c7ffff700 1 -- 172.21.15.115:0/3544666905 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c007ba0 con 0x558e6581c400 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.415+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3544666905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 msgr2=0x558e6581c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.415+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e6581c810 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00b0a0 tx=0x7f9c8c0092e0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.416+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3544666905 shutdown_connections 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.416+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65822240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.416+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e6581c810 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.416+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3544666905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e65822780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.416+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3544666905 >> 172.21.15.115:0/3544666905 conn(0x558e656a2f70 msgr2=0x558e65817220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.416+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3544666905 shutdown_connections 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.416+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3544666905 wait complete. 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.416+0000 7f9caa90ab80 1 -- start start 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.416+0000 7f9caa90ab80 1 Processor -- start 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e658260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65826610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e65826b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558e6581d100 con 0x558e6581c400 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558e656d9b20 con 0x558e65719d00 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558e65832220 con 0x558e6569f4a0 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9caa90ab80 10 client.0 _ll_register_callbacks cb 0x558e655b06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9a491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e65826b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9948f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65826610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9a491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e65826b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57500/0 (socket says 172.21.15.115:57500) 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9948f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65826610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:37094/0 (socket says 172.21.15.115:37094) 2024-05-09T05:44:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9a491700 1 -- 172.21.15.115:0/1678803199 learned_addr learned my addr 172.21.15.115:0/1678803199 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c99c90700 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e658260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9948f700 1 -- 172.21.15.115:0/1678803199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 msgr2=0x558e658260d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9948f700 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e658260d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9948f700 1 -- 172.21.15.115:0/1678803199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 msgr2=0x558e65826b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9948f700 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e65826b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9948f700 1 -- 172.21.15.115:0/1678803199 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e655ba490 con 0x558e65719d00 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9a491700 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e65826b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c9948f700 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65826610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c840df7f0 tx=0x7f9c840dfd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bfeb0cccc1a1bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.417+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/1678803199 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84020040 con 0x558e65719d00 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9caa90ab80 1 -- 172.21.15.115:0/1678803199 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c8c006000 con 0x558e65719d00 2024-05-09T05:44:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/1678803199 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9c840b1d50 con 0x558e65719d00 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/1678803199 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9c8400fed0 con 0x558e65719d00 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9caa90ab80 10 client.26627 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9caa90ab80 1 -- 172.21.15.115:0/1678803199 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558e65833c10 con 0x558e65719d00 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/1678803199 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c8c006000 con 0x558e65719d00 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/1678803199 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9c840a4cf0 con 0x558e65719d00 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9caa90ab80 10 client.26627 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9caa90ab80 10 client.26627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9caa90ab80 20 client.26627 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558e658361f0 0x558e65838610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9caa90ab80 4 client.26627 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9caa90ab80 1 -- 172.21.15.115:0/1678803199 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e6574ca80 con 0x558e658361f0 2024-05-09T05:44:31.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.418+0000 7f9c99c90700 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558e658361f0 0x558e65838610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.419+0000 7f9c7d7fa700 10 client.26627 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.419+0000 7f9c99c90700 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558e658361f0 0x558e65838610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38e45c54cd42ed3a server_cookie=acbed1aa14755786 in_seq=0 out_seq=0 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/1678803199 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558e6574ca80 con 0x558e658361f0 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9c7d7fa700 10 client.26627 handle_command_reply: tid=0 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 client.26627 shutdown 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 -- 172.21.15.115:0/1678803199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 msgr2=0x558e65826610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65826610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c840df7f0 tx=0x7f9c840dfd20 comp rx=0 tx=0).stop 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 -- 172.21.15.115:0/1678803199 shutdown_connections 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558e658361f0 0x558e65838610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e6581c400 0x558e65826b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e658260d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9c7d7fa700 0 client.26627 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:44:31.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/1678803199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e65719d00 0x558e65826610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 -- 172.21.15.115:0/1678803199 >> 172.21.15.115:0/1678803199 conn(0x558e656a2f70 msgr2=0x558e65810ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:31.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 -- 172.21.15.115:0/1678803199 shutdown_connections 2024-05-09T05:44:31.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 1 -- 172.21.15.115:0/1678803199 wait complete. 2024-05-09T05:44:31.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.420+0000 7f9caa90ab80 20 client.26627 trim_cache size 0 max 16384 2024-05-09T05:44:31.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9caa90ab80 1 Processor -- start 2024-05-09T05:44:31.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9caa90ab80 1 -- start start 2024-05-09T05:44:31.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e656a2f70 0x558e65811a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e6569f4a0 0x558e6581bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6583ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941b0320 con 0x558e65719d00 2024-05-09T05:44:31.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941b1e60 con 0x558e6569f4a0 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941b09f0 con 0x558e656a2f70 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9c9a491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6583ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9c99c90700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e656a2f70 0x558e65811a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9c9a491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6583ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57508/0 (socket says 172.21.15.115:57508) 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9c9a491700 1 -- 172.21.15.115:0/911408596 learned_addr learned my addr 172.21.15.115:0/911408596 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.435+0000 7f9c9948f700 1 --2- 172.21.15.115:0/911408596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e6569f4a0 0x558e6581bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.436+0000 7f9c9a491700 1 -- 172.21.15.115:0/911408596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e656a2f70 msgr2=0x558e65811a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.436+0000 7f9c9a491700 1 --2- 172.21.15.115:0/911408596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e656a2f70 0x558e65811a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.436+0000 7f9c9a491700 1 -- 172.21.15.115:0/911408596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e6569f4a0 msgr2=0x558e6581bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.436+0000 7f9c9a491700 1 --2- 172.21.15.115:0/911408596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e6569f4a0 0x558e6581bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.436+0000 7f9c9a491700 1 -- 172.21.15.115:0/911408596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e65817760 con 0x558e65719d00 2024-05-09T05:44:31.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.436+0000 7f9c9a491700 1 --2- 172.21.15.115:0/911408596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6583ddb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00b9c0 tx=0x7f9c8c007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b40427862ac559 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:31.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.436+0000 7f9c7f7fe700 1 -- 172.21.15.115:0/911408596 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c020040 con 0x558e65719d00 2024-05-09T05:44:31.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.436+0000 7f9c7f7fe700 1 -- 172.21.15.115:0/911408596 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c8c006000 con 0x558e65719d00 2024-05-09T05:44:31.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.436+0000 7f9c7f7fe700 1 -- 172.21.15.115:0/911408596 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c007cf0 con 0x558e65719d00 2024-05-09T05:44:31.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 -- 172.21.15.115:0/911408596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 msgr2=0x558e6583ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:31.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/911408596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6583ddb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00b9c0 tx=0x7f9c8c007740 comp rx=0 tx=0).stop 2024-05-09T05:44:31.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 -- 172.21.15.115:0/911408596 shutdown_connections 2024-05-09T05:44:31.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/911408596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6583ddb0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/911408596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e656a2f70 0x558e65811a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/911408596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e6569f4a0 0x558e6581bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 -- 172.21.15.115:0/911408596 >> 172.21.15.115:0/911408596 conn(0x558e658388e0 msgr2=0x558e6581b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:31.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 -- 172.21.15.115:0/911408596 shutdown_connections 2024-05-09T05:44:31.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 -- 172.21.15.115:0/911408596 wait complete. 2024-05-09T05:44:31.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.437+0000 7f9caa90ab80 1 -- start start 2024-05-09T05:44:31.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9caa90ab80 1 Processor -- start 2024-05-09T05:44:31.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e6582bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e656a2f70 0x558e6582c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9caa90ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6582c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941b1790 con 0x558e65719d00 2024-05-09T05:44:31.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941b10c0 con 0x558e656a2f70 2024-05-09T05:44:31.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9caa90ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941ae2c0 con 0x558e6569f4a0 2024-05-09T05:44:31.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9caa90ab80 10 client.0 _ll_register_callbacks cb 0x558e655b06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:31.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9c9a491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6582c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9c9a491700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6582c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57510/0 (socket says 172.21.15.115:57510) 2024-05-09T05:44:31.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9c9a491700 1 -- 172.21.15.115:0/3916213 learned_addr learned my addr 172.21.15.115:0/3916213 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:31.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9c9948f700 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e656a2f70 0x558e6582c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.438+0000 7f9c99c90700 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e6582bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c9a491700 1 -- 172.21.15.115:0/3916213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 msgr2=0x558e6582bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:31.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c9a491700 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e6582bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c9a491700 1 -- 172.21.15.115:0/3916213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e656a2f70 msgr2=0x558e6582c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:31.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c9a491700 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e656a2f70 0x558e6582c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c9a491700 1 -- 172.21.15.115:0/3916213 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e6589f970 con 0x558e65719d00 2024-05-09T05:44:31.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c99c90700 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e6582bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:31.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c9a491700 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6582c680 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009320 tx=0x7f9c8c0079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd43661a72be869c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:31.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:31.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/3916213 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c034070 con 0x558e65719d00 2024-05-09T05:44:31.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/3916213 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9c8c0050b0 con 0x558e65719d00 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3916213 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c8c006000 con 0x558e65719d00 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/3916213 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9c8c020040 con 0x558e65719d00 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9caa90ab80 10 client.17262 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3916213 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558e658a0a50 con 0x558e65719d00 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/3916213 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c8c006000 con 0x558e65719d00 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.439+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/3916213 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9c8c011070 con 0x558e65719d00 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9caa90ab80 10 client.17262 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9caa90ab80 10 client.17262 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9caa90ab80 20 client.17262 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558e6581c400 0x558e658a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:31.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9caa90ab80 4 client.17262 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3916213 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e65746b20 con 0x558e6581c400 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9c99c90700 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558e6581c400 0x558e658a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9c99c90700 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558e6581c400 0x558e658a2c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7825f5462781548 server_cookie=4889a3d3e6a02849 in_seq=0 out_seq=0 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9c7d7fa700 10 client.17262 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9c7d7fa700 1 -- 172.21.15.115:0/3916213 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e65746b20 con 0x558e6581c400 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9c7d7fa700 10 client.17262 handle_command_reply: tid=0 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.440+0000 7f9caa90ab80 1 client.17262 shutdown 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3916213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 msgr2=0x558e6582c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6582c680 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009320 tx=0x7f9c8c0079e0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3916213 shutdown_connections 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x558e6581c400 0x558e658a2c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e65719d00 0x558e6582c680 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558e6569f4a0 0x558e6582bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 --2- 172.21.15.115:0/3916213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e656a2f70 0x558e6582c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c7d7fa700 0 client.17262 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:44:31.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3916213 >> 172.21.15.115:0/3916213 conn(0x558e658388e0 msgr2=0x558e65811780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:31.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3916213 shutdown_connections 2024-05-09T05:44:31.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 1 -- 172.21.15.115:0/3916213 wait complete. 2024-05-09T05:44:31.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9caa90ab80 20 client.17262 trim_cache size 0 max 16384 2024-05-09T05:44:31.445 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:44:31.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 -- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c800b29f0 msgr2=0x7f9c800b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:31.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c800b29f0 0x7f9c800b4f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c94103b00 tx=0x7f9c90009260 comp rx=0 tx=0).stop 2024-05-09T05:44:31.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 -- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 msgr2=0x7f9c9419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:31.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 0x7f9c9419c080 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400d980 tx=0x7f9c8400dc90 comp rx=0 tx=0).stop 2024-05-09T05:44:31.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 -- 172.21.15.115:0/3739997361 shutdown_connections 2024-05-09T05:44:31.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c800b29f0 0x7f9c800b4f00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c941089f0 0x7f9c9419c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c941029c0 0x7f9c9419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 --2- 172.21.15.115:0/3739997361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c94103310 0x7f9c9419c080 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:31.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.441+0000 7f9c9bef4700 1 -- 172.21.15.115:0/3739997361 >> 172.21.15.115:0/3739997361 conn(0x7f9c940fe4e0 msgr2=0x7f9c94106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:31.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.442+0000 7f9c9bef4700 1 -- 172.21.15.115:0/3739997361 shutdown_connections 2024-05-09T05:44:31.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:31.442+0000 7f9c9bef4700 1 -- 172.21.15.115:0/3739997361 wait complete. 2024-05-09T05:44:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:31 smithi195 ceph-mon[33844]: pgmap v192: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 409 KiB/s wr, 10 op/s 2024-05-09T05:44:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:31 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:44:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:31 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:44:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:31 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:44:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:31 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:44:32.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:44:32.480 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:33.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:33 smithi115 ceph-mon[30349]: pgmap v193: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 409 KiB/s wr, 10 op/s 2024-05-09T05:44:33.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:33 smithi161 ceph-mon[33805]: pgmap v193: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 409 KiB/s wr, 10 op/s 2024-05-09T05:44:33.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:33 smithi195 ceph-mon[33844]: pgmap v193: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 409 KiB/s wr, 10 op/s 2024-05-09T05:44:35.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:35 smithi115 ceph-mon[30349]: pgmap v194: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 410 KiB/s wr, 15 op/s 2024-05-09T05:44:35.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:35 smithi161 ceph-mon[33805]: pgmap v194: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 410 KiB/s wr, 15 op/s 2024-05-09T05:44:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.434+0000 7f4be4815640 1 -- 172.21.15.115:0/1680429785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be0108ba0 msgr2=0x7f4be0102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.434+0000 7f4be4815640 1 --2- 172.21.15.115:0/1680429785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be0108ba0 0x7f4be0102740 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009ca0 tx=0x7f4bd002a8c0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.434+0000 7f4be4815640 1 -- 172.21.15.115:0/1680429785 shutdown_connections 2024-05-09T05:44:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.434+0000 7f4be4815640 1 --2- 172.21.15.115:0/1680429785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4be01032f0 0x7f4be010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.434+0000 7f4be4815640 1 --2- 172.21.15.115:0/1680429785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be0109040 0x7f4be0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.434+0000 7f4be4815640 1 --2- 172.21.15.115:0/1680429785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be0108ba0 0x7f4be0102740 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.434+0000 7f4be4815640 1 -- 172.21.15.115:0/1680429785 >> 172.21.15.115:0/1680429785 conn(0x7f4be00782e0 msgr2=0x7f4be0078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.434+0000 7f4be4815640 1 -- 172.21.15.115:0/1680429785 shutdown_connections 2024-05-09T05:44:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.434+0000 7f4be4815640 1 -- 172.21.15.115:0/1680429785 wait complete. 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.435+0000 7f4be4815640 1 Processor -- start 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.435+0000 7f4be4815640 1 -- start start 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4be4815640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be01032f0 0x7f4be01a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4be4815640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be0108ba0 0x7f4be01a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4be4815640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4be0109040 0x7f4be019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4be4815640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4be0112c90 con 0x7f4be01032f0 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4be4815640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4be0112b10 con 0x7f4be0109040 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4be4815640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4be0112e10 con 0x7f4be0108ba0 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bde59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be01032f0 0x7f4be01a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bde59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be01032f0 0x7f4be01a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50008/0 (socket says 172.21.15.115:50008) 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bde59c640 1 -- 172.21.15.115:0/3408965976 learned_addr learned my addr 172.21.15.115:0/3408965976 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be0108ba0 0x7f4be01a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bded9d640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4be0109040 0x7f4be019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bde59c640 1 -- 172.21.15.115:0/3408965976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be0108ba0 msgr2=0x7f4be01a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bde59c640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be0108ba0 0x7f4be01a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bde59c640 1 -- 172.21.15.115:0/3408965976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4be0109040 msgr2=0x7f4be019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bde59c640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4be0109040 0x7f4be019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bde59c640 1 -- 172.21.15.115:0/3408965976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be019adb0 con 0x7f4be01032f0 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bded9d640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4be0109040 0x7f4be019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bde59c640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be01032f0 0x7f4be01a0500 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0003f90 tx=0x7f4bd0009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.436+0000 7f4bcf7fe640 1 -- 172.21.15.115:0/3408965976 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0032cd0 con 0x7f4be01032f0 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.437+0000 7f4be4815640 1 -- 172.21.15.115:0/3408965976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be019afe0 con 0x7f4be01032f0 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.437+0000 7f4bcf7fe640 1 -- 172.21.15.115:0/3408965976 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd0032e70 con 0x7f4be01032f0 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.437+0000 7f4bcf7fe640 1 -- 172.21.15.115:0/3408965976 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0004500 con 0x7f4be01032f0 2024-05-09T05:44:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.437+0000 7f4be4815640 1 -- 172.21.15.115:0/3408965976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be01a72a0 con 0x7f4be01032f0 2024-05-09T05:44:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4beb7a5740 1 Processor -- start 2024-05-09T05:44:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4beb7a5740 1 -- start start 2024-05-09T05:44:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 0x560a3b6fd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 0x560a3b6fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b6fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560a3b6627c0 con 0x560a3b3394d0 2024-05-09T05:44:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560a3b5fe0e0 con 0x560a3b35a1b0 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560a3b65e780 con 0x560a3b3312f0 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4bddd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b6fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4bddd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b6fe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50020/0 (socket says 172.21.15.115:50020) 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4bddd9b640 1 -- 172.21.15.115:0/3889941818 learned_addr learned my addr 172.21.15.115:0/3889941818 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4bcf7fe640 1 -- 172.21.15.115:0/3408965976 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4bd00046c0 con 0x7f4be01032f0 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.440+0000 7f4bded9d640 1 --2- 172.21.15.115:0/3889941818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 0x560a3b6fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bddd9b640 1 -- 172.21.15.115:0/3889941818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 msgr2=0x560a3b6fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/3889941818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 0x560a3b6fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bddd9b640 1 -- 172.21.15.115:0/3889941818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 msgr2=0x560a3b6fd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/3889941818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 0x560a3b6fd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bddd9b640 1 -- 172.21.15.115:0/3889941818 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3b6277c0 con 0x560a3b3394d0 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bded9d640 1 --2- 172.21.15.115:0/3889941818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 0x560a3b6fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/3889941818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b6fe100 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009af0 tx=0x7f4bd40113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bea9e3e60e34d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bccff9640 1 -- 172.21.15.115:0/3889941818 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4011e10 con 0x560a3b3394d0 2024-05-09T05:44:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bccff9640 1 -- 172.21.15.115:0/3889941818 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd4016070 con 0x560a3b3394d0 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bccff9640 1 -- 172.21.15.115:0/3889941818 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd401b470 con 0x560a3b3394d0 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.441+0000 7f4bcf7fe640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bc80b2950 0x7f4bc80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4bded9d640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bc80b2950 0x7f4bc80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4bcf7fe640 1 -- 172.21.15.115:0/3408965976 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4bd0103d70 con 0x7f4be01032f0 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3889941818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 msgr2=0x560a3b6fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4bded9d640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bc80b2950 0x7f4bc80b4e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x560a3b6fa7f0 tx=0x7f4bd8005f90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3889941818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b6fe100 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009af0 tx=0x7f4bd40113e0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3889941818 shutdown_connections 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3889941818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b6fe100 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3889941818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 0x560a3b6fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3889941818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 0x560a3b6fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3889941818 >> 172.21.15.115:0/3889941818 conn(0x560a3b34e300 msgr2=0x560a3b6f9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3889941818 shutdown_connections 2024-05-09T05:44:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.442+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3889941818 wait complete. 2024-05-09T05:44:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4beb7a5740 1 -- start start 2024-05-09T05:44:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4beb7a5740 1 Processor -- start 2024-05-09T05:44:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 0x560a3b709690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b709bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b35a1b0 0x560a3b707b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560a3b70a7b0 con 0x560a3b3394d0 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560a3b5e99c0 con 0x560a3b3312f0 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560a3b70a930 con 0x560a3b35a1b0 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4beb7a5740 10 client.0 _ll_register_callbacks cb 0x560a3b616020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4bddd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b709bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4bddd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b709bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50030/0 (socket says 172.21.15.115:50030) 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4bddd9b640 1 -- 172.21.15.115:0/3002535965 learned_addr learned my addr 172.21.15.115:0/3002535965 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4bded9d640 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b35a1b0 0x560a3b707b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.443+0000 7f4bde59c640 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 0x560a3b709690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bddd9b640 1 -- 172.21.15.115:0/3002535965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b35a1b0 msgr2=0x560a3b707b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b35a1b0 0x560a3b707b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bddd9b640 1 -- 172.21.15.115:0/3002535965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 msgr2=0x560a3b709690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 0x560a3b709690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bddd9b640 1 -- 172.21.15.115:0/3002535965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a3b6101f0 con 0x560a3b3394d0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b709bd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4003b60 tx=0x7f4bd4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dda446bf21d478 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/3002535965 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4004050 con 0x560a3b3394d0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/3002535965 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4bd40041f0 con 0x560a3b3394d0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3002535965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bd4016070 con 0x560a3b3394d0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/3002535965 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4bd4031050 con 0x560a3b3394d0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4beb7a5740 10 client.17274 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3002535965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560a3b3c5790 con 0x560a3b3394d0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.444+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/3002535965 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd4016070 con 0x560a3b3394d0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.445+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/3002535965 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f4bd401bc50 con 0x560a3b3394d0 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.445+0000 7f4beb7a5740 10 client.17274 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.445+0000 7f4beb7a5740 10 client.17274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.445+0000 7f4beb7a5740 20 client.17274 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.445+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a3b318d80 0x560a3b719e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.445+0000 7f4beb7a5740 4 client.17274 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.445+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3002535965 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a3b6703d0 con 0x560a3b318d80 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.445+0000 7f4bde59c640 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a3b318d80 0x560a3b719e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.446+0000 7f4bc5ffb640 10 client.17274 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.446+0000 7f4bde59c640 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a3b318d80 0x560a3b719e50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a35fccb942005f server_cookie=4aca746db81b0377 in_seq=0 out_seq=0 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.446+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/3002535965 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a3b6703d0 con 0x560a3b318d80 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.446+0000 7f4bc5ffb640 10 client.17274 handle_command_reply: tid=0 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.446+0000 7f4beb7a5740 1 client.17274 shutdown 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.446+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3002535965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 msgr2=0x560a3b709bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.446+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b709bd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4003b60 tx=0x7f4bd4003820 comp rx=0 tx=0).stop 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3002535965 shutdown_connections 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a3b318d80 0x560a3b719e50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b35a1b0 0x560a3b707b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4bc5ffb640 0 client.17274 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b709bd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/3002535965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 0x560a3b709690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3002535965 >> 172.21.15.115:0/3002535965 conn(0x560a3b34e300 msgr2=0x560a3b6ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3002535965 shutdown_connections 2024-05-09T05:44:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4beb7a5740 1 -- 172.21.15.115:0/3002535965 wait complete. 2024-05-09T05:44:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.447+0000 7f4beb7a5740 20 client.17274 trim_cache size 0 max 16384 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.461+0000 7f4beb7a5740 1 Processor -- start 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.461+0000 7f4beb7a5740 1 -- start start 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 0x560a3b7077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3394d0 0x560a3b6f9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b35a1b0 0x560a3b6f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01b10c0 con 0x560a3b35a1b0 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01b2c00 con 0x560a3b3312f0 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01b1790 con 0x560a3b3394d0 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bded9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b35a1b0 0x560a3b6f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bded9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b35a1b0 0x560a3b6f9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50038/0 (socket says 172.21.15.115:50038) 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bded9d640 1 -- 172.21.15.115:0/1985008182 learned_addr learned my addr 172.21.15.115:0/1985008182 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/1985008182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3394d0 0x560a3b6f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bde59c640 1 --2- 172.21.15.115:0/1985008182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 0x560a3b7077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bded9d640 1 -- 172.21.15.115:0/1985008182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3394d0 msgr2=0x560a3b6f9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bded9d640 1 --2- 172.21.15.115:0/1985008182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3394d0 0x560a3b6f9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bded9d640 1 -- 172.21.15.115:0/1985008182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 msgr2=0x560a3b7077d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bded9d640 1 --2- 172.21.15.115:0/1985008182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 0x560a3b7077d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bded9d640 1 -- 172.21.15.115:0/1985008182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3b43c5f0 con 0x560a3b35a1b0 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bded9d640 1 --2- 172.21.15.115:0/1985008182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b35a1b0 0x560a3b6f9c50 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4bd801f770 tx=0x7f4bd801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf3d4214e0bd9ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bc7fff640 1 -- 172.21.15.115:0/1985008182 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80116c0 con 0x560a3b35a1b0 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bc7fff640 1 -- 172.21.15.115:0/1985008182 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd8011cf0 con 0x560a3b35a1b0 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.462+0000 7f4bc7fff640 1 -- 172.21.15.115:0/1985008182 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd802ad40 con 0x560a3b35a1b0 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 -- 172.21.15.115:0/1985008182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b35a1b0 msgr2=0x560a3b6f9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/1985008182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b35a1b0 0x560a3b6f9c50 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4bd801f770 tx=0x7f4bd801a000 comp rx=0 tx=0).stop 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 -- 172.21.15.115:0/1985008182 shutdown_connections 2024-05-09T05:44:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/1985008182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b35a1b0 0x560a3b6f9c50 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/1985008182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3394d0 0x560a3b6f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/1985008182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b3312f0 0x560a3b7077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 -- 172.21.15.115:0/1985008182 >> 172.21.15.115:0/1985008182 conn(0x560a3b34e300 msgr2=0x560a3b7070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 -- 172.21.15.115:0/1985008182 shutdown_connections 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 -- 172.21.15.115:0/1985008182 wait complete. 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.463+0000 7f4beb7a5740 1 -- start start 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4beb7a5740 1 Processor -- start 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 0x560a3b71d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b71a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4beb7a5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 0x560a3b71aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01b2530 con 0x560a3b3394d0 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01b1e60 con 0x560a3b35a1b0 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4beb7a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01af0a0 con 0x560a3b3312f0 2024-05-09T05:44:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4beb7a5740 10 client.0 _ll_register_callbacks cb 0x7f4bd80095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bddd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b71a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bddd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b71a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50042/0 (socket says 172.21.15.115:50042) 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bddd9b640 1 -- 172.21.15.115:0/877626135 learned_addr learned my addr 172.21.15.115:0/877626135 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bde59c640 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 0x560a3b71d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bded9d640 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 0x560a3b71aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bddd9b640 1 -- 172.21.15.115:0/877626135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 msgr2=0x560a3b71d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 0x560a3b71d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bddd9b640 1 -- 172.21.15.115:0/877626135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 msgr2=0x560a3b71aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 0x560a3b71aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bddd9b640 1 -- 172.21.15.115:0/877626135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a3b626ed0 con 0x560a3b3394d0 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bde59c640 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 0x560a3b71d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bddd9b640 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b71a5a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40058a0 tx=0x7f4bd40125c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9405859f9eb44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.464+0000 7f4bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/877626135 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40096e0 con 0x560a3b3394d0 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/877626135 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f4bd4005070 con 0x560a3b3394d0 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4beb7a5740 1 -- 172.21.15.115:0/877626135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bd8011cf0 con 0x560a3b3394d0 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/877626135 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4bd4012850 con 0x560a3b3394d0 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4beb7a5740 10 client.17280 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4beb7a5740 1 -- 172.21.15.115:0/877626135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560a3b781af0 con 0x560a3b3394d0 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/877626135 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bd8011cf0 con 0x560a3b3394d0 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/877626135 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f4bd401b9a0 con 0x560a3b3394d0 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4beb7a5740 10 client.17280 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4beb7a5740 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4beb7a5740 20 client.17280 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a3b2ee1c0 0x560a3b784270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4beb7a5740 4 client.17280 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4beb7a5740 1 -- 172.21.15.115:0/877626135 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a3b66a240 con 0x560a3b2ee1c0 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.465+0000 7f4bde59c640 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a3b2ee1c0 0x560a3b784270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.466+0000 7f4bc5ffb640 10 client.17280 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.466+0000 7f4bde59c640 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a3b2ee1c0 0x560a3b784270 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83d5d44c87a612f server_cookie=b00f4643b5619bcb in_seq=0 out_seq=0 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.474+0000 7f4bc5ffb640 1 -- 172.21.15.115:0/877626135 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560a3b66a240 con 0x560a3b2ee1c0 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4bc5ffb640 10 client.17280 handle_command_reply: tid=0 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 client.17280 shutdown 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 -- 172.21.15.115:0/877626135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 msgr2=0x560a3b71a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b71a5a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40058a0 tx=0x7f4bd40125c0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 -- 172.21.15.115:0/877626135 shutdown_connections 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a3b35a1b0 0x560a3b71aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560a3b3394d0 0x560a3b71a5a0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560a3b2ee1c0 0x560a3b784270 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 --2- 172.21.15.115:0/877626135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a3b3312f0 0x560a3b71d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4bc5ffb640 0 client.17280 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 -- 172.21.15.115:0/877626135 >> 172.21.15.115:0/877626135 conn(0x560a3b34e300 msgr2=0x560a3b6fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 -- 172.21.15.115:0/877626135 shutdown_connections 2024-05-09T05:44:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 1 -- 172.21.15.115:0/877626135 wait complete. 2024-05-09T05:44:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4beb7a5740 20 client.17280 trim_cache size 0 max 16384 2024-05-09T05:44:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:44:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:44:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:44:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4be4815640 1 -- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bc80b2950 msgr2=0x7f4bc80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4be4815640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bc80b2950 0x7f4bc80b4e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x560a3b6fa7f0 tx=0x7f4bd8005f90 comp rx=0 tx=0).stop 2024-05-09T05:44:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4be4815640 1 -- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be01032f0 msgr2=0x7f4be01a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.475+0000 7f4be4815640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be01032f0 0x7f4be01a0500 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0003f90 tx=0x7f4bd0009870 comp rx=0 tx=0).stop 2024-05-09T05:44:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.476+0000 7f4be4815640 1 -- 172.21.15.115:0/3408965976 shutdown_connections 2024-05-09T05:44:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.476+0000 7f4be4815640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4bc80b2950 0x7f4bc80b4e60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.476+0000 7f4be4815640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4be0109040 0x7f4be019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.476+0000 7f4be4815640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be0108ba0 0x7f4be01a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.476+0000 7f4be4815640 1 --2- 172.21.15.115:0/3408965976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be01032f0 0x7f4be01a0500 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.476+0000 7f4be4815640 1 -- 172.21.15.115:0/3408965976 >> 172.21.15.115:0/3408965976 conn(0x7f4be00782e0 msgr2=0x7f4be006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.476+0000 7f4be4815640 1 -- 172.21.15.115:0/3408965976 shutdown_connections 2024-05-09T05:44:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:35.476+0000 7f4be4815640 1 -- 172.21.15.115:0/3408965976 wait complete. 2024-05-09T05:44:35.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:35 smithi195 ceph-mon[33844]: pgmap v194: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 410 KiB/s wr, 15 op/s 2024-05-09T05:44:37.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.291+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1503436188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c103c20 msgr2=0x7f1d5c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:37.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.291+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1503436188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c103c20 0x7f1d5c104030 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0099c0 tx=0x7f1d4c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.291+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1503436188 shutdown_connections 2024-05-09T05:44:37.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.291+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1503436188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d5c108770 0x7f1d5c1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.291+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1503436188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c103c20 0x7f1d5c104030 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.291+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1503436188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c107e20 0x7f1d5c108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.291+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1503436188 >> 172.21.15.115:0/1503436188 conn(0x7f1d5c0fe6f0 msgr2=0x7f1d5c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:37.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.291+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1503436188 shutdown_connections 2024-05-09T05:44:37.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.291+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1503436188 wait complete. 2024-05-09T05:44:37.287 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:37 smithi115 ceph-mon[30349]: pgmap v195: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-09T05:44:37.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.292+0000 7f1d61dcd700 1 Processor -- start 2024-05-09T05:44:37.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.292+0000 7f1d61dcd700 1 -- start start 2024-05-09T05:44:37.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.292+0000 7f1d61dcd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c103c20 0x7f1d5c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d61dcd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c107e20 0x7f1d5c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d61dcd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d5c108770 0x7f1d5c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d61dcd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1132d0 con 0x7f1d5c107e20 2024-05-09T05:44:37.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d61dcd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c113150 con 0x7f1d5c103c20 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d61dcd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c113450 con 0x7f1d5c108770 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d52ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c107e20 0x7f1d5c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d52ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c107e20 0x7f1d5c19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50064/0 (socket says 172.21.15.115:50064) 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d52ffd700 1 -- 172.21.15.115:0/1979451651 learned_addr learned my addr 172.21.15.115:0/1979451651 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c103c20 0x7f1d5c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d5bfff700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d5c108770 0x7f1d5c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d52ffd700 1 -- 172.21.15.115:0/1979451651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d5c108770 msgr2=0x7f1d5c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d5c108770 0x7f1d5c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d52ffd700 1 -- 172.21.15.115:0/1979451651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c103c20 msgr2=0x7f1d5c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:37.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c103c20 0x7f1d5c19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d52ffd700 1 -- 172.21.15.115:0/1979451651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c1a6f50 con 0x7f1d5c107e20 2024-05-09T05:44:37.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c107e20 0x7f1d5c19c1b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400c9e0 tx=0x7f1d4400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:37.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d597fa700 1 -- 172.21.15.115:0/1979451651 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44007aa0 con 0x7f1d5c107e20 2024-05-09T05:44:37.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d597fa700 1 -- 172.21.15.115:0/1979451651 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d44007c40 con 0x7f1d5c107e20 2024-05-09T05:44:37.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.293+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1979451651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5c1a7240 con 0x7f1d5c107e20 2024-05-09T05:44:37.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.294+0000 7f1d597fa700 1 -- 172.21.15.115:0/1979451651 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44018660 con 0x7f1d5c107e20 2024-05-09T05:44:37.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.294+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1979451651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5c1a7780 con 0x7f1d5c107e20 2024-05-09T05:44:37.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.296+0000 7f1d597fa700 1 -- 172.21.15.115:0/1979451651 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1d44004830 con 0x7f1d5c107e20 2024-05-09T05:44:37.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.297+0000 7f1d597fa700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d480b29f0 0x7f1d480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.297+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d480b29f0 0x7f1d480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.297+0000 7f1d597fa700 1 -- 172.21.15.115:0/1979451651 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1d440e2980 con 0x7f1d5c107e20 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.297+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d480b29f0 0x7f1d480b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009990 tx=0x7f1d4c005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.297+0000 7f1d707e3b80 1 Processor -- start 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d707e3b80 1 -- start start 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157a3d00 0x55bc158ac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157294a0 0x55bc158ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158a6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc157c90a0 con 0x55bc157294a0 2024-05-09T05:44:37.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bc1576e9a0 con 0x55bc158a6400 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bc157c5060 con 0x55bc157a3d00 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d52ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157294a0 0x55bc158ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d52ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157294a0 0x55bc158ac780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50074/0 (socket says 172.21.15.115:50074) 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d52ffd700 1 -- 172.21.15.115:0/2375725645 learned_addr learned my addr 172.21.15.115:0/2375725645 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157a3d00 0x55bc158ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d5bfff700 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d52ffd700 1 -- 172.21.15.115:0/2375725645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157a3d00 msgr2=0x55bc158ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157a3d00 0x55bc158ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d52ffd700 1 -- 172.21.15.115:0/2375725645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 msgr2=0x55bc158a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d52ffd700 1 -- 172.21.15.115:0/2375725645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc1575f500 con 0x55bc157294a0 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d5bfff700 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157294a0 0x55bc158ac780 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1d440a59a0 tx=0x7f1d440a5130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645af2020c22a9c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.298+0000 7f1d51ffb700 1 -- 172.21.15.115:0/2375725645 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d440aa020 con 0x55bc157294a0 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.299+0000 7f1d51ffb700 1 -- 172.21.15.115:0/2375725645 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d440a9920 con 0x55bc157294a0 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.299+0000 7f1d51ffb700 1 -- 172.21.15.115:0/2375725645 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d440a9b80 con 0x55bc157294a0 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.299+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2375725645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157294a0 msgr2=0x55bc158ac780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.299+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157294a0 0x55bc158ac780 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1d440a59a0 tx=0x7f1d440a5130 comp rx=0 tx=0).stop 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.299+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2375725645 shutdown_connections 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.299+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.299+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157a3d00 0x55bc158ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.299+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2375725645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157294a0 0x55bc158ac780 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.299+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2375725645 >> 172.21.15.115:0/2375725645 conn(0x55bc1572cf70 msgr2=0x55bc158a1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2375725645 shutdown_connections 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2375725645 wait complete. 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 -- start start 2024-05-09T05:44:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 Processor -- start 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bc158a7100 con 0x55bc157a3d00 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bc15763b20 con 0x55bc158a6400 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.300+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bc158bc1d0 con 0x55bc157294a0 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d707e3b80 10 client.0 _ll_register_callbacks cb 0x55bc1563a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d52ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d52ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50078/0 (socket says 172.21.15.115:50078) 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d52ffd700 1 -- 172.21.15.115:0/2368992938 learned_addr learned my addr 172.21.15.115:0/2368992938 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d5bfff700 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d52ffd700 1 -- 172.21.15.115:0/2368992938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 msgr2=0x55bc158afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d52ffd700 1 -- 172.21.15.115:0/2368992938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 msgr2=0x55bc158b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d52ffd700 1 -- 172.21.15.115:0/2368992938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc15644490 con 0x55bc157a3d00 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158b0530 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1d440e0390 tx=0x7f1d44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba39b5ab5ca81249 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d3affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d3affd700 1 -- 172.21.15.115:0/2368992938 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d440aa020 con 0x55bc157a3d00 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d3affd700 1 -- 172.21.15.115:0/2368992938 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1d440a9d90 con 0x55bc157a3d00 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.301+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2368992938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d440a9920 con 0x55bc157a3d00 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d3affd700 1 -- 172.21.15.115:0/2368992938 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1d44084e60 con 0x55bc157a3d00 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d707e3b80 10 client.17292 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2368992938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bc158bda40 con 0x55bc157a3d00 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d3affd700 1 -- 172.21.15.115:0/2368992938 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d440a9920 con 0x55bc157a3d00 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d3affd700 1 -- 172.21.15.115:0/2368992938 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1d440c7020 con 0x55bc157a3d00 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d707e3b80 10 client.17292 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d707e3b80 10 client.17292 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:44:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d707e3b80 20 client.17292 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55bc158bff10 0x55bc158c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d707e3b80 4 client.17292 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2368992938 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc157d6a80 con 0x55bc158bff10 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.302+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55bc158bff10 0x55bc158c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.303+0000 7f1d3affd700 10 client.17292 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.303+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55bc158bff10 0x55bc158c2330 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ada7143e31aa533c server_cookie=8ae02f5ddd0d252b in_seq=0 out_seq=0 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d3affd700 1 -- 172.21.15.115:0/2368992938 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc157d6a80 con 0x55bc158bff10 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d3affd700 10 client.17292 handle_command_reply: tid=0 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 client.17292 shutdown 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2368992938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 msgr2=0x55bc158b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158b0530 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1d440e0390 tx=0x7f1d44000bc0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2368992938 shutdown_connections 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55bc158bff10 0x55bc158c2330 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/2368992938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158b0530 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2368992938 >> 172.21.15.115:0/2368992938 conn(0x55bc1572cf70 msgr2=0x55bc1589aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:37.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d3affd700 0 client.17292 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:44:37.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2368992938 shutdown_connections 2024-05-09T05:44:37.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 1 -- 172.21.15.115:0/2368992938 wait complete. 2024-05-09T05:44:37.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.304+0000 7f1d707e3b80 20 client.17292 trim_cache size 0 max 16384 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d707e3b80 1 Processor -- start 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d707e3b80 1 -- start start 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158bc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc157a3d00 0x55bc158a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc158a6400 0x55bc158ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1b0400 con 0x55bc158a6400 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1b1f40 con 0x55bc157a3d00 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1b0ad0 con 0x55bc157294a0 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d5bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc158a6400 0x55bc158ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d5bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc158a6400 0x55bc158ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50086/0 (socket says 172.21.15.115:50086) 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d5bfff700 1 -- 172.21.15.115:0/1897494724 learned_addr learned my addr 172.21.15.115:0/1897494724 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.319+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc157a3d00 0x55bc158a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d5bfff700 1 -- 172.21.15.115:0/1897494724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 msgr2=0x55bc158bc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d5bfff700 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158bc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d5bfff700 1 -- 172.21.15.115:0/1897494724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc157a3d00 msgr2=0x55bc158a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d5bfff700 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc157a3d00 0x55bc158a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d5bfff700 1 -- 172.21.15.115:0/1897494724 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc158a2ec0 con 0x55bc158a6400 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc157a3d00 0x55bc158a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d5bfff700 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc158a6400 0x55bc158ad250 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009490 tx=0x7f1d540094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7b418fec30753a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d517fa700 1 -- 172.21.15.115:0/1897494724 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5401a020 con 0x55bc158a6400 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d517fa700 1 -- 172.21.15.115:0/1897494724 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d540168d0 con 0x55bc158a6400 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.320+0000 7f1d517fa700 1 -- 172.21.15.115:0/1897494724 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54015860 con 0x55bc158a6400 2024-05-09T05:44:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 -- 172.21.15.115:0/1897494724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc158a6400 msgr2=0x55bc158ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc158a6400 0x55bc158ad250 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009490 tx=0x7f1d540094c0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 -- 172.21.15.115:0/1897494724 shutdown_connections 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc158a6400 0x55bc158ad250 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158bc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/1897494724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc157a3d00 0x55bc158a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 -- 172.21.15.115:0/1897494724 >> 172.21.15.115:0/1897494724 conn(0x55bc1572cf70 msgr2=0x55bc158a53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 -- 172.21.15.115:0/1897494724 shutdown_connections 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 -- 172.21.15.115:0/1897494724 wait complete. 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 -- start start 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.321+0000 7f1d707e3b80 1 Processor -- start 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d707e3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1b1870 con 0x55bc157a3d00 2024-05-09T05:44:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d52ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d52ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50102/0 (socket says 172.21.15.115:50102) 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d707e3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1b11a0 con 0x55bc158a6400 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d707e3b80 1 -- 172.21.15.115:0/3678470137 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1ae3a0 con 0x55bc157294a0 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d52ffd700 1 -- 172.21.15.115:0/3678470137 learned_addr learned my addr 172.21.15.115:0/3678470137 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d707e3b80 10 client.0 _ll_register_callbacks cb 0x55bc1563a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d5bfff700 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d52ffd700 1 -- 172.21.15.115:0/3678470137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 msgr2=0x55bc158c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d52ffd700 1 -- 172.21.15.115:0/3678470137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 msgr2=0x55bc158c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d52ffd700 1 -- 172.21.15.115:0/3678470137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc15922640 con 0x55bc157a3d00 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d52ffd700 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158c42f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1d440e0310 tx=0x7f1d44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb0fcd6b08debf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.322+0000 7f1d3affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d3affd700 1 -- 172.21.15.115:0/3678470137 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d440aa020 con 0x55bc157a3d00 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d3affd700 1 -- 172.21.15.115:0/3678470137 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f1d440a5f10 con 0x55bc157a3d00 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d707e3b80 1 -- 172.21.15.115:0/3678470137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d540168d0 con 0x55bc157a3d00 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d3affd700 1 -- 172.21.15.115:0/3678470137 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1d440a8b80 con 0x55bc157a3d00 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d707e3b80 10 client.17298 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d707e3b80 1 -- 172.21.15.115:0/3678470137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bc15923810 con 0x55bc157a3d00 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d3affd700 1 -- 172.21.15.115:0/3678470137 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d540168d0 con 0x55bc157a3d00 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d3affd700 1 -- 172.21.15.115:0/3678470137 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1d440c7040 con 0x55bc157a3d00 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d707e3b80 10 client.17298 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d707e3b80 10 client.17298 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d707e3b80 20 client.17298 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55bc15923cf0 0x55bc15930870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d707e3b80 4 client.17298 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d707e3b80 1 -- 172.21.15.115:0/3678470137 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc157d0b20 con 0x55bc15923cf0 2024-05-09T05:44:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.323+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55bc15923cf0 0x55bc15930870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d3affd700 10 client.17298 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d5b7fe700 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55bc15923cf0 0x55bc15930870 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec69678ac677922b server_cookie=6f7f973885455a72 in_seq=0 out_seq=0 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d3affd700 1 -- 172.21.15.115:0/3678470137 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc157d0b20 con 0x55bc15923cf0 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d3affd700 10 client.17298 handle_command_reply: tid=0 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 client.17298 shutdown 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 -- 172.21.15.115:0/3678470137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 msgr2=0x55bc158c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158c42f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1d440e0310 tx=0x7f1d44004750 comp rx=0 tx=0).stop 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 -- 172.21.15.115:0/3678470137 shutdown_connections 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55bc15923cf0 0x55bc15930870 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc158a6400 0x55bc158c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bc157294a0 0x55bc158c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 --2- 172.21.15.115:0/3678470137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bc157a3d00 0x55bc158c42f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 -- 172.21.15.115:0/3678470137 >> 172.21.15.115:0/3678470137 conn(0x55bc1572cf70 msgr2=0x55bc158a9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:37.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d3affd700 0 client.17298 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:44:37.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 -- 172.21.15.115:0/3678470137 shutdown_connections 2024-05-09T05:44:37.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 1 -- 172.21.15.115:0/3678470137 wait complete. 2024-05-09T05:44:37.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.324+0000 7f1d707e3b80 20 client.17298 trim_cache size 0 max 16384 2024-05-09T05:44:37.324 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d480b29f0 msgr2=0x7f1d480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d480b29f0 0x7f1d480b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009990 tx=0x7f1d4c005fb0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c107e20 msgr2=0x7f1d5c19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c107e20 0x7f1d5c19c1b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400c9e0 tx=0x7f1d4400ccf0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1979451651 shutdown_connections 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1d480b29f0 0x7f1d480b4f00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c103c20 0x7f1d5c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d5c108770 0x7f1d5c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 --2- 172.21.15.115:0/1979451651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d5c107e20 0x7f1d5c19c1b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1979451651 >> 172.21.15.115:0/1979451651 conn(0x7f1d5c0fe6f0 msgr2=0x7f1d5c109680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:37.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1979451651 shutdown_connections 2024-05-09T05:44:37.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:37.325+0000 7f1d61dcd700 1 -- 172.21.15.115:0/1979451651 wait complete. 2024-05-09T05:44:37.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:37 smithi161 ceph-mon[33805]: pgmap v195: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-09T05:44:37.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:37 smithi195 ceph-mon[33844]: pgmap v195: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-09T05:44:38.062 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:44:38.306 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:39.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:39 smithi115 ceph-mon[30349]: pgmap v196: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-05-09T05:44:39.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:39 smithi161 ceph-mon[33805]: pgmap v196: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-05-09T05:44:39.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:39 smithi195 ceph-mon[33844]: pgmap v196: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-05-09T05:44:39.821 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:44:40.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.159+0000 7f023c6aa700 1 -- 172.21.15.115:0/1032521158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0234107df0 msgr2=0x7f0234108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:40.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.159+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1032521158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0234107df0 0x7f0234108200 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f02280099c0 tx=0x7f0228009cd0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.160+0000 7f023c6aa700 1 -- 172.21.15.115:0/1032521158 shutdown_connections 2024-05-09T05:44:40.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.160+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1032521158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0234108740 0x7f02341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.160+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1032521158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f02341034c0 0x7f02341038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.160+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1032521158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0234107df0 0x7f0234108200 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.160+0000 7f023c6aa700 1 -- 172.21.15.115:0/1032521158 >> 172.21.15.115:0/1032521158 conn(0x7f02340fedc0 msgr2=0x7f02341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:40.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.160+0000 7f023c6aa700 1 -- 172.21.15.115:0/1032521158 shutdown_connections 2024-05-09T05:44:40.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.160+0000 7f023c6aa700 1 -- 172.21.15.115:0/1032521158 wait complete. 2024-05-09T05:44:40.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.160+0000 7f023c6aa700 1 Processor -- start 2024-05-09T05:44:40.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.160+0000 7f023c6aa700 1 -- start start 2024-05-09T05:44:40.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023c6aa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02341034c0 0x7f023419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023c6aa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0234107df0 0x7f023419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023c6aa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0234108740 0x7f023419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023c6aa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0234113270 con 0x7f02341034c0 2024-05-09T05:44:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023c6aa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f02341130f0 con 0x7f0234108740 2024-05-09T05:44:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023c6aa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f02341133f0 con 0x7f0234107df0 2024-05-09T05:44:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023a446700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02341034c0 0x7f023419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023a446700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02341034c0 0x7f023419bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50114/0 (socket says 172.21.15.115:50114) 2024-05-09T05:44:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023a446700 1 -- 172.21.15.115:0/1091294256 learned_addr learned my addr 172.21.15.115:0/1091294256 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023ac47700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0234108740 0x7f023419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f0239c45700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0234107df0 0x7f023419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023a446700 1 -- 172.21.15.115:0/1091294256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0234107df0 msgr2=0x7f023419c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023a446700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0234107df0 0x7f023419c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023a446700 1 -- 172.21.15.115:0/1091294256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0234108740 msgr2=0x7f023419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023a446700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0234108740 0x7f023419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023a446700 1 -- 172.21.15.115:0/1091294256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02341a7000 con 0x7f02341034c0 2024-05-09T05:44:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f0239c45700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0234107df0 0x7f023419c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:40.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.161+0000 7f023a446700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02341034c0 0x7f023419bcb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f022c00ba50 tx=0x7f022c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:40.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.162+0000 7f02337fe700 1 -- 172.21.15.115:0/1091294256 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c019070 con 0x7f02341034c0 2024-05-09T05:44:40.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.162+0000 7f02337fe700 1 -- 172.21.15.115:0/1091294256 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f022c00ce40 con 0x7f02341034c0 2024-05-09T05:44:40.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.162+0000 7f023c6aa700 1 -- 172.21.15.115:0/1091294256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02341a72f0 con 0x7f02341034c0 2024-05-09T05:44:40.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.162+0000 7f02337fe700 1 -- 172.21.15.115:0/1091294256 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c007370 con 0x7f02341034c0 2024-05-09T05:44:40.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.162+0000 7f023c6aa700 1 -- 172.21.15.115:0/1091294256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02341a77e0 con 0x7f02341034c0 2024-05-09T05:44:40.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.165+0000 7f02337fe700 1 -- 172.21.15.115:0/1091294256 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f022c01c070 con 0x7f02341034c0 2024-05-09T05:44:40.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.165+0000 7f02337fe700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f021c0b29f0 0x7f021c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f024b0c0b80 1 Processor -- start 2024-05-09T05:44:40.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f0239c45700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f021c0b29f0 0x7f021c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f02337fe700 1 -- 172.21.15.115:0/1091294256 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f022c0e1e20 con 0x7f02341034c0 2024-05-09T05:44:40.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f024b0c0b80 1 -- start start 2024-05-09T05:44:40.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f0239c45700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f021c0b29f0 0x7f021c0b4f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0228000c00 tx=0x7f0228005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:40.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a91935810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556a918580a0 con 0x556a91832d00 2024-05-09T05:44:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556a917fd9a0 con 0x556a917b84a0 2024-05-09T05:44:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556a91854060 con 0x556a91935400 2024-05-09T05:44:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f023a446700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f023a446700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50124/0 (socket says 172.21.15.115:50124) 2024-05-09T05:44:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f023a446700 1 -- 172.21.15.115:0/365033551 learned_addr learned my addr 172.21.15.115:0/365033551 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.166+0000 7f023ac47700 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a91935810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f0239c45700 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f023a446700 1 -- 172.21.15.115:0/365033551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 msgr2=0x556a91935810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f023a446700 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a91935810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f023a446700 1 -- 172.21.15.115:0/365033551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 msgr2=0x556a9193b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f023a446700 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f023a446700 1 -- 172.21.15.115:0/365033551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a917ee500 con 0x556a91832d00 2024-05-09T05:44:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f0239c45700 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f023a446700 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193b240 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f022c0df830 tx=0x7f022c0a8ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0e07ea801ba6b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f0230ff9700 1 -- 172.21.15.115:0/365033551 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c0ad280 con 0x556a91832d00 2024-05-09T05:44:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f0230ff9700 1 -- 172.21.15.115:0/365033551 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f022c0cb070 con 0x556a91832d00 2024-05-09T05:44:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.167+0000 7f0230ff9700 1 -- 172.21.15.115:0/365033551 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c0a9250 con 0x556a91832d00 2024-05-09T05:44:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 -- 172.21.15.115:0/365033551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 msgr2=0x556a9193b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193b240 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f022c0df830 tx=0x7f022c0a8ab0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 -- 172.21.15.115:0/365033551 shutdown_connections 2024-05-09T05:44:40.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a91935810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193b240 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/365033551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 -- 172.21.15.115:0/365033551 >> 172.21.15.115:0/365033551 conn(0x556a917bbf70 msgr2=0x556a91930220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:40.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 -- 172.21.15.115:0/365033551 shutdown_connections 2024-05-09T05:44:40.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 -- 172.21.15.115:0/365033551 wait complete. 2024-05-09T05:44:40.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 -- start start 2024-05-09T05:44:40.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.168+0000 7f024b0c0b80 1 Processor -- start 2024-05-09T05:44:40.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91832d00 0x556a9193f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91935400 0x556a9193fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556a91936160 con 0x556a91935400 2024-05-09T05:44:40.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556a917f2b20 con 0x556a917b84a0 2024-05-09T05:44:40.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556a91942020 con 0x556a91832d00 2024-05-09T05:44:40.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f024b0c0b80 10 client.0 _ll_register_callbacks cb 0x556a916c96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:40.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023ac47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91935400 0x556a9193fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023ac47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91935400 0x556a9193fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50136/0 (socket says 172.21.15.115:50136) 2024-05-09T05:44:40.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023ac47700 1 -- 172.21.15.115:0/3442523868 learned_addr learned my addr 172.21.15.115:0/3442523868 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023a446700 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f0239c45700 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91832d00 0x556a9193f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023ac47700 1 -- 172.21.15.115:0/3442523868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91832d00 msgr2=0x556a9193f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023ac47700 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91832d00 0x556a9193f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023ac47700 1 -- 172.21.15.115:0/3442523868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 msgr2=0x556a9193f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023ac47700 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023ac47700 1 -- 172.21.15.115:0/3442523868 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a916d3490 con 0x556a91935400 2024-05-09T05:44:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f0239c45700 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91832d00 0x556a9193f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f023ac47700 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91935400 0x556a9193fb00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0224003950 tx=0x7f0224003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59c4c42da37d061 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f0219ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.169+0000 7f0219ffb700 1 -- 172.21.15.115:0/3442523868 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224003ea0 con 0x556a91935400 2024-05-09T05:44:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f0219ffb700 1 -- 172.21.15.115:0/3442523868 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f022400bc90 con 0x556a91935400 2024-05-09T05:44:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3442523868 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f022c0cb070 con 0x556a91935400 2024-05-09T05:44:40.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f0219ffb700 1 -- 172.21.15.115:0/3442523868 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0224024050 con 0x556a91935400 2024-05-09T05:44:40.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f024b0c0b80 10 client.17310 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:40.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3442523868 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556a9194c9e0 con 0x556a91935400 2024-05-09T05:44:40.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f0219ffb700 1 -- 172.21.15.115:0/3442523868 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f022c0cb070 con 0x556a91935400 2024-05-09T05:44:40.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f0219ffb700 1 -- 172.21.15.115:0/3442523868 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f0224011440 con 0x556a91935400 2024-05-09T05:44:40.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f024b0c0b80 10 client.17310 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:40.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f024b0c0b80 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f024b0c0b80 20 client.17310 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556a9194f020 0x556a91951490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f024b0c0b80 4 client.17310 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.170+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3442523868 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a91865a80 con 0x556a9194f020 2024-05-09T05:44:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.171+0000 7f023a446700 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556a9194f020 0x556a91951490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.171+0000 7f0219ffb700 10 client.17310 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.171+0000 7f023a446700 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556a9194f020 0x556a91951490 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6533709c957c96 server_cookie=124f34e6d2f7d435 in_seq=0 out_seq=0 2024-05-09T05:44:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f0219ffb700 1 -- 172.21.15.115:0/3442523868 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a91865a80 con 0x556a9194f020 2024-05-09T05:44:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f0219ffb700 10 client.17310 handle_command_reply: tid=0 2024-05-09T05:44:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 client.17310 shutdown 2024-05-09T05:44:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3442523868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91935400 msgr2=0x556a9193fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91935400 0x556a9193fb00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0224003950 tx=0x7f0224003c60 comp rx=0 tx=0).stop 2024-05-09T05:44:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3442523868 shutdown_connections 2024-05-09T05:44:40.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556a9194f020 0x556a91951490 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91935400 0x556a9193fb00 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f0219ffb700 0 client.17310 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:40.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3442523868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91832d00 0x556a9193f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3442523868 >> 172.21.15.115:0/3442523868 conn(0x556a917bbf70 msgr2=0x556a91929ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:40.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3442523868 shutdown_connections 2024-05-09T05:44:40.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3442523868 wait complete. 2024-05-09T05:44:40.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.172+0000 7f024b0c0b80 20 client.17310 trim_cache size 0 max 16384 2024-05-09T05:44:40.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.187+0000 7f024b0c0b80 1 Processor -- start 2024-05-09T05:44:40.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.187+0000 7f024b0c0b80 1 -- start start 2024-05-09T05:44:40.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.187+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.187+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.187+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a9192a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.187+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b04c0 con 0x556a91832d00 2024-05-09T05:44:40.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.187+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b2000 con 0x556a917b84a0 2024-05-09T05:44:40.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.187+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b0b90 con 0x556a91935400 2024-05-09T05:44:40.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f0239c45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f0239c45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50144/0 (socket says 172.21.15.115:50144) 2024-05-09T05:44:40.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f0239c45700 1 -- 172.21.15.115:0/3053315342 learned_addr learned my addr 172.21.15.115:0/3053315342 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:40.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f0239c45700 1 -- 172.21.15.115:0/3053315342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 msgr2=0x556a9192a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f0239c45700 1 --2- 172.21.15.115:0/3053315342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a9192a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f0239c45700 1 -- 172.21.15.115:0/3053315342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 msgr2=0x556a9193e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f0239c45700 1 --2- 172.21.15.115:0/3053315342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f0239c45700 1 -- 172.21.15.115:0/3053315342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a919418d0 con 0x556a91832d00 2024-05-09T05:44:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f0239c45700 1 --2- 172.21.15.115:0/3053315342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193f940 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0234076ab0 tx=0x7f0228020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e56e76a9d9daeb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f021bfff700 1 -- 172.21.15.115:0/3053315342 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022800f840 con 0x556a91832d00 2024-05-09T05:44:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f021bfff700 1 -- 172.21.15.115:0/3053315342 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0228033070 con 0x556a91832d00 2024-05-09T05:44:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.188+0000 7f021bfff700 1 -- 172.21.15.115:0/3053315342 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022802ec30 con 0x556a91832d00 2024-05-09T05:44:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3053315342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 msgr2=0x556a9193f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3053315342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193f940 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0234076ab0 tx=0x7f0228020000 comp rx=0 tx=0).stop 2024-05-09T05:44:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3053315342 shutdown_connections 2024-05-09T05:44:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3053315342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a917b84a0 0x556a9193e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3053315342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a9192a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/3053315342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a91832d00 0x556a9193f940 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3053315342 >> 172.21.15.115:0/3053315342 conn(0x556a917bbf70 msgr2=0x556a91929ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3053315342 shutdown_connections 2024-05-09T05:44:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 -- 172.21.15.115:0/3053315342 wait complete. 2024-05-09T05:44:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.189+0000 7f024b0c0b80 1 -- start start 2024-05-09T05:44:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f024b0c0b80 1 Processor -- start 2024-05-09T05:44:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a917b84a0 0x556a91947150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a91832d00 0x556a91947690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f024b0c0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a91955c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b1930 con 0x556a917b84a0 2024-05-09T05:44:40.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b1260 con 0x556a91832d00 2024-05-09T05:44:40.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f024b0c0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f02341ae460 con 0x556a91935400 2024-05-09T05:44:40.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f023a446700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a917b84a0 0x556a91947150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f023a446700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a917b84a0 0x556a91947150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50150/0 (socket says 172.21.15.115:50150) 2024-05-09T05:44:40.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f023a446700 1 -- 172.21.15.115:0/378974085 learned_addr learned my addr 172.21.15.115:0/378974085 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:40.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.190+0000 7f023ac47700 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a91955c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f024b0c0b80 10 client.0 _ll_register_callbacks cb 0x556a91930880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:40.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f0239c45700 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a91832d00 0x556a91947690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f023a446700 1 -- 172.21.15.115:0/378974085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 msgr2=0x556a91955c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:40.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f023a446700 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a91955c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f023a446700 1 -- 172.21.15.115:0/378974085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a91832d00 msgr2=0x556a91947690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f023a446700 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a91832d00 0x556a91947690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f023a446700 1 -- 172.21.15.115:0/378974085 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a91957630 con 0x556a917b84a0 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f023a446700 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a917b84a0 0x556a91947150 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f022c0a8bd0 tx=0x7f022c0a8ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677a82f81a707613 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f0219ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f0219ffb700 1 -- 172.21.15.115:0/378974085 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c014660 con 0x556a917b84a0 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f0219ffb700 1 -- 172.21.15.115:0/378974085 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f022c0a54c0 con 0x556a917b84a0 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f024b0c0b80 1 -- 172.21.15.115:0/378974085 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0228033070 con 0x556a917b84a0 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f0219ffb700 1 -- 172.21.15.115:0/378974085 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f022c014800 con 0x556a917b84a0 2024-05-09T05:44:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f024b0c0b80 10 client.17316 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.191+0000 7f024b0c0b80 1 -- 172.21.15.115:0/378974085 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556a919b8cf0 con 0x556a917b84a0 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f0219ffb700 1 -- 172.21.15.115:0/378974085 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0228033070 con 0x556a917b84a0 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f0219ffb700 1 -- 172.21.15.115:0/378974085 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f022c0a4840 con 0x556a917b84a0 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f024b0c0b80 10 client.17316 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f024b0c0b80 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f024b0c0b80 20 client.17316 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556a9194f020 0x556a919baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f024b0c0b80 4 client.17316 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f024b0c0b80 1 -- 172.21.15.115:0/378974085 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a9185fb20 con 0x556a9194f020 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f0239c45700 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556a9194f020 0x556a919baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.192+0000 7f0219ffb700 10 client.17316 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f0239c45700 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556a9194f020 0x556a919baf90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01dd4394c3bd907 server_cookie=ac018fdb051363ee in_seq=0 out_seq=0 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f0219ffb700 1 -- 172.21.15.115:0/378974085 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a9185fb20 con 0x556a9194f020 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f0219ffb700 10 client.17316 handle_command_reply: tid=0 2024-05-09T05:44:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 client.17316 shutdown 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 -- 172.21.15.115:0/378974085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a917b84a0 msgr2=0x556a91947150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a917b84a0 0x556a91947150 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f022c0a8bd0 tx=0x7f022c0a8ca0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 -- 172.21.15.115:0/378974085 shutdown_connections 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556a9194f020 0x556a919baf90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a91935400 0x556a91955c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556a917b84a0 0x556a91947150 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 --2- 172.21.15.115:0/378974085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a91832d00 0x556a91947690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f0219ffb700 0 client.17316 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 -- 172.21.15.115:0/378974085 >> 172.21.15.115:0/378974085 conn(0x556a917bbf70 msgr2=0x556a9193e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 -- 172.21.15.115:0/378974085 shutdown_connections 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 1 -- 172.21.15.115:0/378974085 wait complete. 2024-05-09T05:44:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.193+0000 7f024b0c0b80 20 client.17316 trim_cache size 0 max 16384 2024-05-09T05:44:40.209 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 -- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f021c0b29f0 msgr2=0x7f021c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f021c0b29f0 0x7f021c0b4f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0228000c00 tx=0x7f0228005fb0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 -- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02341034c0 msgr2=0x7f023419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02341034c0 0x7f023419bcb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f022c00ba50 tx=0x7f022c00bd60 comp rx=0 tx=0).stop 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 -- 172.21.15.115:0/1091294256 shutdown_connections 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f021c0b29f0 0x7f021c0b4f00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0234108740 0x7f023419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02341034c0 0x7f023419bcb0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 --2- 172.21.15.115:0/1091294256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0234107df0 0x7f023419c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 -- 172.21.15.115:0/1091294256 >> 172.21.15.115:0/1091294256 conn(0x7f02340fedc0 msgr2=0x7f02341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 -- 172.21.15.115:0/1091294256 shutdown_connections 2024-05-09T05:44:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:40.194+0000 7f023c6aa700 1 -- 172.21.15.115:0/1091294256 wait complete. 2024-05-09T05:44:40.983 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:41.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:41 smithi115 ceph-mon[30349]: pgmap v197: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-05-09T05:44:41.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:41 smithi161 ceph-mon[33805]: pgmap v197: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-05-09T05:44:41.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:41 smithi195 ceph-mon[33844]: pgmap v197: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-05-09T05:44:41.939 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:44:41.939 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:44:42.232 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:42.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.937+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2691010932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 msgr2=0x7fbeb81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:42.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.937+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2691010932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 0x7fbeb81100a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000b0d0 tx=0x7fbeb000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:44:42.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.937+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2691010932 shutdown_connections 2024-05-09T05:44:42.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.937+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2691010932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb81034c0 0x7fbeb81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:42.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.937+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2691010932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 0x7fbeb81100a0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:42.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.937+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2691010932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbeb8107df0 0x7fbeb8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:42.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.937+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2691010932 >> 172.21.15.115:0/2691010932 conn(0x7fbeb80fedc0 msgr2=0x7fbeb81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.937+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2691010932 shutdown_connections 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.937+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2691010932 wait complete. 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebf9ec700 1 Processor -- start 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebf9ec700 1 -- start start 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebf9ec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb81034c0 0x7fbeb819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebf9ec700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbeb8107df0 0x7fbeb819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebf9ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 0x7fbeb81999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebf9ec700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8113220 con 0x7fbeb8108740 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebf9ec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81130a0 con 0x7fbeb81034c0 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebf9ec700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81133a0 con 0x7fbeb8107df0 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebdf89700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 0x7fbeb81999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebdf89700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 0x7fbeb81999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42660/0 (socket says 172.21.15.115:42660) 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebdf89700 1 -- 172.21.15.115:0/2227801019 learned_addr learned my addr 172.21.15.115:0/2227801019 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:42.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.938+0000 7fbebd788700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb81034c0 0x7fbeb819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebcf87700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbeb8107df0 0x7fbeb819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebdf89700 1 -- 172.21.15.115:0/2227801019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbeb8107df0 msgr2=0x7fbeb819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebdf89700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbeb8107df0 0x7fbeb819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebdf89700 1 -- 172.21.15.115:0/2227801019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb81034c0 msgr2=0x7fbeb819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebdf89700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb81034c0 0x7fbeb819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebdf89700 1 -- 172.21.15.115:0/2227801019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb819a090 con 0x7fbeb8108740 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebcf87700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbeb8107df0 0x7fbeb819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebdf89700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 0x7fbeb81999e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009850 tx=0x7fbeb0007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbeae7fc700 1 -- 172.21.15.115:0/2227801019 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb001f070 con 0x7fbeb8108740 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbeae7fc700 1 -- 172.21.15.115:0/2227801019 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbeb0005500 con 0x7fbeb8108740 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb819a320 con 0x7fbeb8108740 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbeae7fc700 1 -- 172.21.15.115:0/2227801019 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb001b9a0 con 0x7fbeb8108740 2024-05-09T05:44:42.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.939+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb819a800 con 0x7fbeb8108740 2024-05-09T05:44:42.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.940+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb80512c0 con 0x7fbeb8108740 2024-05-09T05:44:42.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.942+0000 7fbeae7fc700 1 -- 172.21.15.115:0/2227801019 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fbeb0022070 con 0x7fbeb8108740 2024-05-09T05:44:42.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.942+0000 7fbeae7fc700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbea40b29f0 0x7fbea40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:42.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.942+0000 7fbebd788700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbea40b29f0 0x7fbea40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:42.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.943+0000 7fbeae7fc700 1 -- 172.21.15.115:0/2227801019 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7fbeb000f020 con 0x7fbeb8108740 2024-05-09T05:44:42.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.943+0000 7fbebd788700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbea40b29f0 0x7fbea40b4f00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbeb819ab20 tx=0x7fbeb400b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:42.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:42.946+0000 7fbeae7fc700 1 -- 172.21.15.115:0/2227801019 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbeb0029020 con 0x7fbeb8108740 2024-05-09T05:44:43.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.209+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeb810b1d0 con 0x7fbeb8108740 2024-05-09T05:44:43.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.210+0000 7fbeae7fc700 1 -- 172.21.15.115:0/2227801019 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11953 (secure 0 0 0) 0x7fbeb00ab940 con 0x7fbeb8108740 2024-05-09T05:44:43.204 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:44:43.204 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:43:58.362294+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:44:43.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbea40b29f0 msgr2=0x7fbea40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:43.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbea40b29f0 0x7fbea40b4f00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbeb819ab20 tx=0x7fbeb400b410 comp rx=0 tx=0).stop 2024-05-09T05:44:43.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 msgr2=0x7fbeb81999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:43.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 0x7fbeb81999e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009850 tx=0x7fbeb0007d90 comp rx=0 tx=0).stop 2024-05-09T05:44:43.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 shutdown_connections 2024-05-09T05:44:43.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbea40b29f0 0x7fbea40b4f00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:43.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb81034c0 0x7fbeb819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:43.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb8108740 0x7fbeb81999e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:43.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 --2- 172.21.15.115:0/2227801019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbeb8107df0 0x7fbeb819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:43.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 >> 172.21.15.115:0/2227801019 conn(0x7fbeb80fedc0 msgr2=0x7fbeb8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:43.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 shutdown_connections 2024-05-09T05:44:43.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:43.212+0000 7fbebf9ec700 1 -- 172.21.15.115:0/2227801019 wait complete. 2024-05-09T05:44:43.207 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 38 2024-05-09T05:44:43.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:43 smithi115 ceph-mon[30349]: pgmap v198: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-05-09T05:44:43.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:43 smithi161 ceph-mon[33805]: pgmap v198: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-05-09T05:44:43.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:43 smithi195 ceph-mon[33844]: pgmap v198: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-05-09T05:44:44.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:44 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2227801019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:44.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:44 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2227801019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:44.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:44 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2227801019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:44.786 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id c2030c2b --release --await' 2024-05-09T05:44:44.786 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id c2030c2b --release --await 2024-05-09T05:44:45.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.278+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/3783654783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 msgr2=0x7f6a9c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:45.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.278+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/3783654783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 0x7f6a9c1038f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6a900099c0 tx=0x7f6a90009cd0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.278+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/3783654783 shutdown_connections 2024-05-09T05:44:45.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.278+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/3783654783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a9c108760 0x7f6a9c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.278+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/3783654783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 0x7f6a9c1038f0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.278+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/3783654783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9c107e10 0x7f6a9c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.278+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/3783654783 >> 172.21.15.115:0/3783654783 conn(0x7f6a9c0fef30 msgr2=0x7f6a9c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:45.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.279+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/3783654783 shutdown_connections 2024-05-09T05:44:45.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.279+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/3783654783 wait complete. 2024-05-09T05:44:45.274 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:45 smithi115 ceph-mon[30349]: pgmap v199: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-05-09T05:44:45.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.279+0000 7f6aa2e1e700 1 Processor -- start 2024-05-09T05:44:45.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa2e1e700 1 -- start start 2024-05-09T05:44:45.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa2e1e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 0x7f6a9c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa2e1e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a9c107e10 0x7f6a9c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa2e1e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9c108760 0x7f6a9c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa2e1e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c113240 con 0x7f6a9c1034e0 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa2e1e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1130c0 con 0x7f6a9c107e10 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa2e1e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1133c0 con 0x7f6a9c108760 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa0bba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 0x7f6a9c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa0bba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 0x7f6a9c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42678/0 (socket says 172.21.15.115:42678) 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa0bba700 1 -- 172.21.15.115:0/710559288 learned_addr learned my addr 172.21.15.115:0/710559288 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9c108760 0x7f6a9c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.280+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a9c107e10 0x7f6a9c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6aa0bba700 1 -- 172.21.15.115:0/710559288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9c108760 msgr2=0x7f6a9c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9c108760 0x7f6a9c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6aa0bba700 1 -- 172.21.15.115:0/710559288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a9c107e10 msgr2=0x7f6a9c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:45.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a9c107e10 0x7f6a9c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6aa0bba700 1 -- 172.21.15.115:0/710559288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c19a060 con 0x7f6a9c1034e0 2024-05-09T05:44:45.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 0x7f6a9c19f800 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6a90000c00 tx=0x7f6a9000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:45.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6a99ffb700 1 -- 172.21.15.115:0/710559288 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90025070 con 0x7f6a9c1034e0 2024-05-09T05:44:45.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6a99ffb700 1 -- 172.21.15.115:0/710559288 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a9000fd40 con 0x7f6a9c1034e0 2024-05-09T05:44:45.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/710559288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c19a2f0 con 0x7f6a9c1034e0 2024-05-09T05:44:45.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6a99ffb700 1 -- 172.21.15.115:0/710559288 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90004d20 con 0x7f6a9c1034e0 2024-05-09T05:44:45.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.281+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/710559288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c19a820 con 0x7f6a9c1034e0 2024-05-09T05:44:45.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.285+0000 7f6a99ffb700 1 -- 172.21.15.115:0/710559288 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f6a90028070 con 0x7f6a9c1034e0 2024-05-09T05:44:45.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.285+0000 7f6ab1834b80 1 Processor -- start 2024-05-09T05:44:45.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6ab1834b80 1 -- start start 2024-05-09T05:44:45.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a99ffb700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a8c0b2a70 0x7f6a8c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15d481e0 0x55ac15de1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15c5e520 0x55ac15de1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15ddbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ac15cfe070 con 0x55ac15c5e520 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ac15c9c870 con 0x55ac15d481e0 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac15cfa030 con 0x55ac15ddb720 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a99ffb700 1 -- 172.21.15.115:0/710559288 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6a900e3510 con 0x7f6a9c1034e0 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a8c0b2a70 0x7f6a8c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a9bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15c5e520 0x55ac15de1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a9bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15c5e520 0x55ac15de1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42680/0 (socket says 172.21.15.115:42680) 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a9bfff700 1 -- 172.21.15.115:0/2629598647 learned_addr learned my addr 172.21.15.115:0/2629598647 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15d481e0 0x55ac15de1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a9bfff700 1 -- 172.21.15.115:0/2629598647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 msgr2=0x55ac15ddbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15ddbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15ddbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a9bfff700 1 -- 172.21.15.115:0/2629598647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15d481e0 msgr2=0x55ac15de1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:45.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15d481e0 0x55ac15de1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6a9bfff700 1 -- 172.21.15.115:0/2629598647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac15c943f0 con 0x55ac15c5e520 2024-05-09T05:44:45.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.286+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15d481e0 0x55ac15de1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:45.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.287+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15ddbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:45.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.287+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a8c0b2a70 0x7f6a8c0b4f80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a94007fd0 tx=0x7f6a94007950 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:45.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.287+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15c5e520 0x55ac15de1ab0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800aca0 tx=0x7f6a8800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5831f92a90f426d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:45.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.287+0000 7f6a86ffd700 1 -- 172.21.15.115:0/2629598647 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88004030 con 0x55ac15c5e520 2024-05-09T05:44:45.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.287+0000 7f6a86ffd700 1 -- 172.21.15.115:0/2629598647 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a88003680 con 0x55ac15c5e520 2024-05-09T05:44:45.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.287+0000 7f6a86ffd700 1 -- 172.21.15.115:0/2629598647 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88018690 con 0x55ac15c5e520 2024-05-09T05:44:45.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.288+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2629598647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15c5e520 msgr2=0x55ac15de1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:45.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.288+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15c5e520 0x55ac15de1ab0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800aca0 tx=0x7f6a8800f3c0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.288+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2629598647 shutdown_connections 2024-05-09T05:44:45.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.288+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15d481e0 0x55ac15de1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.288+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15ddbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.288+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2629598647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15c5e520 0x55ac15de1ab0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.288+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2629598647 >> 172.21.15.115:0/2629598647 conn(0x55ac15cbe720 msgr2=0x55ac15dd6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.288+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2629598647 shutdown_connections 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.288+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2629598647 wait complete. 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.289+0000 7f6ab1834b80 1 -- start start 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.289+0000 7f6ab1834b80 1 Processor -- start 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.289+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15de5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.289+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15d481e0 0x55ac15de57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.289+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15de5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.289+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ac15b79c20 con 0x55ac15d481e0 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.289+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ac15cb81d0 con 0x55ac15c5e520 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.289+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ac15de8250 con 0x55ac15ddb720 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6ab1834b80 10 client.0 _ll_register_callbacks cb 0x55ac15b6f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a9bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15d481e0 0x55ac15de57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a9bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15d481e0 0x55ac15de57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42688/0 (socket says 172.21.15.115:42688) 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a9bfff700 1 -- 172.21.15.115:0/3384388298 learned_addr learned my addr 172.21.15.115:0/3384388298 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15de5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15de5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a9bfff700 1 -- 172.21.15.115:0/3384388298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 msgr2=0x55ac15de5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15de5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a9bfff700 1 -- 172.21.15.115:0/3384388298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 msgr2=0x55ac15de5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15de5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a9bfff700 1 -- 172.21.15.115:0/3384388298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac15b79510 con 0x55ac15d481e0 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15de5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15d481e0 0x55ac15de57c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800ac70 tx=0x7f6a88003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975b8838972cb0a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.290+0000 7f6a7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6a7bfff700 1 -- 172.21.15.115:0/3384388298 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88026020 con 0x55ac15d481e0 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6a7bfff700 1 -- 172.21.15.115:0/3384388298 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6a880053e0 con 0x55ac15d481e0 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3384388298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a88003680 con 0x55ac15d481e0 2024-05-09T05:44:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6a7bfff700 1 -- 172.21.15.115:0/3384388298 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6a88031050 con 0x55ac15d481e0 2024-05-09T05:44:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6ab1834b80 10 client.17334 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3384388298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac15df2bd0 con 0x55ac15d481e0 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6a7bfff700 1 -- 172.21.15.115:0/3384388298 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a88003680 con 0x55ac15d481e0 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6a7bfff700 1 -- 172.21.15.115:0/3384388298 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6a88010430 con 0x55ac15d481e0 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6ab1834b80 10 client.17334 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6ab1834b80 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6ab1834b80 20 client.17334 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ac15df51b0 0x55ac15df75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6ab1834b80 4 client.17334 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.291+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3384388298 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac15d0ba50 con 0x55ac15df51b0 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.292+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ac15df51b0 0x55ac15df75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.292+0000 7f6a7bfff700 10 client.17334 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.292+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ac15df51b0 0x55ac15df75d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9f61cd95217e38 server_cookie=545dc66ddfa702ef in_seq=0 out_seq=0 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6a7bfff700 1 -- 172.21.15.115:0/3384388298 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac15d0ba50 con 0x55ac15df51b0 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6a7bfff700 10 client.17334 handle_command_reply: tid=0 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 client.17334 shutdown 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3384388298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15d481e0 msgr2=0x55ac15de57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15d481e0 0x55ac15de57c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800ac70 tx=0x7f6a88003c20 comp rx=0 tx=0).stop 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3384388298 shutdown_connections 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15de5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ac15df51b0 0x55ac15df75d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15ddb720 0x55ac15de5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3384388298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15d481e0 0x55ac15de57c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3384388298 >> 172.21.15.115:0/3384388298 conn(0x55ac15cbe720 msgr2=0x55ac15dd6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6a7bfff700 0 client.17334 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3384388298 shutdown_connections 2024-05-09T05:44:45.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3384388298 wait complete. 2024-05-09T05:44:45.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.293+0000 7f6ab1834b80 20 client.17334 trim_cache size 0 max 16384 2024-05-09T05:44:45.295 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.319+0000 7f6ab1834b80 1 Processor -- start 2024-05-09T05:44:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.319+0000 7f6ab1834b80 1 -- start start 2024-05-09T05:44:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15de45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 0x55ac15de5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15ddae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1adfb0 con 0x55ac15ddb720 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1afaf0 con 0x55ac15c5e520 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1ae680 con 0x55ac15d481e0 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa13bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15ddae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa13bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15ddae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42690/0 (socket says 172.21.15.115:42690) 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa13bb700 1 -- 172.21.15.115:0/3843993244 learned_addr learned my addr 172.21.15.115:0/3843993244 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15de45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 0x55ac15de5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa13bb700 1 -- 172.21.15.115:0/3843993244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 msgr2=0x55ac15de5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 0x55ac15de5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa13bb700 1 -- 172.21.15.115:0/3843993244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 msgr2=0x55ac15de45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15de45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa13bb700 1 -- 172.21.15.115:0/3843993244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac15dcffe0 con 0x55ac15ddb720 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15de45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.320+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15ddae20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6a9401ab40 tx=0x7f6a94010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfd1033ae717 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.321+0000 7f6a867fc700 1 -- 172.21.15.115:0/3843993244 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9401ca30 con 0x55ac15ddb720 2024-05-09T05:44:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.321+0000 7f6a867fc700 1 -- 172.21.15.115:0/3843993244 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a9402d070 con 0x55ac15ddb720 2024-05-09T05:44:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.321+0000 7f6a867fc700 1 -- 172.21.15.115:0/3843993244 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9401b370 con 0x55ac15ddb720 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.322+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3843993244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 msgr2=0x55ac15ddae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.322+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15ddae20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6a9401ab40 tx=0x7f6a94010040 comp rx=0 tx=0).stop 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.322+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3843993244 shutdown_connections 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.322+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15de45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.322+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15ddae20 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.322+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/3843993244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 0x55ac15de5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.322+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3843993244 >> 172.21.15.115:0/3843993244 conn(0x55ac15cbe720 msgr2=0x55ac15dda710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.322+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3843993244 shutdown_connections 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.323+0000 7f6ab1834b80 1 -- 172.21.15.115:0/3843993244 wait complete. 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.323+0000 7f6ab1834b80 1 -- start start 2024-05-09T05:44:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.323+0000 7f6ab1834b80 1 Processor -- start 2024-05-09T05:44:45.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15dfa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 0x55ac15df8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6ab1834b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15df8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1af420 con 0x55ac15ddb720 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1aed50 con 0x55ac15c5e520 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6ab1834b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1abf50 con 0x55ac15d481e0 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6ab1834b80 10 client.0 _ll_register_callbacks cb 0x55ac15de0fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6aa13bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15df8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6aa13bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15df8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42702/0 (socket says 172.21.15.115:42702) 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6aa13bb700 1 -- 172.21.15.115:0/2870095656 learned_addr learned my addr 172.21.15.115:0/2870095656 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15dfa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.324+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 0x55ac15df8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6aa13bb700 1 -- 172.21.15.115:0/2870095656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 msgr2=0x55ac15df8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 0x55ac15df8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6aa13bb700 1 -- 172.21.15.115:0/2870095656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 msgr2=0x55ac15dfa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15dfa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6aa13bb700 1 -- 172.21.15.115:0/2870095656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac15df94d0 con 0x55ac15ddb720 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6a9bfff700 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 0x55ac15df8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:45.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6aa13bb700 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15df8ec0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6a94028950 tx=0x7f6a94028980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee198beae5ff794c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6a7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6a7bfff700 1 -- 172.21.15.115:0/2870095656 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94035070 con 0x55ac15ddb720 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2870095656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a9402d070 con 0x55ac15ddb720 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6a7bfff700 1 -- 172.21.15.115:0/2870095656 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6a9401ca30 con 0x55ac15ddb720 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6a7bfff700 1 -- 172.21.15.115:0/2870095656 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6a94041050 con 0x55ac15ddb720 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6ab1834b80 10 client.17340 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.325+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2870095656 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac15e604f0 con 0x55ac15ddb720 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.326+0000 7f6a7bfff700 1 -- 172.21.15.115:0/2870095656 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a9402d070 con 0x55ac15ddb720 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.326+0000 7f6a7bfff700 1 -- 172.21.15.115:0/2870095656 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6a9401b050 con 0x55ac15ddb720 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.326+0000 7f6ab1834b80 10 client.17340 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.326+0000 7f6ab1834b80 10 client.17340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.326+0000 7f6ab1834b80 20 client.17340 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.326+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ac15e60b50 0x55ac15e62f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:45.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.326+0000 7f6ab1834b80 4 client.17340 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:44:45.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.326+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2870095656 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ac15d05af0 con 0x55ac15e60b50 2024-05-09T05:44:45.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.326+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ac15e60b50 0x55ac15e62f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:45.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.327+0000 7f6aa0bba700 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ac15e60b50 0x55ac15e62f70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472399e6c80cbfb2 server_cookie=5e9fd9c026a3dace in_seq=0 out_seq=0 2024-05-09T05:44:45.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.327+0000 7f6a7bfff700 10 client.17340 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:45.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6a7bfff700 1 -- 172.21.15.115:0/2870095656 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ac15d05af0 con 0x55ac15e60b50 2024-05-09T05:44:45.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6a7bfff700 10 client.17340 handle_command_reply: tid=0 2024-05-09T05:44:45.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 client.17340 shutdown 2024-05-09T05:44:45.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2870095656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 msgr2=0x55ac15df8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:45.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15df8ec0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6a94028950 tx=0x7f6a94028980 comp rx=0 tx=0).stop 2024-05-09T05:44:45.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2870095656 shutdown_connections 2024-05-09T05:44:45.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac15c5e520 0x55ac15dfa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ac15e60b50 0x55ac15e62f70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac15ddb720 0x55ac15df8ec0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 --2- 172.21.15.115:0/2870095656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac15d481e0 0x55ac15df8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2870095656 >> 172.21.15.115:0/2870095656 conn(0x55ac15cbe720 msgr2=0x55ac15de4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6a7bfff700 0 client.17340 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.328+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2870095656 shutdown_connections 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.329+0000 7f6ab1834b80 1 -- 172.21.15.115:0/2870095656 wait complete. 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.329+0000 7f6ab1834b80 20 client.17340 trim_cache size 0 max 16384 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:44:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:44:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "3ff255c9-91bf-40d4-8c31-c969402988f6", 2024-05-09T05:44:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:44:45.326 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.329+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a8c0b2a70 msgr2=0x7f6a8c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.329+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a8c0b2a70 0x7f6a8c0b4f80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6a94007fd0 tx=0x7f6a94007950 comp rx=0 tx=0).stop 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.329+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 msgr2=0x7f6a9c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.329+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 0x7f6a9c19f800 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6a90000c00 tx=0x7f6a9000f710 comp rx=0 tx=0).stop 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.329+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/710559288 shutdown_connections 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.330+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a8c0b2a70 0x7f6a8c0b4f80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.330+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a9c108760 0x7f6a9c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.330+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a9c1034e0 0x7f6a9c19f800 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.330+0000 7f6aa2e1e700 1 --2- 172.21.15.115:0/710559288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a9c107e10 0x7f6a9c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.330+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/710559288 >> 172.21.15.115:0/710559288 conn(0x7f6a9c0fef30 msgr2=0x7f6a9c1071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.330+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/710559288 shutdown_connections 2024-05-09T05:44:45.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:45.330+0000 7f6aa2e1e700 1 -- 172.21.15.115:0/710559288 wait complete. 2024-05-09T05:44:45.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:45 smithi161 ceph-mon[33805]: pgmap v199: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-05-09T05:44:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:45 smithi195 ceph-mon[33844]: pgmap v199: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-05-09T05:44:45.615 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 05:44:45 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T05:44:45.339+0000 7f9d69520700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c7 [...5,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ auth{1=1} v3945 ap=4 f(v36 m2024-05-09T05:40:55.664108+0000 18=18+0) n(v1 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1) old_inodes=1 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:388 nref=3)) caps={15339=pAsLsXs/-@178} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55d6cd1f4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c7:[<0x10000000007/.deps v2123>,<0x10000000006/ffsb v3080>,<0x10000000005/tmp v44>,<0x10000000003/client.0 v40>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v56>,<0x10000000001/sv_0 v71>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c7:[<0x10000000007/.deps v3945>,<0x10000000006/ffsb v3523>,<0x10000000005/tmp v121>,<0x10000000003/client.0 v114>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v134>,<0x10000000001/sv_0 v152>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-05-09T05:40:55.643149+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1)","memory_value.dirstat":"f(v36 m2024-05-09T05:40:55.664108+0000 18=18+0)","memory_value.rstat":"n(v1 rc2024-05-09T05:40:55.664108+0000 b126581 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-05-09T05:44:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:44:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:44:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:44:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:44:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:44:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:44:46.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:44:46.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:44:46.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.397 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:46 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:44:46.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:46 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:44:46.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:44:46.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:46.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:46 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:47.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:44:47.410 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:47 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:47.411 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:47 smithi115 ceph-mon[30349]: pgmap v200: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-09T05:44:47.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:47 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:47.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:47 smithi195 ceph-mon[33844]: pgmap v200: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-09T05:44:47.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:47 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:44:47.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:47 smithi161 ceph-mon[33805]: pgmap v200: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-09T05:44:47.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.809+0000 7f61267f0700 1 -- 172.21.15.115:0/546452668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612010e900 msgr2=0x7f6120110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:47.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.809+0000 7f61267f0700 1 --2- 172.21.15.115:0/546452668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612010e900 0x7f6120110de0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f611800b0a0 tx=0x7f611800b3b0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.809+0000 7f61267f0700 1 -- 172.21.15.115:0/546452668 shutdown_connections 2024-05-09T05:44:47.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.809+0000 7f61267f0700 1 --2- 172.21.15.115:0/546452668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612010e900 0x7f6120110de0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.809+0000 7f61267f0700 1 --2- 172.21.15.115:0/546452668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6120105520 0x7f6120105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.809+0000 7f61267f0700 1 --2- 172.21.15.115:0/546452668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6120105e70 0x7f612010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.809+0000 7f61267f0700 1 -- 172.21.15.115:0/546452668 >> 172.21.15.115:0/546452668 conn(0x7f61200fef90 msgr2=0x7f61201013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:47.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.810+0000 7f61267f0700 1 -- 172.21.15.115:0/546452668 shutdown_connections 2024-05-09T05:44:47.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.810+0000 7f61267f0700 1 -- 172.21.15.115:0/546452668 wait complete. 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f61267f0700 1 Processor -- start 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f61267f0700 1 -- start start 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f61267f0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6120105520 0x7f61201a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f61267f0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6120105e70 0x7f61201a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f61267f0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612010e900 0x7f612019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f61267f0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6120114300 con 0x7f6120105520 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f61267f0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6120114180 con 0x7f6120105e70 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f61267f0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6120114480 con 0x7f612010e900 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6120105520 0x7f61201a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6120105520 0x7f61201a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42726/0 (socket says 172.21.15.115:42726) 2024-05-09T05:44:47.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.811+0000 7f611ffff700 1 -- 172.21.15.115:0/236794149 learned_addr learned my addr 172.21.15.115:0/236794149 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611f7fe700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6120105e70 0x7f61201a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f6124d8d700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612010e900 0x7f612019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611ffff700 1 -- 172.21.15.115:0/236794149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612010e900 msgr2=0x7f612019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611ffff700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612010e900 0x7f612019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611ffff700 1 -- 172.21.15.115:0/236794149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6120105e70 msgr2=0x7f61201a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611ffff700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6120105e70 0x7f61201a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611ffff700 1 -- 172.21.15.115:0/236794149 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612019b170 con 0x7f6120105520 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611ffff700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6120105520 0x7f61201a08a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f611000d980 tx=0x7f611000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611d7fa700 1 -- 172.21.15.115:0/236794149 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110016070 con 0x7f6120105520 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611d7fa700 1 -- 172.21.15.115:0/236794149 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6110004830 con 0x7f6120105520 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f611d7fa700 1 -- 172.21.15.115:0/236794149 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100054c0 con 0x7f6120105520 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f61267f0700 1 -- 172.21.15.115:0/236794149 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612019b400 con 0x7f6120105520 2024-05-09T05:44:47.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.812+0000 7f61267f0700 1 -- 172.21.15.115:0/236794149 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6120103140 con 0x7f6120105520 2024-05-09T05:44:47.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.815+0000 7f611d7fa700 1 -- 172.21.15.115:0/236794149 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f6110004220 con 0x7f6120105520 2024-05-09T05:44:47.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.816+0000 7f611d7fa700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f610c0b29f0 0x7f610c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.816+0000 7f611f7fe700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f610c0b29f0 0x7f610c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.816+0000 7f611d7fa700 1 -- 172.21.15.115:0/236794149 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f61100e2680 con 0x7f6120105520 2024-05-09T05:44:47.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.816+0000 7f6135206b80 1 Processor -- start 2024-05-09T05:44:47.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.816+0000 7f6135206b80 1 -- start start 2024-05-09T05:44:47.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611f7fe700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f610c0b29f0 0x7f610c0b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6120102440 tx=0x7f610800b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f6135206b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc720260 0x562adc7b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f6135206b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc6365a0 0x562adc7b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f6135206b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f6135206b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562adc6d60f0 con 0x562adc720260 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f6135206b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562adc6748f0 con 0x562adc7b3b50 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f6135206b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562adc6d20b0 con 0x562adc6365a0 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc720260 0x562adc7b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc720260 0x562adc7b9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42738/0 (socket says 172.21.15.115:42738) 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611ffff700 1 -- 172.21.15.115:0/4074978324 learned_addr learned my addr 172.21.15.115:0/4074978324 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f6124d8d700 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611f7fe700 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc6365a0 0x562adc7b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611ffff700 1 -- 172.21.15.115:0/4074978324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc6365a0 msgr2=0x562adc7b9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611ffff700 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc6365a0 0x562adc7b9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611ffff700 1 -- 172.21.15.115:0/4074978324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 msgr2=0x562adc7b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:47.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611ffff700 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f611ffff700 1 -- 172.21.15.115:0/4074978324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562adc66c470 con 0x562adc720260 2024-05-09T05:44:47.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.817+0000 7f6124d8d700 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:47.812 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.818+0000 7f611ffff700 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc720260 0x562adc7b9a90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f61100a56a0 tx=0x7f61100a9200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa980cae3e88467 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.818+0000 7f61167fc700 1 -- 172.21.15.115:0/4074978324 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100ad9d0 con 0x562adc720260 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.818+0000 7f61167fc700 1 -- 172.21.15.115:0/4074978324 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61100b2480 con 0x562adc720260 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.818+0000 7f61167fc700 1 -- 172.21.15.115:0/4074978324 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110022350 con 0x562adc720260 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 -- 172.21.15.115:0/4074978324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc720260 msgr2=0x562adc7b9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc720260 0x562adc7b9a90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f61100a56a0 tx=0x7f61100a9200 comp rx=0 tx=0).stop 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 -- 172.21.15.115:0/4074978324 shutdown_connections 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc720260 0x562adc7b9a90 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 --2- 172.21.15.115:0/4074978324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc6365a0 0x562adc7b9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 -- 172.21.15.115:0/4074978324 >> 172.21.15.115:0/4074978324 conn(0x562adc6967a0 msgr2=0x562adc7aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 -- 172.21.15.115:0/4074978324 shutdown_connections 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 -- 172.21.15.115:0/4074978324 wait complete. 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.819+0000 7f6135206b80 1 -- start start 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f6135206b80 1 Processor -- start 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f6135206b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7bd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f6135206b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc720260 0x562adc7bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f6135206b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7be2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f6135206b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562adc551ca0 con 0x562adc6365a0 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f6135206b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562adc690250 con 0x562adc7b3b50 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f6135206b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562adc7b4910 con 0x562adc720260 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f6135206b80 10 client.0 _ll_register_callbacks cb 0x562adc5477c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7bd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f6124d8d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7be2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7bd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42742/0 (socket says 172.21.15.115:42742) 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f611ffff700 1 -- 172.21.15.115:0/1654409601 learned_addr learned my addr 172.21.15.115:0/1654409601 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.820+0000 7f611f7fe700 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc720260 0x562adc7bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f6124d8d700 1 -- 172.21.15.115:0/1654409601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc720260 msgr2=0x562adc7bdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f6124d8d700 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc720260 0x562adc7bdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f6124d8d700 1 -- 172.21.15.115:0/1654409601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 msgr2=0x562adc7bd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f6124d8d700 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7bd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f6124d8d700 1 -- 172.21.15.115:0/1654409601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562adc551590 con 0x562adc7b3b50 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f611ffff700 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7bd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f6124d8d700 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7be2d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6118009fd0 tx=0x7f6118004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab33fe851858448 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f60ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f60ff7fe700 1 -- 172.21.15.115:0/1654409601 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61180079d0 con 0x562adc7b3b50 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f60ff7fe700 1 -- 172.21.15.115:0/1654409601 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f6118007b70 con 0x562adc7b3b50 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f6135206b80 1 -- 172.21.15.115:0/1654409601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61100b2480 con 0x562adc7b3b50 2024-05-09T05:44:47.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f60ff7fe700 1 -- 172.21.15.115:0/1654409601 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6118007d70 con 0x562adc7b3b50 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f6135206b80 10 client.26702 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.821+0000 7f6135206b80 1 -- 172.21.15.115:0/1654409601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562adc7cb0e0 con 0x562adc7b3b50 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.822+0000 7f60ff7fe700 1 -- 172.21.15.115:0/1654409601 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61100b2480 con 0x562adc7b3b50 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.822+0000 7f60ff7fe700 1 -- 172.21.15.115:0/1654409601 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6118011070 con 0x562adc7b3b50 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.822+0000 7f6135206b80 10 client.26702 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.822+0000 7f6135206b80 10 client.26702 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.822+0000 7f6135206b80 20 client.26702 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.822+0000 7f6135206b80 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562adc7cd720 0x562adc7cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.822+0000 7f6135206b80 4 client.26702 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.822+0000 7f6135206b80 1 -- 172.21.15.115:0/1654409601 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562adc6e3ad0 con 0x562adc7cd720 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.822+0000 7f611ffff700 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562adc7cd720 0x562adc7cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.823+0000 7f60ff7fe700 10 client.26702 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:47.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.823+0000 7f611ffff700 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562adc7cd720 0x562adc7cfb60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76070fa4bb0db602 server_cookie=3d576f7ff9557d38 in_seq=0 out_seq=0 2024-05-09T05:44:47.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f60ff7fe700 1 -- 172.21.15.115:0/1654409601 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562adc6e3ad0 con 0x562adc7cd720 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f60ff7fe700 10 client.26702 handle_command_reply: tid=0 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 client.26702 shutdown 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 -- 172.21.15.115:0/1654409601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 msgr2=0x562adc7be2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7be2d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6118009fd0 tx=0x7f6118004cb0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 -- 172.21.15.115:0/1654409601 shutdown_connections 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc7b3b50 0x562adc7be2d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562adc7cd720 0x562adc7cfb60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7bd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 --2- 172.21.15.115:0/1654409601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc720260 0x562adc7bdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f60ff7fe700 0 client.26702 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:47.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 -- 172.21.15.115:0/1654409601 >> 172.21.15.115:0/1654409601 conn(0x562adc6967a0 msgr2=0x562adc7ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:47.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 -- 172.21.15.115:0/1654409601 shutdown_connections 2024-05-09T05:44:47.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 1 -- 172.21.15.115:0/1654409601 wait complete. 2024-05-09T05:44:47.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.824+0000 7f6135206b80 20 client.26702 trim_cache size 0 max 16384 2024-05-09T05:44:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f6135206b80 1 Processor -- start 2024-05-09T05:44:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f6135206b80 1 -- start start 2024-05-09T05:44:47.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f6135206b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f6135206b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 0x562adc7bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f6135206b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 0x562adc7d5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f6135206b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61201b25a0 con 0x562adc6365a0 2024-05-09T05:44:47.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f6135206b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f61201b40e0 con 0x562adc720260 2024-05-09T05:44:47.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f6135206b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61201b2c70 con 0x562adc7b3b50 2024-05-09T05:44:47.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42746/0 (socket says 172.21.15.115:42746) 2024-05-09T05:44:47.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f611ffff700 1 -- 172.21.15.115:0/728738709 learned_addr learned my addr 172.21.15.115:0/728738709 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:47.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f611f7fe700 1 --2- 172.21.15.115:0/728738709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 0x562adc7bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.853+0000 7f6124d8d700 1 --2- 172.21.15.115:0/728738709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 0x562adc7d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.854+0000 7f611ffff700 1 -- 172.21.15.115:0/728738709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 msgr2=0x562adc7d5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:47.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.854+0000 7f611ffff700 1 --2- 172.21.15.115:0/728738709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 0x562adc7d5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.854+0000 7f611ffff700 1 -- 172.21.15.115:0/728738709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 msgr2=0x562adc7bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:47.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.854+0000 7f611ffff700 1 --2- 172.21.15.115:0/728738709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 0x562adc7bf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.854+0000 7f611ffff700 1 -- 172.21.15.115:0/728738709 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562adc7b0280 con 0x562adc6365a0 2024-05-09T05:44:47.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.854+0000 7f611ffff700 1 --2- 172.21.15.115:0/728738709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7b3570 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f61100a6890 tx=0x7f6110091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3083dc0d7fb93977 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:47.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.854+0000 7f6115ffb700 1 -- 172.21.15.115:0/728738709 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100c8070 con 0x562adc6365a0 2024-05-09T05:44:47.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.854+0000 7f6115ffb700 1 -- 172.21.15.115:0/728738709 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6110004cc0 con 0x562adc6365a0 2024-05-09T05:44:47.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.854+0000 7f6115ffb700 1 -- 172.21.15.115:0/728738709 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611006d030 con 0x562adc6365a0 2024-05-09T05:44:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.855+0000 7f6135206b80 1 -- 172.21.15.115:0/728738709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 msgr2=0x562adc7b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.855+0000 7f6135206b80 1 --2- 172.21.15.115:0/728738709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7b3570 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f61100a6890 tx=0x7f6110091040 comp rx=0 tx=0).stop 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.855+0000 7f6135206b80 1 -- 172.21.15.115:0/728738709 shutdown_connections 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.855+0000 7f6135206b80 1 --2- 172.21.15.115:0/728738709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 0x562adc7d5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.855+0000 7f6135206b80 1 --2- 172.21.15.115:0/728738709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7b3570 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.855+0000 7f6135206b80 1 --2- 172.21.15.115:0/728738709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 0x562adc7bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.855+0000 7f6135206b80 1 -- 172.21.15.115:0/728738709 >> 172.21.15.115:0/728738709 conn(0x562adc6967a0 msgr2=0x562adc7bef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.855+0000 7f6135206b80 1 -- 172.21.15.115:0/728738709 shutdown_connections 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.855+0000 7f6135206b80 1 -- 172.21.15.115:0/728738709 wait complete. 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f6135206b80 1 -- start start 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f6135206b80 1 Processor -- start 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f6135206b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f6135206b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 0x562adc7c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f6135206b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 0x562adc7c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f6135206b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61201b3a10 con 0x562adc6365a0 2024-05-09T05:44:47.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f6135206b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f61201b3340 con 0x562adc720260 2024-05-09T05:44:47.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f6135206b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61201b0540 con 0x562adc7b3b50 2024-05-09T05:44:47.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f6135206b80 10 client.0 _ll_register_callbacks cb 0x562adc5477c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:47.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.856+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42748/0 (socket says 172.21.15.115:42748) 2024-05-09T05:44:47.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611ffff700 1 -- 172.21.15.115:0/926559857 learned_addr learned my addr 172.21.15.115:0/926559857 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:47.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611f7fe700 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 0x562adc7c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f6124d8d700 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 0x562adc7c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611ffff700 1 -- 172.21.15.115:0/926559857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 msgr2=0x562adc7c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:47.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611ffff700 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 0x562adc7c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611ffff700 1 -- 172.21.15.115:0/926559857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 msgr2=0x562adc7c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611ffff700 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 0x562adc7c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611ffff700 1 -- 172.21.15.115:0/926559857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562adc837ee0 con 0x562adc6365a0 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611f7fe700 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 0x562adc7c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f6124d8d700 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 0x562adc7c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f611ffff700 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7c2110 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6110091010 tx=0x7f61100adaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c2b8509a9f09f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f60ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f60ff7fe700 1 -- 172.21.15.115:0/926559857 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100c8070 con 0x562adc6365a0 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f60ff7fe700 1 -- 172.21.15.115:0/926559857 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f61100910b0 con 0x562adc6365a0 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f6135206b80 1 -- 172.21.15.115:0/926559857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6110004cc0 con 0x562adc6365a0 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f60ff7fe700 1 -- 172.21.15.115:0/926559857 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f611006d070 con 0x562adc6365a0 2024-05-09T05:44:47.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f6135206b80 10 client.17358 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:47.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.857+0000 7f6135206b80 1 -- 172.21.15.115:0/926559857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562adc839130 con 0x562adc6365a0 2024-05-09T05:44:47.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.858+0000 7f60ff7fe700 1 -- 172.21.15.115:0/926559857 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6110004cc0 con 0x562adc6365a0 2024-05-09T05:44:47.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.858+0000 7f60ff7fe700 1 -- 172.21.15.115:0/926559857 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6110083ce0 con 0x562adc6365a0 2024-05-09T05:44:47.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.858+0000 7f6135206b80 10 client.17358 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:47.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.858+0000 7f6135206b80 10 client.17358 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:44:47.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.858+0000 7f6135206b80 20 client.17358 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:47.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.858+0000 7f6135206b80 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562adc839360 0x562adc83b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:47.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.858+0000 7f6135206b80 4 client.17358 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c2030c2b", "release": true, "await": true}] 2024-05-09T05:44:47.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.858+0000 7f6135206b80 1 -- 172.21.15.115:0/926559857 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c2030c2b", "release": true, "await": true}) -- 0x562adc6ddb70 con 0x562adc839360 2024-05-09T05:44:47.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.858+0000 7f611f7fe700 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562adc839360 0x562adc83b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:47.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.859+0000 7f60ff7fe700 10 client.17358 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:47.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.859+0000 7f611f7fe700 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562adc839360 0x562adc83b780 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44f2a5e19907a501 server_cookie=8198c9cca5704b17 in_seq=0 out_seq=0 2024-05-09T05:44:47.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.874+0000 7f60ff7fe700 1 -- 172.21.15.115:0/926559857 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562adc6ddb70 con 0x562adc839360 2024-05-09T05:44:47.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.874+0000 7f60ff7fe700 10 client.17358 handle_command_reply: tid=0 2024-05-09T05:44:47.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.874+0000 7f6135206b80 1 client.17358 shutdown 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.874+0000 7f6135206b80 1 -- 172.21.15.115:0/926559857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 msgr2=0x562adc7c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.874+0000 7f6135206b80 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7c2110 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6110091010 tx=0x7f61100adaf0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.874+0000 7f6135206b80 1 -- 172.21.15.115:0/926559857 shutdown_connections 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.874+0000 7f6135206b80 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562adc839360 0x562adc83b780 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.874+0000 7f6135206b80 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562adc7b3b50 0x562adc7c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.874+0000 7f6135206b80 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562adc6365a0 0x562adc7c2110 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f6135206b80 1 --2- 172.21.15.115:0/926559857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562adc720260 0x562adc7c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f60ff7fe700 0 client.17358 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f6135206b80 1 -- 172.21.15.115:0/926559857 >> 172.21.15.115:0/926559857 conn(0x562adc6967a0 msgr2=0x562adc7b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f6135206b80 1 -- 172.21.15.115:0/926559857 shutdown_connections 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f6135206b80 1 -- 172.21.15.115:0/926559857 wait complete. 2024-05-09T05:44:47.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f6135206b80 20 client.17358 trim_cache size 0 max 16384 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 38, 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 20, 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "c2030c2b": { 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 20, 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:44:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:47.870 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f61267f0700 1 -- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f610c0b29f0 msgr2=0x7f610c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f61267f0700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f610c0b29f0 0x7f610c0b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6120102440 tx=0x7f610800b410 comp rx=0 tx=0).stop 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f61267f0700 1 -- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6120105520 msgr2=0x7f61201a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.875+0000 7f61267f0700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6120105520 0x7f61201a08a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f611000d980 tx=0x7f611000dc90 comp rx=0 tx=0).stop 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.876+0000 7f61267f0700 1 -- 172.21.15.115:0/236794149 shutdown_connections 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.876+0000 7f61267f0700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f610c0b29f0 0x7f610c0b4f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.876+0000 7f61267f0700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f612010e900 0x7f612019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.876+0000 7f61267f0700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6120105520 0x7f61201a08a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.876+0000 7f61267f0700 1 --2- 172.21.15.115:0/236794149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6120105e70 0x7f61201a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:47.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.876+0000 7f61267f0700 1 -- 172.21.15.115:0/236794149 >> 172.21.15.115:0/236794149 conn(0x7f61200fef90 msgr2=0x7f6120100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:47.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.876+0000 7f61267f0700 1 -- 172.21.15.115:0/236794149 shutdown_connections 2024-05-09T05:44:47.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:47.876+0000 7f61267f0700 1 -- 172.21.15.115:0/236794149 wait complete. 2024-05-09T05:44:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.599+0000 7f3373a29640 1 -- 172.21.15.115:0/1381689758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c10e6c0 msgr2=0x7f336c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.599+0000 7f3373a29640 1 --2- 172.21.15.115:0/1381689758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c10e6c0 0x7f336c110ae0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3364009ca0 tx=0x7f336402a8c0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.599+0000 7f3373a29640 1 -- 172.21.15.115:0/1381689758 shutdown_connections 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.599+0000 7f3373a29640 1 --2- 172.21.15.115:0/1381689758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c10e6c0 0x7f336c110ae0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.599+0000 7f3373a29640 1 --2- 172.21.15.115:0/1381689758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c1021c0 0x7f336c1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.599+0000 7f3373a29640 1 --2- 172.21.15.115:0/1381689758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f336c109d10 0x7f336c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.599+0000 7f3373a29640 1 -- 172.21.15.115:0/1381689758 >> 172.21.15.115:0/1381689758 conn(0x7f336c0fd9d0 msgr2=0x7f336c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.599+0000 7f3373a29640 1 -- 172.21.15.115:0/1381689758 shutdown_connections 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.599+0000 7f3373a29640 1 -- 172.21.15.115:0/1381689758 wait complete. 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f3373a29640 1 Processor -- start 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f3373a29640 1 -- start start 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f3373a29640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c1021c0 0x7f336c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f3373a29640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f336c109d10 0x7f336c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f3373a29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c10e6c0 0x7f336c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f3373a29640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f336c113d60 con 0x7f336c1021c0 2024-05-09T05:44:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f3373a29640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f336c113be0 con 0x7f336c109d10 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f3373a29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f336c113ee0 con 0x7f336c10e6c0 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f33717c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c1021c0 0x7f336c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f33717c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c1021c0 0x7f336c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42780/0 (socket says 172.21.15.115:42780) 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.600+0000 7f33717c5640 1 -- 172.21.15.115:0/501513796 learned_addr learned my addr 172.21.15.115:0/501513796 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f3370fc4640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f336c109d10 0x7f336c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f3371fc6640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c10e6c0 0x7f336c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f33717c5640 1 -- 172.21.15.115:0/501513796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c10e6c0 msgr2=0x7f336c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f33717c5640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c10e6c0 0x7f336c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f33717c5640 1 -- 172.21.15.115:0/501513796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f336c109d10 msgr2=0x7f336c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f33717c5640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f336c109d10 0x7f336c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f33717c5640 1 -- 172.21.15.115:0/501513796 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c19f500 con 0x7f336c1021c0 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f3370fc4640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f336c109d10 0x7f336c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f33717c5640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c1021c0 0x7f336c19a720 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f336800ea20 tx=0x7f336800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f33627fc640 1 -- 172.21.15.115:0/501513796 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336800cd00 con 0x7f336c1021c0 2024-05-09T05:44:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f33627fc640 1 -- 172.21.15.115:0/501513796 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33680044b0 con 0x7f336c1021c0 2024-05-09T05:44:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f3373a29640 1 -- 172.21.15.115:0/501513796 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c19f7f0 con 0x7f336c1021c0 2024-05-09T05:44:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f33627fc640 1 -- 172.21.15.115:0/501513796 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3368010430 con 0x7f336c1021c0 2024-05-09T05:44:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.601+0000 7f3373a29640 1 -- 172.21.15.115:0/501513796 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c19fd40 con 0x7f336c1021c0 2024-05-09T05:44:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.604+0000 7f33627fc640 1 -- 172.21.15.115:0/501513796 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f3368003ec0 con 0x7f336c1021c0 2024-05-09T05:44:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f337a9b5740 1 Processor -- start 2024-05-09T05:44:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f337a9b5740 1 -- start start 2024-05-09T05:44:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decd41b0 0x5606df077510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df077a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f33627fc640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33580b2950 0x7f33580b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decb34d0 0x5606df077f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f337a9b5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5606defdc7c0 con 0x5606decab2f0 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f337a9b5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5606def780e0 con 0x5606decb34d0 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f337a9b5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5606defd8780 con 0x5606decd41b0 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f33717c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f3370fc4640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33580b2950 0x7f33580b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f3371fc6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decb34d0 0x5606df077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f3371fc6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decb34d0 0x5606df077f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:47638/0 (socket says 172.21.15.115:47638) 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f3371fc6640 1 -- 172.21.15.115:0/3903573353 learned_addr learned my addr 172.21.15.115:0/3903573353 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3903573353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decd41b0 0x5606df077510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f33627fc640 1 -- 172.21.15.115:0/501513796 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f33680e2db0 con 0x7f336c1021c0 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f3371fc6640 1 -- 172.21.15.115:0/3903573353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decd41b0 msgr2=0x5606df077510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.605+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3903573353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decd41b0 0x5606df077510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.606+0000 7f3371fc6640 1 -- 172.21.15.115:0/3903573353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 msgr2=0x5606df077a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.606+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3903573353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df077a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.606+0000 7f3371fc6640 1 -- 172.21.15.115:0/3903573353 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606defa17c0 con 0x5606decb34d0 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.606+0000 7f3370fc4640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33580b2950 0x7f33580b4e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f336c0ff510 tx=0x7f335c00a5c0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.606+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3903573353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decb34d0 0x5606df077f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3364009ca0 tx=0x7f33640093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d36830dd6d22e1de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.606+0000 7f33577fe640 1 -- 172.21.15.115:0/3903573353 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364049070 con 0x5606decb34d0 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.606+0000 7f33577fe640 1 -- 172.21.15.115:0/3903573353 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3364036040 con 0x5606decb34d0 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.606+0000 7f33577fe640 1 -- 172.21.15.115:0/3903573353 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364004800 con 0x5606decb34d0 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 -- 172.21.15.115:0/3903573353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decb34d0 msgr2=0x5606df077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3903573353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decb34d0 0x5606df077f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3364009ca0 tx=0x7f33640093c0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 -- 172.21.15.115:0/3903573353 shutdown_connections 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3903573353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decb34d0 0x5606df077f90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3903573353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df077a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3903573353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decd41b0 0x5606df077510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 -- 172.21.15.115:0/3903573353 >> 172.21.15.115:0/3903573353 conn(0x5606decc8300 msgr2=0x5606df073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 -- 172.21.15.115:0/3903573353 shutdown_connections 2024-05-09T05:44:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 -- 172.21.15.115:0/3903573353 wait complete. 2024-05-09T05:44:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 -- start start 2024-05-09T05:44:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.607+0000 7f337a9b5740 1 Processor -- start 2024-05-09T05:44:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df087b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df088070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 0x5606df085fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f337a9b5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5606df088bc0 con 0x5606decab2f0 2024-05-09T05:44:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f337a9b5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5606def639c0 con 0x5606decd41b0 2024-05-09T05:44:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f337a9b5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5606df088d40 con 0x5606decb34d0 2024-05-09T05:44:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f337a9b5740 10 client.0 _ll_register_callbacks cb 0x5606def90020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f33717c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df087b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f33717c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df087b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42802/0 (socket says 172.21.15.115:42802) 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f33717c5640 1 -- 172.21.15.115:0/1656636205 learned_addr learned my addr 172.21.15.115:0/1656636205 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f3371fc6640 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 0x5606df085fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f3370fc4640 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df088070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f33717c5640 1 -- 172.21.15.115:0/1656636205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 msgr2=0x5606df088070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f33717c5640 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df088070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f33717c5640 1 -- 172.21.15.115:0/1656636205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 msgr2=0x5606df085fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f33717c5640 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 0x5606df085fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.608+0000 7f33717c5640 1 -- 172.21.15.115:0/1656636205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606def8a1f0 con 0x5606decab2f0 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f33717c5640 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df087b30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f33680e0270 tx=0x7f33680735f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8511b209053a65a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f3354ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f3354ff9640 1 -- 172.21.15.115:0/1656636205 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33680a9860 con 0x5606decab2f0 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f3354ff9640 1 -- 172.21.15.115:0/1656636205 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f33680a5fe0 con 0x5606decab2f0 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f337a9b5740 1 -- 172.21.15.115:0/1656636205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3364036040 con 0x5606decab2f0 2024-05-09T05:44:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f3354ff9640 1 -- 172.21.15.115:0/1656636205 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f33680aa450 con 0x5606decab2f0 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f337a9b5740 10 client.17370 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f337a9b5740 1 -- 172.21.15.115:0/1656636205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5606ded3f790 con 0x5606decab2f0 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f3354ff9640 1 -- 172.21.15.115:0/1656636205 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3364036040 con 0x5606decab2f0 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f3354ff9640 1 -- 172.21.15.115:0/1656636205 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f33680dc730 con 0x5606decab2f0 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f337a9b5740 10 client.17370 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.609+0000 7f337a9b5740 10 client.17370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.610+0000 7f337a9b5740 20 client.17370 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.610+0000 7f337a9b5740 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5606dec92d80 0x5606df098340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.610+0000 7f337a9b5740 4 client.17370 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.610+0000 7f337a9b5740 1 -- 172.21.15.115:0/1656636205 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606defea3d0 con 0x5606dec92d80 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.610+0000 7f3370fc4640 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5606dec92d80 0x5606df098340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.610+0000 7f3354ff9640 10 client.17370 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.610+0000 7f3370fc4640 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5606dec92d80 0x5606df098340 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36b7b188d0f32807 server_cookie=a7b7601951314b73 in_seq=0 out_seq=0 2024-05-09T05:44:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f3354ff9640 1 -- 172.21.15.115:0/1656636205 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5606defea3d0 con 0x5606dec92d80 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f3354ff9640 10 client.17370 handle_command_reply: tid=0 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 client.17370 shutdown 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 -- 172.21.15.115:0/1656636205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 msgr2=0x5606df087b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df087b30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f33680e0270 tx=0x7f33680735f0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 -- 172.21.15.115:0/1656636205 shutdown_connections 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 0x5606df085fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5606dec92d80 0x5606df098340 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df088070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 --2- 172.21.15.115:0/1656636205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df087b30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 -- 172.21.15.115:0/1656636205 >> 172.21.15.115:0/1656636205 conn(0x5606decc8300 msgr2=0x5606df072700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f3354ff9640 0 client.17370 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 -- 172.21.15.115:0/1656636205 shutdown_connections 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.611+0000 7f337a9b5740 1 -- 172.21.15.115:0/1656636205 wait complete. 2024-05-09T05:44:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.612+0000 7f337a9b5740 20 client.17370 trim_cache size 0 max 16384 2024-05-09T05:44:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f337a9b5740 1 Processor -- start 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f337a9b5740 1 -- start start 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df073f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df07f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 0x5606df07fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f337a9b5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1aee20 con 0x5606decab2f0 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f337a9b5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1b0960 con 0x5606decd41b0 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f337a9b5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1af4f0 con 0x5606decb34d0 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f33717c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f33717c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42804/0 (socket says 172.21.15.115:42804) 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f33717c5640 1 -- 172.21.15.115:0/3995409659 learned_addr learned my addr 172.21.15.115:0/3995409659 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f3370fc4640 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df07f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 0x5606df07fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f3371fc6640 1 -- 172.21.15.115:0/3995409659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 msgr2=0x5606df07f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df07f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f3371fc6640 1 -- 172.21.15.115:0/3995409659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 msgr2=0x5606df073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f3371fc6640 1 -- 172.21.15.115:0/3995409659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606dedb65f0 con 0x5606decd41b0 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.632+0000 7f33717c5640 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.633+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 0x5606df07fe00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f336402ade0 tx=0x7f3364005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d9877c2335e26b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.633+0000 7f3356ffd640 1 -- 172.21.15.115:0/3995409659 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364049070 con 0x5606decd41b0 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.633+0000 7f3356ffd640 1 -- 172.21.15.115:0/3995409659 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3364036070 con 0x5606decd41b0 2024-05-09T05:44:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.633+0000 7f3356ffd640 1 -- 172.21.15.115:0/3995409659 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33640042a0 con 0x5606decd41b0 2024-05-09T05:44:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 -- 172.21.15.115:0/3995409659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 msgr2=0x5606df07fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 0x5606df07fe00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f336402ade0 tx=0x7f3364005bf0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 -- 172.21.15.115:0/3995409659 shutdown_connections 2024-05-09T05:44:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decd41b0 0x5606df07fe00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df07f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3995409659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decab2f0 0x5606df073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 -- 172.21.15.115:0/3995409659 >> 172.21.15.115:0/3995409659 conn(0x5606decc8300 msgr2=0x5606df0737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 -- 172.21.15.115:0/3995409659 shutdown_connections 2024-05-09T05:44:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 -- 172.21.15.115:0/3995409659 wait complete. 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.634+0000 7f337a9b5740 1 -- start start 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f337a9b5740 1 Processor -- start 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decab2f0 0x5606df089d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df08a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f337a9b5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decd41b0 0x5606df08a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f337a9b5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1b0290 con 0x5606decd41b0 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f337a9b5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1afbc0 con 0x5606decab2f0 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f337a9b5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1ace00 con 0x5606decb34d0 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f337a9b5740 10 client.0 _ll_register_callbacks cb 0x7f33680046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f3371fc6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decd41b0 0x5606df08a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f3371fc6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decd41b0 0x5606df08a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42808/0 (socket says 172.21.15.115:42808) 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f3371fc6640 1 -- 172.21.15.115:0/3055926942 learned_addr learned my addr 172.21.15.115:0/3055926942 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.635+0000 7f3370fc4640 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df08a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f33717c5640 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decab2f0 0x5606df089d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3371fc6640 1 -- 172.21.15.115:0/3055926942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 msgr2=0x5606df08a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df08a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3371fc6640 1 -- 172.21.15.115:0/3055926942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decab2f0 msgr2=0x5606df089d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decab2f0 0x5606df089d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3371fc6640 1 -- 172.21.15.115:0/3055926942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606defa0ed0 con 0x5606decd41b0 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3371fc6640 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decd41b0 0x5606df08a7a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3364031520 tx=0x7f3364009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e58e6fba408a7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3354ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3354ff9640 1 -- 172.21.15.115:0/3055926942 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364051070 con 0x5606decd41b0 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3354ff9640 1 -- 172.21.15.115:0/3055926942 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f3364003d80 con 0x5606decd41b0 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f337a9b5740 1 -- 172.21.15.115:0/3055926942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3364036070 con 0x5606decd41b0 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f3354ff9640 1 -- 172.21.15.115:0/3055926942 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3364049070 con 0x5606decd41b0 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f337a9b5740 10 client.17376 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.636+0000 7f337a9b5740 1 -- 172.21.15.115:0/3055926942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5606df100890 con 0x5606decd41b0 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f3354ff9640 1 -- 172.21.15.115:0/3055926942 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3364036070 con 0x5606decd41b0 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f3354ff9640 1 -- 172.21.15.115:0/3055926942 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3364004600 con 0x5606decd41b0 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f337a9b5740 10 client.17376 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f337a9b5740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f337a9b5740 20 client.17376 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5606dec681c0 0x5606df102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f337a9b5740 4 client.17376 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:44:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f337a9b5740 1 -- 172.21.15.115:0/3055926942 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606defe4240 con 0x5606dec681c0 2024-05-09T05:44:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f33717c5640 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5606dec681c0 0x5606df102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f3354ff9640 10 client.17376 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.637+0000 7f33717c5640 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5606dec681c0 0x5606df102ad0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b955988e5fd1cffd server_cookie=f279274a962445e3 in_seq=0 out_seq=0 2024-05-09T05:44:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f3354ff9640 1 -- 172.21.15.115:0/3055926942 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5606defe4240 con 0x5606dec681c0 2024-05-09T05:44:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f3354ff9640 10 client.17376 handle_command_reply: tid=0 2024-05-09T05:44:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 client.17376 shutdown 2024-05-09T05:44:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 -- 172.21.15.115:0/3055926942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decd41b0 msgr2=0x5606df08a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decd41b0 0x5606df08a7a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3364031520 tx=0x7f3364009360 comp rx=0 tx=0).stop 2024-05-09T05:44:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 -- 172.21.15.115:0/3055926942 shutdown_connections 2024-05-09T05:44:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606decd41b0 0x5606df08a7a0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5606dec681c0 0x5606df102ad0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5606decb34d0 0x5606df08a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f3354ff9640 0 client.17376 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 --2- 172.21.15.115:0/3055926942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606decab2f0 0x5606df089d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 -- 172.21.15.115:0/3055926942 >> 172.21.15.115:0/3055926942 conn(0x5606decc8300 msgr2=0x5606df087490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 -- 172.21.15.115:0/3055926942 shutdown_connections 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 1 -- 172.21.15.115:0/3055926942 wait complete. 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.642+0000 7f337a9b5740 20 client.17376 trim_cache size 0 max 16384 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:44:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:44:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 -- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33580b2950 msgr2=0x7f33580b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33580b2950 0x7f33580b4e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f336c0ff510 tx=0x7f335c00a5c0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 -- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c1021c0 msgr2=0x7f336c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c1021c0 0x7f336c19a720 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f336800ea20 tx=0x7f336800eee0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 -- 172.21.15.115:0/501513796 shutdown_connections 2024-05-09T05:44:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33580b2950 0x7f33580b4e60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c10e6c0 0x7f336c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f336c109d10 0x7f336c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 --2- 172.21.15.115:0/501513796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f336c1021c0 0x7f336c19a720 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 -- 172.21.15.115:0/501513796 >> 172.21.15.115:0/501513796 conn(0x7f336c0fd9d0 msgr2=0x7f336c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 -- 172.21.15.115:0/501513796 shutdown_connections 2024-05-09T05:44:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:44:48.643+0000 7f3373a29640 1 -- 172.21.15.115:0/501513796 wait complete. 2024-05-09T05:44:49.345 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:49 smithi115 ceph-mon[30349]: pgmap v201: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.7 KiB/s wr, 36 op/s 2024-05-09T05:44:49.375 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c2030c2b', total seconds elapsed: 39.6 2024-05-09T05:44:49.375 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-09T05:44:49.375 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:44:49.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:49 smithi195 ceph-mon[33844]: pgmap v201: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.7 KiB/s wr, 36 op/s 2024-05-09T05:44:49.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:49 smithi161 ceph-mon[33805]: pgmap v201: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.7 KiB/s wr, 36 op/s 2024-05-09T05:44:50.563 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.573+0000 7f986ef64700 1 -- 172.21.15.115:0/2311367689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868107df0 msgr2=0x7f9868108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.573+0000 7f986ef64700 1 --2- 172.21.15.115:0/2311367689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868107df0 0x7f9868108200 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f98540099c0 tx=0x7f9854009cd0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.573+0000 7f986ef64700 1 -- 172.21.15.115:0/2311367689 shutdown_connections 2024-05-09T05:44:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.573+0000 7f986ef64700 1 --2- 172.21.15.115:0/2311367689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9868108740 0x7f98681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.573+0000 7f986ef64700 1 --2- 172.21.15.115:0/2311367689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98681034c0 0x7f98681038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.573+0000 7f986ef64700 1 --2- 172.21.15.115:0/2311367689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868107df0 0x7f9868108200 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.573+0000 7f986ef64700 1 -- 172.21.15.115:0/2311367689 >> 172.21.15.115:0/2311367689 conn(0x7f98680fedc0 msgr2=0x7f98681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.573+0000 7f986ef64700 1 -- 172.21.15.115:0/2311367689 shutdown_connections 2024-05-09T05:44:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.573+0000 7f986ef64700 1 -- 172.21.15.115:0/2311367689 wait complete. 2024-05-09T05:44:50.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.574+0000 7f986ef64700 1 Processor -- start 2024-05-09T05:44:50.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.574+0000 7f986ef64700 1 -- start start 2024-05-09T05:44:50.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.574+0000 7f986ef64700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98681034c0 0x7f986819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.574+0000 7f986ef64700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868107df0 0x7f986819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.574+0000 7f986ef64700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868108740 0x7f986819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.574+0000 7f986ef64700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9868113200 con 0x7f9868108740 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.574+0000 7f986ef64700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9868113080 con 0x7f98681034c0 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.574+0000 7f986ef64700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9868113380 con 0x7f9868107df0 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f9867fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868107df0 0x7f986819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f986d501700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868108740 0x7f986819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f9867fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868107df0 0x7f986819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33598/0 (socket says 172.21.15.115:33598) 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f9867fff700 1 -- 172.21.15.115:0/1818540085 learned_addr learned my addr 172.21.15.115:0/1818540085 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f9867fff700 1 -- 172.21.15.115:0/1818540085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98681034c0 msgr2=0x7f986819ada0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f986cd00700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98681034c0 0x7f986819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f9867fff700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98681034c0 0x7f986819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f9867fff700 1 -- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868108740 msgr2=0x7f986819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f9867fff700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868108740 0x7f986819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f9867fff700 1 -- 172.21.15.115:0/1818540085 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986819fc70 con 0x7f9868107df0 2024-05-09T05:44:50.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.575+0000 7f986d501700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868108740 0x7f986819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:50.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.576+0000 7f9867fff700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868107df0 0x7f986819b340 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98540045e0 tx=0x7f9854004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:50.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.576+0000 7f9865ffb700 1 -- 172.21.15.115:0/1818540085 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854026070 con 0x7f9868107df0 2024-05-09T05:44:50.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.576+0000 7f9865ffb700 1 -- 172.21.15.115:0/1818540085 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f985400f460 con 0x7f9868107df0 2024-05-09T05:44:50.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.576+0000 7f986ef64700 1 -- 172.21.15.115:0/1818540085 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986819ff00 con 0x7f9868107df0 2024-05-09T05:44:50.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.576+0000 7f9865ffb700 1 -- 172.21.15.115:0/1818540085 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854009e60 con 0x7f9868107df0 2024-05-09T05:44:50.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.576+0000 7f986ef64700 1 -- 172.21.15.115:0/1818540085 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a03e0 con 0x7f9868107df0 2024-05-09T05:44:50.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.580+0000 7f9865ffb700 1 -- 172.21.15.115:0/1818540085 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f9854029070 con 0x7f9868107df0 2024-05-09T05:44:50.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.581+0000 7f987d97ab80 1 Processor -- start 2024-05-09T05:44:50.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.581+0000 7f987d97ab80 1 -- start start 2024-05-09T05:44:50.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.581+0000 7f9865ffb700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98580b29f0 0x7f98580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.581+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.581+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2a4a0 0x563e53cad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.581+0000 7f986d501700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98580b29f0 0x7f98580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.581+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53ca7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f987d97ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563e53bca0a0 con 0x563e53ca7400 2024-05-09T05:44:50.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f987d97ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563e53b6f9a0 con 0x563e53b2a4a0 2024-05-09T05:44:50.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f987d97ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563e53bc6060 con 0x563e53ba4d00 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986cd00700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f9865ffb700 1 -- 172.21.15.115:0/1818540085 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f98540e3190 con 0x7f9868107df0 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986cd00700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33602/0 (socket says 172.21.15.115:33602) 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986cd00700 1 -- 172.21.15.115:0/978945810 learned_addr learned my addr 172.21.15.115:0/978945810 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986d501700 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53ca7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986cd00700 1 -- 172.21.15.115:0/978945810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2a4a0 msgr2=0x563e53cad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f9867fff700 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2a4a0 0x563e53cad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986cd00700 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2a4a0 0x563e53cad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986cd00700 1 -- 172.21.15.115:0/978945810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 msgr2=0x563e53ca7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986cd00700 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53ca7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986cd00700 1 -- 172.21.15.115:0/978945810 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e53b60500 con 0x563e53ba4d00 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.582+0000 7f986d501700 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53ca7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.583+0000 7f986cd00700 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cad240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f985c012d40 tx=0x7f985c009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307f4bc528926afd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.583+0000 7f986d501700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98580b29f0 0x7f98580b4f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9860009420 tx=0x7f98600093b0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:50.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.583+0000 7f9852ffd700 1 -- 172.21.15.115:0/978945810 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0120d0 con 0x563e53ba4d00 2024-05-09T05:44:50.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.583+0000 7f9852ffd700 1 -- 172.21.15.115:0/978945810 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f985c004020 con 0x563e53ba4d00 2024-05-09T05:44:50.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.583+0000 7f9852ffd700 1 -- 172.21.15.115:0/978945810 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c010b60 con 0x563e53ba4d00 2024-05-09T05:44:50.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.584+0000 7f987d97ab80 1 -- 172.21.15.115:0/978945810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 msgr2=0x563e53cad240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.584+0000 7f987d97ab80 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cad240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f985c012d40 tx=0x7f985c009900 comp rx=0 tx=0).stop 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.584+0000 7f987d97ab80 1 -- 172.21.15.115:0/978945810 shutdown_connections 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.584+0000 7f987d97ab80 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53ca7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.584+0000 7f987d97ab80 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cad240 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.584+0000 7f987d97ab80 1 --2- 172.21.15.115:0/978945810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2a4a0 0x563e53cad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.584+0000 7f987d97ab80 1 -- 172.21.15.115:0/978945810 >> 172.21.15.115:0/978945810 conn(0x563e53b2df70 msgr2=0x563e53ca2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.584+0000 7f987d97ab80 1 -- 172.21.15.115:0/978945810 shutdown_connections 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.584+0000 7f987d97ab80 1 -- 172.21.15.115:0/978945810 wait complete. 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.585+0000 7f987d97ab80 1 -- start start 2024-05-09T05:44:50.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.585+0000 7f987d97ab80 1 Processor -- start 2024-05-09T05:44:50.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53b2a4a0 0x563e53cb1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53ba4d00 0x563e53cb1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53cb1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f987d97ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563e53ca8160 con 0x563e53ca7400 2024-05-09T05:44:50.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f987d97ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563e53b64b20 con 0x563e53ba4d00 2024-05-09T05:44:50.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f987d97ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563e53cb4020 con 0x563e53b2a4a0 2024-05-09T05:44:50.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f986d501700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53cb1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f986d501700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53cb1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42832/0 (socket says 172.21.15.115:42832) 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f986d501700 1 -- 172.21.15.115:0/3097199555 learned_addr learned my addr 172.21.15.115:0/3097199555 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f986cd00700 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53b2a4a0 0x563e53cb1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f987d97ab80 10 client.0 _ll_register_callbacks cb 0x563e53a3b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f986d501700 1 -- 172.21.15.115:0/3097199555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53b2a4a0 msgr2=0x563e53cb1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f986d501700 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53b2a4a0 0x563e53cb1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f986d501700 1 -- 172.21.15.115:0/3097199555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53ba4d00 msgr2=0x563e53cb1540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f986d501700 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53ba4d00 0x563e53cb1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.586+0000 7f986d501700 1 -- 172.21.15.115:0/3097199555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e53a45490 con 0x563e53ca7400 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.587+0000 7f986d501700 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53cb1a80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9860019f80 tx=0x7f9860009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f39ba8051217564 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.587+0000 7f9847fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.587+0000 7f9847fff700 1 -- 172.21.15.115:0/3097199555 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986002b060 con 0x563e53ca7400 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.587+0000 7f9847fff700 1 -- 172.21.15.115:0/3097199555 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9860004440 con 0x563e53ca7400 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.587+0000 7f987d97ab80 1 -- 172.21.15.115:0/3097199555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985c004020 con 0x563e53ca7400 2024-05-09T05:44:50.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.587+0000 7f9847fff700 1 -- 172.21.15.115:0/3097199555 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9860032050 con 0x563e53ca7400 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.587+0000 7f987d97ab80 10 client.17388 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.587+0000 7f987d97ab80 1 -- 172.21.15.115:0/3097199555 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563e53cbe7d0 con 0x563e53ca7400 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.588+0000 7f9847fff700 1 -- 172.21.15.115:0/3097199555 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9860004640 con 0x563e53ca7400 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.588+0000 7f9847fff700 1 -- 172.21.15.115:0/3097199555 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9860018040 con 0x563e53ca7400 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.588+0000 7f987d97ab80 10 client.17388 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.588+0000 7f987d97ab80 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.588+0000 7f987d97ab80 20 client.17388 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.588+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563e53cc0e10 0x563e53cc3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.588+0000 7f987d97ab80 4 client.17388 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.588+0000 7f987d97ab80 1 -- 172.21.15.115:0/3097199555 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f985c004020 con 0x563e53cc0e10 2024-05-09T05:44:50.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.589+0000 7f986cd00700 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563e53cc0e10 0x563e53cc3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.589+0000 7f9847fff700 10 client.17388 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.589+0000 7f986cd00700 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563e53cc0e10 0x563e53cc3230 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50172ddb035e718 server_cookie=441eb181e5b5e067 in_seq=0 out_seq=0 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f9847fff700 1 -- 172.21.15.115:0/3097199555 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f985c004020 con 0x563e53cc0e10 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f9847fff700 10 client.17388 handle_command_reply: tid=0 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f987d97ab80 1 client.17388 shutdown 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f987d97ab80 1 -- 172.21.15.115:0/3097199555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 msgr2=0x563e53cb1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53cb1a80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9860019f80 tx=0x7f9860009130 comp rx=0 tx=0).stop 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f987d97ab80 1 -- 172.21.15.115:0/3097199555 shutdown_connections 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563e53cc0e10 0x563e53cc3230 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f9847fff700 0 client.17388 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53ca7400 0x563e53cb1a80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53b2a4a0 0x563e53cb1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3097199555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53ba4d00 0x563e53cb1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.590+0000 7f987d97ab80 1 -- 172.21.15.115:0/3097199555 >> 172.21.15.115:0/3097199555 conn(0x563e53b2df70 msgr2=0x563e53c9bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:50.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.591+0000 7f987d97ab80 1 -- 172.21.15.115:0/3097199555 shutdown_connections 2024-05-09T05:44:50.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.591+0000 7f987d97ab80 1 -- 172.21.15.115:0/3097199555 wait complete. 2024-05-09T05:44:50.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.591+0000 7f987d97ab80 20 client.17388 trim_cache size 0 max 16384 2024-05-09T05:44:50.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.611+0000 7f987d97ab80 1 Processor -- start 2024-05-09T05:44:50.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f987d97ab80 1 -- start start 2024-05-09T05:44:50.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2df70 0x563e53c9c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 0x563e53cbcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f987d97ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f98681ae100 con 0x563e53b2a4a0 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f987d97ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f98681afc40 con 0x563e53b2df70 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f987d97ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98681ae7d0 con 0x563e53ba4d00 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f9867fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 0x563e53cbcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f986cd00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2df70 0x563e53c9c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f986cd00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2df70 0x563e53c9c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:47700/0 (socket says 172.21.15.115:47700) 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.612+0000 7f986cd00700 1 -- 172.21.15.115:0/1655946655 learned_addr learned my addr 172.21.15.115:0/1655946655 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f986d501700 1 --2- 172.21.15.115:0/1655946655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f986cd00700 1 -- 172.21.15.115:0/1655946655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 msgr2=0x563e53cc89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f986cd00700 1 --2- 172.21.15.115:0/1655946655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f986cd00700 1 -- 172.21.15.115:0/1655946655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 msgr2=0x563e53cbcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f986cd00700 1 --2- 172.21.15.115:0/1655946655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 0x563e53cbcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f986cd00700 1 -- 172.21.15.115:0/1655946655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e53cb0dd0 con 0x563e53b2df70 2024-05-09T05:44:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f986cd00700 1 --2- 172.21.15.115:0/1655946655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2df70 0x563e53c9c700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f985c009c50 tx=0x7f985c012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87687fdd1da56352 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f98527fc700 1 -- 172.21.15.115:0/1655946655 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c012190 con 0x563e53b2df70 2024-05-09T05:44:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f98527fc700 1 -- 172.21.15.115:0/1655946655 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f985c0054d0 con 0x563e53b2df70 2024-05-09T05:44:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.613+0000 7f98527fc700 1 -- 172.21.15.115:0/1655946655 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c004b70 con 0x563e53b2df70 2024-05-09T05:44:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.614+0000 7f987d97ab80 1 -- 172.21.15.115:0/1655946655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2df70 msgr2=0x563e53c9c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.614+0000 7f987d97ab80 1 --2- 172.21.15.115:0/1655946655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2df70 0x563e53c9c700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f985c009c50 tx=0x7f985c012820 comp rx=0 tx=0).stop 2024-05-09T05:44:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.614+0000 7f987d97ab80 1 -- 172.21.15.115:0/1655946655 shutdown_connections 2024-05-09T05:44:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.614+0000 7f987d97ab80 1 --2- 172.21.15.115:0/1655946655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53b2df70 0x563e53c9c700 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f985c009c50 tx=0x7f985c012820 comp rx=0 tx=0).stop 2024-05-09T05:44:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.614+0000 7f987d97ab80 1 --2- 172.21.15.115:0/1655946655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.614+0000 7f987d97ab80 1 --2- 172.21.15.115:0/1655946655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 0x563e53cbcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.614+0000 7f987d97ab80 1 -- 172.21.15.115:0/1655946655 >> 172.21.15.115:0/1655946655 conn(0x563e53ca2520 msgr2=0x563e53c9bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.614+0000 7f987d97ab80 1 -- 172.21.15.115:0/1655946655 shutdown_connections 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.614+0000 7f987d97ab80 1 -- 172.21.15.115:0/1655946655 wait complete. 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.615+0000 7f987d97ab80 1 -- start start 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.615+0000 7f987d97ab80 1 Processor -- start 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.615+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 0x563e53cb9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.615+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.615+0000 7f987d97ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53ca7400 0x563e53cc87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.615+0000 7f987d97ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f98681af570 con 0x563e53b2a4a0 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.615+0000 7f987d97ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f98681aeea0 con 0x563e53ca7400 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.615+0000 7f987d97ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f98681ac0a0 con 0x563e53ba4d00 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f987d97ab80 10 client.0 _ll_register_callbacks cb 0x563e53a3b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9867fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9867fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33620/0 (socket says 172.21.15.115:33620) 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9867fff700 1 -- 172.21.15.115:0/3268999614 learned_addr learned my addr 172.21.15.115:0/3268999614 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f986cd00700 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 0x563e53cb9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f986d501700 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53ca7400 0x563e53cc87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9867fff700 1 -- 172.21.15.115:0/3268999614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53ca7400 msgr2=0x563e53cc87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9867fff700 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53ca7400 0x563e53cc87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9867fff700 1 -- 172.21.15.115:0/3268999614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 msgr2=0x563e53cb9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9867fff700 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 0x563e53cb9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9867fff700 1 -- 172.21.15.115:0/3268999614 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e53cc8ef0 con 0x563e53ba4d00 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9867fff700 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc8280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98540a74b0 tx=0x7f985400fc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c5de0cffd52d92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9847fff700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f9847fff700 1 -- 172.21.15.115:0/3268999614 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540a99b0 con 0x563e53ba4d00 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f9847fff700 1 -- 172.21.15.115:0/3268999614 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f9854089e20 con 0x563e53ba4d00 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.616+0000 7f987d97ab80 1 -- 172.21.15.115:0/3268999614 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985c0054d0 con 0x563e53ba4d00 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f9847fff700 1 -- 172.21.15.115:0/3268999614 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f98540e09d0 con 0x563e53ba4d00 2024-05-09T05:44:50.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f987d97ab80 10 client.26827 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f987d97ab80 1 -- 172.21.15.115:0/3268999614 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563e53d2b4d0 con 0x563e53ba4d00 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f9847fff700 1 -- 172.21.15.115:0/3268999614 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f985c0054d0 con 0x563e53ba4d00 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f9847fff700 1 -- 172.21.15.115:0/3268999614 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f98540ae150 con 0x563e53ba4d00 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f987d97ab80 10 client.26827 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f987d97ab80 10 client.26827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f987d97ab80 20 client.26827 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563e53cc0e10 0x563e53d2dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f987d97ab80 4 client.26827 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f987d97ab80 1 -- 172.21.15.115:0/3268999614 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e53bd7a80 con 0x563e53cc0e10 2024-05-09T05:44:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.617+0000 7f986cd00700 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563e53cc0e10 0x563e53d2dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:50.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f9847fff700 10 client.26827 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:50.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f986cd00700 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563e53cc0e10 0x563e53d2dbf0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17230afb9eea46f server_cookie=14f168fd968268e9 in_seq=0 out_seq=0 2024-05-09T05:44:50.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f9847fff700 1 -- 172.21.15.115:0/3268999614 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e53bd7a80 con 0x563e53cc0e10 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f9847fff700 10 client.26827 handle_command_reply: tid=0 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f987d97ab80 1 client.26827 shutdown 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f987d97ab80 1 -- 172.21.15.115:0/3268999614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 msgr2=0x563e53cc8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc8280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98540a74b0 tx=0x7f985400fc10 comp rx=0 tx=0).stop 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f987d97ab80 1 -- 172.21.15.115:0/3268999614 shutdown_connections 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e53ca7400 0x563e53cc87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563e53cc0e10 0x563e53d2dbf0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f9847fff700 0 client.26827 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563e53b2a4a0 0x563e53cb9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f987d97ab80 1 --2- 172.21.15.115:0/3268999614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563e53ba4d00 0x563e53cc8280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.618+0000 7f987d97ab80 1 -- 172.21.15.115:0/3268999614 >> 172.21.15.115:0/3268999614 conn(0x563e53ca2520 msgr2=0x563e53ca63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f987d97ab80 1 -- 172.21.15.115:0/3268999614 shutdown_connections 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f987d97ab80 1 -- 172.21.15.115:0/3268999614 wait complete. 2024-05-09T05:44:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f987d97ab80 20 client.26827 trim_cache size 0 max 16384 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": { 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout: "3ff255c9-91bf-40d4-8c31-c969402988f6": { 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/", 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout: "tag": "3ff255c9-91bf-40d4-8c31-c969402988f6", 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout: "options": "recursive,force" 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:50.616 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:44:50.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 -- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98580b29f0 msgr2=0x7f98580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:50.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98580b29f0 0x7f98580b4f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9860009420 tx=0x7f98600093b0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 -- 172.21.15.115:0/1818540085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868107df0 msgr2=0x7f986819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:50.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868107df0 0x7f986819b340 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98540045e0 tx=0x7f9854004610 comp rx=0 tx=0).stop 2024-05-09T05:44:50.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 -- 172.21.15.115:0/1818540085 shutdown_connections 2024-05-09T05:44:50.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98580b29f0 0x7f98580b4f00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98681034c0 0x7f986819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868108740 0x7f986819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 --2- 172.21.15.115:0/1818540085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868107df0 0x7f986819b340 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 -- 172.21.15.115:0/1818540085 >> 172.21.15.115:0/1818540085 conn(0x7f98680fedc0 msgr2=0x7f986806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.619+0000 7f986ef64700 1 -- 172.21.15.115:0/1818540085 shutdown_connections 2024-05-09T05:44:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:50.620+0000 7f986ef64700 1 -- 172.21.15.115:0/1818540085 wait complete. 2024-05-09T05:44:51.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:51 smithi115 ceph-mon[30349]: pgmap v202: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 op/s 2024-05-09T05:44:51.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:51 smithi195 ceph-mon[33844]: pgmap v202: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 op/s 2024-05-09T05:44:51.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:51 smithi161 ceph-mon[33805]: pgmap v202: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 op/s 2024-05-09T05:44:52.770 INFO:tasks.cephfs.filesystem:scrub status for tag:3ff255c9-91bf-40d4-8c31-c969402988f6 - {'path': '/', 'tag': '3ff255c9-91bf-40d4-8c31-c969402988f6', 'options': 'recursive,force'} 2024-05-09T05:44:53.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:53 smithi115 ceph-mon[30349]: pgmap v203: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 op/s 2024-05-09T05:44:53.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 -- 172.21.15.115:0/3661028904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 msgr2=0x7f76481111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:53.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 --2- 172.21.15.115:0/3661028904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 0x7f76481111f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f764000b0d0 tx=0x7f764000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 -- 172.21.15.115:0/3661028904 shutdown_connections 2024-05-09T05:44:53.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 --2- 172.21.15.115:0/3661028904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648104610 0x7f7648104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 --2- 172.21.15.115:0/3661028904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 0x7f76481111f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 --2- 172.21.15.115:0/3661028904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7648108f40 0x7f7648109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 -- 172.21.15.115:0/3661028904 >> 172.21.15.115:0/3661028904 conn(0x7f764807af70 msgr2=0x7f764807b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:53.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 -- 172.21.15.115:0/3661028904 shutdown_connections 2024-05-09T05:44:53.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 -- 172.21.15.115:0/3661028904 wait complete. 2024-05-09T05:44:53.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.354+0000 7f764e249700 1 Processor -- start 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f764e249700 1 -- start start 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f764e249700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648104610 0x7f764819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f764e249700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7648108f40 0x7f764819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f764e249700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 0x7f764819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f764e249700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76481143e0 con 0x7f7648109890 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f764e249700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7648114260 con 0x7f7648104610 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f764e249700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7648114560 con 0x7f7648108f40 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f7647fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 0x7f764819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f76477fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648104610 0x7f764819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f7647fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 0x7f764819f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48786/0 (socket says 172.21.15.115:48786) 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f7647fff700 1 -- 172.21.15.115:0/511465193 learned_addr learned my addr 172.21.15.115:0/511465193 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f7646ffd700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7648108f40 0x7f764819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:53.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f7647fff700 1 -- 172.21.15.115:0/511465193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7648108f40 msgr2=0x7f764819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f7647fff700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7648108f40 0x7f764819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f7647fff700 1 -- 172.21.15.115:0/511465193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648104610 msgr2=0x7f764819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f7647fff700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648104610 0x7f764819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.355+0000 7f7647fff700 1 -- 172.21.15.115:0/511465193 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764819fc40 con 0x7f7648109890 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.356+0000 7f7647fff700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 0x7f764819f5a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f76400079a0 tx=0x7f76400079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.356+0000 7f7644ff9700 1 -- 172.21.15.115:0/511465193 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764001f070 con 0x7f7648109890 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.356+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764819fed0 con 0x7f7648109890 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.356+0000 7f7644ff9700 1 -- 172.21.15.115:0/511465193 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7640003c00 con 0x7f7648109890 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.356+0000 7f7644ff9700 1 -- 172.21.15.115:0/511465193 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764001b430 con 0x7f7648109890 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.356+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a7850 con 0x7f7648109890 2024-05-09T05:44:53.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.357+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76480512c0 con 0x7f7648109890 2024-05-09T05:44:53.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.359+0000 7f7644ff9700 1 -- 172.21.15.115:0/511465193 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f7640022070 con 0x7f7648109890 2024-05-09T05:44:53.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.360+0000 7f7644ff9700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f76300b2a70 0x7f76300b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:53.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.360+0000 7f76477fe700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f76300b2a70 0x7f76300b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:53.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.360+0000 7f7644ff9700 1 -- 172.21.15.115:0/511465193 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f764000f020 con 0x7f7648109890 2024-05-09T05:44:53.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.361+0000 7f76477fe700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f76300b2a70 0x7f76300b4f80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7638009c60 tx=0x7f7638009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:53.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.364+0000 7f7644ff9700 1 -- 172.21.15.115:0/511465193 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7640029020 con 0x7f7648109890 2024-05-09T05:44:53.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:53 smithi195 ceph-mon[33844]: pgmap v203: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 op/s 2024-05-09T05:44:53.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:53 smithi161 ceph-mon[33805]: pgmap v203: 129 pgs: 129 active+clean; 4.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 op/s 2024-05-09T05:44:53.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.668+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f764819c0d0 con 0x7f7648109890 2024-05-09T05:44:53.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.669+0000 7f7644ff9700 1 -- 172.21.15.115:0/511465193 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11953 (secure 0 0 0) 0x7f76400ab410 con 0x7f7648109890 2024-05-09T05:44:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:44:53.662 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:43:58.362294+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/2784864553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2784864553},{"type":"v1","addr":"172.21.15.195:6833","nonce":2784864553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:44:53.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.671+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f76300b2a70 msgr2=0x7f76300b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:53.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.671+0000 7f764e249700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f76300b2a70 0x7f76300b4f80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7638009c60 tx=0x7f7638009260 comp rx=0 tx=0).stop 2024-05-09T05:44:53.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.671+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 msgr2=0x7f764819f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:53.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.671+0000 7f764e249700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 0x7f764819f5a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f76400079a0 tx=0x7f76400079d0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.672+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 shutdown_connections 2024-05-09T05:44:53.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.672+0000 7f764e249700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f76300b2a70 0x7f76300b4f80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.672+0000 7f764e249700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7648104610 0x7f764819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.672+0000 7f764e249700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7648109890 0x7f764819f5a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.672+0000 7f764e249700 1 --2- 172.21.15.115:0/511465193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7648108f40 0x7f764819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:53.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.672+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 >> 172.21.15.115:0/511465193 conn(0x7f764807af70 msgr2=0x7f7648101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:53.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.672+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 shutdown_connections 2024-05-09T05:44:53.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:53.672+0000 7f764e249700 1 -- 172.21.15.115:0/511465193 wait complete. 2024-05-09T05:44:53.665 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 38 2024-05-09T05:44:54.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:54 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/511465193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:54.442 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-09T05:44:54.442 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-09T05:44:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:54 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/511465193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:54.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:54 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/511465193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:44:54.726 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:44:55.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:55 smithi115 ceph-mon[30349]: pgmap v204: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 39 op/s 2024-05-09T05:44:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:55 smithi195 ceph-mon[33844]: pgmap v204: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 39 op/s 2024-05-09T05:44:55.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:55 smithi161 ceph-mon[33805]: pgmap v204: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 39 op/s 2024-05-09T05:44:56.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.619+0000 7f80e0eb2700 1 -- 172.21.15.115:0/2740809694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc108740 msgr2=0x7f80dc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:56.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.619+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/2740809694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc108740 0x7f80dc1100a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f80d400b0d0 tx=0x7f80d400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.619+0000 7f80e0eb2700 1 -- 172.21.15.115:0/2740809694 shutdown_connections 2024-05-09T05:44:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.619+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/2740809694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80dc1034c0 0x7f80dc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.619+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/2740809694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc108740 0x7f80dc1100a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.619+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/2740809694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80dc107df0 0x7f80dc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.619+0000 7f80e0eb2700 1 -- 172.21.15.115:0/2740809694 >> 172.21.15.115:0/2740809694 conn(0x7f80dc0fedc0 msgr2=0x7f80dc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.619+0000 7f80e0eb2700 1 -- 172.21.15.115:0/2740809694 shutdown_connections 2024-05-09T05:44:56.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.619+0000 7f80e0eb2700 1 -- 172.21.15.115:0/2740809694 wait complete. 2024-05-09T05:44:56.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.620+0000 7f80e0eb2700 1 Processor -- start 2024-05-09T05:44:56.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.620+0000 7f80e0eb2700 1 -- start start 2024-05-09T05:44:56.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80e0eb2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80dc1034c0 0x7f80dc19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80e0eb2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc107df0 0x7f80dc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80e0eb2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80dc108740 0x7f80dc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80e0eb2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc113220 con 0x7f80dc107df0 2024-05-09T05:44:56.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80e0eb2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1130a0 con 0x7f80dc108740 2024-05-09T05:44:56.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80e0eb2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1133a0 con 0x7f80dc1034c0 2024-05-09T05:44:56.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80d9d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc107df0 0x7f80dc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80d9d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc107df0 0x7f80dc19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48806/0 (socket says 172.21.15.115:48806) 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80d9d9b700 1 -- 172.21.15.115:0/3432713709 learned_addr learned my addr 172.21.15.115:0/3432713709 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80da59c700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80dc1034c0 0x7f80dc19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80dad9d700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80dc108740 0x7f80dc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80d9d9b700 1 -- 172.21.15.115:0/3432713709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80dc1034c0 msgr2=0x7f80dc19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80dc1034c0 0x7f80dc19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80d9d9b700 1 -- 172.21.15.115:0/3432713709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80dc108740 msgr2=0x7f80dc1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80dc108740 0x7f80dc1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80d9d9b700 1 -- 172.21.15.115:0/3432713709 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc19a020 con 0x7f80dc107df0 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.621+0000 7f80da59c700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80dc1034c0 0x7f80dc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.622+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc107df0 0x7f80dc19fd00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c9e0 tx=0x7f80d000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.622+0000 7f80cb7fe700 1 -- 172.21.15.115:0/3432713709 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0007aa0 con 0x7f80dc107df0 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.622+0000 7f80e0eb2700 1 -- 172.21.15.115:0/3432713709 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80dc19a310 con 0x7f80dc107df0 2024-05-09T05:44:56.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.622+0000 7f80cb7fe700 1 -- 172.21.15.115:0/3432713709 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80d0007c40 con 0x7f80dc107df0 2024-05-09T05:44:56.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.622+0000 7f80cb7fe700 1 -- 172.21.15.115:0/3432713709 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0018660 con 0x7f80dc107df0 2024-05-09T05:44:56.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.622+0000 7f80e0eb2700 1 -- 172.21.15.115:0/3432713709 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80dc06b610 con 0x7f80dc107df0 2024-05-09T05:44:56.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.625+0000 7f80cb7fe700 1 -- 172.21.15.115:0/3432713709 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f80d0018800 con 0x7f80dc107df0 2024-05-09T05:44:56.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80ef8c8b80 1 Processor -- start 2024-05-09T05:44:56.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80ef8c8b80 1 -- start start 2024-05-09T05:44:56.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80cb7fe700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80c40b29f0 0x7f80c40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfd352f0 0x556bcfdceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 0x556bcfdcf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfdc8be0 0x556bcfdc8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556bcfceb180 con 0x556bcfd352f0 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556bcfc89980 con 0x556bcfc4b630 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556bcfce7140 con 0x556bcfdc8be0 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.626+0000 7f80da59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfd352f0 0x556bcfdceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80da59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfd352f0 0x556bcfdceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48810/0 (socket says 172.21.15.115:48810) 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80da59c700 1 -- 172.21.15.115:0/394277395 learned_addr learned my addr 172.21.15.115:0/394277395 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80dad9d700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80c40b29f0 0x7f80c40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80cb7fe700 1 -- 172.21.15.115:0/3432713709 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f80d00e28b0 con 0x7f80dc107df0 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80dad9d700 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfdc8be0 0x556bcfdc8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80da59c700 1 -- 172.21.15.115:0/394277395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfdc8be0 msgr2=0x556bcfdc8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80da59c700 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfdc8be0 0x556bcfdc8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80da59c700 1 -- 172.21.15.115:0/394277395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 msgr2=0x556bcfdcf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:56.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 0x556bcfdcf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80da59c700 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 0x556bcfdcf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80da59c700 1 -- 172.21.15.115:0/394277395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bcfc81500 con 0x556bcfd352f0 2024-05-09T05:44:56.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80dad9d700 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfdc8be0 0x556bcfdc8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:56.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80da59c700 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfd352f0 0x556bcfdceb20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f80cc008c60 tx=0x7f80cc008f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b6041a16ba49a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:56.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80dad9d700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80c40b29f0 0x7f80c40b4f00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009e60 tx=0x7f80d40090f0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80c8ff9700 1 -- 172.21.15.115:0/394277395 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00ec10 con 0x556bcfd352f0 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80c8ff9700 1 -- 172.21.15.115:0/394277395 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80cc0044b0 con 0x556bcfd352f0 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.627+0000 7f80c8ff9700 1 -- 172.21.15.115:0/394277395 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc003680 con 0x556bcfd352f0 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.628+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/394277395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfd352f0 msgr2=0x556bcfdceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.628+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfd352f0 0x556bcfdceb20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f80cc008c60 tx=0x7f80cc008f70 comp rx=0 tx=0).stop 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.628+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/394277395 shutdown_connections 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.628+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfdc8be0 0x556bcfdc8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.628+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfd352f0 0x556bcfdceb20 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.628+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/394277395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 0x556bcfdcf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.628+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/394277395 >> 172.21.15.115:0/394277395 conn(0x556bcfcab830 msgr2=0x556bcfdc3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:56.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/394277395 shutdown_connections 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/394277395 wait complete. 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 -- start start 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 Processor -- start 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 0x556bcfdd2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfdc8be0 0x556bcfdd32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556bcfb66d30 con 0x556bcfdc8be0 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556bcfca52e0 con 0x556bcfc4b630 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.629+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556bcfdde990 con 0x556bcfd352f0 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80ef8c8b80 10 client.0 _ll_register_callbacks cb 0x556bcfb5c850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80dad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfdc8be0 0x556bcfdd32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80dad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfdc8be0 0x556bcfdd32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48824/0 (socket says 172.21.15.115:48824) 2024-05-09T05:44:56.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80dad9d700 1 -- 172.21.15.115:0/1132024881 learned_addr learned my addr 172.21.15.115:0/1132024881 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80da59c700 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 0x556bcfdd2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80dad9d700 1 -- 172.21.15.115:0/1132024881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 msgr2=0x556bcfdd2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80dad9d700 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80dad9d700 1 -- 172.21.15.115:0/1132024881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 msgr2=0x556bcfdd2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80dad9d700 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 0x556bcfdd2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80dad9d700 1 -- 172.21.15.115:0/1132024881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bcfb66620 con 0x556bcfdc8be0 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80dad9d700 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfdc8be0 0x556bcfdd32c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f80d4004e70 tx=0x7f80d40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11a8b0666d3984c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1132024881 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d402e020 con 0x556bcfdc8be0 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.630+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1132024881 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f80d4007c50 con 0x556bcfdc8be0 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1132024881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80cc0044b0 con 0x556bcfdc8be0 2024-05-09T05:44:56.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1132024881 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f80d4039050 con 0x556bcfdc8be0 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80ef8c8b80 10 client.17412 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1132024881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556bcfde02a0 con 0x556bcfdc8be0 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1132024881 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80cc0044b0 con 0x556bcfdc8be0 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1132024881 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f80d4011070 con 0x556bcfdc8be0 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80ef8c8b80 10 client.17412 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80ef8c8b80 10 client.17412 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80ef8c8b80 20 client.17412 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556bcfde2930 0x556bcfde4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80ef8c8b80 4 client.17412 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:44:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.631+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1132024881 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bcfcf8b60 con 0x556bcfde2930 2024-05-09T05:44:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.632+0000 7f80da59c700 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556bcfde2930 0x556bcfde4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.632+0000 7f80c1ffb700 10 client.17412 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.632+0000 7f80da59c700 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556bcfde2930 0x556bcfde4d90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52493cb3433cc164 server_cookie=5091c375e1293b5a in_seq=0 out_seq=0 2024-05-09T05:44:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1132024881 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556bcfcf8b60 con 0x556bcfde2930 2024-05-09T05:44:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80c1ffb700 10 client.17412 handle_command_reply: tid=0 2024-05-09T05:44:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 client.17412 shutdown 2024-05-09T05:44:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1132024881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfdc8be0 msgr2=0x556bcfdd32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfdc8be0 0x556bcfdd32c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f80d4004e70 tx=0x7f80d40099e0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1132024881 shutdown_connections 2024-05-09T05:44:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfc4b630 0x556bcfdd2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556bcfde2930 0x556bcfde4d90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfdc8be0 0x556bcfdd32c0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1132024881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1132024881 >> 172.21.15.115:0/1132024881 conn(0x556bcfcab830 msgr2=0x556bcfdc37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80c1ffb700 0 client.17412 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.633+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1132024881 shutdown_connections 2024-05-09T05:44:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.634+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1132024881 wait complete. 2024-05-09T05:44:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.634+0000 7f80ef8c8b80 20 client.17412 trim_cache size 0 max 16384 2024-05-09T05:44:56.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.659+0000 7f80ef8c8b80 1 Processor -- start 2024-05-09T05:44:56.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.659+0000 7f80ef8c8b80 1 -- start start 2024-05-09T05:44:56.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.659+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfdc8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.659+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.659+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.659+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1adff0 con 0x556bcfc4b630 2024-05-09T05:44:56.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.659+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1afb30 con 0x556bcfdc8be0 2024-05-09T05:44:56.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.659+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1ae6c0 con 0x556bcfd352f0 2024-05-09T05:44:56.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.659+0000 7f80da59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfdc8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80da59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfdc8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48838/0 (socket says 172.21.15.115:48838) 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80da59c700 1 -- 172.21.15.115:0/3021868589 learned_addr learned my addr 172.21.15.115:0/3021868589 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80dad9d700 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80da59c700 1 -- 172.21.15.115:0/3021868589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 msgr2=0x556bcfdd01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80da59c700 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80da59c700 1 -- 172.21.15.115:0/3021868589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 msgr2=0x556bcfdd4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80da59c700 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80da59c700 1 -- 172.21.15.115:0/3021868589 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bcfdcd440 con 0x556bcfc4b630 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80dad9d700 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80da59c700 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfdc8600 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f80cc014c00 tx=0x7f80cc013b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2f6a22070726c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:56.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80c3fff700 1 -- 172.21.15.115:0/3021868589 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc033070 con 0x556bcfc4b630 2024-05-09T05:44:56.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80c3fff700 1 -- 172.21.15.115:0/3021868589 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80cc00ee30 con 0x556bcfc4b630 2024-05-09T05:44:56.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.660+0000 7f80c3fff700 1 -- 172.21.15.115:0/3021868589 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc004d30 con 0x556bcfc4b630 2024-05-09T05:44:56.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.661+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/3021868589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 msgr2=0x556bcfdc8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:56.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.661+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfdc8600 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f80cc014c00 tx=0x7f80cc013b80 comp rx=0 tx=0).stop 2024-05-09T05:44:56.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.661+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/3021868589 shutdown_connections 2024-05-09T05:44:56.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.661+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.661+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfdc8600 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.661+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/3021868589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.661+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/3021868589 >> 172.21.15.115:0/3021868589 conn(0x556bcfcab830 msgr2=0x556bcfdd3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/3021868589 shutdown_connections 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/3021868589 wait complete. 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 -- start start 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 Processor -- start 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfddac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1af460 con 0x556bcfc4b630 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1aed90 con 0x556bcfdc8be0 2024-05-09T05:44:56.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc1abf90 con 0x556bcfd352f0 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80da59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfddac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80ef8c8b80 10 client.0 _ll_register_callbacks cb 0x556bcfb5c850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80da59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfddac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48844/0 (socket says 172.21.15.115:48844) 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.662+0000 7f80da59c700 1 -- 172.21.15.115:0/1144948499 learned_addr learned my addr 172.21.15.115:0/1144948499 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80dad9d700 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80da59c700 1 -- 172.21.15.115:0/1144948499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 msgr2=0x556bcfdd8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80da59c700 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80da59c700 1 -- 172.21.15.115:0/1144948499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 msgr2=0x556bcfdd9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80da59c700 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80da59c700 1 -- 172.21.15.115:0/1144948499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bcfdd9aa0 con 0x556bcfc4b630 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80dad9d700 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80da59c700 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfddac60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f80cc005950 tx=0x7f80cc0085a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c8976b857b3567 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:44:56.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1144948499 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc033040 con 0x556bcfc4b630 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1144948499 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f80cc005380 con 0x556bcfc4b630 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1144948499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80cc00ee30 con 0x556bcfc4b630 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1144948499 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f80cc045050 con 0x556bcfc4b630 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80ef8c8b80 10 client.17418 fetch_fsmap learned FSMap version 38 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.663+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1144948499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556bcfe4dde0 con 0x556bcfc4b630 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.664+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1144948499 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80cc03c070 con 0x556bcfc4b630 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.664+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1144948499 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f80cc02d370 con 0x556bcfc4b630 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.664+0000 7f80ef8c8b80 10 client.17418 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.664+0000 7f80ef8c8b80 10 client.17418 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.664+0000 7f80ef8c8b80 20 client.17418 populate_metadata read hostname 'smithi115' 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.664+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556bcfe4e2e0 0x556bcfe50730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.664+0000 7f80ef8c8b80 4 client.17418 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.664+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1144948499 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f80cc00ee30 con 0x556bcfe4e2e0 2024-05-09T05:44:56.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.664+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556bcfe4e2e0 0x556bcfe50730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:44:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.665+0000 7f80c1ffb700 10 client.17418 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:56.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.665+0000 7f80d9d9b700 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556bcfe4e2e0 0x556bcfe50730 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2845ec536c5ee6e server_cookie=728c4080e18e6608 in_seq=0 out_seq=0 2024-05-09T05:44:56.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80c1ffb700 1 -- 172.21.15.115:0/1144948499 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f80cc00ee30 con 0x556bcfe4e2e0 2024-05-09T05:44:56.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80c1ffb700 10 client.17418 handle_command_reply: tid=0 2024-05-09T05:44:56.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 client.17418 shutdown 2024-05-09T05:44:56.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1144948499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 msgr2=0x556bcfddac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfddac60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f80cc005950 tx=0x7f80cc0085a0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1144948499 shutdown_connections 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bcfdc8be0 0x556bcfdd9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556bcfe4e2e0 0x556bcfe50730 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556bcfc4b630 0x556bcfddac60 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 --2- 172.21.15.115:0/1144948499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bcfd352f0 0x556bcfdd8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1144948499 >> 172.21.15.115:0/1144948499 conn(0x556bcfcab830 msgr2=0x556bcfdc8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80c1ffb700 0 client.17418 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1144948499 shutdown_connections 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.756+0000 7f80ef8c8b80 1 -- 172.21.15.115:0/1144948499 wait complete. 2024-05-09T05:44:56.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80ef8c8b80 20 client.17418 trim_cache size 0 max 16384 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 38, 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 22, 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "b7601b26": { 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 22, 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:44:56.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:44:56.752 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:44:56.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 -- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80c40b29f0 msgr2=0x7f80c40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:56.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80c40b29f0 0x7f80c40b4f00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009e60 tx=0x7f80d40090f0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 -- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc107df0 msgr2=0x7f80dc19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:44:56.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc107df0 0x7f80dc19fd00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c9e0 tx=0x7f80d000cda0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 -- 172.21.15.115:0/3432713709 shutdown_connections 2024-05-09T05:44:56.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80c40b29f0 0x7f80c40b4f00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80dc108740 0x7f80dc1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80dc1034c0 0x7f80dc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 --2- 172.21.15.115:0/3432713709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80dc107df0 0x7f80dc19fd00 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:44:56.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 -- 172.21.15.115:0/3432713709 >> 172.21.15.115:0/3432713709 conn(0x7f80dc0fedc0 msgr2=0x7f80dc1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:44:56.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 -- 172.21.15.115:0/3432713709 shutdown_connections 2024-05-09T05:44:56.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:44:56.757+0000 7f80e0eb2700 1 -- 172.21.15.115:0/3432713709 wait complete. 2024-05-09T05:44:57.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:57 smithi115 ceph-mon[30349]: pgmap v205: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 35 op/s 2024-05-09T05:44:57.567 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b7601b26', quiesce took 8.2 seconds. Will release after: 21.8 2024-05-09T05:44:57.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:57 smithi195 ceph-mon[33844]: pgmap v205: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 35 op/s 2024-05-09T05:44:57.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:57 smithi161 ceph-mon[33805]: pgmap v205: 129 pgs: 129 active+clean; 4.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 35 op/s 2024-05-09T05:44:59.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:44:59 smithi195 ceph-mon[33844]: pgmap v206: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 MiB/s wr, 43 op/s 2024-05-09T05:44:59.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:44:59 smithi115 ceph-mon[30349]: pgmap v206: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 MiB/s wr, 43 op/s 2024-05-09T05:44:59.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:44:59 smithi161 ceph-mon[33805]: pgmap v206: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 MiB/s wr, 43 op/s 2024-05-09T05:45:00.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:00 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:45:00.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:00 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:45:00.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:00 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:45:00.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:00 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:45:00.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:00 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:45:00.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:00 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:45:00.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:00 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:45:00.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:00 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:45:00.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:00 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:45:00.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:00 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:45:00.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:00 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:45:00.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:00 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:45:01.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:01 smithi195 ceph-mon[33844]: pgmap v207: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:45:01.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:01 smithi115 ceph-mon[30349]: pgmap v207: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:45:01.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:01 smithi161 ceph-mon[33805]: pgmap v207: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:45:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.755+0000 7f8eaf746640 1 -- 172.21.15.115:0/1538995443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8108b90 msgr2=0x7f8ea8102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.755+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1538995443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8108b90 0x7f8ea8102730 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009900 tx=0x7f8ea402a410 comp rx=0 tx=0).stop 2024-05-09T05:45:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.755+0000 7f8eaf746640 1 -- 172.21.15.115:0/1538995443 shutdown_connections 2024-05-09T05:45:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.755+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1538995443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea81032e0 0x7f8ea810f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.755+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1538995443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ea8109030 0x7f8ea8102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.755+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1538995443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8108b90 0x7f8ea8102730 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.755+0000 7f8eaf746640 1 -- 172.21.15.115:0/1538995443 >> 172.21.15.115:0/1538995443 conn(0x7f8ea80fe390 msgr2=0x7f8ea81007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.755+0000 7f8eaf746640 1 -- 172.21.15.115:0/1538995443 shutdown_connections 2024-05-09T05:45:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.755+0000 7f8eaf746640 1 -- 172.21.15.115:0/1538995443 wait complete. 2024-05-09T05:45:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eaf746640 1 Processor -- start 2024-05-09T05:45:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eaf746640 1 -- start start 2024-05-09T05:45:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eaf746640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ea81032e0 0x7f8ea8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eaf746640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea8108b90 0x7f8ea8199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eaf746640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8109030 0x7f8ea819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eaf746640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8112c30 con 0x7f8ea8109030 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eaf746640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8112ab0 con 0x7f8ea81032e0 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eaf746640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8112db0 con 0x7f8ea8108b90 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eadce3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8109030 0x7f8ea819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eadce3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8109030 0x7f8ea819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48858/0 (socket says 172.21.15.115:48858) 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eadce3640 1 -- 172.21.15.115:0/1777379621 learned_addr learned my addr 172.21.15.115:0/1777379621 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ea81032e0 0x7f8ea8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eacce1640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea8108b90 0x7f8ea8199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eadce3640 1 -- 172.21.15.115:0/1777379621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea8108b90 msgr2=0x7f8ea8199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eadce3640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea8108b90 0x7f8ea8199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eadce3640 1 -- 172.21.15.115:0/1777379621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ea81032e0 msgr2=0x7f8ea8199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eadce3640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ea81032e0 0x7f8ea8199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8eadce3640 1 -- 172.21.15.115:0/1777379621 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea819e590 con 0x7f8ea8109030 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.756+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ea81032e0 0x7f8ea8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.757+0000 7f8eadce3640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8109030 0x7f8ea819df50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000eb10 tx=0x7f8ea000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.757+0000 7f8e9e7fc640 1 -- 172.21.15.115:0/1777379621 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0004b20 con 0x7f8ea8109030 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.757+0000 7f8e9e7fc640 1 -- 172.21.15.115:0/1777379621 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ea0004cc0 con 0x7f8ea8109030 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.757+0000 7f8eaf746640 1 -- 172.21.15.115:0/1777379621 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea819e880 con 0x7f8ea8109030 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.757+0000 7f8e9e7fc640 1 -- 172.21.15.115:0/1777379621 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0010690 con 0x7f8ea8109030 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.757+0000 7f8eaf746640 1 -- 172.21.15.115:0/1777379621 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea81a6200 con 0x7f8ea8109030 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eb65ff740 1 Processor -- start 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8e9e7fc640 1 -- 172.21.15.115:0/1777379621 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f8ea0005020 con 0x7f8ea8109030 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eb65ff740 1 -- start start 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a32490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e66662f0 0x5644e6a329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e666e4d0 0x5644e6a32f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644e69977c0 con 0x5644e668f1b0 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644e69330e0 con 0x5644e666e4d0 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644e6993780 con 0x5644e66662f0 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eacce1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a32490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eacce1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a32490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48866/0 (socket says 172.21.15.115:48866) 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.760+0000 7f8eacce1640 1 -- 172.21.15.115:0/1415772462 learned_addr learned my addr 172.21.15.115:0/1415772462 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/1415772462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e66662f0 0x5644e6a329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8eacce1640 1 -- 172.21.15.115:0/1415772462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e66662f0 msgr2=0x5644e6a329d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8eacce1640 1 --2- 172.21.15.115:0/1415772462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e66662f0 0x5644e6a329d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8eacce1640 1 -- 172.21.15.115:0/1415772462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e666e4d0 msgr2=0x5644e6a32f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8eacce1640 1 --2- 172.21.15.115:0/1415772462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e666e4d0 0x5644e6a32f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8eacce1640 1 -- 172.21.15.115:0/1415772462 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e695c7c0 con 0x5644e668f1b0 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8e9e7fc640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e940b2950 0x7f8e940b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8eacce1640 1 --2- 172.21.15.115:0/1415772462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a32490 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800dc60 tx=0x7f8e98011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e638df07a6ae2e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e940b2950 0x7f8e940b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8e9e7fc640 1 -- 172.21.15.115:0/1777379621 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f8ea00e3000 con 0x7f8ea8109030 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8e937fe640 1 -- 172.21.15.115:0/1415772462 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98016070 con 0x5644e668f1b0 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8e937fe640 1 -- 172.21.15.115:0/1415772462 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e980054b0 con 0x5644e668f1b0 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.761+0000 7f8e937fe640 1 -- 172.21.15.115:0/1415772462 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e980123f0 con 0x5644e668f1b0 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1415772462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 msgr2=0x5644e6a32490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e940b2950 0x7f8e940b4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4006010 tx=0x7f8ea40058b0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/1415772462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a32490 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800dc60 tx=0x7f8e98011750 comp rx=0 tx=0).stop 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1415772462 shutdown_connections 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/1415772462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e666e4d0 0x5644e6a32f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/1415772462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e66662f0 0x5644e6a329d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/1415772462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a32490 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1415772462 >> 172.21.15.115:0/1415772462 conn(0x5644e6683300 msgr2=0x5644e6a2ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1415772462 shutdown_connections 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.762+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1415772462 wait complete. 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eb65ff740 1 -- start start 2024-05-09T05:45:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eb65ff740 1 Processor -- start 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a3ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e666e4d0 0x5644e6a3f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e668f1b0 0x5644e6a44f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5644e6a41a20 con 0x5644e66662f0 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644e691e9c0 con 0x5644e666e4d0 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644e6a41ba0 con 0x5644e668f1b0 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eb65ff740 10 client.0 _ll_register_callbacks cb 0x5644e694b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8ead4e2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a3ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8ead4e2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a3ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48870/0 (socket says 172.21.15.115:48870) 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8ead4e2640 1 -- 172.21.15.115:0/2179971183 learned_addr learned my addr 172.21.15.115:0/2179971183 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.763+0000 7f8eadce3640 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e668f1b0 0x5644e6a44f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8eacce1640 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e666e4d0 0x5644e6a3f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8ead4e2640 1 -- 172.21.15.115:0/2179971183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e668f1b0 msgr2=0x5644e6a44f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e668f1b0 0x5644e6a44f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8ead4e2640 1 -- 172.21.15.115:0/2179971183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e666e4d0 msgr2=0x5644e6a3f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e666e4d0 0x5644e6a3f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8ead4e2640 1 -- 172.21.15.115:0/2179971183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e69451f0 con 0x5644e66662f0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a3ef50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4005940 tx=0x7f8ea4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947422e9d7723976 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2179971183 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea404b070 con 0x5644e66662f0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2179971183 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f8ea4005260 con 0x5644e66662f0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2179971183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e980054b0 con 0x5644e66662f0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2179971183 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8ea4057050 con 0x5644e66662f0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8eb65ff740 10 client.17430 fetch_fsmap learned FSMap version 38 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.764+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2179971183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5644e66fa790 con 0x5644e66662f0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.765+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2179971183 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e980054b0 con 0x5644e66662f0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.765+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2179971183 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f8ea40035f0 con 0x5644e66662f0 2024-05-09T05:45:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.765+0000 7f8eb65ff740 10 client.17430 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.765+0000 7f8eb65ff740 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.765+0000 7f8eb65ff740 20 client.17430 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.765+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5644e664dd80 0x5644e6a511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.765+0000 7f8eb65ff740 4 client.17430 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.765+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2179971183 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644e69a53d0 con 0x5644e664dd80 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.765+0000 7f8eacce1640 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5644e664dd80 0x5644e6a511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.766+0000 7f8eacce1640 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5644e664dd80 0x5644e6a511e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13dfc51dc08daab6 server_cookie=58c9190b11f33fd9 in_seq=0 out_seq=0 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.766+0000 7f8e90ff9640 10 client.17430 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.766+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2179971183 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644e69a53d0 con 0x5644e664dd80 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.766+0000 7f8e90ff9640 10 client.17430 handle_command_reply: tid=0 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 client.17430 shutdown 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2179971183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 msgr2=0x5644e6a3ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a3ef50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4005940 tx=0x7f8ea4009260 comp rx=0 tx=0).stop 2024-05-09T05:45:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2179971183 shutdown_connections 2024-05-09T05:45:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5644e664dd80 0x5644e6a511e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e668f1b0 0x5644e6a44f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8e90ff9640 0 client.17430 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e666e4d0 0x5644e6a3f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2179971183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a3ef50 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2179971183 >> 172.21.15.115:0/2179971183 conn(0x5644e6683300 msgr2=0x5644e6a34330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2179971183 shutdown_connections 2024-05-09T05:45:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2179971183 wait complete. 2024-05-09T05:45:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.767+0000 7f8eb65ff740 20 client.17430 trim_cache size 0 max 16384 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eb65ff740 1 Processor -- start 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eb65ff740 1 -- start start 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e66662f0 0x5644e6a569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 0x5644e6a56f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a44dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea81aeed0 con 0x5644e668f1b0 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea81b0a10 con 0x5644e66662f0 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea81af5a0 con 0x5644e666e4d0 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eadce3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a44dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eadce3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a44dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48876/0 (socket says 172.21.15.115:48876) 2024-05-09T05:45:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.782+0000 7f8eadce3640 1 -- 172.21.15.115:0/1658980315 learned_addr learned my addr 172.21.15.115:0/1658980315 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8eacce1640 1 --2- 172.21.15.115:0/1658980315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 0x5644e6a56f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/1658980315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e66662f0 0x5644e6a569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8eadce3640 1 -- 172.21.15.115:0/1658980315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 msgr2=0x5644e6a56f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8eadce3640 1 --2- 172.21.15.115:0/1658980315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 0x5644e6a56f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8eadce3640 1 -- 172.21.15.115:0/1658980315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e66662f0 msgr2=0x5644e6a569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8eadce3640 1 --2- 172.21.15.115:0/1658980315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e66662f0 0x5644e6a569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8eadce3640 1 -- 172.21.15.115:0/1658980315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e67715f0 con 0x5644e668f1b0 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8eadce3640 1 --2- 172.21.15.115:0/1658980315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a44dd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00a9ca0 tx=0x7f8ea00a99b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78486e83557f0023 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8e92ffd640 1 -- 172.21.15.115:0/1658980315 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea006e070 con 0x5644e668f1b0 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8e92ffd640 1 -- 172.21.15.115:0/1658980315 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ea00a9fa0 con 0x5644e668f1b0 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.783+0000 7f8e92ffd640 1 -- 172.21.15.115:0/1658980315 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00aa460 con 0x5644e668f1b0 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1658980315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 msgr2=0x5644e6a44dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/1658980315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a44dd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00a9ca0 tx=0x7f8ea00a99b0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1658980315 shutdown_connections 2024-05-09T05:45:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/1658980315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e668f1b0 0x5644e6a44dd0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/1658980315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 0x5644e6a56f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/1658980315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e66662f0 0x5644e6a569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1658980315 >> 172.21.15.115:0/1658980315 conn(0x5644e6683300 msgr2=0x5644e6a32e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1658980315 shutdown_connections 2024-05-09T05:45:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 -- 172.21.15.115:0/1658980315 wait complete. 2024-05-09T05:45:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 -- start start 2024-05-09T05:45:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.784+0000 7f8eb65ff740 1 Processor -- start 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a4b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 0x5644e6a46470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eb65ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e668f1b0 0x5644e6a469b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea81b0340 con 0x5644e66662f0 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea81afc70 con 0x5644e668f1b0 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eb65ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea81aceb0 con 0x5644e666e4d0 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eb65ff740 10 client.0 _ll_register_callbacks cb 0x7f8ea00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8ead4e2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a4b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8ead4e2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a4b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48884/0 (socket says 172.21.15.115:48884) 2024-05-09T05:45:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8ead4e2640 1 -- 172.21.15.115:0/2499621043 learned_addr learned my addr 172.21.15.115:0/2499621043 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eacce1640 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 0x5644e6a46470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eadce3640 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e668f1b0 0x5644e6a469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8ead4e2640 1 -- 172.21.15.115:0/2499621043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 msgr2=0x5644e6a46470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 0x5644e6a46470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8ead4e2640 1 -- 172.21.15.115:0/2499621043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e668f1b0 msgr2=0x5644e6a469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e668f1b0 0x5644e6a469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8ead4e2640 1 -- 172.21.15.115:0/2499621043 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e695bed0 con 0x5644e66662f0 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8eacce1640 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 0x5644e6a46470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8ead4e2640 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a4b120 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8ea402a450 tx=0x7f8ea4032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e269b26d892fdd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.785+0000 7f8e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2499621043 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea405f070 con 0x5644e66662f0 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2499621043 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9350+0+0 (secure 0 0 0) 0x7f8ea4003e30 con 0x5644e66662f0 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2499621043 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea00a9fa0 con 0x5644e66662f0 2024-05-09T05:45:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2499621043 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8ea406b050 con 0x5644e66662f0 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eb65ff740 10 client.17436 fetch_fsmap learned FSMap version 38 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2499621043 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5644e6ab9250 con 0x5644e66662f0 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2499621043 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ea00a9fa0 con 0x5644e66662f0 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2499621043 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f8ea4004aa0 con 0x5644e66662f0 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eb65ff740 10 client.17436 fetch_fsmap finished waiting for FSMap version 38 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eb65ff740 10 client.17436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eb65ff740 20 client.17436 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5644e66231c0 0x5644e6abba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eb65ff740 4 client.17436 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2499621043 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644e699f240 con 0x5644e66231c0 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.786+0000 7f8eacce1640 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5644e66231c0 0x5644e6abba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.787+0000 7f8e90ff9640 10 client.17436 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:01.787+0000 7f8eacce1640 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5644e66231c0 0x5644e6abba10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f06d6ebd81d782 server_cookie=eaac322e73780fcb in_seq=0 out_seq=0 2024-05-09T05:45:03.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:03 smithi195 ceph-mon[33844]: pgmap v208: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:45:03.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:03 smithi115 ceph-mon[30349]: pgmap v208: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:45:03.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:03 smithi161 ceph-mon[33805]: pgmap v208: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-05-09T05:45:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8e90ff9640 1 -- 172.21.15.115:0/2499621043 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5644e699f240 con 0x5644e66231c0 2024-05-09T05:45:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8e90ff9640 10 client.17436 handle_command_reply: tid=0 2024-05-09T05:45:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8eb65ff740 1 client.17436 shutdown 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2499621043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 msgr2=0x5644e6a4b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a4b120 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8ea402a450 tx=0x7f8ea4032fb0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2499621043 shutdown_connections 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644e668f1b0 0x5644e6a469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5644e66231c0 0x5644e6abba10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644e666e4d0 0x5644e6a46470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8eb65ff740 1 --2- 172.21.15.115:0/2499621043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5644e66662f0 0x5644e6a4b120 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8e90ff9640 0 client.17436 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.283+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2499621043 >> 172.21.15.115:0/2499621043 conn(0x5644e6683300 msgr2=0x5644e6a40d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.284+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2499621043 shutdown_connections 2024-05-09T05:45:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.284+0000 7f8eb65ff740 1 -- 172.21.15.115:0/2499621043 wait complete. 2024-05-09T05:45:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.284+0000 7f8eb65ff740 20 client.17436 trim_cache size 0 max 16384 2024-05-09T05:45:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:45:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:45:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:45:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.284+0000 7f8eaf746640 1 -- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e940b2950 msgr2=0x7f8e940b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.284+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e940b2950 0x7f8e940b4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4006010 tx=0x7f8ea40058b0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.284+0000 7f8eaf746640 1 -- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8109030 msgr2=0x7f8ea819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.284+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8109030 0x7f8ea819df50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000eb10 tx=0x7f8ea000c2d0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.285+0000 7f8eaf746640 1 -- 172.21.15.115:0/1777379621 shutdown_connections 2024-05-09T05:45:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.285+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e940b2950 0x7f8e940b4e60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.285+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea8109030 0x7f8ea819df50 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.285+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ea8108b90 0x7f8ea8199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.285+0000 7f8eaf746640 1 --2- 172.21.15.115:0/1777379621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ea81032e0 0x7f8ea8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.285+0000 7f8eaf746640 1 -- 172.21.15.115:0/1777379621 >> 172.21.15.115:0/1777379621 conn(0x7f8ea80fe390 msgr2=0x7f8ea80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.285+0000 7f8eaf746640 1 -- 172.21.15.115:0/1777379621 shutdown_connections 2024-05-09T05:45:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:04.285+0000 7f8eaf746640 1 -- 172.21.15.115:0/1777379621 wait complete. 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: *** Caught signal (Segmentation fault) ** 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: in thread 7fd1bb741700 thread_name:md_log_replay 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd1ccb03d20] 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x564795acbda1] 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x564795acc238] 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x564795ac68af] 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x564795787cff] 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 6: (MDLog::_replay_thread()+0x1f38) [0x564795b2c92e] 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 7: (MDLog::ReplayThread::entry()+0x11) [0x564795b35e6d] 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 8: (Thread::entry_wrapper()+0x43) [0x7fd1cdf569f5] 2024-05-09T05:45:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 9: (Thread::_entry_func(void*)+0xd) [0x7fd1cdf56a11] 2024-05-09T05:45:05.117 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7fd1ccaf91ca] 2024-05-09T05:45:05.117 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 11: clone() 2024-05-09T05:45:05.117 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2024-05-09T05:45:04.663+0000 7fd1bb741700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:45:05.117 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: in thread 7fd1bb741700 thread_name:md_log_replay 2024-05-09T05:45:05.117 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2024-05-09T05:45:05.117 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:45:05.117 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd1ccb03d20] 2024-05-09T05:45:05.117 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x564795acbda1] 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x564795acc238] 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x564795ac68af] 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x564795787cff] 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 6: (MDLog::_replay_thread()+0x1f38) [0x564795b2c92e] 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 7: (MDLog::ReplayThread::entry()+0x11) [0x564795b35e6d] 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 8: (Thread::entry_wrapper()+0x43) [0x7fd1cdf569f5] 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 9: (Thread::_entry_func(void*)+0xd) [0x7fd1cdf56a11] 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7fd1ccaf91ca] 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 11: clone() 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2024-05-09T05:45:05.118 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 0> 2024-05-09T05:45:04.663+0000 7fd1bb741700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: in thread 7fd1bb741700 thread_name:md_log_replay 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd1ccb03d20] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x564795acbda1] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x564795acc238] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x564795ac68af] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x564795787cff] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 6: (MDLog::_replay_thread()+0x1f38) [0x564795b2c92e] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 7: (MDLog::ReplayThread::entry()+0x11) [0x564795b35e6d] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 8: (Thread::entry_wrapper()+0x43) [0x7fd1cdf569f5] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 9: (Thread::_entry_func(void*)+0xd) [0x7fd1cdf56a11] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7fd1ccaf91ca] 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 11: clone() 2024-05-09T05:45:05.119 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 0> 2024-05-09T05:45:04.663+0000 7fd1bb741700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: in thread 7fd1bb741700 thread_name:md_log_replay 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd1ccb03d20] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x564795acbda1] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x564795acc238] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x564795ac68af] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x564795787cff] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 6: (MDLog::_replay_thread()+0x1f38) [0x564795b2c92e] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 7: (MDLog::ReplayThread::entry()+0x11) [0x564795b35e6d] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 8: (Thread::entry_wrapper()+0x43) [0x7fd1cdf569f5] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 9: (Thread::_entry_func(void*)+0xd) [0x7fd1cdf56a11] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7fd1ccaf91ca] 2024-05-09T05:45:05.120 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 11: clone() 2024-05-09T05:45:05.121 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:45:05.121 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[63258]: 2024-05-09T05:45:05.121 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:04 smithi195 podman[68789]: 2024-05-09 05:45:04.728806872 +0000 UTC m=+0.023033254 container died 4ab77f7ac162183406c6b4c1d71443f1dbae1d3db5818693acc6cd646cce81c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T05:45:05.536 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:05 smithi195 ceph-mon[33844]: pgmap v209: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 40 op/s 2024-05-09T05:45:05.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:05 smithi115 ceph-mon[30349]: pgmap v209: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 40 op/s 2024-05-09T05:45:05.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:05 smithi161 ceph-mon[33805]: pgmap v209: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 40 op/s 2024-05-09T05:45:05.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:05 smithi195 podman[68789]: 2024-05-09 05:45:05.544407458 +0000 UTC m=+0.838633853 container remove 4ab77f7ac162183406c6b4c1d71443f1dbae1d3db5818693acc6cd646cce81c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:45:05.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:05 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-05-09T05:45:05.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:05 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-05-09T05:45:05.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:05 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Consumed 2.317s CPU time. 2024-05-09T05:45:06.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:06 smithi195 ceph-mon[33844]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T05:45:06.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:06 smithi115 ceph-mon[30349]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T05:45:06.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:06 smithi161 ceph-mon[33805]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T05:45:07.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:07 smithi195 ceph-mon[33844]: pgmap v211: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 38 op/s 2024-05-09T05:45:07.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:07 smithi115 ceph-mon[30349]: pgmap v211: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 38 op/s 2024-05-09T05:45:07.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:07 smithi161 ceph-mon[33805]: pgmap v211: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 38 op/s 2024-05-09T05:45:09.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:09 smithi195 ceph-mon[33844]: pgmap v212: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 707 KiB/s wr, 33 op/s 2024-05-09T05:45:09.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:09 smithi195 ceph-mon[33844]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T05:45:09.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:09 smithi115 ceph-mon[30349]: pgmap v212: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 707 KiB/s wr, 33 op/s 2024-05-09T05:45:09.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:09 smithi115 ceph-mon[30349]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T05:45:09.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:09 smithi161 ceph-mon[33805]: pgmap v212: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 707 KiB/s wr, 33 op/s 2024-05-09T05:45:09.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:09 smithi161 ceph-mon[33805]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T05:45:11.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:11 smithi195 ceph-mon[33844]: pgmap v214: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 883 KiB/s wr, 42 op/s 2024-05-09T05:45:11.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:11 smithi115 ceph-mon[30349]: pgmap v214: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 883 KiB/s wr, 42 op/s 2024-05-09T05:45:11.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:11 smithi161 ceph-mon[33805]: pgmap v214: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 883 KiB/s wr, 42 op/s 2024-05-09T05:45:13.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:13 smithi195 ceph-mon[33844]: pgmap v215: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-09T05:45:13.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:13 smithi161 ceph-mon[33805]: pgmap v215: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-09T05:45:13.662 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:13 smithi115 ceph-mon[30349]: pgmap v215: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-09T05:45:15.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:15 smithi195 ceph-mon[33844]: pgmap v216: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 694 B/s wr, 10 op/s 2024-05-09T05:45:15.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:15 smithi115 ceph-mon[30349]: pgmap v216: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 694 B/s wr, 10 op/s 2024-05-09T05:45:15.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:15 smithi161 ceph-mon[33805]: pgmap v216: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 694 B/s wr, 10 op/s 2024-05-09T05:45:16.137 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:15 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 1. 2024-05-09T05:45:16.138 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:15 smithi195 systemd[1]: Stopped Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:45:16.138 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:15 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Consumed 2.317s CPU time. 2024-05-09T05:45:16.138 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:15 smithi195 systemd[1]: Starting Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:45:16.138 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:16 smithi195 podman[68888]: 2024-05-09 05:45:16.01481835 +0000 UTC m=+0.018489857 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:45:16.138 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:16 smithi195 podman[68888]: 2024-05-09 05:45:16.145899646 +0000 UTC m=+0.149571135 container create 7231fae4353efd7d28cf9d936789ef46897b78fe7b0d9c9867eb9a977fa29407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T05:45:16.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:16 smithi195 podman[68888]: 2024-05-09 05:45:16.407200299 +0000 UTC m=+0.410871794 container init 7231fae4353efd7d28cf9d936789ef46897b78fe7b0d9c9867eb9a977fa29407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:45:16.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:16 smithi195 podman[68888]: 2024-05-09 05:45:16.410030658 +0000 UTC m=+0.413702150 container start 7231fae4353efd7d28cf9d936789ef46897b78fe7b0d9c9867eb9a977fa29407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T05:45:16.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:16 smithi195 ceph-mds[68907]: -- 172.21.15.195:0/1958389922 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55928a61ba40 con 0x55928a663a80 2024-05-09T05:45:16.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:16 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: starting mds.l at 2024-05-09T05:45:16.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:16 smithi195 bash[68888]: 7231fae4353efd7d28cf9d936789ef46897b78fe7b0d9c9867eb9a977fa29407 2024-05-09T05:45:16.865 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:45:16 smithi195 systemd[1]: Started Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:45:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 -- 172.21.15.115:0/462125846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f810e660 msgr2=0x7f11f8110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/462125846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f810e660 0x7f11f8110a80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009ca0 tx=0x7f11f002a8c0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 -- 172.21.15.115:0/462125846 shutdown_connections 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/462125846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f810e660 0x7f11f8110a80 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/462125846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11f81022c0 0x7f11f81026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/462125846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f8108c10 0x7f11f8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 -- 172.21.15.115:0/462125846 >> 172.21.15.115:0/462125846 conn(0x7f11f80fd9d0 msgr2=0x7f11f80ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 -- 172.21.15.115:0/462125846 shutdown_connections 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 -- 172.21.15.115:0/462125846 wait complete. 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 Processor -- start 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.398+0000 7f11ff1b2640 1 -- start start 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11ff1b2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11f81022c0 0x7f11f8102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11ff1b2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f8108c10 0x7f11f81034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11ff1b2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f810e660 0x7f11f81039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11ff1b2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8113e60 con 0x7f11f8108c10 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11ff1b2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8113ce0 con 0x7f11f810e660 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11ff1b2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8113fe0 con 0x7f11f81022c0 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f8108c10 0x7f11f81034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f8108c10 0x7f11f81034a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44662/0 (socket says 172.21.15.115:44662) 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11f7fff640 1 -- 172.21.15.115:0/4163139973 learned_addr learned my addr 172.21.15.115:0/4163139973 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11fd74f640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f810e660 0x7f11f81039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11f81022c0 0x7f11f8102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11f7fff640 1 -- 172.21.15.115:0/4163139973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11f81022c0 msgr2=0x7f11f8102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11f7fff640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11f81022c0 0x7f11f8102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11f7fff640 1 -- 172.21.15.115:0/4163139973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f810e660 msgr2=0x7f11f81039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11f7fff640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f810e660 0x7f11f81039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11f7fff640 1 -- 172.21.15.115:0/4163139973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f81a8b80 con 0x7f11f8108c10 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.399+0000 7f11f7fff640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f8108c10 0x7f11f81034a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f11e400d770 tx=0x7f11e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.400+0000 7f11f5ffb640 1 -- 172.21.15.115:0/4163139973 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4017070 con 0x7f11f8108c10 2024-05-09T05:45:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.400+0000 7f11f5ffb640 1 -- 172.21.15.115:0/4163139973 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11e4009c30 con 0x7f11f8108c10 2024-05-09T05:45:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.400+0000 7f11ff1b2640 1 -- 172.21.15.115:0/4163139973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f81a8e70 con 0x7f11f8108c10 2024-05-09T05:45:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.400+0000 7f11f5ffb640 1 -- 172.21.15.115:0/4163139973 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e400f620 con 0x7f11f8108c10 2024-05-09T05:45:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.400+0000 7f11ff1b2640 1 -- 172.21.15.115:0/4163139973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f81a92e0 con 0x7f11f8108c10 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.402+0000 7f11f5ffb640 1 -- 172.21.15.115:0/4163139973 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f11e401a070 con 0x7f11f8108c10 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f1206064740 1 Processor -- start 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f1206064740 1 -- start start 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f1206064740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe9a1b0 0x55c86023d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f1206064740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe712f0 0x55c86023dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f1206064740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe794d0 0x55c86023e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f1206064740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c8601a27c0 con 0x55c85fe9a1b0 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f1206064740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c86013e0e0 con 0x55c85fe712f0 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f1206064740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c86019e780 con 0x55c85fe794d0 2024-05-09T05:45:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11fd74f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe9a1b0 0x55c86023d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11fd74f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe9a1b0 0x55c86023d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44666/0 (socket says 172.21.15.115:44666) 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11fd74f640 1 -- 172.21.15.115:0/2335922663 learned_addr learned my addr 172.21.15.115:0/2335922663 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/2335922663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe712f0 0x55c86023dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11f7fff640 1 --2- 172.21.15.115:0/2335922663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe794d0 0x55c86023e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11f5ffb640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11e80b2950 0x7f11e80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11fd74f640 1 -- 172.21.15.115:0/2335922663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe794d0 msgr2=0x55c86023e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11fd74f640 1 --2- 172.21.15.115:0/2335922663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe794d0 0x55c86023e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11fd74f640 1 -- 172.21.15.115:0/2335922663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe712f0 msgr2=0x55c86023dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11fd74f640 1 --2- 172.21.15.115:0/2335922663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe712f0 0x55c86023dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.403+0000 7f11fd74f640 1 -- 172.21.15.115:0/2335922663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8601677c0 con 0x55c85fe9a1b0 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.404+0000 7f11f5ffb640 1 -- 172.21.15.115:0/4163139973 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f11e40e2450 con 0x7f11f8108c10 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.404+0000 7f11fd74f640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11e80b2950 0x7f11e80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.404+0000 7f11fd74f640 1 --2- 172.21.15.115:0/2335922663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe9a1b0 0x55c86023d680 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009ca0 tx=0x7f11f00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bb93a110c55b9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.404+0000 7f11e2ffd640 1 -- 172.21.15.115:0/2335922663 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0042040 con 0x55c85fe9a1b0 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.404+0000 7f11fd74f640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11e80b2950 0x7f11e80b4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11f004d000 tx=0x7f11f004e000 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.404+0000 7f11e2ffd640 1 -- 172.21.15.115:0/2335922663 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11f0036070 con 0x55c85fe9a1b0 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.404+0000 7f11e2ffd640 1 -- 172.21.15.115:0/2335922663 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f00307e0 con 0x55c85fe9a1b0 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 -- 172.21.15.115:0/2335922663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe9a1b0 msgr2=0x55c86023d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 --2- 172.21.15.115:0/2335922663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe9a1b0 0x55c86023d680 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009ca0 tx=0x7f11f00315b0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 -- 172.21.15.115:0/2335922663 shutdown_connections 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 --2- 172.21.15.115:0/2335922663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe794d0 0x55c86023e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 --2- 172.21.15.115:0/2335922663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe712f0 0x55c86023dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 --2- 172.21.15.115:0/2335922663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe9a1b0 0x55c86023d680 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 -- 172.21.15.115:0/2335922663 >> 172.21.15.115:0/2335922663 conn(0x55c85fe8e300 msgr2=0x55c860239e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 -- 172.21.15.115:0/2335922663 shutdown_connections 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 -- 172.21.15.115:0/2335922663 wait complete. 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 -- start start 2024-05-09T05:45:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.405+0000 7f1206064740 1 Processor -- start 2024-05-09T05:45:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f1206064740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860249770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f1206064740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c860249cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f1206064740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c860247bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f1206064740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c86024a7b0 con 0x55c85fe712f0 2024-05-09T05:45:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f1206064740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c8601299c0 con 0x55c85fe794d0 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f1206064740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c86024a930 con 0x55c85fe9a1b0 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f1206064740 10 client.0 _ll_register_callbacks cb 0x55c860156020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fcf4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860249770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fcf4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860249770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44678/0 (socket says 172.21.15.115:44678) 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fcf4e640 1 -- 172.21.15.115:0/187124161 learned_addr learned my addr 172.21.15.115:0/187124161 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11f7fff640 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c860249cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fd74f640 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c860247bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fcf4e640 1 -- 172.21.15.115:0/187124161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 msgr2=0x55c860247bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c860247bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fcf4e640 1 -- 172.21.15.115:0/187124161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 msgr2=0x55c860249cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c860249cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fcf4e640 1 -- 172.21.15.115:0/187124161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8601501f0 con 0x55c85fe712f0 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11f7fff640 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c860249cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.406+0000 7f11fd74f640 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c860247bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860249770 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f11f806b2e0 tx=0x7f11ec011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f326fbad3c2654d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f11d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f11d7fff640 1 -- 172.21.15.115:0/187124161 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00fe10 con 0x55c85fe712f0 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f11d7fff640 1 -- 172.21.15.115:0/187124161 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f11ec005020 con 0x55c85fe712f0 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f1206064740 1 -- 172.21.15.115:0/187124161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f0036070 con 0x55c85fe712f0 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f11d7fff640 1 -- 172.21.15.115:0/187124161 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f11ec024050 con 0x55c85fe712f0 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f1206064740 10 client.17448 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f1206064740 1 -- 172.21.15.115:0/187124161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c85ff05790 con 0x55c85fe712f0 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f11d7fff640 1 -- 172.21.15.115:0/187124161 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11f0036070 con 0x55c85fe712f0 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f11d7fff640 1 -- 172.21.15.115:0/187124161 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f11ec018030 con 0x55c85fe712f0 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f1206064740 10 client.17448 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f1206064740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f1206064740 20 client.17448 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f1206064740 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c85fe58d80 0x55c86025a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f1206064740 4 client.17448 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.407+0000 7f1206064740 1 -- 172.21.15.115:0/187124161 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8601b03d0 con 0x55c85fe58d80 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.408+0000 7f11f7fff640 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c85fe58d80 0x55c86025a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.408+0000 7f11d7fff640 10 client.17448 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.408+0000 7f11f7fff640 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c85fe58d80 0x55c86025a070 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14bf46e5e092876b server_cookie=322f62cfc143b6e7 in_seq=0 out_seq=0 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f11d7fff640 1 -- 172.21.15.115:0/187124161 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c8601b03d0 con 0x55c85fe58d80 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f11d7fff640 10 client.17448 handle_command_reply: tid=0 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 client.17448 shutdown 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 -- 172.21.15.115:0/187124161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 msgr2=0x55c860249770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860249770 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f11f806b2e0 tx=0x7f11ec011bc0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 -- 172.21.15.115:0/187124161 shutdown_connections 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c85fe58d80 0x55c86025a070 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c860247bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c860249cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f11d7fff640 0 client.17448 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 --2- 172.21.15.115:0/187124161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860249770 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 -- 172.21.15.115:0/187124161 >> 172.21.15.115:0/187124161 conn(0x55c85fe8e300 msgr2=0x55c860238700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 -- 172.21.15.115:0/187124161 shutdown_connections 2024-05-09T05:45:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.409+0000 7f1206064740 1 -- 172.21.15.115:0/187124161 wait complete. 2024-05-09T05:45:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.410+0000 7f1206064740 20 client.17448 trim_cache size 0 max 16384 2024-05-09T05:45:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f1206064740 1 Processor -- start 2024-05-09T05:45:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f1206064740 1 -- start start 2024-05-09T05:45:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f1206064740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860238e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f1206064740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c86024fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f1206064740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c860250100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f1206064740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81b3170 con 0x55c85fe712f0 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f1206064740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81b4cb0 con 0x55c85fe794d0 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f1206064740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81b3840 con 0x55c85fe9a1b0 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fcf4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860238e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fcf4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860238e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44694/0 (socket says 172.21.15.115:44694) 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fcf4e640 1 -- 172.21.15.115:0/3814516667 learned_addr learned my addr 172.21.15.115:0/3814516667 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11f7fff640 1 --2- 172.21.15.115:0/3814516667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c86024fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fd74f640 1 --2- 172.21.15.115:0/3814516667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c860250100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fcf4e640 1 -- 172.21.15.115:0/3814516667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 msgr2=0x55c860250100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/3814516667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c860250100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fcf4e640 1 -- 172.21.15.115:0/3814516667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 msgr2=0x55c86024fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/3814516667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c86024fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fcf4e640 1 -- 172.21.15.115:0/3814516667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c85ff7c5f0 con 0x55c85fe712f0 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.425+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/3814516667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860238e10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005940 tx=0x7f11ec00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d45f261417bd2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f11e27fc640 1 -- 172.21.15.115:0/3814516667 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec02c070 con 0x55c85fe712f0 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f11e27fc640 1 -- 172.21.15.115:0/3814516667 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11ec011040 con 0x55c85fe712f0 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f11e27fc640 1 -- 172.21.15.115:0/3814516667 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec004aa0 con 0x55c85fe712f0 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f1206064740 1 -- 172.21.15.115:0/3814516667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 msgr2=0x55c860238e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f1206064740 1 --2- 172.21.15.115:0/3814516667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860238e10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005940 tx=0x7f11ec00f810 comp rx=0 tx=0).stop 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f1206064740 1 -- 172.21.15.115:0/3814516667 shutdown_connections 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f1206064740 1 --2- 172.21.15.115:0/3814516667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c860250100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f1206064740 1 --2- 172.21.15.115:0/3814516667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c86024fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f1206064740 1 --2- 172.21.15.115:0/3814516667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c860238e10 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f1206064740 1 -- 172.21.15.115:0/3814516667 >> 172.21.15.115:0/3814516667 conn(0x55c85fe8e300 msgr2=0x55c860238700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f1206064740 1 -- 172.21.15.115:0/3814516667 shutdown_connections 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.426+0000 7f1206064740 1 -- 172.21.15.115:0/3814516667 wait complete. 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f1206064740 1 -- start start 2024-05-09T05:45:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f1206064740 1 Processor -- start 2024-05-09T05:45:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f1206064740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c8602501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f1206064740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c86024d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f1206064740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c86024db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f1206064740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81b45e0 con 0x55c85fe712f0 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f1206064740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81b3f10 con 0x55c85fe794d0 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f1206064740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81b1150 con 0x55c85fe9a1b0 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f1206064740 10 client.0 _ll_register_callbacks cb 0x7f11ec00ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f11fcf4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c8602501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f11fcf4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c8602501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44708/0 (socket says 172.21.15.115:44708) 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.427+0000 7f11fcf4e640 1 -- 172.21.15.115:0/673157662 learned_addr learned my addr 172.21.15.115:0/673157662 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11fd74f640 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c86024db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11f7fff640 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c86024d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11fcf4e640 1 -- 172.21.15.115:0/673157662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 msgr2=0x55c86024db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c86024db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11fcf4e640 1 -- 172.21.15.115:0/673157662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 msgr2=0x55c86024d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c86024d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11fcf4e640 1 -- 172.21.15.115:0/673157662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c860166ed0 con 0x55c85fe712f0 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11fcf4e640 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c8602501f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00c440 tx=0x7f11ec00f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8edcd1424b1904f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11d7fff640 1 -- 172.21.15.115:0/673157662 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec02c040 con 0x55c85fe712f0 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11d7fff640 1 -- 172.21.15.115:0/673157662 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f11ec004220 con 0x55c85fe712f0 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f1206064740 1 -- 172.21.15.115:0/673157662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ec011040 con 0x55c85fe712f0 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11d7fff640 1 -- 172.21.15.115:0/673157662 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f11ec042050 con 0x55c85fe712f0 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f1206064740 10 client.17454 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f1206064740 1 -- 172.21.15.115:0/673157662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c8602c2210 con 0x55c85fe712f0 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.428+0000 7f11d7fff640 1 -- 172.21.15.115:0/673157662 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11ec011040 con 0x55c85fe712f0 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f11d7fff640 1 -- 172.21.15.115:0/673157662 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f11ec038030 con 0x55c85fe712f0 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f1206064740 10 client.17454 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f1206064740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f1206064740 20 client.17454 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f1206064740 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c85fe2e1c0 0x55c8602c4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f1206064740 4 client.17454 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:45:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f1206064740 1 -- 172.21.15.115:0/673157662 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8601aa240 con 0x55c85fe2e1c0 2024-05-09T05:45:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f11f7fff640 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c85fe2e1c0 0x55c8602c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f11f7fff640 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c85fe2e1c0 0x55c8602c4950 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc1a0fe5b5d92d9 server_cookie=c9e1bcc35edbfa95 in_seq=0 out_seq=0 2024-05-09T05:45:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.429+0000 7f11d7fff640 10 client.17454 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f11d7fff640 1 -- 172.21.15.115:0/673157662 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c8601aa240 con 0x55c85fe2e1c0 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f11d7fff640 10 client.17454 handle_command_reply: tid=0 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 client.17454 shutdown 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 -- 172.21.15.115:0/673157662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 msgr2=0x55c8602501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c8602501f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00c440 tx=0x7f11ec00f8b0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 -- 172.21.15.115:0/673157662 shutdown_connections 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c85fe2e1c0 0x55c8602c4950 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c85fe9a1b0 0x55c86024db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c85fe794d0 0x55c86024d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f11d7fff640 0 client.17454 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 --2- 172.21.15.115:0/673157662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c85fe712f0 0x55c8602501f0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 -- 172.21.15.115:0/673157662 >> 172.21.15.115:0/673157662 conn(0x55c85fe8e300 msgr2=0x55c860239680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 -- 172.21.15.115:0/673157662 shutdown_connections 2024-05-09T05:45:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 1 -- 172.21.15.115:0/673157662 wait complete. 2024-05-09T05:45:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.436+0000 7f1206064740 20 client.17454 trim_cache size 0 max 16384 2024-05-09T05:45:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:45:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:45:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:45:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 -- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11e80b2950 msgr2=0x7f11e80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11e80b2950 0x7f11e80b4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11f004d000 tx=0x7f11f004e000 comp rx=0 tx=0).stop 2024-05-09T05:45:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 -- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f8108c10 msgr2=0x7f11f81034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f8108c10 0x7f11f81034a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f11e400d770 tx=0x7f11e400dc30 comp rx=0 tx=0).stop 2024-05-09T05:45:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 -- 172.21.15.115:0/4163139973 shutdown_connections 2024-05-09T05:45:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11e80b2950 0x7f11e80b4e60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f810e660 0x7f11f81039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11f8108c10 0x7f11f81034a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 --2- 172.21.15.115:0/4163139973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11f81022c0 0x7f11f8102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 -- 172.21.15.115:0/4163139973 >> 172.21.15.115:0/4163139973 conn(0x7f11f80fd9d0 msgr2=0x7f11f810fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 -- 172.21.15.115:0/4163139973 shutdown_connections 2024-05-09T05:45:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:17.437+0000 7f11ff1b2640 1 -- 172.21.15.115:0/4163139973 wait complete. 2024-05-09T05:45:17.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:17 smithi195 ceph-mon[33844]: pgmap v217: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 614 B/s wr, 9 op/s 2024-05-09T05:45:17.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:17 smithi195 ceph-mon[33844]: daemon mds.l restarted 2024-05-09T05:45:17.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:17 smithi115 ceph-mon[30349]: pgmap v217: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 614 B/s wr, 9 op/s 2024-05-09T05:45:17.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:17 smithi115 ceph-mon[30349]: daemon mds.l restarted 2024-05-09T05:45:17.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:17 smithi161 ceph-mon[33805]: pgmap v217: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 614 B/s wr, 9 op/s 2024-05-09T05:45:17.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:17 smithi161 ceph-mon[33805]: daemon mds.l restarted 2024-05-09T05:45:18.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:18 smithi195 ceph-mon[33844]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T05:45:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:18 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6832/3665923682,v1:172.21.15.195:6833/3665923682] up:boot 2024-05-09T05:45:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:18 smithi195 ceph-mon[33844]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:45:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:45:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:18 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:45:18.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:18 smithi115 ceph-mon[30349]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T05:45:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:18 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6832/3665923682,v1:172.21.15.195:6833/3665923682] up:boot 2024-05-09T05:45:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:18 smithi115 ceph-mon[30349]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:45:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:45:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:18 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:45:18.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:18 smithi161 ceph-mon[33805]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T05:45:18.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:18 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6832/3665923682,v1:172.21.15.195:6833/3665923682] up:boot 2024-05-09T05:45:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:18 smithi161 ceph-mon[33805]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:45:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:45:18.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:18 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:45:19.368 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:45:19.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:19 smithi195 ceph-mon[33844]: pgmap v219: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-09T05:45:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:19.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:19 smithi115 ceph-mon[30349]: pgmap v219: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-09T05:45:19.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:19 smithi161 ceph-mon[33805]: pgmap v219: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-09T05:45:21.355 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:21 smithi115 ceph-mon[30349]: pgmap v220: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-09T05:45:21.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.472+0000 7fe993a4b700 1 -- 172.21.15.115:0/1663457812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 msgr2=0x7fe98c108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:21.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.472+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1663457812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 0x7fe98c108b90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe98400b0a0 tx=0x7fe98400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.472+0000 7fe993a4b700 1 -- 172.21.15.115:0/1663457812 shutdown_connections 2024-05-09T05:45:21.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.472+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1663457812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c103270 0x7fe98c103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.472+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1663457812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 0x7fe98c108b90 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.472+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1663457812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe98c103bc0 0x7fe98c1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.472+0000 7fe993a4b700 1 -- 172.21.15.115:0/1663457812 >> 172.21.15.115:0/1663457812 conn(0x7fe98c0fef30 msgr2=0x7fe98c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:21.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.472+0000 7fe993a4b700 1 -- 172.21.15.115:0/1663457812 shutdown_connections 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.472+0000 7fe993a4b700 1 -- 172.21.15.115:0/1663457812 wait complete. 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.473+0000 7fe993a4b700 1 Processor -- start 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.473+0000 7fe993a4b700 1 -- start start 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.473+0000 7fe993a4b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe98c103270 0x7fe98c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.473+0000 7fe993a4b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c103bc0 0x7fe98c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.473+0000 7fe993a4b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 0x7fe98c1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.473+0000 7fe993a4b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c11a910 con 0x7fe98c1086c0 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.473+0000 7fe993a4b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c11a790 con 0x7fe98c103bc0 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.473+0000 7fe993a4b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c11aa90 con 0x7fe98c103270 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe991fe8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 0x7fe98c1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe9917e7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe98c103270 0x7fe98c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe991fe8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 0x7fe98c1a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44722/0 (socket says 172.21.15.115:44722) 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe991fe8700 1 -- 172.21.15.115:0/1417393275 learned_addr learned my addr 172.21.15.115:0/1417393275 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:21.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe990fe6700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c103bc0 0x7fe98c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe991fe8700 1 -- 172.21.15.115:0/1417393275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe98c103270 msgr2=0x7fe98c1a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe991fe8700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe98c103270 0x7fe98c1a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe991fe8700 1 -- 172.21.15.115:0/1417393275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c103bc0 msgr2=0x7fe98c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe991fe8700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c103bc0 0x7fe98c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe991fe8700 1 -- 172.21.15.115:0/1417393275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c1a61d0 con 0x7fe98c1086c0 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe991fe8700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 0x7fe98c1a5b30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe98400b830 tx=0x7fe984004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe9827fc700 1 -- 172.21.15.115:0/1417393275 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984021070 con 0x7fe98c1086c0 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe9827fc700 1 -- 172.21.15.115:0/1417393275 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe984007b80 con 0x7fe98c1086c0 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c1a6460 con 0x7fe98c1086c0 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe9827fc700 1 -- 172.21.15.115:0/1417393275 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9840052a0 con 0x7fe98c1086c0 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.474+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c1adde0 con 0x7fe98c1086c0 2024-05-09T05:45:21.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.475+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98c06c920 con 0x7fe98c1086c0 2024-05-09T05:45:21.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.477+0000 7fe9827fc700 1 -- 172.21.15.115:0/1417393275 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fe984024070 con 0x7fe98c1086c0 2024-05-09T05:45:21.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.478+0000 7fe9827fc700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9780b2a70 0x7fe9780b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:21.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.478+0000 7fe9917e7700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9780b2a70 0x7fe9780b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:21.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.478+0000 7fe9827fc700 1 -- 172.21.15.115:0/1417393275 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe98400f020 con 0x7fe98c1086c0 2024-05-09T05:45:21.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.478+0000 7fe9917e7700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9780b2a70 0x7fe9780b4f80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe988005950 tx=0x7fe98800b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:21.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.483+0000 7fe9827fc700 1 -- 172.21.15.115:0/1417393275 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe98402b020 con 0x7fe98c1086c0 2024-05-09T05:45:21.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:21 smithi195 ceph-mon[33844]: pgmap v220: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-09T05:45:21.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:21 smithi161 ceph-mon[33805]: pgmap v220: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-09T05:45:21.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.733+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe98c1adfb0 con 0x7fe98c1086c0 2024-05-09T05:45:21.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.734+0000 7fe9827fc700 1 -- 172.21.15.115:0/1417393275 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11953 (secure 0 0 0) 0x7fe9840ab590 con 0x7fe98c1086c0 2024-05-09T05:45:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:45:21.728 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:45:17.254442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26869":{"gid":26869,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/3665923682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3665923682},{"type":"v1","addr":"172.21.15.195:6833","nonce":3665923682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:45:21.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.736+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9780b2a70 msgr2=0x7fe9780b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:21.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.736+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9780b2a70 0x7fe9780b4f80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe988005950 tx=0x7fe98800b410 comp rx=0 tx=0).stop 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.736+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 msgr2=0x7fe98c1a5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.736+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 0x7fe98c1a5b30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe98400b830 tx=0x7fe984004d00 comp rx=0 tx=0).stop 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.736+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 shutdown_connections 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.736+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9780b2a70 0x7fe9780b4f80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.737+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe98c1086c0 0x7fe98c1a5b30 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.737+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe98c103270 0x7fe98c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.737+0000 7fe993a4b700 1 --2- 172.21.15.115:0/1417393275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c103bc0 0x7fe98c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.737+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 >> 172.21.15.115:0/1417393275 conn(0x7fe98c0fef30 msgr2=0x7fe98c106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.737+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 shutdown_connections 2024-05-09T05:45:21.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:21.737+0000 7fe993a4b700 1 -- 172.21.15.115:0/1417393275 wait complete. 2024-05-09T05:45:21.732 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2024-05-09T05:45:22.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:22 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1417393275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:45:22.542 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id b7601b26 --release --await' 2024-05-09T05:45:22.543 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id b7601b26 --release --await 2024-05-09T05:45:22.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:22 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1417393275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:45:22.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:22 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1417393275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:45:22.771 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:45:22.791 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:23.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:23 smithi115 ceph-mon[30349]: pgmap v221: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-09T05:45:23.564 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:23.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:23 smithi195 ceph-mon[33844]: pgmap v221: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-09T05:45:23.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:23 smithi161 ceph-mon[33805]: pgmap v221: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-09T05:45:25.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:25 smithi115 ceph-mon[30349]: pgmap v222: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-09T05:45:25.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:25 smithi195 ceph-mon[33844]: pgmap v222: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-09T05:45:25.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:25 smithi161 ceph-mon[33805]: pgmap v222: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-09T05:45:26.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.116+0000 7f39c602c700 1 -- 172.21.15.115:0/3581622568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c01045f0 msgr2=0x7f39c0104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.116+0000 7f39c602c700 1 --2- 172.21.15.115:0/3581622568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c01045f0 0x7f39c0104a00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f39b000b0d0 tx=0x7f39b000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.116+0000 7f39c602c700 1 -- 172.21.15.115:0/3581622568 shutdown_connections 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.116+0000 7f39c602c700 1 --2- 172.21.15.115:0/3581622568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39c0109870 0x7f39c01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.116+0000 7f39c602c700 1 --2- 172.21.15.115:0/3581622568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c01045f0 0x7f39c0104a00 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.116+0000 7f39c602c700 1 --2- 172.21.15.115:0/3581622568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0108f20 0x7f39c0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.116+0000 7f39c602c700 1 -- 172.21.15.115:0/3581622568 >> 172.21.15.115:0/3581622568 conn(0x7f39c00ffec0 msgr2=0x7f39c0102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.116+0000 7f39c602c700 1 -- 172.21.15.115:0/3581622568 shutdown_connections 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.116+0000 7f39c602c700 1 -- 172.21.15.115:0/3581622568 wait complete. 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.117+0000 7f39c602c700 1 Processor -- start 2024-05-09T05:45:26.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.117+0000 7f39c602c700 1 -- start start 2024-05-09T05:45:26.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.117+0000 7f39c602c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39c01045f0 0x7f39c0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39c602c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c0108f20 0x7f39c0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39c602c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0109870 0x7f39c0077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39c602c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39c0114350 con 0x7f39c0108f20 2024-05-09T05:45:26.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39c602c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01141d0 con 0x7f39c01045f0 2024-05-09T05:45:26.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39c602c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01144d0 con 0x7f39c0109870 2024-05-09T05:45:26.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39beffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c0108f20 0x7f39c0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39beffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c0108f20 0x7f39c0077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53506/0 (socket says 172.21.15.115:53506) 2024-05-09T05:45:26.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39beffd700 1 -- 172.21.15.115:0/1026663856 learned_addr learned my addr 172.21.15.115:0/1026663856 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39c01045f0 0x7f39c0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39bffff700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0109870 0x7f39c0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39beffd700 1 -- 172.21.15.115:0/1026663856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0109870 msgr2=0x7f39c0077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39beffd700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0109870 0x7f39c0077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39beffd700 1 -- 172.21.15.115:0/1026663856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39c01045f0 msgr2=0x7f39c0076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39beffd700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39c01045f0 0x7f39c0076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39beffd700 1 -- 172.21.15.115:0/1026663856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c01a9120 con 0x7f39c0108f20 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39beffd700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c0108f20 0x7f39c0077110 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f39b400d980 tx=0x7f39b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39bcff9700 1 -- 172.21.15.115:0/1026663856 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4017070 con 0x7f39c0108f20 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39c602c700 1 -- 172.21.15.115:0/1026663856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c01a9410 con 0x7f39c0108f20 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.118+0000 7f39bcff9700 1 -- 172.21.15.115:0/1026663856 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39b4009dc0 con 0x7f39c0108f20 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.119+0000 7f39bcff9700 1 -- 172.21.15.115:0/1026663856 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b400f3c0 con 0x7f39c0108f20 2024-05-09T05:45:26.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.119+0000 7f39c602c700 1 -- 172.21.15.115:0/1026663856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c01a9950 con 0x7f39c0108f20 2024-05-09T05:45:26.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.122+0000 7f39bcff9700 1 -- 172.21.15.115:0/1026663856 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f39b400f560 con 0x7f39c0108f20 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39bcff9700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f39a80b29f0 0x7f39a80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f39a80b29f0 0x7f39a80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39bcff9700 1 -- 172.21.15.115:0/1026663856 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f39b40e2410 con 0x7f39c0108f20 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39d4a42b80 1 Processor -- start 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39d4a42b80 1 -- start start 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f39a80b29f0 0x7f39a80b4f00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009fd0 tx=0x7f39b000b040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097d43e00 0x55e097e45c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 0x55e097e46190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e52d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e097d691a0 con 0x55e097e50900 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e097d0eaa0 con 0x55e097cc95a0 2024-05-09T05:45:26.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e097d65160 con 0x55e097d43e00 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.123+0000 7f39bffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e52d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39bffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e52d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53520/0 (socket says 172.21.15.115:53520) 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39bffff700 1 -- 172.21.15.115:0/2281867647 learned_addr learned my addr 172.21.15.115:0/2281867647 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/2281867647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097d43e00 0x55e097e45c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39beffd700 1 --2- 172.21.15.115:0/2281867647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 0x55e097e46190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39bffff700 1 -- 172.21.15.115:0/2281867647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097d43e00 msgr2=0x55e097e45c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39bffff700 1 --2- 172.21.15.115:0/2281867647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097d43e00 0x55e097e45c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39bffff700 1 -- 172.21.15.115:0/2281867647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 msgr2=0x55e097e46190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39bffff700 1 --2- 172.21.15.115:0/2281867647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 0x55e097e46190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39bffff700 1 -- 172.21.15.115:0/2281867647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e097cff600 con 0x55e097e50900 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39bffff700 1 --2- 172.21.15.115:0/2281867647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e52d20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f39b800cb50 tx=0x7f39b800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71832cdb6fe106a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39a5ffb700 1 -- 172.21.15.115:0/2281867647 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b800ec50 con 0x55e097e50900 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39a5ffb700 1 -- 172.21.15.115:0/2281867647 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39b80044b0 con 0x55e097e50900 2024-05-09T05:45:26.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.124+0000 7f39a5ffb700 1 -- 172.21.15.115:0/2281867647 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8010430 con 0x55e097e50900 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.125+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2281867647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 msgr2=0x55e097e52d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.125+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2281867647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e52d20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f39b800cb50 tx=0x7f39b800ce60 comp rx=0 tx=0).stop 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.125+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2281867647 shutdown_connections 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.125+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2281867647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e52d20 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.125+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2281867647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097d43e00 0x55e097e45c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.125+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2281867647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 0x55e097e46190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.125+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2281867647 >> 172.21.15.115:0/2281867647 conn(0x55e097ccd070 msgr2=0x55e097e41b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.125+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2281867647 shutdown_connections 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.125+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2281867647 wait complete. 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39d4a42b80 1 -- start start 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39d4a42b80 1 Processor -- start 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 0x55e097e4d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e4de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097e50900 0x55e097e4e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e097e53320 con 0x55e097d43e00 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e097d03c20 con 0x55e097cc95a0 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e097e534a0 con 0x55e097e50900 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39d4a42b80 10 client.0 _ll_register_callbacks cb 0x55e097bda7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39beffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e4de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39beffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e4de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53532/0 (socket says 172.21.15.115:53532) 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39beffd700 1 -- 172.21.15.115:0/2756891221 learned_addr learned my addr 172.21.15.115:0/2756891221 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39bffff700 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097e50900 0x55e097e4e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.126+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 0x55e097e4d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f39beffd700 1 -- 172.21.15.115:0/2756891221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097e50900 msgr2=0x55e097e4e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f39beffd700 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097e50900 0x55e097e4e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f39beffd700 1 -- 172.21.15.115:0/2756891221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 msgr2=0x55e097e4d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f39beffd700 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 0x55e097e4d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f39beffd700 1 -- 172.21.15.115:0/2756891221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e097be4590 con 0x55e097d43e00 2024-05-09T05:45:26.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f39beffd700 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e4de90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f39b40a66c0 tx=0x7f39b40a8e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a01384004255b9a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f399effd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f399effd700 1 -- 172.21.15.115:0/2756891221 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4020040 con 0x55e097d43e00 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f399effd700 1 -- 172.21.15.115:0/2756891221 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f39b40b1e20 con 0x55e097d43e00 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2756891221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39b80044b0 con 0x55e097d43e00 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f399effd700 1 -- 172.21.15.115:0/2756891221 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f39b40a8f60 con 0x55e097d43e00 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f39d4a42b80 10 client.17472 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2756891221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e097e5dfd0 con 0x55e097d43e00 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.127+0000 7f399effd700 1 -- 172.21.15.115:0/2756891221 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39b80044b0 con 0x55e097d43e00 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f399effd700 1 -- 172.21.15.115:0/2756891221 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f39b40ca310 con 0x55e097d43e00 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f39d4a42b80 10 client.17472 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f39d4a42b80 10 client.17472 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:45:26.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f39d4a42b80 20 client.17472 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e097e60560 0x55e097e629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f39d4a42b80 4 client.17472 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2756891221 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e097d76b80 con 0x55e097e60560 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e097e60560 0x55e097e629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e097e60560 0x55e097e629c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e58ab9bfd72a6d71 server_cookie=870e3ae152407e1d in_seq=0 out_seq=0 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.128+0000 7f399effd700 10 client.17472 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f399effd700 1 -- 172.21.15.115:0/2756891221 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e097d76b80 con 0x55e097e60560 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f399effd700 10 client.17472 handle_command_reply: tid=0 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 client.17472 shutdown 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2756891221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 msgr2=0x55e097e4de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e4de90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f39b40a66c0 tx=0x7f39b40a8e90 comp rx=0 tx=0).stop 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2756891221 shutdown_connections 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097cc95a0 0x55e097e4d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e097e60560 0x55e097e629c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097e50900 0x55e097e4e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2756891221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e4de90 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2756891221 >> 172.21.15.115:0/2756891221 conn(0x55e097ccd070 msgr2=0x55e097e40de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:26.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f399effd700 0 client.17472 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:26.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.129+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2756891221 shutdown_connections 2024-05-09T05:45:26.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.130+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2756891221 wait complete. 2024-05-09T05:45:26.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.130+0000 7f39d4a42b80 20 client.17472 trim_cache size 0 max 16384 2024-05-09T05:45:26.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39d4a42b80 1 Processor -- start 2024-05-09T05:45:26.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39d4a42b80 1 -- start start 2024-05-09T05:45:26.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 0x55e097e4fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097d43e00 0x55e097e5bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e5bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01b26b0 con 0x55e097e50900 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01b41f0 con 0x55e097d43e00 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01b2d80 con 0x55e097cc95a0 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39bffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e5bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39bffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e5bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53546/0 (socket says 172.21.15.115:53546) 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39bffff700 1 -- 172.21.15.115:0/2171377206 learned_addr learned my addr 172.21.15.115:0/2171377206 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 0x55e097e4fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.153+0000 7f39beffd700 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097d43e00 0x55e097e5bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39bffff700 1 -- 172.21.15.115:0/2171377206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 msgr2=0x55e097e4fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39bffff700 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 0x55e097e4fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39bffff700 1 -- 172.21.15.115:0/2171377206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097d43e00 msgr2=0x55e097e5bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39bffff700 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097d43e00 0x55e097e5bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39bffff700 1 -- 172.21.15.115:0/2171377206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e097e3c030 con 0x55e097e50900 2024-05-09T05:45:26.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 0x55e097e4fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:26.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39bffff700 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e5bff0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f39b8000c00 tx=0x7f39b8003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22b4db0e94a76e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39a57fa700 1 -- 172.21.15.115:0/2171377206 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b802a070 con 0x55e097e50900 2024-05-09T05:45:26.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39a57fa700 1 -- 172.21.15.115:0/2171377206 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39b8005030 con 0x55e097e50900 2024-05-09T05:45:26.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.154+0000 7f39a57fa700 1 -- 172.21.15.115:0/2171377206 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8004970 con 0x55e097e50900 2024-05-09T05:45:26.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2171377206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 msgr2=0x55e097e5bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e5bff0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f39b8000c00 tx=0x7f39b8003810 comp rx=0 tx=0).stop 2024-05-09T05:45:26.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2171377206 shutdown_connections 2024-05-09T05:45:26.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097e50900 0x55e097e5bff0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 0x55e097e4fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/2171377206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097d43e00 0x55e097e5bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2171377206 >> 172.21.15.115:0/2171377206 conn(0x55e097ccd070 msgr2=0x55e097e5b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2171377206 shutdown_connections 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 -- 172.21.15.115:0/2171377206 wait complete. 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.155+0000 7f39d4a42b80 1 -- start start 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39d4a42b80 1 Processor -- start 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 0x55e097e55710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e55c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39d4a42b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097e50900 0x55e097ecad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01b3b20 con 0x55e097d43e00 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01b3450 con 0x55e097e50900 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39d4a42b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01b0650 con 0x55e097cc95a0 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39d4a42b80 10 client.0 _ll_register_callbacks cb 0x55e097bda7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39beffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e55c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39beffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e55c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53562/0 (socket says 172.21.15.115:53562) 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39beffd700 1 -- 172.21.15.115:0/920982975 learned_addr learned my addr 172.21.15.115:0/920982975 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:26.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 0x55e097e55710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39bffff700 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097e50900 0x55e097ecad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39beffd700 1 -- 172.21.15.115:0/920982975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 msgr2=0x55e097e55710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39beffd700 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 0x55e097e55710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39beffd700 1 -- 172.21.15.115:0/920982975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097e50900 msgr2=0x55e097ecad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39beffd700 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097e50900 0x55e097ecad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.156+0000 7f39beffd700 1 -- 172.21.15.115:0/920982975 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e097e56530 con 0x55e097d43e00 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39beffd700 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e55c50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f39b40a9150 tx=0x7f39b40a3d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaa17f3f3708d6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f399effd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f399effd700 1 -- 172.21.15.115:0/920982975 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4020030 con 0x55e097d43e00 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f399effd700 1 -- 172.21.15.115:0/920982975 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f39b4009b40 con 0x55e097d43e00 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39d4a42b80 1 -- 172.21.15.115:0/920982975 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39b8005030 con 0x55e097d43e00 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f399effd700 1 -- 172.21.15.115:0/920982975 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f39b408b050 con 0x55e097d43e00 2024-05-09T05:45:26.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39d4a42b80 10 client.17478 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39d4a42b80 1 -- 172.21.15.115:0/920982975 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e097ecc330 con 0x55e097d43e00 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f399effd700 1 -- 172.21.15.115:0/920982975 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39b8005030 con 0x55e097d43e00 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f399effd700 1 -- 172.21.15.115:0/920982975 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f39b40ca020 con 0x55e097d43e00 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39d4a42b80 10 client.17478 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39d4a42b80 10 client.17478 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39d4a42b80 20 client.17478 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e097ecc8b0 0x55e097ececd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39d4a42b80 4 client.17478 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b7601b26", "release": true, "await": true}] 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.157+0000 7f39d4a42b80 1 -- 172.21.15.115:0/920982975 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b7601b26", "release": true, "await": true}) -- 0x55e097d70c20 con 0x55e097ecc8b0 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.158+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e097ecc8b0 0x55e097ececd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.158+0000 7f399effd700 10 client.17478 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:26.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.158+0000 7f39bf7fe700 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e097ecc8b0 0x55e097ececd0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca0e58e3bfcc34f server_cookie=fa056a8d8b1b07ff in_seq=0 out_seq=0 2024-05-09T05:45:26.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f399effd700 1 -- 172.21.15.115:0/920982975 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e097d70c20 con 0x55e097ecc8b0 2024-05-09T05:45:26.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f399effd700 10 client.17478 handle_command_reply: tid=0 2024-05-09T05:45:26.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 client.17478 shutdown 2024-05-09T05:45:26.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 -- 172.21.15.115:0/920982975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 msgr2=0x55e097e55c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e55c50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f39b40a9150 tx=0x7f39b40a3d80 comp rx=0 tx=0).stop 2024-05-09T05:45:26.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 -- 172.21.15.115:0/920982975 shutdown_connections 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e097e50900 0x55e097ecad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e097ecc8b0 0x55e097ececd0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e097cc95a0 0x55e097e55710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 --2- 172.21.15.115:0/920982975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e097d43e00 0x55e097e55c50 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 -- 172.21.15.115:0/920982975 >> 172.21.15.115:0/920982975 conn(0x55e097ccd070 msgr2=0x55e097e4be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f399effd700 0 client.17478 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.168+0000 7f39d4a42b80 1 -- 172.21.15.115:0/920982975 shutdown_connections 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39d4a42b80 1 -- 172.21.15.115:0/920982975 wait complete. 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39d4a42b80 20 client.17478 trim_cache size 0 max 16384 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:45:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 40, 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 24, 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "b7601b26": { 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 24, 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:45:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:45:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:45:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:45:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:45:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:45:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:45:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:45:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:45:26.165 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 -- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f39a80b29f0 msgr2=0x7f39a80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f39a80b29f0 0x7f39a80b4f00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009fd0 tx=0x7f39b000b040 comp rx=0 tx=0).stop 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 -- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c0108f20 msgr2=0x7f39c0077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c0108f20 0x7f39c0077110 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f39b400d980 tx=0x7f39b400dc90 comp rx=0 tx=0).stop 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 -- 172.21.15.115:0/1026663856 shutdown_connections 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f39a80b29f0 0x7f39a80b4f00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39c01045f0 0x7f39c0076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0109870 0x7f39c0077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 --2- 172.21.15.115:0/1026663856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c0108f20 0x7f39c0077110 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.169+0000 7f39c602c700 1 -- 172.21.15.115:0/1026663856 >> 172.21.15.115:0/1026663856 conn(0x7f39c00ffec0 msgr2=0x7f39c0102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.170+0000 7f39c602c700 1 -- 172.21.15.115:0/1026663856 shutdown_connections 2024-05-09T05:45:26.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.170+0000 7f39c602c700 1 -- 172.21.15.115:0/1026663856 wait complete. 2024-05-09T05:45:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.968+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/1371017583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f8810e8e0 msgr2=0x7f9f88110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.968+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/1371017583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f8810e8e0 0x7f9f88110dc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000b0a0 tx=0x7f9f8000b3b0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.968+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/1371017583 shutdown_connections 2024-05-09T05:45:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.968+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/1371017583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f8810e8e0 0x7f9f88110dc0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.968+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/1371017583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f88105500 0x7f9f88105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.968+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/1371017583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88105e50 0x7f9f8810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.968+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/1371017583 >> 172.21.15.115:0/1371017583 conn(0x7f9f880fedc0 msgr2=0x7f9f881011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.968+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/1371017583 shutdown_connections 2024-05-09T05:45:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.968+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/1371017583 wait complete. 2024-05-09T05:45:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8ec7b700 1 Processor -- start 2024-05-09T05:45:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8ec7b700 1 -- start start 2024-05-09T05:45:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8ec7b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88105500 0x7f9f881a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8ec7b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f88105e50 0x7f9f881a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8ec7b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f8810e8e0 0x7f9f8819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8ec7b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88114310 con 0x7f9f88105e50 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8ec7b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88114190 con 0x7f9f88105500 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8ec7b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88114490 con 0x7f9f8810e8e0 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f87fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f88105e50 0x7f9f881a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f87fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f88105e50 0x7f9f881a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53582/0 (socket says 172.21.15.115:53582) 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f87fff700 1 -- 172.21.15.115:0/2108222253 learned_addr learned my addr 172.21.15.115:0/2108222253 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88105500 0x7f9f881a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.969+0000 7f9f8d218700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f8810e8e0 0x7f9f8819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f87fff700 1 -- 172.21.15.115:0/2108222253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f8810e8e0 msgr2=0x7f9f8819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f87fff700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f8810e8e0 0x7f9f8819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f87fff700 1 -- 172.21.15.115:0/2108222253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88105500 msgr2=0x7f9f881a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f87fff700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88105500 0x7f9f881a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f87fff700 1 -- 172.21.15.115:0/2108222253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8819b160 con 0x7f9f88105e50 2024-05-09T05:45:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f87fff700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f88105e50 0x7f9f881a0dd0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400d950 tx=0x7f9f7400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f85ffb700 1 -- 172.21.15.115:0/2108222253 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74016070 con 0x7f9f88105e50 2024-05-09T05:45:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/2108222253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8819b3f0 con 0x7f9f88105e50 2024-05-09T05:45:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f85ffb700 1 -- 172.21.15.115:0/2108222253 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f7400fd20 con 0x7f9f88105e50 2024-05-09T05:45:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f85ffb700 1 -- 172.21.15.115:0/2108222253 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7400e3b0 con 0x7f9f88105e50 2024-05-09T05:45:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.970+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/2108222253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f881a7720 con 0x7f9f88105e50 2024-05-09T05:45:26.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.973+0000 7f9f85ffb700 1 -- 172.21.15.115:0/2108222253 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f9f74019070 con 0x7f9f88105e50 2024-05-09T05:45:26.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.974+0000 7f9f85ffb700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9f780b29f0 0x7f9f780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.974+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9f780b29f0 0x7f9f780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.974+0000 7f9f85ffb700 1 -- 172.21.15.115:0/2108222253 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f9f740e23f0 con 0x7f9f88105e50 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9f780b29f0 0x7f9f780b4f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9f8819bbf0 tx=0x7f9f7c00b3c0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f9d691b80 1 Processor -- start 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f9d691b80 1 -- start start 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40800240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40800780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca407fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f9d691b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ca4071d0a0 con 0x55ca4067d4a0 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f9d691b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ca406c29a0 con 0x55ca406f7d00 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f9d691b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ca40719060 con 0x55ca407fa400 2024-05-09T05:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f87fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40800780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f87fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40800780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53596/0 (socket says 172.21.15.115:53596) 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f87fff700 1 -- 172.21.15.115:0/853808147 learned_addr learned my addr 172.21.15.115:0/853808147 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.975+0000 7f9f8d218700 1 --2- 172.21.15.115:0/853808147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca407fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/853808147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40800240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f87fff700 1 -- 172.21.15.115:0/853808147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 msgr2=0x55ca407fa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f87fff700 1 --2- 172.21.15.115:0/853808147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca407fa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f87fff700 1 -- 172.21.15.115:0/853808147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 msgr2=0x55ca40800240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f87fff700 1 --2- 172.21.15.115:0/853808147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40800240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f87fff700 1 -- 172.21.15.115:0/853808147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca406b3500 con 0x55ca4067d4a0 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f87fff700 1 --2- 172.21.15.115:0/853808147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40800780 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9f740a5410 tx=0x7f9f740a8ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd461b9cd67d49c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f72ffd700 1 -- 172.21.15.115:0/853808147 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74020070 con 0x55ca4067d4a0 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f72ffd700 1 -- 172.21.15.115:0/853808147 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f740ad740 con 0x55ca4067d4a0 2024-05-09T05:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.976+0000 7f9f72ffd700 1 -- 172.21.15.115:0/853808147 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740a9850 con 0x55ca4067d4a0 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.977+0000 7f9f9d691b80 1 -- 172.21.15.115:0/853808147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 msgr2=0x55ca40800780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.977+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/853808147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40800780 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9f740a5410 tx=0x7f9f740a8ea0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.977+0000 7f9f9d691b80 1 -- 172.21.15.115:0/853808147 shutdown_connections 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.977+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/853808147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40800240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.977+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/853808147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca407fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.977+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/853808147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40800780 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.977+0000 7f9f9d691b80 1 -- 172.21.15.115:0/853808147 >> 172.21.15.115:0/853808147 conn(0x55ca40680f70 msgr2=0x55ca407f5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.977+0000 7f9f9d691b80 1 -- 172.21.15.115:0/853808147 shutdown_connections 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.977+0000 7f9f9d691b80 1 -- 172.21.15.115:0/853808147 wait complete. 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.978+0000 7f9f9d691b80 1 -- start start 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.978+0000 7f9f9d691b80 1 Processor -- start 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.978+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40804110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.978+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40804650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.978+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca40804b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.978+0000 7f9f8ca17700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40804110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f8ca17700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40804110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53608/0 (socket says 172.21.15.115:53608) 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f8ca17700 1 -- 172.21.15.115:0/3591880003 learned_addr learned my addr 172.21.15.115:0/3591880003 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ca407fb160 con 0x55ca4067d4a0 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ca406b7b20 con 0x55ca406f7d00 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ca40807020 con 0x55ca407fa400 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f9d691b80 10 client.0 _ll_register_callbacks cb 0x55ca4058e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f8d218700 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca40804b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f87fff700 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40804650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f8ca17700 1 -- 172.21.15.115:0/3591880003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 msgr2=0x55ca40804b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca40804b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f8ca17700 1 -- 172.21.15.115:0/3591880003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 msgr2=0x55ca40804650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:26.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40804650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f8ca17700 1 -- 172.21.15.115:0/3591880003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca40598490 con 0x55ca4067d4a0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40804110 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c003df0 tx=0x7f9f7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772e4859a26878dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f67fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f67fff700 1 -- 172.21.15.115:0/3591880003 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c028670 con 0x55ca4067d4a0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f67fff700 1 -- 172.21.15.115:0/3591880003 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f9f7c028810 con 0x55ca4067d4a0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.979+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f740ad740 con 0x55ca4067d4a0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f67fff700 1 -- 172.21.15.115:0/3591880003 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9f7c031050 con 0x55ca4067d4a0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f9d691b80 10 client.17490 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ca4081a350 con 0x55ca4067d4a0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f67fff700 1 -- 172.21.15.115:0/3591880003 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f740ad740 con 0x55ca4067d4a0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f67fff700 1 -- 172.21.15.115:0/3591880003 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9f7c01e430 con 0x55ca4067d4a0 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f9d691b80 10 client.17490 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f9d691b80 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:26.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f9d691b80 20 client.17490 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ca4081c9e0 0x55ca4081ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f9d691b80 4 client.17490 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.980+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca4072aa80 con 0x55ca4081c9e0 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.981+0000 7f9f87fff700 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ca4081c9e0 0x55ca4081ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.981+0000 7f9f67fff700 10 client.17490 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.981+0000 7f9f87fff700 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ca4081c9e0 0x55ca4081ee60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=defb15faf0d740c4 server_cookie=1e31efe74833300b in_seq=0 out_seq=0 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f67fff700 1 -- 172.21.15.115:0/3591880003 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca4072aa80 con 0x55ca4081c9e0 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f67fff700 10 client.17490 handle_command_reply: tid=0 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 client.17490 shutdown 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 msgr2=0x55ca40804110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40804110 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c003df0 tx=0x7f9f7c0047c0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 shutdown_connections 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ca4081c9e0 0x55ca4081ee60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f67fff700 0 client.17490 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:26.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca40804b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4067d4a0 0x55ca40804110 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/3591880003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40804650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:26.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 >> 172.21.15.115:0/3591880003 conn(0x55ca40680f70 msgr2=0x55ca407eeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:26.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 shutdown_connections 2024-05-09T05:45:26.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 1 -- 172.21.15.115:0/3591880003 wait complete. 2024-05-09T05:45:26.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:26.982+0000 7f9f9d691b80 20 client.17490 trim_cache size 0 max 16384 2024-05-09T05:45:26.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f9d691b80 1 Processor -- start 2024-05-09T05:45:26.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f9d691b80 1 -- start start 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40804a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca407fa400 0x55ca407ef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca4081c9e0 0x55ca408186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f9d691b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881ae170 con 0x55ca407fa400 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f9d691b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881afcb0 con 0x55ca406f7d00 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f9d691b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881ae840 con 0x55ca4081c9e0 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f87fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca407fa400 0x55ca407ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f87fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca407fa400 0x55ca407ef700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53612/0 (socket says 172.21.15.115:53612) 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f8d218700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca4081c9e0 0x55ca408186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.004+0000 7f9f87fff700 1 -- 172.21.15.115:0/4000819515 learned_addr learned my addr 172.21.15.115:0/4000819515 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/4000819515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40804a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f8d218700 1 -- 172.21.15.115:0/4000819515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 msgr2=0x55ca40804a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f8d218700 1 --2- 172.21.15.115:0/4000819515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40804a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f8d218700 1 -- 172.21.15.115:0/4000819515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca407fa400 msgr2=0x55ca407ef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f8d218700 1 --2- 172.21.15.115:0/4000819515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca407fa400 0x55ca407ef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f8d218700 1 -- 172.21.15.115:0/4000819515 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4081f3f0 con 0x55ca4081c9e0 2024-05-09T05:45:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f87fff700 1 --2- 172.21.15.115:0/4000819515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca407fa400 0x55ca407ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f8d218700 1 --2- 172.21.15.115:0/4000819515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca4081c9e0 0x55ca408186b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009fd0 tx=0x7f9f8000b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7f0e70151dc0956 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f727fc700 1 -- 172.21.15.115:0/4000819515 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80009c30 con 0x55ca4081c9e0 2024-05-09T05:45:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f727fc700 1 -- 172.21.15.115:0/4000819515 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f800075c0 con 0x55ca4081c9e0 2024-05-09T05:45:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.005+0000 7f9f727fc700 1 -- 172.21.15.115:0/4000819515 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f800266b0 con 0x55ca4081c9e0 2024-05-09T05:45:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.006+0000 7f9f9d691b80 1 -- 172.21.15.115:0/4000819515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca4081c9e0 msgr2=0x55ca408186b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.006+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/4000819515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca4081c9e0 0x55ca408186b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009fd0 tx=0x7f9f8000b7d0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.006+0000 7f9f9d691b80 1 -- 172.21.15.115:0/4000819515 shutdown_connections 2024-05-09T05:45:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.006+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/4000819515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40804a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.006+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/4000819515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca4081c9e0 0x55ca408186b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.006+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/4000819515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca407fa400 0x55ca407ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.006+0000 7f9f9d691b80 1 -- 172.21.15.115:0/4000819515 >> 172.21.15.115:0/4000819515 conn(0x55ca40680f70 msgr2=0x55ca407eeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.006+0000 7f9f9d691b80 1 -- 172.21.15.115:0/4000819515 shutdown_connections 2024-05-09T05:45:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.006+0000 7f9f9d691b80 1 -- 172.21.15.115:0/4000819515 wait complete. 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.007+0000 7f9f9d691b80 1 -- start start 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.007+0000 7f9f9d691b80 1 Processor -- start 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.007+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40814a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.007+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca40814f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.007+0000 7f9f9d691b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4081c9e0 0x55ca40811a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.007+0000 7f9f9d691b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881af5e0 con 0x55ca4081c9e0 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.007+0000 7f9f9d691b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881aef10 con 0x55ca406f7d00 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.007+0000 7f9f9d691b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881ac110 con 0x55ca407fa400 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.007+0000 7f9f9d691b80 10 client.0 _ll_register_callbacks cb 0x55ca4058e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8d218700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4081c9e0 0x55ca40811a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8d218700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4081c9e0 0x55ca40811a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53626/0 (socket says 172.21.15.115:53626) 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8d218700 1 -- 172.21.15.115:0/2881218130 learned_addr learned my addr 172.21.15.115:0/2881218130 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f87fff700 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca40814f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40814a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8d218700 1 -- 172.21.15.115:0/2881218130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 msgr2=0x55ca40814f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8d218700 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca40814f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8d218700 1 -- 172.21.15.115:0/2881218130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 msgr2=0x55ca40814a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8d218700 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40814a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8d218700 1 -- 172.21.15.115:0/2881218130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca407f19d0 con 0x55ca4081c9e0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f8d218700 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4081c9e0 0x55ca40811a00 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000d010 tx=0x7f9f80003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe02451e16439ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f67fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f67fff700 1 -- 172.21.15.115:0/2881218130 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80007c00 con 0x55ca4081c9e0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f9d691b80 1 -- 172.21.15.115:0/2881218130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f800075c0 con 0x55ca4081c9e0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.008+0000 7f9f67fff700 1 -- 172.21.15.115:0/2881218130 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f9f80026450 con 0x55ca4081c9e0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f67fff700 1 -- 172.21.15.115:0/2881218130 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9f8003a050 con 0x55ca4081c9e0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f9d691b80 10 client.17496 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f9d691b80 1 -- 172.21.15.115:0/2881218130 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ca40887810 con 0x55ca4081c9e0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f67fff700 1 -- 172.21.15.115:0/2881218130 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f800075c0 con 0x55ca4081c9e0 2024-05-09T05:45:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f67fff700 1 -- 172.21.15.115:0/2881218130 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9f8002f020 con 0x55ca4081c9e0 2024-05-09T05:45:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f9d691b80 10 client.17496 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f9d691b80 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f9d691b80 20 client.17496 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ca40887ed0 0x55ca4088a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f9d691b80 4 client.17496 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:45:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.009+0000 7f9f9d691b80 1 -- 172.21.15.115:0/2881218130 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca40724b20 con 0x55ca40887ed0 2024-05-09T05:45:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.010+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ca40887ed0 0x55ca4088a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.010+0000 7f9f67fff700 10 client.17496 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.010+0000 7f9f8ca17700 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ca40887ed0 0x55ca4088a2f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800e58585b5ce055 server_cookie=7b2d03bda3ef1584 in_seq=0 out_seq=0 2024-05-09T05:45:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.010+0000 7f9f67fff700 1 -- 172.21.15.115:0/2881218130 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ca40724b20 con 0x55ca40887ed0 2024-05-09T05:45:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.010+0000 7f9f67fff700 10 client.17496 handle_command_reply: tid=0 2024-05-09T05:45:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.010+0000 7f9f9d691b80 1 client.17496 shutdown 2024-05-09T05:45:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.010+0000 7f9f9d691b80 1 -- 172.21.15.115:0/2881218130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4081c9e0 msgr2=0x55ca40811a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.010+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4081c9e0 0x55ca40811a00 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000d010 tx=0x7f9f80003e50 comp rx=0 tx=0).stop 2024-05-09T05:45:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f9d691b80 1 -- 172.21.15.115:0/2881218130 shutdown_connections 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca406f7d00 0x55ca40814a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55ca40887ed0 0x55ca4088a2f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ca4081c9e0 0x55ca40811a00 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f9d691b80 1 --2- 172.21.15.115:0/2881218130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ca407fa400 0x55ca40814f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f9d691b80 1 -- 172.21.15.115:0/2881218130 >> 172.21.15.115:0/2881218130 conn(0x55ca40680f70 msgr2=0x55ca40805700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f67fff700 0 client.17496 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f9d691b80 1 -- 172.21.15.115:0/2881218130 shutdown_connections 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f9d691b80 1 -- 172.21.15.115:0/2881218130 wait complete. 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.011+0000 7f9f9d691b80 20 client.17496 trim_cache size 0 max 16384 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:45:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:45:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:45:27.009 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:45:27.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9f780b29f0 msgr2=0x7f9f780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:27.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9f780b29f0 0x7f9f780b4f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9f8819bbf0 tx=0x7f9f7c00b3c0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f88105e50 msgr2=0x7f9f881a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:27.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f88105e50 0x7f9f881a0dd0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400d950 tx=0x7f9f7400dc60 comp rx=0 tx=0).stop 2024-05-09T05:45:27.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/2108222253 shutdown_connections 2024-05-09T05:45:27.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9f780b29f0 0x7f9f780b4f00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88105500 0x7f9f881a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f8810e8e0 0x7f9f8819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 --2- 172.21.15.115:0/2108222253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f88105e50 0x7f9f881a0dd0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:27.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/2108222253 >> 172.21.15.115:0/2108222253 conn(0x7f9f880fedc0 msgr2=0x7f9f880ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:27.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/2108222253 shutdown_connections 2024-05-09T05:45:27.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:27.012+0000 7f9f8ec7b700 1 -- 172.21.15.115:0/2108222253 wait complete. 2024-05-09T05:45:27.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:27 smithi195 ceph-mon[33844]: pgmap v223: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-09T05:45:27.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:27 smithi115 ceph-mon[30349]: pgmap v223: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-09T05:45:27.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:27 smithi161 ceph-mon[33805]: pgmap v223: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-05-09T05:45:27.798 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:45:27.798 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:45:28.381 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b7601b26', total seconds elapsed: 39.0 2024-05-09T05:45:28.447 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:29.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:29 smithi195 ceph-mon[33844]: pgmap v224: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 16 op/s 2024-05-09T05:45:29.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:29 smithi115 ceph-mon[30349]: pgmap v224: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 16 op/s 2024-05-09T05:45:29.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:29 smithi161 ceph-mon[33805]: pgmap v224: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 16 op/s 2024-05-09T05:45:30.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.315+0000 7f11aa44e700 1 -- 172.21.15.115:0/3688024415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 msgr2=0x7f11a4102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.315+0000 7f11aa44e700 1 --2- 172.21.15.115:0/3688024415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 0x7f11a4102bf0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f118c009990 tx=0x7f118c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.315+0000 7f11aa44e700 1 -- 172.21.15.115:0/3688024415 shutdown_connections 2024-05-09T05:45:30.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.315+0000 7f11aa44e700 1 --2- 172.21.15.115:0/3688024415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a410a3e0 0x7f11a410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.315+0000 7f11aa44e700 1 --2- 172.21.15.115:0/3688024415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11a4103130 0x7f11a4111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.315+0000 7f11aa44e700 1 --2- 172.21.15.115:0/3688024415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 0x7f11a4102bf0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.315+0000 7f11aa44e700 1 -- 172.21.15.115:0/3688024415 >> 172.21.15.115:0/3688024415 conn(0x7f11a40fe6f0 msgr2=0x7f11a4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:30.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.315+0000 7f11aa44e700 1 -- 172.21.15.115:0/3688024415 shutdown_connections 2024-05-09T05:45:30.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.315+0000 7f11aa44e700 1 -- 172.21.15.115:0/3688024415 wait complete. 2024-05-09T05:45:30.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.316+0000 7f11aa44e700 1 Processor -- start 2024-05-09T05:45:30.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.316+0000 7f11aa44e700 1 -- start start 2024-05-09T05:45:30.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11aa44e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 0x7f11a4110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11aa44e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a4103130 0x7f11a410e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11aa44e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11a410a3e0 0x7f11a410e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11aa44e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11a4104e50 con 0x7f11a41027e0 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11aa44e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11a4104cd0 con 0x7f11a4103130 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11aa44e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f11a4104fd0 con 0x7f11a410a3e0 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 0x7f11a4110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 0x7f11a4110fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53636/0 (socket says 172.21.15.115:53636) 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a3fff700 1 -- 172.21.15.115:0/626979480 learned_addr learned my addr 172.21.15.115:0/626979480 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f119bfff700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a4103130 0x7f11a410e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a89eb700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11a410a3e0 0x7f11a410e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a3fff700 1 -- 172.21.15.115:0/626979480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11a410a3e0 msgr2=0x7f11a410e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a3fff700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11a410a3e0 0x7f11a410e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a3fff700 1 -- 172.21.15.115:0/626979480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a4103130 msgr2=0x7f11a410e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a3fff700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a4103130 0x7f11a410e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a3fff700 1 -- 172.21.15.115:0/626979480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a410ee70 con 0x7f11a41027e0 2024-05-09T05:45:30.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a89eb700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11a410a3e0 0x7f11a410e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:45:30.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a3fff700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 0x7f11a4110fb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f119400d950 tx=0x7f119400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:30.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.317+0000 7f11a1ffb700 1 -- 172.21.15.115:0/626979480 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194016070 con 0x7f11a41027e0 2024-05-09T05:45:30.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.318+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a41ad7f0 con 0x7f11a41027e0 2024-05-09T05:45:30.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.318+0000 7f11a1ffb700 1 -- 172.21.15.115:0/626979480 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1194010460 con 0x7f11a41027e0 2024-05-09T05:45:30.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.318+0000 7f11a1ffb700 1 -- 172.21.15.115:0/626979480 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119400f620 con 0x7f11a41027e0 2024-05-09T05:45:30.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.318+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a41add80 con 0x7f11a41027e0 2024-05-09T05:45:30.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.319+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a4105b00 con 0x7f11a41027e0 2024-05-09T05:45:30.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.320+0000 7f11a1ffb700 1 -- 172.21.15.115:0/626979480 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1194019070 con 0x7f11a41027e0 2024-05-09T05:45:30.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.321+0000 7f11a1ffb700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11900b29f0 0x7f11900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.321+0000 7f11a1ffb700 1 -- 172.21.15.115:0/626979480 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f11940e2410 con 0x7f11a41027e0 2024-05-09T05:45:30.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.321+0000 7f119bfff700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11900b29f0 0x7f11900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.322+0000 7f119bfff700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11900b29f0 0x7f11900b4f00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f118c00b5c0 tx=0x7f118c005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:30.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.326+0000 7f11a1ffb700 1 -- 172.21.15.115:0/626979480 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f11940a9b50 con 0x7f11a41027e0 2024-05-09T05:45:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.549+0000 7f840b5a8640 1 -- 172.21.15.115:0/1381636657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410e210 msgr2=0x7f8404110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.549+0000 7f840b5a8640 1 --2- 172.21.15.115:0/1381636657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410e210 0x7f8404110700 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ca0 tx=0x7f83fc02a8c0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.549+0000 7f840b5a8640 1 -- 172.21.15.115:0/1381636657 shutdown_connections 2024-05-09T05:45:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.549+0000 7f840b5a8640 1 --2- 172.21.15.115:0/1381636657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410e210 0x7f8404110700 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.549+0000 7f840b5a8640 1 --2- 172.21.15.115:0/1381636657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f840410b820 0x7f840410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.549+0000 7f840b5a8640 1 --2- 172.21.15.115:0/1381636657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8404104dc0 0x7f84041051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.549+0000 7f840b5a8640 1 -- 172.21.15.115:0/1381636657 >> 172.21.15.115:0/1381636657 conn(0x7f84040781b0 msgr2=0x7f8404078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.549+0000 7f840b5a8640 1 -- 172.21.15.115:0/1381636657 shutdown_connections 2024-05-09T05:45:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.549+0000 7f840b5a8640 1 -- 172.21.15.115:0/1381636657 wait complete. 2024-05-09T05:45:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f840b5a8640 1 Processor -- start 2024-05-09T05:45:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f840b5a8640 1 -- start start 2024-05-09T05:45:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f840b5a8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8404104dc0 0x7f8404102680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f840b5a8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410b820 0x7f8404102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f840b5a8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f840410e210 0x7f8404103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f840b5a8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8404113d70 con 0x7f840410b820 2024-05-09T05:45:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f840b5a8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8404113bf0 con 0x7f8404104dc0 2024-05-09T05:45:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f840b5a8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8404113ef0 con 0x7f840410e210 2024-05-09T05:45:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f8408b43640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410b820 0x7f8404102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f8408b43640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410b820 0x7f8404102bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53660/0 (socket says 172.21.15.115:53660) 2024-05-09T05:45:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.550+0000 7f8408b43640 1 -- 172.21.15.115:0/4288170949 learned_addr learned my addr 172.21.15.115:0/4288170949 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f8409b45640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f840410e210 0x7f8404103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f8409344640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8404104dc0 0x7f8404102680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f8408b43640 1 -- 172.21.15.115:0/4288170949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f840410e210 msgr2=0x7f8404103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f8408b43640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f840410e210 0x7f8404103100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f8408b43640 1 -- 172.21.15.115:0/4288170949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8404104dc0 msgr2=0x7f8404102680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f8408b43640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8404104dc0 0x7f8404102680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f8408b43640 1 -- 172.21.15.115:0/4288170949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84041a8a90 con 0x7f840410b820 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f8408b43640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410b820 0x7f8404102bc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f83f400e9f0 tx=0x7f83f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f83fa7fc640 1 -- 172.21.15.115:0/4288170949 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f400cd80 con 0x7f840410b820 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f840b5a8640 1 -- 172.21.15.115:0/4288170949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84041a8d80 con 0x7f840410b820 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f83fa7fc640 1 -- 172.21.15.115:0/4288170949 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83f40108c0 con 0x7f840410b820 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f83fa7fc640 1 -- 172.21.15.115:0/4288170949 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f40187c0 con 0x7f840410b820 2024-05-09T05:45:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.551+0000 7f840b5a8640 1 -- 172.21.15.115:0/4288170949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84041a9250 con 0x7f840410b820 2024-05-09T05:45:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f83fa7fc640 1 -- 172.21.15.115:0/4288170949 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f83f4010430 con 0x7f840410b820 2024-05-09T05:45:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8412458740 1 Processor -- start 2024-05-09T05:45:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8412458740 1 -- start start 2024-05-09T05:45:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8412458740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8412458740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8412458740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 0x5623a91b7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8412458740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5623a911b7c0 con 0x5623a8dea2f0 2024-05-09T05:45:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8412458740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5623a90b70e0 con 0x5623a8df24d0 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8412458740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623a9117780 con 0x5623a8e131b0 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8409b45640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8409b45640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46058/0 (socket says 172.21.15.115:46058) 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8409b45640 1 -- 172.21.15.115:0/3968385019 learned_addr learned my addr 172.21.15.115:0/3968385019 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8408b43640 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 0x5623a91b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.555+0000 7f8409344640 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f8409b45640 1 -- 172.21.15.115:0/3968385019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 msgr2=0x5623a91b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f8409b45640 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 0x5623a91b7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f8409b45640 1 -- 172.21.15.115:0/3968385019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 msgr2=0x5623a91b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f8409b45640 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f83fa7fc640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f83f00b2950 0x7f83f00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f8409b45640 1 -- 172.21.15.115:0/3968385019 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a90e07c0 con 0x5623a8e131b0 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f8409344640 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f83fa7fc640 1 -- 172.21.15.115:0/4288170949 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f83f40e2cb0 con 0x7f840410b820 2024-05-09T05:45:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f8408b43640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f83f00b2950 0x7f83f00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f8409b45640 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91b6680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ca0 tx=0x7f83fc009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56220df352fd01d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f83ef7fe640 1 -- 172.21.15.115:0/3968385019 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc031980 con 0x5623a8e131b0 2024-05-09T05:45:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f8408b43640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f83f00b2950 0x7f83f00b4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83f40e0170 tx=0x7f83f40e6800 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f83ef7fe640 1 -- 172.21.15.115:0/3968385019 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83fc036030 con 0x5623a8e131b0 2024-05-09T05:45:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.556+0000 7f83ef7fe640 1 -- 172.21.15.115:0/3968385019 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc004460 con 0x5623a8e131b0 2024-05-09T05:45:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.557+0000 7f8412458740 1 -- 172.21.15.115:0/3968385019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 msgr2=0x5623a91b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.557+0000 7f8412458740 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91b6680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009ca0 tx=0x7f83fc009a20 comp rx=0 tx=0).stop 2024-05-09T05:45:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.557+0000 7f8412458740 1 -- 172.21.15.115:0/3968385019 shutdown_connections 2024-05-09T05:45:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.557+0000 7f8412458740 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 0x5623a91b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.557+0000 7f8412458740 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.557+0000 7f8412458740 1 --2- 172.21.15.115:0/3968385019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91b6680 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.557+0000 7f8412458740 1 -- 172.21.15.115:0/3968385019 >> 172.21.15.115:0/3968385019 conn(0x5623a8e07300 msgr2=0x5623a91b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.557+0000 7f8412458740 1 -- 172.21.15.115:0/3968385019 shutdown_connections 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.557+0000 7f8412458740 1 -- 172.21.15.115:0/3968385019 wait complete. 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.558+0000 7f8412458740 1 -- start start 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.558+0000 7f8412458740 1 Processor -- start 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.558+0000 7f8412458740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.558+0000 7f8412458740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 0x5623a91c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.558+0000 7f8412458740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.558+0000 7f8412458740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5623a91c35d0 con 0x5623a8dea2f0 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.558+0000 7f8412458740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5623a90a29c0 con 0x5623a8df24d0 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.558+0000 7f8412458740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623a91c3750 con 0x5623a8e131b0 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.558+0000 7f8412458740 10 client.0 _ll_register_callbacks cb 0x5623a90cf020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53670/0 (socket says 172.21.15.115:53670) 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 -- 172.21.15.115:0/3931173854 learned_addr learned my addr 172.21.15.115:0/3931173854 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409344640 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 0x5623a91c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 -- 172.21.15.115:0/3931173854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 msgr2=0x5623a91c0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91c0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 -- 172.21.15.115:0/3931173854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 msgr2=0x5623a91c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 0x5623a91c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 -- 172.21.15.115:0/3931173854 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a90c91f0 con 0x5623a8dea2f0 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409344640 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 0x5623a91c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8409b45640 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c2690 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009620 tx=0x7f83fc0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1b06a68a3979dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f83ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f83ecff9640 1 -- 172.21.15.115:0/3931173854 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc058060 con 0x5623a8dea2f0 2024-05-09T05:45:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f83ecff9640 1 -- 172.21.15.115:0/3931173854 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f83fc0041e0 con 0x5623a8dea2f0 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.559+0000 7f8412458740 1 -- 172.21.15.115:0/3931173854 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83fc036030 con 0x5623a8dea2f0 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f83ecff9640 1 -- 172.21.15.115:0/3931173854 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f83fc05f050 con 0x5623a8dea2f0 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f8412458740 10 client.17514 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f8412458740 1 -- 172.21.15.115:0/3931173854 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5623a8e7e790 con 0x5623a8dea2f0 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f83ecff9640 1 -- 172.21.15.115:0/3931173854 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83fc036030 con 0x5623a8dea2f0 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f83ecff9640 1 -- 172.21.15.115:0/3931173854 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f83fc043070 con 0x5623a8dea2f0 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f8412458740 10 client.17514 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f8412458740 10 client.17514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f8412458740 20 client.17514 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f8412458740 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623a8dd1d80 0x5623a91d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f8412458740 4 client.17514 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f8412458740 1 -- 172.21.15.115:0/3931173854 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623a91293d0 con 0x5623a8dd1d80 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.560+0000 7f8409344640 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623a8dd1d80 0x5623a91d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.561+0000 7f8409344640 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623a8dd1d80 0x5623a91d2bc0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e88df2cddfd975 server_cookie=a35b87a90ee3669b in_seq=0 out_seq=0 2024-05-09T05:45:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.561+0000 7f83ecff9640 10 client.17514 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f83ecff9640 1 -- 172.21.15.115:0/3931173854 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5623a91293d0 con 0x5623a8dd1d80 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f83ecff9640 10 client.17514 handle_command_reply: tid=0 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 client.17514 shutdown 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 -- 172.21.15.115:0/3931173854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 msgr2=0x5623a91c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c2690 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009620 tx=0x7f83fc0094a0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 -- 172.21.15.115:0/3931173854 shutdown_connections 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623a8dd1d80 0x5623a91d2bc0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91c0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8df24d0 0x5623a91c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f83ecff9640 0 client.17514 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 --2- 172.21.15.115:0/3931173854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c2690 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 -- 172.21.15.115:0/3931173854 >> 172.21.15.115:0/3931173854 conn(0x5623a8e07300 msgr2=0x5623a91b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 -- 172.21.15.115:0/3931173854 shutdown_connections 2024-05-09T05:45:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 1 -- 172.21.15.115:0/3931173854 wait complete. 2024-05-09T05:45:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.562+0000 7f8412458740 20 client.17514 trim_cache size 0 max 16384 2024-05-09T05:45:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8412458740 1 Processor -- start 2024-05-09T05:45:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8412458740 1 -- start start 2024-05-09T05:45:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8412458740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8dea2f0 0x5623a91c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8412458740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8df24d0 0x5623a91b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8412458740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8412458740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84041b31e0 con 0x5623a8df24d0 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8412458740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84041b4d20 con 0x5623a8dea2f0 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8412458740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84041b38b0 con 0x5623a8e131b0 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409344640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8df24d0 0x5623a91b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409344640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8df24d0 0x5623a91b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53678/0 (socket says 172.21.15.115:53678) 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409344640 1 -- 172.21.15.115:0/96958615 learned_addr learned my addr 172.21.15.115:0/96958615 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409b45640 1 --2- 172.21.15.115:0/96958615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8dea2f0 0x5623a91c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409b45640 1 --2- 172.21.15.115:0/96958615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409344640 1 -- 172.21.15.115:0/96958615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 msgr2=0x5623a91c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409344640 1 --2- 172.21.15.115:0/96958615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409344640 1 -- 172.21.15.115:0/96958615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8dea2f0 msgr2=0x5623a91c0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409344640 1 --2- 172.21.15.115:0/96958615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8dea2f0 0x5623a91c0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.582+0000 7f8409344640 1 -- 172.21.15.115:0/96958615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a8ef55f0 con 0x5623a8df24d0 2024-05-09T05:45:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.583+0000 7f8409344640 1 --2- 172.21.15.115:0/96958615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8df24d0 0x5623a91b1e10 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f84000045d0 tx=0x7f840000bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2fe1a510516788 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.583+0000 7f83eeffd640 1 -- 172.21.15.115:0/96958615 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400021030 con 0x5623a8df24d0 2024-05-09T05:45:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.583+0000 7f83eeffd640 1 -- 172.21.15.115:0/96958615 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84000054b0 con 0x5623a8df24d0 2024-05-09T05:45:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.583+0000 7f83eeffd640 1 -- 172.21.15.115:0/96958615 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840001d8a0 con 0x5623a8df24d0 2024-05-09T05:45:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.584+0000 7f8412458740 1 -- 172.21.15.115:0/96958615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8df24d0 msgr2=0x5623a91b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.584+0000 7f8412458740 1 --2- 172.21.15.115:0/96958615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8df24d0 0x5623a91b1e10 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f84000045d0 tx=0x7f840000bd90 comp rx=0 tx=0).stop 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.584+0000 7f8412458740 1 -- 172.21.15.115:0/96958615 shutdown_connections 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.584+0000 7f8412458740 1 --2- 172.21.15.115:0/96958615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8e131b0 0x5623a91c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.584+0000 7f8412458740 1 --2- 172.21.15.115:0/96958615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8df24d0 0x5623a91b1e10 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.584+0000 7f8412458740 1 --2- 172.21.15.115:0/96958615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8dea2f0 0x5623a91c0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.584+0000 7f8412458740 1 -- 172.21.15.115:0/96958615 >> 172.21.15.115:0/96958615 conn(0x5623a8e07300 msgr2=0x5623a91b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.584+0000 7f8412458740 1 -- 172.21.15.115:0/96958615 shutdown_connections 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.584+0000 7f8412458740 1 -- 172.21.15.115:0/96958615 wait complete. 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.585+0000 7f8412458740 1 -- start start 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.585+0000 7f8412458740 1 Processor -- start 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.585+0000 7f8412458740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.585+0000 7f8412458740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8df24d0 0x5623a91c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8412458740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8e131b0 0x5623a91c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8412458740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f84041b4650 con 0x5623a8dea2f0 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8412458740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84041b3f80 con 0x5623a8e131b0 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8412458740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84041b11c0 con 0x5623a8df24d0 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8412458740 10 client.0 _ll_register_callbacks cb 0x7f83f40e66d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53680/0 (socket says 172.21.15.115:53680) 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 -- 172.21.15.115:0/4183694725 learned_addr learned my addr 172.21.15.115:0/4183694725 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409344640 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8df24d0 0x5623a91c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8e131b0 0x5623a91c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 -- 172.21.15.115:0/4183694725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8df24d0 msgr2=0x5623a91c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8df24d0 0x5623a91c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 -- 172.21.15.115:0/4183694725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8e131b0 msgr2=0x5623a91c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8e131b0 0x5623a91c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 -- 172.21.15.115:0/4183694725 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a90dfed0 con 0x5623a8dea2f0 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f8409b45640 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c6620 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f83fc02aeb0 tx=0x7f83fc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be89feb7b1617b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f83ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.586+0000 7f83ecff9640 1 -- 172.21.15.115:0/4183694725 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc030ed0 con 0x5623a8dea2f0 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f83ecff9640 1 -- 172.21.15.115:0/4183694725 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f83fc004d40 con 0x5623a8dea2f0 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f8412458740 1 -- 172.21.15.115:0/4183694725 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84000054b0 con 0x5623a8dea2f0 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f83ecff9640 1 -- 172.21.15.115:0/4183694725 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f83fc058030 con 0x5623a8dea2f0 2024-05-09T05:45:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f8412458740 10 client.17520 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f8412458740 1 -- 172.21.15.115:0/4183694725 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5623a923add0 con 0x5623a8dea2f0 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f83ecff9640 1 -- 172.21.15.115:0/4183694725 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84000054b0 con 0x5623a8dea2f0 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f83ecff9640 1 -- 172.21.15.115:0/4183694725 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f83fc043070 con 0x5623a8dea2f0 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f8412458740 10 client.17520 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f8412458740 10 client.17520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f8412458740 20 client.17520 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f8412458740 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623a8da71c0 0x5623a923d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f8412458740 4 client.17520 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.587+0000 7f8412458740 1 -- 172.21.15.115:0/4183694725 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623a9123240 con 0x5623a8da71c0 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.588+0000 7f8409344640 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623a8da71c0 0x5623a923d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.588+0000 7f83ecff9640 10 client.17520 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.588+0000 7f8409344640 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623a8da71c0 0x5623a923d470 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf73f5a22e39fa0c server_cookie=4fd84cd8c3da9a3a in_seq=0 out_seq=0 2024-05-09T05:45:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f83ecff9640 1 -- 172.21.15.115:0/4183694725 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5623a9123240 con 0x5623a8da71c0 2024-05-09T05:45:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f83ecff9640 10 client.17520 handle_command_reply: tid=0 2024-05-09T05:45:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 client.17520 shutdown 2024-05-09T05:45:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 -- 172.21.15.115:0/4183694725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 msgr2=0x5623a91c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c6620 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f83fc02aeb0 tx=0x7f83fc02a8c0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 -- 172.21.15.115:0/4183694725 shutdown_connections 2024-05-09T05:45:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5623a8e131b0 0x5623a91c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5623a8da71c0 0x5623a923d470 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623a8df24d0 0x5623a91c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 --2- 172.21.15.115:0/4183694725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5623a8dea2f0 0x5623a91c6620 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 -- 172.21.15.115:0/4183694725 >> 172.21.15.115:0/4183694725 conn(0x5623a8e07300 msgr2=0x5623a91b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f83ecff9640 0 client.17520 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 -- 172.21.15.115:0/4183694725 shutdown_connections 2024-05-09T05:45:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 1 -- 172.21.15.115:0/4183694725 wait complete. 2024-05-09T05:45:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.592+0000 7f8412458740 20 client.17520 trim_cache size 0 max 16384 2024-05-09T05:45:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:45:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:45:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:45:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 -- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f83f00b2950 msgr2=0x7f83f00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f83f00b2950 0x7f83f00b4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83f40e0170 tx=0x7f83f40e6800 comp rx=0 tx=0).stop 2024-05-09T05:45:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 -- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410b820 msgr2=0x7f8404102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410b820 0x7f8404102bc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f83f400e9f0 tx=0x7f83f400eeb0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 -- 172.21.15.115:0/4288170949 shutdown_connections 2024-05-09T05:45:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f83f00b2950 0x7f83f00b4e60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f840410e210 0x7f8404103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840410b820 0x7f8404102bc0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 --2- 172.21.15.115:0/4288170949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8404104dc0 0x7f8404102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 -- 172.21.15.115:0/4288170949 >> 172.21.15.115:0/4288170949 conn(0x7f84040781b0 msgr2=0x7f84041012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 -- 172.21.15.115:0/4288170949 shutdown_connections 2024-05-09T05:45:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:30.593+0000 7f840b5a8640 1 -- 172.21.15.115:0/4288170949 wait complete. 2024-05-09T05:45:30.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.664+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11a40512c0 con 0x7f11a41027e0 2024-05-09T05:45:30.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.664+0000 7f11a1ffb700 1 -- 172.21.15.115:0/626979480 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11953 (secure 0 0 0) 0x7f11940a9450 con 0x7f11a41027e0 2024-05-09T05:45:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:45:30.658 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:45:17.254442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26869":{"gid":26869,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/3665923682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3665923682},{"type":"v1","addr":"172.21.15.195:6833","nonce":3665923682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:45:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11900b29f0 msgr2=0x7f11900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11900b29f0 0x7f11900b4f00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f118c00b5c0 tx=0x7f118c005fb0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 msgr2=0x7f11a4110fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 0x7f11a4110fb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f119400d950 tx=0x7f119400dd10 comp rx=0 tx=0).stop 2024-05-09T05:45:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 shutdown_connections 2024-05-09T05:45:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11900b29f0 0x7f11900b4f00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11a410a3e0 0x7f11a410e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11a41027e0 0x7f11a4110fb0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 --2- 172.21.15.115:0/626979480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a4103130 0x7f11a410e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 >> 172.21.15.115:0/626979480 conn(0x7f11a40fe6f0 msgr2=0x7f11a4107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 shutdown_connections 2024-05-09T05:45:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:30.667+0000 7f11aa44e700 1 -- 172.21.15.115:0/626979480 wait complete. 2024-05-09T05:45:30.662 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2024-05-09T05:45:31.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:31 smithi115 ceph-mon[30349]: pgmap v225: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.8 MiB/s wr, 10 op/s 2024-05-09T05:45:31.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:31 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/626979480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:45:31.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:31 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:45:31.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:45:31.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:31 smithi195 ceph-mon[33844]: pgmap v225: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.8 MiB/s wr, 10 op/s 2024-05-09T05:45:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:31 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/626979480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:45:31.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:31 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:45:31.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:31 smithi161 ceph-mon[33805]: pgmap v225: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.8 MiB/s wr, 10 op/s 2024-05-09T05:45:31.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:31 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/626979480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:45:31.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:31 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:45:31.772 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:32.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:32 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:45:32.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:32 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:45:32.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:32 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:45:32.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:32 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:45:32.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:32 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:45:32.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:32 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:45:32.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:32 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:45:32.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:32 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:45:32.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:32 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:45:33.469 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:33 smithi115 ceph-mon[30349]: pgmap v226: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.8 MiB/s wr, 10 op/s 2024-05-09T05:45:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.608+0000 7fcecdfc8700 1 -- 172.21.15.115:0/2591664629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec81034c0 msgr2=0x7fcec81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.608+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/2591664629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec81034c0 0x7fcec81038d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fceb80099c0 tx=0x7fceb8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.608+0000 7fcecdfc8700 1 -- 172.21.15.115:0/2591664629 shutdown_connections 2024-05-09T05:45:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.608+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/2591664629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec8108740 0x7fcec81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.608+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/2591664629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec81034c0 0x7fcec81038d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.608+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/2591664629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec8107df0 0x7fcec8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.608+0000 7fcecdfc8700 1 -- 172.21.15.115:0/2591664629 >> 172.21.15.115:0/2591664629 conn(0x7fcec80fedc0 msgr2=0x7fcec81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.608+0000 7fcecdfc8700 1 -- 172.21.15.115:0/2591664629 shutdown_connections 2024-05-09T05:45:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.608+0000 7fcecdfc8700 1 -- 172.21.15.115:0/2591664629 wait complete. 2024-05-09T05:45:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.609+0000 7fcecdfc8700 1 Processor -- start 2024-05-09T05:45:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.609+0000 7fcecdfc8700 1 -- start start 2024-05-09T05:45:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcecdfc8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec81034c0 0x7fcec819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcecdfc8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec8107df0 0x7fcec819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcecdfc8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec8108740 0x7fcec8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcecdfc8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8113220 con 0x7fcec8108740 2024-05-09T05:45:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcecdfc8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81130a0 con 0x7fcec81034c0 2024-05-09T05:45:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcecdfc8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81133a0 con 0x7fcec8107df0 2024-05-09T05:45:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec8108740 0x7fcec8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec8108740 0x7fcec8199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57872/0 (socket says 172.21.15.115:57872) 2024-05-09T05:45:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcec7fff700 1 -- 172.21.15.115:0/3089244885 learned_addr learned my addr 172.21.15.115:0/3089244885 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcec6ffd700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec8107df0 0x7fcec819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcec77fe700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec81034c0 0x7fcec819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcec7fff700 1 -- 172.21.15.115:0/3089244885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec8107df0 msgr2=0x7fcec819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.610+0000 7fcec7fff700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec8107df0 0x7fcec819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.611+0000 7fcec7fff700 1 -- 172.21.15.115:0/3089244885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec81034c0 msgr2=0x7fcec819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.611+0000 7fcec7fff700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec81034c0 0x7fcec819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.611+0000 7fcec7fff700 1 -- 172.21.15.115:0/3089244885 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec819a0f0 con 0x7fcec8108740 2024-05-09T05:45:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.611+0000 7fcec77fe700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec81034c0 0x7fcec819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:45:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.611+0000 7fcec7fff700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec8108740 0x7fcec8199a40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcec0007ee0 tx=0x7fcec000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.611+0000 7fcec4ff9700 1 -- 172.21.15.115:0/3089244885 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0004db0 con 0x7fcec8108740 2024-05-09T05:45:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.611+0000 7fcec4ff9700 1 -- 172.21.15.115:0/3089244885 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcec000de50 con 0x7fcec8108740 2024-05-09T05:45:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.611+0000 7fcecdfc8700 1 -- 172.21.15.115:0/3089244885 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec819a3e0 con 0x7fcec8108740 2024-05-09T05:45:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.612+0000 7fcec4ff9700 1 -- 172.21.15.115:0/3089244885 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0005020 con 0x7fcec8108740 2024-05-09T05:45:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.612+0000 7fcecdfc8700 1 -- 172.21.15.115:0/3089244885 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec806b610 con 0x7fcec8108740 2024-05-09T05:45:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.615+0000 7fcec4ff9700 1 -- 172.21.15.115:0/3089244885 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcec00051c0 con 0x7fcec8108740 2024-05-09T05:45:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.616+0000 7fcec4ff9700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb00b2a70 0x7fceb00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.616+0000 7fcec77fe700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb00b2a70 0x7fceb00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.616+0000 7fcec4ff9700 1 -- 172.21.15.115:0/3089244885 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcec00e6420 con 0x7fcec8108740 2024-05-09T05:45:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.616+0000 7fcec77fe700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb00b2a70 0x7fceb00b4f80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fceb800b580 tx=0x7fceb8005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.616+0000 7fcedc9deb80 1 Processor -- start 2024-05-09T05:45:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.616+0000 7fcedc9deb80 1 -- start start 2024-05-09T05:45:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7898e0160 0x55d7899794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7897f64a0 0x55d789979a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d789973ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d789895ff0 con 0x55d7899736a0 2024-05-09T05:45:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d7898347f0 con 0x55d7898e0160 2024-05-09T05:45:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d789891fb0 con 0x55d7897f64a0 2024-05-09T05:45:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d789973ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d789973ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57888/0 (socket says 172.21.15.115:57888) 2024-05-09T05:45:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec7fff700 1 -- 172.21.15.115:0/260083436 learned_addr learned my addr 172.21.15.115:0/260083436 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec6ffd700 1 --2- 172.21.15.115:0/260083436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7897f64a0 0x55d789979a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec77fe700 1 --2- 172.21.15.115:0/260083436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7898e0160 0x55d7899794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec7fff700 1 -- 172.21.15.115:0/260083436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7897f64a0 msgr2=0x55d789979a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec7fff700 1 --2- 172.21.15.115:0/260083436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7897f64a0 0x55d789979a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec7fff700 1 -- 172.21.15.115:0/260083436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7898e0160 msgr2=0x55d7899794f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec7fff700 1 --2- 172.21.15.115:0/260083436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7898e0160 0x55d7899794f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec7fff700 1 -- 172.21.15.115:0/260083436 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78982c370 con 0x55d7899736a0 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.617+0000 7fcec7fff700 1 --2- 172.21.15.115:0/260083436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d789973ad0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcec00ad460 tx=0x7fcec0021f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985ee85c9c58e82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.618+0000 7fceadffb700 1 -- 172.21.15.115:0/260083436 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec00adbd0 con 0x55d7899736a0 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.618+0000 7fceadffb700 1 -- 172.21.15.115:0/260083436 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcec00ad4d0 con 0x55d7899736a0 2024-05-09T05:45:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.618+0000 7fceadffb700 1 -- 172.21.15.115:0/260083436 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec00a9760 con 0x55d7899736a0 2024-05-09T05:45:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.618+0000 7fcedc9deb80 1 -- 172.21.15.115:0/260083436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 msgr2=0x55d789973ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/260083436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d789973ad0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcec00ad460 tx=0x7fcec0021f70 comp rx=0 tx=0).stop 2024-05-09T05:45:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 -- 172.21.15.115:0/260083436 shutdown_connections 2024-05-09T05:45:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/260083436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7898e0160 0x55d7899794f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/260083436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d789973ad0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/260083436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7897f64a0 0x55d789979a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 -- 172.21.15.115:0/260083436 >> 172.21.15.115:0/260083436 conn(0x55d7898566a0 msgr2=0x55d78996e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 -- 172.21.15.115:0/260083436 shutdown_connections 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 -- 172.21.15.115:0/260083436 wait complete. 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 -- start start 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.619+0000 7fcedc9deb80 1 Processor -- start 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78997d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7898e0160 0x55d78997d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7899736a0 0x55d78997dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d789711ba0 con 0x55d7898e0160 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d789850150 con 0x55d7897f64a0 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d7899801d0 con 0x55d7899736a0 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcedc9deb80 10 client.0 _ll_register_callbacks cb 0x55d7897076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec6ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7898e0160 0x55d78997d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec77fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78997d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec6ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7898e0160 0x55d78997d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57896/0 (socket says 172.21.15.115:57896) 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec6ffd700 1 -- 172.21.15.115:0/1408584797 learned_addr learned my addr 172.21.15.115:0/1408584797 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec7fff700 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7899736a0 0x55d78997dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec6ffd700 1 -- 172.21.15.115:0/1408584797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7899736a0 msgr2=0x55d78997dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec6ffd700 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7899736a0 0x55d78997dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec6ffd700 1 -- 172.21.15.115:0/1408584797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 msgr2=0x55d78997d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec6ffd700 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78997d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec6ffd700 1 -- 172.21.15.115:0/1408584797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d789711490 con 0x55d7898e0160 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec77fe700 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78997d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcec6ffd700 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7898e0160 0x55d78997d780 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00f3b0 tx=0x7fcebc00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63944615a8baf11a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.620+0000 7fcea6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcea6ffd700 1 -- 172.21.15.115:0/1408584797 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc01e060 con 0x55d7898e0160 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcea6ffd700 1 -- 172.21.15.115:0/1408584797 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcebc004460 con 0x55d7898e0160 2024-05-09T05:45:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1408584797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec00ad4d0 con 0x55d7898e0160 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcea6ffd700 1 -- 172.21.15.115:0/1408584797 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcebc025050 con 0x55d7898e0160 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcedc9deb80 10 client.17532 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1408584797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d789993480 con 0x55d7898e0160 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcea6ffd700 1 -- 172.21.15.115:0/1408584797 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcec00ad4d0 con 0x55d7898e0160 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcea6ffd700 1 -- 172.21.15.115:0/1408584797 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fcebc00f040 con 0x55d7898e0160 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcedc9deb80 10 client.17532 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcedc9deb80 10 client.17532 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcedc9deb80 20 client.17532 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55d789995b10 0x55d789997f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcedc9deb80 4 client.17532 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.621+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1408584797 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7898a39d0 con 0x55d789995b10 2024-05-09T05:45:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.622+0000 7fcec77fe700 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55d789995b10 0x55d789997f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.619 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:33 smithi195 ceph-mon[33844]: pgmap v226: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.8 MiB/s wr, 10 op/s 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.622+0000 7fcea6ffd700 10 client.17532 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.622+0000 7fcec77fe700 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55d789995b10 0x55d789997f30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1e2f47b8eb9a873 server_cookie=c40d847416b2ff1f in_seq=0 out_seq=0 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcea6ffd700 1 -- 172.21.15.115:0/1408584797 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7898a39d0 con 0x55d789995b10 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcea6ffd700 10 client.17532 handle_command_reply: tid=0 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 client.17532 shutdown 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1408584797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7898e0160 msgr2=0x55d78997d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7898e0160 0x55d78997d780 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00f3b0 tx=0x7fcebc00f6c0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1408584797 shutdown_connections 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78997d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55d789995b10 0x55d789997f30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d7899736a0 0x55d78997dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1408584797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7898e0160 0x55d78997d780 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcea6ffd700 0 client.17532 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:45:33.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1408584797 >> 172.21.15.115:0/1408584797 conn(0x55d7898566a0 msgr2=0x55d78996e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:33.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.623+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1408584797 shutdown_connections 2024-05-09T05:45:33.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.624+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1408584797 wait complete. 2024-05-09T05:45:33.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.624+0000 7fcedc9deb80 20 client.17532 trim_cache size 0 max 16384 2024-05-09T05:45:33.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcedc9deb80 1 Processor -- start 2024-05-09T05:45:33.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcedc9deb80 1 -- start start 2024-05-09T05:45:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7897f64a0 0x55d78997c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7899736a0 0x55d789977510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78997dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81ae250 con 0x55d7897f64a0 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81afd90 con 0x55d7899736a0 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81ae920 con 0x55d78997d7b0 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcec77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7897f64a0 0x55d78997c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78997dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78997dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:57392/0 (socket says 172.21.15.115:57392) 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcec77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7897f64a0 0x55d78997c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57898/0 (socket says 172.21.15.115:57898) 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcec7fff700 1 -- 172.21.15.115:0/1188291373 learned_addr learned my addr 172.21.15.115:0/1188291373 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.643+0000 7fcec6ffd700 1 --2- 172.21.15.115:0/1188291373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7899736a0 0x55d789977510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcec7fff700 1 -- 172.21.15.115:0/1188291373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7899736a0 msgr2=0x55d789977510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcec7fff700 1 --2- 172.21.15.115:0/1188291373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7899736a0 0x55d789977510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcec7fff700 1 -- 172.21.15.115:0/1188291373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7897f64a0 msgr2=0x55d78997c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcec7fff700 1 --2- 172.21.15.115:0/1188291373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7897f64a0 0x55d78997c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcec7fff700 1 -- 172.21.15.115:0/1188291373 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78996f0f0 con 0x55d78997d7b0 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcec77fe700 1 --2- 172.21.15.115:0/1188291373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7897f64a0 0x55d78997c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcec7fff700 1 --2- 172.21.15.115:0/1188291373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78997dbc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcec00a9500 tx=0x7fcec00a9420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7efa4367c4d77e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcead7fa700 1 -- 172.21.15.115:0/1188291373 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec00b1880 con 0x55d78997d7b0 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcead7fa700 1 -- 172.21.15.115:0/1188291373 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcec00adbd0 con 0x55d78997d7b0 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.644+0000 7fcead7fa700 1 -- 172.21.15.115:0/1188291373 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec00ad730 con 0x55d78997d7b0 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.645+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1188291373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 msgr2=0x55d78997dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.645+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1188291373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78997dbc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcec00a9500 tx=0x7fcec00a9420 comp rx=0 tx=0).stop 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.645+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1188291373 shutdown_connections 2024-05-09T05:45:33.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.645+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1188291373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78997dbc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.645+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1188291373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7897f64a0 0x55d78997c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.645+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/1188291373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7899736a0 0x55d789977510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.645+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1188291373 >> 172.21.15.115:0/1188291373 conn(0x55d7898566a0 msgr2=0x55d789976e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.645+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1188291373 shutdown_connections 2024-05-09T05:45:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.645+0000 7fcedc9deb80 1 -- 172.21.15.115:0/1188291373 wait complete. 2024-05-09T05:45:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.646+0000 7fcedc9deb80 1 -- start start 2024-05-09T05:45:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.646+0000 7fcedc9deb80 1 Processor -- start 2024-05-09T05:45:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.646+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78998dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.646+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d78998e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.646+0000 7fcedc9deb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78999c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.646+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81af6c0 con 0x55d7899736a0 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.646+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81aeff0 con 0x55d7897f64a0 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.646+0000 7fcedc9deb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81ac1f0 con 0x55d78997d7b0 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.646+0000 7fcedc9deb80 10 client.0 _ll_register_callbacks cb 0x55d7897076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78999c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec6ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d78998e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78999c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:57400/0 (socket says 172.21.15.115:57400) 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec7fff700 1 -- 172.21.15.115:0/2531386931 learned_addr learned my addr 172.21.15.115:0/2531386931 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec77fe700 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78998dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec7fff700 1 -- 172.21.15.115:0/2531386931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 msgr2=0x55d78998dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec7fff700 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78998dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec7fff700 1 -- 172.21.15.115:0/2531386931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 msgr2=0x55d78998e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:33.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec7fff700 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d78998e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec7fff700 1 -- 172.21.15.115:0/2531386931 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78999cc20 con 0x55d78997d7b0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec77fe700 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78998dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec6ffd700 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d78998e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcec7fff700 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78999c6e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcec00ad460 tx=0x7fcec0021fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6739e7b3f84fc829 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcea6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcea6ffd700 1 -- 172.21.15.115:0/2531386931 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec00cb070 con 0x55d78997d7b0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcedc9deb80 1 -- 172.21.15.115:0/2531386931 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec00adbd0 con 0x55d78997d7b0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.647+0000 7fcea6ffd700 1 -- 172.21.15.115:0/2531386931 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcec00a9760 con 0x55d78997d7b0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcea6ffd700 1 -- 172.21.15.115:0/2531386931 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcec0012270 con 0x55d78997d7b0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcedc9deb80 10 client.26965 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcedc9deb80 1 -- 172.21.15.115:0/2531386931 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d7899ff8c0 con 0x55d78997d7b0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcea6ffd700 1 -- 172.21.15.115:0/2531386931 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcec00adbd0 con 0x55d78997d7b0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcea6ffd700 1 -- 172.21.15.115:0/2531386931 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fcec00b5ea0 con 0x55d78997d7b0 2024-05-09T05:45:33.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcedc9deb80 10 client.26965 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcedc9deb80 10 client.26965 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcedc9deb80 20 client.26965 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55d7899ff8c0 0x55d789a01ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcedc9deb80 4 client.26965 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcedc9deb80 1 -- 172.21.15.115:0/2531386931 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d78989da70 con 0x55d7899ff8c0 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.648+0000 7fcec77fe700 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55d7899ff8c0 0x55d789a01ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcec77fe700 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55d7899ff8c0 0x55d789a01ce0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30c7598f9c750750 server_cookie=2b190dfdc7677348 in_seq=0 out_seq=0 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcea6ffd700 10 client.26965 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcea6ffd700 1 -- 172.21.15.115:0/2531386931 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d78989da70 con 0x55d7899ff8c0 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcea6ffd700 10 client.26965 handle_command_reply: tid=0 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 client.26965 shutdown 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 -- 172.21.15.115:0/2531386931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 msgr2=0x55d78999c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78999c6e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcec00ad460 tx=0x7fcec0021fe0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 -- 172.21.15.115:0/2531386931 shutdown_connections 2024-05-09T05:45:33.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7897f64a0 0x55d78998dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55d7899ff8c0 0x55d789a01ce0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d78997d7b0 0x55d78999c6e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 --2- 172.21.15.115:0/2531386931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d7899736a0 0x55d78998e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 -- 172.21.15.115:0/2531386931 >> 172.21.15.115:0/2531386931 conn(0x55d7898566a0 msgr2=0x55d78997dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:33.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcea6ffd700 0 client.26965 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:45:33.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 -- 172.21.15.115:0/2531386931 shutdown_connections 2024-05-09T05:45:33.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.649+0000 7fcedc9deb80 1 -- 172.21.15.115:0/2531386931 wait complete. 2024-05-09T05:45:33.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcedc9deb80 20 client.26965 trim_cache size 0 max 16384 2024-05-09T05:45:33.648 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:45:33.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 -- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb00b2a70 msgr2=0x7fceb00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:33.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb00b2a70 0x7fceb00b4f80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fceb800b580 tx=0x7fceb8005fb0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 -- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec8108740 msgr2=0x7fcec8199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:33.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec8108740 0x7fcec8199a40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcec0007ee0 tx=0x7fcec000d3b0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 -- 172.21.15.115:0/3089244885 shutdown_connections 2024-05-09T05:45:33.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fceb00b2a70 0x7fceb00b4f80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcec81034c0 0x7fcec819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcec8108740 0x7fcec8199a40 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 --2- 172.21.15.115:0/3089244885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec8107df0 0x7fcec819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:33.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 -- 172.21.15.115:0/3089244885 >> 172.21.15.115:0/3089244885 conn(0x7fcec80fedc0 msgr2=0x7fcec8101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:33.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.650+0000 7fcecdfc8700 1 -- 172.21.15.115:0/3089244885 shutdown_connections 2024-05-09T05:45:33.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:33.651+0000 7fcecdfc8700 1 -- 172.21.15.115:0/3089244885 wait complete. 2024-05-09T05:45:33.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:33 smithi161 ceph-mon[33805]: pgmap v226: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.8 MiB/s wr, 10 op/s 2024-05-09T05:45:34.463 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:45:34.735 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:35.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:35 smithi115 ceph-mon[30349]: pgmap v227: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 19 op/s 2024-05-09T05:45:35.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:35 smithi195 ceph-mon[33844]: pgmap v227: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 19 op/s 2024-05-09T05:45:35.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:35 smithi161 ceph-mon[33805]: pgmap v227: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 19 op/s 2024-05-09T05:45:36.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.646+0000 7fca9413c700 1 -- 172.21.15.115:0/444567603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c108740 msgr2=0x7fca8c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:36.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.646+0000 7fca9413c700 1 --2- 172.21.15.115:0/444567603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c108740 0x7fca8c1100a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fca8400b0d0 tx=0x7fca8400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.646+0000 7fca9413c700 1 -- 172.21.15.115:0/444567603 shutdown_connections 2024-05-09T05:45:36.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.646+0000 7fca9413c700 1 --2- 172.21.15.115:0/444567603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c108740 0x7fca8c1100a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.646+0000 7fca9413c700 1 --2- 172.21.15.115:0/444567603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c1034c0 0x7fca8c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.646+0000 7fca9413c700 1 --2- 172.21.15.115:0/444567603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca8c107df0 0x7fca8c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.646+0000 7fca9413c700 1 -- 172.21.15.115:0/444567603 >> 172.21.15.115:0/444567603 conn(0x7fca8c0fedc0 msgr2=0x7fca8c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:36.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.646+0000 7fca9413c700 1 -- 172.21.15.115:0/444567603 shutdown_connections 2024-05-09T05:45:36.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.646+0000 7fca9413c700 1 -- 172.21.15.115:0/444567603 wait complete. 2024-05-09T05:45:36.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca9413c700 1 Processor -- start 2024-05-09T05:45:36.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca9413c700 1 -- start start 2024-05-09T05:45:36.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca9413c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c1034c0 0x7fca8c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca9413c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c107df0 0x7fca8c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca9413c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca8c108740 0x7fca8c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca9413c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c113220 con 0x7fca8c1034c0 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca9413c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1130a0 con 0x7fca8c108740 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca9413c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1133a0 con 0x7fca8c107df0 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca91ed8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c1034c0 0x7fca8c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca91ed8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c1034c0 0x7fca8c19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57932/0 (socket says 172.21.15.115:57932) 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca91ed8700 1 -- 172.21.15.115:0/3795217597 learned_addr learned my addr 172.21.15.115:0/3795217597 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.647+0000 7fca916d7700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c107df0 0x7fca8c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca926d9700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca8c108740 0x7fca8c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca91ed8700 1 -- 172.21.15.115:0/3795217597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c107df0 msgr2=0x7fca8c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca91ed8700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c107df0 0x7fca8c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca91ed8700 1 -- 172.21.15.115:0/3795217597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca8c108740 msgr2=0x7fca8c19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca91ed8700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca8c108740 0x7fca8c19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca91ed8700 1 -- 172.21.15.115:0/3795217597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c19fc30 con 0x7fca8c1034c0 2024-05-09T05:45:36.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca916d7700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c107df0 0x7fca8c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca926d9700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca8c108740 0x7fca8c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:45:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca91ed8700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c1034c0 0x7fca8c19ad60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fca8800e9a0 tx=0x7fca8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca82ffd700 1 -- 172.21.15.115:0/3795217597 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8800cbd0 con 0x7fca8c1034c0 2024-05-09T05:45:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca82ffd700 1 -- 172.21.15.115:0/3795217597 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca8800cd70 con 0x7fca8c1034c0 2024-05-09T05:45:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca82ffd700 1 -- 172.21.15.115:0/3795217597 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88018680 con 0x7fca8c1034c0 2024-05-09T05:45:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca9413c700 1 -- 172.21.15.115:0/3795217597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c19ff20 con 0x7fca8c1034c0 2024-05-09T05:45:36.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.648+0000 7fca9413c700 1 -- 172.21.15.115:0/3795217597 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c1a7840 con 0x7fca8c1034c0 2024-05-09T05:45:36.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.652+0000 7fca82ffd700 1 -- 172.21.15.115:0/3795217597 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fca88010430 con 0x7fca8c1034c0 2024-05-09T05:45:36.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.652+0000 7fcaa2b52b80 1 Processor -- start 2024-05-09T05:45:36.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.652+0000 7fcaa2b52b80 1 -- start start 2024-05-09T05:45:36.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.652+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 0x55cff553f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.652+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff53bc4a0 0x55cff553fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 0x55cff5539ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cff545bff0 con 0x55cff53bc4a0 2024-05-09T05:45:36.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca82ffd700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fca780b29f0 0x7fca780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cff53fa7f0 con 0x55cff55396a0 2024-05-09T05:45:36.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cff5457fb0 con 0x55cff54a6160 2024-05-09T05:45:36.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca91ed8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff53bc4a0 0x55cff553fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca91ed8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff53bc4a0 0x55cff553fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57946/0 (socket says 172.21.15.115:57946) 2024-05-09T05:45:36.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca91ed8700 1 -- 172.21.15.115:0/333412195 learned_addr learned my addr 172.21.15.115:0/333412195 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:36.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca916d7700 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 0x55cff5539ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca926d9700 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 0x55cff553f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca926d9700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fca780b29f0 0x7fca780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca82ffd700 1 -- 172.21.15.115:0/3795217597 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fca880e2bf0 con 0x7fca8c1034c0 2024-05-09T05:45:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca91ed8700 1 -- 172.21.15.115:0/333412195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 msgr2=0x55cff553f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca91ed8700 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 0x55cff553f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca91ed8700 1 -- 172.21.15.115:0/333412195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 msgr2=0x55cff5539ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca91ed8700 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 0x55cff5539ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca91ed8700 1 -- 172.21.15.115:0/333412195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff53f2370 con 0x55cff53bc4a0 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.653+0000 7fca916d7700 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 0x55cff5539ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.654+0000 7fca91ed8700 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff53bc4a0 0x55cff553fa30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fca88069fe0 tx=0x7fca880a6310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff4155c637e4dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.654+0000 7fca926d9700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fca780b29f0 0x7fca780b4f00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca8400b770 tx=0x7fca8400b040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.654+0000 7fca77fff700 1 -- 172.21.15.115:0/333412195 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca880e6030 con 0x55cff53bc4a0 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.654+0000 7fca77fff700 1 -- 172.21.15.115:0/333412195 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca880e7040 con 0x55cff53bc4a0 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.654+0000 7fca77fff700 1 -- 172.21.15.115:0/333412195 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca880a9b90 con 0x55cff53bc4a0 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.655+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/333412195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff53bc4a0 msgr2=0x55cff553fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.655+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff53bc4a0 0x55cff553fa30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fca88069fe0 tx=0x7fca880a6310 comp rx=0 tx=0).stop 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.655+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/333412195 shutdown_connections 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.655+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 0x55cff5539ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.655+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 0x55cff553f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.655+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/333412195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff53bc4a0 0x55cff553fa30 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.655+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/333412195 >> 172.21.15.115:0/333412195 conn(0x55cff541c6a0 msgr2=0x55cff5534400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.655+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/333412195 shutdown_connections 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.655+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/333412195 wait complete. 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fcaa2b52b80 1 -- start start 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fcaa2b52b80 1 Processor -- start 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5543260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff54a6160 0x55cff55437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 0x55cff5543ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cff52d7ba0 con 0x55cff54a6160 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cff5416150 con 0x55cff55396a0 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cff55461d0 con 0x55cff53bc4a0 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fcaa2b52b80 10 client.0 _ll_register_callbacks cb 0x55cff52cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fca91ed8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5543260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fca916d7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff54a6160 0x55cff55437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fca91ed8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5543260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:57444/0 (socket says 172.21.15.115:57444) 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.656+0000 7fca91ed8700 1 -- 172.21.15.115:0/2758567838 learned_addr learned my addr 172.21.15.115:0/2758567838 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca926d9700 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 0x55cff5543ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca916d7700 1 -- 172.21.15.115:0/2758567838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 msgr2=0x55cff5543260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca916d7700 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5543260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca916d7700 1 -- 172.21.15.115:0/2758567838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 msgr2=0x55cff5543ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca916d7700 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 0x55cff5543ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca916d7700 1 -- 172.21.15.115:0/2758567838 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff52d7490 con 0x55cff54a6160 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca91ed8700 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5543260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca916d7700 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff54a6160 0x55cff55437a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x55cff55400c0 tx=0x7fca7c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb88c78fb3ad7c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca757fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca757fa700 1 -- 172.21.15.115:0/2758567838 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c004030 con 0x55cff54a6160 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca757fa700 1 -- 172.21.15.115:0/2758567838 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fca7c0041d0 con 0x55cff54a6160 2024-05-09T05:45:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/2758567838 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca880e7040 con 0x55cff54a6160 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fca757fa700 1 -- 172.21.15.115:0/2758567838 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fca7c0043d0 con 0x55cff54a6160 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fcaa2b52b80 10 client.17550 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.657+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/2758567838 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cff5555070 con 0x55cff54a6160 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.658+0000 7fca757fa700 1 -- 172.21.15.115:0/2758567838 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca880e7040 con 0x55cff54a6160 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.658+0000 7fca757fa700 1 -- 172.21.15.115:0/2758567838 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fca7c003ad0 con 0x55cff54a6160 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.658+0000 7fcaa2b52b80 10 client.17550 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.658+0000 7fcaa2b52b80 10 client.17550 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.658+0000 7fcaa2b52b80 20 client.17550 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.658+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55cff55576b0 0x55cff5559ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.658+0000 7fcaa2b52b80 4 client.17550 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.658+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/2758567838 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cff54699d0 con 0x55cff55576b0 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.658+0000 7fca91ed8700 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55cff55576b0 0x55cff5559ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.659+0000 7fca757fa700 10 client.17550 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.659+0000 7fca91ed8700 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55cff55576b0 0x55cff5559ad0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1121d708c4d216d4 server_cookie=6b9f6a12af5dd77c in_seq=0 out_seq=0 2024-05-09T05:45:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fca757fa700 1 -- 172.21.15.115:0/2758567838 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cff54699d0 con 0x55cff55576b0 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fca757fa700 10 client.17550 handle_command_reply: tid=0 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 client.17550 shutdown 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/2758567838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff54a6160 msgr2=0x55cff55437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff54a6160 0x55cff55437a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x55cff55400c0 tx=0x7fca7c0058e0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/2758567838 shutdown_connections 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff55396a0 0x55cff5543ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55cff55576b0 0x55cff5559ad0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5543260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/2758567838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff54a6160 0x55cff55437a0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fca757fa700 0 client.17550 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/2758567838 >> 172.21.15.115:0/2758567838 conn(0x55cff541c6a0 msgr2=0x55cff55341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/2758567838 shutdown_connections 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/2758567838 wait complete. 2024-05-09T05:45:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.660+0000 7fcaa2b52b80 20 client.17550 trim_cache size 0 max 16384 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fcaa2b52b80 1 Processor -- start 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fcaa2b52b80 1 -- start start 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff53bc4a0 0x55cff5543220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 0x55cff5543760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff5543ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1ae1f0 con 0x55cff55396a0 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1afd30 con 0x55cff53bc4a0 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1ae8c0 con 0x55cff54a6160 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fca926d9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff5543ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fca926d9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff5543ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57970/0 (socket says 172.21.15.115:57970) 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fca926d9700 1 -- 172.21.15.115:0/1588298168 learned_addr learned my addr 172.21.15.115:0/1588298168 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fca916d7700 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 0x55cff5543760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.684+0000 7fca91ed8700 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff53bc4a0 0x55cff5543220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca926d9700 1 -- 172.21.15.115:0/1588298168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 msgr2=0x55cff5543760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca926d9700 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 0x55cff5543760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca926d9700 1 -- 172.21.15.115:0/1588298168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff53bc4a0 msgr2=0x55cff5543220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca926d9700 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff53bc4a0 0x55cff5543220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca926d9700 1 -- 172.21.15.115:0/1588298168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff5539d00 con 0x55cff55396a0 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca916d7700 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 0x55cff5543760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca926d9700 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff5543ca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fca8400b0d0 tx=0x7fca84013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443261797e74f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca777fe700 1 -- 172.21.15.115:0/1588298168 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84007c20 con 0x55cff55396a0 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca777fe700 1 -- 172.21.15.115:0/1588298168 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca84026040 con 0x55cff55396a0 2024-05-09T05:45:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.685+0000 7fca777fe700 1 -- 172.21.15.115:0/1588298168 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8401f440 con 0x55cff55396a0 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.686+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/1588298168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 msgr2=0x55cff5543ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.686+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff5543ca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fca8400b0d0 tx=0x7fca84013000 comp rx=0 tx=0).stop 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.686+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/1588298168 shutdown_connections 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.686+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff53bc4a0 0x55cff5543220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.686+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff5543ca0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.686+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/1588298168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff54a6160 0x55cff5543760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.686+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/1588298168 >> 172.21.15.115:0/1588298168 conn(0x55cff541c6a0 msgr2=0x55cff5541b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.686+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/1588298168 shutdown_connections 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.686+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/1588298168 wait complete. 2024-05-09T05:45:36.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.687+0000 7fcaa2b52b80 1 -- start start 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.687+0000 7fcaa2b52b80 1 Processor -- start 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5540f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff54a6160 0x55cff554b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fcaa2b52b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff554b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1af660 con 0x55cff55396a0 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1aef90 con 0x55cff54a6160 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fcaa2b52b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1ac190 con 0x55cff53bc4a0 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fcaa2b52b80 10 client.0 _ll_register_callbacks cb 0x55cff52cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca91ed8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5540f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca91ed8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5540f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:57464/0 (socket says 172.21.15.115:57464) 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca91ed8700 1 -- 172.21.15.115:0/3336905525 learned_addr learned my addr 172.21.15.115:0/3336905525 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca916d7700 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff54a6160 0x55cff554b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca926d9700 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff554b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca91ed8700 1 -- 172.21.15.115:0/3336905525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff54a6160 msgr2=0x55cff554b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca91ed8700 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff54a6160 0x55cff554b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca91ed8700 1 -- 172.21.15.115:0/3336905525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 msgr2=0x55cff554b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca91ed8700 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff554b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca91ed8700 1 -- 172.21.15.115:0/3336905525 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff554e1b0 con 0x55cff53bc4a0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca926d9700 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff554b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca91ed8700 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5540f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fca880e0580 tx=0x7fca880a96a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b8768501f8771ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.688+0000 7fca757fa700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fca757fa700 1 -- 172.21.15.115:0/3336905525 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca880c2070 con 0x55cff53bc4a0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/3336905525 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca84026040 con 0x55cff53bc4a0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fca757fa700 1 -- 172.21.15.115:0/3336905525 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fca88010720 con 0x55cff53bc4a0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fca757fa700 1 -- 172.21.15.115:0/3336905525 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fca880a5a50 con 0x55cff53bc4a0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fcaa2b52b80 10 client.26983 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/3336905525 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cff55c1280 con 0x55cff53bc4a0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fca757fa700 1 -- 172.21.15.115:0/3336905525 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca84026040 con 0x55cff53bc4a0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fca757fa700 1 -- 172.21.15.115:0/3336905525 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fca880a5500 con 0x55cff53bc4a0 2024-05-09T05:45:36.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fcaa2b52b80 10 client.26983 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fcaa2b52b80 10 client.26983 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fcaa2b52b80 20 client.26983 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55cff55c1450 0x55cff55c3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fcaa2b52b80 4 client.26983 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.689+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/3336905525 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cff5463a70 con 0x55cff55c1450 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.690+0000 7fca916d7700 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55cff55c1450 0x55cff55c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.690+0000 7fca757fa700 10 client.26983 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fca916d7700 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55cff55c1450 0x55cff55c3870 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b2196ed17f1200ef server_cookie=77245f540d4d4cbc in_seq=0 out_seq=0 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fca757fa700 1 -- 172.21.15.115:0/3336905525 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cff5463a70 con 0x55cff55c1450 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fca757fa700 10 client.26983 handle_command_reply: tid=0 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 client.26983 shutdown 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/3336905525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 msgr2=0x55cff5540f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5540f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fca880e0580 tx=0x7fca880a96a0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/3336905525 shutdown_connections 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55cff55c1450 0x55cff55c3870 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cff55396a0 0x55cff554b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cff53bc4a0 0x55cff5540f80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fca757fa700 0 client.26983 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 --2- 172.21.15.115:0/3336905525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cff54a6160 0x55cff554b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/3336905525 >> 172.21.15.115:0/3336905525 conn(0x55cff541c6a0 msgr2=0x55cff5541750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/3336905525 shutdown_connections 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.691+0000 7fcaa2b52b80 1 -- 172.21.15.115:0/3336905525 wait complete. 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fcaa2b52b80 20 client.26983 trim_cache size 0 max 16384 2024-05-09T05:45:36.689 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:45:36.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 -- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fca780b29f0 msgr2=0x7fca780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fca780b29f0 0x7fca780b4f00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca8400b770 tx=0x7fca8400b040 comp rx=0 tx=0).stop 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 -- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c1034c0 msgr2=0x7fca8c19ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c1034c0 0x7fca8c19ad60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fca8800e9a0 tx=0x7fca8800ecb0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 -- 172.21.15.115:0/3795217597 shutdown_connections 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fca780b29f0 0x7fca780b4f00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca8c108740 0x7fca8c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca8c1034c0 0x7fca8c19ad60 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 --2- 172.21.15.115:0/3795217597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c107df0 0x7fca8c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 -- 172.21.15.115:0/3795217597 >> 172.21.15.115:0/3795217597 conn(0x7fca8c0fedc0 msgr2=0x7fca8c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.692+0000 7fca9413c700 1 -- 172.21.15.115:0/3795217597 shutdown_connections 2024-05-09T05:45:36.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:36.693+0000 7fca9413c700 1 -- 172.21.15.115:0/3795217597 wait complete. 2024-05-09T05:45:37.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:37 smithi115 ceph-mon[30349]: pgmap v228: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-05-09T05:45:37.503 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:45:37.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:37 smithi195 ceph-mon[33844]: pgmap v228: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-05-09T05:45:37.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:37 smithi161 ceph-mon[33805]: pgmap v228: 129 pgs: 129 active+clean; 5.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-05-09T05:45:37.768 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:39.542 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:39 smithi115 ceph-mon[30349]: pgmap v229: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-05-09T05:45:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.578+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/297713251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 msgr2=0x7fc3e01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.578+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/297713251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 0x7fc3e01100a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800b0d0 tx=0x7fc3d800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.578+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/297713251 shutdown_connections 2024-05-09T05:45:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.578+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/297713251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.578+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/297713251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 0x7fc3e01100a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.578+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/297713251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3e0107df0 0x7fc3e0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.578+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/297713251 >> 172.21.15.115:0/297713251 conn(0x7fc3e00fedc0 msgr2=0x7fc3e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.578+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/297713251 shutdown_connections 2024-05-09T05:45:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.578+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/297713251 wait complete. 2024-05-09T05:45:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e6a6c700 1 Processor -- start 2024-05-09T05:45:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e6a6c700 1 -- start start 2024-05-09T05:45:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e6a6c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e6a6c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3e0107df0 0x7fc3e019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e6a6c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 0x7fc3e019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e6a6c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e0113220 con 0x7fc3e0108740 2024-05-09T05:45:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e6a6c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01130a0 con 0x7fc3e01034c0 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e6a6c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01133a0 con 0x7fc3e0107df0 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e5009700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 0x7fc3e019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e5009700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 0x7fc3e019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58002/0 (socket says 172.21.15.115:58002) 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e5009700 1 -- 172.21.15.115:0/403685000 learned_addr learned my addr 172.21.15.115:0/403685000 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3dffff700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3e0107df0 0x7fc3e019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.579+0000 7fc3e4808700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3e5009700 1 -- 172.21.15.115:0/403685000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3e0107df0 msgr2=0x7fc3e019c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3e5009700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3e0107df0 0x7fc3e019c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3e5009700 1 -- 172.21.15.115:0/403685000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 msgr2=0x7fc3e019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3e5009700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3e5009700 1 -- 172.21.15.115:0/403685000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e0199910 con 0x7fc3e0108740 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3e4808700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3e5009700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 0x7fc3e019c5c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009850 tx=0x7fc3d8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3ddffb700 1 -- 172.21.15.115:0/403685000 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d801f070 con 0x7fc3e0108740 2024-05-09T05:45:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3ddffb700 1 -- 172.21.15.115:0/403685000 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3d8005500 con 0x7fc3e0108740 2024-05-09T05:45:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3ddffb700 1 -- 172.21.15.115:0/403685000 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d801b9a0 con 0x7fc3e0108740 2024-05-09T05:45:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/403685000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e0199ba0 con 0x7fc3e0108740 2024-05-09T05:45:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.580+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/403685000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e019a030 con 0x7fc3e0108740 2024-05-09T05:45:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.583+0000 7fc3ddffb700 1 -- 172.21.15.115:0/403685000 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc3d8022070 con 0x7fc3e0108740 2024-05-09T05:45:39.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.584+0000 7fc3ddffb700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3d00b2a70 0x7fc3d00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.584+0000 7fc3e4808700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3d00b2a70 0x7fc3d00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.584+0000 7fc3ddffb700 1 -- 172.21.15.115:0/403685000 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc3d800f020 con 0x7fc3e0108740 2024-05-09T05:45:39.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3e4808700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3d00b2a70 0x7fc3d00b4f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4005fd0 tx=0x7fc3d4005e30 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:39.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3f5482b80 1 Processor -- start 2024-05-09T05:45:39.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3f5482b80 1 -- start start 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 0x55919e814240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e6914a0 0x55919e814780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e80e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55919e7310a0 con 0x55919e80e400 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55919e6d69a0 con 0x55919e70bd00 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55919e72d060 con 0x55919e6914a0 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3e5009700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e80e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3e5009700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e80e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58004/0 (socket says 172.21.15.115:58004) 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3e5009700 1 -- 172.21.15.115:0/375388008 learned_addr learned my addr 172.21.15.115:0/375388008 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.585+0000 7fc3dffff700 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e6914a0 0x55919e814780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3e4808700 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 0x55919e814240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3dffff700 1 -- 172.21.15.115:0/375388008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 msgr2=0x55919e814240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3dffff700 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 0x55919e814240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3dffff700 1 -- 172.21.15.115:0/375388008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 msgr2=0x55919e80e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3dffff700 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e80e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3dffff700 1 -- 172.21.15.115:0/375388008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919e6c7500 con 0x55919e6914a0 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3e5009700 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e80e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3dffff700 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e6914a0 0x55919e814780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc005950 tx=0x7fc3cc00bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d6678ec2f81c2bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3caffd700 1 -- 172.21.15.115:0/375388008 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc010c10 con 0x55919e6914a0 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3caffd700 1 -- 172.21.15.115:0/375388008 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3cc0044c0 con 0x55919e6914a0 2024-05-09T05:45:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.586+0000 7fc3caffd700 1 -- 172.21.15.115:0/375388008 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc005020 con 0x55919e6914a0 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.587+0000 7fc3f5482b80 1 -- 172.21.15.115:0/375388008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e6914a0 msgr2=0x55919e814780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.587+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e6914a0 0x55919e814780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc005950 tx=0x7fc3cc00bee0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.587+0000 7fc3f5482b80 1 -- 172.21.15.115:0/375388008 shutdown_connections 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.587+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 0x55919e814240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.587+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e80e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.587+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/375388008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e6914a0 0x55919e814780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.587+0000 7fc3f5482b80 1 -- 172.21.15.115:0/375388008 >> 172.21.15.115:0/375388008 conn(0x55919e694f70 msgr2=0x55919e809220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.587+0000 7fc3f5482b80 1 -- 172.21.15.115:0/375388008 shutdown_connections 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.587+0000 7fc3f5482b80 1 -- 172.21.15.115:0/375388008 wait complete. 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.588+0000 7fc3f5482b80 1 -- start start 2024-05-09T05:45:39.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.588+0000 7fc3f5482b80 1 Processor -- start 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.588+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e817ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.588+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e70bd00 0x55919e818530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.588+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 0x55919e818a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.588+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55919e80f160 con 0x55919e70bd00 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.588+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55919e6cbb20 con 0x55919e6914a0 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.588+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55919e81b020 con 0x55919e80e400 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.588+0000 7fc3f5482b80 10 client.0 _ll_register_callbacks cb 0x55919e5a26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3dffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e70bd00 0x55919e818530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3dffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e70bd00 0x55919e818530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58010/0 (socket says 172.21.15.115:58010) 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3dffff700 1 -- 172.21.15.115:0/2575605512 learned_addr learned my addr 172.21.15.115:0/2575605512 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3e4808700 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e817ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3e5009700 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 0x55919e818a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3dffff700 1 -- 172.21.15.115:0/2575605512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 msgr2=0x55919e818a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:39.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3dffff700 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 0x55919e818a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3dffff700 1 -- 172.21.15.115:0/2575605512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 msgr2=0x55919e817ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3dffff700 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e817ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3dffff700 1 -- 172.21.15.115:0/2575605512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55919e5ac490 con 0x55919e70bd00 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3e4808700 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e817ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3dffff700 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e70bd00 0x55919e818530 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00b6c0 tx=0x7fc3cc010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5019224e03d32d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3bffff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3bffff700 1 -- 172.21.15.115:0/2575605512 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc011650 con 0x55919e70bd00 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.589+0000 7fc3bffff700 1 -- 172.21.15.115:0/2575605512 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc3cc0117f0 con 0x55919e70bd00 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.590+0000 7fc3f5482b80 1 -- 172.21.15.115:0/2575605512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3cc0044c0 con 0x55919e70bd00 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.590+0000 7fc3bffff700 1 -- 172.21.15.115:0/2575605512 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc3cc0119f0 con 0x55919e70bd00 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.590+0000 7fc3f5482b80 10 client.17565 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.590+0000 7fc3f5482b80 1 -- 172.21.15.115:0/2575605512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55919e8259c0 con 0x55919e70bd00 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.590+0000 7fc3bffff700 1 -- 172.21.15.115:0/2575605512 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3cc02e070 con 0x55919e70bd00 2024-05-09T05:45:39.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.590+0000 7fc3bffff700 1 -- 172.21.15.115:0/2575605512 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc3cc01b940 con 0x55919e70bd00 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.590+0000 7fc3f5482b80 10 client.17565 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.590+0000 7fc3f5482b80 10 client.17565 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.590+0000 7fc3f5482b80 20 client.17565 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.591+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55919e827f40 0x55919e82a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.591+0000 7fc3f5482b80 4 client.17565 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.591+0000 7fc3f5482b80 1 -- 172.21.15.115:0/2575605512 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55919e73ea80 con 0x55919e827f40 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.591+0000 7fc3e4808700 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55919e827f40 0x55919e82a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.591+0000 7fc3bffff700 10 client.17565 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.591+0000 7fc3e4808700 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55919e827f40 0x55919e82a360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=981e745b0e5a9ae2 server_cookie=4b88fa006d87d9b9 in_seq=0 out_seq=0 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.592+0000 7fc3bffff700 1 -- 172.21.15.115:0/2575605512 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55919e73ea80 con 0x55919e827f40 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.592+0000 7fc3bffff700 10 client.17565 handle_command_reply: tid=0 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.592+0000 7fc3f5482b80 1 client.17565 shutdown 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.592+0000 7fc3f5482b80 1 -- 172.21.15.115:0/2575605512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e70bd00 msgr2=0x55919e818530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.592+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e70bd00 0x55919e818530 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00b6c0 tx=0x7fc3cc010c40 comp rx=0 tx=0).stop 2024-05-09T05:45:39.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3f5482b80 1 -- 172.21.15.115:0/2575605512 shutdown_connections 2024-05-09T05:45:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e817ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55919e827f40 0x55919e82a360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 0x55919e818a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/2575605512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e70bd00 0x55919e818530 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3bffff700 0 client.17565 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:45:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3f5482b80 1 -- 172.21.15.115:0/2575605512 >> 172.21.15.115:0/2575605512 conn(0x55919e694f70 msgr2=0x55919e802ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3f5482b80 1 -- 172.21.15.115:0/2575605512 shutdown_connections 2024-05-09T05:45:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3f5482b80 1 -- 172.21.15.115:0/2575605512 wait complete. 2024-05-09T05:45:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.593+0000 7fc3f5482b80 20 client.17565 trim_cache size 0 max 16384 2024-05-09T05:45:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.611+0000 7fc3f5482b80 1 Processor -- start 2024-05-09T05:45:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.611+0000 7fc3f5482b80 1 -- start start 2024-05-09T05:45:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.611+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e6914a0 0x55919e824190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.611+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 0x55919e8123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.611+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 0x55919e82cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.611+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01b24a0 con 0x55919e6914a0 2024-05-09T05:45:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.611+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01b3fe0 con 0x55919e70bd00 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.611+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01b2b70 con 0x55919e80e400 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e4808700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e6914a0 0x55919e824190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e4808700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e6914a0 0x55919e824190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58020/0 (socket says 172.21.15.115:58020) 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e4808700 1 -- 172.21.15.115:0/1020586361 learned_addr learned my addr 172.21.15.115:0/1020586361 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e5009700 1 --2- 172.21.15.115:0/1020586361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 0x55919e82cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3dffff700 1 --2- 172.21.15.115:0/1020586361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 0x55919e8123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e4808700 1 -- 172.21.15.115:0/1020586361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 msgr2=0x55919e82cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e4808700 1 --2- 172.21.15.115:0/1020586361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 0x55919e82cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e4808700 1 -- 172.21.15.115:0/1020586361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 msgr2=0x55919e8123e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e4808700 1 --2- 172.21.15.115:0/1020586361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 0x55919e8123e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e4808700 1 -- 172.21.15.115:0/1020586361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919e817dc0 con 0x55919e6914a0 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3e4808700 1 --2- 172.21.15.115:0/1020586361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e6914a0 0x55919e824190 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400a5c0 tx=0x7fc3d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1bf0cd8a4dc995 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3ca7fc700 1 -- 172.21.15.115:0/1020586361 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4020310 con 0x55919e6914a0 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3ca7fc700 1 -- 172.21.15.115:0/1020586361 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3d4030070 con 0x55919e6914a0 2024-05-09T05:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.612+0000 7fc3ca7fc700 1 -- 172.21.15.115:0/1020586361 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4029800 con 0x55919e6914a0 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.613+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1020586361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e6914a0 msgr2=0x55919e824190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.613+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1020586361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e6914a0 0x55919e824190 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400a5c0 tx=0x7fc3d4011000 comp rx=0 tx=0).stop 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.613+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1020586361 shutdown_connections 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.613+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1020586361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e80e400 0x55919e82cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.613+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1020586361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e6914a0 0x55919e824190 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.613+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1020586361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e70bd00 0x55919e8123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.613+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1020586361 >> 172.21.15.115:0/1020586361 conn(0x55919e694f70 msgr2=0x55919e811cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.613+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1020586361 shutdown_connections 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.613+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1020586361 wait complete. 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.614+0000 7fc3f5482b80 1 -- start start 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.614+0000 7fc3f5482b80 1 Processor -- start 2024-05-09T05:45:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.614+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e82d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.614+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e70bd00 0x55919e829f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.614+0000 7fc3f5482b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e82a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.614+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01b3910 con 0x55919e80e400 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.614+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01b3240 con 0x55919e6914a0 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.614+0000 7fc3f5482b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01b0440 con 0x55919e70bd00 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3f5482b80 10 client.0 _ll_register_callbacks cb 0x55919e5a26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e5009700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e82a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e5009700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e82a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58026/0 (socket says 172.21.15.115:58026) 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e4808700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e82d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e5009700 1 -- 172.21.15.115:0/1356710415 learned_addr learned my addr 172.21.15.115:0/1356710415 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3dffff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e70bd00 0x55919e829f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e4808700 1 -- 172.21.15.115:0/1356710415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e70bd00 msgr2=0x55919e829f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e4808700 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e70bd00 0x55919e829f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e4808700 1 -- 172.21.15.115:0/1356710415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 msgr2=0x55919e82a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e4808700 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e82a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e4808700 1 -- 172.21.15.115:0/1356710415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55919e82a9d0 con 0x55919e6914a0 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e5009700 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e82a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3e4808700 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e82d1f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400aab0 tx=0x7fc3d40203c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7732e71400f7ca4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3bffff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.615+0000 7fc3bffff700 1 -- 172.21.15.115:0/1356710415 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4004c60 con 0x55919e6914a0 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3bffff700 1 -- 172.21.15.115:0/1356710415 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc3d4004e00 con 0x55919e6914a0 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1356710415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d4030070 con 0x55919e6914a0 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3bffff700 1 -- 172.21.15.115:0/1356710415 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc3d4045050 con 0x55919e6914a0 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3f5482b80 10 client.26888 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1356710415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55919e891060 con 0x55919e6914a0 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3bffff700 1 -- 172.21.15.115:0/1356710415 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3d4030070 con 0x55919e6914a0 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3bffff700 1 -- 172.21.15.115:0/1356710415 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc3d40053e0 con 0x55919e6914a0 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3f5482b80 10 client.26888 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3f5482b80 10 client.26888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3f5482b80 20 client.26888 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55919e891600 0x55919e893a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3f5482b80 4 client.26888 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.616+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1356710415 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55919e738b20 con 0x55919e891600 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.617+0000 7fc3dffff700 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55919e891600 0x55919e893a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.617+0000 7fc3bffff700 10 client.26888 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.617+0000 7fc3dffff700 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55919e891600 0x55919e893a20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a44b812421116a2 server_cookie=6ff23972cf6c2f51 in_seq=0 out_seq=0 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.617+0000 7fc3bffff700 1 -- 172.21.15.115:0/1356710415 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55919e738b20 con 0x55919e891600 2024-05-09T05:45:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.617+0000 7fc3bffff700 10 client.26888 handle_command_reply: tid=0 2024-05-09T05:45:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.617+0000 7fc3f5482b80 1 client.26888 shutdown 2024-05-09T05:45:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1356710415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 msgr2=0x55919e82d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e82d1f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400aab0 tx=0x7fc3d40203c0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:39 smithi195 ceph-mon[33844]: pgmap v229: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1356710415 shutdown_connections 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55919e6914a0 0x55919e82d1f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55919e891600 0x55919e893a20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55919e80e400 0x55919e82a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 --2- 172.21.15.115:0/1356710415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55919e70bd00 0x55919e829f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1356710415 >> 172.21.15.115:0/1356710415 conn(0x55919e694f70 msgr2=0x55919e802ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3bffff700 0 client.26888 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1356710415 shutdown_connections 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 1 -- 172.21.15.115:0/1356710415 wait complete. 2024-05-09T05:45:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.618+0000 7fc3f5482b80 20 client.26888 trim_cache size 0 max 16384 2024-05-09T05:45:39.617 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:45:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3d00b2a70 msgr2=0x7fc3d00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3d00b2a70 0x7fc3d00b4f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4005fd0 tx=0x7fc3d4005e30 comp rx=0 tx=0).stop 2024-05-09T05:45:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 msgr2=0x7fc3e019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 0x7fc3e019c5c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009850 tx=0x7fc3d8007d90 comp rx=0 tx=0).stop 2024-05-09T05:45:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/403685000 shutdown_connections 2024-05-09T05:45:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3d00b2a70 0x7fc3d00b4f80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e01034c0 0x7fc3e019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3e0108740 0x7fc3e019c5c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 --2- 172.21.15.115:0/403685000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3e0107df0 0x7fc3e019c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/403685000 >> 172.21.15.115:0/403685000 conn(0x7fc3e00fedc0 msgr2=0x7fc3e0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/403685000 shutdown_connections 2024-05-09T05:45:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:39.619+0000 7fc3e6a6c700 1 -- 172.21.15.115:0/403685000 wait complete. 2024-05-09T05:45:39.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:39 smithi161 ceph-mon[33805]: pgmap v229: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-05-09T05:45:41.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:41 smithi195 ceph-mon[33844]: pgmap v230: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-05-09T05:45:41.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:41 smithi115 ceph-mon[30349]: pgmap v230: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-05-09T05:45:41.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:41 smithi161 ceph-mon[33805]: pgmap v230: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-05-09T05:45:42.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:45:42.911 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:43.617 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:43 smithi195 ceph-mon[33844]: pgmap v231: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-05-09T05:45:43.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:43 smithi115 ceph-mon[30349]: pgmap v231: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-05-09T05:45:43.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:43 smithi161 ceph-mon[33805]: pgmap v231: 129 pgs: 129 active+clean; 5.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-05-09T05:45:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.699+0000 7f116da89640 1 -- 172.21.15.115:0/1250805751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 msgr2=0x7f1168112990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.699+0000 7f116da89640 1 --2- 172.21.15.115:0/1250805751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 0x7f1168112990 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1160009ca0 tx=0x7f116002a8c0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.699+0000 7f116da89640 1 -- 172.21.15.115:0/1250805751 shutdown_connections 2024-05-09T05:45:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.699+0000 7f116da89640 1 --2- 172.21.15.115:0/1250805751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 0x7f1168112990 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.699+0000 7f116da89640 1 --2- 172.21.15.115:0/1250805751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f116810dc10 0x7f1168110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.699+0000 7f116da89640 1 --2- 172.21.15.115:0/1250805751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11681029d0 0x7f1168102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.699+0000 7f116da89640 1 -- 172.21.15.115:0/1250805751 >> 172.21.15.115:0/1250805751 conn(0x7f11680782e0 msgr2=0x7f1168078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.699+0000 7f116da89640 1 -- 172.21.15.115:0/1250805751 shutdown_connections 2024-05-09T05:45:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.699+0000 7f116da89640 1 -- 172.21.15.115:0/1250805751 wait complete. 2024-05-09T05:45:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.700+0000 7f116da89640 1 Processor -- start 2024-05-09T05:45:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.700+0000 7f116da89640 1 -- start start 2024-05-09T05:45:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.700+0000 7f116da89640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11681029d0 0x7f11681a2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.700+0000 7f116da89640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f116810dc10 0x7f11681a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.700+0000 7f116da89640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 0x7f116819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.700+0000 7f116da89640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1168115fc0 con 0x7f1168110570 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.700+0000 7f116da89640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1168115e40 con 0x7f11681029d0 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.700+0000 7f116da89640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1168116140 con 0x7f116810dc10 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f1167fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 0x7f116819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f115effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f116810dc10 0x7f11681a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f1167fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 0x7f116819c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53192/0 (socket says 172.21.15.115:53192) 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f1167fff640 1 -- 172.21.15.115:0/1524131202 learned_addr learned my addr 172.21.15.115:0/1524131202 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f115effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f116810dc10 0x7f11681a2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:59848/0 (socket says 172.21.15.115:59848) 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f11677fe640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11681029d0 0x7f11681a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f1167fff640 1 -- 172.21.15.115:0/1524131202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f116810dc10 msgr2=0x7f11681a2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f1167fff640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f116810dc10 0x7f11681a2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f1167fff640 1 -- 172.21.15.115:0/1524131202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11681029d0 msgr2=0x7f11681a2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f1167fff640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11681029d0 0x7f11681a2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f1167fff640 1 -- 172.21.15.115:0/1524131202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116819cf20 con 0x7f1168110570 2024-05-09T05:45:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f1167fff640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 0x7f116819c880 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1160009dc0 tx=0x7f1160003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.701+0000 7f11657fa640 1 -- 172.21.15.115:0/1524131202 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160004040 con 0x7f1168110570 2024-05-09T05:45:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.702+0000 7f11657fa640 1 -- 172.21.15.115:0/1524131202 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1160004670 con 0x7f1168110570 2024-05-09T05:45:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.702+0000 7f11657fa640 1 -- 172.21.15.115:0/1524131202 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160030cc0 con 0x7f1168110570 2024-05-09T05:45:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.702+0000 7f116da89640 1 -- 172.21.15.115:0/1524131202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116819d1b0 con 0x7f1168110570 2024-05-09T05:45:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.702+0000 7f116da89640 1 -- 172.21.15.115:0/1524131202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11681a94f0 con 0x7f1168110570 2024-05-09T05:45:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.704+0000 7f11657fa640 1 -- 172.21.15.115:0/1524131202 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f11600041e0 con 0x7f1168110570 2024-05-09T05:45:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.705+0000 7f1174a1b740 1 Processor -- start 2024-05-09T05:45:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.705+0000 7f1174a1b740 1 -- start start 2024-05-09T05:45:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.705+0000 7f11657fa640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11540b2950 0x7f11540b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.705+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1d94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.705+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19c82f0 0x5628c1d94bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.705+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1d95100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.705+0000 7f1174a1b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5628c1cf97c0 con 0x5628c19f11b0 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f1174a1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5628c1c950e0 con 0x5628c19c82f0 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f1174a1b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5628c1cf5780 con 0x5628c19d04d0 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11677fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1d94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11677fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1d94680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53208/0 (socket says 172.21.15.115:53208) 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11657fa640 1 -- 172.21.15.115:0/1524131202 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f1160104050 con 0x7f1168110570 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11677fe640 1 -- 172.21.15.115:0/4218310495 learned_addr learned my addr 172.21.15.115:0/4218310495 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f115effd640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11540b2950 0x7f11540b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f115effd640 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19c82f0 0x5628c1d94bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f1167fff640 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1d95100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11677fe640 1 -- 172.21.15.115:0/4218310495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 msgr2=0x5628c1d95100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11677fe640 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1d95100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11677fe640 1 -- 172.21.15.115:0/4218310495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19c82f0 msgr2=0x5628c1d94bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11677fe640 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19c82f0 0x5628c1d94bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11677fe640 1 -- 172.21.15.115:0/4218310495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c1cbe7c0 con 0x5628c19f11b0 2024-05-09T05:45:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f1167fff640 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1d95100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:45:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f115effd640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11540b2950 0x7f11540b4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1168111100 tx=0x7f11500156d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f11677fe640 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1d94680 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1158009a40 tx=0x7f1158009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c94609d9056e1c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f115dffb640 1 -- 172.21.15.115:0/4218310495 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158017070 con 0x5628c19f11b0 2024-05-09T05:45:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f115dffb640 1 -- 172.21.15.115:0/4218310495 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1158013470 con 0x5628c19f11b0 2024-05-09T05:45:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.706+0000 7f115dffb640 1 -- 172.21.15.115:0/4218310495 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158012630 con 0x5628c19f11b0 2024-05-09T05:45:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.707+0000 7f1174a1b740 1 -- 172.21.15.115:0/4218310495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 msgr2=0x5628c1d94680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.707+0000 7f1174a1b740 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1d94680 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1158009a40 tx=0x7f1158009f00 comp rx=0 tx=0).stop 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.707+0000 7f1174a1b740 1 -- 172.21.15.115:0/4218310495 shutdown_connections 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.707+0000 7f1174a1b740 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1d95100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.707+0000 7f1174a1b740 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19c82f0 0x5628c1d94bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.707+0000 7f1174a1b740 1 --2- 172.21.15.115:0/4218310495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1d94680 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.707+0000 7f1174a1b740 1 -- 172.21.15.115:0/4218310495 >> 172.21.15.115:0/4218310495 conn(0x5628c19e5300 msgr2=0x5628c1d90e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.707+0000 7f1174a1b740 1 -- 172.21.15.115:0/4218310495 shutdown_connections 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.707+0000 7f1174a1b740 1 -- 172.21.15.115:0/4218310495 wait complete. 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.708+0000 7f1174a1b740 1 -- start start 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.708+0000 7f1174a1b740 1 Processor -- start 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 0x5628c1d9ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19d04d0 0x5628c1d9f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1da4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1174a1b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5628c1da17b0 con 0x5628c19f11b0 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1174a1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5628c1c809c0 con 0x5628c19d04d0 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1167fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1da4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1167fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1da4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53224/0 (socket says 172.21.15.115:53224) 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1167fff640 1 -- 172.21.15.115:0/1344112665 learned_addr learned my addr 172.21.15.115:0/1344112665 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1174a1b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5628c1da1930 con 0x5628c19c82f0 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f11677fe640 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 0x5628c1d9ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1174a1b740 10 client.0 _ll_register_callbacks cb 0x5628c1cad020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f115effd640 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19d04d0 0x5628c1d9f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1167fff640 1 -- 172.21.15.115:0/1344112665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 msgr2=0x5628c1d9ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1167fff640 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 0x5628c1d9ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1167fff640 1 -- 172.21.15.115:0/1344112665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19d04d0 msgr2=0x5628c1d9f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1167fff640 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19d04d0 0x5628c1d9f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1167fff640 1 -- 172.21.15.115:0/1344112665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c1ca71f0 con 0x5628c19f11b0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f115effd640 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19d04d0 0x5628c1d9f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1167fff640 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1da4ca0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f116004e000 tx=0x7f11600044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f105baab9bcb4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1146ffd640 1 -- 172.21.15.115:0/1344112665 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600cb800 con 0x5628c19f11b0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1146ffd640 1 -- 172.21.15.115:0/1344112665 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f11600cf120 con 0x5628c19f11b0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.709+0000 7f1174a1b740 1 -- 172.21.15.115:0/1344112665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1158013470 con 0x5628c19f11b0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1146ffd640 1 -- 172.21.15.115:0/1344112665 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f11600cb100 con 0x5628c19f11b0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1174a1b740 10 client.17583 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1174a1b740 1 -- 172.21.15.115:0/1344112665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5628c1a5c790 con 0x5628c19f11b0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1146ffd640 1 -- 172.21.15.115:0/1344112665 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1158013470 con 0x5628c19f11b0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1146ffd640 1 -- 172.21.15.115:0/1344112665 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f116008abe0 con 0x5628c19f11b0 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1174a1b740 10 client.17583 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1174a1b740 10 client.17583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1174a1b740 20 client.17583 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1174a1b740 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5628c19afd80 0x5628c1db0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1174a1b740 4 client.17583 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f1174a1b740 1 -- 172.21.15.115:0/1344112665 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628c1d073d0 con 0x5628c19afd80 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.710+0000 7f11677fe640 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5628c19afd80 0x5628c1db0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.711+0000 7f1146ffd640 10 client.17583 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.711+0000 7f11677fe640 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5628c19afd80 0x5628c1db0f70 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc29427020f797c server_cookie=8ede1652f1a14b15 in_seq=0 out_seq=0 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1146ffd640 1 -- 172.21.15.115:0/1344112665 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628c1d073d0 con 0x5628c19afd80 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1146ffd640 10 client.17583 handle_command_reply: tid=0 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 client.17583 shutdown 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 -- 172.21.15.115:0/1344112665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 msgr2=0x5628c1da4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1da4ca0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f116004e000 tx=0x7f11600044d0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 -- 172.21.15.115:0/1344112665 shutdown_connections 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19f11b0 0x5628c1da4ca0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19d04d0 0x5628c1d9f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5628c19afd80 0x5628c1db0f70 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 --2- 172.21.15.115:0/1344112665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 0x5628c1d9ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 -- 172.21.15.115:0/1344112665 >> 172.21.15.115:0/1344112665 conn(0x5628c19e5300 msgr2=0x5628c1d8f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1146ffd640 0 client.17583 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 -- 172.21.15.115:0/1344112665 shutdown_connections 2024-05-09T05:45:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 1 -- 172.21.15.115:0/1344112665 wait complete. 2024-05-09T05:45:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.712+0000 7f1174a1b740 20 client.17583 trim_cache size 0 max 16384 2024-05-09T05:45:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f1174a1b740 1 Processor -- start 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f1174a1b740 1 -- start start 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 0x5628c1d9e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19d04d0 0x5628c1db6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 0x5628c1db6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f1174a1b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b32f0 con 0x5628c19d04d0 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f1174a1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b4e30 con 0x5628c19f11b0 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f1174a1b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b39c0 con 0x5628c19c82f0 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f115effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19d04d0 0x5628c1db6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f115effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19d04d0 0x5628c1db6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53234/0 (socket says 172.21.15.115:53234) 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f115effd640 1 -- 172.21.15.115:0/2226798381 learned_addr learned my addr 172.21.15.115:0/2226798381 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f11677fe640 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 0x5628c1d9e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f1167fff640 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 0x5628c1db6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f115effd640 1 -- 172.21.15.115:0/2226798381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 msgr2=0x5628c1d9e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f115effd640 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 0x5628c1d9e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f115effd640 1 -- 172.21.15.115:0/2226798381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 msgr2=0x5628c1db6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f115effd640 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 0x5628c1db6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f115effd640 1 -- 172.21.15.115:0/2226798381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c1d9f650 con 0x5628c19d04d0 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.730+0000 7f11677fe640 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 0x5628c1d9e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.731+0000 7f1167fff640 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 0x5628c1db6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.731+0000 7f115effd640 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19d04d0 0x5628c1db6740 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1168076100 tx=0x7f1150009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb5be368ce1ae03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.731+0000 7f115d7fa640 1 -- 172.21.15.115:0/2226798381 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150021040 con 0x5628c19d04d0 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.731+0000 7f115d7fa640 1 -- 172.21.15.115:0/2226798381 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115002a070 con 0x5628c19d04d0 2024-05-09T05:45:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.731+0000 7f115d7fa640 1 -- 172.21.15.115:0/2226798381 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150025980 con 0x5628c19d04d0 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 -- 172.21.15.115:0/2226798381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19d04d0 msgr2=0x5628c1db6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19d04d0 0x5628c1db6740 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1168076100 tx=0x7f1150009110 comp rx=0 tx=0).stop 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 -- 172.21.15.115:0/2226798381 shutdown_connections 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 0x5628c1db6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19d04d0 0x5628c1db6740 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 --2- 172.21.15.115:0/2226798381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19c82f0 0x5628c1d9e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 -- 172.21.15.115:0/2226798381 >> 172.21.15.115:0/2226798381 conn(0x5628c19e5300 msgr2=0x5628c1d9e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 -- 172.21.15.115:0/2226798381 shutdown_connections 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 -- 172.21.15.115:0/2226798381 wait complete. 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.732+0000 7f1174a1b740 1 -- start start 2024-05-09T05:45:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f1174a1b740 1 Processor -- start 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19c82f0 0x5628c1daa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1daadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f1174a1b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 0x5628c1da5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f1174a1b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b4760 con 0x5628c19c82f0 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f1174a1b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b4090 con 0x5628c19f11b0 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f1174a1b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b12d0 con 0x5628c19d04d0 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f1174a1b740 10 client.0 _ll_register_callbacks cb 0x7f1160030e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f11677fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19c82f0 0x5628c1daa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f11677fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19c82f0 0x5628c1daa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53240/0 (socket says 172.21.15.115:53240) 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f11677fe640 1 -- 172.21.15.115:0/3875786065 learned_addr learned my addr 172.21.15.115:0/3875786065 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f1167fff640 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 0x5628c1da5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f115effd640 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1daadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.733+0000 7f11677fe640 1 -- 172.21.15.115:0/3875786065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 msgr2=0x5628c1daadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f11677fe640 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1daadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f11677fe640 1 -- 172.21.15.115:0/3875786065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 msgr2=0x5628c1da5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f11677fe640 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 0x5628c1da5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f11677fe640 1 -- 172.21.15.115:0/3875786065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c1ad35f0 con 0x5628c19c82f0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f115effd640 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1daadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f11677fe640 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19c82f0 0x5628c1daa8b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1158008010 tx=0x7f1158013cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ceac5445ed16f64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f1146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f1146ffd640 1 -- 172.21.15.115:0/3875786065 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158017040 con 0x5628c19c82f0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f1146ffd640 1 -- 172.21.15.115:0/3875786065 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f1158005dd0 con 0x5628c19c82f0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f1174a1b740 1 -- 172.21.15.115:0/3875786065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115002a070 con 0x5628c19c82f0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f1146ffd640 1 -- 172.21.15.115:0/3875786065 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1158036050 con 0x5628c19c82f0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f1174a1b740 10 client.17589 fetch_fsmap learned FSMap version 40 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f1174a1b740 1 -- 172.21.15.115:0/3875786065 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5628c1cbded0 con 0x5628c19c82f0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.734+0000 7f1146ffd640 1 -- 172.21.15.115:0/3875786065 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115002a070 con 0x5628c19c82f0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f1146ffd640 1 -- 172.21.15.115:0/3875786065 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f11580263b0 con 0x5628c19c82f0 2024-05-09T05:45:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f1174a1b740 10 client.17589 fetch_fsmap finished waiting for FSMap version 40 2024-05-09T05:45:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f1174a1b740 10 client.17589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f1174a1b740 20 client.17589 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f1174a1b740 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5628c19851c0 0x5628c1e1ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f1174a1b740 4 client.17589 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:45:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f1174a1b740 1 -- 172.21.15.115:0/3875786065 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628c1d01240 con 0x5628c19851c0 2024-05-09T05:45:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f115effd640 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5628c19851c0 0x5628c1e1ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f1146ffd640 10 client.17589 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.735+0000 7f115effd640 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5628c19851c0 0x5628c1e1ada0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4626e8f232d5c8 server_cookie=792ff12110415b17 in_seq=0 out_seq=0 2024-05-09T05:45:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1146ffd640 1 -- 172.21.15.115:0/3875786065 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5628c1d01240 con 0x5628c19851c0 2024-05-09T05:45:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1146ffd640 10 client.17589 handle_command_reply: tid=0 2024-05-09T05:45:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 client.17589 shutdown 2024-05-09T05:45:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 -- 172.21.15.115:0/3875786065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19c82f0 msgr2=0x5628c1daa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19c82f0 0x5628c1daa8b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1158008010 tx=0x7f1158013cf0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 -- 172.21.15.115:0/3875786065 shutdown_connections 2024-05-09T05:45:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628c19f11b0 0x5628c1da5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5628c19851c0 0x5628c1e1ada0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5628c19d04d0 0x5628c1daadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 --2- 172.21.15.115:0/3875786065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5628c19c82f0 0x5628c1daa8b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1146ffd640 0 client.17589 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 -- 172.21.15.115:0/3875786065 >> 172.21.15.115:0/3875786065 conn(0x5628c19e5300 msgr2=0x5628c1d8f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 -- 172.21.15.115:0/3875786065 shutdown_connections 2024-05-09T05:45:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 1 -- 172.21.15.115:0/3875786065 wait complete. 2024-05-09T05:45:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.744+0000 7f1174a1b740 20 client.17589 trim_cache size 0 max 16384 2024-05-09T05:45:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:45:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:45:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:45:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 -- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11540b2950 msgr2=0x7f11540b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11540b2950 0x7f11540b4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1168111100 tx=0x7f11500156d0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 -- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 msgr2=0x7f116819c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 0x7f116819c880 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1160009dc0 tx=0x7f1160003ba0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 -- 172.21.15.115:0/1524131202 shutdown_connections 2024-05-09T05:45:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f11540b2950 0x7f11540b4e60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1168110570 0x7f116819c880 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f116810dc10 0x7f11681a2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 --2- 172.21.15.115:0/1524131202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11681029d0 0x7f11681a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 -- 172.21.15.115:0/1524131202 >> 172.21.15.115:0/1524131202 conn(0x7f11680782e0 msgr2=0x7f11681012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 -- 172.21.15.115:0/1524131202 shutdown_connections 2024-05-09T05:45:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:43.745+0000 7f116da89640 1 -- 172.21.15.115:0/1524131202 wait complete. 2024-05-09T05:45:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:45 smithi195 ceph-mon[33844]: pgmap v232: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 26 op/s 2024-05-09T05:45:45.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:45 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:45:45.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:45 smithi115 ceph-mon[30349]: pgmap v232: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 26 op/s 2024-05-09T05:45:45.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:45 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:45:45.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:45 smithi161 ceph-mon[33805]: pgmap v232: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 26 op/s 2024-05-09T05:45:45.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:45 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:45:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.358+0000 7f4612d3f700 1 -- 172.21.15.115:0/3261747804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c108760 msgr2=0x7f460c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.358+0000 7f4612d3f700 1 --2- 172.21.15.115:0/3261747804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c108760 0x7f460c1100c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f460400b0d0 tx=0x7f460400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.358+0000 7f4612d3f700 1 -- 172.21.15.115:0/3261747804 shutdown_connections 2024-05-09T05:45:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.358+0000 7f4612d3f700 1 --2- 172.21.15.115:0/3261747804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c1034e0 0x7f460c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.358+0000 7f4612d3f700 1 --2- 172.21.15.115:0/3261747804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c108760 0x7f460c1100c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.358+0000 7f4612d3f700 1 --2- 172.21.15.115:0/3261747804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f460c107e10 0x7f460c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.358+0000 7f4612d3f700 1 -- 172.21.15.115:0/3261747804 >> 172.21.15.115:0/3261747804 conn(0x7f460c0fef30 msgr2=0x7f460c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.359+0000 7f4612d3f700 1 -- 172.21.15.115:0/3261747804 shutdown_connections 2024-05-09T05:45:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.359+0000 7f4612d3f700 1 -- 172.21.15.115:0/3261747804 wait complete. 2024-05-09T05:45:46.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f4612d3f700 1 Processor -- start 2024-05-09T05:45:46.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f4612d3f700 1 -- start start 2024-05-09T05:45:46.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f4612d3f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f460c1034e0 0x7f460c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f4612d3f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c107e10 0x7f460c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f4612d3f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c108760 0x7f460c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f4612d3f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f460c113240 con 0x7f460c107e10 2024-05-09T05:45:46.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f4612d3f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1130c0 con 0x7f460c108760 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f4612d3f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1133c0 con 0x7f460c1034e0 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f460bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c107e10 0x7f460c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f460bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c107e10 0x7f460c199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53260/0 (socket says 172.21.15.115:53260) 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.360+0000 7f460bfff700 1 -- 172.21.15.115:0/2024903003 learned_addr learned my addr 172.21.15.115:0/2024903003 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f4610adb700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f460c1034e0 0x7f460c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f46112dc700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c108760 0x7f460c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f460bfff700 1 -- 172.21.15.115:0/2024903003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f460c1034e0 msgr2=0x7f460c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f460bfff700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f460c1034e0 0x7f460c199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f460bfff700 1 -- 172.21.15.115:0/2024903003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c108760 msgr2=0x7f460c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f460bfff700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c108760 0x7f460c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f460bfff700 1 -- 172.21.15.115:0/2024903003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c1a9070 con 0x7f460c107e10 2024-05-09T05:45:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f46112dc700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c108760 0x7f460c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f460bfff700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c107e10 0x7f460c199eb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d980 tx=0x7f45f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f4609ffb700 1 -- 172.21.15.115:0/2024903003 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8017070 con 0x7f460c107e10 2024-05-09T05:45:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f4612d3f700 1 -- 172.21.15.115:0/2024903003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460c1a9360 con 0x7f460c107e10 2024-05-09T05:45:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f4609ffb700 1 -- 172.21.15.115:0/2024903003 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45f8009dc0 con 0x7f460c107e10 2024-05-09T05:45:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.361+0000 7f4609ffb700 1 -- 172.21.15.115:0/2024903003 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f800f3c0 con 0x7f460c107e10 2024-05-09T05:45:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.362+0000 7f4612d3f700 1 -- 172.21.15.115:0/2024903003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460c1a98a0 con 0x7f460c107e10 2024-05-09T05:45:46.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.365+0000 7f4609ffb700 1 -- 172.21.15.115:0/2024903003 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f45f8009930 con 0x7f460c107e10 2024-05-09T05:45:46.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.366+0000 7f4609ffb700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f45fc0b29f0 0x7f45fc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.366+0000 7f4610adb700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f45fc0b29f0 0x7f45fc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.366+0000 7f4609ffb700 1 -- 172.21.15.115:0/2024903003 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f45f80e2420 con 0x7f460c107e10 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.366+0000 7f4621755b80 1 Processor -- start 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.366+0000 7f4621755b80 1 -- start start 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.366+0000 7f4621755b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0bf5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.366+0000 7f4610adb700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f45fc0b29f0 0x7f45fc0b4f00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4600005950 tx=0x7f460000b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.366+0000 7f4621755b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 0x5557f0bf5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f4621755b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bef810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f4621755b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5557f0b120a0 con 0x5557f0bef400 2024-05-09T05:45:46.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f4621755b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5557f0ab79a0 con 0x5557f0aecd00 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f4621755b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5557f0b0e060 con 0x5557f0a724a0 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f46112dc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f46112dc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bef810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53266/0 (socket says 172.21.15.115:53266) 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f46112dc700 1 -- 172.21.15.115:0/3063832106 learned_addr learned my addr 172.21.15.115:0/3063832106 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f4610adb700 1 --2- 172.21.15.115:0/3063832106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0bf5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f460bfff700 1 --2- 172.21.15.115:0/3063832106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 0x5557f0bf5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f46112dc700 1 -- 172.21.15.115:0/3063832106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 msgr2=0x5557f0bf5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f46112dc700 1 --2- 172.21.15.115:0/3063832106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 0x5557f0bf5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f46112dc700 1 -- 172.21.15.115:0/3063832106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 msgr2=0x5557f0bf5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f46112dc700 1 --2- 172.21.15.115:0/3063832106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0bf5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f46112dc700 1 -- 172.21.15.115:0/3063832106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f0aa8500 con 0x5557f0bef400 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f46112dc700 1 --2- 172.21.15.115:0/3063832106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bef810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4604009dc0 tx=0x7f4604009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b87e12f9db973b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.367+0000 7f45f6ffd700 1 -- 172.21.15.115:0/3063832106 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604022070 con 0x5557f0bef400 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.368+0000 7f45f6ffd700 1 -- 172.21.15.115:0/3063832106 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4604009350 con 0x5557f0bef400 2024-05-09T05:45:46.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.368+0000 7f45f6ffd700 1 -- 172.21.15.115:0/3063832106 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460401c8c0 con 0x5557f0bef400 2024-05-09T05:45:46.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 -- 172.21.15.115:0/3063832106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 msgr2=0x5557f0bef810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:46.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 --2- 172.21.15.115:0/3063832106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bef810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4604009dc0 tx=0x7f4604009df0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 -- 172.21.15.115:0/3063832106 shutdown_connections 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 --2- 172.21.15.115:0/3063832106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0bf5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 --2- 172.21.15.115:0/3063832106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bef810 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 --2- 172.21.15.115:0/3063832106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 0x5557f0bf5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 -- 172.21.15.115:0/3063832106 >> 172.21.15.115:0/3063832106 conn(0x5557f0a75f70 msgr2=0x5557f0bea220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 -- 172.21.15.115:0/3063832106 shutdown_connections 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 -- 172.21.15.115:0/3063832106 wait complete. 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 -- start start 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.369+0000 7f4621755b80 1 Processor -- start 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f4621755b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0a724a0 0x5557f0bf9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f4621755b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0aecd00 0x5557f0bf9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f4621755b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0bf9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f4621755b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5557f0bf0100 con 0x5557f0aecd00 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f4621755b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5557f0aacb20 con 0x5557f0a724a0 2024-05-09T05:45:46.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f4621755b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5557f0c05280 con 0x5557f0bef400 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f4621755b80 10 client.0 _ll_register_callbacks cb 0x5557f09836c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f460bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0aecd00 0x5557f0bf9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f460bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0aecd00 0x5557f0bf9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53274/0 (socket says 172.21.15.115:53274) 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f460bfff700 1 -- 172.21.15.115:0/2205772129 learned_addr learned my addr 172.21.15.115:0/2205772129 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f46112dc700 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0bf9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f4610adb700 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0a724a0 0x5557f0bf9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f460bfff700 1 -- 172.21.15.115:0/2205772129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 msgr2=0x5557f0bf9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f460bfff700 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0bf9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f460bfff700 1 -- 172.21.15.115:0/2205772129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0a724a0 msgr2=0x5557f0bf9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f460bfff700 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0a724a0 0x5557f0bf9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f460bfff700 1 -- 172.21.15.115:0/2205772129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f098d490 con 0x5557f0aecd00 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.370+0000 7f46112dc700 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0bf9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.371+0000 7f460bfff700 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0aecd00 0x5557f0bf9680 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f45f80a6630 tx=0x7f45f80dfd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671501d28c02f054 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.371+0000 7f45ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.371+0000 7f45ebfff700 1 -- 172.21.15.115:0/2205772129 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8020070 con 0x5557f0aecd00 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.371+0000 7f45ebfff700 1 -- 172.21.15.115:0/2205772129 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f45f80a59b0 con 0x5557f0aecd00 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.371+0000 7f4621755b80 1 -- 172.21.15.115:0/2205772129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4604009350 con 0x5557f0aecd00 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.371+0000 7f45ebfff700 1 -- 172.21.15.115:0/2205772129 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f45f80a93c0 con 0x5557f0aecd00 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.371+0000 7f4621755b80 10 client.17601 fetch_fsmap learned FSMap version 41 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.371+0000 7f4621755b80 1 -- 172.21.15.115:0/2205772129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5557f0c06c20 con 0x5557f0aecd00 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.371+0000 7f45ebfff700 1 -- 172.21.15.115:0/2205772129 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4604009350 con 0x5557f0aecd00 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.372+0000 7f45ebfff700 1 -- 172.21.15.115:0/2205772129 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f45f80b1d90 con 0x5557f0aecd00 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.372+0000 7f4621755b80 10 client.17601 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.372+0000 7f4621755b80 10 client.17601 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.372+0000 7f4621755b80 20 client.17601 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.372+0000 7f4621755b80 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5557f0c09200 0x5557f0c0b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.372+0000 7f4621755b80 4 client.17601 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.372+0000 7f4621755b80 1 -- 172.21.15.115:0/2205772129 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557f0b1fa80 con 0x5557f0c09200 2024-05-09T05:45:46.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.372+0000 7f4610adb700 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5557f0c09200 0x5557f0c0b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.373+0000 7f4610adb700 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5557f0c09200 0x5557f0c0b620 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ae163db5b41f63d server_cookie=c2f8569d7021ccb in_seq=0 out_seq=0 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.373+0000 7f45ebfff700 10 client.17601 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.373+0000 7f45ebfff700 1 -- 172.21.15.115:0/2205772129 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5557f0b1fa80 con 0x5557f0c09200 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.373+0000 7f45ebfff700 10 client.17601 handle_command_reply: tid=0 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.373+0000 7f4621755b80 1 client.17601 shutdown 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.373+0000 7f4621755b80 1 -- 172.21.15.115:0/2205772129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0aecd00 msgr2=0x5557f0bf9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.373+0000 7f4621755b80 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0aecd00 0x5557f0bf9680 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f45f80a6630 tx=0x7f45f80dfd30 comp rx=0 tx=0).stop 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f4621755b80 1 -- 172.21.15.115:0/2205772129 shutdown_connections 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f4621755b80 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5557f0c09200 0x5557f0c0b620 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f4621755b80 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0a724a0 0x5557f0bf9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f4621755b80 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0bf9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f45ebfff700 0 client.17601 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:45:46.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f4621755b80 1 --2- 172.21.15.115:0/2205772129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0aecd00 0x5557f0bf9680 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f4621755b80 1 -- 172.21.15.115:0/2205772129 >> 172.21.15.115:0/2205772129 conn(0x5557f0a75f70 msgr2=0x5557f0be3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:46.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f4621755b80 1 -- 172.21.15.115:0/2205772129 shutdown_connections 2024-05-09T05:45:46.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f4621755b80 1 -- 172.21.15.115:0/2205772129 wait complete. 2024-05-09T05:45:46.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.374+0000 7f4621755b80 20 client.17601 trim_cache size 0 max 16384 2024-05-09T05:45:46.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.393+0000 7f4621755b80 1 Processor -- start 2024-05-09T05:45:46.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.393+0000 7f4621755b80 1 -- start start 2024-05-09T05:45:46.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.393+0000 7f4621755b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 0x5557f0bfb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.393+0000 7f4621755b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0be4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.393+0000 7f4621755b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bf96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.393+0000 7f4621755b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1b2430 con 0x5557f0bef400 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.393+0000 7f4621755b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1b3fa0 con 0x5557f0aecd00 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.393+0000 7f4621755b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1b2b00 con 0x5557f0a724a0 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f46112dc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bf96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f46112dc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bf96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53286/0 (socket says 172.21.15.115:53286) 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f46112dc700 1 -- 172.21.15.115:0/1954662923 learned_addr learned my addr 172.21.15.115:0/1954662923 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f460bfff700 1 --2- 172.21.15.115:0/1954662923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0be4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f4610adb700 1 --2- 172.21.15.115:0/1954662923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 0x5557f0bfb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f46112dc700 1 -- 172.21.15.115:0/1954662923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 msgr2=0x5557f0bfb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f46112dc700 1 --2- 172.21.15.115:0/1954662923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 0x5557f0bfb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f46112dc700 1 -- 172.21.15.115:0/1954662923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 msgr2=0x5557f0be4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f46112dc700 1 --2- 172.21.15.115:0/1954662923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0be4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f46112dc700 1 -- 172.21.15.115:0/1954662923 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f0be69d0 con 0x5557f0bef400 2024-05-09T05:45:46.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f46112dc700 1 --2- 172.21.15.115:0/1954662923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bf96b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4604009fd0 tx=0x7f460400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b81f5a75461fa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:46.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.394+0000 7f45f67fc700 1 -- 172.21.15.115:0/1954662923 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604022040 con 0x5557f0bef400 2024-05-09T05:45:46.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.395+0000 7f45f67fc700 1 -- 172.21.15.115:0/1954662923 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46040093d0 con 0x5557f0bef400 2024-05-09T05:45:46.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.395+0000 7f45f67fc700 1 -- 172.21.15.115:0/1954662923 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604027450 con 0x5557f0bef400 2024-05-09T05:45:46.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.395+0000 7f4621755b80 1 -- 172.21.15.115:0/1954662923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 msgr2=0x5557f0bf96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:46.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.395+0000 7f4621755b80 1 --2- 172.21.15.115:0/1954662923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bf96b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4604009fd0 tx=0x7f460400b840 comp rx=0 tx=0).stop 2024-05-09T05:45:46.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.396+0000 7f4621755b80 1 -- 172.21.15.115:0/1954662923 shutdown_connections 2024-05-09T05:45:46.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.396+0000 7f4621755b80 1 --2- 172.21.15.115:0/1954662923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0bef400 0x5557f0bf96b0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.396+0000 7f4621755b80 1 --2- 172.21.15.115:0/1954662923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0a724a0 0x5557f0bfb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.396+0000 7f4621755b80 1 --2- 172.21.15.115:0/1954662923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0be4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.396+0000 7f4621755b80 1 -- 172.21.15.115:0/1954662923 >> 172.21.15.115:0/1954662923 conn(0x5557f0a75f70 msgr2=0x5557f0be3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.396+0000 7f4621755b80 1 -- 172.21.15.115:0/1954662923 shutdown_connections 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.396+0000 7f4621755b80 1 -- 172.21.15.115:0/1954662923 wait complete. 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.396+0000 7f4621755b80 1 -- start start 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.396+0000 7f4621755b80 1 Processor -- start 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4621755b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0a724a0 0x5557f0c104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4621755b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0c10a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4621755b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0c10f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4621755b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1b38d0 con 0x5557f0a724a0 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4610adb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0a724a0 0x5557f0c104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4621755b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1b3200 con 0x5557f0aecd00 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4621755b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1b03d0 con 0x5557f0bef400 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4610adb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0a724a0 0x5557f0c104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53288/0 (socket says 172.21.15.115:53288) 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4610adb700 1 -- 172.21.15.115:0/391848113 learned_addr learned my addr 172.21.15.115:0/391848113 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f460bfff700 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0c10a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f46112dc700 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0c10f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4621755b80 10 client.0 _ll_register_callbacks cb 0x5557f09836c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4610adb700 1 -- 172.21.15.115:0/391848113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 msgr2=0x5557f0c10f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4610adb700 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0c10f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4610adb700 1 -- 172.21.15.115:0/391848113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 msgr2=0x5557f0c10a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4610adb700 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0c10a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f4610adb700 1 -- 172.21.15.115:0/391848113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f0bf5c70 con 0x5557f0a724a0 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f46112dc700 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0c10f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.397+0000 7f460bfff700 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0c10a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:46.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.398+0000 7f4610adb700 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0a724a0 0x5557f0c104d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4600005d90 tx=0x7f46000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d85c972bec3dd3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.398+0000 7f45ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.398+0000 7f45ebfff700 1 -- 172.21.15.115:0/391848113 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460002a580 con 0x5557f0a724a0 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.398+0000 7f45ebfff700 1 -- 172.21.15.115:0/391848113 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f460002a720 con 0x5557f0a724a0 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.398+0000 7f4621755b80 1 -- 172.21.15.115:0/391848113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46040093d0 con 0x5557f0a724a0 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.398+0000 7f45ebfff700 1 -- 172.21.15.115:0/391848113 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f460003b050 con 0x5557f0a724a0 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.398+0000 7f4621755b80 10 client.17607 fetch_fsmap learned FSMap version 41 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.398+0000 7f4621755b80 1 -- 172.21.15.115:0/391848113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5557f0beb0c0 con 0x5557f0a724a0 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f45ebfff700 1 -- 172.21.15.115:0/391848113 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f460002e070 con 0x5557f0a724a0 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f45ebfff700 1 -- 172.21.15.115:0/391848113 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f460001e420 con 0x5557f0a724a0 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f4621755b80 10 client.17607 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T05:45:46.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f4621755b80 10 client.17607 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f4621755b80 20 client.17607 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f4621755b80 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5557f0c73dd0 0x5557f0c761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f4621755b80 4 client.17607 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f4621755b80 1 -- 172.21.15.115:0/391848113 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f46040093d0 con 0x5557f0c73dd0 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f460bfff700 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5557f0c73dd0 0x5557f0c761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.399+0000 7f45ebfff700 10 client.17607 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f460bfff700 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5557f0c73dd0 0x5557f0c761f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8265cba748a5a492 server_cookie=da4245b500d735b0 in_seq=0 out_seq=0 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f45ebfff700 1 -- 172.21.15.115:0/391848113 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f46040093d0 con 0x5557f0c73dd0 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f45ebfff700 10 client.17607 handle_command_reply: tid=0 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 client.17607 shutdown 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 -- 172.21.15.115:0/391848113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0a724a0 msgr2=0x5557f0c104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0a724a0 0x5557f0c104d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4600005d90 tx=0x7f46000047c0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 -- 172.21.15.115:0/391848113 shutdown_connections 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x5557f0c73dd0 0x5557f0c761f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5557f0bef400 0x5557f0c10f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5557f0a724a0 0x5557f0c104d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f45ebfff700 0 client.17607 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 --2- 172.21.15.115:0/391848113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5557f0aecd00 0x5557f0c10a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 -- 172.21.15.115:0/391848113 >> 172.21.15.115:0/391848113 conn(0x5557f0a75f70 msgr2=0x5557f0bf2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 -- 172.21.15.115:0/391848113 shutdown_connections 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 1 -- 172.21.15.115:0/391848113 wait complete. 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.400+0000 7f4621755b80 20 client.17607 trim_cache size 0 max 16384 2024-05-09T05:45:46.399 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:45:46.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 -- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f45fc0b29f0 msgr2=0x7f45fc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:46.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f45fc0b29f0 0x7f45fc0b4f00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4600005950 tx=0x7f460000b410 comp rx=0 tx=0).stop 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 -- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c107e10 msgr2=0x7f460c199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c107e10 0x7f460c199eb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d980 tx=0x7f45f800dc90 comp rx=0 tx=0).stop 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 -- 172.21.15.115:0/2024903003 shutdown_connections 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f45fc0b29f0 0x7f45fc0b4f00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f460c108760 0x7f460c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f460c1034e0 0x7f460c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 --2- 172.21.15.115:0/2024903003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f460c107e10 0x7f460c199eb0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 -- 172.21.15.115:0/2024903003 >> 172.21.15.115:0/2024903003 conn(0x7f460c0fef30 msgr2=0x7f460c107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 -- 172.21.15.115:0/2024903003 shutdown_connections 2024-05-09T05:45:46.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:46.401+0000 7f4612d3f700 1 -- 172.21.15.115:0/2024903003 wait complete. 2024-05-09T05:45:47.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:45:47.395 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:47.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:47 smithi115 ceph-mon[30349]: pgmap v233: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-05-09T05:45:47.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:47 smithi195 ceph-mon[33844]: pgmap v233: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-05-09T05:45:47.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:47 smithi161 ceph-mon[33805]: pgmap v233: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 17 op/s 2024-05-09T05:45:49.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.272+0000 7f4ebb190700 1 -- 172.21.15.115:0/301313336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4109300 msgr2=0x7f4eb4109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.272+0000 7f4ebb190700 1 --2- 172.21.15.115:0/301313336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4109300 0x7f4eb4109710 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009990 tx=0x7f4ea8009ca0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.272+0000 7f4ebb190700 1 -- 172.21.15.115:0/301313336 shutdown_connections 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.272+0000 7f4ebb190700 1 --2- 172.21.15.115:0/301313336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb41030e0 0x7f4eb4111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.272+0000 7f4ebb190700 1 --2- 172.21.15.115:0/301313336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4109300 0x7f4eb4109710 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.272+0000 7f4ebb190700 1 --2- 172.21.15.115:0/301313336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb4102790 0x7f4eb4102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.272+0000 7f4ebb190700 1 -- 172.21.15.115:0/301313336 >> 172.21.15.115:0/301313336 conn(0x7f4eb40fe520 msgr2=0x7f4eb4100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.272+0000 7f4ebb190700 1 -- 172.21.15.115:0/301313336 shutdown_connections 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.272+0000 7f4ebb190700 1 -- 172.21.15.115:0/301313336 wait complete. 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.273+0000 7f4ebb190700 1 Processor -- start 2024-05-09T05:45:49.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.273+0000 7f4ebb190700 1 -- start start 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.273+0000 7f4ebb190700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4102790 0x7f4eb419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.273+0000 7f4ebb190700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb41030e0 0x7f4eb419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.273+0000 7f4ebb190700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4109300 0x7f4eb419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.273+0000 7f4ebb190700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb4114450 con 0x7f4eb4102790 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.273+0000 7f4ebb190700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41142d0 con 0x7f4eb4109300 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.273+0000 7f4ebb190700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41145d0 con 0x7f4eb41030e0 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb8f2c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4102790 0x7f4eb419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb8f2c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4102790 0x7f4eb419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53302/0 (socket says 172.21.15.115:53302) 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/1233374997 learned_addr learned my addr 172.21.15.115:0/1233374997 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb41030e0 0x7f4eb419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb972d700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4109300 0x7f4eb419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/1233374997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb41030e0 msgr2=0x7f4eb419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb41030e0 0x7f4eb419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/1233374997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4109300 msgr2=0x7f4eb419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4109300 0x7f4eb419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/1233374997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb419fd50 con 0x7f4eb4102790 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4102790 0x7f4eb419ae80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8000c00 tx=0x7f4ea80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb1ffb700 1 -- 172.21.15.115:0/1233374997 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8026070 con 0x7f4eb4102790 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4eb1ffb700 1 -- 172.21.15.115:0/1233374997 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ea8021410 con 0x7f4eb4102790 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.274+0000 7f4ebb190700 1 -- 172.21.15.115:0/1233374997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb419ffe0 con 0x7f4eb4102790 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.275+0000 7f4eb1ffb700 1 -- 172.21.15.115:0/1233374997 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8017650 con 0x7f4eb4102790 2024-05-09T05:45:49.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.275+0000 7f4ebb190700 1 -- 172.21.15.115:0/1233374997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb41a04c0 con 0x7f4eb4102790 2024-05-09T05:45:49.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.278+0000 7f4eb1ffb700 1 -- 172.21.15.115:0/1233374997 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4ea8029070 con 0x7f4eb4102790 2024-05-09T05:45:49.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4eb1ffb700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ea40b29f0 0x7f4ea40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ea40b29f0 0x7f4ea40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4eb1ffb700 1 -- 172.21.15.115:0/1233374997 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4ea80e3840 con 0x7f4eb4102790 2024-05-09T05:45:49.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4ec9ba6b80 1 Processor -- start 2024-05-09T05:45:49.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4ec9ba6b80 1 -- start start 2024-05-09T05:45:49.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50d914a0 0x557d50f14a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50f0e6a0 0x557d50f0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557d50e30ff0 con 0x557d50f0e6a0 2024-05-09T05:45:49.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557d50dcf7f0 con 0x557d50d914a0 2024-05-09T05:45:49.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557d50e2cfb0 con 0x557d50e7b160 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.279+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ea40b29f0 0x7f4ea40b4f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40feac0 tx=0x7f4ea0009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb8f2c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb8f2c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f144f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:59976/0 (socket says 172.21.15.115:59976) 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/1329837169 learned_addr learned my addr 172.21.15.115:0/1329837169 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb972d700 1 --2- 172.21.15.115:0/1329837169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50f0e6a0 0x557d50f0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/1329837169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50d914a0 0x557d50f14a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/1329837169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50d914a0 msgr2=0x557d50f14a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/1329837169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50d914a0 0x557d50f14a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/1329837169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50f0e6a0 msgr2=0x557d50f0ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/1329837169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50f0e6a0 0x557d50f0ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/1329837169 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d50dc7370 con 0x557d50e7b160 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/1329837169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f144f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80a68f0 tx=0x7f4ea80a6860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e9df4206d95f66c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.280+0000 7f4e9effd700 1 -- 172.21.15.115:0/1329837169 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80aa380 con 0x557d50e7b160 2024-05-09T05:45:49.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.281+0000 7f4e9effd700 1 -- 172.21.15.115:0/1329837169 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ea80aaf70 con 0x557d50e7b160 2024-05-09T05:45:49.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.281+0000 7f4e9effd700 1 -- 172.21.15.115:0/1329837169 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80aaad0 con 0x557d50e7b160 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.281+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1329837169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 msgr2=0x557d50f144f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.281+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1329837169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f144f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80a68f0 tx=0x7f4ea80a6860 comp rx=0 tx=0).stop 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.281+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1329837169 shutdown_connections 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.281+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1329837169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50f0e6a0 0x557d50f0ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.281+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1329837169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f144f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.281+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1329837169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50d914a0 0x557d50f14a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.281+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1329837169 >> 172.21.15.115:0/1329837169 conn(0x557d50df16a0 msgr2=0x557d50f09400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.282+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1329837169 shutdown_connections 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.282+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1329837169 wait complete. 2024-05-09T05:45:49.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.282+0000 7f4ec9ba6b80 1 -- start start 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.282+0000 7f4ec9ba6b80 1 Processor -- start 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f1dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f1e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557d50cacba0 con 0x557d50d914a0 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb8f2c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f1dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557d50deb150 con 0x557d50f0e6a0 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557d50f0f360 con 0x557d50e7b160 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb8f2c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f1dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53318/0 (socket says 172.21.15.115:53318) 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/2657548198 learned_addr learned my addr 172.21.15.115:0/2657548198 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4ec9ba6b80 10 client.0 _ll_register_callbacks cb 0x557d50ca26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f1e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb972d700 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/2657548198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 msgr2=0x557d50f1e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f1e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/2657548198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 msgr2=0x557d50f180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f180c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/2657548198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d50cac490 con 0x557d50d914a0 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.283+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f1e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f1dcc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005930 tx=0x7f4ea80a6130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc7123c25e0fb9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4e93fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4e93fff700 1 -- 172.21.15.115:0/2657548198 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80c7020 con 0x557d50d914a0 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4e93fff700 1 -- 172.21.15.115:0/2657548198 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4ea80aaad0 con 0x557d50d914a0 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2657548198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea80aaf70 con 0x557d50d914a0 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4e93fff700 1 -- 172.21.15.115:0/2657548198 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4ea80aec20 con 0x557d50d914a0 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4ec9ba6b80 10 client.17619 fetch_fsmap learned FSMap version 41 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2657548198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557d50f258e0 con 0x557d50d914a0 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4e93fff700 1 -- 172.21.15.115:0/2657548198 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ea80aaf70 con 0x557d50d914a0 2024-05-09T05:45:49.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.284+0000 7f4e93fff700 1 -- 172.21.15.115:0/2657548198 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f4ea80abba0 con 0x557d50d914a0 2024-05-09T05:45:49.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.285+0000 7f4ec9ba6b80 10 client.17619 fetch_fsmap finished waiting for FSMap version 41 2024-05-09T05:45:49.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.285+0000 7f4ec9ba6b80 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:49.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.285+0000 7f4ec9ba6b80 20 client.17619 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:49.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.285+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557d50f27e50 0x557d50f2a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.285+0000 7f4ec9ba6b80 4 client.17619 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:49.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.285+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2657548198 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d50e3e9d0 con 0x557d50f27e50 2024-05-09T05:45:49.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.285+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557d50f27e50 0x557d50f2a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.285+0000 7f4e93fff700 10 client.17619 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:49.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.285+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557d50f27e50 0x557d50f2a2d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57f6641882ad0fa server_cookie=7b999c1fd54d4847 in_seq=0 out_seq=0 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4e93fff700 1 -- 172.21.15.115:0/2657548198 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d50e3e9d0 con 0x557d50f27e50 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4e93fff700 10 client.17619 handle_command_reply: tid=0 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4ec9ba6b80 1 client.17619 shutdown 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2657548198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 msgr2=0x557d50f1dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f1dcc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005930 tx=0x7f4ea80a6130 comp rx=0 tx=0).stop 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2657548198 shutdown_connections 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557d50f27e50 0x557d50f2a2d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f1dcc0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2657548198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f1e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4e93fff700 0 client.17619 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.286+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2657548198 >> 172.21.15.115:0/2657548198 conn(0x557d50df16a0 msgr2=0x557d50f091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.287+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2657548198 shutdown_connections 2024-05-09T05:45:49.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.287+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2657548198 wait complete. 2024-05-09T05:45:49.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.287+0000 7f4ec9ba6b80 20 client.17619 trim_cache size 0 max 16384 2024-05-09T05:45:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4ec9ba6b80 1 Processor -- start 2024-05-09T05:45:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4ec9ba6b80 1 -- start start 2024-05-09T05:45:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f10e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f2fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41ae100 con 0x557d50d914a0 2024-05-09T05:45:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41afc70 con 0x557d50f0e6a0 2024-05-09T05:45:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41ae7d0 con 0x557d50e7b160 2024-05-09T05:45:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4eb3fff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f10e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4eb8f2c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4eb8f2c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53328/0 (socket says 172.21.15.115:53328) 2024-05-09T05:45:49.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.309+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/2286082520 learned_addr learned my addr 172.21.15.115:0/2286082520 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:49.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.310+0000 7f4eb972d700 1 --2- 172.21.15.115:0/2286082520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f2fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.310+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/2286082520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 msgr2=0x557d50f10e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:49.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.310+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/2286082520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f10e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.310+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/2286082520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 msgr2=0x557d50f2fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:49.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.310+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/2286082520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f2fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.310+0000 7f4eb8f2c700 1 -- 172.21.15.115:0/2286082520 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d50f0ad00 con 0x557d50d914a0 2024-05-09T05:45:49.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.310+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/2286082520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f126c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80aa800 tx=0x7f4ea80a6860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c079905fdbdf62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:49.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.312+0000 7f4e9e7fc700 1 -- 172.21.15.115:0/2286082520 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80c7070 con 0x557d50d914a0 2024-05-09T05:45:49.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.312+0000 7f4e9e7fc700 1 -- 172.21.15.115:0/2286082520 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ea80aa380 con 0x557d50d914a0 2024-05-09T05:45:49.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.312+0000 7f4e9e7fc700 1 -- 172.21.15.115:0/2286082520 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80b31b0 con 0x557d50d914a0 2024-05-09T05:45:49.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2286082520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 msgr2=0x557d50f126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:49.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2286082520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f126c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80aa800 tx=0x7f4ea80a6860 comp rx=0 tx=0).stop 2024-05-09T05:45:49.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2286082520 shutdown_connections 2024-05-09T05:45:49.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2286082520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f2fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2286082520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50d914a0 0x557d50f126c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/2286082520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50e7b160 0x557d50f10e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2286082520 >> 172.21.15.115:0/2286082520 conn(0x557d50df16a0 msgr2=0x557d50f169d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:49.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2286082520 shutdown_connections 2024-05-09T05:45:49.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/2286082520 wait complete. 2024-05-09T05:45:49.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.313+0000 7f4ec9ba6b80 1 -- start start 2024-05-09T05:45:49.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4ec9ba6b80 1 Processor -- start 2024-05-09T05:45:49.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50d914a0 0x557d50f24780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50e7b160 0x557d50f1eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4ec9ba6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f1f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41af5a0 con 0x557d50e7b160 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41aeed0 con 0x557d50f0e6a0 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4ec9ba6b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb41ac0a0 con 0x557d50d914a0 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4ec9ba6b80 10 client.0 _ll_register_callbacks cb 0x557d50ca26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4eb3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50e7b160 0x557d50f1eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4eb3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50e7b160 0x557d50f1eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53334/0 (socket says 172.21.15.115:53334) 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4eb3fff700 1 -- 172.21.15.115:0/1669399993 learned_addr learned my addr 172.21.15.115:0/1669399993 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.314+0000 7f4eb972d700 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f1f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.315+0000 7f4eb3fff700 1 -- 172.21.15.115:0/1669399993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50d914a0 msgr2=0x557d50f24780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.315+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50d914a0 0x557d50f24780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.315+0000 7f4eb3fff700 1 -- 172.21.15.115:0/1669399993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 msgr2=0x557d50f1f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:49.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.315+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f1f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.315+0000 7f4eb3fff700 1 -- 172.21.15.115:0/1669399993 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d50f90630 con 0x557d50e7b160 2024-05-09T05:45:49.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.315+0000 7f4eb3fff700 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50e7b160 0x557d50f1eb60 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x557d50f0aed0 tx=0x7f4ea00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17c2ac837cb68a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:49.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.315+0000 7f4e93fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:49.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.317+0000 7f4e93fff700 1 -- 172.21.15.115:0/1669399993 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea002b890 con 0x557d50e7b160 2024-05-09T05:45:49.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.317+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1669399993 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea80aa380 con 0x557d50e7b160 2024-05-09T05:45:49.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.317+0000 7f4e93fff700 1 -- 172.21.15.115:0/1669399993 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4ea002ba30 con 0x557d50e7b160 2024-05-09T05:45:49.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.317+0000 7f4e93fff700 1 -- 172.21.15.115:0/1669399993 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4ea003a050 con 0x557d50e7b160 2024-05-09T05:45:49.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.317+0000 7f4ec9ba6b80 10 client.17625 fetch_fsmap learned FSMap version 42 2024-05-09T05:45:49.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.317+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1669399993 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557d50f91830 con 0x557d50e7b160 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.317+0000 7f4e93fff700 1 -- 172.21.15.115:0/1669399993 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ea80aa380 con 0x557d50e7b160 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.317+0000 7f4e93fff700 1 -- 172.21.15.115:0/1669399993 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f4ea0028070 con 0x557d50e7b160 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.318+0000 7f4ec9ba6b80 10 client.17625 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.318+0000 7f4ec9ba6b80 10 client.17625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.318+0000 7f4ec9ba6b80 20 client.17625 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.318+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557d50f91f50 0x557d50f94370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.318+0000 7f4ec9ba6b80 4 client.17625 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.318+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1669399993 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d50e38a70 con 0x557d50f91f50 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.318+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557d50f91f50 0x557d50f94370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.318+0000 7f4e93fff700 10 client.17625 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.318+0000 7f4eb8f2c700 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557d50f91f50 0x557d50f94370 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ebd1af72777d9d2 server_cookie=a42a98d94089ef4f in_seq=0 out_seq=0 2024-05-09T05:45:49.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4e93fff700 1 -- 172.21.15.115:0/1669399993 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d50e38a70 con 0x557d50f91f50 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4e93fff700 10 client.17625 handle_command_reply: tid=0 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 client.17625 shutdown 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1669399993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50e7b160 msgr2=0x557d50f1eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50e7b160 0x557d50f1eb60 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x557d50f0aed0 tx=0x7f4ea00047c0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1669399993 shutdown_connections 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557d50f0e6a0 0x557d50f1f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557d50f91f50 0x557d50f94370 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557d50d914a0 0x557d50f24780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 --2- 172.21.15.115:0/1669399993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557d50e7b160 0x557d50f1eb60 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4e93fff700 0 client.17625 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1669399993 >> 172.21.15.115:0/1669399993 conn(0x557d50df16a0 msgr2=0x557d50f11e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1669399993 shutdown_connections 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 1 -- 172.21.15.115:0/1669399993 wait complete. 2024-05-09T05:45:49.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.319+0000 7f4ec9ba6b80 20 client.17625 trim_cache size 0 max 16384 2024-05-09T05:45:49.315 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:45:49.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 -- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ea40b29f0 msgr2=0x7f4ea40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:49.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ea40b29f0 0x7f4ea40b4f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40feac0 tx=0x7f4ea0009260 comp rx=0 tx=0).stop 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 -- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4102790 msgr2=0x7f4eb419ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4102790 0x7f4eb419ae80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8000c00 tx=0x7f4ea80047b0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 -- 172.21.15.115:0/1233374997 shutdown_connections 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ea40b29f0 0x7f4ea40b4f00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4eb4109300 0x7f4eb419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eb4102790 0x7f4eb419ae80 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 --2- 172.21.15.115:0/1233374997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb41030e0 0x7f4eb419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 -- 172.21.15.115:0/1233374997 >> 172.21.15.115:0/1233374997 conn(0x7f4eb40fe520 msgr2=0x7f4eb41072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 -- 172.21.15.115:0/1233374997 shutdown_connections 2024-05-09T05:45:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:49.320+0000 7f4ebb190700 1 -- 172.21.15.115:0/1233374997 wait complete. 2024-05-09T05:45:49.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:49 smithi115 ceph-mon[30349]: pgmap v234: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 MiB/s wr, 29 op/s 2024-05-09T05:45:49.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:49 smithi195 ceph-mon[33844]: pgmap v234: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 MiB/s wr, 29 op/s 2024-05-09T05:45:49.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:49 smithi161 ceph-mon[33805]: pgmap v234: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 MiB/s wr, 29 op/s 2024-05-09T05:45:50.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:50 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:45:50.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:50 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:45:50.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:50 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:45:51.134 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:45:51.134 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:45:51.410 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:51.536 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:51 smithi115 ceph-mon[30349]: pgmap v235: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:45:51.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:51 smithi195 ceph-mon[33844]: pgmap v235: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:45:51.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:51 smithi161 ceph-mon[33805]: pgmap v235: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:45:53.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.296+0000 7f902bbfe700 1 -- 172.21.15.115:0/3911434553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 msgr2=0x7f902410e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:53.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.296+0000 7f902bbfe700 1 --2- 172.21.15.115:0/3911434553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 0x7f902410e380 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9014009990 tx=0x7f9014009ca0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.296+0000 7f902bbfe700 1 -- 172.21.15.115:0/3911434553 shutdown_connections 2024-05-09T05:45:53.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.296+0000 7f902bbfe700 1 --2- 172.21.15.115:0/3911434553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90241054e0 0x7f90241058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.296+0000 7f902bbfe700 1 --2- 172.21.15.115:0/3911434553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902410e910 0x7f9024110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.296+0000 7f902bbfe700 1 --2- 172.21.15.115:0/3911434553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 0x7f902410e380 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.296+0000 7f902bbfe700 1 -- 172.21.15.115:0/3911434553 >> 172.21.15.115:0/3911434553 conn(0x7f90240fef30 msgr2=0x7f9024101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:53.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.296+0000 7f902bbfe700 1 -- 172.21.15.115:0/3911434553 shutdown_connections 2024-05-09T05:45:53.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.296+0000 7f902bbfe700 1 -- 172.21.15.115:0/3911434553 wait complete. 2024-05-09T05:45:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.297+0000 7f902bbfe700 1 Processor -- start 2024-05-09T05:45:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.297+0000 7f902bbfe700 1 -- start start 2024-05-09T05:45:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.297+0000 7f902bbfe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90241054e0 0x7f90241a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.297+0000 7f902bbfe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 0x7f90241a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f902bbfe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902410e910 0x7f902419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f902bbfe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9024114320 con 0x7f9024105e30 2024-05-09T05:45:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f902bbfe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90241141a0 con 0x7f90241054e0 2024-05-09T05:45:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f902bbfe700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90241144a0 con 0x7f902410e910 2024-05-09T05:45:53.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f9029199700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 0x7f90241a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f9029199700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 0x7f90241a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37280/0 (socket says 172.21.15.115:37280) 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f9029199700 1 -- 172.21.15.115:0/184417867 learned_addr learned my addr 172.21.15.115:0/184417867 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f902999a700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90241054e0 0x7f90241a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f902a19b700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902410e910 0x7f902419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f9029199700 1 -- 172.21.15.115:0/184417867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902410e910 msgr2=0x7f902419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f9029199700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902410e910 0x7f902419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f9029199700 1 -- 172.21.15.115:0/184417867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90241054e0 msgr2=0x7f90241a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f9029199700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90241054e0 0x7f90241a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f9029199700 1 -- 172.21.15.115:0/184417867 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902419b270 con 0x7f9024105e30 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f902999a700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90241054e0 0x7f90241a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.298+0000 7f9029199700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 0x7f90241a0ef0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9014000c00 tx=0x7f90140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.299+0000 7f901affd700 1 -- 172.21.15.115:0/184417867 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014027070 con 0x7f9024105e30 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.299+0000 7f901affd700 1 -- 172.21.15.115:0/184417867 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f901400fb80 con 0x7f9024105e30 2024-05-09T05:45:53.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.299+0000 7f901affd700 1 -- 172.21.15.115:0/184417867 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014022d00 con 0x7f9024105e30 2024-05-09T05:45:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.299+0000 7f902bbfe700 1 -- 172.21.15.115:0/184417867 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902419b500 con 0x7f9024105e30 2024-05-09T05:45:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.299+0000 7f902bbfe700 1 -- 172.21.15.115:0/184417867 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902419b990 con 0x7f9024105e30 2024-05-09T05:45:53.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.302+0000 7f901affd700 1 -- 172.21.15.115:0/184417867 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f901402a070 con 0x7f9024105e30 2024-05-09T05:45:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f901affd700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90100b2a70 0x7f90100b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f902999a700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90100b2a70 0x7f90100b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f901affd700 1 -- 172.21.15.115:0/184417867 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f90140e37e0 con 0x7f9024105e30 2024-05-09T05:45:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f903a614b80 1 Processor -- start 2024-05-09T05:45:53.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f903a614b80 1 -- start start 2024-05-09T05:45:53.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f903a614b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa82980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f902999a700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90100b2a70 0x7f90100b4f80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f902000bc10 tx=0x7f902000b5f0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:53.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f903a614b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa82ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f903a614b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa7cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f903a614b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fb1a99b070 con 0x55fb1aa7cb30 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f903a614b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fb1a939870 con 0x55fb1a9e51e0 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.303+0000 7f903a614b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fb1a997030 con 0x55fb1a8fb520 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902a19b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa7cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902a19b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa7cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37290/0 (socket says 172.21.15.115:37290) 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902a19b700 1 -- 172.21.15.115:0/4087829929 learned_addr learned my addr 172.21.15.115:0/4087829929 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f9029199700 1 --2- 172.21.15.115:0/4087829929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa82ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902999a700 1 --2- 172.21.15.115:0/4087829929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa82980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902a19b700 1 -- 172.21.15.115:0/4087829929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 msgr2=0x55fb1aa82ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902a19b700 1 --2- 172.21.15.115:0/4087829929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa82ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902a19b700 1 -- 172.21.15.115:0/4087829929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 msgr2=0x55fb1aa82980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902a19b700 1 --2- 172.21.15.115:0/4087829929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa82980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902a19b700 1 -- 172.21.15.115:0/4087829929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb1a9313f0 con 0x55fb1aa7cb30 2024-05-09T05:45:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f902a19b700 1 --2- 172.21.15.115:0/4087829929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa7cf60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f901c003fa0 tx=0x7f901c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dd7993653df712 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:53.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f900ffff700 1 -- 172.21.15.115:0/4087829929 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c004bb0 con 0x55fb1aa7cb30 2024-05-09T05:45:53.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f900ffff700 1 -- 172.21.15.115:0/4087829929 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f901c0048e0 con 0x55fb1aa7cb30 2024-05-09T05:45:53.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.304+0000 7f900ffff700 1 -- 172.21.15.115:0/4087829929 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c007c80 con 0x55fb1aa7cb30 2024-05-09T05:45:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.305+0000 7f903a614b80 1 -- 172.21.15.115:0/4087829929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 msgr2=0x55fb1aa7cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.305+0000 7f903a614b80 1 --2- 172.21.15.115:0/4087829929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa7cf60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f901c003fa0 tx=0x7f901c009ba0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.305+0000 7f903a614b80 1 -- 172.21.15.115:0/4087829929 shutdown_connections 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.305+0000 7f903a614b80 1 --2- 172.21.15.115:0/4087829929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa82980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.305+0000 7f903a614b80 1 --2- 172.21.15.115:0/4087829929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa7cf60 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.305+0000 7f903a614b80 1 --2- 172.21.15.115:0/4087829929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa82ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.305+0000 7f903a614b80 1 -- 172.21.15.115:0/4087829929 >> 172.21.15.115:0/4087829929 conn(0x55fb1a95b720 msgr2=0x55fb1aa73480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.306+0000 7f903a614b80 1 -- 172.21.15.115:0/4087829929 shutdown_connections 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.306+0000 7f903a614b80 1 -- 172.21.15.115:0/4087829929 wait complete. 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.306+0000 7f903a614b80 1 -- start start 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.306+0000 7f903a614b80 1 Processor -- start 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f903a614b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a8fb520 0x55fb1aa865d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f903a614b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa86b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f903a614b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa87050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f903a614b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fb1a816c20 con 0x55fb1a9e51e0 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f903a614b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fb1a9551d0 con 0x55fb1a8fb520 2024-05-09T05:45:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f903a614b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fb1aa89660 con 0x55fb1aa7cb30 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f9029199700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa86b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f9029199700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa86b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37306/0 (socket says 172.21.15.115:37306) 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f9029199700 1 -- 172.21.15.115:0/1150411776 learned_addr learned my addr 172.21.15.115:0/1150411776 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f903a614b80 10 client.0 _ll_register_callbacks cb 0x55fb1a80c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f902999a700 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a8fb520 0x55fb1aa865d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f902a19b700 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa87050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f9029199700 1 -- 172.21.15.115:0/1150411776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1aa7cb30 msgr2=0x55fb1aa87050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f9029199700 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa87050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f9029199700 1 -- 172.21.15.115:0/1150411776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a8fb520 msgr2=0x55fb1aa865d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f9029199700 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a8fb520 0x55fb1aa865d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f9029199700 1 -- 172.21.15.115:0/1150411776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb1a816510 con 0x55fb1a9e51e0 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.307+0000 7f902a19b700 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa87050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f9029199700 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa86b10 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f90140e0c50 tx=0x7f90140227e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4576223a87ab2242 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f900d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f900d7fa700 1 -- 172.21.15.115:0/1150411776 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90140aa780 con 0x55fb1a9e51e0 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f900d7fa700 1 -- 172.21.15.115:0/1150411776 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f90140b3150 con 0x55fb1a9e51e0 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f903a614b80 1 -- 172.21.15.115:0/1150411776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f901c0048e0 con 0x55fb1a9e51e0 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f900d7fa700 1 -- 172.21.15.115:0/1150411776 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f90140aeb30 con 0x55fb1a9e51e0 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f903a614b80 10 client.17637 fetch_fsmap learned FSMap version 42 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f903a614b80 1 -- 172.21.15.115:0/1150411776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fb1aa93fa0 con 0x55fb1a9e51e0 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f900d7fa700 1 -- 172.21.15.115:0/1150411776 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f901c0048e0 con 0x55fb1a9e51e0 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.308+0000 7f900d7fa700 1 -- 172.21.15.115:0/1150411776 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f90140a99e0 con 0x55fb1a9e51e0 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.309+0000 7f903a614b80 10 client.17637 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.309+0000 7f903a614b80 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.309+0000 7f903a614b80 20 client.17637 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.309+0000 7f903a614b80 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fb1aa964c0 0x55fb1aa98930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.309+0000 7f903a614b80 4 client.17637 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.309+0000 7f903a614b80 1 -- 172.21.15.115:0/1150411776 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb1a9a8a50 con 0x55fb1aa964c0 2024-05-09T05:45:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.309+0000 7f902999a700 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fb1aa964c0 0x55fb1aa98930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.310+0000 7f900d7fa700 10 client.17637 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.310+0000 7f902999a700 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fb1aa964c0 0x55fb1aa98930 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746d28740342cd33 server_cookie=a8d6948c9c0d907d in_seq=0 out_seq=0 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.310+0000 7f900d7fa700 1 -- 172.21.15.115:0/1150411776 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fb1a9a8a50 con 0x55fb1aa964c0 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.310+0000 7f900d7fa700 10 client.17637 handle_command_reply: tid=0 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.310+0000 7f903a614b80 1 client.17637 shutdown 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 -- 172.21.15.115:0/1150411776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1a9e51e0 msgr2=0x55fb1aa86b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa86b10 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f90140e0c50 tx=0x7f90140227e0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 -- 172.21.15.115:0/1150411776 shutdown_connections 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a8fb520 0x55fb1aa865d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fb1aa964c0 0x55fb1aa98930 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f900d7fa700 0 client.17637 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa87050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 --2- 172.21.15.115:0/1150411776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa86b10 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 -- 172.21.15.115:0/1150411776 >> 172.21.15.115:0/1150411776 conn(0x55fb1a95b720 msgr2=0x55fb1aa73240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:53.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 -- 172.21.15.115:0/1150411776 shutdown_connections 2024-05-09T05:45:53.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 1 -- 172.21.15.115:0/1150411776 wait complete. 2024-05-09T05:45:53.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.311+0000 7f903a614b80 20 client.17637 trim_cache size 0 max 16384 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f903a614b80 1 Processor -- start 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f903a614b80 1 -- start start 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f903a614b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa7c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f903a614b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa7f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f903a614b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa924a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f903a614b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90241ae080 con 0x55fb1aa7cb30 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f903a614b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90241afc50 con 0x55fb1a9e51e0 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f903a614b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90241ae750 con 0x55fb1a8fb520 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f902a19b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa924a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f902a19b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa924a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37318/0 (socket says 172.21.15.115:37318) 2024-05-09T05:45:53.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f902a19b700 1 -- 172.21.15.115:0/751150810 learned_addr learned my addr 172.21.15.115:0/751150810 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.336+0000 7f902999a700 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa7c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f9029199700 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa7f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f902999a700 1 -- 172.21.15.115:0/751150810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 msgr2=0x55fb1aa7f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f902999a700 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa7f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f902999a700 1 -- 172.21.15.115:0/751150810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 msgr2=0x55fb1aa924a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f902999a700 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa924a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f902999a700 1 -- 172.21.15.115:0/751150810 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb1aa7d4c0 con 0x55fb1a8fb520 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f902a19b700 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa924a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f902999a700 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa7c550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55fb1aa99060 tx=0x7f9020012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac01a2868ea68c01 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f900f7fe700 1 -- 172.21.15.115:0/751150810 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020011e40 con 0x55fb1a8fb520 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f900f7fe700 1 -- 172.21.15.115:0/751150810 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90200044b0 con 0x55fb1a8fb520 2024-05-09T05:45:53.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.337+0000 7f900f7fe700 1 -- 172.21.15.115:0/751150810 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902001f430 con 0x55fb1a8fb520 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.338+0000 7f903a614b80 1 -- 172.21.15.115:0/751150810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 msgr2=0x55fb1aa7c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.338+0000 7f903a614b80 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa7c550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55fb1aa99060 tx=0x7f9020012000 comp rx=0 tx=0).stop 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.338+0000 7f903a614b80 1 -- 172.21.15.115:0/751150810 shutdown_connections 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.338+0000 7f903a614b80 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1aa924a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.338+0000 7f903a614b80 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa7c550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.338+0000 7f903a614b80 1 --2- 172.21.15.115:0/751150810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa7f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.338+0000 7f903a614b80 1 -- 172.21.15.115:0/751150810 >> 172.21.15.115:0/751150810 conn(0x55fb1a95b720 msgr2=0x55fb1aa83580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.338+0000 7f903a614b80 1 -- 172.21.15.115:0/751150810 shutdown_connections 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.338+0000 7f903a614b80 1 -- 172.21.15.115:0/751150810 wait complete. 2024-05-09T05:45:53.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.339+0000 7f903a614b80 1 -- start start 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.339+0000 7f903a614b80 1 Processor -- start 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.339+0000 7f903a614b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa8bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f903a614b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa8c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f903a614b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1ab00cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f903a614b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f90241af580 con 0x55fb1aa7cb30 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f903a614b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90241aeeb0 con 0x55fb1a9e51e0 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f903a614b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90241ac020 con 0x55fb1a8fb520 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f903a614b80 10 client.0 _ll_register_callbacks cb 0x55fb1a80c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902a19b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1ab00cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902999a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa8bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902999a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa8bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33664/0 (socket says 172.21.15.115:33664) 2024-05-09T05:45:53.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902999a700 1 -- 172.21.15.115:0/14307165 learned_addr learned my addr 172.21.15.115:0/14307165 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f9029199700 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa8c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902a19b700 1 -- 172.21.15.115:0/14307165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 msgr2=0x55fb1aa8bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902a19b700 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa8bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902a19b700 1 -- 172.21.15.115:0/14307165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 msgr2=0x55fb1aa8c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902a19b700 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa8c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902a19b700 1 -- 172.21.15.115:0/14307165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb1ab01220 con 0x55fb1aa7cb30 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902999a700 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa8bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f902a19b700 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1ab00cb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f901c0093f0 tx=0x7f901c008c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cd2137234b5ba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.340+0000 7f900d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f900d7fa700 1 -- 172.21.15.115:0/14307165 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c015480 con 0x55fb1aa7cb30 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f900d7fa700 1 -- 172.21.15.115:0/14307165 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f901c015620 con 0x55fb1aa7cb30 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f903a614b80 1 -- 172.21.15.115:0/14307165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90200044b0 con 0x55fb1aa7cb30 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f900d7fa700 1 -- 172.21.15.115:0/14307165 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f901c030050 con 0x55fb1aa7cb30 2024-05-09T05:45:53.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f903a614b80 10 client.17643 fetch_fsmap learned FSMap version 42 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f903a614b80 1 -- 172.21.15.115:0/14307165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fb1ab024c0 con 0x55fb1aa7cb30 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f900d7fa700 1 -- 172.21.15.115:0/14307165 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f901c024070 con 0x55fb1aa7cb30 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f900d7fa700 1 -- 172.21.15.115:0/14307165 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f901c0047c0 con 0x55fb1aa7cb30 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f903a614b80 10 client.17643 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f903a614b80 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f903a614b80 20 client.17643 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f903a614b80 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fb1aa964c0 0x55fb1ab048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f903a614b80 4 client.17643 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.341+0000 7f903a614b80 1 -- 172.21.15.115:0/14307165 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f90200044b0 con 0x55fb1aa964c0 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.342+0000 7f902999a700 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fb1aa964c0 0x55fb1ab048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.342+0000 7f900d7fa700 10 client.17643 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.342+0000 7f902999a700 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fb1aa964c0 0x55fb1ab048a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20997ed4f8460ba9 server_cookie=ac93baab7b3c7649 in_seq=0 out_seq=0 2024-05-09T05:45:53.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.343+0000 7f900d7fa700 1 -- 172.21.15.115:0/14307165 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f90200044b0 con 0x55fb1aa964c0 2024-05-09T05:45:53.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.343+0000 7f900d7fa700 10 client.17643 handle_command_reply: tid=0 2024-05-09T05:45:53.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.343+0000 7f903a614b80 1 client.17643 shutdown 2024-05-09T05:45:53.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.343+0000 7f903a614b80 1 -- 172.21.15.115:0/14307165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 msgr2=0x55fb1ab00cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.343+0000 7f903a614b80 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1ab00cb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f901c0093f0 tx=0x7f901c008c20 comp rx=0 tx=0).stop 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f903a614b80 1 -- 172.21.15.115:0/14307165 shutdown_connections 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f903a614b80 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fb1aa964c0 0x55fb1ab048a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f903a614b80 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fb1aa7cb30 0x55fb1ab00cb0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f903a614b80 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fb1a8fb520 0x55fb1aa8bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f903a614b80 1 --2- 172.21.15.115:0/14307165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb1a9e51e0 0x55fb1aa8c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f903a614b80 1 -- 172.21.15.115:0/14307165 >> 172.21.15.115:0/14307165 conn(0x55fb1a95b720 msgr2=0x55fb1aa7c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f900d7fa700 0 client.17643 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f903a614b80 1 -- 172.21.15.115:0/14307165 shutdown_connections 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f903a614b80 1 -- 172.21.15.115:0/14307165 wait complete. 2024-05-09T05:45:53.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f903a614b80 20 client.17643 trim_cache size 0 max 16384 2024-05-09T05:45:53.341 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:45:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:45:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "62cf5198-3119-4120-97f3-fbefadef9297", 2024-05-09T05:45:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:45:53.341 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:45:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f902bbfe700 1 -- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90100b2a70 msgr2=0x7f90100b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.344+0000 7f902bbfe700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90100b2a70 0x7f90100b4f80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f902000bc10 tx=0x7f902000b5f0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 -- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 msgr2=0x7f90241a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 0x7f90241a0ef0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9014000c00 tx=0x7f90140047b0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 -- 172.21.15.115:0/184417867 shutdown_connections 2024-05-09T05:45:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f90100b2a70 0x7f90100b4f80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90241054e0 0x7f90241a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f902410e910 0x7f902419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 --2- 172.21.15.115:0/184417867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9024105e30 0x7f90241a0ef0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:53.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 -- 172.21.15.115:0/184417867 >> 172.21.15.115:0/184417867 conn(0x7f90240fef30 msgr2=0x7f9024100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:53.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 -- 172.21.15.115:0/184417867 shutdown_connections 2024-05-09T05:45:53.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:53.345+0000 7f902bbfe700 1 -- 172.21.15.115:0/184417867 wait complete. 2024-05-09T05:45:53.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:53 smithi195 ceph-mon[33844]: pgmap v236: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:45:53.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:53 smithi115 ceph-mon[30349]: pgmap v236: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:45:53.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:53 smithi161 ceph-mon[33805]: pgmap v236: 129 pgs: 129 active+clean; 5.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:45:54.223 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:45:54.497 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:45:54.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:45:54.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.616 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:54 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.621 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:45:54.621 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.621 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:45:54.621 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:45:54.622 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.622 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:45:54.622 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.622 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:45:54.622 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:45:54.622 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.622 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:45:54.622 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.622 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:54 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:45:54.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:45:54.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:45:54.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:45:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:45:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:45:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:45:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:54.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:54 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:55 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:55.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:55 smithi195 ceph-mon[33844]: pgmap v237: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 29 op/s 2024-05-09T05:45:55.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:55 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:55.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:55 smithi115 ceph-mon[30349]: pgmap v237: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 29 op/s 2024-05-09T05:45:55.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:55 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:45:55.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:55 smithi161 ceph-mon[33805]: pgmap v237: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 29 op/s 2024-05-09T05:45:56.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.587+0000 7fc7dbde5700 1 -- 172.21.15.115:0/4017626411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d41030e0 msgr2=0x7fc7d4111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.587+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/4017626411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d41030e0 0x7fc7d4111170 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00b0a0 tx=0x7fc7cc00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.587+0000 7fc7dbde5700 1 -- 172.21.15.115:0/4017626411 shutdown_connections 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.587+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/4017626411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d41030e0 0x7fc7d4111170 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.587+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/4017626411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7d4109300 0x7fc7d4109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.587+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/4017626411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7d4102790 0x7fc7d4102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.587+0000 7fc7dbde5700 1 -- 172.21.15.115:0/4017626411 >> 172.21.15.115:0/4017626411 conn(0x7fc7d40fe520 msgr2=0x7fc7d4100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.587+0000 7fc7dbde5700 1 -- 172.21.15.115:0/4017626411 shutdown_connections 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.587+0000 7fc7dbde5700 1 -- 172.21.15.115:0/4017626411 wait complete. 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.588+0000 7fc7dbde5700 1 Processor -- start 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.588+0000 7fc7dbde5700 1 -- start start 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.588+0000 7fc7dbde5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7d4102790 0x7fc7d4103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.588+0000 7fc7dbde5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7d41030e0 0x7fc7d4103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7dbde5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4109300 0x7fc7d41043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7dbde5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41144a0 con 0x7fc7d4109300 2024-05-09T05:45:56.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7dbde5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d4114320 con 0x7fc7d41030e0 2024-05-09T05:45:56.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7dbde5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d4114620 con 0x7fc7d4102790 2024-05-09T05:45:56.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7da382700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4109300 0x7fc7d41043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9b81700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7d4102790 0x7fc7d4103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9b81700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7d4102790 0x7fc7d4103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33684/0 (socket says 172.21.15.115:33684) 2024-05-09T05:45:56.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9b81700 1 -- 172.21.15.115:0/3121781379 learned_addr learned my addr 172.21.15.115:0/3121781379 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7da382700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4109300 0x7fc7d41043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37334/0 (socket says 172.21.15.115:37334) 2024-05-09T05:45:56.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9380700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7d41030e0 0x7fc7d4103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9b81700 1 -- 172.21.15.115:0/3121781379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7d41030e0 msgr2=0x7fc7d4103eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7d41030e0 0x7fc7d4103eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9b81700 1 -- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4109300 msgr2=0x7fc7d41043f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4109300 0x7fc7d41043f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9b81700 1 -- 172.21.15.115:0/3121781379 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d41a9150 con 0x7fc7d4102790 2024-05-09T05:45:56.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7da382700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4109300 0x7fc7d41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:56.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.589+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7d4102790 0x7fc7d4103970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000e940 tx=0x7fc7d000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.590+0000 7fc7caffd700 1 -- 172.21.15.115:0/3121781379 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0004220 con 0x7fc7d4102790 2024-05-09T05:45:56.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.590+0000 7fc7caffd700 1 -- 172.21.15.115:0/3121781379 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7d00043c0 con 0x7fc7d4102790 2024-05-09T05:45:56.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.590+0000 7fc7caffd700 1 -- 172.21.15.115:0/3121781379 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0003bf0 con 0x7fc7d4102790 2024-05-09T05:45:56.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.590+0000 7fc7dbde5700 1 -- 172.21.15.115:0/3121781379 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d41a93e0 con 0x7fc7d4102790 2024-05-09T05:45:56.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.590+0000 7fc7dbde5700 1 -- 172.21.15.115:0/3121781379 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d41a9920 con 0x7fc7d4102790 2024-05-09T05:45:56.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.593+0000 7fc7caffd700 1 -- 172.21.15.115:0/3121781379 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc7d000c7e0 con 0x7fc7d4102790 2024-05-09T05:45:56.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.594+0000 7fc7caffd700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc7c00b2a70 0x7fc7c00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.594+0000 7fc7d9380700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc7c00b2a70 0x7fc7c00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.594+0000 7fc7caffd700 1 -- 172.21.15.115:0/3121781379 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc7d00e2aa0 con 0x7fc7d4102790 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.594+0000 7fc7ea7fbb80 1 Processor -- start 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7ea7fbb80 1 -- start start 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7d9380700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc7c00b2a70 0x7fc7c00b4f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4108a60 tx=0x7fc7c40073d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d669804160 0x55d66989d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d66989da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 0x55d669897ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d6697b9ff0 con 0x55d66971a4a0 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d6697587f0 con 0x55d669804160 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d6697b5fb0 con 0x55d6698976a0 2024-05-09T05:45:56.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7d9380700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d66989da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7d9380700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d66989da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37340/0 (socket says 172.21.15.115:37340) 2024-05-09T05:45:56.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7d9380700 1 -- 172.21.15.115:0/3476870704 learned_addr learned my addr 172.21.15.115:0/3476870704 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d669804160 0x55d66989d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7da382700 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 0x55d669897ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7d9380700 1 -- 172.21.15.115:0/3476870704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 msgr2=0x55d669897ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7d9380700 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 0x55d669897ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7d9380700 1 -- 172.21.15.115:0/3476870704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d669804160 msgr2=0x55d66989d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.595+0000 7fc7d9380700 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d669804160 0x55d66989d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.596+0000 7fc7d9380700 1 -- 172.21.15.115:0/3476870704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d669750370 con 0x55d66971a4a0 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.596+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d669804160 0x55d66989d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.596+0000 7fc7da382700 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 0x55d669897ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.596+0000 7fc7d9380700 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d66989da30 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4004500 tx=0x7fc7c4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39a83bcdc174596 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.596+0000 7fc7bffff700 1 -- 172.21.15.115:0/3476870704 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4024070 con 0x55d66971a4a0 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.596+0000 7fc7bffff700 1 -- 172.21.15.115:0/3476870704 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7c4015070 con 0x55d66971a4a0 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.596+0000 7fc7bffff700 1 -- 172.21.15.115:0/3476870704 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4011770 con 0x55d66971a4a0 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.597+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3476870704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 msgr2=0x55d66989da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.597+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d66989da30 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4004500 tx=0x7fc7c4013000 comp rx=0 tx=0).stop 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.597+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3476870704 shutdown_connections 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.597+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d669804160 0x55d66989d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.597+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 0x55d669897ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.597+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3476870704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d66989da30 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.597+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3476870704 >> 172.21.15.115:0/3476870704 conn(0x55d66977a6a0 msgr2=0x55d669892400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.597+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3476870704 shutdown_connections 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.597+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3476870704 wait complete. 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.598+0000 7fc7ea7fbb80 1 -- start start 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.598+0000 7fc7ea7fbb80 1 Processor -- start 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.598+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 0x55d6698a1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.598+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d669804160 0x55d6698a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.598+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 0x55d6698a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d669635ba0 con 0x55d669804160 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d669774150 con 0x55d66971a4a0 2024-05-09T05:45:56.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d6698a41d0 con 0x55d6698976a0 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7ea7fbb80 10 client.0 _ll_register_callbacks cb 0x55d66962b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9380700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d669804160 0x55d6698a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9380700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d669804160 0x55d6698a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37352/0 (socket says 172.21.15.115:37352) 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9380700 1 -- 172.21.15.115:0/3133453821 learned_addr learned my addr 172.21.15.115:0/3133453821 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7da382700 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 0x55d6698a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 0x55d6698a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9b81700 1 -- 172.21.15.115:0/3133453821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 msgr2=0x55d6698a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 0x55d6698a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9b81700 1 -- 172.21.15.115:0/3133453821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d669804160 msgr2=0x55d6698a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d669804160 0x55d6698a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9b81700 1 -- 172.21.15.115:0/3133453821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d669635490 con 0x55d66971a4a0 2024-05-09T05:45:56.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 0x55d6698a1280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00dfe50 tx=0x7fc7d00e03d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2399ff12180a9905 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.599+0000 7fc7bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:45:56.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/3133453821 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00c3060 con 0x55d66971a4a0 2024-05-09T05:45:56.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/3133453821 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc7d00cc2d0 con 0x55d66971a4a0 2024-05-09T05:45:56.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3133453821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7c4015070 con 0x55d66971a4a0 2024-05-09T05:45:56.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/3133453821 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc7d00aa0e0 con 0x55d66971a4a0 2024-05-09T05:45:56.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7ea7fbb80 10 client.26969 fetch_fsmap learned FSMap version 42 2024-05-09T05:45:56.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3133453821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d6698aec80 con 0x55d66971a4a0 2024-05-09T05:45:56.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/3133453821 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7c4015070 con 0x55d66971a4a0 2024-05-09T05:45:56.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/3133453821 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fc7d006f1a0 con 0x55d66971a4a0 2024-05-09T05:45:56.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7ea7fbb80 10 client.26969 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7ea7fbb80 10 client.26969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7ea7fbb80 20 client.26969 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6698b1210 0x55d6698b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7ea7fbb80 4 client.26969 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.600+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3133453821 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6697c79d0 con 0x55d6698b1210 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.601+0000 7fc7d9380700 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6698b1210 0x55d6698b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.601+0000 7fc7d9380700 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6698b1210 0x55d6698b3650 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254c20a6bcb74d2f server_cookie=10821907586a6541 in_seq=0 out_seq=0 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.601+0000 7fc7bd7fa700 10 client.26969 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/3133453821 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6697c79d0 con 0x55d6698b1210 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7bd7fa700 10 client.26969 handle_command_reply: tid=0 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 client.26969 shutdown 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3133453821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 msgr2=0x55d6698a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 0x55d6698a1280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00dfe50 tx=0x7fc7d00e03d0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3133453821 shutdown_connections 2024-05-09T05:45:56.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 0x55d6698a1280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6698b1210 0x55d6698b3650 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6698976a0 0x55d6698a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/3133453821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d669804160 0x55d6698a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7bd7fa700 0 client.26969 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:56.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3133453821 >> 172.21.15.115:0/3133453821 conn(0x55d66977a6a0 msgr2=0x55d6698921c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3133453821 shutdown_connections 2024-05-09T05:45:56.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.602+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/3133453821 wait complete. 2024-05-09T05:45:56.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.603+0000 7fc7ea7fbb80 20 client.26969 trim_cache size 0 max 16384 2024-05-09T05:45:56.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7ea7fbb80 1 Processor -- start 2024-05-09T05:45:56.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7ea7fbb80 1 -- start start 2024-05-09T05:45:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 0x55d66989fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 0x55d6698a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6698976a0 0x55d6698a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41b2540 con 0x55d6698976a0 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41b4110 con 0x55d66971a4a0 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41b2c10 con 0x55d669804160 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7da382700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6698976a0 0x55d6698a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7da382700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6698976a0 0x55d6698a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37364/0 (socket says 172.21.15.115:37364) 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7da382700 1 -- 172.21.15.115:0/4117304541 learned_addr learned my addr 172.21.15.115:0/4117304541 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/4117304541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 0x55d66989fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.627+0000 7fc7d9380700 1 --2- 172.21.15.115:0/4117304541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 0x55d6698a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.628+0000 7fc7da382700 1 -- 172.21.15.115:0/4117304541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 msgr2=0x55d6698a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.628+0000 7fc7da382700 1 --2- 172.21.15.115:0/4117304541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 0x55d6698a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.628+0000 7fc7da382700 1 -- 172.21.15.115:0/4117304541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 msgr2=0x55d66989fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.628+0000 7fc7da382700 1 --2- 172.21.15.115:0/4117304541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 0x55d66989fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.628+0000 7fc7da382700 1 -- 172.21.15.115:0/4117304541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d66989d050 con 0x55d6698976a0 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.628+0000 7fc7da382700 1 --2- 172.21.15.115:0/4117304541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6698976a0 0x55d6698a1240 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00b070 tx=0x7fc7cc0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5893e010c668d95f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.628+0000 7fc7bf7fe700 1 -- 172.21.15.115:0/4117304541 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc029070 con 0x55d6698976a0 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.628+0000 7fc7bf7fe700 1 -- 172.21.15.115:0/4117304541 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7cc0093d0 con 0x55d6698976a0 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.628+0000 7fc7bf7fe700 1 -- 172.21.15.115:0/4117304541 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc005040 con 0x55d6698976a0 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.629+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4117304541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6698976a0 msgr2=0x55d6698a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.629+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4117304541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6698976a0 0x55d6698a1240 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00b070 tx=0x7fc7cc0092e0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.629+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4117304541 shutdown_connections 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.629+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4117304541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d66971a4a0 0x55d66989fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.629+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4117304541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d6698976a0 0x55d6698a1240 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.629+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4117304541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 0x55d6698a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.629+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4117304541 >> 172.21.15.115:0/4117304541 conn(0x55d66977a6a0 msgr2=0x55d66989c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.629+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4117304541 shutdown_connections 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.629+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4117304541 wait complete. 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7ea7fbb80 1 -- start start 2024-05-09T05:45:56.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7ea7fbb80 1 Processor -- start 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d6698a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 0x55d6698a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7ea7fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6698976a0 0x55d6698a5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41b3a40 con 0x55d66971a4a0 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41b3370 con 0x55d6698976a0 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7ea7fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d41b04e0 con 0x55d669804160 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7d9b81700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d6698a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7ea7fbb80 10 client.0 _ll_register_callbacks cb 0x55d669897fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7d9b81700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d6698a4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37380/0 (socket says 172.21.15.115:37380) 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.630+0000 7fc7d9b81700 1 -- 172.21.15.115:0/4221351239 learned_addr learned my addr 172.21.15.115:0/4221351239 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7d9380700 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 0x55d6698a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7da382700 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6698976a0 0x55d6698a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7d9b81700 1 -- 172.21.15.115:0/4221351239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 msgr2=0x55d6698a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 0x55d6698a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7d9b81700 1 -- 172.21.15.115:0/4221351239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6698976a0 msgr2=0x55d6698a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6698976a0 0x55d6698a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7d9b81700 1 -- 172.21.15.115:0/4221351239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66991a9c0 con 0x55d66971a4a0 2024-05-09T05:45:56.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7da382700 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6698976a0 0x55d6698a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:45:56.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7d9b81700 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d6698a4eb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00a5f80 tx=0x7fc7d000cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05a59d43618355d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:56.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/4221351239 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00a99e0 con 0x55d66971a4a0 2024-05-09T05:45:56.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/4221351239 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc7d00a8e50 con 0x55d66971a4a0 2024-05-09T05:45:56.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4221351239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7cc0093d0 con 0x55d66971a4a0 2024-05-09T05:45:56.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/4221351239 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc7d00add90 con 0x55d66971a4a0 2024-05-09T05:45:56.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.631+0000 7fc7ea7fbb80 10 client.17661 fetch_fsmap learned FSMap version 42 2024-05-09T05:45:56.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4221351239 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d66991bb10 con 0x55d66971a4a0 2024-05-09T05:45:56.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/4221351239 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7cc0093d0 con 0x55d66971a4a0 2024-05-09T05:45:56.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/4221351239 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fc7d00a86b0 con 0x55d66971a4a0 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7ea7fbb80 10 client.17661 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7ea7fbb80 10 client.17661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7ea7fbb80 20 client.17661 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d66991c040 0x55d66991e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7ea7fbb80 4 client.17661 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4221351239 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d6697c1a70 con 0x55d66991c040 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.632+0000 7fc7d9380700 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d66991c040 0x55d66991e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7bd7fa700 10 client.17661 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7d9380700 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d66991c040 0x55d66991e460 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3fce91bd26c8c8 server_cookie=d849dad1d056250d in_seq=0 out_seq=0 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7bd7fa700 1 -- 172.21.15.115:0/4221351239 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d6697c1a70 con 0x55d66991c040 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7bd7fa700 10 client.17661 handle_command_reply: tid=0 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7ea7fbb80 1 client.17661 shutdown 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4221351239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 msgr2=0x55d6698a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d6698a4eb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00a5f80 tx=0x7fc7d000cd00 comp rx=0 tx=0).stop 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4221351239 shutdown_connections 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6698976a0 0x55d6698a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d66991c040 0x55d66991e460 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66971a4a0 0x55d6698a4eb0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7bd7fa700 0 client.17661 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7ea7fbb80 1 --2- 172.21.15.115:0/4221351239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d669804160 0x55d6698a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.633+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4221351239 >> 172.21.15.115:0/4221351239 conn(0x55d66977a6a0 msgr2=0x55d6698a11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4221351239 shutdown_connections 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7ea7fbb80 1 -- 172.21.15.115:0/4221351239 wait complete. 2024-05-09T05:45:56.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7ea7fbb80 20 client.17661 trim_cache size 0 max 16384 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": { 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "62cf5198-3119-4120-97f3-fbefadef9297": { 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/", 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "tag": "62cf5198-3119-4120-97f3-fbefadef9297", 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "options": "recursive,force" 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:45:56.635 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 -- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc7c00b2a70 msgr2=0x7fc7c00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc7c00b2a70 0x7fc7c00b4f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4108a60 tx=0x7fc7c40073d0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 -- 172.21.15.115:0/3121781379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7d4102790 msgr2=0x7fc7d4103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7d4102790 0x7fc7d4103970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000e940 tx=0x7fc7d000ec50 comp rx=0 tx=0).stop 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 -- 172.21.15.115:0/3121781379 shutdown_connections 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc7c00b2a70 0x7fc7c00b4f80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7d4109300 0x7fc7d41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7d4102790 0x7fc7d4103970 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 --2- 172.21.15.115:0/3121781379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7d41030e0 0x7fc7d4103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 -- 172.21.15.115:0/3121781379 >> 172.21.15.115:0/3121781379 conn(0x7fc7d40fe520 msgr2=0x7fc7d4107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.634+0000 7fc7dbde5700 1 -- 172.21.15.115:0/3121781379 shutdown_connections 2024-05-09T05:45:56.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:45:56.635+0000 7fc7dbde5700 1 -- 172.21.15.115:0/3121781379 wait complete. 2024-05-09T05:45:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.862+0000 7f8451a4c640 1 -- 172.21.15.115:0/490834387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10b8c0 msgr2=0x7f844c10dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.862+0000 7f8451a4c640 1 --2- 172.21.15.115:0/490834387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10b8c0 0x7f844c10dce0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8434009900 tx=0x7f843402a480 comp rx=0 tx=0).stop 2024-05-09T05:45:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.862+0000 7f8451a4c640 1 -- 172.21.15.115:0/490834387 shutdown_connections 2024-05-09T05:45:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.862+0000 7f8451a4c640 1 --2- 172.21.15.115:0/490834387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c10e220 0x7f844c110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.862+0000 7f8451a4c640 1 --2- 172.21.15.115:0/490834387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10b8c0 0x7f844c10dce0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.862+0000 7f8451a4c640 1 --2- 172.21.15.115:0/490834387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c104e60 0x7f844c105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.862+0000 7f8451a4c640 1 -- 172.21.15.115:0/490834387 >> 172.21.15.115:0/490834387 conn(0x7f844c0781b0 msgr2=0x7f844c078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.862+0000 7f8451a4c640 1 -- 172.21.15.115:0/490834387 shutdown_connections 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.862+0000 7f8451a4c640 1 -- 172.21.15.115:0/490834387 wait complete. 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.863+0000 7f8451a4c640 1 Processor -- start 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.863+0000 7f8451a4c640 1 -- start start 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8451a4c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c104e60 0x7f844c1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8451a4c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c10b8c0 0x7f844c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8451a4c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10e220 0x7f844c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8451a4c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f844c113d30 con 0x7f844c10e220 2024-05-09T05:45:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8451a4c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f844c113bb0 con 0x7f844c104e60 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8451a4c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f844c113eb0 con 0x7f844c10b8c0 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10e220 0x7f844c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10e220 0x7f844c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37406/0 (socket says 172.21.15.115:37406) 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844b7fe640 1 -- 172.21.15.115:0/3339470479 learned_addr learned my addr 172.21.15.115:0/3339470479 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844affd640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c104e60 0x7f844c1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f84427fc640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c10b8c0 0x7f844c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844b7fe640 1 -- 172.21.15.115:0/3339470479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c10b8c0 msgr2=0x7f844c1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844b7fe640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c10b8c0 0x7f844c1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844b7fe640 1 -- 172.21.15.115:0/3339470479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c104e60 msgr2=0x7f844c1a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844b7fe640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c104e60 0x7f844c1a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844b7fe640 1 -- 172.21.15.115:0/3339470479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c19ad80 con 0x7f844c10e220 2024-05-09T05:45:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f844b7fe640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10e220 0x7f844c19a650 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f844400eb10 tx=0x7f844400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8450a4a640 1 -- 172.21.15.115:0/3339470479 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444004b20 con 0x7f844c10e220 2024-05-09T05:45:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8450a4a640 1 -- 172.21.15.115:0/3339470479 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8444004cc0 con 0x7f844c10e220 2024-05-09T05:45:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8451a4c640 1 -- 172.21.15.115:0/3339470479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c19b010 con 0x7f844c10e220 2024-05-09T05:45:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.864+0000 7f8450a4a640 1 -- 172.21.15.115:0/3339470479 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444010690 con 0x7f844c10e220 2024-05-09T05:45:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.865+0000 7f8451a4c640 1 -- 172.21.15.115:0/3339470479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c1a7270 con 0x7f844c10e220 2024-05-09T05:45:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.867+0000 7f84589de740 1 Processor -- start 2024-05-09T05:45:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.868+0000 7f8450a4a640 1 -- 172.21.15.115:0/3339470479 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f8444010830 con 0x7f844c10e220 2024-05-09T05:45:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.868+0000 7f84589de740 1 -- start start 2024-05-09T05:45:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84589de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f79a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84589de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75d92f0 0x5615f79a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84589de740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84589de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5615f790a7c0 con 0x5615f76021b0 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84589de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5615f78a60e0 con 0x5615f75d92f0 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84589de740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5615f7906780 con 0x5615f75e14d0 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f79a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f79a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37416/0 (socket says 172.21.15.115:37416) 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 -- 172.21.15.115:0/3893011360 learned_addr learned my addr 172.21.15.115:0/3893011360 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f844affd640 1 --2- 172.21.15.115:0/3893011360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75d92f0 0x5615f79a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 --2- 172.21.15.115:0/3893011360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 -- 172.21.15.115:0/3893011360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 msgr2=0x5615f79a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 --2- 172.21.15.115:0/3893011360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 -- 172.21.15.115:0/3893011360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75d92f0 msgr2=0x5615f79a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 --2- 172.21.15.115:0/3893011360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75d92f0 0x5615f79a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 -- 172.21.15.115:0/3893011360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f78cf7c0 con 0x5615f76021b0 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f8450a4a640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f84380b2950 0x7f84380b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f84427fc640 1 --2- 172.21.15.115:0/3893011360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f79a5680 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f843402a9a0 tx=0x7f8434005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e70b8665af8fe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.869+0000 7f844affd640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f84380b2950 0x7f84380b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.870+0000 7f8450a4a640 1 -- 172.21.15.115:0/3339470479 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f84440e2ef0 con 0x7f844c10e220 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.870+0000 7f8441ffb640 1 -- 172.21.15.115:0/3893011360 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434047020 con 0x5615f76021b0 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.870+0000 7f8441ffb640 1 -- 172.21.15.115:0/3893011360 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84340044c0 con 0x5615f76021b0 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.870+0000 7f8441ffb640 1 -- 172.21.15.115:0/3893011360 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434035050 con 0x5615f76021b0 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.870+0000 7f84589de740 1 -- 172.21.15.115:0/3893011360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 msgr2=0x5615f79a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.870+0000 7f84589de740 1 --2- 172.21.15.115:0/3893011360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f79a5680 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f843402a9a0 tx=0x7f8434005e50 comp rx=0 tx=0).stop 2024-05-09T05:45:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.870+0000 7f844affd640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f84380b2950 0x7f84380b4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5615f79a6a30 tx=0x7f843c00b390 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.871+0000 7f84589de740 1 -- 172.21.15.115:0/3893011360 shutdown_connections 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.871+0000 7f84589de740 1 --2- 172.21.15.115:0/3893011360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.871+0000 7f84589de740 1 --2- 172.21.15.115:0/3893011360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75d92f0 0x5615f79a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.871+0000 7f84589de740 1 --2- 172.21.15.115:0/3893011360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f79a5680 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.871+0000 7f84589de740 1 -- 172.21.15.115:0/3893011360 >> 172.21.15.115:0/3893011360 conn(0x5615f75f6300 msgr2=0x5615f79a1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.871+0000 7f84589de740 1 -- 172.21.15.115:0/3893011360 shutdown_connections 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.871+0000 7f84589de740 1 -- 172.21.15.115:0/3893011360 wait complete. 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.871+0000 7f84589de740 1 -- start start 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.871+0000 7f84589de740 1 Processor -- start 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f84589de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f84589de740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f84589de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 0x5615f79afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f84589de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5615f79b2750 con 0x5615f75d92f0 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f84589de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5615f78919c0 con 0x5615f76021b0 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f84589de740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5615f79b28d0 con 0x5615f75e14d0 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f84589de740 10 client.0 _ll_register_callbacks cb 0x5615f78be020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37428/0 (socket says 172.21.15.115:37428) 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844affd640 1 -- 172.21.15.115:0/1915639892 learned_addr learned my addr 172.21.15.115:0/1915639892 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844b7fe640 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 0x5615f79afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f84427fc640 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844affd640 1 -- 172.21.15.115:0/1915639892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 msgr2=0x5615f79b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844affd640 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844affd640 1 -- 172.21.15.115:0/1915639892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 msgr2=0x5615f79afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844affd640 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 0x5615f79afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844affd640 1 -- 172.21.15.115:0/1915639892 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f78b81f0 con 0x5615f75d92f0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f844affd640 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79b1820 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f843c003dd0 tx=0x7f843c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923c6969af69d68a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.872+0000 7f842affd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f842affd640 1 -- 172.21.15.115:0/1915639892 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c02c060 con 0x5615f75d92f0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f842affd640 1 -- 172.21.15.115:0/1915639892 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f843c017030 con 0x5615f75d92f0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f84589de740 1 -- 172.21.15.115:0/1915639892 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615f79183d0 con 0x5615f75d92f0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f842affd640 1 -- 172.21.15.115:0/1915639892 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f843c033050 con 0x5615f75d92f0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f84589de740 10 client.17673 fetch_fsmap learned FSMap version 42 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f84589de740 1 -- 172.21.15.115:0/1915639892 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5615f766d790 con 0x5615f75d92f0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f842affd640 1 -- 172.21.15.115:0/1915639892 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5615f79183d0 con 0x5615f75d92f0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f842affd640 1 -- 172.21.15.115:0/1915639892 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f843c01c3f0 con 0x5615f75d92f0 2024-05-09T05:45:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f84589de740 10 client.17673 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f84589de740 10 client.17673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f84589de740 20 client.17673 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f84589de740 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5615f75c0d80 0x5615f79c2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f84589de740 4 client.17673 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.873+0000 7f84589de740 1 -- 172.21.15.115:0/1915639892 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f7912240 con 0x5615f75c0d80 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.874+0000 7f84427fc640 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5615f75c0d80 0x5615f79c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.874+0000 7f842affd640 10 client.17673 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.874+0000 7f84427fc640 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5615f75c0d80 0x5615f79c2040 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcfab6346bb95e4e server_cookie=68c1b606e825e6b9 in_seq=0 out_seq=0 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f842affd640 1 -- 172.21.15.115:0/1915639892 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5615f7912240 con 0x5615f75c0d80 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f842affd640 10 client.17673 handle_command_reply: tid=0 2024-05-09T05:45:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 client.17673 shutdown 2024-05-09T05:45:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 -- 172.21.15.115:0/1915639892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 msgr2=0x5615f79b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79b1820 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f843c003dd0 tx=0x7f843c004730 comp rx=0 tx=0).stop 2024-05-09T05:45:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 -- 172.21.15.115:0/1915639892 shutdown_connections 2024-05-09T05:45:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 0x5615f79afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5615f75c0d80 0x5615f79c2040 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 --2- 172.21.15.115:0/1915639892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79b1820 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 -- 172.21.15.115:0/1915639892 >> 172.21.15.115:0/1915639892 conn(0x5615f75f6300 msgr2=0x5615f79a0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f842affd640 0 client.17673 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 -- 172.21.15.115:0/1915639892 shutdown_connections 2024-05-09T05:45:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 1 -- 172.21.15.115:0/1915639892 wait complete. 2024-05-09T05:45:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.875+0000 7f84589de740 20 client.17673 trim_cache size 0 max 16384 2024-05-09T05:45:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.891+0000 7f84589de740 1 Processor -- start 2024-05-09T05:45:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.891+0000 7f84589de740 1 -- start start 2024-05-09T05:45:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.891+0000 7f84589de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79a6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.891+0000 7f84589de740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.891+0000 7f84589de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 0x5615f79afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.891+0000 7f84589de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1aedd0 con 0x5615f75d92f0 2024-05-09T05:45:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.891+0000 7f84589de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1b09a0 con 0x5615f76021b0 2024-05-09T05:45:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.891+0000 7f84589de740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1af4a0 con 0x5615f75e14d0 2024-05-09T05:45:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79a6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37430/0 (socket says 172.21.15.115:37430) 2024-05-09T05:45:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844affd640 1 -- 172.21.15.115:0/2680629610 learned_addr learned my addr 172.21.15.115:0/2680629610 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f84427fc640 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844b7fe640 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 0x5615f79afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844affd640 1 -- 172.21.15.115:0/2680629610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 msgr2=0x5615f79a99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844affd640 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79a99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844affd640 1 -- 172.21.15.115:0/2680629610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 msgr2=0x5615f79afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844affd640 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 0x5615f79afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844affd640 1 -- 172.21.15.115:0/2680629610 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f76e45f0 con 0x5615f75d92f0 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844b7fe640 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 0x5615f79afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.892+0000 7f844affd640 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79a6420 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f843c018040 tx=0x7f843c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58c3afabcba44f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.893+0000 7f84417fa640 1 -- 172.21.15.115:0/2680629610 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c03c070 con 0x5615f75d92f0 2024-05-09T05:45:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.893+0000 7f84417fa640 1 -- 172.21.15.115:0/2680629610 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f843c005040 con 0x5615f75d92f0 2024-05-09T05:45:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.893+0000 7f84417fa640 1 -- 172.21.15.115:0/2680629610 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c004a00 con 0x5615f75d92f0 2024-05-09T05:45:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.893+0000 7f84589de740 1 -- 172.21.15.115:0/2680629610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 msgr2=0x5615f79a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.893+0000 7f84589de740 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79a6420 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f843c018040 tx=0x7f843c005fb0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.894+0000 7f84589de740 1 -- 172.21.15.115:0/2680629610 shutdown_connections 2024-05-09T05:45:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.894+0000 7f84589de740 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f76021b0 0x5615f79afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.894+0000 7f84589de740 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75e14d0 0x5615f79a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.894+0000 7f84589de740 1 --2- 172.21.15.115:0/2680629610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f75d92f0 0x5615f79a6420 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.894+0000 7f84589de740 1 -- 172.21.15.115:0/2680629610 >> 172.21.15.115:0/2680629610 conn(0x5615f75f6300 msgr2=0x5615f79af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.894+0000 7f84589de740 1 -- 172.21.15.115:0/2680629610 shutdown_connections 2024-05-09T05:45:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.894+0000 7f84589de740 1 -- 172.21.15.115:0/2680629610 wait complete. 2024-05-09T05:45:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.894+0000 7f84589de740 1 -- start start 2024-05-09T05:45:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.894+0000 7f84589de740 1 Processor -- start 2024-05-09T05:45:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f84589de740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75d92f0 0x5615f79b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f84589de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75e14d0 0x5615f79b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f84589de740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f7a22430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f84589de740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1b02d0 con 0x5615f76021b0 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f84589de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1afc00 con 0x5615f75e14d0 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f84589de740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1acdb0 con 0x5615f75d92f0 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f84589de740 10 client.0 _ll_register_callbacks cb 0x7f843c003c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f7a22430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75d92f0 0x5615f79b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f7a22430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37446/0 (socket says 172.21.15.115:37446) 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844b7fe640 1 -- 172.21.15.115:0/2395711135 learned_addr learned my addr 172.21.15.115:0/2395711135 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f84427fc640 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75e14d0 0x5615f79b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844affd640 1 -- 172.21.15.115:0/2395711135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75e14d0 msgr2=0x5615f79b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844affd640 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75e14d0 0x5615f79b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844affd640 1 -- 172.21.15.115:0/2395711135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 msgr2=0x5615f7a22430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844affd640 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f7a22430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844affd640 1 -- 172.21.15.115:0/2395711135 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f78ceed0 con 0x5615f75d92f0 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844b7fe640 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f7a22430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f844affd640 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75d92f0 0x5615f79b5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f843c003dd0 tx=0x7f843c01c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d85a2fe0b0ad002 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.895+0000 7f842affd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f842affd640 1 -- 172.21.15.115:0/2395711135 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c03c070 con 0x5615f75d92f0 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f84589de740 1 -- 172.21.15.115:0/2395711135 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843c005040 con 0x5615f75d92f0 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f842affd640 1 -- 172.21.15.115:0/2395711135 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f843c0051f0 con 0x5615f75d92f0 2024-05-09T05:45:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f842affd640 1 -- 172.21.15.115:0/2395711135 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f843c01cdc0 con 0x5615f75d92f0 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f84589de740 10 client.27097 fetch_fsmap learned FSMap version 42 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f84589de740 1 -- 172.21.15.115:0/2395711135 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5615f7a29ac0 con 0x5615f75d92f0 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f842affd640 1 -- 172.21.15.115:0/2395711135 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f843c005040 con 0x5615f75d92f0 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f842affd640 1 -- 172.21.15.115:0/2395711135 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f843c04a030 con 0x5615f75d92f0 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f84589de740 10 client.27097 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f84589de740 10 client.27097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f84589de740 20 client.27097 populate_metadata read hostname 'smithi115' 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f84589de740 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5615f75961c0 0x5615f7a2c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f84589de740 4 client.27097 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.896+0000 7f84589de740 1 -- 172.21.15.115:0/2395711135 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615f788d350 con 0x5615f75961c0 2024-05-09T05:45:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.897+0000 7f84427fc640 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5615f75961c0 0x5615f7a2c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:45:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.897+0000 7f842affd640 10 client.27097 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.897+0000 7f84427fc640 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5615f75961c0 0x5615f7a2c160 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b671c492747c922e server_cookie=7273caf9d65857e9 in_seq=0 out_seq=0 2024-05-09T05:45:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f842affd640 1 -- 172.21.15.115:0/2395711135 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5615f788d350 con 0x5615f75961c0 2024-05-09T05:45:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f842affd640 10 client.27097 handle_command_reply: tid=0 2024-05-09T05:45:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 client.27097 shutdown 2024-05-09T05:45:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 -- 172.21.15.115:0/2395711135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75d92f0 msgr2=0x5615f79b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75d92f0 0x5615f79b5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f843c003dd0 tx=0x7f843c01c620 comp rx=0 tx=0).stop 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 -- 172.21.15.115:0/2395711135 shutdown_connections 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5615f76021b0 0x5615f7a22430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615f75e14d0 0x5615f79b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5615f75961c0 0x5615f7a2c160 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 --2- 172.21.15.115:0/2395711135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5615f75d92f0 0x5615f79b5050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f842affd640 0 client.27097 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 -- 172.21.15.115:0/2395711135 >> 172.21.15.115:0/2395711135 conn(0x5615f75f6300 msgr2=0x5615f79a0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 -- 172.21.15.115:0/2395711135 shutdown_connections 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 1 -- 172.21.15.115:0/2395711135 wait complete. 2024-05-09T05:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.923+0000 7f84589de740 20 client.27097 trim_cache size 0 max 16384 2024-05-09T05:45:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:45:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:45:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:45:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 -- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f84380b2950 msgr2=0x7f84380b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f84380b2950 0x7f84380b4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5615f79a6a30 tx=0x7f843c00b390 comp rx=0 tx=0).stop 2024-05-09T05:45:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 -- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10e220 msgr2=0x7f844c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:45:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10e220 0x7f844c19a650 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f844400eb10 tx=0x7f844400c2d0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 -- 172.21.15.115:0/3339470479 shutdown_connections 2024-05-09T05:45:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f84380b2950 0x7f84380b4e60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f844c10e220 0x7f844c19a650 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c10b8c0 0x7f844c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 --2- 172.21.15.115:0/3339470479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c104e60 0x7f844c1a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:45:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 -- 172.21.15.115:0/3339470479 >> 172.21.15.115:0/3339470479 conn(0x7f844c0781b0 msgr2=0x7f844c1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:45:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 -- 172.21.15.115:0/3339470479 shutdown_connections 2024-05-09T05:45:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:45:56.924+0000 7f8451a4c640 1 -- 172.21.15.115:0/3339470479 wait complete. 2024-05-09T05:45:57.471 INFO:tasks.cephfs.filesystem:scrub status for tag:62cf5198-3119-4120-97f3-fbefadef9297 - {'path': '/', 'tag': '62cf5198-3119-4120-97f3-fbefadef9297', 'options': 'recursive,force'} 2024-05-09T05:45:57.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:57 smithi195 ceph-mon[33844]: pgmap v238: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:45:57.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:57 smithi115 ceph-mon[30349]: pgmap v238: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:45:57.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:57 smithi161 ceph-mon[33805]: pgmap v238: 129 pgs: 129 active+clean; 5.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 20 op/s 2024-05-09T05:45:59.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:45:59 smithi195 ceph-mon[33844]: pgmap v239: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 34 MiB/s wr, 61 op/s 2024-05-09T05:45:59.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:45:59 smithi115 ceph-mon[30349]: pgmap v239: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 34 MiB/s wr, 61 op/s 2024-05-09T05:45:59.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:45:59 smithi161 ceph-mon[33805]: pgmap v239: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 34 MiB/s wr, 61 op/s 2024-05-09T05:46:01.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:01 smithi195 ceph-mon[33844]: pgmap v240: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 49 op/s 2024-05-09T05:46:01.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:01 smithi115 ceph-mon[30349]: pgmap v240: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 49 op/s 2024-05-09T05:46:01.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:01 smithi161 ceph-mon[33805]: pgmap v240: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 49 op/s 2024-05-09T05:46:03.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:03 smithi195 ceph-mon[33844]: pgmap v241: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 49 op/s 2024-05-09T05:46:03.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:03 smithi115 ceph-mon[30349]: pgmap v241: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 49 op/s 2024-05-09T05:46:03.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:03 smithi161 ceph-mon[33805]: pgmap v241: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 49 op/s 2024-05-09T05:46:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:05 smithi195 ceph-mon[33844]: pgmap v242: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 34 MiB/s wr, 75 op/s 2024-05-09T05:46:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:05 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:46:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:05 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:46:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:05 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:46:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:05 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:46:05.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:05 smithi195 ceph-mon[33844]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T05:46:05.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:05 smithi115 ceph-mon[30349]: pgmap v242: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 34 MiB/s wr, 75 op/s 2024-05-09T05:46:05.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:05 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:46:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:05 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:46:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:05 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:46:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:05 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:46:05.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:05 smithi115 ceph-mon[30349]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T05:46:05.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:05 smithi161 ceph-mon[33805]: pgmap v242: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 34 MiB/s wr, 75 op/s 2024-05-09T05:46:05.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:05 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:46:05.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:05 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:46:05.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:05 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:46:05.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:05 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:46:05.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:05 smithi161 ceph-mon[33805]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T05:46:07.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:07 smithi195 ceph-mon[33844]: pgmap v244: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 28 MiB/s wr, 79 op/s 2024-05-09T05:46:07.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:07 smithi115 ceph-mon[30349]: pgmap v244: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 28 MiB/s wr, 79 op/s 2024-05-09T05:46:07.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:07 smithi161 ceph-mon[33805]: pgmap v244: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 28 MiB/s wr, 79 op/s 2024-05-09T05:46:09.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:09 smithi195 ceph-mon[33844]: pgmap v245: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 42 op/s 2024-05-09T05:46:09.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:09 smithi195 ceph-mon[33844]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T05:46:09.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:09 smithi115 ceph-mon[30349]: pgmap v245: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 42 op/s 2024-05-09T05:46:09.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:09 smithi115 ceph-mon[30349]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T05:46:09.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:09 smithi161 ceph-mon[33805]: pgmap v245: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 42 op/s 2024-05-09T05:46:09.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:09 smithi161 ceph-mon[33805]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T05:46:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.032+0000 7f3212500640 1 -- 172.21.15.115:0/921074559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c108bc0 msgr2=0x7f320c102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.032+0000 7f3212500640 1 --2- 172.21.15.115:0/921074559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c108bc0 0x7f320c102760 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009900 tx=0x7f31fc02a3f0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.032+0000 7f3212500640 1 -- 172.21.15.115:0/921074559 shutdown_connections 2024-05-09T05:46:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.032+0000 7f3212500640 1 --2- 172.21.15.115:0/921074559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c103310 0x7f320c10f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.032+0000 7f3212500640 1 --2- 172.21.15.115:0/921074559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320c109060 0x7f320c102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.032+0000 7f3212500640 1 --2- 172.21.15.115:0/921074559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c108bc0 0x7f320c102760 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.032+0000 7f3212500640 1 -- 172.21.15.115:0/921074559 >> 172.21.15.115:0/921074559 conn(0x7f320c0781b0 msgr2=0x7f320c078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.032+0000 7f3212500640 1 -- 172.21.15.115:0/921074559 shutdown_connections 2024-05-09T05:46:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.032+0000 7f3212500640 1 -- 172.21.15.115:0/921074559 wait complete. 2024-05-09T05:46:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.033+0000 7f3212500640 1 Processor -- start 2024-05-09T05:46:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.033+0000 7f3212500640 1 -- start start 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.033+0000 7f3212500640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c103310 0x7f320c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.033+0000 7f3212500640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c108bc0 0x7f320c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.033+0000 7f3212500640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320c109060 0x7f320c19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.033+0000 7f3212500640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f320c112c60 con 0x7f320c103310 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.033+0000 7f3212500640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f320c112ae0 con 0x7f320c109060 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.033+0000 7f3212500640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f320c112de0 con 0x7f320c108bc0 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c103310 0x7f320c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c103310 0x7f320c1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45586/0 (socket says 172.21.15.115:45586) 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320bfff640 1 -- 172.21.15.115:0/350550557 learned_addr learned my addr 172.21.15.115:0/350550557 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320b7fe640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c108bc0 0x7f320c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f3210a9d640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320c109060 0x7f320c19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320bfff640 1 -- 172.21.15.115:0/350550557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c108bc0 msgr2=0x7f320c199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320bfff640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c108bc0 0x7f320c199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320bfff640 1 -- 172.21.15.115:0/350550557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320c109060 msgr2=0x7f320c19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320bfff640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320c109060 0x7f320c19dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320bfff640 1 -- 172.21.15.115:0/350550557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c19e480 con 0x7f320c103310 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f320bfff640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c103310 0x7f320c1996a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009230 tx=0x7f31fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f32097fa640 1 -- 172.21.15.115:0/350550557 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc03f030 con 0x7f320c103310 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f32097fa640 1 -- 172.21.15.115:0/350550557 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31fc003c90 con 0x7f320c103310 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f3212500640 1 -- 172.21.15.115:0/350550557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320c19e710 con 0x7f320c103310 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f32097fa640 1 -- 172.21.15.115:0/350550557 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc035070 con 0x7f320c103310 2024-05-09T05:46:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.034+0000 7f3212500640 1 -- 172.21.15.115:0/350550557 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320c19ec00 con 0x7f320c103310 2024-05-09T05:46:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.037+0000 7f32097fa640 1 -- 172.21.15.115:0/350550557 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f31fc042030 con 0x7f320c103310 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.037+0000 7f32193bb740 1 Processor -- start 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.037+0000 7f32193bb740 1 -- start start 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f32193bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 0x55c50c9b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f32193bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f32193bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 0x55c50c9b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f32193bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c50c9187c0 con 0x55c50c5e72f0 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f32193bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c50c8b40e0 con 0x55c50c5ef4d0 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f32193bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c50c914780 con 0x55c50c6101b0 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f320bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f320bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45596/0 (socket says 172.21.15.115:45596) 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f320bfff640 1 -- 172.21.15.115:0/3777970494 learned_addr learned my addr 172.21.15.115:0/3777970494 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f3210a9d640 1 --2- 172.21.15.115:0/3777970494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 0x55c50c9b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f320b7fe640 1 --2- 172.21.15.115:0/3777970494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 0x55c50c9b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f32097fa640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31f40b2950 0x7f31f40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f320bfff640 1 -- 172.21.15.115:0/3777970494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 msgr2=0x55c50c9b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f320bfff640 1 --2- 172.21.15.115:0/3777970494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 0x55c50c9b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f320bfff640 1 -- 172.21.15.115:0/3777970494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 msgr2=0x55c50c9b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f320bfff640 1 --2- 172.21.15.115:0/3777970494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 0x55c50c9b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f320bfff640 1 -- 172.21.15.115:0/3777970494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50c8dd7c0 con 0x55c50c5e72f0 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f3210a9d640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31f40b2950 0x7f31f40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.038+0000 7f32097fa640 1 -- 172.21.15.115:0/350550557 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f31fc1030c0 con 0x7f320c103310 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.039+0000 7f320bfff640 1 --2- 172.21.15.115:0/3777970494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9b3bc0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f31fc1003f0 tx=0x7f31fc04b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661cb3bbb44697af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.039+0000 7f3210a9d640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31f40b2950 0x7f31f40b4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55c50c9b07f0 tx=0x7f3204004a40 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.039+0000 7f31f27fc640 1 -- 172.21.15.115:0/3777970494 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0c9de0 con 0x55c50c5e72f0 2024-05-09T05:46:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.039+0000 7f31f27fc640 1 -- 172.21.15.115:0/3777970494 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31fc0c9b20 con 0x55c50c5e72f0 2024-05-09T05:46:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.039+0000 7f31f27fc640 1 -- 172.21.15.115:0/3777970494 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0044e0 con 0x55c50c5e72f0 2024-05-09T05:46:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 -- 172.21.15.115:0/3777970494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 msgr2=0x55c50c9b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 --2- 172.21.15.115:0/3777970494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9b3bc0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f31fc1003f0 tx=0x7f31fc04b000 comp rx=0 tx=0).stop 2024-05-09T05:46:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 -- 172.21.15.115:0/3777970494 shutdown_connections 2024-05-09T05:46:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 --2- 172.21.15.115:0/3777970494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 0x55c50c9b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 --2- 172.21.15.115:0/3777970494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9b3bc0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 --2- 172.21.15.115:0/3777970494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 0x55c50c9b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 -- 172.21.15.115:0/3777970494 >> 172.21.15.115:0/3777970494 conn(0x55c50c604300 msgr2=0x55c50c9afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 -- 172.21.15.115:0/3777970494 shutdown_connections 2024-05-09T05:46:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 -- 172.21.15.115:0/3777970494 wait complete. 2024-05-09T05:46:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.040+0000 7f32193bb740 1 -- start start 2024-05-09T05:46:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f32193bb740 1 Processor -- start 2024-05-09T05:46:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f32193bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9bdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f32193bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 0x55c50c9be220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f32193bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 0x55c50c9c3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f32193bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c50c9c0570 con 0x55c50c5e72f0 2024-05-09T05:46:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f32193bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c50c89f9c0 con 0x55c50c5ef4d0 2024-05-09T05:46:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f32193bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c50c9c06f0 con 0x55c50c6101b0 2024-05-09T05:46:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f32193bb740 10 client.0 _ll_register_callbacks cb 0x55c50c8cc020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f320bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9bdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f320bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9bdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45598/0 (socket says 172.21.15.115:45598) 2024-05-09T05:46:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f320bfff640 1 -- 172.21.15.115:0/2207868777 learned_addr learned my addr 172.21.15.115:0/2207868777 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f3210a9d640 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 0x55c50c9c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.041+0000 7f320b7fe640 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 0x55c50c9be220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f320bfff640 1 -- 172.21.15.115:0/2207868777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 msgr2=0x55c50c9c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f320bfff640 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 0x55c50c9c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f320bfff640 1 -- 172.21.15.115:0/2207868777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 msgr2=0x55c50c9be220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f320bfff640 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 0x55c50c9be220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f320bfff640 1 -- 172.21.15.115:0/2207868777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50c8c61f0 con 0x55c50c5e72f0 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f3210a9d640 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 0x55c50c9c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f320b7fe640 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 0x55c50c9be220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f320bfff640 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9bdce0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0c6090 tx=0x7f31fc03b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75140497162458fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f31eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f31eb7fe640 1 -- 172.21.15.115:0/2207868777 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0a7ff0 con 0x55c50c5e72f0 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f31eb7fe640 1 -- 172.21.15.115:0/2207868777 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f31fc106550 con 0x55c50c5e72f0 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f32193bb740 1 -- 172.21.15.115:0/2207868777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31fc0c9b20 con 0x55c50c5e72f0 2024-05-09T05:46:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f31eb7fe640 1 -- 172.21.15.115:0/2207868777 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f31fc049020 con 0x55c50c5e72f0 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f32193bb740 10 client.17691 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.042+0000 7f32193bb740 1 -- 172.21.15.115:0/2207868777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c50c67b790 con 0x55c50c5e72f0 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.043+0000 7f31eb7fe640 1 -- 172.21.15.115:0/2207868777 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31fc0ca710 con 0x55c50c5e72f0 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.043+0000 7f31eb7fe640 1 -- 172.21.15.115:0/2207868777 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f31fc100700 con 0x55c50c5e72f0 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.043+0000 7f32193bb740 10 client.17691 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.043+0000 7f32193bb740 10 client.17691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.043+0000 7f32193bb740 20 client.17691 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.043+0000 7f32193bb740 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c50c5ced80 0x55c50c9cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.043+0000 7f32193bb740 4 client.17691 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.043+0000 7f32193bb740 1 -- 172.21.15.115:0/2207868777 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f31fc0c9b20 con 0x55c50c5ced80 2024-05-09T05:46:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.043+0000 7f320b7fe640 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c50c5ced80 0x55c50c9cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.044+0000 7f31eb7fe640 10 client.17691 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.044+0000 7f320b7fe640 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c50c5ced80 0x55c50c9cfaa0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17beadb1f000a576 server_cookie=31834a5eff02d28c in_seq=0 out_seq=0 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.044+0000 7f31eb7fe640 1 -- 172.21.15.115:0/2207868777 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f31fc0c9b20 con 0x55c50c5ced80 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.044+0000 7f31eb7fe640 10 client.17691 handle_command_reply: tid=0 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 client.17691 shutdown 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 -- 172.21.15.115:0/2207868777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 msgr2=0x55c50c9bdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9bdce0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0c6090 tx=0x7f31fc03b510 comp rx=0 tx=0).stop 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 -- 172.21.15.115:0/2207868777 shutdown_connections 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c50c5ced80 0x55c50c9cfaa0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c6101b0 0x55c50c9c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f31eb7fe640 0 client.17691 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ef4d0 0x55c50c9be220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 --2- 172.21.15.115:0/2207868777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9bdce0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 -- 172.21.15.115:0/2207868777 >> 172.21.15.115:0/2207868777 conn(0x55c50c604300 msgr2=0x55c50c9ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 -- 172.21.15.115:0/2207868777 shutdown_connections 2024-05-09T05:46:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 1 -- 172.21.15.115:0/2207868777 wait complete. 2024-05-09T05:46:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.045+0000 7f32193bb740 20 client.17691 trim_cache size 0 max 16384 2024-05-09T05:46:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.062+0000 7f32193bb740 1 Processor -- start 2024-05-09T05:46:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.062+0000 7f32193bb740 1 -- start start 2024-05-09T05:46:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.062+0000 7f32193bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ced80 0x55c50c9b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.062+0000 7f32193bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5e72f0 0x55c50c9aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.062+0000 7f32193bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5ef4d0 0x55c50c9d5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.062+0000 7f32193bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1aeeb0 con 0x55c50c5ef4d0 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.062+0000 7f32193bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1b0a80 con 0x55c50c5e72f0 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.062+0000 7f32193bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1af580 con 0x55c50c5ced80 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.062+0000 7f3210a9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5ef4d0 0x55c50c9d5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f3210a9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5ef4d0 0x55c50c9d5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45606/0 (socket says 172.21.15.115:45606) 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f3210a9d640 1 -- 172.21.15.115:0/2262128664 learned_addr learned my addr 172.21.15.115:0/2262128664 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f320bfff640 1 --2- 172.21.15.115:0/2262128664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ced80 0x55c50c9b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f320b7fe640 1 --2- 172.21.15.115:0/2262128664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5e72f0 0x55c50c9aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f3210a9d640 1 -- 172.21.15.115:0/2262128664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ced80 msgr2=0x55c50c9b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f3210a9d640 1 --2- 172.21.15.115:0/2262128664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ced80 0x55c50c9b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f3210a9d640 1 -- 172.21.15.115:0/2262128664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5e72f0 msgr2=0x55c50c9aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f3210a9d640 1 --2- 172.21.15.115:0/2262128664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5e72f0 0x55c50c9aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f3210a9d640 1 -- 172.21.15.115:0/2262128664 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50c6f25f0 con 0x55c50c5ef4d0 2024-05-09T05:46:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f3210a9d640 1 --2- 172.21.15.115:0/2262128664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5ef4d0 0x55c50c9d5270 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f320400e8d0 tx=0x7f3204016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cb89e448689345 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f31f1ffb640 1 -- 172.21.15.115:0/2262128664 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320402c070 con 0x55c50c5ef4d0 2024-05-09T05:46:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f31f1ffb640 1 -- 172.21.15.115:0/2262128664 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f320400ee50 con 0x55c50c5ef4d0 2024-05-09T05:46:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.063+0000 7f31f1ffb640 1 -- 172.21.15.115:0/2262128664 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204027e20 con 0x55c50c5ef4d0 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.064+0000 7f32193bb740 1 -- 172.21.15.115:0/2262128664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5ef4d0 msgr2=0x55c50c9d5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.064+0000 7f32193bb740 1 --2- 172.21.15.115:0/2262128664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5ef4d0 0x55c50c9d5270 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f320400e8d0 tx=0x7f3204016000 comp rx=0 tx=0).stop 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.064+0000 7f32193bb740 1 -- 172.21.15.115:0/2262128664 shutdown_connections 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.064+0000 7f32193bb740 1 --2- 172.21.15.115:0/2262128664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5ef4d0 0x55c50c9d5270 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.064+0000 7f32193bb740 1 --2- 172.21.15.115:0/2262128664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5e72f0 0x55c50c9aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.064+0000 7f32193bb740 1 --2- 172.21.15.115:0/2262128664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ced80 0x55c50c9b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.064+0000 7f32193bb740 1 -- 172.21.15.115:0/2262128664 >> 172.21.15.115:0/2262128664 conn(0x55c50c604300 msgr2=0x55c50c9ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.064+0000 7f32193bb740 1 -- 172.21.15.115:0/2262128664 shutdown_connections 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.064+0000 7f32193bb740 1 -- 172.21.15.115:0/2262128664 wait complete. 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f32193bb740 1 -- start start 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f32193bb740 1 Processor -- start 2024-05-09T05:46:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f32193bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ced80 0x55c50c9c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f32193bb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f32193bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ef4d0 0x55c50c9c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f32193bb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1b03b0 con 0x55c50c5e72f0 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f32193bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1afce0 con 0x55c50c5ced80 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f32193bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1aceb0 con 0x55c50c5ef4d0 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f32193bb740 10 client.0 _ll_register_callbacks cb 0x7f320000c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f320b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f320b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9c4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45608/0 (socket says 172.21.15.115:45608) 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.065+0000 7f320b7fe640 1 -- 172.21.15.115:0/1592410874 learned_addr learned my addr 172.21.15.115:0/1592410874 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f3210a9d640 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ef4d0 0x55c50c9c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f320bfff640 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ced80 0x55c50c9c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f320b7fe640 1 -- 172.21.15.115:0/1592410874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ef4d0 msgr2=0x55c50c9c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f320b7fe640 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ef4d0 0x55c50c9c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f320b7fe640 1 -- 172.21.15.115:0/1592410874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ced80 msgr2=0x55c50c9c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f320b7fe640 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ced80 0x55c50c9c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f320b7fe640 1 -- 172.21.15.115:0/1592410874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50c8dced0 con 0x55c50c5e72f0 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f320bfff640 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ced80 0x55c50c9c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f320b7fe640 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9c4fd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f320c10e9d0 tx=0x7f32000039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a43aff69a9d5387 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f31eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f31eb7fe640 1 -- 172.21.15.115:0/1592410874 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32000044b0 con 0x55c50c5e72f0 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f31eb7fe640 1 -- 172.21.15.115:0/1592410874 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f320001c630 con 0x55c50c5e72f0 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f32193bb740 1 -- 172.21.15.115:0/1592410874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f320400ee50 con 0x55c50c5e72f0 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f31eb7fe640 1 -- 172.21.15.115:0/1592410874 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f320002b050 con 0x55c50c5e72f0 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f32193bb740 10 client.17697 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.066+0000 7f32193bb740 1 -- 172.21.15.115:0/1592410874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c50ca37cd0 con 0x55c50c5e72f0 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f31eb7fe640 1 -- 172.21.15.115:0/1592410874 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f320400ee50 con 0x55c50c5e72f0 2024-05-09T05:46:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f31eb7fe640 1 -- 172.21.15.115:0/1592410874 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f3200010bc0 con 0x55c50c5e72f0 2024-05-09T05:46:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f32193bb740 10 client.17697 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f32193bb740 10 client.17697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f32193bb740 20 client.17697 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f32193bb740 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c50c6101b0 0x55c50ca3a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f32193bb740 4 client.17697 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:46:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f32193bb740 1 -- 172.21.15.115:0/1592410874 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c50c9263d0 con 0x55c50c6101b0 2024-05-09T05:46:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f320bfff640 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c50c6101b0 0x55c50ca3a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.067+0000 7f31eb7fe640 10 client.17697 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.068+0000 7f320bfff640 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c50c6101b0 0x55c50ca3a510 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2c9f4f4a6bd4478 server_cookie=7e4c0fb56e0834c2 in_seq=0 out_seq=0 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.076+0000 7f31eb7fe640 1 -- 172.21.15.115:0/1592410874 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c50c9263d0 con 0x55c50c6101b0 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.076+0000 7f31eb7fe640 10 client.17697 handle_command_reply: tid=0 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 client.17697 shutdown 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 -- 172.21.15.115:0/1592410874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 msgr2=0x55c50c9c4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9c4fd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f320c10e9d0 tx=0x7f32000039c0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 -- 172.21.15.115:0/1592410874 shutdown_connections 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c50c6101b0 0x55c50ca3a510 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c50c5ef4d0 0x55c50c9c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c50c5e72f0 0x55c50c9c4fd0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f31eb7fe640 0 client.17697 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 --2- 172.21.15.115:0/1592410874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c50c5ced80 0x55c50c9c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 -- 172.21.15.115:0/1592410874 >> 172.21.15.115:0/1592410874 conn(0x55c50c604300 msgr2=0x55c50c9af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 -- 172.21.15.115:0/1592410874 shutdown_connections 2024-05-09T05:46:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 1 -- 172.21.15.115:0/1592410874 wait complete. 2024-05-09T05:46:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.077+0000 7f32193bb740 20 client.17697 trim_cache size 0 max 16384 2024-05-09T05:46:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:46:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:46:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:46:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 -- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31f40b2950 msgr2=0x7f31f40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31f40b2950 0x7f31f40b4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55c50c9b07f0 tx=0x7f3204004a40 comp rx=0 tx=0).stop 2024-05-09T05:46:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 -- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c103310 msgr2=0x7f320c1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c103310 0x7f320c1996a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009230 tx=0x7f31fc0042f0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 -- 172.21.15.115:0/350550557 shutdown_connections 2024-05-09T05:46:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31f40b2950 0x7f31f40b4e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320c109060 0x7f320c19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f320c108bc0 0x7f320c199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 --2- 172.21.15.115:0/350550557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c103310 0x7f320c1996a0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 -- 172.21.15.115:0/350550557 >> 172.21.15.115:0/350550557 conn(0x7f320c0781b0 msgr2=0x7f320c104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 -- 172.21.15.115:0/350550557 shutdown_connections 2024-05-09T05:46:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:10.078+0000 7f3212500640 1 -- 172.21.15.115:0/350550557 wait complete. 2024-05-09T05:46:11.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:11 smithi195 ceph-mon[33844]: pgmap v247: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 36 MiB/s wr, 53 op/s 2024-05-09T05:46:11.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:11 smithi115 ceph-mon[30349]: pgmap v247: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 36 MiB/s wr, 53 op/s 2024-05-09T05:46:11.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:11 smithi161 ceph-mon[33805]: pgmap v247: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 36 MiB/s wr, 53 op/s 2024-05-09T05:46:13.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:13 smithi195 ceph-mon[33844]: pgmap v248: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-05-09T05:46:13.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:13 smithi115 ceph-mon[30349]: pgmap v248: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-05-09T05:46:13.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:13 smithi161 ceph-mon[33805]: pgmap v248: 129 pgs: 129 active+clean; 6.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-05-09T05:46:15.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:15 smithi195 ceph-mon[33844]: pgmap v249: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-05-09T05:46:15.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:15 smithi115 ceph-mon[30349]: pgmap v249: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-05-09T05:46:15.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:15 smithi161 ceph-mon[33805]: pgmap v249: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-05-09T05:46:17.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:17 smithi195 ceph-mon[33844]: pgmap v250: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:46:17.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:17 smithi115 ceph-mon[30349]: pgmap v250: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:46:17.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:17 smithi161 ceph-mon[33805]: pgmap v250: 129 pgs: 129 active+clean; 6.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:46:19.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:19 smithi195 ceph-mon[33844]: pgmap v251: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:46:19.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:19 smithi115 ceph-mon[30349]: pgmap v251: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:46:19.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:19 smithi161 ceph-mon[33805]: pgmap v251: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:46:21.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:21 smithi195 ceph-mon[33844]: pgmap v252: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-05-09T05:46:21.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:21 smithi115 ceph-mon[30349]: pgmap v252: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-05-09T05:46:21.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:21 smithi161 ceph-mon[33805]: pgmap v252: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-05-09T05:46:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.201+0000 7f94ccfdc640 1 -- 172.21.15.115:0/2542244538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c81032f0 msgr2=0x7f94c810f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.201+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/2542244538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c81032f0 0x7f94c810f6c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009ca0 tx=0x7f94c002a8c0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.201+0000 7f94ccfdc640 1 -- 172.21.15.115:0/2542244538 shutdown_connections 2024-05-09T05:46:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.201+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/2542244538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c81032f0 0x7f94c810f6c0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.201+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/2542244538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8109040 0x7f94c8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.201+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/2542244538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94c8108ba0 0x7f94c8102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.201+0000 7f94ccfdc640 1 -- 172.21.15.115:0/2542244538 >> 172.21.15.115:0/2542244538 conn(0x7f94c80782e0 msgr2=0x7f94c8078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.201+0000 7f94ccfdc640 1 -- 172.21.15.115:0/2542244538 shutdown_connections 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.201+0000 7f94ccfdc640 1 -- 172.21.15.115:0/2542244538 wait complete. 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94ccfdc640 1 Processor -- start 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94ccfdc640 1 -- start start 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94ccfdc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94c81032f0 0x7f94c81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94ccfdc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8108ba0 0x7f94c8199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94ccfdc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8109040 0x7f94c819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94ccfdc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8112cb0 con 0x7f94c8108ba0 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94ccfdc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8112b30 con 0x7f94c8109040 2024-05-09T05:46:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94ccfdc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8112e30 con 0x7f94c81032f0 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8108ba0 0x7f94c8199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8108ba0 0x7f94c8199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47960/0 (socket says 172.21.15.115:47960) 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94c5d9b640 1 -- 172.21.15.115:0/4128311503 learned_addr learned my addr 172.21.15.115:0/4128311503 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94c659c640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94c81032f0 0x7f94c81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.202+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8109040 0x7f94c819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94c5d9b640 1 -- 172.21.15.115:0/4128311503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94c81032f0 msgr2=0x7f94c81996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94c81032f0 0x7f94c81996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94c5d9b640 1 -- 172.21.15.115:0/4128311503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8109040 msgr2=0x7f94c819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8109040 0x7f94c819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94c5d9b640 1 -- 172.21.15.115:0/4128311503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c819e510 con 0x7f94c8108ba0 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8109040 0x7f94c819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8108ba0 0x7f94c8199c10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f94bc005fd0 tx=0x7f94bc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94b77fe640 1 -- 172.21.15.115:0/4128311503 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc017070 con 0x7f94c8108ba0 2024-05-09T05:46:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94b77fe640 1 -- 172.21.15.115:0/4128311503 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94bc009c30 con 0x7f94c8108ba0 2024-05-09T05:46:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94b77fe640 1 -- 172.21.15.115:0/4128311503 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc003c90 con 0x7f94c8108ba0 2024-05-09T05:46:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94ccfdc640 1 -- 172.21.15.115:0/4128311503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c819e800 con 0x7f94c8108ba0 2024-05-09T05:46:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.203+0000 7f94ccfdc640 1 -- 172.21.15.115:0/4128311503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81a6180 con 0x7f94c8108ba0 2024-05-09T05:46:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.206+0000 7f94d3f69740 1 Processor -- start 2024-05-09T05:46:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.206+0000 7f94b77fe640 1 -- 172.21.15.115:0/4128311503 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f94bc003e30 con 0x7f94c8108ba0 2024-05-09T05:46:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.206+0000 7f94d3f69740 1 -- start start 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.206+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b773d1b0 0x55f4b7ae0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.206+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7ae0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.206+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b771c4d0 0x55f4b7ae1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.206+0000 7f94d3f69740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f4b7a457c0 con 0x55f4b771c4d0 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.206+0000 7f94d3f69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4b79e10e0 con 0x55f4b773d1b0 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.206+0000 7f94d3f69740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f4b7a41780 con 0x55f4b77142f0 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b771c4d0 0x55f4b7ae1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b771c4d0 0x55f4b7ae1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47974/0 (socket says 172.21.15.115:47974) 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c5d9b640 1 -- 172.21.15.115:0/1332919990 learned_addr learned my addr 172.21.15.115:0/1332919990 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c659c640 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7ae0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b773d1b0 0x55f4b7ae0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c5d9b640 1 -- 172.21.15.115:0/1332919990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 msgr2=0x55f4b7ae0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7ae0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c5d9b640 1 -- 172.21.15.115:0/1332919990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b773d1b0 msgr2=0x55f4b7ae0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b773d1b0 0x55f4b7ae0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c5d9b640 1 -- 172.21.15.115:0/1332919990 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4b7a0a7c0 con 0x55f4b771c4d0 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c659c640 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7ae0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94b77fe640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94b00b2950 0x7f94b00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94b00b2950 0x7f94b00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b771c4d0 0x55f4b7ae1100 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0df8f0 tx=0x7f94bc0ea690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e249e0fd37a3df9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.207+0000 7f94b77fe640 1 -- 172.21.15.115:0/4128311503 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f94bc0e25c0 con 0x7f94c8108ba0 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.208+0000 7f94b4ff9640 1 -- 172.21.15.115:0/1332919990 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0a53d0 con 0x55f4b771c4d0 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.208+0000 7f94b4ff9640 1 -- 172.21.15.115:0/1332919990 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94bc0910d0 con 0x55f4b771c4d0 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.208+0000 7f94b4ff9640 1 -- 172.21.15.115:0/1332919990 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0a8c80 con 0x55f4b771c4d0 2024-05-09T05:46:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.208+0000 7f94d3f69740 1 -- 172.21.15.115:0/1332919990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b771c4d0 msgr2=0x55f4b7ae1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.208+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94b00b2950 0x7f94b00b4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f94c0005e30 tx=0x7f94c0005d80 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.208+0000 7f94d3f69740 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b771c4d0 0x55f4b7ae1100 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0df8f0 tx=0x7f94bc0ea690 comp rx=0 tx=0).stop 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.209+0000 7f94d3f69740 1 -- 172.21.15.115:0/1332919990 shutdown_connections 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.209+0000 7f94d3f69740 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b771c4d0 0x55f4b7ae1100 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.209+0000 7f94d3f69740 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7ae0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.209+0000 7f94d3f69740 1 --2- 172.21.15.115:0/1332919990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b773d1b0 0x55f4b7ae0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.209+0000 7f94d3f69740 1 -- 172.21.15.115:0/1332919990 >> 172.21.15.115:0/1332919990 conn(0x55f4b7731300 msgr2=0x55f4b7adce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.209+0000 7f94d3f69740 1 -- 172.21.15.115:0/1332919990 shutdown_connections 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.209+0000 7f94d3f69740 1 -- 172.21.15.115:0/1332919990 wait complete. 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.209+0000 7f94d3f69740 1 -- start start 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.209+0000 7f94d3f69740 1 Processor -- start 2024-05-09T05:46:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7aec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 0x55f4b7aecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7aeac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94d3f69740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f4b7aef9c0 con 0x55f4b773d1b0 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94d3f69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4b79cc9c0 con 0x55f4b771c4d0 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94d3f69740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f4b7aefb40 con 0x55f4b77142f0 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94d3f69740 10 client.0 _ll_register_callbacks cb 0x55f4b79f9020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7aeac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7aeac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47984/0 (socket says 172.21.15.115:47984) 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c6d9d640 1 -- 172.21.15.115:0/3754972120 learned_addr learned my addr 172.21.15.115:0/3754972120 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 0x55f4b7aecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c659c640 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7aec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c6d9d640 1 -- 172.21.15.115:0/3754972120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 msgr2=0x55f4b7aec7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7aec7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c6d9d640 1 -- 172.21.15.115:0/3754972120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 msgr2=0x55f4b7aecd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 0x55f4b7aecd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c6d9d640 1 -- 172.21.15.115:0/3754972120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4b79f31f0 con 0x55f4b773d1b0 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7aeac60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f94c0035000 tx=0x7f94c002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4a6fe362c78d13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.210+0000 7f94adffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94adffb640 1 -- 172.21.15.115:0/3754972120 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0004040 con 0x55f4b773d1b0 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94adffb640 1 -- 172.21.15.115:0/3754972120 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f94c00041e0 con 0x55f4b773d1b0 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94d3f69740 1 -- 172.21.15.115:0/3754972120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94bc0910d0 con 0x55f4b773d1b0 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94adffb640 1 -- 172.21.15.115:0/3754972120 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f94c00043e0 con 0x55f4b773d1b0 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94d3f69740 10 client.17709 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94d3f69740 1 -- 172.21.15.115:0/3754972120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4b77a8790 con 0x55f4b773d1b0 2024-05-09T05:46:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94adffb640 1 -- 172.21.15.115:0/3754972120 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94bc0910d0 con 0x55f4b773d1b0 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94adffb640 1 -- 172.21.15.115:0/3754972120 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f94c0051070 con 0x55f4b773d1b0 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94d3f69740 10 client.17709 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94d3f69740 10 client.17709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94d3f69740 20 client.17709 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4b76fbd80 0x55f4b7aff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94d3f69740 4 client.17709 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.211+0000 7f94d3f69740 1 -- 172.21.15.115:0/3754972120 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4b7a533d0 con 0x55f4b76fbd80 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.212+0000 7f94c659c640 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4b76fbd80 0x55f4b7aff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.212+0000 7f94adffb640 10 client.17709 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.212+0000 7f94c659c640 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4b76fbd80 0x55f4b7aff300 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b045848b86ee74d3 server_cookie=44ff3a0b0c7bb58d in_seq=0 out_seq=0 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94adffb640 1 -- 172.21.15.115:0/3754972120 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4b7a533d0 con 0x55f4b76fbd80 2024-05-09T05:46:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94adffb640 10 client.17709 handle_command_reply: tid=0 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 client.17709 shutdown 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 -- 172.21.15.115:0/3754972120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 msgr2=0x55f4b7aeac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7aeac60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f94c0035000 tx=0x7f94c002a8c0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 -- 172.21.15.115:0/3754972120 shutdown_connections 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7aeac60 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 0x55f4b7aecd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4b76fbd80 0x55f4b7aff300 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3754972120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7aec7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 -- 172.21.15.115:0/3754972120 >> 172.21.15.115:0/3754972120 conn(0x55f4b7731300 msgr2=0x55f4b7adb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94adffb640 0 client.17709 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 -- 172.21.15.115:0/3754972120 shutdown_connections 2024-05-09T05:46:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 1 -- 172.21.15.115:0/3754972120 wait complete. 2024-05-09T05:46:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.213+0000 7f94d3f69740 20 client.17709 trim_cache size 0 max 16384 2024-05-09T05:46:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.234+0000 7f94d3f69740 1 Processor -- start 2024-05-09T05:46:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.234+0000 7f94d3f69740 1 -- start start 2024-05-09T05:46:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.234+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7adce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.234+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 0x55f4b7b04b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.234+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b05040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.234+0000 7f94d3f69740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81af090 con 0x55f4b773d1b0 2024-05-09T05:46:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.234+0000 7f94d3f69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81b0c60 con 0x55f4b771c4d0 2024-05-09T05:46:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.234+0000 7f94d3f69740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81af760 con 0x55f4b77142f0 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b05040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 0x55f4b7b04b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7adce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b05040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47994/0 (socket says 172.21.15.115:47994) 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c6d9d640 1 -- 172.21.15.115:0/2645457485 learned_addr learned my addr 172.21.15.115:0/2645457485 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7adce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46062/0 (socket says 172.21.15.115:46062) 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c6d9d640 1 -- 172.21.15.115:0/2645457485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 msgr2=0x55f4b7adce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/2645457485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7adce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c6d9d640 1 -- 172.21.15.115:0/2645457485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 msgr2=0x55f4b7b04b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/2645457485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 0x55f4b7b04b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c6d9d640 1 -- 172.21.15.115:0/2645457485 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4b781f5f0 con 0x55f4b773d1b0 2024-05-09T05:46:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.235+0000 7f94c6d9d640 1 --2- 172.21.15.115:0/2645457485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b05040 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f94c0030dd0 tx=0x7f94c0009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b46133b1e47faf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.236+0000 7f94affff640 1 -- 172.21.15.115:0/2645457485 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0003d10 con 0x55f4b773d1b0 2024-05-09T05:46:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.236+0000 7f94affff640 1 -- 172.21.15.115:0/2645457485 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94c0036040 con 0x55f4b773d1b0 2024-05-09T05:46:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.236+0000 7f94affff640 1 -- 172.21.15.115:0/2645457485 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0053440 con 0x55f4b773d1b0 2024-05-09T05:46:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 -- 172.21.15.115:0/2645457485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 msgr2=0x55f4b7b05040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 --2- 172.21.15.115:0/2645457485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b05040 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f94c0030dd0 tx=0x7f94c0009540 comp rx=0 tx=0).stop 2024-05-09T05:46:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 -- 172.21.15.115:0/2645457485 shutdown_connections 2024-05-09T05:46:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 --2- 172.21.15.115:0/2645457485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b05040 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 --2- 172.21.15.115:0/2645457485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b771c4d0 0x55f4b7b04b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 --2- 172.21.15.115:0/2645457485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b77142f0 0x55f4b7adce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 -- 172.21.15.115:0/2645457485 >> 172.21.15.115:0/2645457485 conn(0x55f4b7731300 msgr2=0x55f4b7adc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 -- 172.21.15.115:0/2645457485 shutdown_connections 2024-05-09T05:46:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 -- 172.21.15.115:0/2645457485 wait complete. 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 -- start start 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.237+0000 7f94d3f69740 1 Processor -- start 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b77142f0 0x55f4b7af4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b771c4d0 0x55f4b7af5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94d3f69740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b67aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94d3f69740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81b0590 con 0x55f4b773d1b0 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94d3f69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81afec0 con 0x55f4b77142f0 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94d3f69740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81ad090 con 0x55f4b771c4d0 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94d3f69740 10 client.0 _ll_register_callbacks cb 0x7f94c0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b67aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b67aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48002/0 (socket says 172.21.15.115:48002) 2024-05-09T05:46:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b77142f0 0x55f4b7af4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b771c4d0 0x55f4b7af5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c6d9d640 1 -- 172.21.15.115:0/3855756718 learned_addr learned my addr 172.21.15.115:0/3855756718 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c659c640 1 -- 172.21.15.115:0/3855756718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b771c4d0 msgr2=0x55f4b7af5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c659c640 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b771c4d0 0x55f4b7af5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c659c640 1 -- 172.21.15.115:0/3855756718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 msgr2=0x55f4b7b67aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c659c640 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b67aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c659c640 1 -- 172.21.15.115:0/3855756718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4b7a09ed0 con 0x55f4b77142f0 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b771c4d0 0x55f4b7af5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94c659c640 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b77142f0 0x55f4b7af4b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55f4b7affaa0 tx=0x7f94b8020610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=667c242c2666aa16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.238+0000 7f94adffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94adffb640 1 -- 172.21.15.115:0/3855756718 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8005220 con 0x55f4b77142f0 2024-05-09T05:46:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94d3f69740 1 -- 172.21.15.115:0/3855756718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94c0036040 con 0x55f4b77142f0 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94adffb640 1 -- 172.21.15.115:0/3855756718 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f94b80053c0 con 0x55f4b77142f0 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94adffb640 1 -- 172.21.15.115:0/3855756718 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f94b8031050 con 0x55f4b77142f0 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94d3f69740 10 client.27026 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94d3f69740 1 -- 172.21.15.115:0/3855756718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4b7b69100 con 0x55f4b77142f0 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94adffb640 1 -- 172.21.15.115:0/3855756718 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94c0036040 con 0x55f4b77142f0 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94adffb640 1 -- 172.21.15.115:0/3855756718 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f94b8004960 con 0x55f4b77142f0 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94d3f69740 10 client.27026 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94d3f69740 10 client.27026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94d3f69740 20 client.27026 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4b76d11c0 0x55f4b7b6b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94d3f69740 4 client.27026 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:46:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.239+0000 7f94d3f69740 1 -- 172.21.15.115:0/3855756718 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4b7a4d240 con 0x55f4b76d11c0 2024-05-09T05:46:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.240+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4b76d11c0 0x55f4b7b6b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.240+0000 7f94adffb640 10 client.27026 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.240+0000 7f94c5d9b640 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4b76d11c0 0x55f4b7b6b840 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97be66d64be65c9 server_cookie=cac65e999fc3455b in_seq=0 out_seq=0 2024-05-09T05:46:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94adffb640 1 -- 172.21.15.115:0/3855756718 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f4b7a4d240 con 0x55f4b76d11c0 2024-05-09T05:46:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94adffb640 10 client.27026 handle_command_reply: tid=0 2024-05-09T05:46:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 client.27026 shutdown 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 -- 172.21.15.115:0/3855756718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b77142f0 msgr2=0x55f4b7af4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b77142f0 0x55f4b7af4b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55f4b7affaa0 tx=0x7f94b8020610 comp rx=0 tx=0).stop 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 -- 172.21.15.115:0/3855756718 shutdown_connections 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f4b773d1b0 0x55f4b7b67aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f4b76d11c0 0x55f4b7b6b840 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f4b771c4d0 0x55f4b7af5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 --2- 172.21.15.115:0/3855756718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4b77142f0 0x55f4b7af4b30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94adffb640 0 client.27026 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 -- 172.21.15.115:0/3855756718 >> 172.21.15.115:0/3855756718 conn(0x55f4b7731300 msgr2=0x55f4b7ae42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 -- 172.21.15.115:0/3855756718 shutdown_connections 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 1 -- 172.21.15.115:0/3855756718 wait complete. 2024-05-09T05:46:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.249+0000 7f94d3f69740 20 client.27026 trim_cache size 0 max 16384 2024-05-09T05:46:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:46:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:46:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:46:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 -- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94b00b2950 msgr2=0x7f94b00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94b00b2950 0x7f94b00b4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f94c0005e30 tx=0x7f94c0005d80 comp rx=0 tx=0).stop 2024-05-09T05:46:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 -- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8108ba0 msgr2=0x7f94c8199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8108ba0 0x7f94c8199c10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f94bc005fd0 tx=0x7f94bc00dde0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 -- 172.21.15.115:0/4128311503 shutdown_connections 2024-05-09T05:46:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94b00b2950 0x7f94b00b4e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94c8109040 0x7f94c819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8108ba0 0x7f94c8199c10 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 --2- 172.21.15.115:0/4128311503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94c81032f0 0x7f94c81996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 -- 172.21.15.115:0/4128311503 >> 172.21.15.115:0/4128311503 conn(0x7f94c80782e0 msgr2=0x7f94c80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 -- 172.21.15.115:0/4128311503 shutdown_connections 2024-05-09T05:46:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:23.250+0000 7f94ccfdc640 1 -- 172.21.15.115:0/4128311503 wait complete. 2024-05-09T05:46:23.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:23 smithi195 ceph-mon[33844]: pgmap v253: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:23.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:23 smithi115 ceph-mon[30349]: pgmap v253: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:23.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:23 smithi161 ceph-mon[33805]: pgmap v253: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:25.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:25 smithi195 ceph-mon[33844]: pgmap v254: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:46:25.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:25 smithi115 ceph-mon[30349]: pgmap v254: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:46:25.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:25 smithi161 ceph-mon[33805]: pgmap v254: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:46:27.472 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:46:27.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:27 smithi115 ceph-mon[30349]: pgmap v255: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:46:27.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:27 smithi161 ceph-mon[33805]: pgmap v255: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:46:27.737 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:46:27.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:27 smithi195 ceph-mon[33844]: pgmap v255: 129 pgs: 129 active+clean; 6.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:46:29.511 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:29 smithi115 ceph-mon[30349]: pgmap v256: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:46:29.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.644+0000 7f4ee9de5700 1 -- 172.21.15.115:0/3544375082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 msgr2=0x7f4ee4110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:29.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.644+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/3544375082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 0x7f4ee4110150 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00b0d0 tx=0x7f4edc00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.644+0000 7f4ee9de5700 1 -- 172.21.15.115:0/3544375082 shutdown_connections 2024-05-09T05:46:29.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.644+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/3544375082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee4103ca0 0x7f4ee41040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.644+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/3544375082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 0x7f4ee4110150 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.644+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/3544375082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee4107e10 0x7f4ee4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.644+0000 7f4ee9de5700 1 -- 172.21.15.115:0/3544375082 >> 172.21.15.115:0/3544375082 conn(0x7f4ee40fe6f0 msgr2=0x7f4ee4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:29.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.644+0000 7f4ee9de5700 1 -- 172.21.15.115:0/3544375082 shutdown_connections 2024-05-09T05:46:29.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.644+0000 7f4ee9de5700 1 -- 172.21.15.115:0/3544375082 wait complete. 2024-05-09T05:46:29.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.645+0000 7f4ee9de5700 1 Processor -- start 2024-05-09T05:46:29.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.645+0000 7f4ee9de5700 1 -- start start 2024-05-09T05:46:29.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee9de5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee4103ca0 0x7f4ee419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee9de5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee4107e10 0x7f4ee419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee9de5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 0x7f4ee419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee9de5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4113280 con 0x7f4ee4108760 2024-05-09T05:46:29.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee9de5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4113100 con 0x7f4ee4103ca0 2024-05-09T05:46:29.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee9de5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4113400 con 0x7f4ee4107e10 2024-05-09T05:46:29.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 0x7f4ee419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 0x7f4ee419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48022/0 (socket says 172.21.15.115:48022) 2024-05-09T05:46:29.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee3fff700 1 -- 172.21.15.115:0/1653164562 learned_addr learned my addr 172.21.15.115:0/1653164562 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:29.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee4103ca0 0x7f4ee419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee3fff700 1 -- 172.21.15.115:0/1653164562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee4107e10 msgr2=0x7f4ee419c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:29.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee3fff700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee4107e10 0x7f4ee419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee3fff700 1 -- 172.21.15.115:0/1653164562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee4103ca0 msgr2=0x7f4ee419bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:29.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee3fff700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee4103ca0 0x7f4ee419bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee3fff700 1 -- 172.21.15.115:0/1653164562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee41a6f90 con 0x7f4ee4108760 2024-05-09T05:46:29.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.646+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee4103ca0 0x7f4ee419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:46:29.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.647+0000 7f4ee3fff700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 0x7f4ee419c6e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4edc007d10 tx=0x7f4edc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:29.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.647+0000 7f4ee0ff9700 1 -- 172.21.15.115:0/1653164562 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc01f070 con 0x7f4ee4108760 2024-05-09T05:46:29.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.647+0000 7f4ee9de5700 1 -- 172.21.15.115:0/1653164562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41a7220 con 0x7f4ee4108760 2024-05-09T05:46:29.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.647+0000 7f4ee0ff9700 1 -- 172.21.15.115:0/1653164562 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4edc003e10 con 0x7f4ee4108760 2024-05-09T05:46:29.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.647+0000 7f4ee0ff9700 1 -- 172.21.15.115:0/1653164562 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc01b820 con 0x7f4ee4108760 2024-05-09T05:46:29.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.647+0000 7f4ee9de5700 1 -- 172.21.15.115:0/1653164562 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41a76d0 con 0x7f4ee4108760 2024-05-09T05:46:29.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.650+0000 7f4ee0ff9700 1 -- 172.21.15.115:0/1653164562 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4edc022070 con 0x7f4ee4108760 2024-05-09T05:46:29.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ee0ff9700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ecc0b29f0 0x7f4ecc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ecc0b29f0 0x7f4ecc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ee0ff9700 1 -- 172.21.15.115:0/1653164562 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f4edc00f020 con 0x7f4ee4108760 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ef87fbb80 1 Processor -- start 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ef87fbb80 1 -- start start 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ecc0b29f0 0x7f4ecc0b4f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4005fd0 tx=0x7f4ed4005ee0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1d7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1d7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b1d1400 0x55cf9b1d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cf9b0f40a0 con 0x55cf9b0ced00 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cf9b0999a0 con 0x55cf9b0544a0 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.651+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cf9b0f0060 con 0x55cf9b1d1400 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1d7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1d7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48030/0 (socket says 172.21.15.115:48030) 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee37fe700 1 -- 172.21.15.115:0/535191093 learned_addr learned my addr 172.21.15.115:0/535191093 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee3fff700 1 --2- 172.21.15.115:0/535191093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b1d1400 0x55cf9b1d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee2ffd700 1 --2- 172.21.15.115:0/535191093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1d7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee37fe700 1 -- 172.21.15.115:0/535191093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b1d1400 msgr2=0x55cf9b1d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/535191093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b1d1400 0x55cf9b1d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee37fe700 1 -- 172.21.15.115:0/535191093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 msgr2=0x55cf9b1d7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/535191093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1d7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee37fe700 1 -- 172.21.15.115:0/535191093 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9b08a500 con 0x55cf9b0ced00 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/535191093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1d7240 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400edd0 tx=0x7f4ed4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f957ed9f1206dedc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ec9ffb700 1 -- 172.21.15.115:0/535191093 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400ee00 con 0x55cf9b0ced00 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ec9ffb700 1 -- 172.21.15.115:0/535191093 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ed4017030 con 0x55cf9b0ced00 2024-05-09T05:46:29.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.652+0000 7f4ec9ffb700 1 -- 172.21.15.115:0/535191093 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4005440 con 0x55cf9b0ced00 2024-05-09T05:46:29.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.653+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/535191093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 msgr2=0x55cf9b1d7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:29.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.653+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/535191093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1d7240 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400edd0 tx=0x7f4ed4009f90 comp rx=0 tx=0).stop 2024-05-09T05:46:29.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.654+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/535191093 shutdown_connections 2024-05-09T05:46:29.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.654+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/535191093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b1d1400 0x55cf9b1d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.654+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/535191093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1d7240 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.654+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/535191093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1d7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.654+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/535191093 >> 172.21.15.115:0/535191093 conn(0x55cf9b057f70 msgr2=0x55cf9b1cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.654+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/535191093 shutdown_connections 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.654+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/535191093 wait complete. 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.654+0000 7f4ef87fbb80 1 -- start start 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.654+0000 7f4ef87fbb80 1 Processor -- start 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0544a0 0x55cf9b1db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1db5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 0x55cf9b1dbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55cf9b1d2160 con 0x55cf9b0ced00 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cf9b08eb20 con 0x55cf9b1d1400 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cf9b1de020 con 0x55cf9b0544a0 2024-05-09T05:46:29.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0544a0 0x55cf9b1db080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1db5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ef87fbb80 10 client.0 _ll_register_callbacks cb 0x55cf9af656c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0544a0 0x55cf9b1db080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46098/0 (socket says 172.21.15.115:46098) 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee37fe700 1 -- 172.21.15.115:0/1612687958 learned_addr learned my addr 172.21.15.115:0/1612687958 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee3fff700 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 0x55cf9b1dbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee2ffd700 1 -- 172.21.15.115:0/1612687958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0544a0 msgr2=0x55cf9b1db080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee2ffd700 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0544a0 0x55cf9b1db080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee2ffd700 1 -- 172.21.15.115:0/1612687958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 msgr2=0x55cf9b1dbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee2ffd700 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 0x55cf9b1dbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.655+0000 7f4ee2ffd700 1 -- 172.21.15.115:0/1612687958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9af6f490 con 0x55cf9b0ced00 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.656+0000 7f4ee2ffd700 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1db5c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x55cf9b1cdc90 tx=0x7f4ed800e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52332237ed4efaf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.656+0000 7f4ec2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.656+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/1612687958 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8004220 con 0x55cf9b0ced00 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.656+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1612687958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed4017030 con 0x55cf9b0ced00 2024-05-09T05:46:29.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.656+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/1612687958 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f4ed80043c0 con 0x55cf9b0ced00 2024-05-09T05:46:29.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.656+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/1612687958 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4ed8022050 con 0x55cf9b0ced00 2024-05-09T05:46:29.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.656+0000 7f4ef87fbb80 10 client.17727 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:29.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.656+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1612687958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cf9b1e8ac0 con 0x55cf9b0ced00 2024-05-09T05:46:29.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.656+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/1612687958 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ed4017030 con 0x55cf9b0ced00 2024-05-09T05:46:29.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.657+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/1612687958 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f4ed8010650 con 0x55cf9b0ced00 2024-05-09T05:46:29.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.657+0000 7f4ef87fbb80 10 client.17727 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:29.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.657+0000 7f4ef87fbb80 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:29.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.657+0000 7f4ef87fbb80 20 client.17727 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:29.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.657+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf9b1eb0a0 0x55cf9b1ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.657+0000 7f4ef87fbb80 4 client.17727 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.657+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1612687958 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf9b101a80 con 0x55cf9b1eb0a0 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.657+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf9b1eb0a0 0x55cf9b1ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.657+0000 7f4ec2ffd700 10 client.17727 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.659+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf9b1eb0a0 0x55cf9b1ed500 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd045021584423da server_cookie=d63df2b9a120825e in_seq=0 out_seq=0 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.659+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/1612687958 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cf9b101a80 con 0x55cf9b1eb0a0 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.659+0000 7f4ec2ffd700 10 client.17727 handle_command_reply: tid=0 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.659+0000 7f4ef87fbb80 1 client.17727 shutdown 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1612687958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 msgr2=0x55cf9b1db5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1db5c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x55cf9b1cdc90 tx=0x7f4ed800e930 comp rx=0 tx=0).stop 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1612687958 shutdown_connections 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 0x55cf9b1dbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf9b1eb0a0 0x55cf9b1ed500 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0544a0 0x55cf9b1db080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1612687958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0ced00 0x55cf9b1db5c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ec2ffd700 0 client.17727 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:29.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1612687958 >> 172.21.15.115:0/1612687958 conn(0x55cf9b057f70 msgr2=0x55cf9b1c5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:29.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1612687958 shutdown_connections 2024-05-09T05:46:29.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1612687958 wait complete. 2024-05-09T05:46:29.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.660+0000 7f4ef87fbb80 20 client.17727 trim_cache size 0 max 16384 2024-05-09T05:46:29.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:29 smithi161 ceph-mon[33805]: pgmap v256: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:46:29.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.686+0000 7f4ef87fbb80 1 Processor -- start 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.686+0000 7f4ef87fbb80 1 -- start start 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0544a0 0x55cf9b1c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 0x55cf9b1e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41ae2b0 con 0x55cf9b0544a0 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41afe80 con 0x55cf9b1d1400 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41ae980 con 0x55cf9b0ced00 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0544a0 0x55cf9b1c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0544a0 0x55cf9b1c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48050/0 (socket says 172.21.15.115:48050) 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee37fe700 1 -- 172.21.15.115:0/1751064524 learned_addr learned my addr 172.21.15.115:0/1751064524 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee2ffd700 1 --2- 172.21.15.115:0/1751064524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 0x55cf9b1e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee3fff700 1 --2- 172.21.15.115:0/1751064524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee37fe700 1 -- 172.21.15.115:0/1751064524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 msgr2=0x55cf9b1e6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:29.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/1751064524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 0x55cf9b1e6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee37fe700 1 -- 172.21.15.115:0/1751064524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 msgr2=0x55cf9b1f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:29.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/1751064524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee37fe700 1 -- 172.21.15.115:0/1751064524 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9b1c89d0 con 0x55cf9b0544a0 2024-05-09T05:46:29.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.687+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/1751064524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0544a0 0x55cf9b1c6a20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4013040 tx=0x7f4ed400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bf88fe147b5553 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:29.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.688+0000 7f4ec97fa700 1 -- 172.21.15.115:0/1751064524 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4004020 con 0x55cf9b0544a0 2024-05-09T05:46:29.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.688+0000 7f4ec97fa700 1 -- 172.21.15.115:0/1751064524 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ed4017070 con 0x55cf9b0544a0 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.688+0000 7f4ec97fa700 1 -- 172.21.15.115:0/1751064524 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40056b0 con 0x55cf9b0544a0 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1751064524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0544a0 msgr2=0x55cf9b1c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1751064524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0544a0 0x55cf9b1c6a20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4013040 tx=0x7f4ed400e990 comp rx=0 tx=0).stop 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1751064524 shutdown_connections 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1751064524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1751064524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b0544a0 0x55cf9b1c6a20 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/1751064524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 0x55cf9b1e6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1751064524 >> 172.21.15.115:0/1751064524 conn(0x55cf9b057f70 msgr2=0x55cf9b1d6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1751064524 shutdown_connections 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/1751064524 wait complete. 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 -- start start 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.689+0000 7f4ef87fbb80 1 Processor -- start 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 0x55cf9b1f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ef87fbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41af7b0 con 0x55cf9b1d1400 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41af0e0 con 0x55cf9b0544a0 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ef87fbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41ac270 con 0x55cf9b0ced00 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ef87fbb80 10 client.0 _ll_register_callbacks cb 0x55cf9af656c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1e34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:37912/0 (socket says 172.21.15.115:37912) 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee37fe700 1 -- 172.21.15.115:0/3333056207 learned_addr learned my addr 172.21.15.115:0/3333056207 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48052/0 (socket says 172.21.15.115:48052) 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee2ffd700 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 0x55cf9b1f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee37fe700 1 -- 172.21.15.115:0/3333056207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 msgr2=0x55cf9b1f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 0x55cf9b1f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee37fe700 1 -- 172.21.15.115:0/3333056207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b1d1400 msgr2=0x55cf9b1f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee37fe700 1 -- 172.21.15.115:0/3333056207 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9b1f2b30 con 0x55cf9b0544a0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee2ffd700 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 0x55cf9b1f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.690+0000 7f4ee3fff700 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.691+0000 7f4ee37fe700 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1e34a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400ee90 tx=0x7f4ed4011d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39370367b0a9893 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.691+0000 7f4ec2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.691+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/3333056207 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed403f060 con 0x55cf9b0544a0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.691+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/3333056207 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed4017070 con 0x55cf9b0544a0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.691+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/3333056207 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f4ed4037040 con 0x55cf9b0544a0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/3333056207 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4ed4046050 con 0x55cf9b0544a0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ef87fbb80 10 client.27041 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/3333056207 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cf9b2554f0 con 0x55cf9b0544a0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/3333056207 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ed4017070 con 0x55cf9b0544a0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/3333056207 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f4ed400f040 con 0x55cf9b0544a0 2024-05-09T05:46:29.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ef87fbb80 10 client.27041 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:29.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ef87fbb80 10 client.27041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:29.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ef87fbb80 20 client.27041 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:29.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf9b2559b0 0x55cf9b257dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:29.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ef87fbb80 4 client.27041 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:46:29.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/3333056207 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf9b0fbb20 con 0x55cf9b2559b0 2024-05-09T05:46:29.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.692+0000 7f4ee2ffd700 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf9b2559b0 0x55cf9b257dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ec2ffd700 10 client.27041 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ee2ffd700 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf9b2559b0 0x55cf9b257dd0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c1ed585882db9a5 server_cookie=d1aa2563ac5420d0 in_seq=0 out_seq=0 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ec2ffd700 1 -- 172.21.15.115:0/3333056207 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cf9b0fbb20 con 0x55cf9b2559b0 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ec2ffd700 10 client.27041 handle_command_reply: tid=0 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 client.27041 shutdown 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/3333056207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 msgr2=0x55cf9b1e34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1e34a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400ee90 tx=0x7f4ed4011d20 comp rx=0 tx=0).stop 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/3333056207 shutdown_connections 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cf9b0544a0 0x55cf9b1e34a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55cf9b2559b0 0x55cf9b257dd0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55cf9b1d1400 0x55cf9b1f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 --2- 172.21.15.115:0/3333056207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cf9b0ced00 0x55cf9b1f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/3333056207 >> 172.21.15.115:0/3333056207 conn(0x55cf9b057f70 msgr2=0x55cf9b1d6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:29.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ec2ffd700 0 client.27041 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:29.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.693+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/3333056207 shutdown_connections 2024-05-09T05:46:29.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.694+0000 7f4ef87fbb80 1 -- 172.21.15.115:0/3333056207 wait complete. 2024-05-09T05:46:29.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.694+0000 7f4ef87fbb80 20 client.27041 trim_cache size 0 max 16384 2024-05-09T05:46:29.692 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:46:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:46:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:46:29.692 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.694+0000 7f4ee9de5700 1 -- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ecc0b29f0 msgr2=0x7f4ecc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.694+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ecc0b29f0 0x7f4ecc0b4f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4005fd0 tx=0x7f4ed4005ee0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.694+0000 7f4ee9de5700 1 -- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 msgr2=0x7f4ee419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.694+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 0x7f4ee419c6e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4edc007d10 tx=0x7f4edc007df0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.694+0000 7f4ee9de5700 1 -- 172.21.15.115:0/1653164562 shutdown_connections 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.694+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ecc0b29f0 0x7f4ecc0b4f00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.694+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee4103ca0 0x7f4ee419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.695+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ee4108760 0x7f4ee419c6e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.695+0000 7f4ee9de5700 1 --2- 172.21.15.115:0/1653164562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ee4107e10 0x7f4ee419c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.695+0000 7f4ee9de5700 1 -- 172.21.15.115:0/1653164562 >> 172.21.15.115:0/1653164562 conn(0x7f4ee40fe6f0 msgr2=0x7f4ee4107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.695+0000 7f4ee9de5700 1 -- 172.21.15.115:0/1653164562 shutdown_connections 2024-05-09T05:46:29.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:29.695+0000 7f4ee9de5700 1 -- 172.21.15.115:0/1653164562 wait complete. 2024-05-09T05:46:29.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:29 smithi195 ceph-mon[33844]: pgmap v256: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:46:30.411 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:46:30.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:46:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:46:31.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:31 smithi115 ceph-mon[30349]: pgmap v257: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:31.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:31 smithi161 ceph-mon[33805]: pgmap v257: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:31.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:31 smithi195 ceph-mon[33844]: pgmap v257: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:32.477 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:32 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:46:32.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.484+0000 7f692accb700 1 -- 172.21.15.115:0/4047780503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6924108760 msgr2=0x7f6924110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:32.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.484+0000 7f692accb700 1 --2- 172.21.15.115:0/4047780503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6924108760 0x7f6924110150 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f691c00b0d0 tx=0x7f691c00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.484+0000 7f692accb700 1 -- 172.21.15.115:0/4047780503 shutdown_connections 2024-05-09T05:46:32.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.484+0000 7f692accb700 1 --2- 172.21.15.115:0/4047780503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6924103ca0 0x7f69241040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.484+0000 7f692accb700 1 --2- 172.21.15.115:0/4047780503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6924108760 0x7f6924110150 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.484+0000 7f692accb700 1 --2- 172.21.15.115:0/4047780503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6924107e10 0x7f6924108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.484+0000 7f692accb700 1 -- 172.21.15.115:0/4047780503 >> 172.21.15.115:0/4047780503 conn(0x7f69240fe6f0 msgr2=0x7f6924100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:32.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.484+0000 7f692accb700 1 -- 172.21.15.115:0/4047780503 shutdown_connections 2024-05-09T05:46:32.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.484+0000 7f692accb700 1 -- 172.21.15.115:0/4047780503 wait complete. 2024-05-09T05:46:32.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.485+0000 7f692accb700 1 Processor -- start 2024-05-09T05:46:32.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.485+0000 7f692accb700 1 -- start start 2024-05-09T05:46:32.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f692accb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6924103ca0 0x7f6924075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:32.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f692accb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6924107e10 0x7f692406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:32.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f692accb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6924108760 0x7f692406ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:32.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f692accb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6924113230 con 0x7f6924107e10 2024-05-09T05:46:32.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f692accb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69241130b0 con 0x7f6924108760 2024-05-09T05:46:32.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f692accb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69241133b0 con 0x7f6924103ca0 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6923fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6924107e10 0x7f692406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6929268700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6924108760 0x7f692406ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6929268700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6924108760 0x7f692406ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:37928/0 (socket says 172.21.15.115:37928) 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6929268700 1 -- 172.21.15.115:0/1835779423 learned_addr learned my addr 172.21.15.115:0/1835779423 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6928a67700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6924103ca0 0x7f6924075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6923fff700 1 -- 172.21.15.115:0/1835779423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6924103ca0 msgr2=0x7f6924075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6923fff700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6924103ca0 0x7f6924075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6923fff700 1 -- 172.21.15.115:0/1835779423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6924108760 msgr2=0x7f692406ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6923fff700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6924108760 0x7f692406ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6923fff700 1 -- 172.21.15.115:0/1835779423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6924072f80 con 0x7f6924107e10 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6929268700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6924108760 0x7f692406ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.486+0000 7f6923fff700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6924107e10 0x7f692406e5d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f691000e9c0 tx=0x7f691000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:32.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.487+0000 7f6921ffb700 1 -- 172.21.15.115:0/1835779423 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691000c9c0 con 0x7f6924107e10 2024-05-09T05:46:32.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.488+0000 7f6921ffb700 1 -- 172.21.15.115:0/1835779423 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6910010430 con 0x7f6924107e10 2024-05-09T05:46:32.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.488+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6924073270 con 0x7f6924107e10 2024-05-09T05:46:32.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.489+0000 7f6921ffb700 1 -- 172.21.15.115:0/1835779423 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910018680 con 0x7f6924107e10 2024-05-09T05:46:32.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.489+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69240737b0 con 0x7f6924107e10 2024-05-09T05:46:32.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.490+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6924104a20 con 0x7f6924107e10 2024-05-09T05:46:32.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.491+0000 7f6921ffb700 1 -- 172.21.15.115:0/1835779423 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f69100108e0 con 0x7f6924107e10 2024-05-09T05:46:32.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.492+0000 7f6921ffb700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69140b2b40 0x7f69140b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:32.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.492+0000 7f6928a67700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69140b2b40 0x7f69140b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:32.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.492+0000 7f6921ffb700 1 -- 172.21.15.115:0/1835779423 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f69100e2910 con 0x7f6924107e10 2024-05-09T05:46:32.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.493+0000 7f6928a67700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69140b2b40 0x7f69140b5050 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f692406fdd0 tx=0x7f69180073d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:32.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.499+0000 7f6921ffb700 1 -- 172.21.15.115:0/1835779423 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69100a9f60 con 0x7f6924107e10 2024-05-09T05:46:32.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:32 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:46:32.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.817+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f692406fae0 con 0x7f6924107e10 2024-05-09T05:46:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.818+0000 7f6921ffb700 1 -- 172.21.15.115:0/1835779423 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11958 (secure 0 0 0) 0x7f692406fae0 con 0x7f6924107e10 2024-05-09T05:46:32.812 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:46:32.812 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:45:48.364068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":26,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26869":{"gid":26869,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/3665923682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3665923682},{"type":"v1","addr":"172.21.15.195:6833","nonce":3665923682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:46:32.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.822+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69140b2b40 msgr2=0x7f69140b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:32.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.822+0000 7f692accb700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69140b2b40 0x7f69140b5050 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f692406fdd0 tx=0x7f69180073d0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.822+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6924107e10 msgr2=0x7f692406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:32.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.822+0000 7f692accb700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6924107e10 0x7f692406e5d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f691000e9c0 tx=0x7f691000ecd0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.822+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 shutdown_connections 2024-05-09T05:46:32.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.822+0000 7f692accb700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69140b2b40 0x7f69140b5050 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.822+0000 7f692accb700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6924108760 0x7f692406ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.823+0000 7f692accb700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6924103ca0 0x7f6924075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.823+0000 7f692accb700 1 --2- 172.21.15.115:0/1835779423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6924107e10 0x7f692406e5d0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.823+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 >> 172.21.15.115:0/1835779423 conn(0x7f69240fe6f0 msgr2=0x7f6924107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.823+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 shutdown_connections 2024-05-09T05:46:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:32.823+0000 7f692accb700 1 -- 172.21.15.115:0/1835779423 wait complete. 2024-05-09T05:46:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 42 2024-05-09T05:46:32.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:32 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:46:33.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:33 smithi115 ceph-mon[30349]: pgmap v258: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:33.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:33 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1835779423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:46:33.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:33 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:46:33.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:33 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:46:33.513 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:33 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:46:33.539 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:46:33.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:33 smithi161 ceph-mon[33805]: pgmap v258: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:33.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:33 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1835779423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:46:33.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:33 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:46:33.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:33 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:46:33.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:33 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:46:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:46:33.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:33 smithi195 ceph-mon[33844]: pgmap v258: 129 pgs: 129 active+clean; 6.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:33.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:33 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1835779423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:46:33.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:33 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:46:33.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:33 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:46:33.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:33 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:46:35.586 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:35 smithi115 ceph-mon[30349]: pgmap v259: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:46:35.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.623+0000 7ff1f0f85700 1 -- 172.21.15.115:0/1952060360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ec107df0 msgr2=0x7ff1ec108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:35.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.623+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/1952060360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ec107df0 0x7ff1ec108200 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00099c0 tx=0x7ff1e0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.624+0000 7ff1f0f85700 1 -- 172.21.15.115:0/1952060360 shutdown_connections 2024-05-09T05:46:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.624+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/1952060360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec108740 0x7ff1ec1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.624+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/1952060360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ec1034c0 0x7ff1ec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.624+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/1952060360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ec107df0 0x7ff1ec108200 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.624+0000 7ff1f0f85700 1 -- 172.21.15.115:0/1952060360 >> 172.21.15.115:0/1952060360 conn(0x7ff1ec0fedc0 msgr2=0x7ff1ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.624+0000 7ff1f0f85700 1 -- 172.21.15.115:0/1952060360 shutdown_connections 2024-05-09T05:46:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.624+0000 7ff1f0f85700 1 -- 172.21.15.115:0/1952060360 wait complete. 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.625+0000 7ff1f0f85700 1 Processor -- start 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.625+0000 7ff1f0f85700 1 -- start start 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.625+0000 7ff1f0f85700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec1034c0 0x7ff1ec199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.625+0000 7ff1f0f85700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ec107df0 0x7ff1ec199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.625+0000 7ff1f0f85700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ec108740 0x7ff1ec19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1f0f85700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec113200 con 0x7ff1ec108740 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1f0f85700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec113080 con 0x7ff1ec1034c0 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1f0f85700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec113380 con 0x7ff1ec107df0 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1ea59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec1034c0 0x7ff1ec199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1ea59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec1034c0 0x7ff1ec199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:53724/0 (socket says 172.21.15.115:53724) 2024-05-09T05:46:35.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1ea59c700 1 -- 172.21.15.115:0/2137039554 learned_addr learned my addr 172.21.15.115:0/2137039554 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:35.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1ead9d700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ec108740 0x7ff1ec19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ec107df0 0x7ff1ec199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1ea59c700 1 -- 172.21.15.115:0/2137039554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ec107df0 msgr2=0x7ff1ec199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:35.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ec107df0 0x7ff1ec199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1ea59c700 1 -- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ec108740 msgr2=0x7ff1ec19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:35.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ec108740 0x7ff1ec19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.626+0000 7ff1ea59c700 1 -- 172.21.15.115:0/2137039554 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec19a700 con 0x7ff1ec1034c0 2024-05-09T05:46:35.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.627+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec1034c0 0x7ff1ec199910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00d950 tx=0x7ff1dc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:35.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.627+0000 7ff1db7fe700 1 -- 172.21.15.115:0/2137039554 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc016070 con 0x7ff1ec1034c0 2024-05-09T05:46:35.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.627+0000 7ff1db7fe700 1 -- 172.21.15.115:0/2137039554 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1dc009dd0 con 0x7ff1ec1034c0 2024-05-09T05:46:35.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.627+0000 7ff1f0f85700 1 -- 172.21.15.115:0/2137039554 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec19efc0 con 0x7ff1ec1034c0 2024-05-09T05:46:35.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.627+0000 7ff1db7fe700 1 -- 172.21.15.115:0/2137039554 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00f3c0 con 0x7ff1ec1034c0 2024-05-09T05:46:35.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.627+0000 7ff1f0f85700 1 -- 172.21.15.115:0/2137039554 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec19f480 con 0x7ff1ec1034c0 2024-05-09T05:46:35.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.631+0000 7ff1db7fe700 1 -- 172.21.15.115:0/2137039554 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff1dc019070 con 0x7ff1ec1034c0 2024-05-09T05:46:35.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.632+0000 7ff1db7fe700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1d40b29f0 0x7ff1d40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.632+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1d40b29f0 0x7ff1d40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.632+0000 7ff1db7fe700 1 -- 172.21.15.115:0/2137039554 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7ff1dc0e2600 con 0x7ff1ec1034c0 2024-05-09T05:46:35.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.632+0000 7ff1ff99bb80 1 Processor -- start 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ff99bb80 1 -- start start 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1d40b29f0 0x7ff1d40b4f00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009990 tx=0x7ff1e0005fd0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffce0160 0x55bbffd794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd79a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 0x55bbffd73ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bbffc95ff0 con 0x55bbffbf64a0 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bbffc347f0 con 0x55bbffd736a0 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bbffc91fb0 con 0x55bbffce0160 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ea59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffce0160 0x55bbffd794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ea59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffce0160 0x55bbffd794f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:52226/0 (socket says 172.21.15.115:52226) 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1e9d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd79a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ea59c700 1 -- 172.21.15.115:0/265054457 learned_addr learned my addr 172.21.15.115:0/265054457 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.633+0000 7ff1ead9d700 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 0x55bbffd73ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1ea59c700 1 -- 172.21.15.115:0/265054457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 msgr2=0x55bbffd73ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 0x55bbffd73ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1ea59c700 1 -- 172.21.15.115:0/265054457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 msgr2=0x55bbffd79a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd79a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1ea59c700 1 -- 172.21.15.115:0/265054457 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbffc2c370 con 0x55bbffce0160 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1ead9d700 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 0x55bbffd73ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd79a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffce0160 0x55bbffd794f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0a55d0 tx=0x7ff1dc0a9c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18e7e65cddea8915 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1d8ff9700 1 -- 172.21.15.115:0/265054457 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc020040 con 0x55bbffce0160 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1d8ff9700 1 -- 172.21.15.115:0/265054457 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1dc0e2070 con 0x55bbffce0160 2024-05-09T05:46:35.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.634+0000 7ff1d8ff9700 1 -- 172.21.15.115:0/265054457 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0e2460 con 0x55bbffce0160 2024-05-09T05:46:35.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.635+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/265054457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffce0160 msgr2=0x55bbffd794f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:35.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.635+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffce0160 0x55bbffd794f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0a55d0 tx=0x7ff1dc0a9c50 comp rx=0 tx=0).stop 2024-05-09T05:46:35.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.635+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/265054457 shutdown_connections 2024-05-09T05:46:35.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.635+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 0x55bbffd73ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.635+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffce0160 0x55bbffd794f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.635+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/265054457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd79a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.635+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/265054457 >> 172.21.15.115:0/265054457 conn(0x55bbffc566a0 msgr2=0x55bbffd6e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:35.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.635+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/265054457 shutdown_connections 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.636+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/265054457 wait complete. 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.636+0000 7ff1ff99bb80 1 -- start start 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.636+0000 7ff1ff99bb80 1 Processor -- start 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffbf64a0 0x55bbffd7d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 0x55bbffd7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bbffb11ba0 con 0x55bbffce0160 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bbffc50150 con 0x55bbffd736a0 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bbffd801d0 con 0x55bbffbf64a0 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1ff99bb80 10 client.0 _ll_register_callbacks cb 0x55bbffb076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1e9d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1e9d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd7d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47402/0 (socket says 172.21.15.115:47402) 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1e9d9b700 1 -- 172.21.15.115:0/1819096146 learned_addr learned my addr 172.21.15.115:0/1819096146 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1ead9d700 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 0x55bbffd7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffbf64a0 0x55bbffd7d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1e9d9b700 1 -- 172.21.15.115:0/1819096146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffbf64a0 msgr2=0x55bbffd7d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffbf64a0 0x55bbffd7d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1e9d9b700 1 -- 172.21.15.115:0/1819096146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 msgr2=0x55bbffd7dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 0x55bbffd7dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.637+0000 7ff1e9d9b700 1 -- 172.21.15.115:0/1819096146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbffb11490 con 0x55bbffce0160 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.638+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd7d800 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00099c0 tx=0x7ff1e0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb899a922019d78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.638+0000 7ff1d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.638+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/1819096146 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0034060 con 0x55bbffce0160 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.638+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/1819096146 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7ff1e00055e0 con 0x55bbffce0160 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.638+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1819096146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1dc0e2070 con 0x55bbffce0160 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.638+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/1819096146 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff1e003b050 con 0x55bbffce0160 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.638+0000 7ff1ff99bb80 10 client.17751 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.638+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1819096146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bbffd8ac90 con 0x55bbffce0160 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.639+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/1819096146 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1dc0e2070 con 0x55bbffce0160 2024-05-09T05:46:35.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.639+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/1819096146 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7ff1e000f460 con 0x55bbffce0160 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.639+0000 7ff1ff99bb80 10 client.17751 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.639+0000 7ff1ff99bb80 10 client.17751 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.639+0000 7ff1ff99bb80 20 client.17751 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.639+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55bbffd8d2d0 0x55bbffd8f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.639+0000 7ff1ff99bb80 4 client.17751 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.639+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1819096146 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbffca39d0 con 0x55bbffd8d2d0 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.639+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55bbffd8d2d0 0x55bbffd8f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.640+0000 7ff1d1ffb700 10 client.17751 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.640+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55bbffd8d2d0 0x55bbffd8f6f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb01412d32baf342 server_cookie=af21fd39046a1e4f in_seq=0 out_seq=0 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.640+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/1819096146 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbffca39d0 con 0x55bbffd8d2d0 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.640+0000 7ff1d1ffb700 10 client.17751 handle_command_reply: tid=0 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.640+0000 7ff1ff99bb80 1 client.17751 shutdown 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1819096146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 msgr2=0x55bbffd7d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:35.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd7d800 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00099c0 tx=0x7ff1e0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1819096146 shutdown_connections 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffd736a0 0x55bbffd7dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55bbffd8d2d0 0x55bbffd8f6f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffbf64a0 0x55bbffd7d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1819096146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd7d800 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1819096146 >> 172.21.15.115:0/1819096146 conn(0x55bbffc566a0 msgr2=0x55bbffd6e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1d1ffb700 0 client.17751 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1819096146 shutdown_connections 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1819096146 wait complete. 2024-05-09T05:46:35.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.641+0000 7ff1ff99bb80 20 client.17751 trim_cache size 0 max 16384 2024-05-09T05:46:35.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.656+0000 7ff1ff99bb80 1 Processor -- start 2024-05-09T05:46:35.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.656+0000 7ff1ff99bb80 1 -- start start 2024-05-09T05:46:35.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.656+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd78620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.656+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffce0160 0x55bbffd7c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.656+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 0x55bbffd77510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.656+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1b6af0 con 0x55bbffbf64a0 2024-05-09T05:46:35.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.656+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1b86c0 con 0x55bbffce0160 2024-05-09T05:46:35.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.656+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1b71c0 con 0x55bbffd736a0 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ea59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd78620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ea59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd78620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47410/0 (socket says 172.21.15.115:47410) 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ea59c700 1 -- 172.21.15.115:0/1983413096 learned_addr learned my addr 172.21.15.115:0/1983413096 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/1983413096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffce0160 0x55bbffd7c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ead9d700 1 --2- 172.21.15.115:0/1983413096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 0x55bbffd77510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ea59c700 1 -- 172.21.15.115:0/1983413096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 msgr2=0x55bbffd77510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/1983413096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 0x55bbffd77510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ea59c700 1 -- 172.21.15.115:0/1983413096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffce0160 msgr2=0x55bbffd7c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/1983413096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffce0160 0x55bbffd7c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ea59c700 1 -- 172.21.15.115:0/1983413096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbffd898e0 con 0x55bbffbf64a0 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/1983413096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd78620 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0a9250 tx=0x7ff1dc0b23b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca815298b6e2fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1d3fff700 1 -- 172.21.15.115:0/1983413096 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc020040 con 0x55bbffbf64a0 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1d3fff700 1 -- 172.21.15.115:0/1983413096 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1dc06e070 con 0x55bbffbf64a0 2024-05-09T05:46:35.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.657+0000 7ff1d3fff700 1 -- 172.21.15.115:0/1983413096 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0a9a10 con 0x55bbffbf64a0 2024-05-09T05:46:35.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.658+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1983413096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 msgr2=0x55bbffd78620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.658+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1983413096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd78620 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0a9250 tx=0x7ff1dc0b23b0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.658+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1983413096 shutdown_connections 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.658+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1983413096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 0x55bbffd77510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.658+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1983413096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffbf64a0 0x55bbffd78620 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.658+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/1983413096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffce0160 0x55bbffd7c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.658+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1983413096 >> 172.21.15.115:0/1983413096 conn(0x55bbffc566a0 msgr2=0x55bbffd76e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.658+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1983413096 shutdown_connections 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.658+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/1983413096 wait complete. 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ff99bb80 1 -- start start 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ff99bb80 1 Processor -- start 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffbf64a0 0x55bbffd854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ff99bb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 0x55bbffd7e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1b7ff0 con 0x55bbffce0160 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1b7920 con 0x55bbffbf64a0 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ff99bb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1b4ab0 con 0x55bbffd736a0 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ff99bb80 10 client.0 _ll_register_callbacks cb 0x55bbffb076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1e9d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1e9d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd859f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47414/0 (socket says 172.21.15.115:47414) 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1e9d9b700 1 -- 172.21.15.115:0/3449791099 learned_addr learned my addr 172.21.15.115:0/3449791099 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ead9d700 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 0x55bbffd7e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.659+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffbf64a0 0x55bbffd854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1e9d9b700 1 -- 172.21.15.115:0/3449791099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 msgr2=0x55bbffd7e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 0x55bbffd7e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1e9d9b700 1 -- 172.21.15.115:0/3449791099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffbf64a0 msgr2=0x55bbffd854b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffbf64a0 0x55bbffd854b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1e9d9b700 1 -- 172.21.15.115:0/3449791099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbffd7ee00 con 0x55bbffce0160 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffbf64a0 0x55bbffd854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1e9d9b700 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd859f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000f940 tx=0x7ff1e0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c923d475fa172aeb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/3449791099 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0034040 con 0x55bbffce0160 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/3449791099 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7ff1e000f9c0 con 0x55bbffce0160 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/3449791099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1dc06e070 con 0x55bbffce0160 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/3449791099 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff1e004c050 con 0x55bbffce0160 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1ff99bb80 10 client.17757 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.660+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/3449791099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bbffdf7480 con 0x55bbffce0160 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.661+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/3449791099 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1e000fbc0 con 0x55bbffce0160 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.661+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/3449791099 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7ff1e0047030 con 0x55bbffce0160 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.661+0000 7ff1ff99bb80 10 client.17757 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.661+0000 7ff1ff99bb80 10 client.17757 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:46:35.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.661+0000 7ff1ff99bb80 20 client.17757 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.661+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55bbffdf7b80 0x55bbffdf9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.661+0000 7ff1ff99bb80 4 client.17757 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.661+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/3449791099 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff1dc06e070 con 0x55bbffdf7b80 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.661+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55bbffdf7b80 0x55bbffdf9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ea59c700 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55bbffdf7b80 0x55bbffdf9fa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c519c8aaab269b73 server_cookie=f4e471b2b03b4414 in_seq=0 out_seq=0 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1d1ffb700 10 client.17757 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1d1ffb700 1 -- 172.21.15.115:0/3449791099 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff1dc06e070 con 0x55bbffdf7b80 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1d1ffb700 10 client.17757 handle_command_reply: tid=0 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 client.17757 shutdown 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/3449791099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 msgr2=0x55bbffd859f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd859f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000f940 tx=0x7ff1e0005810 comp rx=0 tx=0).stop 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/3449791099 shutdown_connections 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bbffbf64a0 0x55bbffd854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55bbffdf7b80 0x55bbffdf9fa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bbffd736a0 0x55bbffd7e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 --2- 172.21.15.115:0/3449791099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bbffce0160 0x55bbffd859f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/3449791099 >> 172.21.15.115:0/3449791099 conn(0x55bbffc566a0 msgr2=0x55bbffd7c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1d1ffb700 0 client.17757 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:46:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/3449791099 shutdown_connections 2024-05-09T05:46:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 1 -- 172.21.15.115:0/3449791099 wait complete. 2024-05-09T05:46:35.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.662+0000 7ff1ff99bb80 20 client.17757 trim_cache size 0 max 16384 2024-05-09T05:46:35.661 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:46:35.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 -- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1d40b29f0 msgr2=0x7ff1d40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:35.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1d40b29f0 0x7ff1d40b4f00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009990 tx=0x7ff1e0005fd0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 -- 172.21.15.115:0/2137039554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec1034c0 msgr2=0x7ff1ec199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:35.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec1034c0 0x7ff1ec199910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00d950 tx=0x7ff1dc00dc60 comp rx=0 tx=0).stop 2024-05-09T05:46:35.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 -- 172.21.15.115:0/2137039554 shutdown_connections 2024-05-09T05:46:35.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff1d40b29f0 0x7ff1d40b4f00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec1034c0 0x7ff1ec199910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1ec108740 0x7ff1ec19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 --2- 172.21.15.115:0/2137039554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1ec107df0 0x7ff1ec199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:35.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 -- 172.21.15.115:0/2137039554 >> 172.21.15.115:0/2137039554 conn(0x7ff1ec0fedc0 msgr2=0x7ff1ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:35.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 -- 172.21.15.115:0/2137039554 shutdown_connections 2024-05-09T05:46:35.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:35.663+0000 7ff1f0f85700 1 -- 172.21.15.115:0/2137039554 wait complete. 2024-05-09T05:46:35.663 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:35 smithi161 ceph-mon[33805]: pgmap v259: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:46:35.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:35 smithi195 ceph-mon[33844]: pgmap v259: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:46:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.356+0000 7f2c63766640 1 -- 172.21.15.115:0/186456398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c109030 msgr2=0x7f2c5c102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.356+0000 7f2c63766640 1 --2- 172.21.15.115:0/186456398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c109030 0x7f2c5c102c70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009900 tx=0x7f2c4c02a410 comp rx=0 tx=0).stop 2024-05-09T05:46:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.356+0000 7f2c63766640 1 -- 172.21.15.115:0/186456398 shutdown_connections 2024-05-09T05:46:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.356+0000 7f2c63766640 1 --2- 172.21.15.115:0/186456398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c5c1032e0 0x7f2c5c10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.356+0000 7f2c63766640 1 --2- 172.21.15.115:0/186456398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c109030 0x7f2c5c102c70 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.356+0000 7f2c63766640 1 --2- 172.21.15.115:0/186456398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c5c108b90 0x7f2c5c102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.356+0000 7f2c63766640 1 -- 172.21.15.115:0/186456398 >> 172.21.15.115:0/186456398 conn(0x7f2c5c0fe390 msgr2=0x7f2c5c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.356+0000 7f2c63766640 1 -- 172.21.15.115:0/186456398 shutdown_connections 2024-05-09T05:46:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.356+0000 7f2c63766640 1 -- 172.21.15.115:0/186456398 wait complete. 2024-05-09T05:46:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.357+0000 7f2c63766640 1 Processor -- start 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.357+0000 7f2c63766640 1 -- start start 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c63766640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c1032e0 0x7f2c5c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c63766640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c5c108b90 0x7f2c5c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c63766640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c5c109030 0x7f2c5c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c63766640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c112c30 con 0x7f2c5c1032e0 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c63766640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c112ab0 con 0x7f2c5c108b90 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c63766640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c112db0 con 0x7f2c5c109030 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61502640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c1032e0 0x7f2c5c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61502640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c1032e0 0x7f2c5c199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47438/0 (socket says 172.21.15.115:47438) 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61502640 1 -- 172.21.15.115:0/231156949 learned_addr learned my addr 172.21.15.115:0/231156949 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c60d01640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c5c108b90 0x7f2c5c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61d03640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c5c109030 0x7f2c5c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61502640 1 -- 172.21.15.115:0/231156949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c5c109030 msgr2=0x7f2c5c19df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61502640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c5c109030 0x7f2c5c19df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61502640 1 -- 172.21.15.115:0/231156949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c5c108b90 msgr2=0x7f2c5c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61502640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c5c108b90 0x7f2c5c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61502640 1 -- 172.21.15.115:0/231156949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5c19e5b0 con 0x7f2c5c1032e0 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c61d03640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c5c109030 0x7f2c5c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.358+0000 7f2c60d01640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c5c108b90 0x7f2c5c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.359+0000 7f2c61502640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c1032e0 0x7f2c5c199750 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800d770 tx=0x7f2c5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.359+0000 7f2c527fc640 1 -- 172.21.15.115:0/231156949 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58017070 con 0x7f2c5c1032e0 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.359+0000 7f2c63766640 1 -- 172.21.15.115:0/231156949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5c19e8a0 con 0x7f2c5c1032e0 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.359+0000 7f2c527fc640 1 -- 172.21.15.115:0/231156949 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c58009c30 con 0x7f2c5c1032e0 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.359+0000 7f2c527fc640 1 -- 172.21.15.115:0/231156949 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800f620 con 0x7f2c5c1032e0 2024-05-09T05:46:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.359+0000 7f2c63766640 1 -- 172.21.15.115:0/231156949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5c19ed70 con 0x7f2c5c1032e0 2024-05-09T05:46:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.363+0000 7f2c527fc640 1 -- 172.21.15.115:0/231156949 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f2c5801a070 con 0x7f2c5c1032e0 2024-05-09T05:46:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.363+0000 7f2c6a617740 1 Processor -- start 2024-05-09T05:46:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.363+0000 7f2c6a617740 1 -- start start 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a7241b0 0x55744aac7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a6fb2f0 0x55744aac7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 0x55744aac8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c6a617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55744aa2c7c0 con 0x55744a6fb2f0 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c6a617740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55744a9c80e0 con 0x55744a7241b0 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c6a617740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55744aa28780 con 0x55744a7034d0 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c61d03640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a7241b0 0x55744aac7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c61502640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a6fb2f0 0x55744aac7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c527fc640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c480b2950 0x7f2c480b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c61d03640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a7241b0 0x55744aac7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:53786/0 (socket says 172.21.15.115:53786) 2024-05-09T05:46:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c61d03640 1 -- 172.21.15.115:0/975497011 learned_addr learned my addr 172.21.15.115:0/975497011 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c60d01640 1 --2- 172.21.15.115:0/975497011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 0x55744aac8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c61d03640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c480b2950 0x7f2c480b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.364+0000 7f2c527fc640 1 -- 172.21.15.115:0/231156949 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f2c580e2480 con 0x7f2c5c1032e0 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c61502640 1 -- 172.21.15.115:0/975497011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 msgr2=0x55744aac8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c61502640 1 --2- 172.21.15.115:0/975497011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 0x55744aac8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c61502640 1 -- 172.21.15.115:0/975497011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a7241b0 msgr2=0x55744aac7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c61502640 1 --2- 172.21.15.115:0/975497011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a7241b0 0x55744aac7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c61502640 1 -- 172.21.15.115:0/975497011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55744a9f17c0 con 0x55744a6fb2f0 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c61d03640 1 --2- 172.21.15.115:0/975497011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a7241b0 0x55744aac7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c61502640 1 --2- 172.21.15.115:0/975497011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a6fb2f0 0x55744aac7bc0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f2c580df840 tx=0x7f2c5800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09ab22635e3b353 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c477fe640 1 -- 172.21.15.115:0/975497011 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58021030 con 0x55744a6fb2f0 2024-05-09T05:46:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c61d03640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c480b2950 0x7f2c480b4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55744aac47f0 tx=0x7f2c54004ab0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c477fe640 1 -- 172.21.15.115:0/975497011 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c580a8ee0 con 0x55744a6fb2f0 2024-05-09T05:46:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.365+0000 7f2c477fe640 1 -- 172.21.15.115:0/975497011 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580a93d0 con 0x55744a6fb2f0 2024-05-09T05:46:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.366+0000 7f2c6a617740 1 -- 172.21.15.115:0/975497011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a6fb2f0 msgr2=0x55744aac7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.366+0000 7f2c6a617740 1 --2- 172.21.15.115:0/975497011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a6fb2f0 0x55744aac7bc0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f2c580df840 tx=0x7f2c5800df10 comp rx=0 tx=0).stop 2024-05-09T05:46:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.366+0000 7f2c6a617740 1 -- 172.21.15.115:0/975497011 shutdown_connections 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.366+0000 7f2c6a617740 1 --2- 172.21.15.115:0/975497011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 0x55744aac8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.366+0000 7f2c6a617740 1 --2- 172.21.15.115:0/975497011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a6fb2f0 0x55744aac7bc0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.366+0000 7f2c6a617740 1 --2- 172.21.15.115:0/975497011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a7241b0 0x55744aac7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.366+0000 7f2c6a617740 1 -- 172.21.15.115:0/975497011 >> 172.21.15.115:0/975497011 conn(0x55744a718300 msgr2=0x55744aac3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.366+0000 7f2c6a617740 1 -- 172.21.15.115:0/975497011 shutdown_connections 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.367+0000 7f2c6a617740 1 -- 172.21.15.115:0/975497011 wait complete. 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.367+0000 7f2c6a617740 1 -- start start 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.367+0000 7f2c6a617740 1 Processor -- start 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 0x55744aad36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 0x55744aad3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7241b0 0x55744aad1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c6a617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55744aad46f0 con 0x55744a7241b0 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c6a617740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55744a9b39c0 con 0x55744a6fb2f0 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c6a617740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55744aad4870 con 0x55744a7034d0 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c6a617740 10 client.0 _ll_register_callbacks cb 0x55744a9e0020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c61d03640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7241b0 0x55744aad1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c61d03640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7241b0 0x55744aad1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47452/0 (socket says 172.21.15.115:47452) 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c61d03640 1 -- 172.21.15.115:0/2115890578 learned_addr learned my addr 172.21.15.115:0/2115890578 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c61502640 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 0x55744aad36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c60d01640 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 0x55744aad3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c61d03640 1 -- 172.21.15.115:0/2115890578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 msgr2=0x55744aad3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c61d03640 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 0x55744aad3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c61d03640 1 -- 172.21.15.115:0/2115890578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 msgr2=0x55744aad36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c61d03640 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 0x55744aad36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.368+0000 7f2c61d03640 1 -- 172.21.15.115:0/2115890578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55744a9da1f0 con 0x55744a7241b0 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c61d03640 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7241b0 0x55744aad1ba0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2c54014010 tx=0x7f2c54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e7fb8af6c833d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c44ff9640 1 -- 172.21.15.115:0/2115890578 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54003e20 con 0x55744a7241b0 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c44ff9640 1 -- 172.21.15.115:0/2115890578 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f2c540047d0 con 0x55744a7241b0 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c6a617740 1 -- 172.21.15.115:0/2115890578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c580a8ee0 con 0x55744a7241b0 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c44ff9640 1 -- 172.21.15.115:0/2115890578 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2c54031050 con 0x55744a7241b0 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c6a617740 10 client.17769 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c6a617740 1 -- 172.21.15.115:0/2115890578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55744a78f790 con 0x55744a7241b0 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c44ff9640 1 -- 172.21.15.115:0/2115890578 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c580a8ee0 con 0x55744a7241b0 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c44ff9640 1 -- 172.21.15.115:0/2115890578 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f2c54028400 con 0x55744a7241b0 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c6a617740 10 client.17769 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c6a617740 10 client.17769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.369+0000 7f2c6a617740 20 client.17769 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.370+0000 7f2c6a617740 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55744a6e2d80 0x55744aae3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.370+0000 7f2c6a617740 4 client.17769 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.370+0000 7f2c6a617740 1 -- 172.21.15.115:0/2115890578 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55744aa3a3d0 con 0x55744a6e2d80 2024-05-09T05:46:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.370+0000 7f2c61502640 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55744a6e2d80 0x55744aae3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.370+0000 7f2c44ff9640 10 client.17769 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.370+0000 7f2c61502640 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55744a6e2d80 0x55744aae3f30 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3631ea2d36d37f server_cookie=567be402e57ed12d in_seq=0 out_seq=0 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c44ff9640 1 -- 172.21.15.115:0/2115890578 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55744aa3a3d0 con 0x55744a6e2d80 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c44ff9640 10 client.17769 handle_command_reply: tid=0 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c6a617740 1 client.17769 shutdown 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c6a617740 1 -- 172.21.15.115:0/2115890578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7241b0 msgr2=0x55744aad1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c6a617740 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7241b0 0x55744aad1ba0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2c54014010 tx=0x7f2c54004a40 comp rx=0 tx=0).stop 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c6a617740 1 -- 172.21.15.115:0/2115890578 shutdown_connections 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c6a617740 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7241b0 0x55744aad1ba0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c6a617740 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55744a6e2d80 0x55744aae3f30 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c6a617740 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a7034d0 0x55744aad3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c6a617740 1 --2- 172.21.15.115:0/2115890578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 0x55744aad36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.371+0000 7f2c44ff9640 0 client.17769 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.372+0000 7f2c6a617740 1 -- 172.21.15.115:0/2115890578 >> 172.21.15.115:0/2115890578 conn(0x55744a718300 msgr2=0x55744aac2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.372+0000 7f2c6a617740 1 -- 172.21.15.115:0/2115890578 shutdown_connections 2024-05-09T05:46:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.372+0000 7f2c6a617740 1 -- 172.21.15.115:0/2115890578 wait complete. 2024-05-09T05:46:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.372+0000 7f2c6a617740 20 client.17769 trim_cache size 0 max 16384 2024-05-09T05:46:36.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:46:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c6a617740 1 Processor -- start 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c6a617740 1 -- start start 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6e2d80 0x55744aac2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6fb2f0 0x55744aad2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744aacb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c6a617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1b3380 con 0x55744a7034d0 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c6a617740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1b4f50 con 0x55744a6e2d80 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c6a617740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1b3a50 con 0x55744a6fb2f0 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c61d03640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744aacb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c61502640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6e2d80 0x55744aac2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c61d03640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744aacb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47466/0 (socket says 172.21.15.115:47466) 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c61d03640 1 -- 172.21.15.115:0/4105107244 learned_addr learned my addr 172.21.15.115:0/4105107244 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.394+0000 7f2c60d01640 1 --2- 172.21.15.115:0/4105107244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6fb2f0 0x55744aad2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c61502640 1 -- 172.21.15.115:0/4105107244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6fb2f0 msgr2=0x55744aad2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c61502640 1 --2- 172.21.15.115:0/4105107244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6fb2f0 0x55744aad2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c61502640 1 -- 172.21.15.115:0/4105107244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 msgr2=0x55744aacb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c61502640 1 --2- 172.21.15.115:0/4105107244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744aacb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c61502640 1 -- 172.21.15.115:0/4105107244 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55744a8065f0 con 0x55744a6e2d80 2024-05-09T05:46:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c61d03640 1 --2- 172.21.15.115:0/4105107244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744aacb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c61502640 1 --2- 172.21.15.115:0/4105107244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6e2d80 0x55744aac2d70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55744aac9130 tx=0x7f2c580ad8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ce3e87a19fcfe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c46ffd640 1 -- 172.21.15.115:0/4105107244 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58021040 con 0x55744a6e2d80 2024-05-09T05:46:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c46ffd640 1 -- 172.21.15.115:0/4105107244 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c580a9ad0 con 0x55744a6e2d80 2024-05-09T05:46:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.395+0000 7f2c46ffd640 1 -- 172.21.15.115:0/4105107244 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580ad300 con 0x55744a6e2d80 2024-05-09T05:46:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.396+0000 7f2c6a617740 1 -- 172.21.15.115:0/4105107244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6e2d80 msgr2=0x55744aac2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.396+0000 7f2c6a617740 1 --2- 172.21.15.115:0/4105107244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6e2d80 0x55744aac2d70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55744aac9130 tx=0x7f2c580ad8c0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.396+0000 7f2c6a617740 1 -- 172.21.15.115:0/4105107244 shutdown_connections 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.396+0000 7f2c6a617740 1 --2- 172.21.15.115:0/4105107244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744aacb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.396+0000 7f2c6a617740 1 --2- 172.21.15.115:0/4105107244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6fb2f0 0x55744aad2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.396+0000 7f2c6a617740 1 --2- 172.21.15.115:0/4105107244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6e2d80 0x55744aac2d70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.397+0000 7f2c6a617740 1 -- 172.21.15.115:0/4105107244 >> 172.21.15.115:0/4105107244 conn(0x55744a718300 msgr2=0x55744aad1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.397+0000 7f2c6a617740 1 -- 172.21.15.115:0/4105107244 shutdown_connections 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.397+0000 7f2c6a617740 1 -- 172.21.15.115:0/4105107244 wait complete. 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.397+0000 7f2c6a617740 1 -- start start 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.397+0000 7f2c6a617740 1 Processor -- start 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6e2d80 0x55744aad5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 0x55744aad5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c6a617740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744ab4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c6a617740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1b4880 con 0x55744a7034d0 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c6a617740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1b41b0 con 0x55744a6fb2f0 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c6a617740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1b1380 con 0x55744a6e2d80 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c6a617740 10 client.0 _ll_register_callbacks cb 0x7f2c54009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61d03640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744ab4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c60d01640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 0x55744aad5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61d03640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744ab4b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47468/0 (socket says 172.21.15.115:47468) 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61d03640 1 -- 172.21.15.115:0/1290700401 learned_addr learned my addr 172.21.15.115:0/1290700401 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61502640 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6e2d80 0x55744aad5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61d03640 1 -- 172.21.15.115:0/1290700401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6e2d80 msgr2=0x55744aad5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61d03640 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6e2d80 0x55744aad5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61d03640 1 -- 172.21.15.115:0/1290700401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 msgr2=0x55744aad5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61d03640 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 0x55744aad5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61d03640 1 -- 172.21.15.115:0/1290700401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55744a9f0ed0 con 0x55744a7034d0 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.398+0000 7f2c61502640 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6e2d80 0x55744aad5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c61d03640 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744ab4b2c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2c54014010 tx=0x7f2c54005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7f9a1ec8c231e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c44ff9640 1 -- 172.21.15.115:0/1290700401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54004410 con 0x55744a7034d0 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c44ff9640 1 -- 172.21.15.115:0/1290700401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f2c540134a0 con 0x55744a7034d0 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c6a617740 1 -- 172.21.15.115:0/1290700401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c580a9ad0 con 0x55744a7034d0 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c44ff9640 1 -- 172.21.15.115:0/1290700401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2c54047050 con 0x55744a7034d0 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c6a617740 10 client.17775 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c6a617740 1 -- 172.21.15.115:0/1290700401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55744ab4c650 con 0x55744a7034d0 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c44ff9640 1 -- 172.21.15.115:0/1290700401 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c580a9ad0 con 0x55744a7034d0 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c44ff9640 1 -- 172.21.15.115:0/1290700401 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f2c54010a00 con 0x55744a7034d0 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c6a617740 10 client.17775 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c6a617740 10 client.17775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.399+0000 7f2c6a617740 20 client.17775 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.400+0000 7f2c6a617740 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55744a7241b0 0x55744ab4ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.400+0000 7f2c6a617740 4 client.17775 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.400+0000 7f2c6a617740 1 -- 172.21.15.115:0/1290700401 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55744aa34240 con 0x55744a7241b0 2024-05-09T05:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.400+0000 7f2c61502640 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55744a7241b0 0x55744ab4ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.400+0000 7f2c44ff9640 10 client.17775 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.400+0000 7f2c61502640 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55744a7241b0 0x55744ab4ed80 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86443f31bc3c792 server_cookie=21f1c61608412a7a in_seq=0 out_seq=0 2024-05-09T05:46:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c44ff9640 1 -- 172.21.15.115:0/1290700401 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55744aa34240 con 0x55744a7241b0 2024-05-09T05:46:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c44ff9640 10 client.17775 handle_command_reply: tid=0 2024-05-09T05:46:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 client.17775 shutdown 2024-05-09T05:46:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 -- 172.21.15.115:0/1290700401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 msgr2=0x55744ab4b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744ab4b2c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2c54014010 tx=0x7f2c54005f50 comp rx=0 tx=0).stop 2024-05-09T05:46:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 -- 172.21.15.115:0/1290700401 shutdown_connections 2024-05-09T05:46:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55744a7034d0 0x55744ab4b2c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55744a6fb2f0 0x55744aad5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55744a7241b0 0x55744ab4ed80 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 --2- 172.21.15.115:0/1290700401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55744a6e2d80 0x55744aad5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 -- 172.21.15.115:0/1290700401 >> 172.21.15.115:0/1290700401 conn(0x55744a718300 msgr2=0x55744aac35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c44ff9640 0 client.17775 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 -- 172.21.15.115:0/1290700401 shutdown_connections 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 1 -- 172.21.15.115:0/1290700401 wait complete. 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.409+0000 7f2c6a617740 20 client.17775 trim_cache size 0 max 16384 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:46:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:46:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 -- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c480b2950 msgr2=0x7f2c480b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c480b2950 0x7f2c480b4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55744aac47f0 tx=0x7f2c54004ab0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 -- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c1032e0 msgr2=0x7f2c5c199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c1032e0 0x7f2c5c199750 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800d770 tx=0x7f2c5800dc30 comp rx=0 tx=0).stop 2024-05-09T05:46:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 -- 172.21.15.115:0/231156949 shutdown_connections 2024-05-09T05:46:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2c480b2950 0x7f2c480b4e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2c5c109030 0x7f2c5c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c5c108b90 0x7f2c5c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 --2- 172.21.15.115:0/231156949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c5c1032e0 0x7f2c5c199750 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 -- 172.21.15.115:0/231156949 >> 172.21.15.115:0/231156949 conn(0x7f2c5c0fe390 msgr2=0x7f2c5c10e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 -- 172.21.15.115:0/231156949 shutdown_connections 2024-05-09T05:46:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:36.410+0000 7f2c63766640 1 -- 172.21.15.115:0/231156949 wait complete. 2024-05-09T05:46:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:46:37.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:37 smithi115 ceph-mon[30349]: pgmap v260: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:37.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:37 smithi161 ceph-mon[33805]: pgmap v260: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:37 smithi195 ceph-mon[33844]: pgmap v260: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:46:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.519+0000 7fd1757b5700 1 -- 172.21.15.115:0/716709367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 msgr2=0x7fd1701100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.519+0000 7fd1757b5700 1 --2- 172.21.15.115:0/716709367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 0x7fd1701100a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd1680099c0 tx=0x7fd168009cd0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.519+0000 7fd1757b5700 1 -- 172.21.15.115:0/716709367 shutdown_connections 2024-05-09T05:46:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.519+0000 7fd1757b5700 1 --2- 172.21.15.115:0/716709367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1701034c0 0x7fd1701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.519+0000 7fd1757b5700 1 --2- 172.21.15.115:0/716709367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 0x7fd1701100a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.519+0000 7fd1757b5700 1 --2- 172.21.15.115:0/716709367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd170107df0 0x7fd170108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.519+0000 7fd1757b5700 1 -- 172.21.15.115:0/716709367 >> 172.21.15.115:0/716709367 conn(0x7fd1700fedc0 msgr2=0x7fd1701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.519+0000 7fd1757b5700 1 -- 172.21.15.115:0/716709367 shutdown_connections 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.519+0000 7fd1757b5700 1 -- 172.21.15.115:0/716709367 wait complete. 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd1757b5700 1 Processor -- start 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd1757b5700 1 -- start start 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd1757b5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1701034c0 0x7fd17019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd1757b5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd170107df0 0x7fd17019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd1757b5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 0x7fd1701a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd1757b5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd170113220 con 0x7fd170108740 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd1757b5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701130a0 con 0x7fd170107df0 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd1757b5700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701133a0 con 0x7fd1701034c0 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd16f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 0x7fd1701a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd16e7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd170107df0 0x7fd17019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd16f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 0x7fd1701a6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47482/0 (socket says 172.21.15.115:47482) 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd16effd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1701034c0 0x7fd17019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.520+0000 7fd16f7fe700 1 -- 172.21.15.115:0/750253762 learned_addr learned my addr 172.21.15.115:0/750253762 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd16f7fe700 1 -- 172.21.15.115:0/750253762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1701034c0 msgr2=0x7fd17019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1701034c0 0x7fd17019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd16f7fe700 1 -- 172.21.15.115:0/750253762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd170107df0 msgr2=0x7fd17019c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd170107df0 0x7fd17019c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd16f7fe700 1 -- 172.21.15.115:0/750253762 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17019c8b0 con 0x7fd170108740 2024-05-09T05:46:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd16effd700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1701034c0 0x7fd17019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 0x7fd1701a6b40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd168009990 tx=0x7fd1680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd167fff700 1 -- 172.21.15.115:0/750253762 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168004830 con 0x7fd170108740 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd167fff700 1 -- 172.21.15.115:0/750253762 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd168005020 con 0x7fd170108740 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd1757b5700 1 -- 172.21.15.115:0/750253762 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd170199970 con 0x7fd170108740 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd167fff700 1 -- 172.21.15.115:0/750253762 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16800f9d0 con 0x7fd170108740 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.521+0000 7fd1757b5700 1 -- 172.21.15.115:0/750253762 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd170199da0 con 0x7fd170108740 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.524+0000 7fd167fff700 1 -- 172.21.15.115:0/750253762 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fd16800fb70 con 0x7fd170108740 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.525+0000 7fd167fff700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd15c0b29f0 0x7fd15c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.525+0000 7fd16effd700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd15c0b29f0 0x7fd15c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.525+0000 7fd167fff700 1 -- 172.21.15.115:0/750253762 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fd1680e3bc0 con 0x7fd170108740 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.525+0000 7fd1841cbb80 1 Processor -- start 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.525+0000 7fd1841cbb80 1 -- start start 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16effd700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd15c0b29f0 0x7fd15c0b4f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd160009fd0 tx=0x7fd1600091b0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e60664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 0x55e0e6066a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e6060ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e0e5f82ff0 con 0x55e0e5fcd160 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e0e5f217f0 con 0x55e0e60606a0 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e0e5f7efb0 con 0x55e0e5ee34a0 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e60664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e60664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47484/0 (socket says 172.21.15.115:47484) 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16effd700 1 -- 172.21.15.115:0/3387036311 learned_addr learned my addr 172.21.15.115:0/3387036311 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16e7fc700 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 0x55e0e6066a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e6060ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16effd700 1 -- 172.21.15.115:0/3387036311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 msgr2=0x55e0e6066a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16effd700 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 0x55e0e6066a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16effd700 1 -- 172.21.15.115:0/3387036311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 msgr2=0x55e0e6060ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16effd700 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e6060ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16effd700 1 -- 172.21.15.115:0/3387036311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e5f19370 con 0x55e0e5fcd160 2024-05-09T05:46:38.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.526+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e6060ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:38.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.527+0000 7fd16effd700 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e60664f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd160015890 tx=0x7fd160013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6454702535cd08da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:38.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.527+0000 7fd1657fa700 1 -- 172.21.15.115:0/3387036311 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16001c050 con 0x55e0e5fcd160 2024-05-09T05:46:38.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.527+0000 7fd1657fa700 1 -- 172.21.15.115:0/3387036311 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd160026070 con 0x55e0e5fcd160 2024-05-09T05:46:38.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.527+0000 7fd1657fa700 1 -- 172.21.15.115:0/3387036311 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160021c30 con 0x55e0e5fcd160 2024-05-09T05:46:38.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.528+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3387036311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 msgr2=0x55e0e60664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.528+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e60664f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd160015890 tx=0x7fd160013000 comp rx=0 tx=0).stop 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.528+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3387036311 shutdown_connections 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.528+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e6060ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.528+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e60664f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.528+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/3387036311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 0x55e0e6066a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.528+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3387036311 >> 172.21.15.115:0/3387036311 conn(0x55e0e5f436a0 msgr2=0x55e0e605b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.528+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3387036311 shutdown_connections 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.528+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3387036311 wait complete. 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.529+0000 7fd1841cbb80 1 -- start start 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.529+0000 7fd1841cbb80 1 Processor -- start 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.529+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 0x55e0e606a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e606a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e606ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e0e5dfeba0 con 0x55e0e5fcd160 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e0e5f3d150 con 0x55e0e60606a0 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e606ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e606ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:53864/0 (socket says 172.21.15.115:53864) 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16f7fe700 1 -- 172.21.15.115:0/633735039 learned_addr learned my addr 172.21.15.115:0/633735039 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd1841cbb80 1 -- 172.21.15.115:0/633735039 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e0e606d1d0 con 0x55e0e5ee34a0 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16effd700 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 0x55e0e606a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16e7fc700 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e606a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16f7fe700 1 -- 172.21.15.115:0/633735039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 msgr2=0x55e0e606a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 0x55e0e606a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16f7fe700 1 -- 172.21.15.115:0/633735039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 msgr2=0x55e0e606a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd1841cbb80 10 client.0 _ll_register_callbacks cb 0x55e0e5df46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e606a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16f7fe700 1 -- 172.21.15.115:0/633735039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e5dfe490 con 0x55e0e60606a0 2024-05-09T05:46:38.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd1841cbb80 1 -- 172.21.15.115:0/633735039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd160026070 con 0x55e0e60606a0 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.530+0000 7fd16e7fc700 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e606a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e606ac80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1680aa8a0 tx=0x7fd16800ff60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef780c17a6e8d70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1567fc700 10 client.27095 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1567fc700 1 -- 172.21.15.115:0/633735039 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1680b32b0 con 0x55e0e60606a0 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1567fc700 1 -- 172.21.15.115:0/633735039 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fd1680b36f0 con 0x55e0e60606a0 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1567fc700 1 -- 172.21.15.115:0/633735039 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd1680aa6b0 con 0x55e0e60606a0 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1841cbb80 10 client.27095 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1841cbb80 1 -- 172.21.15.115:0/633735039 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55e0e60779d0 con 0x55e0e60606a0 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1567fc700 1 -- 172.21.15.115:0/633735039 <== mon.2 v2:172.21.15.161:3300/0 4 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fd1680049d0 con 0x55e0e60606a0 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1841cbb80 10 client.27095 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1841cbb80 10 client.27095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:46:38.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1841cbb80 20 client.27095 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.531+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e0e6079f40 0x55e0e607c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.532+0000 7fd1841cbb80 4 client.27095 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.532+0000 7fd1841cbb80 1 -- 172.21.15.115:0/633735039 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0e5f909d0 con 0x55e0e6079f40 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.532+0000 7fd16effd700 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e0e6079f40 0x55e0e607c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.532+0000 7fd1567fc700 10 client.27095 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.532+0000 7fd16effd700 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e0e6079f40 0x55e0e607c3f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a02a6cec82033358 server_cookie=5ebeacb4757fb175 in_seq=0 out_seq=0 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1567fc700 1 -- 172.21.15.115:0/633735039 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0e5f909d0 con 0x55e0e6079f40 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1567fc700 10 client.27095 handle_command_reply: tid=0 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 client.27095 shutdown 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 -- 172.21.15.115:0/633735039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 msgr2=0x55e0e606ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e606ac80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1680aa8a0 tx=0x7fd16800ff60 comp rx=0 tx=0).stop 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 -- 172.21.15.115:0/633735039 shutdown_connections 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e606ac80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e0e6079f40 0x55e0e607c3f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5ee34a0 0x55e0e606a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1567fc700 0 client.27095 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:46:38.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/633735039 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e606a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 -- 172.21.15.115:0/633735039 >> 172.21.15.115:0/633735039 conn(0x55e0e5f436a0 msgr2=0x55e0e605b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:38.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 -- 172.21.15.115:0/633735039 shutdown_connections 2024-05-09T05:46:38.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 1 -- 172.21.15.115:0/633735039 wait complete. 2024-05-09T05:46:38.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.533+0000 7fd1841cbb80 20 client.27095 trim_cache size 0 max 16384 2024-05-09T05:46:38.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.551+0000 7fd1841cbb80 1 Processor -- start 2024-05-09T05:46:38.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.551+0000 7fd1841cbb80 1 -- start start 2024-05-09T05:46:38.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e5ee34a0 0x55e0e60600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e6069b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e60606a0 0x55e0e6064510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701b24c0 con 0x55e0e5fcd160 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701b4090 con 0x55e0e5ee34a0 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701b2b90 con 0x55e0e60606a0 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e6069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e6069b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47506/0 (socket says 172.21.15.115:47506) 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16e7fc700 1 -- 172.21.15.115:0/3106455539 learned_addr learned my addr 172.21.15.115:0/3106455539 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e60606a0 0x55e0e6064510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16effd700 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e5ee34a0 0x55e0e60600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16e7fc700 1 -- 172.21.15.115:0/3106455539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e60606a0 msgr2=0x55e0e6064510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16e7fc700 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e60606a0 0x55e0e6064510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16e7fc700 1 -- 172.21.15.115:0/3106455539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e5ee34a0 msgr2=0x55e0e60600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16e7fc700 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e5ee34a0 0x55e0e60600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16e7fc700 1 -- 172.21.15.115:0/3106455539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e605cd00 con 0x55e0e5fcd160 2024-05-09T05:46:38.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e60606a0 0x55e0e6064510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:38.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.552+0000 7fd16e7fc700 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e6069b00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd170076ab0 tx=0x7fd158004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae960b613f6773c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:38.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.553+0000 7fd164ff9700 1 -- 172.21.15.115:0/3106455539 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158021070 con 0x55e0e5fcd160 2024-05-09T05:46:38.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.553+0000 7fd164ff9700 1 -- 172.21.15.115:0/3106455539 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd15801c460 con 0x55e0e5fcd160 2024-05-09T05:46:38.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.553+0000 7fd164ff9700 1 -- 172.21.15.115:0/3106455539 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1580116a0 con 0x55e0e5fcd160 2024-05-09T05:46:38.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3106455539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 msgr2=0x55e0e6069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:38.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e6069b00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd170076ab0 tx=0x7fd158004320 comp rx=0 tx=0).stop 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3106455539 shutdown_connections 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e5ee34a0 0x55e0e60600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e60606a0 0x55e0e6064510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/3106455539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5fcd160 0x55e0e6069b00 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3106455539 >> 172.21.15.115:0/3106455539 conn(0x55e0e5f436a0 msgr2=0x55e0e6063e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3106455539 shutdown_connections 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 -- 172.21.15.115:0/3106455539 wait complete. 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 -- start start 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.554+0000 7fd1841cbb80 1 Processor -- start 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5ee34a0 0x55e0e6072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5fcd160 0x55e0e6075730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd1841cbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e6075c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701b39c0 con 0x55e0e5ee34a0 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701b32f0 con 0x55e0e60606a0 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd1841cbb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701b0480 con 0x55e0e5fcd160 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd1841cbb80 10 client.0 _ll_register_callbacks cb 0x55e0e5df46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5ee34a0 0x55e0e6072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5ee34a0 0x55e0e6072700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47508/0 (socket says 172.21.15.115:47508) 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16effd700 1 -- 172.21.15.115:0/303661395 learned_addr learned my addr 172.21.15.115:0/303661395 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16e7fc700 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5fcd160 0x55e0e6075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16f7fe700 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e6075c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16effd700 1 -- 172.21.15.115:0/303661395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5fcd160 msgr2=0x55e0e6075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16effd700 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5fcd160 0x55e0e6075730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16effd700 1 -- 172.21.15.115:0/303661395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 msgr2=0x55e0e6075c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:38.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16effd700 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e6075c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.555+0000 7fd16effd700 1 -- 172.21.15.115:0/303661395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e6054f60 con 0x55e0e5ee34a0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd16effd700 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5ee34a0 0x55e0e6072700 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd16000b010 tx=0x7fd1600215f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96ce4a7727f2e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1567fc700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1567fc700 1 -- 172.21.15.115:0/303661395 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16001c040 con 0x55e0e5ee34a0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1567fc700 1 -- 172.21.15.115:0/303661395 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fd1600219e0 con 0x55e0e5ee34a0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1841cbb80 1 -- 172.21.15.115:0/303661395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15801c460 con 0x55e0e5ee34a0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1567fc700 1 -- 172.21.15.115:0/303661395 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd160043050 con 0x55e0e5ee34a0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1841cbb80 10 client.17793 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1841cbb80 1 -- 172.21.15.115:0/303661395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e0e60e42e0 con 0x55e0e5ee34a0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1567fc700 1 -- 172.21.15.115:0/303661395 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd15801c460 con 0x55e0e5ee34a0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1567fc700 1 -- 172.21.15.115:0/303661395 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fd160016680 con 0x55e0e5ee34a0 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1841cbb80 10 client.17793 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:38.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1841cbb80 10 client.17793 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1841cbb80 20 client.17793 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.556+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e0e6079f40 0x55e0e60e6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.557+0000 7fd1841cbb80 4 client.17793 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.557+0000 7fd1841cbb80 1 -- 172.21.15.115:0/303661395 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0e5f8aa70 con 0x55e0e6079f40 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.557+0000 7fd16e7fc700 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e0e6079f40 0x55e0e60e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.557+0000 7fd16e7fc700 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e0e6079f40 0x55e0e60e6750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9029fdff291f4821 server_cookie=a5b2c2bc5f55357f in_seq=0 out_seq=0 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.557+0000 7fd1567fc700 10 client.17793 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.557+0000 7fd1567fc700 1 -- 172.21.15.115:0/303661395 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0e5f8aa70 con 0x55e0e6079f40 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.557+0000 7fd1567fc700 10 client.17793 handle_command_reply: tid=0 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.557+0000 7fd1841cbb80 1 client.17793 shutdown 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 -- 172.21.15.115:0/303661395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5ee34a0 msgr2=0x55e0e6072700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5ee34a0 0x55e0e6072700 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd16000b010 tx=0x7fd1600215f0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 -- 172.21.15.115:0/303661395 shutdown_connections 2024-05-09T05:46:38.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0e60606a0 0x55e0e6075c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x55e0e6079f40 0x55e0e60e6750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1567fc700 0 client.17793 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:46:38.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e0e5ee34a0 0x55e0e6072700 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 --2- 172.21.15.115:0/303661395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0e5fcd160 0x55e0e6075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 -- 172.21.15.115:0/303661395 >> 172.21.15.115:0/303661395 conn(0x55e0e5f436a0 msgr2=0x55e0e605fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:38.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 -- 172.21.15.115:0/303661395 shutdown_connections 2024-05-09T05:46:38.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 1 -- 172.21.15.115:0/303661395 wait complete. 2024-05-09T05:46:38.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.558+0000 7fd1841cbb80 20 client.17793 trim_cache size 0 max 16384 2024-05-09T05:46:38.556 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:46:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 -- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd15c0b29f0 msgr2=0x7fd15c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd15c0b29f0 0x7fd15c0b4f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd160009fd0 tx=0x7fd1600091b0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 -- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 msgr2=0x7fd1701a6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 0x7fd1701a6b40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd168009990 tx=0x7fd1680047b0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 -- 172.21.15.115:0/750253762 shutdown_connections 2024-05-09T05:46:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd15c0b29f0 0x7fd15c0b4f00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd170108740 0x7fd1701a6b40 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1701034c0 0x7fd17019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 --2- 172.21.15.115:0/750253762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd170107df0 0x7fd17019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:38.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 -- 172.21.15.115:0/750253762 >> 172.21.15.115:0/750253762 conn(0x7fd1700fedc0 msgr2=0x7fd170107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:38.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 -- 172.21.15.115:0/750253762 shutdown_connections 2024-05-09T05:46:38.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:38.559+0000 7fd1757b5700 1 -- 172.21.15.115:0/750253762 wait complete. 2024-05-09T05:46:39.308 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:46:39.612 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:46:39.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:39 smithi115 ceph-mon[30349]: pgmap v261: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 MiB/s wr, 31 op/s 2024-05-09T05:46:39.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:39 smithi161 ceph-mon[33805]: pgmap v261: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 MiB/s wr, 31 op/s 2024-05-09T05:46:39.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:39 smithi195 ceph-mon[33844]: pgmap v261: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 MiB/s wr, 31 op/s 2024-05-09T05:46:41.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.442+0000 7ff2626f2700 1 -- 172.21.15.115:0/522229310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c107df0 msgr2=0x7ff25c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:41.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.442+0000 7ff2626f2700 1 --2- 172.21.15.115:0/522229310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c107df0 0x7ff25c108200 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff2440099c0 tx=0x7ff244009cd0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.443+0000 7ff2626f2700 1 -- 172.21.15.115:0/522229310 shutdown_connections 2024-05-09T05:46:41.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.443+0000 7ff2626f2700 1 --2- 172.21.15.115:0/522229310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff25c108740 0x7ff25c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.443+0000 7ff2626f2700 1 --2- 172.21.15.115:0/522229310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff25c1034c0 0x7ff25c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.443+0000 7ff2626f2700 1 --2- 172.21.15.115:0/522229310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c107df0 0x7ff25c108200 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.443+0000 7ff2626f2700 1 -- 172.21.15.115:0/522229310 >> 172.21.15.115:0/522229310 conn(0x7ff25c0fedc0 msgr2=0x7ff25c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:41.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.443+0000 7ff2626f2700 1 -- 172.21.15.115:0/522229310 shutdown_connections 2024-05-09T05:46:41.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.443+0000 7ff2626f2700 1 -- 172.21.15.115:0/522229310 wait complete. 2024-05-09T05:46:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.443+0000 7ff2626f2700 1 Processor -- start 2024-05-09T05:46:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.443+0000 7ff2626f2700 1 -- start start 2024-05-09T05:46:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff2626f2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c1034c0 0x7ff25c0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff2626f2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff25c107df0 0x7ff25c0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff2626f2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff25c108740 0x7ff25c0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff2626f2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c113200 con 0x7ff25c1034c0 2024-05-09T05:46:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff2626f2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c113080 con 0x7ff25c107df0 2024-05-09T05:46:41.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff2626f2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c113380 con 0x7ff25c108740 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff25bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c1034c0 0x7ff25c0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff25bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c1034c0 0x7ff25c0ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47520/0 (socket says 172.21.15.115:47520) 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff25bfff700 1 -- 172.21.15.115:0/3191561568 learned_addr learned my addr 172.21.15.115:0/3191561568 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff253fff700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff25c107df0 0x7ff25c0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff260c8f700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff25c108740 0x7ff25c0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff25bfff700 1 -- 172.21.15.115:0/3191561568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff25c108740 msgr2=0x7ff25c0ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff25bfff700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff25c108740 0x7ff25c0ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff25bfff700 1 -- 172.21.15.115:0/3191561568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff25c107df0 msgr2=0x7ff25c0ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff25bfff700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff25c107df0 0x7ff25c0ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff25bfff700 1 -- 172.21.15.115:0/3191561568 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c1a9060 con 0x7ff25c1034c0 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.444+0000 7ff253fff700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff25c107df0 0x7ff25c0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.445+0000 7ff25bfff700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c1034c0 0x7ff25c0ff2a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00d950 tx=0x7ff24c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.445+0000 7ff259ffb700 1 -- 172.21.15.115:0/3191561568 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c016070 con 0x7ff25c1034c0 2024-05-09T05:46:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.445+0000 7ff2626f2700 1 -- 172.21.15.115:0/3191561568 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c1a9350 con 0x7ff25c1034c0 2024-05-09T05:46:41.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.445+0000 7ff259ffb700 1 -- 172.21.15.115:0/3191561568 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff24c00fd20 con 0x7ff25c1034c0 2024-05-09T05:46:41.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.445+0000 7ff259ffb700 1 -- 172.21.15.115:0/3191561568 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00e3b0 con 0x7ff25c1034c0 2024-05-09T05:46:41.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.445+0000 7ff2626f2700 1 -- 172.21.15.115:0/3191561568 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c1a98b0 con 0x7ff25c1034c0 2024-05-09T05:46:41.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.448+0000 7ff259ffb700 1 -- 172.21.15.115:0/3191561568 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff24c019070 con 0x7ff25c1034c0 2024-05-09T05:46:41.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff259ffb700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2480b29f0 0x7ff2480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff253fff700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2480b29f0 0x7ff2480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff271108b80 1 Processor -- start 2024-05-09T05:46:41.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff259ffb700 1 -- 172.21.15.115:0/3191561568 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7ff24c0e23c0 con 0x7ff25c1034c0 2024-05-09T05:46:41.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff271108b80 1 -- start start 2024-05-09T05:46:41.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff271108b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fb73d00 0x55ab5fc7eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff271108b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 0x55ab5fc7f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff271108b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fc835b0 0x55ab5fc839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff253fff700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2480b29f0 0x7ff2480b4f00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff244000c00 tx=0x7ff24401a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:41.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff271108b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ab5fb990a0 con 0x55ab5fc835b0 2024-05-09T05:46:41.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff271108b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ab5fb3e9a0 con 0x55ab5fb73d00 2024-05-09T05:46:41.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff271108b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ab5fb95060 con 0x55ab5faf94a0 2024-05-09T05:46:41.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.449+0000 7ff260c8f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fc835b0 0x55ab5fc839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff260c8f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fc835b0 0x55ab5fc839c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47534/0 (socket says 172.21.15.115:47534) 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff260c8f700 1 -- 172.21.15.115:0/2478069660 learned_addr learned my addr 172.21.15.115:0/2478069660 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff253fff700 1 --2- 172.21.15.115:0/2478069660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 0x55ab5fc7f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff25bfff700 1 --2- 172.21.15.115:0/2478069660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fb73d00 0x55ab5fc7eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff260c8f700 1 -- 172.21.15.115:0/2478069660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 msgr2=0x55ab5fc7f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff260c8f700 1 --2- 172.21.15.115:0/2478069660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 0x55ab5fc7f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff260c8f700 1 -- 172.21.15.115:0/2478069660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fb73d00 msgr2=0x55ab5fc7eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff260c8f700 1 --2- 172.21.15.115:0/2478069660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fb73d00 0x55ab5fc7eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff260c8f700 1 -- 172.21.15.115:0/2478069660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5fb2f500 con 0x55ab5fc835b0 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff260c8f700 1 --2- 172.21.15.115:0/2478069660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fc835b0 0x55ab5fc839c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff254011680 tx=0x7ff254009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a103865158dc1a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff2527fc700 1 -- 172.21.15.115:0/2478069660 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254019030 con 0x55ab5fc835b0 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff2527fc700 1 -- 172.21.15.115:0/2478069660 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2540092e0 con 0x55ab5fc835b0 2024-05-09T05:46:41.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.450+0000 7ff2527fc700 1 -- 172.21.15.115:0/2478069660 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254007500 con 0x55ab5fc835b0 2024-05-09T05:46:41.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.451+0000 7ff271108b80 1 -- 172.21.15.115:0/2478069660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fc835b0 msgr2=0x55ab5fc839c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.451+0000 7ff271108b80 1 --2- 172.21.15.115:0/2478069660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fc835b0 0x55ab5fc839c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff254011680 tx=0x7ff254009920 comp rx=0 tx=0).stop 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.451+0000 7ff271108b80 1 -- 172.21.15.115:0/2478069660 shutdown_connections 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.451+0000 7ff271108b80 1 --2- 172.21.15.115:0/2478069660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fb73d00 0x55ab5fc7eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.451+0000 7ff271108b80 1 --2- 172.21.15.115:0/2478069660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fc835b0 0x55ab5fc839c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.451+0000 7ff271108b80 1 --2- 172.21.15.115:0/2478069660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 0x55ab5fc7f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.451+0000 7ff271108b80 1 -- 172.21.15.115:0/2478069660 >> 172.21.15.115:0/2478069660 conn(0x55ab5fafcf70 msgr2=0x55ab5fc71220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.451+0000 7ff271108b80 1 -- 172.21.15.115:0/2478069660 shutdown_connections 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.451+0000 7ff271108b80 1 -- 172.21.15.115:0/2478069660 wait complete. 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.452+0000 7ff271108b80 1 -- start start 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.452+0000 7ff271108b80 1 Processor -- start 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.452+0000 7ff271108b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 0x55ab5fc88810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.452+0000 7ff271108b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc88d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.452+0000 7ff271108b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fc835b0 0x55ab5fc89290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.452+0000 7ff253fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc88d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.452+0000 7ff253fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc88d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47544/0 (socket says 172.21.15.115:47544) 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff253fff700 1 -- 172.21.15.115:0/1190517475 learned_addr learned my addr 172.21.15.115:0/1190517475 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55ab5fc842d0 con 0x55ab5fb73d00 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ab5fb33b20 con 0x55ab5fc835b0 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ab5fc8b860 con 0x55ab5faf94a0 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff271108b80 10 client.0 _ll_register_callbacks cb 0x55ab5fa0a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff25bfff700 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 0x55ab5fc88810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff260c8f700 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fc835b0 0x55ab5fc89290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff253fff700 1 -- 172.21.15.115:0/1190517475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 msgr2=0x55ab5fc88810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff253fff700 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 0x55ab5fc88810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff253fff700 1 -- 172.21.15.115:0/1190517475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fc835b0 msgr2=0x55ab5fc89290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:41.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff253fff700 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fc835b0 0x55ab5fc89290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff253fff700 1 -- 172.21.15.115:0/1190517475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab5fa14490 con 0x55ab5fb73d00 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff25bfff700 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 0x55ab5fc88810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff253fff700 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc88d50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff2440099c0 tx=0x7ff244009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567e8bcbc21f2c4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.453+0000 7ff23b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff23b7fe700 1 -- 172.21.15.115:0/1190517475 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244035060 con 0x55ab5fb73d00 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff23b7fe700 1 -- 172.21.15.115:0/1190517475 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7ff24400f660 con 0x55ab5fb73d00 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2540092e0 con 0x55ab5fb73d00 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff23b7fe700 1 -- 172.21.15.115:0/1190517475 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff244003e30 con 0x55ab5fb73d00 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff271108b80 10 client.17805 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ab5fc961f0 con 0x55ab5fb73d00 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff23b7fe700 1 -- 172.21.15.115:0/1190517475 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2540092e0 con 0x55ab5fb73d00 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff23b7fe700 1 -- 172.21.15.115:0/1190517475 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7ff2440266f0 con 0x55ab5fb73d00 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff271108b80 10 client.17805 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:41.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff271108b80 10 client.17805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:46:41.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff271108b80 20 client.17805 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:41.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff271108b80 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55ab5fc98830 0x55ab5fc9acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff271108b80 4 client.17805 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:41.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.454+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab5fba6a80 con 0x55ab5fc98830 2024-05-09T05:46:41.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.455+0000 7ff25bfff700 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55ab5fc98830 0x55ab5fc9acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.455+0000 7ff23b7fe700 10 client.17805 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:46:41.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.455+0000 7ff25bfff700 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55ab5fc98830 0x55ab5fc9acb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9434c299d709f6e server_cookie=da827524ebab3c52 in_seq=0 out_seq=0 2024-05-09T05:46:41.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff23b7fe700 1 -- 172.21.15.115:0/1190517475 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ab5fba6a80 con 0x55ab5fc98830 2024-05-09T05:46:41.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff23b7fe700 10 client.17805 handle_command_reply: tid=0 2024-05-09T05:46:41.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 client.17805 shutdown 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 msgr2=0x55ab5fc88d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc88d50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff2440099c0 tx=0x7ff244009310 comp rx=0 tx=0).stop 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 shutdown_connections 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5fc835b0 0x55ab5fc89290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55ab5fc98830 0x55ab5fc9acb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5faf94a0 0x55ab5fc88810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 --2- 172.21.15.115:0/1190517475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc88d50 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 >> 172.21.15.115:0/1190517475 conn(0x55ab5fafcf70 msgr2=0x55ab5fc753f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff23b7fe700 0 client.17805 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 shutdown_connections 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.456+0000 7ff271108b80 1 -- 172.21.15.115:0/1190517475 wait complete. 2024-05-09T05:46:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.457+0000 7ff271108b80 20 client.17805 trim_cache size 0 max 16384 2024-05-09T05:46:41.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff271108b80 1 Processor -- start 2024-05-09T05:46:41.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff271108b80 1 -- start start 2024-05-09T05:46:41.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff271108b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc6ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff271108b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc94660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff271108b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fca04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff271108b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c00ecc0 con 0x55ab5fb73d00 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff271108b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c010890 con 0x55ab5faf94a0 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff271108b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c00f390 con 0x55ab5fafcf70 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff25bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc6ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff260c8f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fca04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff25bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc6ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:52404/0 (socket says 172.21.15.115:52404) 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff260c8f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fca04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47548/0 (socket says 172.21.15.115:47548) 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff25bfff700 1 -- 172.21.15.115:0/2167619427 learned_addr learned my addr 172.21.15.115:0/2167619427 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.480+0000 7ff253fff700 1 --2- 172.21.15.115:0/2167619427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc94660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff260c8f700 1 -- 172.21.15.115:0/2167619427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 msgr2=0x55ab5fc6ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff260c8f700 1 --2- 172.21.15.115:0/2167619427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc6ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff260c8f700 1 -- 172.21.15.115:0/2167619427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 msgr2=0x55ab5fc94660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:41.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff260c8f700 1 --2- 172.21.15.115:0/2167619427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc94660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff260c8f700 1 -- 172.21.15.115:0/2167619427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5fc867e0 con 0x55ab5fb73d00 2024-05-09T05:46:41.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff253fff700 1 --2- 172.21.15.115:0/2167619427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc94660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:41.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff260c8f700 1 --2- 172.21.15.115:0/2167619427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fca04b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff254000f80 tx=0x7ff254006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f253b35260f8d80d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:41.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff251ffb700 1 -- 172.21.15.115:0/2167619427 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254003d00 con 0x55ab5fb73d00 2024-05-09T05:46:41.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff251ffb700 1 -- 172.21.15.115:0/2167619427 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff254007990 con 0x55ab5fb73d00 2024-05-09T05:46:41.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.481+0000 7ff251ffb700 1 -- 172.21.15.115:0/2167619427 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25401f950 con 0x55ab5fb73d00 2024-05-09T05:46:41.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.482+0000 7ff271108b80 1 -- 172.21.15.115:0/2167619427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 msgr2=0x55ab5fca04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:41.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.482+0000 7ff271108b80 1 --2- 172.21.15.115:0/2167619427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fca04b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff254000f80 tx=0x7ff254006000 comp rx=0 tx=0).stop 2024-05-09T05:46:41.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.482+0000 7ff271108b80 1 -- 172.21.15.115:0/2167619427 shutdown_connections 2024-05-09T05:46:41.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.482+0000 7ff271108b80 1 --2- 172.21.15.115:0/2167619427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fca04b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.482+0000 7ff271108b80 1 --2- 172.21.15.115:0/2167619427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc6ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.482+0000 7ff271108b80 1 --2- 172.21.15.115:0/2167619427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc94660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.482+0000 7ff271108b80 1 -- 172.21.15.115:0/2167619427 >> 172.21.15.115:0/2167619427 conn(0x55ab5fc89490 msgr2=0x55ab5fc82e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:41.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.482+0000 7ff271108b80 1 -- 172.21.15.115:0/2167619427 shutdown_connections 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.482+0000 7ff271108b80 1 -- 172.21.15.115:0/2167619427 wait complete. 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.483+0000 7ff271108b80 1 -- start start 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.483+0000 7ff271108b80 1 Processor -- start 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff271108b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc910d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff271108b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc8e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff271108b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc8eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff271108b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0101c0 con 0x55ab5fb73d00 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff271108b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c00faf0 con 0x55ab5faf94a0 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff271108b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c00cc80 con 0x55ab5fafcf70 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff271108b80 10 client.0 _ll_register_callbacks cb 0x55ab5fa0a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff25bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc910d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff25bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc910d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:53920/0 (socket says 172.21.15.115:53920) 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff25bfff700 1 -- 172.21.15.115:0/3341628097 learned_addr learned my addr 172.21.15.115:0/3341628097 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:41.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff260c8f700 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc8eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff253fff700 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc8e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff260c8f700 1 -- 172.21.15.115:0/3341628097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 msgr2=0x55ab5fc8e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff260c8f700 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc8e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff260c8f700 1 -- 172.21.15.115:0/3341628097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 msgr2=0x55ab5fc910d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff260c8f700 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc910d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff260c8f700 1 -- 172.21.15.115:0/3341628097 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab5fc8f270 con 0x55ab5fb73d00 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.484+0000 7ff253fff700 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc8e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.485+0000 7ff25bfff700 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc910d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.485+0000 7ff260c8f700 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc8eaf0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff25401a000 tx=0x7ff254008d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613c5615fb7fd058 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:41.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.485+0000 7ff23b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.485+0000 7ff23b7fe700 1 -- 172.21.15.115:0/3341628097 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25401f590 con 0x55ab5fb73d00 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.485+0000 7ff23b7fe700 1 -- 172.21.15.115:0/3341628097 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7ff25401f730 con 0x55ab5fb73d00 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.485+0000 7ff271108b80 1 -- 172.21.15.115:0/3341628097 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff254007990 con 0x55ab5fb73d00 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.485+0000 7ff23b7fe700 1 -- 172.21.15.115:0/3341628097 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff254039050 con 0x55ab5fb73d00 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.485+0000 7ff271108b80 10 client.17811 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.485+0000 7ff271108b80 1 -- 172.21.15.115:0/3341628097 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ab5fd03100 con 0x55ab5fb73d00 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.486+0000 7ff23b7fe700 1 -- 172.21.15.115:0/3341628097 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff254007990 con 0x55ab5fb73d00 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.486+0000 7ff23b7fe700 1 -- 172.21.15.115:0/3341628097 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7ff254026020 con 0x55ab5fb73d00 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.486+0000 7ff271108b80 10 client.17811 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.486+0000 7ff271108b80 10 client.17811 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.486+0000 7ff271108b80 20 client.17811 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.486+0000 7ff271108b80 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55ab5fc835b0 0x55ab5fd05570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:41.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.486+0000 7ff271108b80 4 client.17811 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.486+0000 7ff271108b80 1 -- 172.21.15.115:0/3341628097 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab5fba0b20 con 0x55ab5fc835b0 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.486+0000 7ff25bfff700 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55ab5fc835b0 0x55ab5fd05570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff23b7fe700 10 client.17811 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff25bfff700 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55ab5fc835b0 0x55ab5fd05570 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf7490e8fddbefed server_cookie=c56eee09d5e30640 in_seq=0 out_seq=0 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff23b7fe700 1 -- 172.21.15.115:0/3341628097 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab5fba0b20 con 0x55ab5fc835b0 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff23b7fe700 10 client.17811 handle_command_reply: tid=0 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff271108b80 1 client.17811 shutdown 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff271108b80 1 -- 172.21.15.115:0/3341628097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 msgr2=0x55ab5fc8eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff271108b80 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc8eaf0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff25401a000 tx=0x7ff254008d80 comp rx=0 tx=0).stop 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff271108b80 1 -- 172.21.15.115:0/3341628097 shutdown_connections 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff271108b80 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab5faf94a0 0x55ab5fc910d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff271108b80 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x55ab5fc835b0 0x55ab5fd05570 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff271108b80 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab5fb73d00 0x55ab5fc8eaf0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff271108b80 1 --2- 172.21.15.115:0/3341628097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ab5fafcf70 0x55ab5fc8e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.487+0000 7ff271108b80 1 -- 172.21.15.115:0/3341628097 >> 172.21.15.115:0/3341628097 conn(0x55ab5fc89490 msgr2=0x55ab5fc82e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:41.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff23b7fe700 0 client.17811 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:46:41.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff271108b80 1 -- 172.21.15.115:0/3341628097 shutdown_connections 2024-05-09T05:46:41.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff271108b80 1 -- 172.21.15.115:0/3341628097 wait complete. 2024-05-09T05:46:41.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff271108b80 20 client.17811 trim_cache size 0 max 16384 2024-05-09T05:46:41.485 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:46:41.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 -- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2480b29f0 msgr2=0x7ff2480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:41.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2480b29f0 0x7ff2480b4f00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff244000c00 tx=0x7ff24401a040 comp rx=0 tx=0).stop 2024-05-09T05:46:41.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 -- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c1034c0 msgr2=0x7ff25c0ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:41.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c1034c0 0x7ff25c0ff2a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00d950 tx=0x7ff24c00dc60 comp rx=0 tx=0).stop 2024-05-09T05:46:41.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 -- 172.21.15.115:0/3191561568 shutdown_connections 2024-05-09T05:46:41.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2480b29f0 0x7ff2480b4f00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff25c108740 0x7ff25c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff25c1034c0 0x7ff25c0ff2a0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 --2- 172.21.15.115:0/3191561568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff25c107df0 0x7ff25c0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:41.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 -- 172.21.15.115:0/3191561568 >> 172.21.15.115:0/3191561568 conn(0x7ff25c0fedc0 msgr2=0x7ff25c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:41.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.488+0000 7ff2626f2700 1 -- 172.21.15.115:0/3191561568 shutdown_connections 2024-05-09T05:46:41.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:41.489+0000 7ff2626f2700 1 -- 172.21.15.115:0/3191561568 wait complete. 2024-05-09T05:46:41.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:41 smithi115 ceph-mon[30349]: pgmap v262: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:46:41.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:41 smithi161 ceph-mon[33805]: pgmap v262: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:46:41.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:41 smithi195 ceph-mon[33844]: pgmap v262: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:46:42.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:46:42.448 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:46:43.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:43 smithi115 ceph-mon[30349]: pgmap v263: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:46:43.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:43 smithi161 ceph-mon[33805]: pgmap v263: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:46:43.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:43 smithi195 ceph-mon[33844]: pgmap v263: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:46:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.177+0000 7f40141a2700 1 -- 172.21.15.115:0/1882884841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c109870 msgr2=0x7f400c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.177+0000 7f40141a2700 1 --2- 172.21.15.115:0/1882884841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c109870 0x7f400c1111d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f400400b0d0 tx=0x7f400400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.177+0000 7f40141a2700 1 -- 172.21.15.115:0/1882884841 shutdown_connections 2024-05-09T05:46:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.177+0000 7f40141a2700 1 --2- 172.21.15.115:0/1882884841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f400c1045f0 0x7f400c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.177+0000 7f40141a2700 1 --2- 172.21.15.115:0/1882884841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c109870 0x7f400c1111d0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.177+0000 7f40141a2700 1 --2- 172.21.15.115:0/1882884841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f400c108f20 0x7f400c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.177+0000 7f40141a2700 1 -- 172.21.15.115:0/1882884841 >> 172.21.15.115:0/1882884841 conn(0x7f400c100040 msgr2=0x7f400c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.177+0000 7f40141a2700 1 -- 172.21.15.115:0/1882884841 shutdown_connections 2024-05-09T05:46:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.177+0000 7f40141a2700 1 -- 172.21.15.115:0/1882884841 wait complete. 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f40141a2700 1 Processor -- start 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f40141a2700 1 -- start start 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f40141a2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c1045f0 0x7f400c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f40141a2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f400c108f20 0x7f400c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f40141a2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f400c109870 0x7f400c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f40141a2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f400c114350 con 0x7f400c1045f0 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f40141a2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1141d0 con 0x7f400c108f20 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f40141a2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1144d0 con 0x7f400c109870 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f4011f3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c1045f0 0x7f400c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f4011f3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c1045f0 0x7f400c19ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39026/0 (socket says 172.21.15.115:39026) 2024-05-09T05:46:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f4011f3e700 1 -- 172.21.15.115:0/2971893364 learned_addr learned my addr 172.21.15.115:0/2971893364 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f401273f700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f400c109870 0x7f400c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.178+0000 7f401173d700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f400c108f20 0x7f400c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f4011f3e700 1 -- 172.21.15.115:0/2971893364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f400c109870 msgr2=0x7f400c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f4011f3e700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f400c109870 0x7f400c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f4011f3e700 1 -- 172.21.15.115:0/2971893364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f400c108f20 msgr2=0x7f400c19b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f4011f3e700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f400c108f20 0x7f400c19b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f4011f3e700 1 -- 172.21.15.115:0/2971893364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400c19fbf0 con 0x7f400c1045f0 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f4011f3e700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c1045f0 0x7f400c19ad20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f400800d980 tx=0x7f400800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f4002ffd700 1 -- 172.21.15.115:0/2971893364 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008016070 con 0x7f400c1045f0 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f4002ffd700 1 -- 172.21.15.115:0/2971893364 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4008004830 con 0x7f400c1045f0 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f40141a2700 1 -- 172.21.15.115:0/2971893364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400c19fee0 con 0x7f400c1045f0 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f4002ffd700 1 -- 172.21.15.115:0/2971893364 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080054c0 con 0x7f400c1045f0 2024-05-09T05:46:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.179+0000 7f40141a2700 1 -- 172.21.15.115:0/2971893364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f400c1a7800 con 0x7f400c1045f0 2024-05-09T05:46:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.182+0000 7f4002ffd700 1 -- 172.21.15.115:0/2971893364 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f400800f690 con 0x7f400c1045f0 2024-05-09T05:46:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.183+0000 7f4002ffd700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ff80b2a70 0x7f3ff80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.183+0000 7f401173d700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ff80b2a70 0x7f3ff80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.183+0000 7f4002ffd700 1 -- 172.21.15.115:0/2971893364 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f40080e2670 con 0x7f400c1045f0 2024-05-09T05:46:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.183+0000 7f401173d700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ff80b2a70 0x7f3ff80b4f80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f400c1077f0 tx=0x7f3ffc00b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f4022bb8b80 1 Processor -- start 2024-05-09T05:46:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f4022bb8b80 1 -- start start 2024-05-09T05:46:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 0x55c8e8194240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e80114a0 0x55c8e8194780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e818e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c8e80b10a0 con 0x55c8e818e400 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c8e80569a0 con 0x55c8e80114a0 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c8e80ad060 con 0x55c8e808bd00 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f401273f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e818e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f401273f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e818e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39032/0 (socket says 172.21.15.115:39032) 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f4011f3e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 0x55c8e8194240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f401273f700 1 -- 172.21.15.115:0/21406904 learned_addr learned my addr 172.21.15.115:0/21406904 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.184+0000 7f401173d700 1 --2- 172.21.15.115:0/21406904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e80114a0 0x55c8e8194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f4011f3e700 1 -- 172.21.15.115:0/21406904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e80114a0 msgr2=0x55c8e8194780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f4011f3e700 1 --2- 172.21.15.115:0/21406904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e80114a0 0x55c8e8194780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f4011f3e700 1 -- 172.21.15.115:0/21406904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 msgr2=0x55c8e818e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f4011f3e700 1 --2- 172.21.15.115:0/21406904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e818e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f4011f3e700 1 -- 172.21.15.115:0/21406904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e8047500 con 0x55c8e808bd00 2024-05-09T05:46:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f401273f700 1 --2- 172.21.15.115:0/21406904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e818e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f4011f3e700 1 --2- 172.21.15.115:0/21406904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 0x55c8e8194240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f40080dfa30 tx=0x7f40080a9cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=321245add31a71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f3ff7fff700 1 -- 172.21.15.115:0/21406904 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080b2420 con 0x55c8e808bd00 2024-05-09T05:46:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f3ff7fff700 1 -- 172.21.15.115:0/21406904 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40080e1040 con 0x55c8e808bd00 2024-05-09T05:46:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.185+0000 7f3ff7fff700 1 -- 172.21.15.115:0/21406904 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080e2260 con 0x55c8e808bd00 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.186+0000 7f4022bb8b80 1 -- 172.21.15.115:0/21406904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 msgr2=0x55c8e8194240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.186+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/21406904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 0x55c8e8194240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f40080dfa30 tx=0x7f40080a9cc0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.186+0000 7f4022bb8b80 1 -- 172.21.15.115:0/21406904 shutdown_connections 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.186+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/21406904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e818e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.186+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/21406904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 0x55c8e8194240 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.186+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/21406904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e80114a0 0x55c8e8194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.186+0000 7f4022bb8b80 1 -- 172.21.15.115:0/21406904 >> 172.21.15.115:0/21406904 conn(0x55c8e8014f70 msgr2=0x55c8e8189220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.186+0000 7f4022bb8b80 1 -- 172.21.15.115:0/21406904 shutdown_connections 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.186+0000 7f4022bb8b80 1 -- 172.21.15.115:0/21406904 wait complete. 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.187+0000 7f4022bb8b80 1 -- start start 2024-05-09T05:46:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.187+0000 7f4022bb8b80 1 Processor -- start 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.187+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e80114a0 0x55c8e8198020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.187+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 0x55c8e8198560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.187+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.187+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c8e818f160 con 0x55c8e818e400 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.187+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c8e804bb20 con 0x55c8e808bd00 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.187+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c8e819f430 con 0x55c8e80114a0 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.187+0000 7f4022bb8b80 10 client.0 _ll_register_callbacks cb 0x55c8e7f226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4011f3e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e80114a0 0x55c8e8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4011f3e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e80114a0 0x55c8e8198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:43694/0 (socket says 172.21.15.115:43694) 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4011f3e700 1 -- 172.21.15.115:0/3851877528 learned_addr learned my addr 172.21.15.115:0/3851877528 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f401273f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f401173d700 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 0x55c8e8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4011f3e700 1 -- 172.21.15.115:0/3851877528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 msgr2=0x55c8e8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4011f3e700 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 0x55c8e8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4011f3e700 1 -- 172.21.15.115:0/3851877528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 msgr2=0x55c8e8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4011f3e700 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4011f3e700 1 -- 172.21.15.115:0/3851877528 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e7f2c490 con 0x55c8e80114a0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4011f3e700 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e80114a0 0x55c8e8198020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f40080dfae0 tx=0x7f4008004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d112954cf1c8f634 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f3ff57fa700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3851877528 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080c7070 con 0x55c8e80114a0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3851877528 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40080e1040 con 0x55c8e80114a0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.188+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3851877528 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f40080a5850 con 0x55c8e80114a0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3851877528 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f400808c050 con 0x55c8e80114a0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f4022bb8b80 10 client.27226 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3851877528 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c8e81a9e00 con 0x55c8e80114a0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3851877528 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40080e1040 con 0x55c8e80114a0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3851877528 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f40080a3ce0 con 0x55c8e80114a0 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f4022bb8b80 10 client.27226 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f4022bb8b80 10 client.27226 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:46:44.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f4022bb8b80 20 client.27226 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:44.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55c8e81ac2d0 0x55c8e81ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f4022bb8b80 4 client.27226 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:44.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.189+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3851877528 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8e80bea80 con 0x55c8e81ac2d0 2024-05-09T05:46:44.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.190+0000 7f401173d700 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55c8e81ac2d0 0x55c8e81ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.190+0000 7f3ff57fa700 10 client.27226 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:46:44.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.190+0000 7f401173d700 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55c8e81ac2d0 0x55c8e81ae6f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb5e786c80f1223f server_cookie=5e6f87165e5eb5e in_seq=0 out_seq=0 2024-05-09T05:46:44.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3851877528 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c8e80bea80 con 0x55c8e81ac2d0 2024-05-09T05:46:44.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f3ff57fa700 10 client.27226 handle_command_reply: tid=0 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f4022bb8b80 1 client.27226 shutdown 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3851877528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e80114a0 msgr2=0x55c8e8198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e80114a0 0x55c8e8198020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f40080dfae0 tx=0x7f4008004ef0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3851877528 shutdown_connections 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55c8e81ac2d0 0x55c8e81ae6f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e818e400 0x55c8e8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e80114a0 0x55c8e8198020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3851877528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 0x55c8e8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3851877528 >> 172.21.15.115:0/3851877528 conn(0x55c8e8014f70 msgr2=0x55c8e8182ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.191+0000 7f3ff57fa700 0 client.27226 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.192+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3851877528 shutdown_connections 2024-05-09T05:46:44.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.192+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3851877528 wait complete. 2024-05-09T05:46:44.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.192+0000 7f4022bb8b80 20 client.27226 trim_cache size 0 max 16384 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.210+0000 7f4022bb8b80 1 Processor -- start 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.210+0000 7f4022bb8b80 1 -- start start 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.210+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 0x55c8e8183700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e818e400 0x55c8e8195250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1ae390 con 0x55c8e80114a0 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1aff60 con 0x55c8e818e400 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1aea60 con 0x55c8e808bd00 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4011f3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4011f3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39050/0 (socket says 172.21.15.115:39050) 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4011f3e700 1 -- 172.21.15.115:0/667890754 learned_addr learned my addr 172.21.15.115:0/667890754 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f401173d700 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 0x55c8e8183700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f401273f700 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e818e400 0x55c8e8195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4011f3e700 1 -- 172.21.15.115:0/667890754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 msgr2=0x55c8e8183700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4011f3e700 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 0x55c8e8183700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4011f3e700 1 -- 172.21.15.115:0/667890754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e818e400 msgr2=0x55c8e8195250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4011f3e700 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e818e400 0x55c8e8195250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4011f3e700 1 -- 172.21.15.115:0/667890754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e81859d0 con 0x55c8e80114a0 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f401273f700 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e818e400 0x55c8e8195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f4011f3e700 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81988e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f40080a9e70 tx=0x7f40080a94c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fbef2f063dcaa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.211+0000 7f3ff77fe700 1 -- 172.21.15.115:0/667890754 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080c7070 con 0x55c8e80114a0 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.212+0000 7f3ff77fe700 1 -- 172.21.15.115:0/667890754 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40080e1030 con 0x55c8e80114a0 2024-05-09T05:46:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.212+0000 7f3ff77fe700 1 -- 172.21.15.115:0/667890754 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080e24c0 con 0x55c8e80114a0 2024-05-09T05:46:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.212+0000 7f4022bb8b80 1 -- 172.21.15.115:0/667890754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 msgr2=0x55c8e81988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.212+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81988e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f40080a9e70 tx=0x7f40080a94c0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.212+0000 7f4022bb8b80 1 -- 172.21.15.115:0/667890754 shutdown_connections 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.212+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e818e400 0x55c8e8195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.213+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81988e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.213+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/667890754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e808bd00 0x55c8e8183700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.213+0000 7f4022bb8b80 1 -- 172.21.15.115:0/667890754 >> 172.21.15.115:0/667890754 conn(0x55c8e8014f70 msgr2=0x55c8e8182ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.213+0000 7f4022bb8b80 1 -- 172.21.15.115:0/667890754 shutdown_connections 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.213+0000 7f4022bb8b80 1 -- 172.21.15.115:0/667890754 wait complete. 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.213+0000 7f4022bb8b80 1 -- start start 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.213+0000 7f4022bb8b80 1 Processor -- start 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 0x55c8e81ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4022bb8b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e818e400 0x55c8e81aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1af890 con 0x55c8e80114a0 2024-05-09T05:46:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1af1c0 con 0x55c8e808bd00 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4022bb8b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f400c1ac350 con 0x55c8e818e400 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4022bb8b80 10 client.0 _ll_register_callbacks cb 0x55c8e7f226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4011f3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4011f3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39066/0 (socket says 172.21.15.115:39066) 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4011f3e700 1 -- 172.21.15.115:0/3661690027 learned_addr learned my addr 172.21.15.115:0/3661690027 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f401273f700 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e818e400 0x55c8e81aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f401173d700 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 0x55c8e81ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4011f3e700 1 -- 172.21.15.115:0/3661690027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e818e400 msgr2=0x55c8e81aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4011f3e700 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e818e400 0x55c8e81aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4011f3e700 1 -- 172.21.15.115:0/3661690027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 msgr2=0x55c8e81ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4011f3e700 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 0x55c8e81ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4011f3e700 1 -- 172.21.15.115:0/3661690027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e82149a0 con 0x55c8e80114a0 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f401273f700 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e818e400 0x55c8e81aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f4011f3e700 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81ae0e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f40080a5640 tx=0x7f4008005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7b4dc3e0707588 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.214+0000 7f3ff57fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.215+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3661690027 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080c7040 con 0x55c8e80114a0 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.215+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3661690027 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f40080e2260 con 0x55c8e80114a0 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.215+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3661690027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40080e1030 con 0x55c8e80114a0 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.215+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3661690027 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f40080e24a0 con 0x55c8e80114a0 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.215+0000 7f4022bb8b80 10 client.17829 fetch_fsmap learned FSMap version 42 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.215+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3661690027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c8e82149a0 con 0x55c8e80114a0 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.215+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3661690027 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40080e1030 con 0x55c8e80114a0 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.215+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3661690027 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f40080a3ce0 con 0x55c8e80114a0 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.216+0000 7f4022bb8b80 10 client.17829 fetch_fsmap finished waiting for FSMap version 42 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.216+0000 7f4022bb8b80 10 client.17829 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.216+0000 7f4022bb8b80 20 client.17829 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.216+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55c8e8215eb0 0x55c8e82182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.216+0000 7f4022bb8b80 4 client.17829 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.216+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3661690027 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8e80b8b20 con 0x55c8e8215eb0 2024-05-09T05:46:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.216+0000 7f401173d700 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55c8e8215eb0 0x55c8e82182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.216+0000 7f3ff57fa700 10 client.17829 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.216+0000 7f401173d700 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55c8e8215eb0 0x55c8e82182d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e0797eb10efcd08 server_cookie=dd4d0cb365c8f9e0 in_seq=0 out_seq=0 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f3ff57fa700 1 -- 172.21.15.115:0/3661690027 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8e80b8b20 con 0x55c8e8215eb0 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f3ff57fa700 10 client.17829 handle_command_reply: tid=0 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 client.17829 shutdown 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3661690027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 msgr2=0x55c8e81ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81ae0e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f40080a5640 tx=0x7f4008005dc0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3661690027 shutdown_connections 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x55c8e8215eb0 0x55c8e82182d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c8e818e400 0x55c8e81aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c8e80114a0 0x55c8e81ae0e0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f3ff57fa700 0 client.17829 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 --2- 172.21.15.115:0/3661690027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8e808bd00 0x55c8e81ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3661690027 >> 172.21.15.115:0/3661690027 conn(0x55c8e8014f70 msgr2=0x55c8e818d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:44.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3661690027 shutdown_connections 2024-05-09T05:46:44.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 1 -- 172.21.15.115:0/3661690027 wait complete. 2024-05-09T05:46:44.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.217+0000 7f4022bb8b80 20 client.17829 trim_cache size 0 max 16384 2024-05-09T05:46:44.214 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:46:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 -- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ff80b2a70 msgr2=0x7f3ff80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ff80b2a70 0x7f3ff80b4f80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f400c1077f0 tx=0x7f3ffc00b410 comp rx=0 tx=0).stop 2024-05-09T05:46:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 -- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c1045f0 msgr2=0x7f400c19ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c1045f0 0x7f400c19ad20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f400800d980 tx=0x7f400800dc90 comp rx=0 tx=0).stop 2024-05-09T05:46:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 -- 172.21.15.115:0/2971893364 shutdown_connections 2024-05-09T05:46:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ff80b2a70 0x7f3ff80b4f80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f400c109870 0x7f400c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f400c1045f0 0x7f400c19ad20 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 --2- 172.21.15.115:0/2971893364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f400c108f20 0x7f400c19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 -- 172.21.15.115:0/2971893364 >> 172.21.15.115:0/2971893364 conn(0x7f400c100040 msgr2=0x7f400c10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 -- 172.21.15.115:0/2971893364 shutdown_connections 2024-05-09T05:46:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:44.218+0000 7f40141a2700 1 -- 172.21.15.115:0/2971893364 wait complete. 2024-05-09T05:46:45.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:46:45.454 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:46:45.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:45 smithi161 ceph-mon[33805]: pgmap v264: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 28 op/s 2024-05-09T05:46:45.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:45 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:46:45.724 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:45 smithi115 ceph-mon[30349]: pgmap v264: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 28 op/s 2024-05-09T05:46:45.724 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:45 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:46:45.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:45 smithi195 ceph-mon[33844]: pgmap v264: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 28 op/s 2024-05-09T05:46:45.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:45 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:46:47.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.236+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/3110708936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38107e30 msgr2=0x7fcd38108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:47.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/3110708936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38107e30 0x7fcd38108240 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0099c0 tx=0x7fcd2c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/3110708936 shutdown_connections 2024-05-09T05:46:47.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/3110708936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd38108780 0x7fcd381100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/3110708936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd38103550 0x7fcd38103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/3110708936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38107e30 0x7fcd38108240 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/3110708936 >> 172.21.15.115:0/3110708936 conn(0x7fcd38078e00 msgr2=0x7fcd38079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:47.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/3110708936 shutdown_connections 2024-05-09T05:46:47.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/3110708936 wait complete. 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 Processor -- start 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.237+0000 7fcd3e5fb700 1 -- start start 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd3e5fb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38103550 0x7fcd3819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd3e5fb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd38107e30 0x7fcd3819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd3e5fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd38108780 0x7fcd3819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd3e5fb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38113260 con 0x7fcd38103550 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd3e5fb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381130e0 con 0x7fcd38108780 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd3e5fb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381133e0 con 0x7fcd38107e30 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd37fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38103550 0x7fcd3819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd37fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38103550 0x7fcd3819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39094/0 (socket says 172.21.15.115:39094) 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd37fff700 1 -- 172.21.15.115:0/1089137487 learned_addr learned my addr 172.21.15.115:0/1089137487 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:47.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd377fe700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd38107e30 0x7fcd3819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd3cb98700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd38108780 0x7fcd3819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd377fe700 1 -- 172.21.15.115:0/1089137487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd38108780 msgr2=0x7fcd3819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd377fe700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd38108780 0x7fcd3819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd377fe700 1 -- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38103550 msgr2=0x7fcd3819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.238+0000 7fcd377fe700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38103550 0x7fcd3819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.239+0000 7fcd377fe700 1 -- 172.21.15.115:0/1089137487 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3819fcd0 con 0x7fcd38107e30 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.239+0000 7fcd37fff700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38103550 0x7fcd3819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.239+0000 7fcd377fe700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd38107e30 0x7fcd3819b3a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0098e0 tx=0x7fcd2c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.239+0000 7fcd357fa700 1 -- 172.21.15.115:0/1089137487 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c026070 con 0x7fcd38107e30 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.239+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/1089137487 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3819ff60 con 0x7fcd38107e30 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.239+0000 7fcd357fa700 1 -- 172.21.15.115:0/1089137487 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd2c00f460 con 0x7fcd38107e30 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.239+0000 7fcd357fa700 1 -- 172.21.15.115:0/1089137487 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c009e60 con 0x7fcd38107e30 2024-05-09T05:46:47.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.239+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/1089137487 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd381a0380 con 0x7fcd38107e30 2024-05-09T05:46:47.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.243+0000 7fcd357fa700 1 -- 172.21.15.115:0/1089137487 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcd2c029070 con 0x7fcd38107e30 2024-05-09T05:46:47.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd357fa700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcd200b29f0 0x7fcd200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd4d011b80 1 Processor -- start 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd4d011b80 1 -- start start 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd357fa700 1 -- 172.21.15.115:0/1089137487 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fcd2c0e3110 con 0x7fcd38107e30 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd37fff700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcd200b29f0 0x7fcd200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f064a4a0 0x5584f07cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f07c7400 0x5584f07c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd37fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd37fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39106/0 (socket says 172.21.15.115:39106) 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd37fff700 1 -- 172.21.15.115:0/825982820 learned_addr learned my addr 172.21.15.115:0/825982820 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd4d011b80 1 -- 172.21.15.115:0/825982820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5584f06ea0a0 con 0x5584f06c4d00 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd4d011b80 1 -- 172.21.15.115:0/825982820 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5584f068f9a0 con 0x5584f064a4a0 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd4d011b80 1 -- 172.21.15.115:0/825982820 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5584f06e6060 con 0x5584f07c7400 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.244+0000 7fcd377fe700 1 --2- 172.21.15.115:0/825982820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f064a4a0 0x5584f07cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd3cb98700 1 --2- 172.21.15.115:0/825982820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f07c7400 0x5584f07c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd37fff700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcd200b29f0 0x7fcd200b4f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd28013aa0 tx=0x7fcd28013e60 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd37fff700 1 -- 172.21.15.115:0/825982820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f07c7400 msgr2=0x5584f07c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd37fff700 1 --2- 172.21.15.115:0/825982820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f07c7400 0x5584f07c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd37fff700 1 -- 172.21.15.115:0/825982820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f064a4a0 msgr2=0x5584f07cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd37fff700 1 --2- 172.21.15.115:0/825982820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f064a4a0 0x5584f07cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd37fff700 1 -- 172.21.15.115:0/825982820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f0680500 con 0x5584f06c4d00 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd37fff700 1 --2- 172.21.15.115:0/825982820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07cd240 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800f6a0 tx=0x7fcd28020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc88b30e7351cf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd1e7fc700 1 -- 172.21.15.115:0/825982820 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28004830 con 0x5584f06c4d00 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd1e7fc700 1 -- 172.21.15.115:0/825982820 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd28024070 con 0x5584f06c4d00 2024-05-09T05:46:47.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.245+0000 7fcd1e7fc700 1 -- 172.21.15.115:0/825982820 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28020740 con 0x5584f06c4d00 2024-05-09T05:46:47.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.246+0000 7fcd4d011b80 1 -- 172.21.15.115:0/825982820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 msgr2=0x5584f07cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:47.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.246+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/825982820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07cd240 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800f6a0 tx=0x7fcd28020000 comp rx=0 tx=0).stop 2024-05-09T05:46:47.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.246+0000 7fcd4d011b80 1 -- 172.21.15.115:0/825982820 shutdown_connections 2024-05-09T05:46:47.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.246+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/825982820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f07c7400 0x5584f07c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.246+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/825982820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07cd240 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.246+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/825982820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f064a4a0 0x5584f07cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.246+0000 7fcd4d011b80 1 -- 172.21.15.115:0/825982820 >> 172.21.15.115:0/825982820 conn(0x5584f064df70 msgr2=0x5584f07c2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.246+0000 7fcd4d011b80 1 -- 172.21.15.115:0/825982820 shutdown_connections 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.246+0000 7fcd4d011b80 1 -- 172.21.15.115:0/825982820 wait complete. 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.247+0000 7fcd4d011b80 1 -- start start 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.247+0000 7fcd4d011b80 1 Processor -- start 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f07d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd4d011b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5584f07c8100 con 0x5584f06c4d00 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd4d011b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5584f0684b20 con 0x5584f07c7400 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd4d011b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5584f07dd1d0 con 0x5584f064a4a0 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd377fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd4d011b80 10 client.0 _ll_register_callbacks cb 0x5584f055b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd377fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39112/0 (socket says 172.21.15.115:39112) 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd377fe700 1 -- 172.21.15.115:0/167625738 learned_addr learned my addr 172.21.15.115:0/167625738 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd3cb98700 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f07d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd37fff700 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd37fff700 1 -- 172.21.15.115:0/167625738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 msgr2=0x5584f07d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd37fff700 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f07d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd37fff700 1 -- 172.21.15.115:0/167625738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 msgr2=0x5584f07d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd37fff700 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd37fff700 1 -- 172.21.15.115:0/167625738 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f0565490 con 0x5584f064a4a0 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.248+0000 7fcd377fe700 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.249+0000 7fcd37fff700 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07d1040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800f6a0 tx=0x7fcd28005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eafb33e6782fb74c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.249+0000 7fcd177fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.249+0000 7fcd177fe700 1 -- 172.21.15.115:0/167625738 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2802d070 con 0x5584f064a4a0 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.249+0000 7fcd177fe700 1 -- 172.21.15.115:0/167625738 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fcd28020b20 con 0x5584f064a4a0 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.249+0000 7fcd4d011b80 1 -- 172.21.15.115:0/167625738 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd28024070 con 0x5584f064a4a0 2024-05-09T05:46:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.249+0000 7fcd177fe700 1 -- 172.21.15.115:0/167625738 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcd2803b050 con 0x5584f064a4a0 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.249+0000 7fcd4d011b80 10 client.27244 fetch_fsmap learned FSMap version 43 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.249+0000 7fcd4d011b80 1 -- 172.21.15.115:0/167625738 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5584f07deb20 con 0x5584f064a4a0 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.249+0000 7fcd177fe700 1 -- 172.21.15.115:0/167625738 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd28020d20 con 0x5584f064a4a0 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.250+0000 7fcd177fe700 1 -- 172.21.15.115:0/167625738 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fcd2801f070 con 0x5584f064a4a0 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.250+0000 7fcd4d011b80 10 client.27244 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.250+0000 7fcd4d011b80 10 client.27244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.250+0000 7fcd4d011b80 20 client.27244 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.250+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5584f07e10b0 0x5584f07e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.250+0000 7fcd4d011b80 4 client.27244 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.250+0000 7fcd4d011b80 1 -- 172.21.15.115:0/167625738 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcd28024070 con 0x5584f07e10b0 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.250+0000 7fcd377fe700 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5584f07e10b0 0x5584f07e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.251+0000 7fcd177fe700 10 client.27244 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:47.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.251+0000 7fcd377fe700 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5584f07e10b0 0x5584f07e34d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6aabbfb0b8e7fb4 server_cookie=b1f678fc912ab0a in_seq=0 out_seq=0 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.251+0000 7fcd177fe700 1 -- 172.21.15.115:0/167625738 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fcd28024070 con 0x5584f07e10b0 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.251+0000 7fcd177fe700 10 client.27244 handle_command_reply: tid=0 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.251+0000 7fcd4d011b80 1 client.27244 shutdown 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 -- 172.21.15.115:0/167625738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 msgr2=0x5584f07d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07d1040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800f6a0 tx=0x7fcd28005630 comp rx=0 tx=0).stop 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 -- 172.21.15.115:0/167625738 shutdown_connections 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f07d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5584f07e10b0 0x5584f07e34d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07d1040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/167625738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd177fe700 0 client.27244 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 -- 172.21.15.115:0/167625738 >> 172.21.15.115:0/167625738 conn(0x5584f064df70 msgr2=0x5584f07bbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 -- 172.21.15.115:0/167625738 shutdown_connections 2024-05-09T05:46:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 1 -- 172.21.15.115:0/167625738 wait complete. 2024-05-09T05:46:47.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.252+0000 7fcd4d011b80 20 client.27244 trim_cache size 0 max 16384 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd4d011b80 1 Processor -- start 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd4d011b80 1 -- start start 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f07e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd4d011b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ae280 con 0x5584f06c4d00 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd4d011b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381afe80 con 0x5584f07c7400 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd4d011b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ae950 con 0x5584f064a4a0 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd377fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd377fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39122/0 (socket says 172.21.15.115:39122) 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd377fe700 1 -- 172.21.15.115:0/4228872744 learned_addr learned my addr 172.21.15.115:0/4228872744 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd3cb98700 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f07e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.273+0000 7fcd37fff700 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd377fe700 1 -- 172.21.15.115:0/4228872744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 msgr2=0x5584f07bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd377fe700 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07bca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd377fe700 1 -- 172.21.15.115:0/4228872744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 msgr2=0x5584f07e64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd377fe700 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f07e64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd377fe700 1 -- 172.21.15.115:0/4228872744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f07be9d0 con 0x5584f06c4d00 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd37fff700 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd3cb98700 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f07e64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd377fe700 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d34e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0e0ad0 tx=0x7fcd2c0a9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24426ec7f9e5064e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd1dffb700 1 -- 172.21.15.115:0/4228872744 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c06e070 con 0x5584f06c4d00 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd1dffb700 1 -- 172.21.15.115:0/4228872744 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd2c0ae4a0 con 0x5584f06c4d00 2024-05-09T05:46:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.274+0000 7fcd1dffb700 1 -- 172.21.15.115:0/4228872744 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c00f930 con 0x5584f06c4d00 2024-05-09T05:46:47.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 -- 172.21.15.115:0/4228872744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 msgr2=0x5584f07d34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:47.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d34e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0e0ad0 tx=0x7fcd2c0a9b60 comp rx=0 tx=0).stop 2024-05-09T05:46:47.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 -- 172.21.15.115:0/4228872744 shutdown_connections 2024-05-09T05:46:47.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f07e64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/4228872744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07d34e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 -- 172.21.15.115:0/4228872744 >> 172.21.15.115:0/4228872744 conn(0x5584f064df70 msgr2=0x5584f07cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:47.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 -- 172.21.15.115:0/4228872744 shutdown_connections 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 -- 172.21.15.115:0/4228872744 wait complete. 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.275+0000 7fcd4d011b80 1 -- start start 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd4d011b80 1 Processor -- start 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07e6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd4d011b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f0842c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd4d011b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381af7b0 con 0x5584f06c4d00 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd4d011b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381af0b0 con 0x5584f07c7400 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd4d011b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ac260 con 0x5584f064a4a0 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd4d011b80 10 client.0 _ll_register_callbacks cb 0x5584f07d04b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd377fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd377fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07e67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39136/0 (socket says 172.21.15.115:39136) 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd377fe700 1 -- 172.21.15.115:0/492832426 learned_addr learned my addr 172.21.15.115:0/492832426 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd3cb98700 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f0842c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.276+0000 7fcd37fff700 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd377fe700 1 -- 172.21.15.115:0/492832426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 msgr2=0x5584f07e6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd377fe700 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07e6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd377fe700 1 -- 172.21.15.115:0/492832426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 msgr2=0x5584f0842c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd377fe700 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f0842c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd377fe700 1 -- 172.21.15.115:0/492832426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f08432a0 con 0x5584f06c4d00 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd3cb98700 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f0842c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd37fff700 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd377fe700 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07e67f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0e0580 tx=0x7fcd2c0a9cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e430589ab48e4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd177fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd177fe700 1 -- 172.21.15.115:0/492832426 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c06e070 con 0x5584f06c4d00 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd177fe700 1 -- 172.21.15.115:0/492832426 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fcd2c00f930 con 0x5584f06c4d00 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd4d011b80 1 -- 172.21.15.115:0/492832426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd2c0ae4a0 con 0x5584f06c4d00 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd177fe700 1 -- 172.21.15.115:0/492832426 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcd2c0aa350 con 0x5584f06c4d00 2024-05-09T05:46:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd4d011b80 10 client.17847 fetch_fsmap learned FSMap version 43 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd4d011b80 1 -- 172.21.15.115:0/492832426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5584f084b6e0 con 0x5584f06c4d00 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.277+0000 7fcd177fe700 1 -- 172.21.15.115:0/492832426 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd2c0ae4a0 con 0x5584f06c4d00 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd177fe700 1 -- 172.21.15.115:0/492832426 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fcd2c0b2aa0 con 0x5584f06c4d00 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd4d011b80 10 client.17847 fetch_fsmap finished waiting for FSMap version 43 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd4d011b80 10 client.17847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd4d011b80 20 client.17847 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5584f084be40 0x5584f084e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd4d011b80 4 client.17847 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd4d011b80 1 -- 172.21.15.115:0/492832426 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584f06f7a80 con 0x5584f084be40 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd37fff700 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5584f084be40 0x5584f084e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd177fe700 10 client.17847 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.278+0000 7fcd37fff700 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5584f084be40 0x5584f084e260 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5132a6c4642e1db server_cookie=d78b0ca075ab357a in_seq=0 out_seq=0 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd177fe700 1 -- 172.21.15.115:0/492832426 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584f06f7a80 con 0x5584f084be40 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd177fe700 10 client.17847 handle_command_reply: tid=0 2024-05-09T05:46:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 client.17847 shutdown 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 -- 172.21.15.115:0/492832426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 msgr2=0x5584f07e67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07e67f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0e0580 tx=0x7fcd2c0a9cb0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 -- 172.21.15.115:0/492832426 shutdown_connections 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584f07c7400 0x5584f0842c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5584f084be40 0x5584f084e260 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd177fe700 0 client.17847 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584f064a4a0 0x5584f07e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 --2- 172.21.15.115:0/492832426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5584f06c4d00 0x5584f07e67f0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 -- 172.21.15.115:0/492832426 >> 172.21.15.115:0/492832426 conn(0x5584f064df70 msgr2=0x5584f07cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 -- 172.21.15.115:0/492832426 shutdown_connections 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 1 -- 172.21.15.115:0/492832426 wait complete. 2024-05-09T05:46:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.279+0000 7fcd4d011b80 20 client.17847 trim_cache size 0 max 16384 2024-05-09T05:46:47.277 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:46:47.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcd200b29f0 msgr2=0x7fcd200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:47.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcd200b29f0 0x7fcd200b4f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd28013aa0 tx=0x7fcd28013e60 comp rx=0 tx=0).stop 2024-05-09T05:46:47.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/1089137487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd38107e30 msgr2=0x7fcd3819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:47.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd38107e30 0x7fcd3819b3a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0098e0 tx=0x7fcd2c0045e0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/1089137487 shutdown_connections 2024-05-09T05:46:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcd200b29f0 0x7fcd200b4f00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd38108780 0x7fcd3819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd38103550 0x7fcd3819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 --2- 172.21.15.115:0/1089137487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd38107e30 0x7fcd3819b3a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/1089137487 >> 172.21.15.115:0/1089137487 conn(0x7fcd38078e00 msgr2=0x7fcd38075e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/1089137487 shutdown_connections 2024-05-09T05:46:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:47.280+0000 7fcd3e5fb700 1 -- 172.21.15.115:0/1089137487 wait complete. 2024-05-09T05:46:47.611 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:47 smithi115 ceph-mon[30349]: pgmap v265: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:46:47.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:47 smithi161 ceph-mon[33805]: pgmap v265: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:46:47.667 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:47 smithi195 ceph-mon[33844]: pgmap v265: 129 pgs: 129 active+clean; 6.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:46:49.025 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:46:49.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:46:49.299 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.518+0000 7ff42c18d640 1 -- 172.21.15.115:0/388833031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424109040 msgr2=0x7ff424102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.518+0000 7ff42c18d640 1 --2- 172.21.15.115:0/388833031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424109040 0x7ff424102c80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff414009900 tx=0x7ff41402a410 comp rx=0 tx=0).stop 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.518+0000 7ff42c18d640 1 -- 172.21.15.115:0/388833031 shutdown_connections 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.518+0000 7ff42c18d640 1 --2- 172.21.15.115:0/388833031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4241032f0 0x7ff42410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.518+0000 7ff42c18d640 1 --2- 172.21.15.115:0/388833031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424109040 0x7ff424102c80 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.518+0000 7ff42c18d640 1 --2- 172.21.15.115:0/388833031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff424108ba0 0x7ff424102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.518+0000 7ff42c18d640 1 -- 172.21.15.115:0/388833031 >> 172.21.15.115:0/388833031 conn(0x7ff4240782e0 msgr2=0x7ff424078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.518+0000 7ff42c18d640 1 -- 172.21.15.115:0/388833031 shutdown_connections 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.518+0000 7ff42c18d640 1 -- 172.21.15.115:0/388833031 wait complete. 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42c18d640 1 Processor -- start 2024-05-09T05:46:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42c18d640 1 -- start start 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42c18d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4241032f0 0x7ff42419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42c18d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff424108ba0 0x7ff42419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42c18d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff424109040 0x7ff42419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42c18d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff424112c40 con 0x7ff4241032f0 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42c18d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff424112ac0 con 0x7ff424108ba0 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42c18d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff424112dc0 con 0x7ff424109040 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff429f29640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4241032f0 0x7ff42419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff429f29640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4241032f0 0x7ff42419b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39140/0 (socket says 172.21.15.115:39140) 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff429f29640 1 -- 172.21.15.115:0/4108610859 learned_addr learned my addr 172.21.15.115:0/4108610859 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff429728640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff424108ba0 0x7ff42419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42a72a640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff424109040 0x7ff42419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff429f29640 1 -- 172.21.15.115:0/4108610859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff424109040 msgr2=0x7ff42419c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff429f29640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff424109040 0x7ff42419c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff429f29640 1 -- 172.21.15.115:0/4108610859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff424108ba0 msgr2=0x7ff42419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff429f29640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff424108ba0 0x7ff42419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff429f29640 1 -- 172.21.15.115:0/4108610859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4241a6950 con 0x7ff4241032f0 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.519+0000 7ff42a72a640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff424109040 0x7ff42419c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.520+0000 7ff429f29640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4241032f0 0x7ff42419b640 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ff42000e9c0 tx=0x7ff42000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.520+0000 7ff41affd640 1 -- 172.21.15.115:0/4108610859 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42000cd80 con 0x7ff4241032f0 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.520+0000 7ff42c18d640 1 -- 172.21.15.115:0/4108610859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4241a6c40 con 0x7ff4241032f0 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.520+0000 7ff41affd640 1 -- 172.21.15.115:0/4108610859 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4200054b0 con 0x7ff4241032f0 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.520+0000 7ff41affd640 1 -- 172.21.15.115:0/4108610859 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420010430 con 0x7ff4241032f0 2024-05-09T05:46:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.520+0000 7ff42c18d640 1 -- 172.21.15.115:0/4108610859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4241a71b0 con 0x7ff4241032f0 2024-05-09T05:46:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff433048740 1 Processor -- start 2024-05-09T05:46:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff433048740 1 -- start start 2024-05-09T05:46:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff41affd640 1 -- 172.21.15.115:0/4108610859 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff420004f20 con 0x7ff4241032f0 2024-05-09T05:46:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff433048740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 0x559000a61680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff433048740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006952f0 0x559000a61bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff433048740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55900069d4d0 0x559000a62100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff433048740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5590009c67c0 con 0x55900069d4d0 2024-05-09T05:46:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff433048740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5590009620e0 con 0x5590006be1b0 2024-05-09T05:46:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff433048740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5590009c2780 con 0x5590006952f0 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff429728640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55900069d4d0 0x559000a62100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.523+0000 7ff429728640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55900069d4d0 0x559000a62100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39148/0 (socket says 172.21.15.115:39148) 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff429728640 1 -- 172.21.15.115:0/3886096812 learned_addr learned my addr 172.21.15.115:0/3886096812 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff429f29640 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006952f0 0x559000a61bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff42a72a640 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 0x559000a61680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff429728640 1 -- 172.21.15.115:0/3886096812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006952f0 msgr2=0x559000a61bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff429728640 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006952f0 0x559000a61bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff429728640 1 -- 172.21.15.115:0/3886096812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 msgr2=0x559000a61680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff429728640 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 0x559000a61680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff429728640 1 -- 172.21.15.115:0/3886096812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900098b7c0 con 0x55900069d4d0 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff429f29640 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006952f0 0x559000a61bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff41affd640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff4100b2950 0x7ff4100b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff429728640 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55900069d4d0 0x559000a62100 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff4140057c0 tx=0x7ff414031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4423a7d5f257b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff42a72a640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff4100b2950 0x7ff4100b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff41affd640 1 -- 172.21.15.115:0/4108610859 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7ff4200e2f70 con 0x7ff4241032f0 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.524+0000 7ff40ffff640 1 -- 172.21.15.115:0/3886096812 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414040070 con 0x55900069d4d0 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.525+0000 7ff40ffff640 1 -- 172.21.15.115:0/3886096812 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff414004030 con 0x55900069d4d0 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.525+0000 7ff40ffff640 1 -- 172.21.15.115:0/3886096812 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414035040 con 0x55900069d4d0 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 -- 172.21.15.115:0/3886096812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55900069d4d0 msgr2=0x559000a62100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff42a72a640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff4100b2950 0x7ff4100b4e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x559000a5e7f0 tx=0x7ff41c009500 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55900069d4d0 0x559000a62100 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff4140057c0 tx=0x7ff414031a70 comp rx=0 tx=0).stop 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 -- 172.21.15.115:0/3886096812 shutdown_connections 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55900069d4d0 0x559000a62100 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006952f0 0x559000a61bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 --2- 172.21.15.115:0/3886096812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 0x559000a61680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 -- 172.21.15.115:0/3886096812 >> 172.21.15.115:0/3886096812 conn(0x5590006b2300 msgr2=0x559000a5de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 -- 172.21.15.115:0/3886096812 shutdown_connections 2024-05-09T05:46:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 -- 172.21.15.115:0/3886096812 wait complete. 2024-05-09T05:46:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 -- start start 2024-05-09T05:46:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.526+0000 7ff433048740 1 Processor -- start 2024-05-09T05:46:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff433048740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff433048740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900069d4d0 0x559000a6dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff433048740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006be1b0 0x559000a6bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff433048740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559000a6e750 con 0x5590006952f0 2024-05-09T05:46:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff433048740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55900094d9c0 con 0x55900069d4d0 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff433048740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559000a6e8d0 con 0x5590006be1b0 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff433048740 10 client.0 _ll_register_callbacks cb 0x55900097a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff429f29640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff429f29640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39158/0 (socket says 172.21.15.115:39158) 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff429f29640 1 -- 172.21.15.115:0/3007965159 learned_addr learned my addr 172.21.15.115:0/3007965159 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff429728640 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900069d4d0 0x559000a6dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff42a72a640 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006be1b0 0x559000a6bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff429f29640 1 -- 172.21.15.115:0/3007965159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006be1b0 msgr2=0x559000a6bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff429f29640 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006be1b0 0x559000a6bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff429f29640 1 -- 172.21.15.115:0/3007965159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900069d4d0 msgr2=0x559000a6dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff429f29640 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900069d4d0 0x559000a6dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.527+0000 7ff429f29640 1 -- 172.21.15.115:0/3007965159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590009741f0 con 0x5590006952f0 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff429f29640 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6d6c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff4200a9bc0 tx=0x7ff420005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fd3c8b113158a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff40d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3007965159 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4200cc040 con 0x5590006952f0 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff433048740 1 -- 172.21.15.115:0/3007965159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff414004030 con 0x5590006952f0 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3007965159 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7ff4200e6d30 con 0x5590006952f0 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3007965159 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff4200a9c90 con 0x5590006952f0 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff433048740 10 client.17859 fetch_fsmap learned FSMap version 44 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff433048740 1 -- 172.21.15.115:0/3007965159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559000729790 con 0x5590006952f0 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3007965159 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff414004030 con 0x5590006952f0 2024-05-09T05:46:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3007965159 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7ff4200a6150 con 0x5590006952f0 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff433048740 10 client.17859 fetch_fsmap finished waiting for FSMap version 44 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff433048740 10 client.17859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff433048740 20 client.17859 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff433048740 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55900067cd80 0x559000a7e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff433048740 4 client.17859 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.528+0000 7ff433048740 1 -- 172.21.15.115:0/3007965159 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590009d43d0 con 0x55900067cd80 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.529+0000 7ff429728640 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55900067cd80 0x559000a7e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.529+0000 7ff40d7fa640 10 client.17859 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.529+0000 7ff429728640 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55900067cd80 0x559000a7e5d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200492139814a3e server_cookie=de88ff9c6e6a6d7e in_seq=0 out_seq=0 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3007965159 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590009d43d0 con 0x55900067cd80 2024-05-09T05:46:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff40d7fa640 10 client.17859 handle_command_reply: tid=0 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 client.17859 shutdown 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 -- 172.21.15.115:0/3007965159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 msgr2=0x559000a6d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6d6c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff4200a9bc0 tx=0x7ff420005660 comp rx=0 tx=0).stop 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 -- 172.21.15.115:0/3007965159 shutdown_connections 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55900067cd80 0x559000a7e5d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590006be1b0 0x559000a6bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff40d7fa640 0 client.17859 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900069d4d0 0x559000a6dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 --2- 172.21.15.115:0/3007965159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6d6c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 -- 172.21.15.115:0/3007965159 >> 172.21.15.115:0/3007965159 conn(0x5590006b2300 msgr2=0x559000a633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 -- 172.21.15.115:0/3007965159 shutdown_connections 2024-05-09T05:46:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.530+0000 7ff433048740 1 -- 172.21.15.115:0/3007965159 wait complete. 2024-05-09T05:46:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.531+0000 7ff433048740 20 client.17859 trim_cache size 0 max 16384 2024-05-09T05:46:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff433048740 1 Processor -- start 2024-05-09T05:46:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff433048740 1 -- start start 2024-05-09T05:46:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff433048740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff433048740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 0x559000a639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff433048740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 0x559000a62420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff433048740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b1140 con 0x5590006952f0 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff433048740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b2da0 con 0x5590006be1b0 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff433048740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b1810 con 0x55900069d4d0 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff429f29640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff429f29640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39168/0 (socket says 172.21.15.115:39168) 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff429f29640 1 -- 172.21.15.115:0/3305309855 learned_addr learned my addr 172.21.15.115:0/3305309855 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff429728640 1 --2- 172.21.15.115:0/3305309855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 0x559000a639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff42a72a640 1 --2- 172.21.15.115:0/3305309855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 0x559000a62420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff429f29640 1 -- 172.21.15.115:0/3305309855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 msgr2=0x559000a639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff429f29640 1 --2- 172.21.15.115:0/3305309855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 0x559000a639d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff429f29640 1 -- 172.21.15.115:0/3305309855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 msgr2=0x559000a62420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff429f29640 1 --2- 172.21.15.115:0/3305309855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 0x559000a62420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.548+0000 7ff429f29640 1 -- 172.21.15.115:0/3305309855 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590007a05f0 con 0x5590006952f0 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.549+0000 7ff429f29640 1 --2- 172.21.15.115:0/3305309855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6aad0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff420005f60 tx=0x7ff420005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efd4592cfd21fff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.549+0000 7ff40f7fe640 1 -- 172.21.15.115:0/3305309855 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42006e070 con 0x5590006952f0 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.549+0000 7ff40f7fe640 1 -- 172.21.15.115:0/3305309855 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4200aa5c0 con 0x5590006952f0 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.549+0000 7ff40f7fe640 1 -- 172.21.15.115:0/3305309855 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4200cc040 con 0x5590006952f0 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 -- 172.21.15.115:0/3305309855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 msgr2=0x559000a6aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 --2- 172.21.15.115:0/3305309855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6aad0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff420005f60 tx=0x7ff420005440 comp rx=0 tx=0).stop 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 -- 172.21.15.115:0/3305309855 shutdown_connections 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 --2- 172.21.15.115:0/3305309855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006be1b0 0x559000a62420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 --2- 172.21.15.115:0/3305309855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 0x559000a639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 --2- 172.21.15.115:0/3305309855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006952f0 0x559000a6aad0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 -- 172.21.15.115:0/3305309855 >> 172.21.15.115:0/3305309855 conn(0x5590006b2300 msgr2=0x559000a632c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 -- 172.21.15.115:0/3305309855 shutdown_connections 2024-05-09T05:46:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 -- 172.21.15.115:0/3305309855 wait complete. 2024-05-09T05:46:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 -- start start 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.550+0000 7ff433048740 1 Processor -- start 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff433048740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006952f0 0x559000a6eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff433048740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 0x559000a6f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff433048740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006be1b0 0x559000a6f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff433048740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b26d0 con 0x5590006be1b0 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff433048740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b1f70 con 0x5590006952f0 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff433048740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241af140 con 0x55900069d4d0 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff433048740 10 client.0 _ll_register_callbacks cb 0x7ff420009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff42a72a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006be1b0 0x559000a6f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff42a72a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006be1b0 0x559000a6f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39182/0 (socket says 172.21.15.115:39182) 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff42a72a640 1 -- 172.21.15.115:0/3548622895 learned_addr learned my addr 172.21.15.115:0/3548622895 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff429728640 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 0x559000a6f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff429f29640 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006952f0 0x559000a6eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.551+0000 7ff42a72a640 1 -- 172.21.15.115:0/3548622895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 msgr2=0x559000a6f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff42a72a640 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 0x559000a6f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff42a72a640 1 -- 172.21.15.115:0/3548622895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006952f0 msgr2=0x559000a6eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff42a72a640 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006952f0 0x559000a6eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff42a72a640 1 -- 172.21.15.115:0/3548622895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900098aed0 con 0x5590006be1b0 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff429f29640 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006952f0 0x559000a6eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff42a72a640 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006be1b0 0x559000a6f5c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0109d0 tx=0x7ff41c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2783253b8a3300 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff40d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3548622895 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c003970 con 0x5590006be1b0 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff433048740 1 -- 172.21.15.115:0/3548622895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4200aa5c0 con 0x5590006be1b0 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3548622895 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7ff41c003b10 con 0x5590006be1b0 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3548622895 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff41c033050 con 0x5590006be1b0 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff433048740 10 client.17865 fetch_fsmap learned FSMap version 44 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff433048740 1 -- 172.21.15.115:0/3548622895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559000ae5c20 con 0x5590006be1b0 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.552+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3548622895 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4200aa5c0 con 0x5590006be1b0 2024-05-09T05:46:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3548622895 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7ff41c027070 con 0x5590006be1b0 2024-05-09T05:46:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff433048740 10 client.17865 fetch_fsmap finished waiting for FSMap version 44 2024-05-09T05:46:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff433048740 10 client.17865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff433048740 20 client.17865 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff433048740 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5590006521c0 0x559000ae84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff433048740 4 client.17865 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:46:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff433048740 1 -- 172.21.15.115:0/3548622895 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590009ce240 con 0x5590006521c0 2024-05-09T05:46:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff429f29640 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5590006521c0 0x559000ae84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff40d7fa640 10 client.17865 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.553+0000 7ff429f29640 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5590006521c0 0x559000ae84a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d431b7d8c782e90 server_cookie=e279c5f7c7849a7c in_seq=0 out_seq=0 2024-05-09T05:46:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff40d7fa640 1 -- 172.21.15.115:0/3548622895 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5590009ce240 con 0x5590006521c0 2024-05-09T05:46:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff40d7fa640 10 client.17865 handle_command_reply: tid=0 2024-05-09T05:46:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 client.17865 shutdown 2024-05-09T05:46:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 -- 172.21.15.115:0/3548622895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006be1b0 msgr2=0x559000a6f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006be1b0 0x559000a6f5c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0109d0 tx=0x7ff41c004a40 comp rx=0 tx=0).stop 2024-05-09T05:46:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 -- 172.21.15.115:0/3548622895 shutdown_connections 2024-05-09T05:46:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5590006be1b0 0x559000a6f5c0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5590006521c0 0x559000ae84a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55900069d4d0 0x559000a6f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff40d7fa640 0 client.17865 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 --2- 172.21.15.115:0/3548622895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590006952f0 0x559000a6eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 -- 172.21.15.115:0/3548622895 >> 172.21.15.115:0/3548622895 conn(0x5590006b2300 msgr2=0x559000a6d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 -- 172.21.15.115:0/3548622895 shutdown_connections 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 1 -- 172.21.15.115:0/3548622895 wait complete. 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.561+0000 7ff433048740 20 client.17865 trim_cache size 0 max 16384 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:46:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:46:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:46:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 -- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff4100b2950 msgr2=0x7ff4100b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff4100b2950 0x7ff4100b4e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x559000a5e7f0 tx=0x7ff41c009500 comp rx=0 tx=0).stop 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 -- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4241032f0 msgr2=0x7ff42419b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4241032f0 0x7ff42419b640 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ff42000e9c0 tx=0x7ff42000ee80 comp rx=0 tx=0).stop 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 -- 172.21.15.115:0/4108610859 shutdown_connections 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff4100b2950 0x7ff4100b4e60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff424109040 0x7ff42419c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff424108ba0 0x7ff42419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 --2- 172.21.15.115:0/4108610859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4241032f0 0x7ff42419b640 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 -- 172.21.15.115:0/4108610859 >> 172.21.15.115:0/4108610859 conn(0x7ff4240782e0 msgr2=0x7ff4241012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 -- 172.21.15.115:0/4108610859 shutdown_connections 2024-05-09T05:46:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:46:49.562+0000 7ff42c18d640 1 -- 172.21.15.115:0/4108610859 wait complete. 2024-05-09T05:46:49.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:49 smithi161 ceph-mon[33805]: pgmap v266: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 29 op/s 2024-05-09T05:46:49.750 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:49 smithi115 ceph-mon[30349]: pgmap v266: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 29 op/s 2024-05-09T05:46:49.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:49 smithi195 ceph-mon[33844]: pgmap v266: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 29 op/s 2024-05-09T05:46:50.731 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:50 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:46:50.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:50 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:46:50.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:50 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:46:51.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.349+0000 7f49a13ad700 1 -- 172.21.15.115:0/3245986386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c108740 msgr2=0x7f499c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:51.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.349+0000 7f49a13ad700 1 --2- 172.21.15.115:0/3245986386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c108740 0x7f499c1100a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f499400b0d0 tx=0x7f499400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.349+0000 7f49a13ad700 1 -- 172.21.15.115:0/3245986386 shutdown_connections 2024-05-09T05:46:51.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.349+0000 7f49a13ad700 1 --2- 172.21.15.115:0/3245986386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c1034c0 0x7f499c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.349+0000 7f49a13ad700 1 --2- 172.21.15.115:0/3245986386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c108740 0x7f499c1100a0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.349+0000 7f49a13ad700 1 --2- 172.21.15.115:0/3245986386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f499c107df0 0x7f499c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.349+0000 7f49a13ad700 1 -- 172.21.15.115:0/3245986386 >> 172.21.15.115:0/3245986386 conn(0x7f499c0fedc0 msgr2=0x7f499c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:51.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.349+0000 7f49a13ad700 1 -- 172.21.15.115:0/3245986386 shutdown_connections 2024-05-09T05:46:51.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.349+0000 7f49a13ad700 1 -- 172.21.15.115:0/3245986386 wait complete. 2024-05-09T05:46:51.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.350+0000 7f49a13ad700 1 Processor -- start 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.350+0000 7f49a13ad700 1 -- start start 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f49a13ad700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f499c1034c0 0x7f499c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f49a13ad700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c107df0 0x7f499c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f49a13ad700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c108740 0x7f499c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f49a13ad700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f499c113220 con 0x7f499c107df0 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f49a13ad700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1130a0 con 0x7f499c108740 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f49a13ad700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1133a0 con 0x7f499c1034c0 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c107df0 0x7f499c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c108740 0x7f499c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c107df0 0x7f499c19b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39202/0 (socket says 172.21.15.115:39202) 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499a7fc700 1 -- 172.21.15.115:0/1123157253 learned_addr learned my addr 172.21.15.115:0/1123157253 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499affd700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f499c1034c0 0x7f499c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499b7fe700 1 -- 172.21.15.115:0/1123157253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f499c1034c0 msgr2=0x7f499c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499b7fe700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f499c1034c0 0x7f499c19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499b7fe700 1 -- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c107df0 msgr2=0x7f499c19b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499b7fe700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c107df0 0x7f499c19b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499b7fe700 1 -- 172.21.15.115:0/1123157253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c19fbf0 con 0x7f499c108740 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.351+0000 7f499a7fc700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c107df0 0x7f499c19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.352+0000 7f499b7fe700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c108740 0x7f499c19f550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f499400b0d0 tx=0x7f4994007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.352+0000 7f4983fff700 1 -- 172.21.15.115:0/1123157253 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499401f070 con 0x7f499c108740 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.352+0000 7f4983fff700 1 -- 172.21.15.115:0/1123157253 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4994005500 con 0x7f499c108740 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.352+0000 7f49a13ad700 1 -- 172.21.15.115:0/1123157253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c19fe80 con 0x7f499c108740 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.352+0000 7f4983fff700 1 -- 172.21.15.115:0/1123157253 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499401b690 con 0x7f499c108740 2024-05-09T05:46:51.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.352+0000 7f49a13ad700 1 -- 172.21.15.115:0/1123157253 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c1a7800 con 0x7f499c108740 2024-05-09T05:46:51.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.355+0000 7f4983fff700 1 -- 172.21.15.115:0/1123157253 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4994022070 con 0x7f499c108740 2024-05-09T05:46:51.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.356+0000 7f4983fff700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49840b2a70 0x7f49840b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.356+0000 7f499affd700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49840b2a70 0x7f49840b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.356+0000 7f4983fff700 1 -- 172.21.15.115:0/1123157253 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f499400f020 con 0x7f499c108740 2024-05-09T05:46:51.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.356+0000 7f49afdc3b80 1 Processor -- start 2024-05-09T05:46:51.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f49afdc3b80 1 -- start start 2024-05-09T05:46:51.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499affd700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49840b2a70 0x7f49840b4f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f498c005950 tx=0x7f498c00b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:51.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b496dcd80 0x557b497e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557b49702120 con 0x557b496dcd80 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557b496a7a20 con 0x557b497e8260 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557b496fe0e0 con 0x557b49662520 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b496dcd80 0x557b497e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b496dcd80 0x557b497e3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39216/0 (socket says 172.21.15.115:39216) 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499affd700 1 -- 172.21.15.115:0/882714173 learned_addr learned my addr 172.21.15.115:0/882714173 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499b7fe700 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499a7fc700 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499affd700 1 -- 172.21.15.115:0/882714173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 msgr2=0x557b497e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499affd700 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499affd700 1 -- 172.21.15.115:0/882714173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 msgr2=0x557b497e8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499affd700 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499affd700 1 -- 172.21.15.115:0/882714173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b49698580 con 0x557b496dcd80 2024-05-09T05:46:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.357+0000 7f499a7fc700 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:51.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.358+0000 7f499b7fe700 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:51.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.358+0000 7f499affd700 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b496dcd80 0x557b497e3b50 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f498c010fa0 tx=0x7f498c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c07f47fac6a8b68 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.358+0000 7f49817fa700 1 -- 172.21.15.115:0/882714173 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c00be60 con 0x557b496dcd80 2024-05-09T05:46:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.358+0000 7f49817fa700 1 -- 172.21.15.115:0/882714173 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f498c01eab0 con 0x557b496dcd80 2024-05-09T05:46:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.358+0000 7f49817fa700 1 -- 172.21.15.115:0/882714173 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c01caa0 con 0x557b496dcd80 2024-05-09T05:46:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 -- 172.21.15.115:0/882714173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b496dcd80 msgr2=0x557b497e3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b496dcd80 0x557b497e3b50 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f498c010fa0 tx=0x7f498c013000 comp rx=0 tx=0).stop 2024-05-09T05:46:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 -- 172.21.15.115:0/882714173 shutdown_connections 2024-05-09T05:46:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b496dcd80 0x557b497e3b50 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/882714173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 -- 172.21.15.115:0/882714173 >> 172.21.15.115:0/882714173 conn(0x557b49665ff0 msgr2=0x557b497da2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 -- 172.21.15.115:0/882714173 shutdown_connections 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 -- 172.21.15.115:0/882714173 wait complete. 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.359+0000 7f49afdc3b80 1 -- start start 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f49afdc3b80 1 Processor -- start 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497ed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b497e8260 0x557b497edee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557b497e8f80 con 0x557b49662520 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557b4969cba0 con 0x557b496dcd80 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557b497f04f0 con 0x557b497e8260 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f49afdc3b80 10 client.0 _ll_register_callbacks cb 0x557b49573740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f499affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497ed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f499a7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f499a7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497ed9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:56656/0 (socket says 172.21.15.115:56656) 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f499a7fc700 1 -- 172.21.15.115:0/3783237991 learned_addr learned my addr 172.21.15.115:0/3783237991 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.360+0000 7f499b7fe700 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b497e8260 0x557b497edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f499a7fc700 1 -- 172.21.15.115:0/3783237991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b497e8260 msgr2=0x557b497edee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f499a7fc700 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b497e8260 0x557b497edee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f499a7fc700 1 -- 172.21.15.115:0/3783237991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 msgr2=0x557b497ed460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f499a7fc700 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497ed460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f499a7fc700 1 -- 172.21.15.115:0/3783237991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4957d510 con 0x557b496dcd80 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f499a7fc700 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497ed9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49900128c0 tx=0x7f4990012bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a576eee9b0c23d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f497eff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f497eff5700 1 -- 172.21.15.115:0/3783237991 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990021060 con 0x557b496dcd80 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f497eff5700 1 -- 172.21.15.115:0/3783237991 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f4990010870 con 0x557b496dcd80 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f49afdc3b80 1 -- 172.21.15.115:0/3783237991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498c01eab0 con 0x557b496dcd80 2024-05-09T05:46:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f497eff5700 1 -- 172.21.15.115:0/3783237991 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4990028050 con 0x557b496dcd80 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f49afdc3b80 10 client.27185 fetch_fsmap learned FSMap version 44 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f49afdc3b80 1 -- 172.21.15.115:0/3783237991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557b497fae90 con 0x557b496dcd80 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.361+0000 7f497eff5700 1 -- 172.21.15.115:0/3783237991 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f498c01eab0 con 0x557b496dcd80 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.362+0000 7f497eff5700 1 -- 172.21.15.115:0/3783237991 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f499000ead0 con 0x557b496dcd80 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.362+0000 7f49afdc3b80 10 client.27185 fetch_fsmap finished waiting for FSMap version 44 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.362+0000 7f49afdc3b80 10 client.27185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.362+0000 7f49afdc3b80 20 client.27185 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.362+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557b497fd4d0 0x557b497ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.362+0000 7f49afdc3b80 4 client.27185 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.362+0000 7f49afdc3b80 1 -- 172.21.15.115:0/3783237991 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4970fb00 con 0x557b497fd4d0 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.362+0000 7f499affd700 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557b497fd4d0 0x557b497ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.363+0000 7f497eff5700 10 client.27185 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.363+0000 7f499affd700 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557b497fd4d0 0x557b497ff8f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb92af214808985f server_cookie=60740df444b18fb6 in_seq=0 out_seq=0 2024-05-09T05:46:51.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.363+0000 7f497eff5700 1 -- 172.21.15.115:0/3783237991 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b4970fb00 con 0x557b497fd4d0 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.363+0000 7f497eff5700 10 client.27185 handle_command_reply: tid=0 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.363+0000 7f49afdc3b80 1 client.27185 shutdown 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 -- 172.21.15.115:0/3783237991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 msgr2=0x557b497ed9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497ed9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49900128c0 tx=0x7f4990012bd0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 -- 172.21.15.115:0/3783237991 shutdown_connections 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557b497fd4d0 0x557b497ff8f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b497e8260 0x557b497edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497ed460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f497eff5700 0 client.27185 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/3783237991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497ed9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 -- 172.21.15.115:0/3783237991 >> 172.21.15.115:0/3783237991 conn(0x557b49665ff0 msgr2=0x557b497d4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 -- 172.21.15.115:0/3783237991 shutdown_connections 2024-05-09T05:46:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 1 -- 172.21.15.115:0/3783237991 wait complete. 2024-05-09T05:46:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.364+0000 7f49afdc3b80 20 client.27185 trim_cache size 0 max 16384 2024-05-09T05:46:51.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.385+0000 7f49afdc3b80 1 Processor -- start 2024-05-09T05:46:51.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.385+0000 7f49afdc3b80 1 -- start start 2024-05-09T05:46:51.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b497e8260 0x557b497edd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1ae190 con 0x557b497e8260 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1afdf0 con 0x557b496dcd80 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1ae860 con 0x557b49662520 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b497e8260 0x557b497edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b497e8260 0x557b497edd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39226/0 (socket says 172.21.15.115:39226) 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499b7fe700 1 -- 172.21.15.115:0/1709322655 learned_addr learned my addr 172.21.15.115:0/1709322655 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499a7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499affd700 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499b7fe700 1 -- 172.21.15.115:0/1709322655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 msgr2=0x557b497e43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499b7fe700 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499b7fe700 1 -- 172.21.15.115:0/1709322655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 msgr2=0x557b497d4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:51.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499b7fe700 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497d4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499b7fe700 1 -- 172.21.15.115:0/1709322655 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b497e8090 con 0x557b497e8260 2024-05-09T05:46:51.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499a7fc700 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:46:51.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.386+0000 7f499affd700 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:51.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.387+0000 7f499b7fe700 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b497e8260 0x557b497edd20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f49940e10b0 tx=0x7f4994006180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520e4bef08fe9ada server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:51.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.387+0000 7f4980ff9700 1 -- 172.21.15.115:0/1709322655 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49940c2070 con 0x557b497e8260 2024-05-09T05:46:51.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.387+0000 7f4980ff9700 1 -- 172.21.15.115:0/1709322655 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49940b3b30 con 0x557b497e8260 2024-05-09T05:46:51.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.387+0000 7f4980ff9700 1 -- 172.21.15.115:0/1709322655 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49940b36a0 con 0x557b497e8260 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 -- 172.21.15.115:0/1709322655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b497e8260 msgr2=0x557b497edd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b497e8260 0x557b497edd20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f49940e10b0 tx=0x7f4994006180 comp rx=0 tx=0).stop 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 -- 172.21.15.115:0/1709322655 shutdown_connections 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b497e8260 0x557b497edd20 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b49662520 0x557b497e43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/1709322655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b496dcd80 0x557b497d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 -- 172.21.15.115:0/1709322655 >> 172.21.15.115:0/1709322655 conn(0x557b49665ff0 msgr2=0x557b497de470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 -- 172.21.15.115:0/1709322655 shutdown_connections 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 -- 172.21.15.115:0/1709322655 wait complete. 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 -- start start 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.388+0000 7f49afdc3b80 1 Processor -- start 2024-05-09T05:46:51.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b496dcd80 0x557b497e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f49afdc3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1af720 con 0x557b49662520 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1aefc0 con 0x557b497e8260 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f49afdc3b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f499c1ac150 con 0x557b496dcd80 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f49afdc3b80 10 client.0 _ll_register_callbacks cb 0x557b49573740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497e5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39228/0 (socket says 172.21.15.115:39228) 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499affd700 1 -- 172.21.15.115:0/4102749046 learned_addr learned my addr 172.21.15.115:0/4102749046 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499a7fc700 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b496dcd80 0x557b497e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499b7fe700 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499affd700 1 -- 172.21.15.115:0/4102749046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b496dcd80 msgr2=0x557b497e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499affd700 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b496dcd80 0x557b497e6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499affd700 1 -- 172.21.15.115:0/4102749046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 msgr2=0x557b497e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499affd700 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499affd700 1 -- 172.21.15.115:0/4102749046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b497f3b50 con 0x557b49662520 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499b7fe700 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f499affd700 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497e5cf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f498c010f40 tx=0x7f498c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf9ba75e0a0092 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.389+0000 7f497eff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f497eff5700 1 -- 172.21.15.115:0/4102749046 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c01ea70 con 0x557b49662520 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f497eff5700 1 -- 172.21.15.115:0/4102749046 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f498c005040 con 0x557b49662520 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f49afdc3b80 1 -- 172.21.15.115:0/4102749046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49940b3b30 con 0x557b49662520 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f497eff5700 1 -- 172.21.15.115:0/4102749046 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f498c045050 con 0x557b49662520 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f49afdc3b80 10 client.17883 fetch_fsmap learned FSMap version 44 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f49afdc3b80 1 -- 172.21.15.115:0/4102749046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557b49867fe0 con 0x557b49662520 2024-05-09T05:46:51.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f497eff5700 1 -- 172.21.15.115:0/4102749046 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49940b3b30 con 0x557b49662520 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f497eff5700 1 -- 172.21.15.115:0/4102749046 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f498c004cc0 con 0x557b49662520 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f49afdc3b80 10 client.17883 fetch_fsmap finished waiting for FSMap version 44 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f49afdc3b80 10 client.17883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f49afdc3b80 20 client.17883 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557b49868460 0x557b4986a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f49afdc3b80 4 client.17883 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.390+0000 7f49afdc3b80 1 -- 172.21.15.115:0/4102749046 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557b49709ba0 con 0x557b49868460 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.391+0000 7f499a7fc700 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557b49868460 0x557b4986a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.391+0000 7f497eff5700 10 client.17883 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.391+0000 7f499a7fc700 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557b49868460 0x557b4986a880 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5980d618cd02e44 server_cookie=ca7beac6f0f912b6 in_seq=0 out_seq=0 2024-05-09T05:46:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f497eff5700 1 -- 172.21.15.115:0/4102749046 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557b49709ba0 con 0x557b49868460 2024-05-09T05:46:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f497eff5700 10 client.17883 handle_command_reply: tid=0 2024-05-09T05:46:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 client.17883 shutdown 2024-05-09T05:46:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 -- 172.21.15.115:0/4102749046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 msgr2=0x557b497e5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497e5cf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f498c010f40 tx=0x7f498c00bf10 comp rx=0 tx=0).stop 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 -- 172.21.15.115:0/4102749046 shutdown_connections 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b497e8260 0x557b497e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x557b49868460 0x557b4986a880 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557b49662520 0x557b497e5cf0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 --2- 172.21.15.115:0/4102749046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b496dcd80 0x557b497e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f497eff5700 0 client.17883 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 -- 172.21.15.115:0/4102749046 >> 172.21.15.115:0/4102749046 conn(0x557b49665ff0 msgr2=0x557b497de470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 -- 172.21.15.115:0/4102749046 shutdown_connections 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.392+0000 7f49afdc3b80 1 -- 172.21.15.115:0/4102749046 wait complete. 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49afdc3b80 20 client.17883 trim_cache size 0 max 16384 2024-05-09T05:46:51.389 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:46:51.390 INFO:teuthology.orchestra.run.smithi115.stdout: "return_code": 0, 2024-05-09T05:46:51.390 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_tag": "ed4abb0e-5d49-4cc9-ac08-392f9d013533", 2024-05-09T05:46:51.390 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": "asynchronous" 2024-05-09T05:46:51.390 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 -- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49840b2a70 msgr2=0x7f49840b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49840b2a70 0x7f49840b4f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f498c005950 tx=0x7f498c00b410 comp rx=0 tx=0).stop 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 -- 172.21.15.115:0/1123157253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c108740 msgr2=0x7f499c19f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c108740 0x7f499c19f550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f499400b0d0 tx=0x7f4994007d90 comp rx=0 tx=0).stop 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 -- 172.21.15.115:0/1123157253 shutdown_connections 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49840b2a70 0x7f49840b4f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499c108740 0x7f499c19f550 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f499c1034c0 0x7f499c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 --2- 172.21.15.115:0/1123157253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f499c107df0 0x7f499c19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 -- 172.21.15.115:0/1123157253 >> 172.21.15.115:0/1123157253 conn(0x7f499c0fedc0 msgr2=0x7f499c06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.393+0000 7f49a13ad700 1 -- 172.21.15.115:0/1123157253 shutdown_connections 2024-05-09T05:46:51.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:51.394+0000 7f49a13ad700 1 -- 172.21.15.115:0/1123157253 wait complete. 2024-05-09T05:46:51.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:51 smithi115 ceph-mon[30349]: pgmap v267: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:46:51.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:51 smithi195 ceph-mon[33844]: pgmap v267: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:46:51.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.420+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000200 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_op.Po auth v418 ap=1 s=6532 n(v0 rc2024-05-09T05:40:54.759157+0000 b6532 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e0d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000200:[<0x100000003c7/ffsb_op.Po v418>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x20000000200:[<0x100000003c7/ffsb_op.Po v418>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.420+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fa [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rwlock.Po auth v417 ap=1 s=4760 n(v0 rc2024-05-09T05:40:53.667710+0000 b4760 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e0d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001fa:[<0x100000003c7/rwlock.Po v417>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001fa:[<0x100000003c7/rwlock.Po v417>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.420+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f6 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/main.Po auth v419 ap=1 s=8139 n(v0 rc2024-05-09T05:40:53.193481+0000 b8139 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e14000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001f6:[<0x100000003c7/main.Po v419>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001f6:[<0x100000003c7/main.Po v419>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.420+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000203 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fc.Po auth v420 ap=1 s=6578 n(v0 rc2024-05-09T05:40:55.467964+0000 b6578 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e14580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000203:[<0x100000003c7/ffsb_fc.Po v420>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x20000000203:[<0x100000003c7/ffsb_fc.Po v420>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.866 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.420+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000202 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/parser.Po auth v421 ap=1 s=6565 n(v0 rc2024-05-09T05:40:55.361222+0000 b6565 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e14b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000202:[<0x100000003c7/parser.Po v421>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x20000000202:[<0x100000003c7/parser.Po v421>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.421+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f8 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/filelist.Po auth v422 ap=1 s=8867 n(v0 rc2024-05-09T05:40:53.496818+0000 b8867 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e15080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001f8:[<0x100000003c7/filelist.Po v422>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001f8:[<0x100000003c7/filelist.Po v422>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.421+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fb [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/cirlist.Po auth v423 ap=1 s=5544 n(v0 rc2024-05-09T05:40:53.761397+0000 b5544 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e15600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001fb:[<0x100000003c7/cirlist.Po v423>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001fb:[<0x100000003c7/cirlist.Po v423>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.421+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000201 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/util.Po auth v424 ap=1 s=10402 n(v0 rc2024-05-09T05:40:54.911594+0000 b10402 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e15b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000201:[<0x100000003c7/util.Po v424>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x20000000201:[<0x100000003c7/util.Po v424>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.421+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000000 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fileops.Po auth v426 ap=1 s=8220 n(v0 rc2024-05-09T05:40:52.895380+0000 b8220 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e16680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000000:[<0x100000003c7/fileops.Po v426>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x20000000000:[<0x100000003c7/fileops.Po v426>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.421+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000205 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/list.Po auth v425 ap=1 s=3059 n(v0 rc2024-05-09T05:40:55.664108+0000 b3059 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e16100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000205:[<0x100000003c7/list.Po v425>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x20000000205:[<0x100000003c7/list.Po v425>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.421+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fd [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_tg.Po auth v427 ap=1 s=6493 n(v0 rc2024-05-09T05:40:54.339265+0000 b6493 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e16c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001fd:[<0x100000003c7/ffsb_tg.Po v427>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001fd:[<0x100000003c7/ffsb_tg.Po v427>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.421+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001ff [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_thread.Po auth v428 ap=1 s=6498 n(v0 rc2024-05-09T05:40:54.648353+0000 b6498 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e17180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001ff:[<0x100000003c7/ffsb_thread.Po v428>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001ff:[<0x100000003c7/ffsb_thread.Po v428>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.422+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fc [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rbt.Po auth v429 ap=1 s=5204 n(v0 rc2024-05-09T05:40:54.170745+0000 b5204 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e17700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001fc:[<0x100000003c7/rbt.Po v429>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001fc:[<0x100000003c7/rbt.Po v429>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.422+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fe [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fs.Po auth v430 ap=1 s=8428 n(v0 rc2024-05-09T05:40:54.543664+0000 b8428 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e22000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001fe:[<0x100000003c7/ffsb_fs.Po v430>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001fe:[<0x100000003c7/ffsb_fs.Po v430>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.422+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000204 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_stats.Po auth v431 ap=1 s=6937 n(v0 rc2024-05-09T05:40:55.601839+0000 b6937 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e22580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000204:[<0x100000003c7/ffsb_stats.Po v431>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x20000000204:[<0x100000003c7/ffsb_stats.Po v431>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.422+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f7 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fh.Po auth v432 ap=1 s=8062 n(v0 rc2024-05-09T05:40:53.326597+0000 b8062 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e22b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001f7:[<0x100000003c7/fh.Po v432>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001f7:[<0x100000003c7/fh.Po v432>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.867 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.422+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000001 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rand.Po auth v433 ap=1 s=8490 n(v0 rc2024-05-09T05:40:53.040521+0000 b8490 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e23080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000001:[<0x100000003c7/rand.Po v433>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x20000000001:[<0x100000003c7/rand.Po v433>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.868 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.422+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f9 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/metaops.Po auth v434 ap=1 s=7803 n(v0 rc2024-05-09T05:40:53.592646+0000 b7803 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e23600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x200000001f9:[<0x100000003c7/metaops.Po v434>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x200000001f9:[<0x100000003c7/metaops.Po v434>,<0x10000000007/.deps v3953>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.868 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.423+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dd [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/output.0 auth v22 ap=1 s=192926 n(v0 rc2024-05-09T05:40:45.074870+0000 b192926 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e24100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003dd:[<0x100000003db/output.0 v22>,<0x10000000007/autom4te.cache v3906>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x100000003dd:[<0x100000003db/output.0 v22>,<0x10000000007/autom4te.cache v3906>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.868 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.423+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dc [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/requests auth v34 ap=1 s=3312 n(v0 rc2024-05-09T05:40:52.489690+0000 b3312 1=1+0) caps={15339=pAsLsXs/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e24680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003dc:[<0x100000003db/requests v34>,<0x10000000007/autom4te.cache v3906>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x100000003dc:[<0x100000003db/requests v34>,<0x10000000007/autom4te.cache v3906>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.868 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.423+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003de [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/traces.0 auth v20 ap=1 s=33705 n(v0 rc2024-05-09T05:40:45.071548+0000 b33705 1=1+0) caps={15339=pAsLsXsFc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e24c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003de:[<0x100000003db/traces.0 v20>,<0x10000000007/autom4te.cache v3906>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x100000003de:[<0x100000003db/traces.0 v20>,<0x10000000007/autom4te.cache v3906>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.868 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 05:46:51 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T05:46:51.423+0000 7f7ef9744700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002b0 [4,head] /volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.git/index auth v388 ap=1 s=4539 n(v0 rc2024-05-09T05:40:38.535782+0000 b4539 1=1+0) caps={15339=pAsLsXs/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d977e2c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002b0:[<0x10000000008/index v388>,<0x10000000007/.git v3951>,<0x10000000006/ffsb v3555>,<0x10000000005/tmp v129>,<0x10000000003/client.0 v122>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v142>,<0x10000000001/sv_0 v160>,<0x10000000000/qa v150>,<0x1/volumes v141>]//[3]","memoryvalue":"(4)0x100000002b0:[<0x10000000008/index v388>,<0x10000000007/.git v3951>,<0x10000000006/ffsb v3568>,<0x10000000005/tmp v145>,<0x10000000003/client.0 v136>,<0x10000000002/0735efc5-9f8c-4467-8b69-740c3189ba00 v157>,<0x10000000001/sv_0 v154>,<0x10000000000/qa v142>,<0x1/volumes v133>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-09T05:46:51.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:51 smithi161 ceph-mon[33805]: pgmap v267: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:46:52.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:46:52.521 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:46:52.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub queued for path: / 2024-05-09T05:46:52.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:46:52.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:46:52.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:46:52.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:46:52.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x20000000200 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_op.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001fa (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rwlock.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001f6 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/main.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x20000000203 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fc.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x20000000202 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/parser.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001f8 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/filelist.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001fb (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/cirlist.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x20000000201 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/util.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x20000000000 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fileops.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.647 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x20000000205 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/list.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001fd (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_tg.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001ff (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_thread.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001fc (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rbt.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001fe (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fs.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x20000000204 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_stats.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001f7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fh.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x20000000001 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rand.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x200000001f9 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/metaops.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.648 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/output.0) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/requests) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x100000003de (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/traces.0) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: Scrub error on inode 0x100000002b0 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.git/index) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: active paths [/] 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.649 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:52 smithi115 ceph-mon[30349]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub queued for path: / 2024-05-09T05:46:52.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:46:52.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x20000000200 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_op.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001fa (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rwlock.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001f6 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/main.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x20000000203 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fc.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x20000000202 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/parser.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001f8 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/filelist.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001fb (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/cirlist.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x20000000201 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/util.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.866 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x20000000000 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fileops.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x20000000205 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/list.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001fd (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_tg.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001ff (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_thread.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001fc (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rbt.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001fe (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fs.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x20000000204 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_stats.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001f7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fh.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x20000000001 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rand.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x200000001f9 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/metaops.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/output.0) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/requests) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x100000003de (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/traces.0) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: Scrub error on inode 0x100000002b0 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.git/index) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.867 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:46:52.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: active paths [/] 2024-05-09T05:46:52.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:52 smithi195 ceph-mon[33844]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub queued for path: / 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x20000000200 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_op.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001fa (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rwlock.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001f6 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/main.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x20000000203 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fc.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x20000000202 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/parser.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001f8 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/filelist.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001fb (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/cirlist.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x20000000201 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/util.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x20000000000 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fileops.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x20000000205 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/list.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001fd (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_tg.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001ff (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_thread.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001fc (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rbt.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001fe (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fs.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x20000000204 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_stats.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001f7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fh.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x20000000001 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rand.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x200000001f9 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/metaops.Po) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/output.0) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/requests) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x100000003de (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/traces.0) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: Scrub error on inode 0x100000002b0 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.git/index) see mds.f log and `damage ls` output for details 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: active paths [/] 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:52.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:52 smithi161 ceph-mon[33805]: scrub summary: idle+waiting paths [/] 2024-05-09T05:46:53.691 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:53 smithi115 ceph-mon[30349]: pgmap v268: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:46:53.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:53 smithi195 ceph-mon[33844]: pgmap v268: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:46:53.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:53 smithi161 ceph-mon[33805]: pgmap v268: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-05-09T05:46:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.446+0000 7fe0b8c71700 1 -- 172.21.15.115:0/224271159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 msgr2=0x7fe0b4111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.446+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/224271159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 0x7fe0b4111120 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00b0a0 tx=0x7fe0ac00b3b0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.446+0000 7fe0b8c71700 1 -- 172.21.15.115:0/224271159 shutdown_connections 2024-05-09T05:46:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.446+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/224271159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 0x7fe0b4111120 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.446+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/224271159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b410a3e0 0x7fe0b410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.446+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/224271159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0b41027e0 0x7fe0b4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.446+0000 7fe0b8c71700 1 -- 172.21.15.115:0/224271159 >> 172.21.15.115:0/224271159 conn(0x7fe0b40fe6f0 msgr2=0x7fe0b4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.446+0000 7fe0b8c71700 1 -- 172.21.15.115:0/224271159 shutdown_connections 2024-05-09T05:46:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.446+0000 7fe0b8c71700 1 -- 172.21.15.115:0/224271159 wait complete. 2024-05-09T05:46:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b8c71700 1 Processor -- start 2024-05-09T05:46:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b8c71700 1 -- start start 2024-05-09T05:46:54.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b8c71700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b41027e0 0x7fe0b41a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b8c71700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 0x7fe0b41a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b8c71700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0b410a3e0 0x7fe0b419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b8c71700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b4114330 con 0x7fe0b4103130 2024-05-09T05:46:54.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b8c71700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41141b0 con 0x7fe0b410a3e0 2024-05-09T05:46:54.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b8c71700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41144b0 con 0x7fe0b41027e0 2024-05-09T05:46:54.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 0x7fe0b41a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 0x7fe0b41a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42118/0 (socket says 172.21.15.115:42118) 2024-05-09T05:46:54.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b1d9b700 1 -- 172.21.15.115:0/1764408212 learned_addr learned my addr 172.21.15.115:0/1764408212 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:54.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b259c700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b41027e0 0x7fe0b41a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.447+0000 7fe0b2d9d700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0b410a3e0 0x7fe0b419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0b1d9b700 1 -- 172.21.15.115:0/1764408212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b41027e0 msgr2=0x7fe0b41a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:54.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0b1d9b700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b41027e0 0x7fe0b41a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0b1d9b700 1 -- 172.21.15.115:0/1764408212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0b410a3e0 msgr2=0x7fe0b419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:54.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0b1d9b700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0b410a3e0 0x7fe0b419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0b1d9b700 1 -- 172.21.15.115:0/1764408212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b419b260 con 0x7fe0b4103130 2024-05-09T05:46:54.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0b1d9b700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 0x7fe0b41a0ee0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800e970 tx=0x7fe0a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:54.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0a37fe700 1 -- 172.21.15.115:0/1764408212 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8004a30 con 0x7fe0b4103130 2024-05-09T05:46:54.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0a37fe700 1 -- 172.21.15.115:0/1764408212 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0a800cc70 con 0x7fe0b4103130 2024-05-09T05:46:54.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0b8c71700 1 -- 172.21.15.115:0/1764408212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b419b4f0 con 0x7fe0b4103130 2024-05-09T05:46:54.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0a37fe700 1 -- 172.21.15.115:0/1764408212 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8005520 con 0x7fe0b4103130 2024-05-09T05:46:54.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.448+0000 7fe0b8c71700 1 -- 172.21.15.115:0/1764408212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b41a7750 con 0x7fe0b4103130 2024-05-09T05:46:54.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.451+0000 7fe0a37fe700 1 -- 172.21.15.115:0/1764408212 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fe0a8021440 con 0x7fe0b4103130 2024-05-09T05:46:54.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.452+0000 7fe0a37fe700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe09c0b2a70 0x7fe09c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.452+0000 7fe0b259c700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe09c0b2a70 0x7fe09c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.452+0000 7fe0a37fe700 1 -- 172.21.15.115:0/1764408212 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fe0a80e2b20 con 0x7fe0b4103130 2024-05-09T05:46:54.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.453+0000 7fe0c7687b80 1 Processor -- start 2024-05-09T05:46:54.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.453+0000 7fe0b259c700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe09c0b2a70 0x7fe09c0b4f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4005950 tx=0x7fe0a400b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:54.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.453+0000 7fe0c7687b80 1 -- start start 2024-05-09T05:46:54.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.453+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636ba99160 0x55636bb324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.453+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636b9af4a0 0x55636bb32a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.453+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb2cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.453+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55636ba4eff0 con 0x55636bb2c6a0 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.453+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55636b9ed7f0 con 0x55636b9af4a0 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.453+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55636ba4afb0 con 0x55636ba99160 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636ba99160 0x55636bb324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b2d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb2cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636ba99160 0x55636bb324f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:37216/0 (socket says 172.21.15.115:37216) 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b259c700 1 -- 172.21.15.115:0/2616497192 learned_addr learned my addr 172.21.15.115:0/2616497192 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b1d9b700 1 --2- 172.21.15.115:0/2616497192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636b9af4a0 0x55636bb32a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b259c700 1 -- 172.21.15.115:0/2616497192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636b9af4a0 msgr2=0x55636bb32a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b259c700 1 --2- 172.21.15.115:0/2616497192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636b9af4a0 0x55636bb32a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b259c700 1 -- 172.21.15.115:0/2616497192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 msgr2=0x55636bb2cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b259c700 1 --2- 172.21.15.115:0/2616497192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb2cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b259c700 1 -- 172.21.15.115:0/2616497192 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636b9e5370 con 0x55636ba99160 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b1d9b700 1 --2- 172.21.15.115:0/2616497192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636b9af4a0 0x55636bb32a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0b259c700 1 --2- 172.21.15.115:0/2616497192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636ba99160 0x55636bb324f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4010e60 tx=0x7fe0a4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c348d686b8d82f2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0a0ff9700 1 -- 172.21.15.115:0/2616497192 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4011db0 con 0x55636ba99160 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0a0ff9700 1 -- 172.21.15.115:0/2616497192 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0a4010e90 con 0x55636ba99160 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.454+0000 7fe0a0ff9700 1 -- 172.21.15.115:0/2616497192 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a401c6a0 con 0x55636ba99160 2024-05-09T05:46:54.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2616497192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636ba99160 msgr2=0x55636bb324f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/2616497192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636ba99160 0x55636bb324f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4010e60 tx=0x7fe0a4013000 comp rx=0 tx=0).stop 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2616497192 shutdown_connections 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/2616497192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb2cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/2616497192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636ba99160 0x55636bb324f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/2616497192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636b9af4a0 0x55636bb32a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2616497192 >> 172.21.15.115:0/2616497192 conn(0x55636ba0f6a0 msgr2=0x55636bb27400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2616497192 shutdown_connections 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2616497192 wait complete. 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.456+0000 7fe0c7687b80 1 -- start start 2024-05-09T05:46:54.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0c7687b80 1 Processor -- start 2024-05-09T05:46:54.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 0x55636bb361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636ba99160 0x55636bb366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636bb2c6a0 0x55636bb36c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55636b8caba0 con 0x55636ba99160 2024-05-09T05:46:54.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55636ba09150 con 0x55636bb2c6a0 2024-05-09T05:46:54.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55636bb391d0 con 0x55636b9af4a0 2024-05-09T05:46:54.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0c7687b80 10 client.0 _ll_register_callbacks cb 0x55636b8c06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:54.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0b1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636ba99160 0x55636bb366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0b1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636ba99160 0x55636bb366e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42146/0 (socket says 172.21.15.115:42146) 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0b1d9b700 1 -- 172.21.15.115:0/612744223 learned_addr learned my addr 172.21.15.115:0/612744223 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.457+0000 7fe0b259c700 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 0x55636bb361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe0b2d9d700 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636bb2c6a0 0x55636bb36c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe0b1d9b700 1 -- 172.21.15.115:0/612744223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 msgr2=0x55636bb361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe0b1d9b700 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 0x55636bb361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe0b1d9b700 1 -- 172.21.15.115:0/612744223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636bb2c6a0 msgr2=0x55636bb36c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe0b1d9b700 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636bb2c6a0 0x55636bb36c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe0b1d9b700 1 -- 172.21.15.115:0/612744223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636b8ca490 con 0x55636ba99160 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe0b2d9d700 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636bb2c6a0 0x55636bb36c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:46:54.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe0b1d9b700 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636ba99160 0x55636bb366e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80a9770 tx=0x7fe0a80a52d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad33ee42a3f1a593 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe099ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe099ffb700 1 -- 172.21.15.115:0/612744223 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80aa170 con 0x55636ba99160 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe0c7687b80 1 -- 172.21.15.115:0/612744223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a4010e90 con 0x55636ba99160 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.458+0000 7fe099ffb700 1 -- 172.21.15.115:0/612744223 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fe0a80a9cd0 con 0x55636ba99160 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe099ffb700 1 -- 172.21.15.115:0/612744223 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe0a800ce20 con 0x55636ba99160 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe0c7687b80 10 client.17895 fetch_fsmap learned FSMap version 44 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe0c7687b80 1 -- 172.21.15.115:0/612744223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55636bb47fe0 con 0x55636ba99160 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe099ffb700 1 -- 172.21.15.115:0/612744223 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0a4010e90 con 0x55636ba99160 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe099ffb700 1 -- 172.21.15.115:0/612744223 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7fe0a80b2440 con 0x55636ba99160 2024-05-09T05:46:54.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe0c7687b80 10 client.17895 fetch_fsmap finished waiting for FSMap version 44 2024-05-09T05:46:54.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe0c7687b80 10 client.17895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:54.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe0c7687b80 20 client.17895 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:54.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55636bb4a570 0x55636bb4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe0c7687b80 4 client.17895 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:46:54.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe0c7687b80 1 -- 172.21.15.115:0/612744223 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55636ba5c9d0 con 0x55636bb4a570 2024-05-09T05:46:54.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.459+0000 7fe0b259c700 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55636bb4a570 0x55636bb4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.460+0000 7fe099ffb700 10 client.17895 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:54.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.460+0000 7fe0b259c700 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55636bb4a570 0x55636bb4ca10 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1cc1da838669c server_cookie=559e959ae666ee3a in_seq=0 out_seq=0 2024-05-09T05:46:54.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe099ffb700 1 -- 172.21.15.115:0/612744223 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55636ba5c9d0 con 0x55636bb4a570 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe099ffb700 10 client.17895 handle_command_reply: tid=0 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 client.17895 shutdown 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 -- 172.21.15.115:0/612744223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636ba99160 msgr2=0x55636bb366e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636ba99160 0x55636bb366e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80a9770 tx=0x7fe0a80a52d0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 -- 172.21.15.115:0/612744223 shutdown_connections 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636bb2c6a0 0x55636bb36c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55636bb4a570 0x55636bb4ca10 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 0x55636bb361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe099ffb700 0 client.17895 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/612744223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636ba99160 0x55636bb366e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 -- 172.21.15.115:0/612744223 >> 172.21.15.115:0/612744223 conn(0x55636ba0f6a0 msgr2=0x55636bb271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 -- 172.21.15.115:0/612744223 shutdown_connections 2024-05-09T05:46:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 1 -- 172.21.15.115:0/612744223 wait complete. 2024-05-09T05:46:54.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.461+0000 7fe0c7687b80 20 client.17895 trim_cache size 0 max 16384 2024-05-09T05:46:54.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0c7687b80 1 Processor -- start 2024-05-09T05:46:54.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0c7687b80 1 -- start start 2024-05-09T05:46:54.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636b9af4a0 0x55636bb521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636ba99160 0x55636bb52710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636bb2c6a0 0x55636bb2fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41ae2c0 con 0x55636b9af4a0 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41aff20 con 0x55636ba99160 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41ae990 con 0x55636bb2c6a0 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0b259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636b9af4a0 0x55636bb521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0b259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636b9af4a0 0x55636bb521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42154/0 (socket says 172.21.15.115:42154) 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.487+0000 7fe0b259c700 1 -- 172.21.15.115:0/2764788494 learned_addr learned my addr 172.21.15.115:0/2764788494 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.488+0000 7fe0b259c700 1 -- 172.21.15.115:0/2764788494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636bb2c6a0 msgr2=0x55636bb2fe00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.488+0000 7fe0b259c700 1 --2- 172.21.15.115:0/2764788494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636bb2c6a0 0x55636bb2fe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.488+0000 7fe0b259c700 1 -- 172.21.15.115:0/2764788494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636ba99160 msgr2=0x55636bb52710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.488+0000 7fe0b259c700 1 --2- 172.21.15.115:0/2764788494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636ba99160 0x55636bb52710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.488+0000 7fe0b259c700 1 -- 172.21.15.115:0/2764788494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636bb34710 con 0x55636b9af4a0 2024-05-09T05:46:54.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.488+0000 7fe0b259c700 1 --2- 172.21.15.115:0/2764788494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636b9af4a0 0x55636bb521d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4011a00 tx=0x7fe0a400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17907883bd9ca31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:54.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.488+0000 7fe09bfff700 1 -- 172.21.15.115:0/2764788494 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4011dd0 con 0x55636b9af4a0 2024-05-09T05:46:54.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.488+0000 7fe09bfff700 1 -- 172.21.15.115:0/2764788494 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0a40054d0 con 0x55636b9af4a0 2024-05-09T05:46:54.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.488+0000 7fe09bfff700 1 -- 172.21.15.115:0/2764788494 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a401e420 con 0x55636b9af4a0 2024-05-09T05:46:54.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.489+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2764788494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636b9af4a0 msgr2=0x55636bb521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:54.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.489+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/2764788494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636b9af4a0 0x55636bb521d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4011a00 tx=0x7fe0a400b630 comp rx=0 tx=0).stop 2024-05-09T05:46:54.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.489+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2764788494 shutdown_connections 2024-05-09T05:46:54.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.489+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/2764788494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636bb2c6a0 0x55636bb2fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.489+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/2764788494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636b9af4a0 0x55636bb521d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.490+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/2764788494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636ba99160 0x55636bb52710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.490+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2764788494 >> 172.21.15.115:0/2764788494 conn(0x55636ba0f6a0 msgr2=0x55636bb324f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.490+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2764788494 shutdown_connections 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.490+0000 7fe0c7687b80 1 -- 172.21.15.115:0/2764788494 wait complete. 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.490+0000 7fe0c7687b80 1 -- start start 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.490+0000 7fe0c7687b80 1 Processor -- start 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 0x55636bb41110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636ba99160 0x55636bb41650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0c7687b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb41b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41af850 con 0x55636bb2c6a0 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41af0f0 con 0x55636ba99160 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0c7687b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41ac2a0 con 0x55636b9af4a0 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0c7687b80 10 client.0 _ll_register_callbacks cb 0x55636b8c06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:46:54.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b2d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb41b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b2d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb41b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42168/0 (socket says 172.21.15.115:42168) 2024-05-09T05:46:54.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b2d9d700 1 -- 172.21.15.115:0/593475914 learned_addr learned my addr 172.21.15.115:0/593475914 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:46:54.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b259c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 0x55636bb41110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b1d9b700 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636ba99160 0x55636bb41650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b2d9d700 1 -- 172.21.15.115:0/593475914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 msgr2=0x55636bb41110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b2d9d700 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 0x55636bb41110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b2d9d700 1 -- 172.21.15.115:0/593475914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636ba99160 msgr2=0x55636bb41650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b2d9d700 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636ba99160 0x55636bb41650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b2d9d700 1 -- 172.21.15.115:0/593475914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636bbb3e30 con 0x55636bb2c6a0 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe0b2d9d700 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb41b90 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00b0a0 tx=0x7fe0ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1ab42229e7d315 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.491+0000 7fe099ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe099ffb700 1 -- 172.21.15.115:0/593475914 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac003c00 con 0x55636bb2c6a0 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe099ffb700 1 -- 172.21.15.115:0/593475914 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fe0ac003da0 con 0x55636bb2c6a0 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe0c7687b80 1 -- 172.21.15.115:0/593475914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a40054d0 con 0x55636bb2c6a0 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe099ffb700 1 -- 172.21.15.115:0/593475914 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe0ac0075c0 con 0x55636bb2c6a0 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe0c7687b80 10 client.17901 fetch_fsmap learned FSMap version 44 2024-05-09T05:46:54.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe0c7687b80 1 -- 172.21.15.115:0/593475914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55636bbb50c0 con 0x55636bb2c6a0 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe099ffb700 1 -- 172.21.15.115:0/593475914 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0a40054d0 con 0x55636bb2c6a0 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe099ffb700 1 -- 172.21.15.115:0/593475914 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7fe0ac011070 con 0x55636bb2c6a0 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe0c7687b80 10 client.17901 fetch_fsmap finished waiting for FSMap version 44 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe0c7687b80 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe0c7687b80 20 client.17901 populate_metadata read hostname 'smithi115' 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55636bb4a570 0x55636bbb7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.492+0000 7fe0c7687b80 4 client.17901 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.493+0000 7fe0c7687b80 1 -- 172.21.15.115:0/593475914 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55636ba56a70 con 0x55636bb4a570 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.493+0000 7fe0b259c700 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55636bb4a570 0x55636bbb7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.493+0000 7fe099ffb700 10 client.17901 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.493+0000 7fe0b259c700 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55636bb4a570 0x55636bbb7970 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2922856d8e4247a5 server_cookie=d2e37932dcc98a8d in_seq=0 out_seq=0 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe099ffb700 1 -- 172.21.15.115:0/593475914 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55636ba56a70 con 0x55636bb4a570 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe099ffb700 10 client.17901 handle_command_reply: tid=0 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 client.17901 shutdown 2024-05-09T05:46:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 -- 172.21.15.115:0/593475914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 msgr2=0x55636bb41b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb41b90 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00b0a0 tx=0x7fe0ac00be20 comp rx=0 tx=0).stop 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 -- 172.21.15.115:0/593475914 shutdown_connections 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55636bb4a570 0x55636bbb7970 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55636bb2c6a0 0x55636bb41b90 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55636b9af4a0 0x55636bb41110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe099ffb700 0 client.17901 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 --2- 172.21.15.115:0/593475914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55636ba99160 0x55636bb41650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 -- 172.21.15.115:0/593475914 >> 172.21.15.115:0/593475914 conn(0x55636ba0f6a0 msgr2=0x55636bb2fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 -- 172.21.15.115:0/593475914 shutdown_connections 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 1 -- 172.21.15.115:0/593475914 wait complete. 2024-05-09T05:46:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.494+0000 7fe0c7687b80 20 client.17901 trim_cache size 0 max 16384 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": { 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "ed4abb0e-5d49-4cc9-ac08-392f9d013533": { 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/", 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tag": "ed4abb0e-5d49-4cc9-ac08-392f9d013533", 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "options": "recursive,force" 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:46:54.491 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 -- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe09c0b2a70 msgr2=0x7fe09c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe09c0b2a70 0x7fe09c0b4f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4005950 tx=0x7fe0a400b410 comp rx=0 tx=0).stop 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 -- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 msgr2=0x7fe0b41a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 0x7fe0b41a0ee0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800e970 tx=0x7fe0a800ec80 comp rx=0 tx=0).stop 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 -- 172.21.15.115:0/1764408212 shutdown_connections 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe09c0b2a70 0x7fe09c0b4f80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0b410a3e0 0x7fe0b419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0b41027e0 0x7fe0b41a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 --2- 172.21.15.115:0/1764408212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0b4103130 0x7fe0b41a0ee0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 -- 172.21.15.115:0/1764408212 >> 172.21.15.115:0/1764408212 conn(0x7fe0b40fe6f0 msgr2=0x7fe0b4100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 -- 172.21.15.115:0/1764408212 shutdown_connections 2024-05-09T05:46:54.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:46:54.495+0000 7fe0b8c71700 1 -- 172.21.15.115:0/1764408212 wait complete. 2024-05-09T05:46:55.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:55 smithi195 ceph-mon[33844]: pgmap v269: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 34 MiB/s wr, 53 op/s 2024-05-09T05:46:55.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:55 smithi115 ceph-mon[30349]: pgmap v269: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 34 MiB/s wr, 53 op/s 2024-05-09T05:46:55.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:55 smithi161 ceph-mon[33805]: pgmap v269: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 34 MiB/s wr, 53 op/s 2024-05-09T05:46:56.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:56 smithi195 ceph-mon[33844]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-09T05:46:56.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:56 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] up:active 2024-05-09T05:46:56.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:56 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:46:56.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:56 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a[30345]: 2024-05-09T05:46:56.436+0000 7f5c15a39700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-09T05:46:56.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:56 smithi115 ceph-mon[30349]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-09T05:46:56.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:56 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] up:active 2024-05-09T05:46:56.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:56 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:46:56.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:56 smithi161 ceph-mon[33805]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-09T05:46:56.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:56 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] up:active 2024-05-09T05:46:56.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:56 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:46:57.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:57 smithi115 ceph-mon[30349]: pgmap v270: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:46:57.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:57 smithi195 ceph-mon[33844]: pgmap v270: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:46:57.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:57 smithi161 ceph-mon[33805]: pgmap v270: 129 pgs: 129 active+clean; 7.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 23 MiB/s wr, 45 op/s 2024-05-09T05:46:59.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:46:59 smithi195 ceph-mon[33844]: pgmap v271: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 33 MiB/s wr, 53 op/s 2024-05-09T05:46:59.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:46:59 smithi115 ceph-mon[30349]: pgmap v271: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 33 MiB/s wr, 53 op/s 2024-05-09T05:46:59.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:46:59 smithi161 ceph-mon[33805]: pgmap v271: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 33 MiB/s wr, 53 op/s 2024-05-09T05:47:01.351 INFO:tasks.cephfs.filesystem:scrub status for tag:ed4abb0e-5d49-4cc9-ac08-392f9d013533 - {'path': '/', 'tag': 'ed4abb0e-5d49-4cc9-ac08-392f9d013533', 'options': 'recursive,force'} 2024-05-09T05:47:01.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:01 smithi115 ceph-mon[30349]: pgmap v272: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 22 MiB/s wr, 43 op/s 2024-05-09T05:47:01.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:01 smithi195 ceph-mon[33844]: pgmap v272: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 22 MiB/s wr, 43 op/s 2024-05-09T05:47:01.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:01 smithi161 ceph-mon[33805]: pgmap v272: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 22 MiB/s wr, 43 op/s 2024-05-09T05:47:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.669+0000 7f80c2434640 1 -- 172.21.15.115:0/2620418801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc108bb0 msgr2=0x7f80bc102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.669+0000 7f80c2434640 1 --2- 172.21.15.115:0/2620418801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc108bb0 0x7f80bc102750 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009900 tx=0x7f80ac02a410 comp rx=0 tx=0).stop 2024-05-09T05:47:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.669+0000 7f80c2434640 1 -- 172.21.15.115:0/2620418801 shutdown_connections 2024-05-09T05:47:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.669+0000 7f80c2434640 1 --2- 172.21.15.115:0/2620418801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80bc103300 0x7f80bc10f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.669+0000 7f80c2434640 1 --2- 172.21.15.115:0/2620418801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80bc109050 0x7f80bc102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.669+0000 7f80c2434640 1 --2- 172.21.15.115:0/2620418801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc108bb0 0x7f80bc102750 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.669+0000 7f80c2434640 1 -- 172.21.15.115:0/2620418801 >> 172.21.15.115:0/2620418801 conn(0x7f80bc078280 msgr2=0x7f80bc0786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.669+0000 7f80c2434640 1 -- 172.21.15.115:0/2620418801 shutdown_connections 2024-05-09T05:47:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.669+0000 7f80c2434640 1 -- 172.21.15.115:0/2620418801 wait complete. 2024-05-09T05:47:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c2434640 1 Processor -- start 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c2434640 1 -- start start 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c2434640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80bc103300 0x7f80bc19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c2434640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80bc108bb0 0x7f80bc19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c2434640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc109050 0x7f80bc19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c2434640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc112c50 con 0x7f80bc109050 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c2434640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc112ad0 con 0x7f80bc103300 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c2434640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc112dd0 con 0x7f80bc108bb0 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c09d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc109050 0x7f80bc19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c09d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc109050 0x7f80bc19c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47510/0 (socket says 172.21.15.115:47510) 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80c09d1640 1 -- 172.21.15.115:0/3019671981 learned_addr learned my addr 172.21.15.115:0/3019671981 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.670+0000 7f80bbfff640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80bc103300 0x7f80bc19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80bc108bb0 0x7f80bc19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80c09d1640 1 -- 172.21.15.115:0/3019671981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80bc108bb0 msgr2=0x7f80bc19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80c09d1640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80bc108bb0 0x7f80bc19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80c09d1640 1 -- 172.21.15.115:0/3019671981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80bc103300 msgr2=0x7f80bc19b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80c09d1640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80bc103300 0x7f80bc19b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80c09d1640 1 -- 172.21.15.115:0/3019671981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc1a68f0 con 0x7f80bc109050 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80c09d1640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc109050 0x7f80bc19c0f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f80b400cce0 tx=0x7f80b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80b97fa640 1 -- 172.21.15.115:0/3019671981 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b400f450 con 0x7f80bc109050 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80c2434640 1 -- 172.21.15.115:0/3019671981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc1a6be0 con 0x7f80bc109050 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80b97fa640 1 -- 172.21.15.115:0/3019671981 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80b400fa80 con 0x7f80bc109050 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80b97fa640 1 -- 172.21.15.115:0/3019671981 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4018ad0 con 0x7f80bc109050 2024-05-09T05:47:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.671+0000 7f80c2434640 1 -- 172.21.15.115:0/3019671981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc1a7130 con 0x7f80bc109050 2024-05-09T05:47:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.674+0000 7f80b97fa640 1 -- 172.21.15.115:0/3019671981 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f80b40213f0 con 0x7f80bc109050 2024-05-09T05:47:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.674+0000 7f80c93c5740 1 Processor -- start 2024-05-09T05:47:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80c93c5740 1 -- start start 2024-05-09T05:47:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2ae2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2ae2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2ae3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80c93c5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5618d2a477c0 con 0x5618d271e4d0 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80c93c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5618d29e30e0 con 0x5618d273f1b0 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80c93c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5618d2a43780 con 0x5618d27162f0 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80b97fa640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80a40b2950 0x7f80a40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2ae3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2ae2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2ae3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47516/0 (socket says 172.21.15.115:47516) 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80bb7fe640 1 -- 172.21.15.115:0/2621704524 learned_addr learned my addr 172.21.15.115:0/2621704524 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2ae2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:60412/0 (socket says 172.21.15.115:60412) 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/2621704524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2ae2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.675+0000 7f80b97fa640 1 -- 172.21.15.115:0/3019671981 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f80b40e3200 con 0x7f80bc109050 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80c09d1640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80a40b2950 0x7f80a40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80bbfff640 1 -- 172.21.15.115:0/2621704524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 msgr2=0x5618d2ae2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80bbfff640 1 --2- 172.21.15.115:0/2621704524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2ae2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80bbfff640 1 -- 172.21.15.115:0/2621704524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 msgr2=0x5618d2ae3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80bbfff640 1 --2- 172.21.15.115:0/2621704524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2ae3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80bbfff640 1 -- 172.21.15.115:0/2621704524 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d2a0c7c0 con 0x5618d27162f0 2024-05-09T05:47:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/2621704524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2ae3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80c09d1640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80a40b2950 0x7f80a40b4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80b40a9e50 tx=0x7f80b4018400 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80bbfff640 1 --2- 172.21.15.115:0/2621704524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2ae2bc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009230 tx=0x7f80ac031c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=164253ba60078c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.676+0000 7f80a27fc640 1 -- 172.21.15.115:0/2621704524 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac035030 con 0x5618d27162f0 2024-05-09T05:47:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.677+0000 7f80a27fc640 1 -- 172.21.15.115:0/2621704524 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80ac040070 con 0x5618d27162f0 2024-05-09T05:47:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.677+0000 7f80a27fc640 1 -- 172.21.15.115:0/2621704524 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac003850 con 0x5618d27162f0 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 -- 172.21.15.115:0/2621704524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 msgr2=0x5618d2ae2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 --2- 172.21.15.115:0/2621704524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2ae2bc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009230 tx=0x7f80ac031c50 comp rx=0 tx=0).stop 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 -- 172.21.15.115:0/2621704524 shutdown_connections 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 --2- 172.21.15.115:0/2621704524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2ae3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 --2- 172.21.15.115:0/2621704524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2ae2bc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 --2- 172.21.15.115:0/2621704524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2ae2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 -- 172.21.15.115:0/2621704524 >> 172.21.15.115:0/2621704524 conn(0x5618d2733300 msgr2=0x5618d2adee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 -- 172.21.15.115:0/2621704524 shutdown_connections 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 -- 172.21.15.115:0/2621704524 wait complete. 2024-05-09T05:47:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 -- start start 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.678+0000 7f80c93c5740 1 Processor -- start 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d27162f0 0x5618d2aee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d271e4d0 0x5618d2aeecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2aecbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80c93c5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5618d2af3b60 con 0x5618d27162f0 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80c93c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5618d29ce9c0 con 0x5618d273f1b0 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80c93c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5618d2af3ce0 con 0x5618d271e4d0 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80c93c5740 10 client.0 _ll_register_callbacks cb 0x5618d29fb020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d27162f0 0x5618d2aee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d27162f0 0x5618d2aee770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47522/0 (socket says 172.21.15.115:47522) 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bb7fe640 1 -- 172.21.15.115:0/98753282 learned_addr learned my addr 172.21.15.115:0/98753282 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bbfff640 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d271e4d0 0x5618d2aeecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2aecbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bb7fe640 1 -- 172.21.15.115:0/98753282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d271e4d0 msgr2=0x5618d2aeecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d271e4d0 0x5618d2aeecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bb7fe640 1 -- 172.21.15.115:0/98753282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 msgr2=0x5618d2aecbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2aecbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bb7fe640 1 -- 172.21.15.115:0/98753282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d29f51f0 con 0x5618d27162f0 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.679+0000 7f80bbfff640 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d271e4d0 0x5618d2aeecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d27162f0 0x5618d2aee770 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f80b00143d0 tx=0x7f80b000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ba7dd96c500ae1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f809b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f809b7fe640 1 -- 172.21.15.115:0/98753282 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b000f440 con 0x5618d27162f0 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f809b7fe640 1 -- 172.21.15.115:0/98753282 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f80b000f5e0 con 0x5618d27162f0 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80c93c5740 1 -- 172.21.15.115:0/98753282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ac040070 con 0x5618d27162f0 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f809b7fe640 1 -- 172.21.15.115:0/98753282 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f80b0028050 con 0x5618d27162f0 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80c93c5740 10 client.17913 fetch_fsmap learned FSMap version 45 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80c93c5740 1 -- 172.21.15.115:0/98753282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5618d27aa790 con 0x5618d27162f0 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f809b7fe640 1 -- 172.21.15.115:0/98753282 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80ac040070 con 0x5618d27162f0 2024-05-09T05:47:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f809b7fe640 1 -- 172.21.15.115:0/98753282 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f80b0020020 con 0x5618d27162f0 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80c93c5740 10 client.17913 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80c93c5740 10 client.17913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80c93c5740 20 client.17913 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80c93c5740 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5618d26fdd80 0x5618d2b03440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80c93c5740 4 client.17913 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.680+0000 7f80c93c5740 1 -- 172.21.15.115:0/98753282 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d2a553d0 con 0x5618d26fdd80 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.681+0000 7f80bbfff640 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5618d26fdd80 0x5618d2b03440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.681+0000 7f809b7fe640 10 client.17913 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.681+0000 7f80bbfff640 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5618d26fdd80 0x5618d2b03440 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da82a1300263f57 server_cookie=1ea4996096115a20 in_seq=0 out_seq=0 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.682+0000 7f809b7fe640 1 -- 172.21.15.115:0/98753282 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5618d2a553d0 con 0x5618d26fdd80 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.682+0000 7f809b7fe640 10 client.17913 handle_command_reply: tid=0 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.682+0000 7f80c93c5740 1 client.17913 shutdown 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.682+0000 7f80c93c5740 1 -- 172.21.15.115:0/98753282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d27162f0 msgr2=0x5618d2aee770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.682+0000 7f80c93c5740 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d27162f0 0x5618d2aee770 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f80b00143d0 tx=0x7f80b000cb30 comp rx=0 tx=0).stop 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f80c93c5740 1 -- 172.21.15.115:0/98753282 shutdown_connections 2024-05-09T05:47:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f80c93c5740 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2aecbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f80c93c5740 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5618d26fdd80 0x5618d2b03440 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f80c93c5740 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d271e4d0 0x5618d2aeecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f80c93c5740 1 --2- 172.21.15.115:0/98753282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d27162f0 0x5618d2aee770 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f80c93c5740 1 -- 172.21.15.115:0/98753282 >> 172.21.15.115:0/98753282 conn(0x5618d2733300 msgr2=0x5618d2add700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f809b7fe640 0 client.17913 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f80c93c5740 1 -- 172.21.15.115:0/98753282 shutdown_connections 2024-05-09T05:47:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f80c93c5740 1 -- 172.21.15.115:0/98753282 wait complete. 2024-05-09T05:47:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.683+0000 7f80c93c5740 20 client.17913 trim_cache size 0 max 16384 2024-05-09T05:47:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80c93c5740 1 Processor -- start 2024-05-09T05:47:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80c93c5740 1 -- start start 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2adde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2b08c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2b09180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80c93c5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc1aee70 con 0x5618d271e4d0 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80c93c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc1b0ad0 con 0x5618d273f1b0 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80c93c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc1af540 con 0x5618d27162f0 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2b08c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2b08c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47536/0 (socket says 172.21.15.115:47536) 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80bbfff640 1 -- 172.21.15.115:0/1996719654 learned_addr learned my addr 172.21.15.115:0/1996719654 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/1996719654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2b09180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.703+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/1996719654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2adde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.704+0000 7f80bbfff640 1 -- 172.21.15.115:0/1996719654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 msgr2=0x5618d2adde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.704+0000 7f80bbfff640 1 --2- 172.21.15.115:0/1996719654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2adde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.704+0000 7f80bbfff640 1 -- 172.21.15.115:0/1996719654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 msgr2=0x5618d2b09180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.704+0000 7f80bbfff640 1 --2- 172.21.15.115:0/1996719654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2b09180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.704+0000 7f80bbfff640 1 -- 172.21.15.115:0/1996719654 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d28215f0 con 0x5618d271e4d0 2024-05-09T05:47:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.704+0000 7f80bbfff640 1 --2- 172.21.15.115:0/1996719654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2b08c40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0057c0 tx=0x7f80ac02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9f82fe7c277a6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.704+0000 7f80a1ffb640 1 -- 172.21.15.115:0/1996719654 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac035040 con 0x5618d271e4d0 2024-05-09T05:47:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.704+0000 7f80a1ffb640 1 -- 172.21.15.115:0/1996719654 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80ac040040 con 0x5618d271e4d0 2024-05-09T05:47:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.704+0000 7f80a1ffb640 1 -- 172.21.15.115:0/1996719654 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac003850 con 0x5618d271e4d0 2024-05-09T05:47:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 -- 172.21.15.115:0/1996719654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 msgr2=0x5618d2b08c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 --2- 172.21.15.115:0/1996719654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2b08c40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0057c0 tx=0x7f80ac02a410 comp rx=0 tx=0).stop 2024-05-09T05:47:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 -- 172.21.15.115:0/1996719654 shutdown_connections 2024-05-09T05:47:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 --2- 172.21.15.115:0/1996719654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d273f1b0 0x5618d2b09180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 --2- 172.21.15.115:0/1996719654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2b08c40 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 --2- 172.21.15.115:0/1996719654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d27162f0 0x5618d2adde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 -- 172.21.15.115:0/1996719654 >> 172.21.15.115:0/1996719654 conn(0x5618d2733300 msgr2=0x5618d2add700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 -- 172.21.15.115:0/1996719654 shutdown_connections 2024-05-09T05:47:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 -- 172.21.15.115:0/1996719654 wait complete. 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.705+0000 7f80c93c5740 1 -- start start 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80c93c5740 1 Processor -- start 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d27162f0 0x5618d2af5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2af6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80c93c5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d273f1b0 0x5618d2af6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80c93c5740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc1b0400 con 0x5618d271e4d0 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80c93c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc1afca0 con 0x5618d27162f0 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80c93c5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc1ace70 con 0x5618d273f1b0 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80c93c5740 10 client.0 _ll_register_callbacks cb 0x7f80b0014a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2af6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2af6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47546/0 (socket says 172.21.15.115:47546) 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bbfff640 1 -- 172.21.15.115:0/3761444232 learned_addr learned my addr 172.21.15.115:0/3761444232 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d273f1b0 0x5618d2af6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d27162f0 0x5618d2af5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bbfff640 1 -- 172.21.15.115:0/3761444232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d273f1b0 msgr2=0x5618d2af6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bbfff640 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d273f1b0 0x5618d2af6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bbfff640 1 -- 172.21.15.115:0/3761444232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d27162f0 msgr2=0x5618d2af5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bbfff640 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d27162f0 0x5618d2af5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.706+0000 7f80bbfff640 1 -- 172.21.15.115:0/3761444232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d2a0bed0 con 0x5618d271e4d0 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d273f1b0 0x5618d2af6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f80bbfff640 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2af6330 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f80ac02ab90 tx=0x7f80ac02abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ada21e4c2681385 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f809b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f809b7fe640 1 -- 172.21.15.115:0/3761444232 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac035040 con 0x5618d271e4d0 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f809b7fe640 1 -- 172.21.15.115:0/3761444232 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f80ac031830 con 0x5618d271e4d0 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f80c93c5740 1 -- 172.21.15.115:0/3761444232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ac040040 con 0x5618d271e4d0 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f809b7fe640 1 -- 172.21.15.115:0/3761444232 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f80ac004030 con 0x5618d271e4d0 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f80c93c5740 10 client.17919 fetch_fsmap learned FSMap version 45 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f80c93c5740 1 -- 172.21.15.115:0/3761444232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5618d2b6aa90 con 0x5618d271e4d0 2024-05-09T05:47:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f809b7fe640 1 -- 172.21.15.115:0/3761444232 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80ac040040 con 0x5618d271e4d0 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f809b7fe640 1 -- 172.21.15.115:0/3761444232 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f80ac04c070 con 0x5618d271e4d0 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f80c93c5740 10 client.17919 fetch_fsmap finished waiting for FSMap version 45 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f80c93c5740 10 client.17919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.707+0000 7f80c93c5740 20 client.17919 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.708+0000 7f80c93c5740 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5618d26d31c0 0x5618d2b6ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.708+0000 7f80c93c5740 4 client.17919 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.708+0000 7f80c93c5740 1 -- 172.21.15.115:0/3761444232 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618d2a4f240 con 0x5618d26d31c0 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.708+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5618d26d31c0 0x5618d2b6ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.708+0000 7f809b7fe640 10 client.17919 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:02.708+0000 7f80bb7fe640 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5618d26d31c0 0x5618d2b6ce30 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be50678c283da485 server_cookie=4b376e6d9b8947e7 in_seq=0 out_seq=0 2024-05-09T05:47:03.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:03 smithi195 ceph-mon[33844]: pgmap v273: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 22 MiB/s wr, 43 op/s 2024-05-09T05:47:03.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:03 smithi115 ceph-mon[30349]: pgmap v273: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 22 MiB/s wr, 43 op/s 2024-05-09T05:47:03.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:03 smithi161 ceph-mon[33805]: pgmap v273: 129 pgs: 129 active+clean; 7.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 22 MiB/s wr, 43 op/s 2024-05-09T05:47:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f809b7fe640 1 -- 172.21.15.115:0/3761444232 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5618d2a4f240 con 0x5618d26d31c0 2024-05-09T05:47:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f809b7fe640 10 client.17919 handle_command_reply: tid=0 2024-05-09T05:47:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 client.17919 shutdown 2024-05-09T05:47:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 -- 172.21.15.115:0/3761444232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 msgr2=0x5618d2af6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2af6330 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f80ac02ab90 tx=0x7f80ac02abc0 comp rx=0 tx=0).stop 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 -- 172.21.15.115:0/3761444232 shutdown_connections 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5618d26d31c0 0x5618d2b6ce30 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618d273f1b0 0x5618d2af6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f809b7fe640 0 client.17919 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618d271e4d0 0x5618d2af6330 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 --2- 172.21.15.115:0/3761444232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618d27162f0 0x5618d2af5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 -- 172.21.15.115:0/3761444232 >> 172.21.15.115:0/3761444232 conn(0x5618d2733300 msgr2=0x5618d2ade680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 -- 172.21.15.115:0/3761444232 shutdown_connections 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.286+0000 7f80c93c5740 1 -- 172.21.15.115:0/3761444232 wait complete. 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c93c5740 20 client.17919 trim_cache size 0 max 16384 2024-05-09T05:47:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:47:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:47:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:47:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 -- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80a40b2950 msgr2=0x7f80a40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80a40b2950 0x7f80a40b4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80b40a9e50 tx=0x7f80b4018400 comp rx=0 tx=0).stop 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 -- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc109050 msgr2=0x7f80bc19c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc109050 0x7f80bc19c0f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f80b400cce0 tx=0x7f80b400a620 comp rx=0 tx=0).stop 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 -- 172.21.15.115:0/3019671981 shutdown_connections 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f80a40b2950 0x7f80a40b4e60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80bc109050 0x7f80bc19c0f0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80bc108bb0 0x7f80bc19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 --2- 172.21.15.115:0/3019671981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80bc103300 0x7f80bc19b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 -- 172.21.15.115:0/3019671981 >> 172.21.15.115:0/3019671981 conn(0x7f80bc078280 msgr2=0x7f80bc104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 -- 172.21.15.115:0/3019671981 shutdown_connections 2024-05-09T05:47:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:04.287+0000 7f80c2434640 1 -- 172.21.15.115:0/3019671981 wait complete. 2024-05-09T05:47:04.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:04 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:47:04.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:04 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:47:04.859 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: *** Caught signal (Segmentation fault) ** 2024-05-09T05:47:04.859 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: in thread 7ff631fda700 thread_name:md_log_replay 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff64339cd20] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x559287d68da1] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x559287d69238] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x559287d638af] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x559287a24cff] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 6: (MDLog::_replay_thread()+0x1f38) [0x559287dc992e] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 7: (MDLog::ReplayThread::entry()+0x11) [0x559287dd2e6d] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 8: (Thread::entry_wrapper()+0x43) [0x7ff6447ef9f5] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 9: (Thread::_entry_func(void*)+0xd) [0x7ff6447efa11] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7ff6433921ca] 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 11: clone() 2024-05-09T05:47:04.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2024-05-09T05:47:04.484+0000 7ff631fda700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: in thread 7ff631fda700 thread_name:md_log_replay 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff64339cd20] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x559287d68da1] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x559287d69238] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x559287d638af] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x559287a24cff] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 6: (MDLog::_replay_thread()+0x1f38) [0x559287dc992e] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 7: (MDLog::ReplayThread::entry()+0x11) [0x559287dd2e6d] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 8: (Thread::entry_wrapper()+0x43) [0x7ff6447ef9f5] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 9: (Thread::_entry_func(void*)+0xd) [0x7ff6447efa11] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7ff6433921ca] 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 11: clone() 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:47:04.861 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 0> 2024-05-09T05:47:04.484+0000 7ff631fda700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: in thread 7ff631fda700 thread_name:md_log_replay 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff64339cd20] 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x559287d68da1] 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x559287d69238] 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x559287d638af] 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x559287a24cff] 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 6: (MDLog::_replay_thread()+0x1f38) [0x559287dc992e] 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 7: (MDLog::ReplayThread::entry()+0x11) [0x559287dd2e6d] 2024-05-09T05:47:04.862 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 8: (Thread::entry_wrapper()+0x43) [0x7ff6447ef9f5] 2024-05-09T05:47:04.863 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 9: (Thread::_entry_func(void*)+0xd) [0x7ff6447efa11] 2024-05-09T05:47:04.863 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7ff6433921ca] 2024-05-09T05:47:04.863 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 11: clone() 2024-05-09T05:47:04.863 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:47:04.863 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2024-05-09T05:47:04.863 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 0> 2024-05-09T05:47:04.484+0000 7ff631fda700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:47:04.863 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: in thread 7ff631fda700 thread_name:md_log_replay 2024-05-09T05:47:04.863 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2024-05-09T05:47:04.863 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff64339cd20] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x559287d68da1] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x559287d69238] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x559287d638af] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x559287a24cff] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 6: (MDLog::_replay_thread()+0x1f38) [0x559287dc992e] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 7: (MDLog::ReplayThread::entry()+0x11) [0x559287dd2e6d] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 8: (Thread::entry_wrapper()+0x43) [0x7ff6447ef9f5] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 9: (Thread::_entry_func(void*)+0xd) [0x7ff6447efa11] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7ff6433921ca] 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 11: clone() 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[68903]: 2024-05-09T05:47:04.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:04 smithi195 podman[69235]: 2024-05-09 05:47:04.547285028 +0000 UTC m=+0.021222032 container died 7231fae4353efd7d28cf9d936789ef46897b78fe7b0d9c9867eb9a977fa29407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T05:47:04.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:04 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:47:04.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:04 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:47:04.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:04 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:47:04.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:04 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:47:05.417 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:05 smithi195 podman[69235]: 2024-05-09 05:47:05.166920375 +0000 UTC m=+0.640857377 container remove 7231fae4353efd7d28cf9d936789ef46897b78fe7b0d9c9867eb9a977fa29407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T05:47:05.417 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:05 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-05-09T05:47:05.864 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:05 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-05-09T05:47:05.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:05 smithi195 ceph-mon[33844]: pgmap v274: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 32 MiB/s wr, 71 op/s 2024-05-09T05:47:05.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:05 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:47:05.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:05 smithi195 ceph-mon[33844]: scrub summary: idle 2024-05-09T05:47:05.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:05 smithi195 ceph-mon[33844]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T05:47:05.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:05 smithi115 ceph-mon[30349]: pgmap v274: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 32 MiB/s wr, 71 op/s 2024-05-09T05:47:05.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:05 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:47:05.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:05 smithi115 ceph-mon[30349]: scrub summary: idle 2024-05-09T05:47:05.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:05 smithi115 ceph-mon[30349]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T05:47:05.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:05 smithi161 ceph-mon[33805]: pgmap v274: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 32 MiB/s wr, 71 op/s 2024-05-09T05:47:05.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:05 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:47:05.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:05 smithi161 ceph-mon[33805]: scrub summary: idle 2024-05-09T05:47:05.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:05 smithi161 ceph-mon[33805]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T05:47:07.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:07 smithi195 ceph-mon[33844]: pgmap v276: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 MiB/s wr, 43 op/s 2024-05-09T05:47:07.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:07 smithi115 ceph-mon[30349]: pgmap v276: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 MiB/s wr, 43 op/s 2024-05-09T05:47:07.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:07 smithi161 ceph-mon[33805]: pgmap v276: 129 pgs: 129 active+clean; 7.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 MiB/s wr, 43 op/s 2024-05-09T05:47:09.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:09 smithi195 ceph-mon[33844]: pgmap v277: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 26 MiB/s wr, 44 op/s 2024-05-09T05:47:09.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:09 smithi195 ceph-mon[33844]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T05:47:09.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:09 smithi115 ceph-mon[30349]: pgmap v277: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 26 MiB/s wr, 44 op/s 2024-05-09T05:47:09.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:09 smithi115 ceph-mon[30349]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T05:47:09.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:09 smithi161 ceph-mon[33805]: pgmap v277: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 26 MiB/s wr, 44 op/s 2024-05-09T05:47:09.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:09 smithi161 ceph-mon[33805]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T05:47:11.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:11 smithi195 ceph-mon[33844]: pgmap v279: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 33 MiB/s wr, 55 op/s 2024-05-09T05:47:11.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:11 smithi115 ceph-mon[30349]: pgmap v279: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 33 MiB/s wr, 55 op/s 2024-05-09T05:47:11.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:11 smithi161 ceph-mon[33805]: pgmap v279: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 33 MiB/s wr, 55 op/s 2024-05-09T05:47:12.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:12 smithi195 ceph-mon[33844]: pgmap v280: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 13 op/s 2024-05-09T05:47:12.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:12 smithi115 ceph-mon[30349]: pgmap v280: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 13 op/s 2024-05-09T05:47:12.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:12 smithi161 ceph-mon[33805]: pgmap v280: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 13 op/s 2024-05-09T05:47:15.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:15 smithi115 ceph-mon[30349]: pgmap v281: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-05-09T05:47:15.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:15 smithi161 ceph-mon[33805]: pgmap v281: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-05-09T05:47:15.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:15 smithi195 ceph-mon[33844]: pgmap v281: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-05-09T05:47:15.925 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:15 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 2. 2024-05-09T05:47:15.926 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:15 smithi195 systemd[1]: Stopped Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:47:15.926 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:15 smithi195 systemd[1]: Starting Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:47:15.926 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:15 smithi195 podman[69335]: 2024-05-09 05:47:15.761494831 +0000 UTC m=+0.017992284 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:47:16.199 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:15 smithi195 podman[69335]: 2024-05-09 05:47:15.934222729 +0000 UTC m=+0.190720164 container create 2e229c2e786c64be76f8874048ea44248897bdaf0f295db4deb6345e77e472c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-09T05:47:16.199 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:16 smithi195 podman[69335]: 2024-05-09 05:47:16.171065742 +0000 UTC m=+0.427563175 container init 2e229c2e786c64be76f8874048ea44248897bdaf0f295db4deb6345e77e472c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:47:16.199 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:16 smithi195 podman[69335]: 2024-05-09 05:47:16.17399566 +0000 UTC m=+0.430493098 container start 2e229c2e786c64be76f8874048ea44248897bdaf0f295db4deb6345e77e472c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-05-09T05:47:16.614 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:16 smithi195 ceph-mds[69354]: -- 172.21.15.195:0/3264841799 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4e5809a40 con 0x55c4e5851180 2024-05-09T05:47:16.614 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:16 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: starting mds.l at 2024-05-09T05:47:16.614 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:16 smithi195 bash[69335]: 2e229c2e786c64be76f8874048ea44248897bdaf0f295db4deb6345e77e472c0 2024-05-09T05:47:16.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:47:16 smithi195 systemd[1]: Started Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:47:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.401+0000 7feede6bd640 1 -- 172.21.15.115:0/608277006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed8108bd0 msgr2=0x7feed8102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.401+0000 7feede6bd640 1 --2- 172.21.15.115:0/608277006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed8108bd0 0x7feed8102e30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7feec8009900 tx=0x7feec802a480 comp rx=0 tx=0).stop 2024-05-09T05:47:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.401+0000 7feede6bd640 1 -- 172.21.15.115:0/608277006 shutdown_connections 2024-05-09T05:47:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.401+0000 7feede6bd640 1 --2- 172.21.15.115:0/608277006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8103a20 0x7feed810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.401+0000 7feede6bd640 1 --2- 172.21.15.115:0/608277006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed8108fe0 0x7feed8103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.401+0000 7feede6bd640 1 --2- 172.21.15.115:0/608277006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed8108bd0 0x7feed8102e30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.401+0000 7feede6bd640 1 -- 172.21.15.115:0/608277006 >> 172.21.15.115:0/608277006 conn(0x7feed80fd9d0 msgr2=0x7feed80ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.401+0000 7feede6bd640 1 -- 172.21.15.115:0/608277006 shutdown_connections 2024-05-09T05:47:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.401+0000 7feede6bd640 1 -- 172.21.15.115:0/608277006 wait complete. 2024-05-09T05:47:17.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:17 smithi115 ceph-mon[30349]: pgmap v282: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-05-09T05:47:17.396 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:17 smithi115 ceph-mon[30349]: daemon mds.l restarted 2024-05-09T05:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.402+0000 7feede6bd640 1 Processor -- start 2024-05-09T05:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.402+0000 7feede6bd640 1 -- start start 2024-05-09T05:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feede6bd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed8108bd0 0x7feed81a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feede6bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8108fe0 0x7feed81a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feede6bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed819c560 0x7feed819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feede6bd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feed8112c40 con 0x7feed819c560 2024-05-09T05:47:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feede6bd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feed8112ac0 con 0x7feed8108bd0 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feede6bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feed8112dc0 con 0x7feed8108fe0 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feedcc5a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed819c560 0x7feed819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feed77fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8108fe0 0x7feed81a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feedcc5a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed819c560 0x7feed819c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40016/0 (socket says 172.21.15.115:40016) 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feedcc5a640 1 -- 172.21.15.115:0/2892348752 learned_addr learned my addr 172.21.15.115:0/2892348752 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feed7fff640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed8108bd0 0x7feed81a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feedcc5a640 1 -- 172.21.15.115:0/2892348752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8108fe0 msgr2=0x7feed81a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feedcc5a640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8108fe0 0x7feed81a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feedcc5a640 1 -- 172.21.15.115:0/2892348752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed8108bd0 msgr2=0x7feed81a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feedcc5a640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed8108bd0 0x7feed81a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feedcc5a640 1 -- 172.21.15.115:0/2892348752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed819d0a0 con 0x7feed819c560 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.403+0000 7feed77fe640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8108fe0 0x7feed81a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.404+0000 7feedcc5a640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed819c560 0x7feed819c970 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7feed000eaa0 tx=0x7feed000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.404+0000 7feed57fa640 1 -- 172.21.15.115:0/2892348752 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed000cdb0 con 0x7feed819c560 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.404+0000 7feede6bd640 1 -- 172.21.15.115:0/2892348752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed819d390 con 0x7feed819c560 2024-05-09T05:47:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.404+0000 7feed57fa640 1 -- 172.21.15.115:0/2892348752 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed0005020 con 0x7feed819c560 2024-05-09T05:47:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.404+0000 7feed57fa640 1 -- 172.21.15.115:0/2892348752 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00116d0 con 0x7feed819c560 2024-05-09T05:47:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.404+0000 7feede6bd640 1 -- 172.21.15.115:0/2892348752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed81a94b0 con 0x7feed819c560 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.406+0000 7feee557a740 1 Processor -- start 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feed57fa640 1 -- 172.21.15.115:0/2892348752 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7feed00051d0 con 0x7feed819c560 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feee557a740 1 -- start start 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feee557a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 0x56231d39d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feee557a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d39dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feee557a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d39e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feee557a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56231d3027c0 con 0x56231cfd94d0 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feee557a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56231d29e0e0 con 0x56231cffa1b0 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feee557a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56231d2fe780 con 0x56231cfd12f0 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feed77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d39e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feed77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d39e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40018/0 (socket says 172.21.15.115:40018) 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feed77fe640 1 -- 172.21.15.115:0/1227772662 learned_addr learned my addr 172.21.15.115:0/1227772662 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feed7fff640 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d39dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.407+0000 7feed77fe640 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 0x56231d39d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed7fff640 1 -- 172.21.15.115:0/1227772662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 msgr2=0x56231d39d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed7fff640 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 0x56231d39d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed7fff640 1 -- 172.21.15.115:0/1227772662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 msgr2=0x56231d39e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed7fff640 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d39e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed7fff640 1 -- 172.21.15.115:0/1227772662 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231d2c77c0 con 0x56231cfd12f0 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed77fe640 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d39e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed57fa640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feec00b2950 0x7feec00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed77fe640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feec00b2950 0x7feec00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed57fa640 1 -- 172.21.15.115:0/2892348752 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7feed00e2a20 con 0x7feed819c560 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feed7fff640 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d39dbc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feec8009230 tx=0x7feec8031bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3efc26ef3cd093a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feebe7fc640 1 -- 172.21.15.115:0/1227772662 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8035030 con 0x56231cfd12f0 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feebe7fc640 1 -- 172.21.15.115:0/1227772662 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feec8040070 con 0x56231cfd12f0 2024-05-09T05:47:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.408+0000 7feebe7fc640 1 -- 172.21.15.115:0/1227772662 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80038e0 con 0x56231cfd12f0 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feee557a740 1 -- 172.21.15.115:0/1227772662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 msgr2=0x56231d39dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feed77fe640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feec00b2950 0x7feec00b4e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x56231d39a7f0 tx=0x7feecc00bb60 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feee557a740 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d39dbc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feec8009230 tx=0x7feec8031bc0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feee557a740 1 -- 172.21.15.115:0/1227772662 shutdown_connections 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feee557a740 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d39e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feee557a740 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d39dbc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feee557a740 1 --2- 172.21.15.115:0/1227772662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 0x56231d39d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feee557a740 1 -- 172.21.15.115:0/1227772662 >> 172.21.15.115:0/1227772662 conn(0x56231cfee300 msgr2=0x56231d399e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feee557a740 1 -- 172.21.15.115:0/1227772662 shutdown_connections 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.409+0000 7feee557a740 1 -- 172.21.15.115:0/1227772662 wait complete. 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feee557a740 1 -- start start 2024-05-09T05:47:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feee557a740 1 Processor -- start 2024-05-09T05:47:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feee557a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 0x56231d3a9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feee557a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feee557a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feee557a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56231d3aa750 con 0x56231cffa1b0 2024-05-09T05:47:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feee557a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56231d2899c0 con 0x56231cfd12f0 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feee557a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56231d3aa8d0 con 0x56231cfd94d0 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feee557a740 10 client.0 _ll_register_callbacks cb 0x56231d2b6020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feedcc5a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feed77fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feedcc5a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40028/0 (socket says 172.21.15.115:40028) 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feedcc5a640 1 -- 172.21.15.115:0/637180747 learned_addr learned my addr 172.21.15.115:0/637180747 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feed77fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3a9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:60952/0 (socket says 172.21.15.115:60952) 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.410+0000 7feed7fff640 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 0x56231d3a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feedcc5a640 1 -- 172.21.15.115:0/637180747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 msgr2=0x56231d3a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feedcc5a640 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feedcc5a640 1 -- 172.21.15.115:0/637180747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 msgr2=0x56231d3a9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feedcc5a640 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 0x56231d3a9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feedcc5a640 1 -- 172.21.15.115:0/637180747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231d2b01f0 con 0x56231cffa1b0 2024-05-09T05:47:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feed77fe640 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feedcc5a640 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7bf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7feed00a5af0 tx=0x7feed0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585df84fa13c185e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feeb77fe640 1 -- 172.21.15.115:0/637180747 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed001f040 con 0x56231cffa1b0 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feeb77fe640 1 -- 172.21.15.115:0/637180747 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7feed00cc0d0 con 0x56231cffa1b0 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feee557a740 1 -- 172.21.15.115:0/637180747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec8040070 con 0x56231cffa1b0 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feeb77fe640 1 -- 172.21.15.115:0/637180747 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7feed00b2820 con 0x56231cffa1b0 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feee557a740 10 client.17937 fetch_fsmap learned FSMap version 47 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.411+0000 7feee557a740 1 -- 172.21.15.115:0/637180747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56231d065790 con 0x56231cffa1b0 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feeb77fe640 1 -- 172.21.15.115:0/637180747 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feec8040070 con 0x56231cffa1b0 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feeb77fe640 1 -- 172.21.15.115:0/637180747 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7feed000f040 con 0x56231cffa1b0 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feee557a740 10 client.17937 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feee557a740 10 client.17937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feee557a740 20 client.17937 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feee557a740 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56231cfb8d80 0x56231d3ba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feee557a740 4 client.17937 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feee557a740 1 -- 172.21.15.115:0/637180747 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56231d3103d0 con 0x56231cfb8d80 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feed7fff640 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56231cfb8d80 0x56231d3ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feeb77fe640 10 client.17937 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.412+0000 7feed7fff640 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56231cfb8d80 0x56231d3ba040 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e671b54a4ca703e7 server_cookie=fab98821ea4e9325 in_seq=0 out_seq=0 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feeb77fe640 1 -- 172.21.15.115:0/637180747 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56231d3103d0 con 0x56231cfb8d80 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feeb77fe640 10 client.17937 handle_command_reply: tid=0 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feee557a740 1 client.17937 shutdown 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feee557a740 1 -- 172.21.15.115:0/637180747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 msgr2=0x56231d3a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feee557a740 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7bf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7feed00a5af0 tx=0x7feed0005720 comp rx=0 tx=0).stop 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feee557a740 1 -- 172.21.15.115:0/637180747 shutdown_connections 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feee557a740 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7bf0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feee557a740 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56231cfb8d80 0x56231d3ba040 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feee557a740 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feee557a740 1 --2- 172.21.15.115:0/637180747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 0x56231d3a9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feee557a740 1 -- 172.21.15.115:0/637180747 >> 172.21.15.115:0/637180747 conn(0x56231cfee300 msgr2=0x56231d398700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.413+0000 7feeb77fe640 0 client.17937 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.414+0000 7feee557a740 1 -- 172.21.15.115:0/637180747 shutdown_connections 2024-05-09T05:47:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.414+0000 7feee557a740 1 -- 172.21.15.115:0/637180747 wait complete. 2024-05-09T05:47:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.414+0000 7feee557a740 20 client.17937 trim_cache size 0 max 16384 2024-05-09T05:47:17.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:17 smithi161 ceph-mon[33805]: pgmap v282: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-05-09T05:47:17.413 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:17 smithi161 ceph-mon[33805]: daemon mds.l restarted 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.433+0000 7feee557a740 1 Processor -- start 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.433+0000 7feee557a740 1 -- start start 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.433+0000 7feee557a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 0x56231d3afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feee557a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feee557a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feee557a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b3260 con 0x56231cffa1b0 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feee557a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b4ec0 con 0x56231cfd12f0 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feee557a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b3930 con 0x56231cfd94d0 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feedcc5a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feedcc5a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40032/0 (socket says 172.21.15.115:40032) 2024-05-09T05:47:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feedcc5a640 1 -- 172.21.15.115:0/4032548299 learned_addr learned my addr 172.21.15.115:0/4032548299 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feed7fff640 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 0x56231d3afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feed77fe640 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feedcc5a640 1 -- 172.21.15.115:0/4032548299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 msgr2=0x56231d3b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feedcc5a640 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feedcc5a640 1 -- 172.21.15.115:0/4032548299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 msgr2=0x56231d3afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feedcc5a640 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 0x56231d3afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feedcc5a640 1 -- 172.21.15.115:0/4032548299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231d0dc5f0 con 0x56231cffa1b0 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feed7fff640 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 0x56231d3afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.434+0000 7feedcc5a640 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7480 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7feed00e2d40 tx=0x7feed0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914590e0974122f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.435+0000 7feebdffb640 1 -- 172.21.15.115:0/4032548299 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed001f070 con 0x56231cffa1b0 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.435+0000 7feebdffb640 1 -- 172.21.15.115:0/4032548299 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed00cc0d0 con 0x56231cffa1b0 2024-05-09T05:47:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.435+0000 7feebdffb640 1 -- 172.21.15.115:0/4032548299 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00a5620 con 0x56231cffa1b0 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 -- 172.21.15.115:0/4032548299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 msgr2=0x56231d3a7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7480 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7feed00e2d40 tx=0x7feed0000f40 comp rx=0 tx=0).stop 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 -- 172.21.15.115:0/4032548299 shutdown_connections 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cffa1b0 0x56231d3a7480 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd94d0 0x56231d3b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 --2- 172.21.15.115:0/4032548299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cfd12f0 0x56231d3afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 -- 172.21.15.115:0/4032548299 >> 172.21.15.115:0/4032548299 conn(0x56231cfee300 msgr2=0x56231d3a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 -- 172.21.15.115:0/4032548299 shutdown_connections 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 -- 172.21.15.115:0/4032548299 wait complete. 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.436+0000 7feee557a740 1 -- start start 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.437+0000 7feee557a740 1 Processor -- start 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.437+0000 7feee557a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d3aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.437+0000 7feee557a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d3ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.437+0000 7feee557a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 0x56231d4203f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.437+0000 7feee557a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b47f0 con 0x56231cfd94d0 2024-05-09T05:47:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feee557a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b4090 con 0x56231cffa1b0 2024-05-09T05:47:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feee557a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b1240 con 0x56231cfd12f0 2024-05-09T05:47:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feee557a740 10 client.0 _ll_register_callbacks cb 0x7feed00a59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feed77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d3ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feed77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d3ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40044/0 (socket says 172.21.15.115:40044) 2024-05-09T05:47:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feed77fe640 1 -- 172.21.15.115:0/3709207325 learned_addr learned my addr 172.21.15.115:0/3709207325 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feed7fff640 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d3aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feedcc5a640 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 0x56231d4203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feed77fe640 1 -- 172.21.15.115:0/3709207325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 msgr2=0x56231d3aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feed77fe640 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d3aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.438+0000 7feed77fe640 1 -- 172.21.15.115:0/3709207325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 msgr2=0x56231d4203f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.439+0000 7feed77fe640 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 0x56231d4203f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.439+0000 7feed77fe640 1 -- 172.21.15.115:0/3709207325 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231d2c6ed0 con 0x56231cfd94d0 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.439+0000 7feed7fff640 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d3aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.439+0000 7feed77fe640 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d3ad3f0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b5f0 tx=0x7feecc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bb6fda686b7b94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.439+0000 7feeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.439+0000 7feeb77fe640 1 -- 172.21.15.115:0/3709207325 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc02f060 con 0x56231cfd94d0 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.439+0000 7feeb77fe640 1 -- 172.21.15.115:0/3709207325 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7feecc010840 con 0x56231cfd94d0 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feee557a740 1 -- 172.21.15.115:0/3709207325 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed00cc0d0 con 0x56231cfd94d0 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feeb77fe640 1 -- 172.21.15.115:0/3709207325 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7feecc036050 con 0x56231cfd94d0 2024-05-09T05:47:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feee557a740 10 client.17943 fetch_fsmap learned FSMap version 47 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feee557a740 1 -- 172.21.15.115:0/3709207325 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56231d4219e0 con 0x56231cfd94d0 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feeb77fe640 1 -- 172.21.15.115:0/3709207325 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feed00cc0d0 con 0x56231cfd94d0 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feeb77fe640 1 -- 172.21.15.115:0/3709207325 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7feecc01c040 con 0x56231cfd94d0 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feee557a740 10 client.17943 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feee557a740 10 client.17943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feee557a740 20 client.17943 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feee557a740 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56231cf8e1c0 0x56231d424260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feee557a740 4 client.17943 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.440+0000 7feee557a740 1 -- 172.21.15.115:0/3709207325 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56231d30a240 con 0x56231cf8e1c0 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.441+0000 7feed7fff640 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56231cf8e1c0 0x56231d424260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.441+0000 7feeb77fe640 10 client.17943 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.441+0000 7feed7fff640 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56231cf8e1c0 0x56231d424260 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=818d8ff7e3999479 server_cookie=57a1dfbf2a4b0c58 in_seq=0 out_seq=0 2024-05-09T05:47:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feeb77fe640 1 -- 172.21.15.115:0/3709207325 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56231d30a240 con 0x56231cf8e1c0 2024-05-09T05:47:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feeb77fe640 10 client.17943 handle_command_reply: tid=0 2024-05-09T05:47:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 client.17943 shutdown 2024-05-09T05:47:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 -- 172.21.15.115:0/3709207325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 msgr2=0x56231d3ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d3ad3f0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b5f0 tx=0x7feecc00b390 comp rx=0 tx=0).stop 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 -- 172.21.15.115:0/3709207325 shutdown_connections 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56231cffa1b0 0x56231d4203f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56231cfd94d0 0x56231d3ad3f0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56231cf8e1c0 0x56231d424260 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 --2- 172.21.15.115:0/3709207325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56231cfd12f0 0x56231d3aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feeb77fe640 0 client.17943 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 -- 172.21.15.115:0/3709207325 >> 172.21.15.115:0/3709207325 conn(0x56231cfee300 msgr2=0x56231d398700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 -- 172.21.15.115:0/3709207325 shutdown_connections 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 1 -- 172.21.15.115:0/3709207325 wait complete. 2024-05-09T05:47:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.448+0000 7feee557a740 20 client.17943 trim_cache size 0 max 16384 2024-05-09T05:47:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:47:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:47:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:47:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 -- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feec00b2950 msgr2=0x7feec00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feec00b2950 0x7feec00b4e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x56231d39a7f0 tx=0x7feecc00bb60 comp rx=0 tx=0).stop 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 -- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed819c560 msgr2=0x7feed819c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed819c560 0x7feed819c970 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7feed000eaa0 tx=0x7feed000ef60 comp rx=0 tx=0).stop 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 -- 172.21.15.115:0/2892348752 shutdown_connections 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7feec00b2950 0x7feec00b4e60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feed819c560 0x7feed819c970 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8108fe0 0x7feed81a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 --2- 172.21.15.115:0/2892348752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feed8108bd0 0x7feed81a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 -- 172.21.15.115:0/2892348752 >> 172.21.15.115:0/2892348752 conn(0x7feed80fd9d0 msgr2=0x7feed8101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 -- 172.21.15.115:0/2892348752 shutdown_connections 2024-05-09T05:47:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:17.449+0000 7feede6bd640 1 -- 172.21.15.115:0/2892348752 wait complete. 2024-05-09T05:47:17.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:17 smithi195 ceph-mon[33844]: pgmap v282: 129 pgs: 129 active+clean; 7.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-05-09T05:47:17.516 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:17 smithi195 ceph-mon[33844]: daemon mds.l restarted 2024-05-09T05:47:18.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:18 smithi115 ceph-mon[30349]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T05:47:18.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:18 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6832/391681574,v1:172.21.15.195:6833/391681574] up:boot 2024-05-09T05:47:18.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:18 smithi115 ceph-mon[30349]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:47:18.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:47:18.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:18 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:47:18.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:18 smithi161 ceph-mon[33805]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T05:47:18.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:18 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6832/391681574,v1:172.21.15.195:6833/391681574] up:boot 2024-05-09T05:47:18.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:18 smithi161 ceph-mon[33805]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:47:18.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:47:18.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:18 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:47:18.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:18 smithi195 ceph-mon[33844]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T05:47:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:18 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6832/391681574,v1:172.21.15.195:6833/391681574] up:boot 2024-05-09T05:47:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:18 smithi195 ceph-mon[33844]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:47:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:47:18.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:18 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:47:19.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:19 smithi161 ceph-mon[33805]: pgmap v284: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-05-09T05:47:19.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:19 smithi195 ceph-mon[33844]: pgmap v284: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-05-09T05:47:19.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:19 smithi115 ceph-mon[30349]: pgmap v284: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-05-09T05:47:19.811 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-09T05:47:19.811 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-09T05:47:19.812 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:47:20.087 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:21.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:21 smithi161 ceph-mon[33805]: pgmap v285: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-05-09T05:47:21.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:21 smithi115 ceph-mon[30349]: pgmap v285: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-05-09T05:47:21.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:21 smithi195 ceph-mon[33844]: pgmap v285: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-05-09T05:47:22.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.044+0000 7fb57dc14700 1 -- 172.21.15.115:0/713025675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb578103310 msgr2=0x7fb57810f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:22.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.044+0000 7fb57dc14700 1 --2- 172.21.15.115:0/713025675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb578103310 0x7fb57810f890 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c210 tx=0x7fb57000c520 comp rx=0 tx=0).stop 2024-05-09T05:47:22.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.044+0000 7fb57dc14700 1 -- 172.21.15.115:0/713025675 shutdown_connections 2024-05-09T05:47:22.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.044+0000 7fb57dc14700 1 --2- 172.21.15.115:0/713025675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb578103310 0x7fb57810f890 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:22.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.044+0000 7fb57dc14700 1 --2- 172.21.15.115:0/713025675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5781089f0 0x7fb578108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:22.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.044+0000 7fb57dc14700 1 --2- 172.21.15.115:0/713025675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5781029c0 0x7fb578102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:22.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.044+0000 7fb57dc14700 1 -- 172.21.15.115:0/713025675 >> 172.21.15.115:0/713025675 conn(0x7fb5780fe6f0 msgr2=0x7fb578100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:22.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.044+0000 7fb57dc14700 1 -- 172.21.15.115:0/713025675 shutdown_connections 2024-05-09T05:47:22.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.044+0000 7fb57dc14700 1 -- 172.21.15.115:0/713025675 wait complete. 2024-05-09T05:47:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.045+0000 7fb57dc14700 1 Processor -- start 2024-05-09T05:47:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb57dc14700 1 -- start start 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb57dc14700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5781029c0 0x7fb57819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb57dc14700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb578103310 0x7fb57819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb57dc14700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781089f0 0x7fb578198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb57dc14700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb5781121f0 con 0x7fb5781089f0 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb57dc14700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb578112070 con 0x7fb5781029c0 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb57dc14700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb578112370 con 0x7fb578103310 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb577fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781089f0 0x7fb578198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb577fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781089f0 0x7fb578198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40054/0 (socket says 172.21.15.115:40054) 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb577fff700 1 -- 172.21.15.115:0/768958313 learned_addr learned my addr 172.21.15.115:0/768958313 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb5777fe700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5781029c0 0x7fb57819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb576ffd700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb578103310 0x7fb57819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb577fff700 1 -- 172.21.15.115:0/768958313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb578103310 msgr2=0x7fb57819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb577fff700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb578103310 0x7fb57819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.046+0000 7fb577fff700 1 -- 172.21.15.115:0/768958313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5781029c0 msgr2=0x7fb57819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.047+0000 7fb577fff700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5781029c0 0x7fb57819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.047+0000 7fb577fff700 1 -- 172.21.15.115:0/768958313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb578199000 con 0x7fb5781089f0 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.047+0000 7fb5777fe700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5781029c0 0x7fb57819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.047+0000 7fb577fff700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781089f0 0x7fb578198950 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb57000a530 tx=0x7fb570004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.047+0000 7fb574ff9700 1 -- 172.21.15.115:0/768958313 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570004e60 con 0x7fb5781089f0 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.047+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb578199290 con 0x7fb5781089f0 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.047+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5781a55c0 con 0x7fb5781089f0 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.047+0000 7fb574ff9700 1 -- 172.21.15.115:0/768958313 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5700074e0 con 0x7fb5781089f0 2024-05-09T05:47:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.047+0000 7fb574ff9700 1 -- 172.21.15.115:0/768958313 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57001b720 con 0x7fb5781089f0 2024-05-09T05:47:22.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.048+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb564005330 con 0x7fb5781089f0 2024-05-09T05:47:22.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.050+0000 7fb574ff9700 1 -- 172.21.15.115:0/768958313 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fb5700086d0 con 0x7fb5781089f0 2024-05-09T05:47:22.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.050+0000 7fb574ff9700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5600b2a70 0x7fb5600b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:22.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.051+0000 7fb5777fe700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5600b2a70 0x7fb5600b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:22.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.051+0000 7fb574ff9700 1 -- 172.21.15.115:0/768958313 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fb570010030 con 0x7fb5781089f0 2024-05-09T05:47:22.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.051+0000 7fb5777fe700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5600b2a70 0x7fb5600b4f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb578199a90 tx=0x7fb56800b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.055+0000 7fb574ff9700 1 -- 172.21.15.115:0/768958313 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb57002b060 con 0x7fb5781089f0 2024-05-09T05:47:22.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.344+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb564005620 con 0x7fb5781089f0 2024-05-09T05:47:22.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.345+0000 7fb574ff9700 1 -- 172.21.15.115:0/768958313 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11961 (secure 0 0 0) 0x7fb5700ad660 con 0x7fb5781089f0 2024-05-09T05:47:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:47:22.338 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:47:17.152151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27316":{"gid":27316,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/391681574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":391681574},{"type":"v1","addr":"172.21.15.195:6833","nonce":391681574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24440,24425,24451,24469,24457]},"id":1}]} 2024-05-09T05:47:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.348+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5600b2a70 msgr2=0x7fb5600b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.348+0000 7fb57dc14700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5600b2a70 0x7fb5600b4f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb578199a90 tx=0x7fb56800b410 comp rx=0 tx=0).stop 2024-05-09T05:47:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.348+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781089f0 msgr2=0x7fb578198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.348+0000 7fb57dc14700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781089f0 0x7fb578198950 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb57000a530 tx=0x7fb570004c40 comp rx=0 tx=0).stop 2024-05-09T05:47:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.348+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 shutdown_connections 2024-05-09T05:47:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.348+0000 7fb57dc14700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb5600b2a70 0x7fb5600b4f80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.348+0000 7fb57dc14700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5781029c0 0x7fb57819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.348+0000 7fb57dc14700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5781089f0 0x7fb578198950 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.348+0000 7fb57dc14700 1 --2- 172.21.15.115:0/768958313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb578103310 0x7fb57819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.349+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 >> 172.21.15.115:0/768958313 conn(0x7fb5780fe6f0 msgr2=0x7fb578100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.349+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 shutdown_connections 2024-05-09T05:47:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:22.349+0000 7fb57dc14700 1 -- 172.21.15.115:0/768958313 wait complete. 2024-05-09T05:47:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 47 2024-05-09T05:47:23.070 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-09T05:47:23.070 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-09T05:47:23.346 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:23.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:23 smithi115 ceph-mon[30349]: pgmap v286: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-05-09T05:47:23.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:23 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/768958313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:47:23.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:23 smithi161 ceph-mon[33805]: pgmap v286: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-05-09T05:47:23.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:23 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/768958313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:47:23.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:23 smithi195 ceph-mon[33844]: pgmap v286: 129 pgs: 129 active+clean; 7.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-05-09T05:47:23.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:23 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/768958313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:47:25.228 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:25 smithi115 ceph-mon[30349]: pgmap v287: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:47:25.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.372+0000 7fa93a3d7700 1 -- 172.21.15.115:0/15916572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 msgr2=0x7fa934108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:25.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.372+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/15916572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 0x7fa934108220 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0099c0 tx=0x7fa91c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.372+0000 7fa93a3d7700 1 -- 172.21.15.115:0/15916572 shutdown_connections 2024-05-09T05:47:25.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.372+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/15916572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9341034e0 0x7fa9341038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.372+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/15916572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa934108760 0x7fa9341100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.372+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/15916572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 0x7fa934108220 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.372+0000 7fa93a3d7700 1 -- 172.21.15.115:0/15916572 >> 172.21.15.115:0/15916572 conn(0x7fa9340fef30 msgr2=0x7fa934101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:25.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.372+0000 7fa93a3d7700 1 -- 172.21.15.115:0/15916572 shutdown_connections 2024-05-09T05:47:25.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.372+0000 7fa93a3d7700 1 -- 172.21.15.115:0/15916572 wait complete. 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa93a3d7700 1 Processor -- start 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa93a3d7700 1 -- start start 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa93a3d7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341034e0 0x7fa93419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa93a3d7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 0x7fa93419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa93a3d7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa934108760 0x7fa93419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa93a3d7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa934113270 con 0x7fa934107e10 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa93a3d7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341130f0 con 0x7fa934108760 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa93a3d7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341133f0 con 0x7fa9341034e0 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 0x7fa93419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 0x7fa93419c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46322/0 (socket says 172.21.15.115:46322) 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.373+0000 7fa92bfff700 1 -- 172.21.15.115:0/1204189968 learned_addr learned my addr 172.21.15.115:0/1204189968 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa933fff700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341034e0 0x7fa93419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa938974700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa934108760 0x7fa93419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa92bfff700 1 -- 172.21.15.115:0/1204189968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341034e0 msgr2=0x7fa93419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa92bfff700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341034e0 0x7fa93419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa92bfff700 1 -- 172.21.15.115:0/1204189968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa934108760 msgr2=0x7fa93419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa92bfff700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa934108760 0x7fa93419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa92bfff700 1 -- 172.21.15.115:0/1204189968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9341a6f80 con 0x7fa934107e10 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa92bfff700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 0x7fa93419c150 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0098e0 tx=0x7fa91c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa931ffb700 1 -- 172.21.15.115:0/1204189968 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c027070 con 0x7fa934107e10 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa93a3d7700 1 -- 172.21.15.115:0/1204189968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9341a7210 con 0x7fa934107e10 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa931ffb700 1 -- 172.21.15.115:0/1204189968 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa91c00fc00 con 0x7fa934107e10 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa931ffb700 1 -- 172.21.15.115:0/1204189968 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c022ca0 con 0x7fa934107e10 2024-05-09T05:47:25.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.374+0000 7fa93a3d7700 1 -- 172.21.15.115:0/1204189968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9341a76c0 con 0x7fa934107e10 2024-05-09T05:47:25.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.377+0000 7fa931ffb700 1 -- 172.21.15.115:0/1204189968 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa91c02a070 con 0x7fa934107e10 2024-05-09T05:47:25.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.378+0000 7fa931ffb700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29f0 0x7fa9200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.378+0000 7fa933fff700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29f0 0x7fa9200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.378+0000 7fa931ffb700 1 -- 172.21.15.115:0/1204189968 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fa91c0e3720 con 0x7fa934107e10 2024-05-09T05:47:25.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.379+0000 7fa933fff700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29f0 0x7fa9200b4f00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa924005950 tx=0x7fa92400b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:25.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.379+0000 7fa948dedb80 1 Processor -- start 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.379+0000 7fa948dedb80 1 -- start start 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.379+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f27b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.379+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f28060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.379+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f21ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.379+0000 7fa948dedb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5582d5e44180 con 0x5582d5e8e2f0 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.379+0000 7fa948dedb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5582d5de2980 con 0x5582d5da4630 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.379+0000 7fa948dedb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5582d5e40140 con 0x5582d5f21be0 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f27b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa938974700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f21ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f27b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46332/0 (socket says 172.21.15.115:46332) 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f28060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa933fff700 1 -- 172.21.15.115:0/3631508600 learned_addr learned my addr 172.21.15.115:0/3631508600 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa938974700 1 -- 172.21.15.115:0/3631508600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 msgr2=0x5582d5f28060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa938974700 1 --2- 172.21.15.115:0/3631508600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f28060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa938974700 1 -- 172.21.15.115:0/3631508600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 msgr2=0x5582d5f27b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa938974700 1 --2- 172.21.15.115:0/3631508600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f27b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa938974700 1 -- 172.21.15.115:0/3631508600 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d5dda500 con 0x5582d5f21be0 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa92bfff700 1 --2- 172.21.15.115:0/3631508600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f28060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa933fff700 1 --2- 172.21.15.115:0/3631508600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f27b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa938974700 1 --2- 172.21.15.115:0/3631508600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f21ff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa93406d120 tx=0x7fa92c009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10e2eded01920cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.380+0000 7fa92a7fc700 1 -- 172.21.15.115:0/3631508600 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c004bb0 con 0x5582d5f21be0 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.381+0000 7fa92a7fc700 1 -- 172.21.15.115:0/3631508600 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa92c004d50 con 0x5582d5f21be0 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.381+0000 7fa92a7fc700 1 -- 172.21.15.115:0/3631508600 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c007b30 con 0x5582d5f21be0 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.381+0000 7fa948dedb80 1 -- 172.21.15.115:0/3631508600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 msgr2=0x5582d5f21ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.381+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3631508600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f21ff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa93406d120 tx=0x7fa92c009ba0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.382+0000 7fa948dedb80 1 -- 172.21.15.115:0/3631508600 shutdown_connections 2024-05-09T05:47:25.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.382+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3631508600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f21ff0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.382+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3631508600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f27b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.382+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3631508600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f28060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.382+0000 7fa948dedb80 1 -- 172.21.15.115:0/3631508600 >> 172.21.15.115:0/3631508600 conn(0x5582d5e04830 msgr2=0x5582d5f1cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:25.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.382+0000 7fa948dedb80 1 -- 172.21.15.115:0/3631508600 shutdown_connections 2024-05-09T05:47:25.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.382+0000 7fa948dedb80 1 -- 172.21.15.115:0/3631508600 wait complete. 2024-05-09T05:47:25.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.382+0000 7fa948dedb80 1 -- start start 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.382+0000 7fa948dedb80 1 Processor -- start 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f2b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f2bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f2c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa948dedb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5582d5cbfd30 con 0x5582d5e8e2f0 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa948dedb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5582d5dfe2e0 con 0x5582d5da4630 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa948dedb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5582d5f37990 con 0x5582d5f21be0 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa948dedb80 10 client.0 _ll_register_callbacks cb 0x5582d5cb5850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f2bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa933fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f2b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f2bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46346/0 (socket says 172.21.15.115:46346) 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa92bfff700 1 -- 172.21.15.115:0/3055770896 learned_addr learned my addr 172.21.15.115:0/3055770896 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa938974700 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f2c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa92bfff700 1 -- 172.21.15.115:0/3055770896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 msgr2=0x5582d5f2c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa92bfff700 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f2c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa92bfff700 1 -- 172.21.15.115:0/3055770896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 msgr2=0x5582d5f2b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa92bfff700 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f2b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa92bfff700 1 -- 172.21.15.115:0/3055770896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d5cbf620 con 0x5582d5e8e2f0 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa933fff700 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f2b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa92bfff700 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f2bd50 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0e0be0 tx=0x7fa91c0a4f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4ada815176efc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.383+0000 7fa9137fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa9137fe700 1 -- 172.21.15.115:0/3055770896 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0aa260 con 0x5582d5e8e2f0 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa9137fe700 1 -- 172.21.15.115:0/3055770896 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fa91c0920f0 con 0x5582d5e8e2f0 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa948dedb80 1 -- 172.21.15.115:0/3055770896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa92c004d50 con 0x5582d5e8e2f0 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa9137fe700 1 -- 172.21.15.115:0/3055770896 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa91c06b020 con 0x5582d5e8e2f0 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa948dedb80 10 client.17961 fetch_fsmap learned FSMap version 47 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa948dedb80 1 -- 172.21.15.115:0/3055770896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5582d5f392b0 con 0x5582d5e8e2f0 2024-05-09T05:47:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa9137fe700 1 -- 172.21.15.115:0/3055770896 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa92c004d50 con 0x5582d5e8e2f0 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa9137fe700 1 -- 172.21.15.115:0/3055770896 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fa91c0aa9b0 con 0x5582d5e8e2f0 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa948dedb80 10 client.17961 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa948dedb80 10 client.17961 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa948dedb80 20 client.17961 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5582d5f3b880 0x5582d5f3dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa948dedb80 4 client.17961 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.384+0000 7fa948dedb80 1 -- 172.21.15.115:0/3055770896 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582d5e51b60 con 0x5582d5f3b880 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.385+0000 7fa933fff700 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5582d5f3b880 0x5582d5f3dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.385+0000 7fa9137fe700 10 client.17961 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.385+0000 7fa933fff700 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5582d5f3b880 0x5582d5f3dd50 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7138b90d411922c9 server_cookie=9016549693bfd658 in_seq=0 out_seq=0 2024-05-09T05:47:25.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa9137fe700 1 -- 172.21.15.115:0/3055770896 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5582d5e51b60 con 0x5582d5f3b880 2024-05-09T05:47:25.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa9137fe700 10 client.17961 handle_command_reply: tid=0 2024-05-09T05:47:25.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa948dedb80 1 client.17961 shutdown 2024-05-09T05:47:25.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa948dedb80 1 -- 172.21.15.115:0/3055770896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 msgr2=0x5582d5f2bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f2bd50 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0e0be0 tx=0x7fa91c0a4f60 comp rx=0 tx=0).stop 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa948dedb80 1 -- 172.21.15.115:0/3055770896 shutdown_connections 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5da4630 0x5582d5f2b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5582d5f3b880 0x5582d5f3dd50 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa9137fe700 0 client.17961 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f2c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa948dedb80 1 --2- 172.21.15.115:0/3055770896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5e8e2f0 0x5582d5f2bd50 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.386+0000 7fa948dedb80 1 -- 172.21.15.115:0/3055770896 >> 172.21.15.115:0/3055770896 conn(0x5582d5e04830 msgr2=0x5582d5f1c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.387+0000 7fa948dedb80 1 -- 172.21.15.115:0/3055770896 shutdown_connections 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.387+0000 7fa948dedb80 1 -- 172.21.15.115:0/3055770896 wait complete. 2024-05-09T05:47:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.387+0000 7fa948dedb80 20 client.17961 trim_cache size 0 max 16384 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.415+0000 7fa948dedb80 1 Processor -- start 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa948dedb80 1 -- start start 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f43510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5e8e2f0 0x5582d5f43a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f25430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa948dedb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341ae0e0 con 0x5582d5da4630 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa948dedb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341afd40 con 0x5582d5e8e2f0 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa948dedb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341ae7b0 con 0x5582d5f21be0 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f43510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa933fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f43510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46358/0 (socket says 172.21.15.115:46358) 2024-05-09T05:47:25.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5e8e2f0 0x5582d5f43a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa933fff700 1 -- 172.21.15.115:0/1965108014 learned_addr learned my addr 172.21.15.115:0/1965108014 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:25.410 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:25 smithi161 ceph-mon[33805]: pgmap v287: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:47:25.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa938974700 1 --2- 172.21.15.115:0/1965108014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f25430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa92bfff700 1 -- 172.21.15.115:0/1965108014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 msgr2=0x5582d5f25430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa92bfff700 1 --2- 172.21.15.115:0/1965108014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f25430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa92bfff700 1 -- 172.21.15.115:0/1965108014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 msgr2=0x5582d5f43510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa92bfff700 1 --2- 172.21.15.115:0/1965108014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f43510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa92bfff700 1 -- 172.21.15.115:0/1965108014 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d5f29d80 con 0x5582d5e8e2f0 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.416+0000 7fa933fff700 1 --2- 172.21.15.115:0/1965108014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f43510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.417+0000 7fa92bfff700 1 --2- 172.21.15.115:0/1965108014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5e8e2f0 0x5582d5f43a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0a67d0 tx=0x7fa91c0e0c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193d3b8e212d7e3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.417+0000 7fa929ffb700 1 -- 172.21.15.115:0/1965108014 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0aae50 con 0x5582d5e8e2f0 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.417+0000 7fa929ffb700 1 -- 172.21.15.115:0/1965108014 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa91c0aa750 con 0x5582d5e8e2f0 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.417+0000 7fa929ffb700 1 -- 172.21.15.115:0/1965108014 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0b30e0 con 0x5582d5e8e2f0 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.418+0000 7fa948dedb80 1 -- 172.21.15.115:0/1965108014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5e8e2f0 msgr2=0x5582d5f43a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.418+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1965108014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5e8e2f0 0x5582d5f43a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0a67d0 tx=0x7fa91c0e0c10 comp rx=0 tx=0).stop 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.418+0000 7fa948dedb80 1 -- 172.21.15.115:0/1965108014 shutdown_connections 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.418+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1965108014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5f21be0 0x5582d5f25430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.418+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1965108014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f43510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.418+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1965108014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5e8e2f0 0x5582d5f43a50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.418+0000 7fa948dedb80 1 -- 172.21.15.115:0/1965108014 >> 172.21.15.115:0/1965108014 conn(0x5582d5e04830 msgr2=0x5582d5f268a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:25.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.418+0000 7fa948dedb80 1 -- 172.21.15.115:0/1965108014 shutdown_connections 2024-05-09T05:47:25.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.418+0000 7fa948dedb80 1 -- 172.21.15.115:0/1965108014 wait complete. 2024-05-09T05:47:25.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.419+0000 7fa948dedb80 1 -- start start 2024-05-09T05:47:25.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.419+0000 7fa948dedb80 1 Processor -- start 2024-05-09T05:47:25.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f33e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5e8e2f0 0x5582d5f308c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa948dedb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5f21be0 0x5582d5f30e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa948dedb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341af670 con 0x5582d5da4630 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa948dedb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341aef10 con 0x5582d5f21be0 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa948dedb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9341ac0a0 con 0x5582d5e8e2f0 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa948dedb80 10 client.0 _ll_register_callbacks cb 0x5582d5cb5850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5e8e2f0 0x5582d5f308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5e8e2f0 0x5582d5f308c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:42616/0 (socket says 172.21.15.115:42616) 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa92bfff700 1 -- 172.21.15.115:0/1192500302 learned_addr learned my addr 172.21.15.115:0/1192500302 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa933fff700 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f33e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa938974700 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5f21be0 0x5582d5f30e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa92bfff700 1 -- 172.21.15.115:0/1192500302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5f21be0 msgr2=0x5582d5f30e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa92bfff700 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5f21be0 0x5582d5f30e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa92bfff700 1 -- 172.21.15.115:0/1192500302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 msgr2=0x5582d5f33e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa92bfff700 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f33e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.420+0000 7fa92bfff700 1 -- 172.21.15.115:0/1192500302 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d5f31470 con 0x5582d5e8e2f0 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.421+0000 7fa92bfff700 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5e8e2f0 0x5582d5f308c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0b3540 tx=0x7fa91c0923a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b6f0cc061a0e0ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.421+0000 7fa9137fe700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:47:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.421+0000 7fa9137fe700 1 -- 172.21.15.115:0/1192500302 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0b32f0 con 0x5582d5e8e2f0 2024-05-09T05:47:25.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.421+0000 7fa9137fe700 1 -- 172.21.15.115:0/1192500302 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fa91c00f770 con 0x5582d5e8e2f0 2024-05-09T05:47:25.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.421+0000 7fa948dedb80 1 -- 172.21.15.115:0/1192500302 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa91c0aa750 con 0x5582d5e8e2f0 2024-05-09T05:47:25.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.421+0000 7fa9137fe700 1 -- 172.21.15.115:0/1192500302 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa91c0aae50 con 0x5582d5e8e2f0 2024-05-09T05:47:25.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.421+0000 7fa948dedb80 10 client.27358 fetch_fsmap learned FSMap version 47 2024-05-09T05:47:25.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.421+0000 7fa948dedb80 1 -- 172.21.15.115:0/1192500302 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5582d5f31470 con 0x5582d5e8e2f0 2024-05-09T05:47:25.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.422+0000 7fa9137fe700 1 -- 172.21.15.115:0/1192500302 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa91c0aa750 con 0x5582d5e8e2f0 2024-05-09T05:47:25.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.422+0000 7fa9137fe700 1 -- 172.21.15.115:0/1192500302 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fa91c06e070 con 0x5582d5e8e2f0 2024-05-09T05:47:25.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.422+0000 7fa948dedb80 10 client.27358 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T05:47:25.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.422+0000 7fa948dedb80 10 client.27358 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:47:25.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.422+0000 7fa948dedb80 20 client.27358 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:25.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.422+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5582d5fa71b0 0x5582d5fa95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:25.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.422+0000 7fa948dedb80 4 client.27358 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-09T05:47:25.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.422+0000 7fa948dedb80 1 -- 172.21.15.115:0/1192500302 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5582d5f1c550 con 0x5582d5fa71b0 2024-05-09T05:47:25.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.423+0000 7fa933fff700 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5582d5fa71b0 0x5582d5fa95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:25.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.424+0000 7fa9137fe700 10 client.27358 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:25.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.424+0000 7fa933fff700 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5582d5fa71b0 0x5582d5fa95d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a5b150f9a1c57d9 server_cookie=8abab079224747d2 in_seq=0 out_seq=0 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa9137fe700 1 -- 172.21.15.115:0/1192500302 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5582d5f1c550 con 0x5582d5fa71b0 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa9137fe700 10 client.27358 handle_command_reply: tid=0 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa948dedb80 1 client.27358 shutdown 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa948dedb80 1 -- 172.21.15.115:0/1192500302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5e8e2f0 msgr2=0x5582d5f308c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5e8e2f0 0x5582d5f308c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0b3540 tx=0x7fa91c0923a0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa948dedb80 1 -- 172.21.15.115:0/1192500302 shutdown_connections 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5582d5f21be0 0x5582d5f30e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5582d5fa71b0 0x5582d5fa95d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5582d5da4630 0x5582d5f33e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa948dedb80 1 --2- 172.21.15.115:0/1192500302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5582d5e8e2f0 0x5582d5f308c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.534+0000 7fa9137fe700 0 client.27358 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.535+0000 7fa948dedb80 1 -- 172.21.15.115:0/1192500302 >> 172.21.15.115:0/1192500302 conn(0x5582d5e04830 msgr2=0x5582d5f25430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.535+0000 7fa948dedb80 1 -- 172.21.15.115:0/1192500302 shutdown_connections 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.535+0000 7fa948dedb80 1 -- 172.21.15.115:0/1192500302 wait complete. 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.535+0000 7fa948dedb80 20 client.27358 trim_cache size 0 max 16384 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 47, 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 26, 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "9919d1d8": { 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 26, 2024-05-09T05:47:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:25.531 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:47:25.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.535+0000 7fa93a3d7700 1 -- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29f0 msgr2=0x7fa9200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:25.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.535+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29f0 0x7fa9200b4f00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa924005950 tx=0x7fa92400b410 comp rx=0 tx=0).stop 2024-05-09T05:47:25.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.535+0000 7fa93a3d7700 1 -- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 msgr2=0x7fa93419c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:25.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.536+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 0x7fa93419c150 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0098e0 tx=0x7fa91c0047b0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.536+0000 7fa93a3d7700 1 -- 172.21.15.115:0/1204189968 shutdown_connections 2024-05-09T05:47:25.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.536+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9200b29f0 0x7fa9200b4f00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.536+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa934108760 0x7fa93419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.536+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341034e0 0x7fa93419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.536+0000 7fa93a3d7700 1 --2- 172.21.15.115:0/1204189968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa934107e10 0x7fa93419c150 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:25.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.536+0000 7fa93a3d7700 1 -- 172.21.15.115:0/1204189968 >> 172.21.15.115:0/1204189968 conn(0x7fa9340fef30 msgr2=0x7fa9341096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:25.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.536+0000 7fa93a3d7700 1 -- 172.21.15.115:0/1204189968 shutdown_connections 2024-05-09T05:47:25.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:25.536+0000 7fa93a3d7700 1 -- 172.21.15.115:0/1204189968 wait complete. 2024-05-09T05:47:25.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:25 smithi195 ceph-mon[33844]: pgmap v287: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:47:26.416 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9919d1d8', quiesce took 6.6 seconds. Will release after: 23.4 2024-05-09T05:47:27.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:27 smithi195 ceph-mon[33844]: pgmap v288: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:47:27.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:27 smithi115 ceph-mon[30349]: pgmap v288: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:47:27.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:27 smithi161 ceph-mon[33805]: pgmap v288: 129 pgs: 129 active+clean; 8.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:47:29.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:29 smithi195 ceph-mon[33844]: pgmap v289: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 29 op/s 2024-05-09T05:47:29.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:29 smithi115 ceph-mon[30349]: pgmap v289: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 29 op/s 2024-05-09T05:47:29.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:29 smithi161 ceph-mon[33805]: pgmap v289: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 MiB/s wr, 29 op/s 2024-05-09T05:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.555+0000 7fa9c796c640 1 -- 172.21.15.115:0/2356325536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0103530 msgr2=0x7fa9c0103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.555+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2356325536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0103530 0x7fa9c0103940 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009900 tx=0x7fa9b402a410 comp rx=0 tx=0).stop 2024-05-09T05:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.555+0000 7fa9c796c640 1 -- 172.21.15.115:0/2356325536 shutdown_connections 2024-05-09T05:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.555+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2356325536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9c0107e70 0x7fa9c010f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.555+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2356325536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c01073f0 0x7fa9c0107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.555+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2356325536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0103530 0x7fa9c0103940 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.555+0000 7fa9c796c640 1 -- 172.21.15.115:0/2356325536 >> 172.21.15.115:0/2356325536 conn(0x7fa9c00fd9d0 msgr2=0x7fa9c00ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.555+0000 7fa9c796c640 1 -- 172.21.15.115:0/2356325536 shutdown_connections 2024-05-09T05:47:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.555+0000 7fa9c796c640 1 -- 172.21.15.115:0/2356325536 wait complete. 2024-05-09T05:47:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.556+0000 7fa9c796c640 1 Processor -- start 2024-05-09T05:47:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.556+0000 7fa9c796c640 1 -- start start 2024-05-09T05:47:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.556+0000 7fa9c796c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c0103530 0x7fa9c019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.556+0000 7fa9c796c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9c01073f0 0x7fa9c019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.556+0000 7fa9c796c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0107e70 0x7fa9c019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.556+0000 7fa9c796c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0112c20 con 0x7fa9c0107e70 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.556+0000 7fa9c796c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0112aa0 con 0x7fa9c0103530 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.556+0000 7fa9c796c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0112da0 con 0x7fa9c01073f0 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.556+0000 7fa9c5f09640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0107e70 0x7fa9c019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c5f09640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0107e70 0x7fa9c019c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46402/0 (socket says 172.21.15.115:46402) 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c5f09640 1 -- 172.21.15.115:0/2561807162 learned_addr learned my addr 172.21.15.115:0/2561807162 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9c01073f0 0x7fa9c019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c5708640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c0103530 0x7fa9c019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c5f09640 1 -- 172.21.15.115:0/2561807162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9c01073f0 msgr2=0x7fa9c019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c5f09640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9c01073f0 0x7fa9c019bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c5f09640 1 -- 172.21.15.115:0/2561807162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c0103530 msgr2=0x7fa9c019b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c5f09640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c0103530 0x7fa9c019b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c5f09640 1 -- 172.21.15.115:0/2561807162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c01a69c0 con 0x7fa9c0107e70 2024-05-09T05:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c5f09640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0107e70 0x7fa9c019c120 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800eb10 tx=0x7fa9b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9b27fc640 1 -- 172.21.15.115:0/2561807162 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8004b20 con 0x7fa9c0107e70 2024-05-09T05:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9b27fc640 1 -- 172.21.15.115:0/2561807162 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b8004cc0 con 0x7fa9c0107e70 2024-05-09T05:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c796c640 1 -- 172.21.15.115:0/2561807162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c01a6cb0 con 0x7fa9c0107e70 2024-05-09T05:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9b27fc640 1 -- 172.21.15.115:0/2561807162 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8010690 con 0x7fa9c0107e70 2024-05-09T05:47:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.557+0000 7fa9c796c640 1 -- 172.21.15.115:0/2561807162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c01a71b0 con 0x7fa9c0107e70 2024-05-09T05:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.560+0000 7fa9ce81b740 1 Processor -- start 2024-05-09T05:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.560+0000 7fa9b27fc640 1 -- 172.21.15.115:0/2561807162 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa9b8005020 con 0x7fa9c0107e70 2024-05-09T05:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.560+0000 7fa9ce81b740 1 -- start start 2024-05-09T05:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596db71b0 0x55d59715a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d59715abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 0x55d59715b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d5970bf7c0 con 0x55d596d8e2f0 2024-05-09T05:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d59705b0e0 con 0x55d596db71b0 2024-05-09T05:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d5970bb780 con 0x55d596d964d0 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c5708640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d59715abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c5708640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d59715abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46408/0 (socket says 172.21.15.115:46408) 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c5708640 1 -- 172.21.15.115:0/3508271511 learned_addr learned my addr 172.21.15.115:0/3508271511 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/3508271511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 0x55d59715b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/3508271511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596db71b0 0x55d59715a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9b27fc640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c5708640 1 -- 172.21.15.115:0/3508271511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 msgr2=0x55d59715b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c5708640 1 --2- 172.21.15.115:0/3508271511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 0x55d59715b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c5708640 1 -- 172.21.15.115:0/3508271511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596db71b0 msgr2=0x55d59715a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c5708640 1 --2- 172.21.15.115:0/3508271511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596db71b0 0x55d59715a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c5708640 1 -- 172.21.15.115:0/3508271511 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5970847c0 con 0x55d596d8e2f0 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9c5f09640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.561+0000 7fa9b27fc640 1 -- 172.21.15.115:0/2561807162 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fa9b80e2e70 con 0x7fa9c0107e70 2024-05-09T05:47:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.562+0000 7fa9c5708640 1 --2- 172.21.15.115:0/3508271511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d59715abc0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40057c0 tx=0x7fa9b4031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca685c07e96975f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.562+0000 7fa9ab7fe640 1 -- 172.21.15.115:0/3508271511 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4035030 con 0x55d596d8e2f0 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.562+0000 7fa9ab7fe640 1 -- 172.21.15.115:0/3508271511 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b4040070 con 0x55d596d8e2f0 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.562+0000 7fa9c5f09640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80a7080 tx=0x7fa9b8076740 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.562+0000 7fa9ab7fe640 1 -- 172.21.15.115:0/3508271511 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4003850 con 0x55d596d8e2f0 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3508271511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 msgr2=0x55d59715abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3508271511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d59715abc0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40057c0 tx=0x7fa9b4031a70 comp rx=0 tx=0).stop 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3508271511 shutdown_connections 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3508271511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 0x55d59715b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3508271511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d59715abc0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3508271511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596db71b0 0x55d59715a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3508271511 >> 172.21.15.115:0/3508271511 conn(0x55d596dab300 msgr2=0x55d597156e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3508271511 shutdown_connections 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3508271511 wait complete. 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 -- start start 2024-05-09T05:47:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.563+0000 7fa9ce81b740 1 Processor -- start 2024-05-09T05:47:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d8e2f0 0x55d597166730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 0x55d597166c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d597164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d597167750 con 0x55d596db71b0 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d5970469c0 con 0x55d596d8e2f0 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d5971678d0 con 0x55d596d964d0 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9ce81b740 10 client.0 _ll_register_callbacks cb 0x55d597073020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d597164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d597164bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46418/0 (socket says 172.21.15.115:46418) 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 -- 172.21.15.115:0/2373296958 learned_addr learned my addr 172.21.15.115:0/2373296958 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d8e2f0 0x55d597166730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c5708640 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 0x55d597166c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 -- 172.21.15.115:0/2373296958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 msgr2=0x55d597166c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 0x55d597166c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 -- 172.21.15.115:0/2373296958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d8e2f0 msgr2=0x55d597166730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d8e2f0 0x55d597166730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 -- 172.21.15.115:0/2373296958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d59706d1f0 con 0x55d596db71b0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d597164bb0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa9c006bdd0 tx=0x7fa9bc009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba10b81c570e583 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.564+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/2373296958 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc00fb90 con 0x55d596db71b0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9ce81b740 1 -- 172.21.15.115:0/2373296958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b4040070 con 0x55d596db71b0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/2373296958 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fa9bc00fd30 con 0x55d596db71b0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/2373296958 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa9bc02c050 con 0x55d596db71b0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9ce81b740 10 client.17979 fetch_fsmap learned FSMap version 47 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9ce81b740 1 -- 172.21.15.115:0/2373296958 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d596e22790 con 0x55d596db71b0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/2373296958 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b4040070 con 0x55d596db71b0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/2373296958 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fa9bc0039f0 con 0x55d596db71b0 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9ce81b740 10 client.17979 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9ce81b740 10 client.17979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9ce81b740 20 client.17979 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d596d75d80 0x55d597177100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9ce81b740 4 client.17979 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.565+0000 7fa9ce81b740 1 -- 172.21.15.115:0/2373296958 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5970cd3d0 con 0x55d596d75d80 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.566+0000 7fa9c5708640 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d596d75d80 0x55d597177100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.566+0000 7fa9c5708640 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d596d75d80 0x55d597177100 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4080c11208054df7 server_cookie=8a5d624dc84de92 in_seq=0 out_seq=0 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.566+0000 7fa9a8ff9640 10 client.17979 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/2373296958 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d5970cd3d0 con 0x55d596d75d80 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9a8ff9640 10 client.17979 handle_command_reply: tid=0 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 client.17979 shutdown 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 -- 172.21.15.115:0/2373296958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 msgr2=0x55d597164bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d597164bb0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa9c006bdd0 tx=0x7fa9bc009aa0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 -- 172.21.15.115:0/2373296958 shutdown_connections 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d597164bb0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d596d75d80 0x55d597177100 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d964d0 0x55d597166c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/2373296958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d8e2f0 0x55d597166730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9a8ff9640 0 client.17979 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 -- 172.21.15.115:0/2373296958 >> 172.21.15.115:0/2373296958 conn(0x55d596dab300 msgr2=0x55d597155700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 -- 172.21.15.115:0/2373296958 shutdown_connections 2024-05-09T05:47:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 1 -- 172.21.15.115:0/2373296958 wait complete. 2024-05-09T05:47:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.567+0000 7fa9ce81b740 20 client.17979 trim_cache size 0 max 16384 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9ce81b740 1 Processor -- start 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9ce81b740 1 -- start start 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d8e2f0 0x55d59715ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 0x55d59717c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d59717ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01b10d0 con 0x55d596db71b0 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01b2d30 con 0x55d596d964d0 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01b17a0 con 0x55d596d8e2f0 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9c4f07640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d59717ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9c4f07640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d59717ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46432/0 (socket says 172.21.15.115:46432) 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9c4f07640 1 -- 172.21.15.115:0/124616383 learned_addr learned my addr 172.21.15.115:0/124616383 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.582+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d8e2f0 0x55d59715ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9c5708640 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 0x55d59717c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9c4f07640 1 -- 172.21.15.115:0/124616383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d8e2f0 msgr2=0x55d59715ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d8e2f0 0x55d59715ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9c4f07640 1 -- 172.21.15.115:0/124616383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 msgr2=0x55d59717c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 0x55d59717c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9c4f07640 1 -- 172.21.15.115:0/124616383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d596e995f0 con 0x55d596db71b0 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9c5708640 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 0x55d59717c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d59717ce00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x55d597157eb0 tx=0x7fa9bc013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8f2cc0c54d87fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9aaffd640 1 -- 172.21.15.115:0/124616383 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc011270 con 0x55d596db71b0 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9aaffd640 1 -- 172.21.15.115:0/124616383 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9bc004000 con 0x55d596db71b0 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.583+0000 7fa9aaffd640 1 -- 172.21.15.115:0/124616383 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc003b00 con 0x55d596db71b0 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 -- 172.21.15.115:0/124616383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 msgr2=0x55d59717ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d59717ce00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x55d597157eb0 tx=0x7fa9bc013040 comp rx=0 tx=0).stop 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 -- 172.21.15.115:0/124616383 shutdown_connections 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596db71b0 0x55d59717ce00 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 0x55d59717c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/124616383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596d8e2f0 0x55d59715ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 -- 172.21.15.115:0/124616383 >> 172.21.15.115:0/124616383 conn(0x55d596dab300 msgr2=0x55d59715e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 -- 172.21.15.115:0/124616383 shutdown_connections 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 -- 172.21.15.115:0/124616383 wait complete. 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 -- start start 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.584+0000 7fa9ce81b740 1 Processor -- start 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d5971d7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 0x55d5971d7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9ce81b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596db71b0 0x55d59716a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01b2660 con 0x55d596d8e2f0 2024-05-09T05:47:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01b1f00 con 0x55d596d964d0 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9ce81b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01af0b0 con 0x55d596db71b0 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9ce81b740 10 client.0 _ll_register_callbacks cb 0x7fa9bc0117c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d5971d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d5971d7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46446/0 (socket says 172.21.15.115:46446) 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 -- 172.21.15.115:0/3541243811 learned_addr learned my addr 172.21.15.115:0/3541243811 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c5708640 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 0x55d5971d7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596db71b0 0x55d59716a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 -- 172.21.15.115:0/3541243811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596db71b0 msgr2=0x55d59716a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596db71b0 0x55d59716a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 -- 172.21.15.115:0/3541243811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 msgr2=0x55d5971d7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 0x55d5971d7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 -- 172.21.15.115:0/3541243811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d597083ed0 con 0x55d596d8e2f0 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9c4f07640 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d5971d7880 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc005890 tx=0x7fa9bc00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5efdc3cd50caf0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/3541243811 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc011260 con 0x55d596d8e2f0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/3541243811 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fa9bc00fb90 con 0x55d596d8e2f0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.585+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3541243811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9bc004000 con 0x55d596d8e2f0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/3541243811 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa9bc045050 con 0x55d596d8e2f0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9ce81b740 10 client.17985 fetch_fsmap learned FSMap version 47 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3541243811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d5971ded40 con 0x55d596d8e2f0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/3541243811 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9bc004000 con 0x55d596d8e2f0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/3541243811 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fa9bc004570 con 0x55d596d8e2f0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9ce81b740 10 client.17985 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9ce81b740 10 client.17985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9ce81b740 20 client.17985 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d596d4b1c0 0x55d5971e1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9ce81b740 4 client.17985 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3541243811 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5970c7240 con 0x55d596d4b1c0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.586+0000 7fa9c5708640 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d596d4b1c0 0x55d5971e1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.587+0000 7fa9a8ff9640 10 client.17985 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.587+0000 7fa9c5708640 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d596d4b1c0 0x55d5971e1440 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd997e1ef9ebb558 server_cookie=3b6f2abd1101d1e5 in_seq=0 out_seq=0 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9a8ff9640 1 -- 172.21.15.115:0/3541243811 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d5970c7240 con 0x55d596d4b1c0 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9a8ff9640 10 client.17985 handle_command_reply: tid=0 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 client.17985 shutdown 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3541243811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 msgr2=0x55d5971d7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d5971d7880 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc005890 tx=0x7fa9bc00f810 comp rx=0 tx=0).stop 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3541243811 shutdown_connections 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d596d4b1c0 0x55d5971e1440 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d596db71b0 0x55d59716a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d596d964d0 0x55d5971d7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9a8ff9640 0 client.17985 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 --2- 172.21.15.115:0/3541243811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d596d8e2f0 0x55d5971d7880 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3541243811 >> 172.21.15.115:0/3541243811 conn(0x55d596dab300 msgr2=0x55d5971567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3541243811 shutdown_connections 2024-05-09T05:47:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.594+0000 7fa9ce81b740 1 -- 172.21.15.115:0/3541243811 wait complete. 2024-05-09T05:47:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9ce81b740 20 client.17985 trim_cache size 0 max 16384 2024-05-09T05:47:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:47:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:47:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 -- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9ac0b2950 msgr2=0x7fa9ac0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80a7080 tx=0x7fa9b8076740 comp rx=0 tx=0).stop 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 -- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0107e70 msgr2=0x7fa9c019c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0107e70 0x7fa9c019c120 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800eb10 tx=0x7fa9b800c2d0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 -- 172.21.15.115:0/2561807162 shutdown_connections 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9c0107e70 0x7fa9c019c120 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9c01073f0 0x7fa9c019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 --2- 172.21.15.115:0/2561807162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9c0103530 0x7fa9c019b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 -- 172.21.15.115:0/2561807162 >> 172.21.15.115:0/2561807162 conn(0x7fa9c00fd9d0 msgr2=0x7fa9c0105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 -- 172.21.15.115:0/2561807162 shutdown_connections 2024-05-09T05:47:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:30.595+0000 7fa9c796c640 1 -- 172.21.15.115:0/2561807162 wait complete. 2024-05-09T05:47:31.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:47:31.601 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:31.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:31 smithi195 ceph-mon[33844]: pgmap v290: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-09T05:47:31.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:31 smithi115 ceph-mon[30349]: pgmap v290: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-09T05:47:31.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:31 smithi161 ceph-mon[33805]: pgmap v290: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-09T05:47:33.352 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:33 smithi115 ceph-mon[30349]: pgmap v291: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-09T05:47:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.478+0000 7fd0084d4700 1 -- 172.21.15.115:0/4056242425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd000103310 msgr2=0x7fd00010f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.478+0000 7fd0084d4700 1 --2- 172.21.15.115:0/4056242425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd000103310 0x7fd00010f890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c180 tx=0x7fcff800c490 comp rx=0 tx=0).stop 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.478+0000 7fd0084d4700 1 -- 172.21.15.115:0/4056242425 shutdown_connections 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.478+0000 7fd0084d4700 1 --2- 172.21.15.115:0/4056242425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0001089f0 0x7fd000108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.478+0000 7fd0084d4700 1 --2- 172.21.15.115:0/4056242425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd000103310 0x7fd00010f890 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.478+0000 7fd0084d4700 1 --2- 172.21.15.115:0/4056242425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0001029c0 0x7fd000102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.478+0000 7fd0084d4700 1 -- 172.21.15.115:0/4056242425 >> 172.21.15.115:0/4056242425 conn(0x7fd0000fe6f0 msgr2=0x7fd000100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.478+0000 7fd0084d4700 1 -- 172.21.15.115:0/4056242425 shutdown_connections 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.478+0000 7fd0084d4700 1 -- 172.21.15.115:0/4056242425 wait complete. 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd0084d4700 1 Processor -- start 2024-05-09T05:47:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd0084d4700 1 -- start start 2024-05-09T05:47:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd0084d4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0001029c0 0x7fd00019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd0084d4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd000103310 0x7fd00019d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd0084d4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0001089f0 0x7fd0001a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd0084d4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd000112190 con 0x7fd000103310 2024-05-09T05:47:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd0084d4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd000112010 con 0x7fd0001029c0 2024-05-09T05:47:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd0084d4700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd000112310 con 0x7fd0001089f0 2024-05-09T05:47:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd005a6f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd000103310 0x7fd00019d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd005a6f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd000103310 0x7fd00019d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56324/0 (socket says 172.21.15.115:56324) 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd005a6f700 1 -- 172.21.15.115:0/378970594 learned_addr learned my addr 172.21.15.115:0/378970594 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd006270700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0001029c0 0x7fd00019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.479+0000 7fd006a71700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0001089f0 0x7fd0001a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fd005a6f700 1 -- 172.21.15.115:0/378970594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0001089f0 msgr2=0x7fd0001a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fd005a6f700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0001089f0 0x7fd0001a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fd005a6f700 1 -- 172.21.15.115:0/378970594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0001029c0 msgr2=0x7fd00019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fd005a6f700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0001029c0 0x7fd00019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fd005a6f700 1 -- 172.21.15.115:0/378970594 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0001a1e10 con 0x7fd000103310 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fd006270700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0001029c0 0x7fd00019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fd005a6f700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd000103310 0x7fd00019d4e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fcff000ec70 tx=0x7fcff000ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fcff77fe700 1 -- 172.21.15.115:0/378970594 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff000cc90 con 0x7fd000103310 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fd0084d4700 1 -- 172.21.15.115:0/378970594 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0001a20a0 con 0x7fd000103310 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fcff77fe700 1 -- 172.21.15.115:0/378970594 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcff000ce30 con 0x7fd000103310 2024-05-09T05:47:33.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fcff77fe700 1 -- 172.21.15.115:0/378970594 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0010430 con 0x7fd000103310 2024-05-09T05:47:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.480+0000 7fd0084d4700 1 -- 172.21.15.115:0/378970594 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0001a9a20 con 0x7fd000103310 2024-05-09T05:47:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.483+0000 7fcff77fe700 1 -- 172.21.15.115:0/378970594 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcff00105d0 con 0x7fd000103310 2024-05-09T05:47:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fcff77fe700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfec0b29f0 0x7fcfec0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fd016eeab80 1 Processor -- start 2024-05-09T05:47:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fd006270700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfec0b29f0 0x7fcfec0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fcff77fe700 1 -- 172.21.15.115:0/378970594 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fcff00e2c80 con 0x7fd000103310 2024-05-09T05:47:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fd016eeab80 1 -- start start 2024-05-09T05:47:33.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 0x55eec7aba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec79374a0 0x55eec7aba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec7ab4400 0x55eec7ab4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fd016eeab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eec79d70a0 con 0x55eec7ab4400 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fd016eeab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eec797c9a0 con 0x55eec79b1d00 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.484+0000 7fd016eeab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eec79d3060 con 0x55eec79374a0 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006270700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfec0b29f0 0x7fcfec0b4f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd000103b00 tx=0x7fcffc009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006a71700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec7ab4400 0x55eec7ab4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006a71700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec7ab4400 0x55eec7ab4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56334/0 (socket says 172.21.15.115:56334) 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006270700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 0x55eec7aba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006a71700 1 -- 172.21.15.115:0/1823446834 learned_addr learned my addr 172.21.15.115:0/1823446834 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd005a6f700 1 --2- 172.21.15.115:0/1823446834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec79374a0 0x55eec7aba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006a71700 1 -- 172.21.15.115:0/1823446834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec79374a0 msgr2=0x55eec7aba780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006a71700 1 --2- 172.21.15.115:0/1823446834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec79374a0 0x55eec7aba780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006a71700 1 -- 172.21.15.115:0/1823446834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 msgr2=0x55eec7aba240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006a71700 1 --2- 172.21.15.115:0/1823446834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 0x55eec7aba240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006a71700 1 -- 172.21.15.115:0/1823446834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec796d500 con 0x55eec7ab4400 2024-05-09T05:47:33.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fd006a71700 1 --2- 172.21.15.115:0/1823446834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec7ab4400 0x55eec7ab4810 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fcff800ad90 tx=0x7fcff800adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed92af3ebc176964 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:33.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fcff4d47700 1 -- 172.21.15.115:0/1823446834 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8023070 con 0x55eec7ab4400 2024-05-09T05:47:33.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fcff4d47700 1 -- 172.21.15.115:0/1823446834 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcff800a350 con 0x55eec7ab4400 2024-05-09T05:47:33.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.485+0000 7fcff4d47700 1 -- 172.21.15.115:0/1823446834 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8008cf0 con 0x55eec7ab4400 2024-05-09T05:47:33.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 -- 172.21.15.115:0/1823446834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec7ab4400 msgr2=0x55eec7ab4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:33.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1823446834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec7ab4400 0x55eec7ab4810 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fcff800ad90 tx=0x7fcff800adc0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 -- 172.21.15.115:0/1823446834 shutdown_connections 2024-05-09T05:47:33.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1823446834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 0x55eec7aba240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1823446834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec7ab4400 0x55eec7ab4810 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1823446834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec79374a0 0x55eec7aba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 -- 172.21.15.115:0/1823446834 >> 172.21.15.115:0/1823446834 conn(0x55eec793af70 msgr2=0x55eec7aaf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:33.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 -- 172.21.15.115:0/1823446834 shutdown_connections 2024-05-09T05:47:33.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 -- 172.21.15.115:0/1823446834 wait complete. 2024-05-09T05:47:33.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.487+0000 7fd016eeab80 1 -- start start 2024-05-09T05:47:33.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd016eeab80 1 Processor -- start 2024-05-09T05:47:33.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79374a0 0x55eec7abe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 0x55eec7abe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec7ab4400 0x55eec7abeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd016eeab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eec7ab5100 con 0x55eec79374a0 2024-05-09T05:47:33.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd016eeab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eec7971b20 con 0x55eec79b1d00 2024-05-09T05:47:33.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd016eeab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eec7aca200 con 0x55eec7ab4400 2024-05-09T05:47:33.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd006270700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79374a0 0x55eec7abe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd016eeab80 10 client.0 _ll_register_callbacks cb 0x55eec78486c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:33.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd006270700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79374a0 0x55eec7abe0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56348/0 (socket says 172.21.15.115:56348) 2024-05-09T05:47:33.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd006a71700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec7ab4400 0x55eec7abeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd005a6f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 0x55eec7abe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd006270700 1 -- 172.21.15.115:0/3834072394 learned_addr learned my addr 172.21.15.115:0/3834072394 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd006a71700 1 -- 172.21.15.115:0/3834072394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 msgr2=0x55eec7abe5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd006a71700 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 0x55eec7abe5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.488+0000 7fd006a71700 1 -- 172.21.15.115:0/3834072394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79374a0 msgr2=0x55eec7abe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fd006a71700 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79374a0 0x55eec7abe0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fd006a71700 1 -- 172.21.15.115:0/3834072394 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec7852490 con 0x55eec7ab4400 2024-05-09T05:47:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fd005a6f700 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 0x55eec7abe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fd006270700 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79374a0 0x55eec7abe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fd006a71700 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec7ab4400 0x55eec7abeb30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c4d0 tx=0x7fcff800a580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8375a4643323034b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fcfe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:47:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/3834072394 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8023040 con 0x55eec7ab4400 2024-05-09T05:47:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/3834072394 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fcff8008b20 con 0x55eec7ab4400 2024-05-09T05:47:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fd016eeab80 1 -- 172.21.15.115:0/3834072394 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff800a350 con 0x55eec7ab4400 2024-05-09T05:47:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/3834072394 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcff8038050 con 0x55eec7ab4400 2024-05-09T05:47:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fd016eeab80 10 client.27388 fetch_fsmap learned FSMap version 47 2024-05-09T05:47:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fd016eeab80 1 -- 172.21.15.115:0/3834072394 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55eec7acbba0 con 0x55eec7ab4400 2024-05-09T05:47:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.489+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/3834072394 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcff800a350 con 0x55eec7ab4400 2024-05-09T05:47:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/3834072394 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fcff8012070 con 0x55eec7ab4400 2024-05-09T05:47:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fd016eeab80 10 client.27388 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T05:47:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fd016eeab80 10 client.27388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fd016eeab80 20 client.27388 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fd016eeab80 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55eec7ace070 0x55eec7ad0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fd016eeab80 4 client.27388 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:33.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fd016eeab80 1 -- 172.21.15.115:0/3834072394 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eec79e4a80 con 0x55eec7ace070 2024-05-09T05:47:33.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fd006270700 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55eec7ace070 0x55eec7ad0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fcfe9ffb700 10 client.27388 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:33.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.490+0000 7fd006270700 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55eec7ace070 0x55eec7ad0490 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c29c18ac2501df server_cookie=cad8c4d78f6858f in_seq=0 out_seq=0 2024-05-09T05:47:33.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.491+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/3834072394 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eec79e4a80 con 0x55eec7ace070 2024-05-09T05:47:33.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.491+0000 7fcfe9ffb700 10 client.27388 handle_command_reply: tid=0 2024-05-09T05:47:33.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 client.27388 shutdown 2024-05-09T05:47:33.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 -- 172.21.15.115:0/3834072394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec7ab4400 msgr2=0x55eec7abeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:33.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec7ab4400 0x55eec7abeb30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c4d0 tx=0x7fcff800a580 comp rx=0 tx=0).stop 2024-05-09T05:47:33.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 -- 172.21.15.115:0/3834072394 shutdown_connections 2024-05-09T05:47:33.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55eec7ace070 0x55eec7ad0490 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec7ab4400 0x55eec7abeb30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79374a0 0x55eec7abe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 --2- 172.21.15.115:0/3834072394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79b1d00 0x55eec7abe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 -- 172.21.15.115:0/3834072394 >> 172.21.15.115:0/3834072394 conn(0x55eec793af70 msgr2=0x55eec7aa8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fcfe9ffb700 0 client.27388 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 -- 172.21.15.115:0/3834072394 shutdown_connections 2024-05-09T05:47:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 1 -- 172.21.15.115:0/3834072394 wait complete. 2024-05-09T05:47:33.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.492+0000 7fd016eeab80 20 client.27388 trim_cache size 0 max 16384 2024-05-09T05:47:33.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.534+0000 7fd016eeab80 1 Processor -- start 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.534+0000 7fd016eeab80 1 -- start start 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 0x55eec7aa9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7ac0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7abd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd016eeab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001b24d0 con 0x55eec79b1d00 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd016eeab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001b4130 con 0x55eec79374a0 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd016eeab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001b2ba0 con 0x55eec793af70 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006a71700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7abd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006270700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 0x55eec7aa9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd005a6f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7ac0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006a71700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7abd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56360/0 (socket says 172.21.15.115:56360) 2024-05-09T05:47:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006a71700 1 -- 172.21.15.115:0/19279837 learned_addr learned my addr 172.21.15.115:0/19279837 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006a71700 1 -- 172.21.15.115:0/19279837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 msgr2=0x55eec7aa9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006a71700 1 --2- 172.21.15.115:0/19279837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 0x55eec7aa9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006a71700 1 -- 172.21.15.115:0/19279837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 msgr2=0x55eec7ac0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006a71700 1 --2- 172.21.15.115:0/19279837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7ac0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006a71700 1 -- 172.21.15.115:0/19279837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec7aab9d0 con 0x55eec79b1d00 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd005a6f700 1 --2- 172.21.15.115:0/19279837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7ac0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.535+0000 7fd006a71700 1 --2- 172.21.15.115:0/19279837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7abd880 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fcff8006090 tx=0x7fcff800a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1e5a04fed84a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.536+0000 7fcfebfff700 1 -- 172.21.15.115:0/19279837 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8023070 con 0x55eec79b1d00 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.536+0000 7fcfebfff700 1 -- 172.21.15.115:0/19279837 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcff8008b20 con 0x55eec79b1d00 2024-05-09T05:47:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.536+0000 7fcfebfff700 1 -- 172.21.15.115:0/19279837 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff801d440 con 0x55eec79b1d00 2024-05-09T05:47:33.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 -- 172.21.15.115:0/19279837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 msgr2=0x55eec7abd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:33.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 --2- 172.21.15.115:0/19279837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7abd880 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fcff8006090 tx=0x7fcff800a5f0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 -- 172.21.15.115:0/19279837 shutdown_connections 2024-05-09T05:47:33.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 --2- 172.21.15.115:0/19279837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7abd880 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 --2- 172.21.15.115:0/19279837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 0x55eec7aa9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 --2- 172.21.15.115:0/19279837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7ac0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 -- 172.21.15.115:0/19279837 >> 172.21.15.115:0/19279837 conn(0x55eec7ab4ae0 msgr2=0x55eec7ab7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:33.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 -- 172.21.15.115:0/19279837 shutdown_connections 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 -- 172.21.15.115:0/19279837 wait complete. 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 -- start start 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 Processor -- start 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7b309f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 0x55eec7b30f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7ad04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001b3a60 con 0x55eec79b1d00 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001b3300 con 0x55eec79374a0 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.537+0000 7fd016eeab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001b0470 con 0x55eec793af70 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd016eeab80 10 client.0 _ll_register_callbacks cb 0x55eec7aaf880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd006270700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7b309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd006a71700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7ad04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd005a6f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 0x55eec7b30f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd006270700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7b309f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:57818/0 (socket says 172.21.15.115:57818) 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd006270700 1 -- 172.21.15.115:0/1391150273 learned_addr learned my addr 172.21.15.115:0/1391150273 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd005a6f700 1 -- 172.21.15.115:0/1391150273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 msgr2=0x55eec7b309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd005a6f700 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7b309f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd005a6f700 1 -- 172.21.15.115:0/1391150273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 msgr2=0x55eec7ad04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd005a6f700 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7ad04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd005a6f700 1 -- 172.21.15.115:0/1391150273 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec7ad0b00 con 0x55eec793af70 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd005a6f700 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 0x55eec7b30f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcff00e0090 tx=0x7fcff00aa180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18dbececb9d01ea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fcfe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/1391150273 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00c4300 con 0x55eec793af70 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/1391150273 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fcff00a9090 con 0x55eec793af70 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.538+0000 7fd016eeab80 1 -- 172.21.15.115:0/1391150273 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff8008b20 con 0x55eec793af70 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/1391150273 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fcff00b23c0 con 0x55eec793af70 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fd016eeab80 10 client.27394 fetch_fsmap learned FSMap version 47 2024-05-09T05:47:33.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fd016eeab80 1 -- 172.21.15.115:0/1391150273 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55eec7b38060 con 0x55eec793af70 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/1391150273 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcff8008b20 con 0x55eec793af70 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/1391150273 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 47) ==== 8386+0+0 (secure 0 0 0) 0x7fcff00a88f0 con 0x55eec793af70 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fd016eeab80 10 client.27394 fetch_fsmap finished waiting for FSMap version 47 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fd016eeab80 10 client.27394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fd016eeab80 20 client.27394 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55eec7ab4400 0x55eec7b3a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fd016eeab80 4 client.27394 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fd016eeab80 1 -- 172.21.15.115:0/1391150273 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eec79deb20 con 0x55eec7ab4400 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.539+0000 7fd006270700 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55eec7ab4400 0x55eec7b3a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fcfe9ffb700 10 client.27394 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd006270700 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55eec7ab4400 0x55eec7b3a860 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d49372d2e6649488 server_cookie=eeb6b08e7d2bbbb0 in_seq=0 out_seq=0 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fcfe9ffb700 1 -- 172.21.15.115:0/1391150273 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eec79deb20 con 0x55eec7ab4400 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fcfe9ffb700 10 client.27394 handle_command_reply: tid=0 2024-05-09T05:47:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 client.27394 shutdown 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 -- 172.21.15.115:0/1391150273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 msgr2=0x55eec7b30f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 0x55eec7b30f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcff00e0090 tx=0x7fcff00aa180 comp rx=0 tx=0).stop 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 -- 172.21.15.115:0/1391150273 shutdown_connections 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eec79374a0 0x55eec7b309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55eec7ab4400 0x55eec7b3a860 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eec79b1d00 0x55eec7ad04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fcfe9ffb700 0 client.27394 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 --2- 172.21.15.115:0/1391150273 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eec793af70 0x55eec7b30f30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 -- 172.21.15.115:0/1391150273 >> 172.21.15.115:0/1391150273 conn(0x55eec7ab4ae0 msgr2=0x55eec7aa9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 -- 172.21.15.115:0/1391150273 shutdown_connections 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.540+0000 7fd016eeab80 1 -- 172.21.15.115:0/1391150273 wait complete. 2024-05-09T05:47:33.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd016eeab80 20 client.27394 trim_cache size 0 max 16384 2024-05-09T05:47:33.540 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:47:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: "status": "no active scrubs running", 2024-05-09T05:47:33.540 INFO:teuthology.orchestra.run.smithi115.stdout: "scrubs": {} 2024-05-09T05:47:33.540 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:47:33.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 -- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfec0b29f0 msgr2=0x7fcfec0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:33.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfec0b29f0 0x7fcfec0b4f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd000103b00 tx=0x7fcffc009260 comp rx=0 tx=0).stop 2024-05-09T05:47:33.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 -- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd000103310 msgr2=0x7fd00019d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:33.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd000103310 0x7fd00019d4e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fcff000ec70 tx=0x7fcff000ef80 comp rx=0 tx=0).stop 2024-05-09T05:47:33.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 -- 172.21.15.115:0/378970594 shutdown_connections 2024-05-09T05:47:33.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfec0b29f0 0x7fcfec0b4f00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0001029c0 0x7fd00019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0001089f0 0x7fd0001a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 --2- 172.21.15.115:0/378970594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd000103310 0x7fd00019d4e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:33.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 -- 172.21.15.115:0/378970594 >> 172.21.15.115:0/378970594 conn(0x7fd0000fe6f0 msgr2=0x7fd000100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:33.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 -- 172.21.15.115:0/378970594 shutdown_connections 2024-05-09T05:47:33.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:33.541+0000 7fd0084d4700 1 -- 172.21.15.115:0/378970594 wait complete. 2024-05-09T05:47:33.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:33 smithi195 ceph-mon[33844]: pgmap v291: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-09T05:47:33.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:33 smithi161 ceph-mon[33805]: pgmap v291: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-09T05:47:34.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:34 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:47:34.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:34 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:47:34.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:34 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:47:34.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:34 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:47:34.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:34 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:47:34.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:34 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:47:34.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:34 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:47:34.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:34 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:47:34.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:34 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:47:34.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:34 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:47:34.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:34 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:47:34.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:34 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:47:34.677 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:47:34.677 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:47:34.919 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:35.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:35 smithi115 ceph-mon[30349]: pgmap v292: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 22 op/s 2024-05-09T05:47:35.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:35 smithi195 ceph-mon[33844]: pgmap v292: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 22 op/s 2024-05-09T05:47:35.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:35 smithi161 ceph-mon[33805]: pgmap v292: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 22 op/s 2024-05-09T05:47:37.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:37 smithi115 ceph-mon[30349]: pgmap v293: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-09T05:47:37.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:37 smithi195 ceph-mon[33844]: pgmap v293: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-09T05:47:37.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:37 smithi161 ceph-mon[33805]: pgmap v293: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-09T05:47:39.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:39 smithi195 ceph-mon[33844]: pgmap v294: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 17 op/s 2024-05-09T05:47:39.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:39 smithi115 ceph-mon[30349]: pgmap v294: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 17 op/s 2024-05-09T05:47:39.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:39 smithi161 ceph-mon[33805]: pgmap v294: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 17 op/s 2024-05-09T05:47:40.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:40 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:47:40.624 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:40 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:47:40.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:40 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:47:41.237 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:41 smithi115 ceph-mon[30349]: pgmap v295: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 418 KiB/s wr, 10 op/s 2024-05-09T05:47:41.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 -- 172.21.15.115:0/240744384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 msgr2=0x7f92f8102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:41.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 --2- 172.21.15.115:0/240744384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 0x7f92f8102dd0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009990 tx=0x7f92e4009ca0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 -- 172.21.15.115:0/240744384 shutdown_connections 2024-05-09T05:47:41.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 --2- 172.21.15.115:0/240744384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92f81089f0 0x7f92f8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 --2- 172.21.15.115:0/240744384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92f8103310 0x7f92f810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 --2- 172.21.15.115:0/240744384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 0x7f92f8102dd0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 -- 172.21.15.115:0/240744384 >> 172.21.15.115:0/240744384 conn(0x7f92f80fe520 msgr2=0x7f92f8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:41.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 -- 172.21.15.115:0/240744384 shutdown_connections 2024-05-09T05:47:41.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 -- 172.21.15.115:0/240744384 wait complete. 2024-05-09T05:47:41.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.259+0000 7f92feef0700 1 Processor -- start 2024-05-09T05:47:41.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92feef0700 1 -- start start 2024-05-09T05:47:41.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92feef0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 0x7f92f8110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:41.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92feef0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92f8103310 0x7f92f81114f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:41.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92feef0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92f81089f0 0x7f92f8111a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:41.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92feef0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92f81143a0 con 0x7f92f81029c0 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92feef0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92f8114220 con 0x7f92f81089f0 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92feef0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92f8114520 con 0x7f92f8103310 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fcc8c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 0x7f92f8110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fcc8c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 0x7f92f8110fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56386/0 (socket says 172.21.15.115:56386) 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fcc8c700 1 -- 172.21.15.115:0/559398647 learned_addr learned my addr 172.21.15.115:0/559398647 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fd48d700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92f81089f0 0x7f92f8111a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92f7fff700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92f8103310 0x7f92f81114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fcc8c700 1 -- 172.21.15.115:0/559398647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92f8103310 msgr2=0x7f92f81114f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fcc8c700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92f8103310 0x7f92f81114f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fcc8c700 1 -- 172.21.15.115:0/559398647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92f81089f0 msgr2=0x7f92f8111a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fcc8c700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92f81089f0 0x7f92f8111a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fcc8c700 1 -- 172.21.15.115:0/559398647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f81a9040 con 0x7f92f81029c0 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.260+0000 7f92fcc8c700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 0x7f92f8110fb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c9e0 tx=0x7f92ec00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:41.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.261+0000 7f92f5ffb700 1 -- 172.21.15.115:0/559398647 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec007aa0 con 0x7f92f81029c0 2024-05-09T05:47:41.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.261+0000 7f92f5ffb700 1 -- 172.21.15.115:0/559398647 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92ec007c40 con 0x7f92f81029c0 2024-05-09T05:47:41.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.261+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f81a9330 con 0x7f92f81029c0 2024-05-09T05:47:41.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.261+0000 7f92f5ffb700 1 -- 172.21.15.115:0/559398647 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec018660 con 0x7f92f81029c0 2024-05-09T05:47:41.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.261+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f81a9870 con 0x7f92f81029c0 2024-05-09T05:47:41.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.262+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f80512c0 con 0x7f92f81029c0 2024-05-09T05:47:41.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.263+0000 7f92f5ffb700 1 -- 172.21.15.115:0/559398647 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f92ec00f450 con 0x7f92f81029c0 2024-05-09T05:47:41.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.264+0000 7f92f5ffb700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92e80b29f0 0x7f92e80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:41.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.264+0000 7f92f7fff700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92e80b29f0 0x7f92e80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:41.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.264+0000 7f92f5ffb700 1 -- 172.21.15.115:0/559398647 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f92ec0e2880 con 0x7f92f81029c0 2024-05-09T05:47:41.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.265+0000 7f92f7fff700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92e80b29f0 0x7f92e80b4f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92e4006010 tx=0x7f92e400b540 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:41.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.269+0000 7f92f5ffb700 1 -- 172.21.15.115:0/559398647 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f92ec014030 con 0x7f92f81029c0 2024-05-09T05:47:41.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.518+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92f802e5f0 con 0x7f92f81029c0 2024-05-09T05:47:41.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.519+0000 7f92f5ffb700 1 -- 172.21.15.115:0/559398647 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11957 (secure 0 0 0) 0x7f92ec0a9f20 con 0x7f92f81029c0 2024-05-09T05:47:41.512 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:47:41.512 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:47:38.366444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27316":{"gid":27316,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/391681574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":391681574},{"type":"v1","addr":"172.21.15.195:6833","nonce":391681574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:47:41.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92e80b29f0 msgr2=0x7f92e80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92e80b29f0 0x7f92e80b4f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92e4006010 tx=0x7f92e400b540 comp rx=0 tx=0).stop 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 msgr2=0x7f92f8110fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 0x7f92f8110fb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c9e0 tx=0x7f92ec00ccf0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 shutdown_connections 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92e80b29f0 0x7f92e80b4f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92f81089f0 0x7f92f8111a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92f81029c0 0x7f92f8110fb0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 --2- 172.21.15.115:0/559398647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92f8103310 0x7f92f81114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.521+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 >> 172.21.15.115:0/559398647 conn(0x7f92f80fe520 msgr2=0x7f92f8100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.522+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 shutdown_connections 2024-05-09T05:47:41.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:41.522+0000 7f92feef0700 1 -- 172.21.15.115:0/559398647 wait complete. 2024-05-09T05:47:41.516 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 48 2024-05-09T05:47:41.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:41 smithi195 ceph-mon[33844]: pgmap v295: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 418 KiB/s wr, 10 op/s 2024-05-09T05:47:41.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:41 smithi161 ceph-mon[33805]: pgmap v295: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 418 KiB/s wr, 10 op/s 2024-05-09T05:47:42.244 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-09T05:47:42.486 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:42.608 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:42 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/559398647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:47:42.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:42 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/559398647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:47:42.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:42 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/559398647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:47:43.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:43 smithi195 ceph-mon[33844]: pgmap v296: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 418 KiB/s wr, 10 op/s 2024-05-09T05:47:43.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:43 smithi115 ceph-mon[30349]: pgmap v296: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 418 KiB/s wr, 10 op/s 2024-05-09T05:47:43.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:43 smithi161 ceph-mon[33805]: pgmap v296: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 418 KiB/s wr, 10 op/s 2024-05-09T05:47:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.698+0000 7fc363b3f640 1 -- 172.21.15.115:0/2020582626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c106b10 msgr2=0x7fc35c106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.698+0000 7fc363b3f640 1 --2- 172.21.15.115:0/2020582626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c106b10 0x7fc35c106f40 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009900 tx=0x7fc34c02a410 comp rx=0 tx=0).stop 2024-05-09T05:47:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.699+0000 7fc363b3f640 1 -- 172.21.15.115:0/2020582626 shutdown_connections 2024-05-09T05:47:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.699+0000 7fc363b3f640 1 --2- 172.21.15.115:0/2020582626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c1075d0 0x7fc35c10ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.699+0000 7fc363b3f640 1 --2- 172.21.15.115:0/2020582626 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c106b10 0x7fc35c106f40 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.699+0000 7fc363b3f640 1 --2- 172.21.15.115:0/2020582626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c102580 0x7fc35c102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.699+0000 7fc363b3f640 1 -- 172.21.15.115:0/2020582626 >> 172.21.15.115:0/2020582626 conn(0x7fc35c0779e0 msgr2=0x7fc35c077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.699+0000 7fc363b3f640 1 -- 172.21.15.115:0/2020582626 shutdown_connections 2024-05-09T05:47:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.699+0000 7fc363b3f640 1 -- 172.21.15.115:0/2020582626 wait complete. 2024-05-09T05:47:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.699+0000 7fc363b3f640 1 Processor -- start 2024-05-09T05:47:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.699+0000 7fc363b3f640 1 -- start start 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc363b3f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c102580 0x7fc35c1a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc363b3f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c106b10 0x7fc35c1a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc363b3f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c1075d0 0x7fc35c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc363b3f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c111b10 con 0x7fc35c1075d0 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc363b3f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c111990 con 0x7fc35c102580 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc363b3f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c111c90 con 0x7fc35c106b10 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3620dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c1075d0 0x7fc35c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3618db640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c102580 0x7fc35c1a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3620dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c1075d0 0x7fc35c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43718/0 (socket says 172.21.15.115:43718) 2024-05-09T05:47:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3620dc640 1 -- 172.21.15.115:0/3450543211 learned_addr learned my addr 172.21.15.115:0/3450543211 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3610da640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c106b10 0x7fc35c1a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3620dc640 1 -- 172.21.15.115:0/3450543211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c106b10 msgr2=0x7fc35c1a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3620dc640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c106b10 0x7fc35c1a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3620dc640 1 -- 172.21.15.115:0/3450543211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c102580 msgr2=0x7fc35c1a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3620dc640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c102580 0x7fc35c1a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3620dc640 1 -- 172.21.15.115:0/3450543211 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c19cf40 con 0x7fc35c1075d0 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.700+0000 7fc3620dc640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c1075d0 0x7fc35c19c860 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc35400ddc0 tx=0x7fc35400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.701+0000 7fc352ffd640 1 -- 172.21.15.115:0/3450543211 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354004020 con 0x7fc35c1075d0 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.701+0000 7fc352ffd640 1 -- 172.21.15.115:0/3450543211 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3540041c0 con 0x7fc35c1075d0 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.701+0000 7fc363b3f640 1 -- 172.21.15.115:0/3450543211 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35c19d230 con 0x7fc35c1075d0 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.701+0000 7fc352ffd640 1 -- 172.21.15.115:0/3450543211 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35400e740 con 0x7fc35c1075d0 2024-05-09T05:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.701+0000 7fc363b3f640 1 -- 172.21.15.115:0/3450543211 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35c1a9480 con 0x7fc35c1075d0 2024-05-09T05:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.703+0000 7fc352ffd640 1 -- 172.21.15.115:0/3450543211 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc3540035f0 con 0x7fc35c1075d0 2024-05-09T05:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.703+0000 7fc36a9f6740 1 Processor -- start 2024-05-09T05:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc36a9f6740 1 -- start start 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cda061b0 0x55c3cdda9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 0x55c3cdda9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9e54d0 0x55c3cddaa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c3cdd0e7c0 con 0x55c3cda061b0 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c3cdcaa0e0 con 0x55c3cd9e54d0 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c3cdd0a780 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc3610da640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cda061b0 0x55c3cdda9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc3610da640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cda061b0 0x55c3cdda9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43730/0 (socket says 172.21.15.115:43730) 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc3610da640 1 -- 172.21.15.115:0/3077821132 learned_addr learned my addr 172.21.15.115:0/3077821132 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc3618db640 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 0x55c3cdda9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc3610da640 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9e54d0 0x55c3cddaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc352ffd640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3480b2950 0x7fc3480b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc3610da640 1 -- 172.21.15.115:0/3077821132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 msgr2=0x55c3cdda9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc3610da640 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 0x55c3cdda9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc3610da640 1 -- 172.21.15.115:0/3077821132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9e54d0 msgr2=0x55c3cddaa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.704+0000 7fc3610da640 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9e54d0 0x55c3cddaa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.705+0000 7fc3610da640 1 -- 172.21.15.115:0/3077821132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3cdcd37c0 con 0x55c3cda061b0 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.705+0000 7fc3618db640 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 0x55c3cdda9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.705+0000 7fc3620dc640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3480b2950 0x7fc3480b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.705+0000 7fc352ffd640 1 -- 172.21.15.115:0/3450543211 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fc3540e26a0 con 0x7fc35c1075d0 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.705+0000 7fc3610da640 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cda061b0 0x55c3cdda9680 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0057c0 tx=0x7fc34c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb8c8f76644946f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.705+0000 7fc347fff640 1 -- 172.21.15.115:0/3077821132 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c035070 con 0x55c3cda061b0 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.705+0000 7fc347fff640 1 -- 172.21.15.115:0/3077821132 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc34c0044c0 con 0x55c3cda061b0 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.705+0000 7fc347fff640 1 -- 172.21.15.115:0/3077821132 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c003bd0 con 0x55c3cda061b0 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 -- 172.21.15.115:0/3077821132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cda061b0 msgr2=0x55c3cdda9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cda061b0 0x55c3cdda9680 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0057c0 tx=0x7fc34c005fd0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc3620dc640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3480b2950 0x7fc3480b4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc3540e00b0 tx=0x7fc35409e080 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 -- 172.21.15.115:0/3077821132 shutdown_connections 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9e54d0 0x55c3cddaa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 0x55c3cdda9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/3077821132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cda061b0 0x55c3cdda9680 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 -- 172.21.15.115:0/3077821132 >> 172.21.15.115:0/3077821132 conn(0x55c3cd9fa300 msgr2=0x55c3cdda5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 -- 172.21.15.115:0/3077821132 shutdown_connections 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 -- 172.21.15.115:0/3077821132 wait complete. 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 -- start start 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.706+0000 7fc36a9f6740 1 Processor -- start 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9e54d0 0x55c3cddb5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cda061b0 0x55c3cddb3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c3cddb6750 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c3cdc959c0 con 0x55c3cda061b0 2024-05-09T05:47:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c3cddb68d0 con 0x55c3cd9e54d0 2024-05-09T05:47:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc36a9f6740 10 client.0 _ll_register_callbacks cb 0x55c3cdcc2020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3610da640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3610da640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43746/0 (socket says 172.21.15.115:43746) 2024-05-09T05:47:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3610da640 1 -- 172.21.15.115:0/29757126 learned_addr learned my addr 172.21.15.115:0/29757126 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3618db640 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9e54d0 0x55c3cddb5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3610da640 1 -- 172.21.15.115:0/29757126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9e54d0 msgr2=0x55c3cddb5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3610da640 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9e54d0 0x55c3cddb5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3610da640 1 -- 172.21.15.115:0/29757126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cda061b0 msgr2=0x55c3cddb3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3610da640 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cda061b0 0x55c3cddb3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3610da640 1 -- 172.21.15.115:0/29757126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3cdcbc1f0 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3610da640 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb57e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc34c02a450 tx=0x7fc34c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0079ef408b4abb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.707+0000 7fc3457fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc3457fa640 1 -- 172.21.15.115:0/29757126 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c04f020 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc3457fa640 1 -- 172.21.15.115:0/29757126 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fc34c003950 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc36a9f6740 1 -- 172.21.15.115:0/29757126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c3cdd1c3d0 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc3457fa640 1 -- 172.21.15.115:0/29757126 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc34c05a050 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc36a9f6740 10 client.18021 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc36a9f6740 1 -- 172.21.15.115:0/29757126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c3cda71790 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc3457fa640 1 -- 172.21.15.115:0/29757126 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c3cdd1c3d0 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc3457fa640 1 -- 172.21.15.115:0/29757126 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fc34c03b460 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc36a9f6740 10 client.18021 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc36a9f6740 10 client.18021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc36a9f6740 20 client.18021 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c3cd9c4d80 0x55c3cddc5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc36a9f6740 4 client.18021 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.708+0000 7fc36a9f6740 1 -- 172.21.15.115:0/29757126 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3cdd16240 con 0x55c3cd9c4d80 2024-05-09T05:47:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.709+0000 7fc3618db640 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c3cd9c4d80 0x55c3cddc5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.709+0000 7fc3457fa640 10 client.18021 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.709+0000 7fc3618db640 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c3cd9c4d80 0x55c3cddc5e80 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af50769cd9fd05b server_cookie=45aaad6c7a15f4ec in_seq=0 out_seq=0 2024-05-09T05:47:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.710+0000 7fc3457fa640 1 -- 172.21.15.115:0/29757126 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c3cdd16240 con 0x55c3cd9c4d80 2024-05-09T05:47:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.710+0000 7fc3457fa640 10 client.18021 handle_command_reply: tid=0 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.710+0000 7fc36a9f6740 1 client.18021 shutdown 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.710+0000 7fc36a9f6740 1 -- 172.21.15.115:0/29757126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9dd2f0 msgr2=0x55c3cddb57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.710+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb57e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc34c02a450 tx=0x7fc34c0092c0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc36a9f6740 1 -- 172.21.15.115:0/29757126 shutdown_connections 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cda061b0 0x55c3cddb3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c3cd9c4d80 0x55c3cddc5e80 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9e54d0 0x55c3cddb5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/29757126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb57e0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc3457fa640 0 client.18021 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc36a9f6740 1 -- 172.21.15.115:0/29757126 >> 172.21.15.115:0/29757126 conn(0x55c3cd9fa300 msgr2=0x55c3cdda4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc36a9f6740 1 -- 172.21.15.115:0/29757126 shutdown_connections 2024-05-09T05:47:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc36a9f6740 1 -- 172.21.15.115:0/29757126 wait complete. 2024-05-09T05:47:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.711+0000 7fc36a9f6740 20 client.18021 trim_cache size 0 max 16384 2024-05-09T05:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc36a9f6740 1 Processor -- start 2024-05-09T05:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc36a9f6740 1 -- start start 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cda061b0 0x55c3cddad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b3180 con 0x55c3cd9e54d0 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b4d80 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b3850 con 0x55c3cda061b0 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc3618db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc3618db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddad370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43756/0 (socket says 172.21.15.115:43756) 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc3618db640 1 -- 172.21.15.115:0/478405624 learned_addr learned my addr 172.21.15.115:0/478405624 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc3610da640 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cda061b0 0x55c3cddad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.726+0000 7fc3610da640 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3618db640 1 -- 172.21.15.115:0/478405624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cda061b0 msgr2=0x55c3cddad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3618db640 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cda061b0 0x55c3cddad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3618db640 1 -- 172.21.15.115:0/478405624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9dd2f0 msgr2=0x55c3cddb3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3618db640 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3618db640 1 -- 172.21.15.115:0/478405624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3cdae85f0 con 0x55c3cd9e54d0 2024-05-09T05:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3610da640 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cda061b0 0x55c3cddad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3618db640 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddad370 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x55c3cdda7170 tx=0x7fc358015ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a562bb4f54d281 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3477fe640 1 -- 172.21.15.115:0/478405624 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358004b20 con 0x55c3cd9e54d0 2024-05-09T05:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3477fe640 1 -- 172.21.15.115:0/478405624 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc358004cc0 con 0x55c3cd9e54d0 2024-05-09T05:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.727+0000 7fc3477fe640 1 -- 172.21.15.115:0/478405624 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3580136e0 con 0x55c3cd9e54d0 2024-05-09T05:47:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 -- 172.21.15.115:0/478405624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 msgr2=0x55c3cddad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddad370 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x55c3cdda7170 tx=0x7fc358015ad0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 -- 172.21.15.115:0/478405624 shutdown_connections 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cda061b0 0x55c3cddad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddad370 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/478405624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddb3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 -- 172.21.15.115:0/478405624 >> 172.21.15.115:0/478405624 conn(0x55c3cd9fa300 msgr2=0x55c3cddb34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 -- 172.21.15.115:0/478405624 shutdown_connections 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 -- 172.21.15.115:0/478405624 wait complete. 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.728+0000 7fc36a9f6740 1 -- start start 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc36a9f6740 1 Processor -- start 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddbd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddbd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc36a9f6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cda061b0 0x55c3cde2bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b46b0 con 0x55c3cd9e54d0 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b3f50 con 0x55c3cda061b0 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc36a9f6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b1160 con 0x55c3cd9dd2f0 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc36a9f6740 10 client.0 _ll_register_callbacks cb 0x7fc34c03e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc3618db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddbd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc3618db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddbd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43766/0 (socket says 172.21.15.115:43766) 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc3618db640 1 -- 172.21.15.115:0/2422700278 learned_addr learned my addr 172.21.15.115:0/2422700278 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc3610da640 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddbd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc3610da640 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cda061b0 0x55c3cde2bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc3618db640 1 -- 172.21.15.115:0/2422700278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 msgr2=0x55c3cddbd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc3618db640 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddbd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc3618db640 1 -- 172.21.15.115:0/2422700278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cda061b0 msgr2=0x55c3cde2bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.729+0000 7fc3618db640 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cda061b0 0x55c3cde2bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc3618db640 1 -- 172.21.15.115:0/2422700278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3cdcd2ed0 con 0x55c3cd9e54d0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc3618db640 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddbd840 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc3580135d0 tx=0x7fc358004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fe75cb7dd68c14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc3457fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc3457fa640 1 -- 172.21.15.115:0/2422700278 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358015460 con 0x55c3cd9e54d0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc3457fa640 1 -- 172.21.15.115:0/2422700278 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fc35801e420 con 0x55c3cd9e54d0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc36a9f6740 1 -- 172.21.15.115:0/2422700278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc358004cc0 con 0x55c3cd9e54d0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc3457fa640 1 -- 172.21.15.115:0/2422700278 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc358039050 con 0x55c3cd9e54d0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc36a9f6740 10 client.18027 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc36a9f6740 1 -- 172.21.15.115:0/2422700278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c3cde2d360 con 0x55c3cd9e54d0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.730+0000 7fc3457fa640 1 -- 172.21.15.115:0/2422700278 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc358004cc0 con 0x55c3cd9e54d0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc3457fa640 1 -- 172.21.15.115:0/2422700278 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fc35800f040 con 0x55c3cd9e54d0 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc36a9f6740 10 client.18027 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc36a9f6740 10 client.18027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc36a9f6740 20 client.18027 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c3cd99a1c0 0x55c3cde2faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc36a9f6740 4 client.18027 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:47:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc36a9f6740 1 -- 172.21.15.115:0/2422700278 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3cdc91350 con 0x55c3cd99a1c0 2024-05-09T05:47:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc3610da640 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c3cd99a1c0 0x55c3cde2faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc3610da640 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c3cd99a1c0 0x55c3cde2faf0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5cb923718ae26bf server_cookie=d8d4bace89191a18 in_seq=0 out_seq=0 2024-05-09T05:47:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.731+0000 7fc3457fa640 10 client.18027 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.739+0000 7fc3457fa640 1 -- 172.21.15.115:0/2422700278 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c3cdc91350 con 0x55c3cd99a1c0 2024-05-09T05:47:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.739+0000 7fc3457fa640 10 client.18027 handle_command_reply: tid=0 2024-05-09T05:47:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 client.18027 shutdown 2024-05-09T05:47:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 -- 172.21.15.115:0/2422700278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 msgr2=0x55c3cddbd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddbd840 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc3580135d0 tx=0x7fc358004ab0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 -- 172.21.15.115:0/2422700278 shutdown_connections 2024-05-09T05:47:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3cda061b0 0x55c3cde2bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c3cd9e54d0 0x55c3cddbd840 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c3cd99a1c0 0x55c3cde2faf0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 --2- 172.21.15.115:0/2422700278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c3cd9dd2f0 0x55c3cddbd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 -- 172.21.15.115:0/2422700278 >> 172.21.15.115:0/2422700278 conn(0x55c3cd9fa300 msgr2=0x55c3cddad370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc3457fa640 0 client.18027 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 -- 172.21.15.115:0/2422700278 shutdown_connections 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 1 -- 172.21.15.115:0/2422700278 wait complete. 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc36a9f6740 20 client.18027 trim_cache size 0 max 16384 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:47:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:47:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc363b3f640 1 -- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3480b2950 msgr2=0x7fc3480b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc363b3f640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3480b2950 0x7fc3480b4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc3540e00b0 tx=0x7fc35409e080 comp rx=0 tx=0).stop 2024-05-09T05:47:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc363b3f640 1 -- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c1075d0 msgr2=0x7fc35c19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.740+0000 7fc363b3f640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c1075d0 0x7fc35c19c860 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc35400ddc0 tx=0x7fc35400a620 comp rx=0 tx=0).stop 2024-05-09T05:47:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.741+0000 7fc363b3f640 1 -- 172.21.15.115:0/3450543211 shutdown_connections 2024-05-09T05:47:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.741+0000 7fc363b3f640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc3480b2950 0x7fc3480b4e60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.741+0000 7fc363b3f640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc35c1075d0 0x7fc35c19c860 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.741+0000 7fc363b3f640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc35c106b10 0x7fc35c1a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.741+0000 7fc363b3f640 1 --2- 172.21.15.115:0/3450543211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c102580 0x7fc35c1a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.741+0000 7fc363b3f640 1 -- 172.21.15.115:0/3450543211 >> 172.21.15.115:0/3450543211 conn(0x7fc35c0779e0 msgr2=0x7fc35c100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.741+0000 7fc363b3f640 1 -- 172.21.15.115:0/3450543211 shutdown_connections 2024-05-09T05:47:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:43.741+0000 7fc363b3f640 1 -- 172.21.15.115:0/3450543211 wait complete. 2024-05-09T05:47:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.261+0000 7f4980bc3700 1 -- 172.21.15.115:0/3510903925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c103540 msgr2=0x7f497c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.261+0000 7f4980bc3700 1 --2- 172.21.15.115:0/3510903925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c103540 0x7f497c103950 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f49640099c0 tx=0x7f4964009cd0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.261+0000 7f4980bc3700 1 -- 172.21.15.115:0/3510903925 shutdown_connections 2024-05-09T05:47:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.261+0000 7f4980bc3700 1 --2- 172.21.15.115:0/3510903925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c108770 0x7f497c1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.261+0000 7f4980bc3700 1 --2- 172.21.15.115:0/3510903925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c103540 0x7f497c103950 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.261+0000 7f4980bc3700 1 --2- 172.21.15.115:0/3510903925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c107e20 0x7f497c108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.261+0000 7f4980bc3700 1 -- 172.21.15.115:0/3510903925 >> 172.21.15.115:0/3510903925 conn(0x7f497c078e50 msgr2=0x7f497c079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:44.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.261+0000 7f4980bc3700 1 -- 172.21.15.115:0/3510903925 shutdown_connections 2024-05-09T05:47:44.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.261+0000 7f4980bc3700 1 -- 172.21.15.115:0/3510903925 wait complete. 2024-05-09T05:47:44.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4980bc3700 1 Processor -- start 2024-05-09T05:47:44.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4980bc3700 1 -- start start 2024-05-09T05:47:44.255 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4980bc3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c103540 0x7f497c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4980bc3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c107e20 0x7f497c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4980bc3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c108770 0x7f497c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4980bc3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f497c113250 con 0x7f497c107e20 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4980bc3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f497c1130d0 con 0x7f497c108770 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4980bc3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f497c1133d0 con 0x7f497c103540 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4979d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c107e20 0x7f497c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f497a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c103540 0x7f497c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4979d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c107e20 0x7f497c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43782/0 (socket says 172.21.15.115:43782) 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f4979d9b700 1 -- 172.21.15.115:0/2482908822 learned_addr learned my addr 172.21.15.115:0/2482908822 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f497a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c103540 0x7f497c19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:49232/0 (socket says 172.21.15.115:49232) 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.262+0000 7f497ad9d700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c108770 0x7f497c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f4979d9b700 1 -- 172.21.15.115:0/2482908822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c103540 msgr2=0x7f497c19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f4979d9b700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c103540 0x7f497c19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f4979d9b700 1 -- 172.21.15.115:0/2482908822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c108770 msgr2=0x7f497c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:44.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f4979d9b700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c108770 0x7f497c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f4979d9b700 1 -- 172.21.15.115:0/2482908822 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c19fc20 con 0x7f497c107e20 2024-05-09T05:47:44.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f4979d9b700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c107e20 0x7f497c19b2f0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f496c00d980 tx=0x7f496c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:44.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f49737fe700 1 -- 172.21.15.115:0/2482908822 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c017070 con 0x7f497c107e20 2024-05-09T05:47:44.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f4980bc3700 1 -- 172.21.15.115:0/2482908822 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497c19ff10 con 0x7f497c107e20 2024-05-09T05:47:44.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f49737fe700 1 -- 172.21.15.115:0/2482908822 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f496c00fd20 con 0x7f497c107e20 2024-05-09T05:47:44.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f49737fe700 1 -- 172.21.15.115:0/2482908822 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c00e3b0 con 0x7f497c107e20 2024-05-09T05:47:44.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.263+0000 7f4980bc3700 1 -- 172.21.15.115:0/2482908822 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497c1a7830 con 0x7f497c107e20 2024-05-09T05:47:44.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.266+0000 7f49737fe700 1 -- 172.21.15.115:0/2482908822 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f496c00f890 con 0x7f497c107e20 2024-05-09T05:47:44.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.267+0000 7f49737fe700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49680b29f0 0x7f49680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.267+0000 7f497a59c700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49680b29f0 0x7f49680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.267+0000 7f49737fe700 1 -- 172.21.15.115:0/2482908822 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f496c0e2760 con 0x7f497c107e20 2024-05-09T05:47:44.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.267+0000 7f498f5d9b80 1 Processor -- start 2024-05-09T05:47:44.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.267+0000 7f498f5d9b80 1 -- start start 2024-05-09T05:47:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.267+0000 7f497a59c700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49680b29f0 0x7f49680b4f00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4964000c00 tx=0x7f4964005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.267+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f6ba160 0x55e50f7534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f5d04a0 0x55e50f753a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f74d6a0 0x55e50f74dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e50f66fff0 con 0x55e50f74d6a0 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e50f60e7f0 con 0x55e50f6ba160 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e50f66bfb0 con 0x55e50f5d04a0 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f74d6a0 0x55e50f74dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f74d6a0 0x55e50f74dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43796/0 (socket says 172.21.15.115:43796) 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497ad9d700 1 -- 172.21.15.115:0/713174633 learned_addr learned my addr 172.21.15.115:0/713174633 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f4979d9b700 1 --2- 172.21.15.115:0/713174633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f5d04a0 0x55e50f753a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497a59c700 1 --2- 172.21.15.115:0/713174633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f6ba160 0x55e50f7534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497ad9d700 1 -- 172.21.15.115:0/713174633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f5d04a0 msgr2=0x55e50f753a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497ad9d700 1 --2- 172.21.15.115:0/713174633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f5d04a0 0x55e50f753a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497ad9d700 1 -- 172.21.15.115:0/713174633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f6ba160 msgr2=0x55e50f7534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:44.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497ad9d700 1 --2- 172.21.15.115:0/713174633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f6ba160 0x55e50f7534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497ad9d700 1 -- 172.21.15.115:0/713174633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50f606370 con 0x55e50f74d6a0 2024-05-09T05:47:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f497ad9d700 1 --2- 172.21.15.115:0/713174633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f74d6a0 0x55e50f74dad0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4974003fa0 tx=0x7f4974009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16e158d9edd6d2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f4970ff9700 1 -- 172.21.15.115:0/713174633 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974004bb0 con 0x55e50f74d6a0 2024-05-09T05:47:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f4970ff9700 1 -- 172.21.15.115:0/713174633 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49740048e0 con 0x55e50f74d6a0 2024-05-09T05:47:44.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.268+0000 7f4970ff9700 1 -- 172.21.15.115:0/713174633 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974007d20 con 0x55e50f74d6a0 2024-05-09T05:47:44.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.269+0000 7f498f5d9b80 1 -- 172.21.15.115:0/713174633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f74d6a0 msgr2=0x55e50f74dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.269+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/713174633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f74d6a0 0x55e50f74dad0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4974003fa0 tx=0x7f4974009ba0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.269+0000 7f498f5d9b80 1 -- 172.21.15.115:0/713174633 shutdown_connections 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.269+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/713174633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f6ba160 0x55e50f7534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.269+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/713174633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f74d6a0 0x55e50f74dad0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.269+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/713174633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f5d04a0 0x55e50f753a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.269+0000 7f498f5d9b80 1 -- 172.21.15.115:0/713174633 >> 172.21.15.115:0/713174633 conn(0x55e50f6306a0 msgr2=0x55e50f748400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.269+0000 7f498f5d9b80 1 -- 172.21.15.115:0/713174633 shutdown_connections 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.269+0000 7f498f5d9b80 1 -- 172.21.15.115:0/713174633 wait complete. 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f498f5d9b80 1 -- start start 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f498f5d9b80 1 Processor -- start 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 0x55e50f757160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f7576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 0x55e50f757be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e50f4ebba0 con 0x55e50f6ba160 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e50f62a150 con 0x55e50f5d04a0 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e50f75a1d0 con 0x55e50f74d6a0 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f498f5d9b80 10 client.0 _ll_register_callbacks cb 0x55e50f4e16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f4979d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f7576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f4979d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f7576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43800/0 (socket says 172.21.15.115:43800) 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f4979d9b700 1 -- 172.21.15.115:0/3899095112 learned_addr learned my addr 172.21.15.115:0/3899095112 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f497ad9d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 0x55e50f757be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.270+0000 7f497a59c700 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 0x55e50f757160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4979d9b700 1 -- 172.21.15.115:0/3899095112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 msgr2=0x55e50f757be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4979d9b700 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 0x55e50f757be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4979d9b700 1 -- 172.21.15.115:0/3899095112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 msgr2=0x55e50f757160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4979d9b700 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 0x55e50f757160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4979d9b700 1 -- 172.21.15.115:0/3899095112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50f4eb490 con 0x55e50f6ba160 2024-05-09T05:47:44.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4979d9b700 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f7576a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f496c0a6970 tx=0x7f496c0e0070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0e19a7d1372351 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4961ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4961ffb700 1 -- 172.21.15.115:0/3899095112 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c020040 con 0x55e50f6ba160 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4961ffb700 1 -- 172.21.15.115:0/3899095112 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f496c091070 con 0x55e50f6ba160 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3899095112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49740048e0 con 0x55e50f6ba160 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4961ffb700 1 -- 172.21.15.115:0/3899095112 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f496c00fed0 con 0x55e50f6ba160 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f498f5d9b80 10 client.18039 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3899095112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e50f764b90 con 0x55e50f6ba160 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4961ffb700 1 -- 172.21.15.115:0/3899095112 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49740048e0 con 0x55e50f6ba160 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.271+0000 7f4961ffb700 1 -- 172.21.15.115:0/3899095112 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f496c0a9960 con 0x55e50f6ba160 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.272+0000 7f498f5d9b80 10 client.18039 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.272+0000 7f498f5d9b80 10 client.18039 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.272+0000 7f498f5d9b80 20 client.18039 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.272+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55e50f767220 0x55e50f7696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.272+0000 7f498f5d9b80 4 client.18039 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:44.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.272+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3899095112 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e50f67d9d0 con 0x55e50f767220 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.272+0000 7f497a59c700 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55e50f767220 0x55e50f7696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.272+0000 7f4961ffb700 10 client.18039 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.272+0000 7f497a59c700 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55e50f767220 0x55e50f7696a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3e03c8acdcc94beb server_cookie=5fa99fc773d0046 in_seq=0 out_seq=0 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f4961ffb700 1 -- 172.21.15.115:0/3899095112 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e50f67d9d0 con 0x55e50f767220 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f4961ffb700 10 client.18039 handle_command_reply: tid=0 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 client.18039 shutdown 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3899095112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 msgr2=0x55e50f7576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f7576a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f496c0a6970 tx=0x7f496c0e0070 comp rx=0 tx=0).stop 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3899095112 shutdown_connections 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 0x55e50f757160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55e50f767220 0x55e50f7696a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 0x55e50f757be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f4961ffb700 0 client.18039 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3899095112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f7576a0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3899095112 >> 172.21.15.115:0/3899095112 conn(0x55e50f6306a0 msgr2=0x55e50f7481c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3899095112 shutdown_connections 2024-05-09T05:47:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.273+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3899095112 wait complete. 2024-05-09T05:47:44.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.274+0000 7f498f5d9b80 20 client.18039 trim_cache size 0 max 16384 2024-05-09T05:47:44.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.288+0000 7f498f5d9b80 1 Processor -- start 2024-05-09T05:47:44.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.288+0000 7f498f5d9b80 1 -- start start 2024-05-09T05:47:44.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.288+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 0x55e50f753b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.288+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f751860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.288+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 0x55e50f752980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.288+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f497c1b0350 con 0x55e50f6ba160 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.288+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f497c1b1f50 con 0x55e50f5d04a0 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.288+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f497c1b0a20 con 0x55e50f74d6a0 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4979d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f751860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4979d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f751860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43810/0 (socket says 172.21.15.115:43810) 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4979d9b700 1 -- 172.21.15.115:0/906559267 learned_addr learned my addr 172.21.15.115:0/906559267 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f497ad9d700 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 0x55e50f752980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f497a59c700 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 0x55e50f753b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4979d9b700 1 -- 172.21.15.115:0/906559267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 msgr2=0x55e50f752980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4979d9b700 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 0x55e50f752980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4979d9b700 1 -- 172.21.15.115:0/906559267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 msgr2=0x55e50f753b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4979d9b700 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 0x55e50f753b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4979d9b700 1 -- 172.21.15.115:0/906559267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50f756480 con 0x55e50f6ba160 2024-05-09T05:47:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f497ad9d700 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 0x55e50f752980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:47:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f497a59c700 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 0x55e50f753b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:47:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4979d9b700 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f751860 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f496c0dfc20 tx=0x7f496c010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53cfbd5d882b914 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4963fff700 1 -- 172.21.15.115:0/906559267 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c020050 con 0x55e50f6ba160 2024-05-09T05:47:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4963fff700 1 -- 172.21.15.115:0/906559267 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f496c0a9210 con 0x55e50f6ba160 2024-05-09T05:47:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.289+0000 7f4963fff700 1 -- 172.21.15.115:0/906559267 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c0a9960 con 0x55e50f6ba160 2024-05-09T05:47:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 -- 172.21.15.115:0/906559267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 msgr2=0x55e50f751860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f751860 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f496c0dfc20 tx=0x7f496c010820 comp rx=0 tx=0).stop 2024-05-09T05:47:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 -- 172.21.15.115:0/906559267 shutdown_connections 2024-05-09T05:47:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f5d04a0 0x55e50f753b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f74d6a0 0x55e50f752980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/906559267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f6ba160 0x55e50f751860 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 -- 172.21.15.115:0/906559267 >> 172.21.15.115:0/906559267 conn(0x55e50f6306a0 msgr2=0x55e50f752270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 -- 172.21.15.115:0/906559267 shutdown_connections 2024-05-09T05:47:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 -- 172.21.15.115:0/906559267 wait complete. 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.290+0000 7f498f5d9b80 1 -- start start 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f498f5d9b80 1 Processor -- start 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f5d04a0 0x55e50f76de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f6ba160 0x55e50f76e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f498f5d9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f74d6a0 0x55e50f76e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f497c1b1880 con 0x55e50f5d04a0 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f497c1b1120 con 0x55e50f74d6a0 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f498f5d9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f497c1ae2f0 con 0x55e50f6ba160 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f498f5d9b80 10 client.0 _ll_register_callbacks cb 0x55e50f4e16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f5d04a0 0x55e50f76de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f5d04a0 0x55e50f76de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43814/0 (socket says 172.21.15.115:43814) 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497a59c700 1 -- 172.21.15.115:0/3787328342 learned_addr learned my addr 172.21.15.115:0/3787328342 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497ad9d700 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f74d6a0 0x55e50f76e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f4979d9b700 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f6ba160 0x55e50f76e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497a59c700 1 -- 172.21.15.115:0/3787328342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f6ba160 msgr2=0x55e50f76e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497a59c700 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f6ba160 0x55e50f76e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497a59c700 1 -- 172.21.15.115:0/3787328342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f74d6a0 msgr2=0x55e50f76e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497a59c700 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f74d6a0 0x55e50f76e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497a59c700 1 -- 172.21.15.115:0/3787328342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50f75e2c0 con 0x55e50f5d04a0 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497ad9d700 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f74d6a0 0x55e50f76e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f497a59c700 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f5d04a0 0x55e50f76de90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f49640099c0 tx=0x7f4964009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8ebd0ee204b4a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.291+0000 7f4961ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f4961ffb700 1 -- 172.21.15.115:0/3787328342 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964039070 con 0x55e50f5d04a0 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f4961ffb700 1 -- 172.21.15.115:0/3787328342 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f4964003bd0 con 0x55e50f5d04a0 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3787328342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f496c0a9210 con 0x55e50f5d04a0 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f4961ffb700 1 -- 172.21.15.115:0/3787328342 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4964042050 con 0x55e50f5d04a0 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f498f5d9b80 10 client.18045 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3787328342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e50f75f400 con 0x55e50f5d04a0 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f4961ffb700 1 -- 172.21.15.115:0/3787328342 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f496c0a9210 con 0x55e50f5d04a0 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f4961ffb700 1 -- 172.21.15.115:0/3787328342 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f496403c020 con 0x55e50f5d04a0 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f498f5d9b80 10 client.18045 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f498f5d9b80 10 client.18045 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f498f5d9b80 20 client.18045 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55e50f75fb20 0x55e50f7d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f498f5d9b80 4 client.18045 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.292+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3787328342 --> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e50f677a70 con 0x55e50f75fb20 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f4979d9b700 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55e50f75fb20 0x55e50f7d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f4961ffb700 10 client.18045 ms_handle_connect on v2:172.21.15.161:6836/3591780613 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f4979d9b700 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55e50f75fb20 0x55e50f7d3a60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84955aa7c0923e75 server_cookie=e9420b46b8e01747 in_seq=0 out_seq=0 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f4961ffb700 1 -- 172.21.15.115:0/3787328342 <== mds.4 v2:172.21.15.161:6836/3591780613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e50f677a70 con 0x55e50f75fb20 2024-05-09T05:47:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f4961ffb700 10 client.18045 handle_command_reply: tid=0 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f498f5d9b80 1 client.18045 shutdown 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3787328342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f5d04a0 msgr2=0x55e50f76de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f5d04a0 0x55e50f76de90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f49640099c0 tx=0x7f4964009cd0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3787328342 shutdown_connections 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e50f74d6a0 0x55e50f76e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.161:6836/3591780613,v1:172.21.15.161:6837/3591780613] conn(0x55e50f75fb20 0x55e50f7d3a60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e50f5d04a0 0x55e50f76de90 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f4961ffb700 0 client.18045 ms_handle_reset on v2:172.21.15.161:6836/3591780613 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f498f5d9b80 1 --2- 172.21.15.115:0/3787328342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e50f6ba160 0x55e50f76e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.293+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3787328342 >> 172.21.15.115:0/3787328342 conn(0x55e50f6306a0 msgr2=0x55e50f750e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3787328342 shutdown_connections 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f498f5d9b80 1 -- 172.21.15.115:0/3787328342 wait complete. 2024-05-09T05:47:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f498f5d9b80 20 client.18045 trim_cache size 0 max 16384 2024-05-09T05:47:44.292 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:47:44.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 -- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49680b29f0 msgr2=0x7f49680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:44.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49680b29f0 0x7f49680b4f00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4964000c00 tx=0x7f4964005fb0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 -- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c107e20 msgr2=0x7f497c19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:44.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c107e20 0x7f497c19b2f0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f496c00d980 tx=0x7f496c00dc90 comp rx=0 tx=0).stop 2024-05-09T05:47:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 -- 172.21.15.115:0/2482908822 shutdown_connections 2024-05-09T05:47:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f49680b29f0 0x7f49680b4f00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c108770 0x7f497c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c103540 0x7f497c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 --2- 172.21.15.115:0/2482908822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f497c107e20 0x7f497c19b2f0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 -- 172.21.15.115:0/2482908822 >> 172.21.15.115:0/2482908822 conn(0x7f497c078e50 msgr2=0x7f497c075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 -- 172.21.15.115:0/2482908822 shutdown_connections 2024-05-09T05:47:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:44.294+0000 7f4980bc3700 1 -- 172.21.15.115:0/2482908822 wait complete. 2024-05-09T05:47:45.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-09T05:47:45.270 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:45.392 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:45 smithi115 ceph-mon[30349]: pgmap v297: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 418 KiB/s wr, 15 op/s 2024-05-09T05:47:45.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:45 smithi195 ceph-mon[33844]: pgmap v297: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 418 KiB/s wr, 15 op/s 2024-05-09T05:47:45.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:45 smithi161 ceph-mon[33805]: pgmap v297: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 418 KiB/s wr, 15 op/s 2024-05-09T05:47:47.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.071+0000 7f51256ac700 1 -- 172.21.15.115:0/2428706853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5120108740 msgr2=0x7f51201100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.071+0000 7f51256ac700 1 --2- 172.21.15.115:0/2428706853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5120108740 0x7f51201100a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f511800b0d0 tx=0x7f511800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.071+0000 7f51256ac700 1 -- 172.21.15.115:0/2428706853 shutdown_connections 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.071+0000 7f51256ac700 1 --2- 172.21.15.115:0/2428706853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5120108740 0x7f51201100a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.071+0000 7f51256ac700 1 --2- 172.21.15.115:0/2428706853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51201034c0 0x7f51201038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.071+0000 7f51256ac700 1 --2- 172.21.15.115:0/2428706853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5120107df0 0x7f5120108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.071+0000 7f51256ac700 1 -- 172.21.15.115:0/2428706853 >> 172.21.15.115:0/2428706853 conn(0x7f51200fedc0 msgr2=0x7f51201011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.072+0000 7f51256ac700 1 -- 172.21.15.115:0/2428706853 shutdown_connections 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.072+0000 7f51256ac700 1 -- 172.21.15.115:0/2428706853 wait complete. 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.072+0000 7f51256ac700 1 Processor -- start 2024-05-09T05:47:47.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.072+0000 7f51256ac700 1 -- start start 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f51256ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51201034c0 0x7f512019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f51256ac700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5120107df0 0x7f512019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f51256ac700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5120108740 0x7f512019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f51256ac700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5120113220 con 0x7f51201034c0 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f51256ac700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f51201130a0 con 0x7f5120108740 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f51256ac700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f51201133a0 con 0x7f5120107df0 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51201034c0 0x7f512019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51201034c0 0x7f512019bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43822/0 (socket says 172.21.15.115:43822) 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511effd700 1 -- 172.21.15.115:0/1288223247 learned_addr learned my addr 172.21.15.115:0/1288223247 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511e7fc700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5120107df0 0x7f512019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511f7fe700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5120108740 0x7f512019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511effd700 1 -- 172.21.15.115:0/1288223247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5120107df0 msgr2=0x7f512019c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511effd700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5120107df0 0x7f512019c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511effd700 1 -- 172.21.15.115:0/1288223247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5120108740 msgr2=0x7f512019c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511effd700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5120108740 0x7f512019c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511effd700 1 -- 172.21.15.115:0/1288223247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51201a6e50 con 0x7f51201034c0 2024-05-09T05:47:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f511effd700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51201034c0 0x7f512019bae0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f511400c920 tx=0x7f511400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.073+0000 7f5107fff700 1 -- 172.21.15.115:0/1288223247 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114007b00 con 0x7f51201034c0 2024-05-09T05:47:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.074+0000 7f5107fff700 1 -- 172.21.15.115:0/1288223247 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5114004cc0 con 0x7f51201034c0 2024-05-09T05:47:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.074+0000 7f51256ac700 1 -- 172.21.15.115:0/1288223247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51201a7140 con 0x7f51201034c0 2024-05-09T05:47:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.074+0000 7f5107fff700 1 -- 172.21.15.115:0/1288223247 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114005440 con 0x7f51201034c0 2024-05-09T05:47:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.074+0000 7f51256ac700 1 -- 172.21.15.115:0/1288223247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51201a7680 con 0x7f51201034c0 2024-05-09T05:47:47.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.076+0000 7f5107fff700 1 -- 172.21.15.115:0/1288223247 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f511400f450 con 0x7f51201034c0 2024-05-09T05:47:47.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.077+0000 7f5107fff700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f51080b29f0 0x7f51080b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.077+0000 7f511e7fc700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f51080b29f0 0x7f51080b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.077+0000 7f5107fff700 1 -- 172.21.15.115:0/1288223247 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f51140e2520 con 0x7f51201034c0 2024-05-09T05:47:47.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.077+0000 7f51340c2b80 1 Processor -- start 2024-05-09T05:47:47.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.077+0000 7f51340c2b80 1 -- start start 2024-05-09T05:47:47.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.077+0000 7f511e7fc700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f51080b29f0 0x7f51080b4f00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f51201024b0 tx=0x7f511000b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.077+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec8ff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec8ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec8f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f51340c2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561fec817ff0 con 0x561fec7784a0 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f51340c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561fec7b67f0 con 0x561fec862160 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f51340c2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561fec813fb0 con 0x561fec8f9ab0 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec8ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511e7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec8ffe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43828/0 (socket says 172.21.15.115:43828) 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511e7fc700 1 -- 172.21.15.115:0/3585133588 learned_addr learned my addr 172.21.15.115:0/3585133588 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511effd700 1 --2- 172.21.15.115:0/3585133588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec8ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511e7fc700 1 -- 172.21.15.115:0/3585133588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 msgr2=0x561fec8f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511e7fc700 1 --2- 172.21.15.115:0/3585133588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec8f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511e7fc700 1 -- 172.21.15.115:0/3585133588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 msgr2=0x561fec8ff900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:47.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511e7fc700 1 --2- 172.21.15.115:0/3585133588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec8ff900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511e7fc700 1 -- 172.21.15.115:0/3585133588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fec7ae370 con 0x561fec7784a0 2024-05-09T05:47:47.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f511e7fc700 1 --2- 172.21.15.115:0/3585133588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec8ffe40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f5110010dc0 tx=0x7f5110013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3633b79aa1e7dd0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:47.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f51057fa700 1 -- 172.21.15.115:0/3585133588 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110011bb0 con 0x561fec7784a0 2024-05-09T05:47:47.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f51057fa700 1 -- 172.21.15.115:0/3585133588 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5110011d50 con 0x561fec7784a0 2024-05-09T05:47:47.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.078+0000 7f51057fa700 1 -- 172.21.15.115:0/3585133588 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511001c5b0 con 0x561fec7784a0 2024-05-09T05:47:47.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.079+0000 7f51340c2b80 1 -- 172.21.15.115:0/3585133588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 msgr2=0x561fec8ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:47.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.079+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3585133588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec8ffe40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f5110010dc0 tx=0x7f5110013000 comp rx=0 tx=0).stop 2024-05-09T05:47:47.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.079+0000 7f51340c2b80 1 -- 172.21.15.115:0/3585133588 shutdown_connections 2024-05-09T05:47:47.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.079+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3585133588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec8ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.079+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3585133588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec8f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.079+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3585133588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec8ffe40 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.079+0000 7f51340c2b80 1 -- 172.21.15.115:0/3585133588 >> 172.21.15.115:0/3585133588 conn(0x561fec7d86a0 msgr2=0x561fec8f0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.079+0000 7f51340c2b80 1 -- 172.21.15.115:0/3585133588 shutdown_connections 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.079+0000 7f51340c2b80 1 -- 172.21.15.115:0/3585133588 wait complete. 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f51340c2b80 1 -- start start 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f51340c2b80 1 Processor -- start 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec7784a0 0x561fec903620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec862160 0x561fec903b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec8f9ab0 0x561fec9040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f51340c2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561fec693ba0 con 0x561fec8f9ab0 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f51340c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561fec7d2150 con 0x561fec7784a0 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f51340c2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561fec9065e0 con 0x561fec862160 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f51340c2b80 10 client.0 _ll_register_callbacks cb 0x561fec6896c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f511f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec8f9ab0 0x561fec9040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f511f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec8f9ab0 0x561fec9040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43832/0 (socket says 172.21.15.115:43832) 2024-05-09T05:47:47.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f511f7fe700 1 -- 172.21.15.115:0/3352902663 learned_addr learned my addr 172.21.15.115:0/3352902663 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.080+0000 7f511effd700 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec7784a0 0x561fec903620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f511e7fc700 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec862160 0x561fec903b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f511f7fe700 1 -- 172.21.15.115:0/3352902663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec862160 msgr2=0x561fec903b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f511f7fe700 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec862160 0x561fec903b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f511f7fe700 1 -- 172.21.15.115:0/3352902663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec7784a0 msgr2=0x561fec903620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f511f7fe700 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec7784a0 0x561fec903620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f511f7fe700 1 -- 172.21.15.115:0/3352902663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fec693490 con 0x561fec8f9ab0 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f511e7fc700 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec862160 0x561fec903b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f511f7fe700 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec8f9ab0 0x561fec9040a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f5118009a20 tx=0x7f51180094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f60beacb3c8263 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f5102ff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f5102ff5700 1 -- 172.21.15.115:0/3352902663 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118022070 con 0x561fec8f9ab0 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f5102ff5700 1 -- 172.21.15.115:0/3352902663 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f5118007ad0 con 0x561fec8f9ab0 2024-05-09T05:47:47.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f51340c2b80 1 -- 172.21.15.115:0/3352902663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5110011d50 con 0x561fec8f9ab0 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f5102ff5700 1 -- 172.21.15.115:0/3352902663 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f511802e050 con 0x561fec8f9ab0 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f51340c2b80 10 client.18057 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f51340c2b80 1 -- 172.21.15.115:0/3352902663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561fec910ff0 con 0x561fec8f9ab0 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.081+0000 7f5102ff5700 1 -- 172.21.15.115:0/3352902663 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5110011d50 con 0x561fec8f9ab0 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f5102ff5700 1 -- 172.21.15.115:0/3352902663 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f5118011070 con 0x561fec8f9ab0 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f51340c2b80 10 client.18057 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f51340c2b80 10 client.18057 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f51340c2b80 20 client.18057 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x561fec9135d0 0x561fec915a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f51340c2b80 4 client.18057 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f51340c2b80 1 -- 172.21.15.115:0/3352902663 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fec8259d0 con 0x561fec9135d0 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f511effd700 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x561fec9135d0 0x561fec915a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f5102ff5700 10 client.18057 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:47:47.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.082+0000 7f511effd700 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x561fec9135d0 0x561fec915a10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a869ff2e6c3b48e2 server_cookie=83d3ec200d18f404 in_seq=0 out_seq=0 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f5102ff5700 1 -- 172.21.15.115:0/3352902663 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561fec8259d0 con 0x561fec9135d0 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f5102ff5700 10 client.18057 handle_command_reply: tid=0 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f51340c2b80 1 client.18057 shutdown 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f51340c2b80 1 -- 172.21.15.115:0/3352902663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec8f9ab0 msgr2=0x561fec9040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec8f9ab0 0x561fec9040a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f5118009a20 tx=0x7f51180094d0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f51340c2b80 1 -- 172.21.15.115:0/3352902663 shutdown_connections 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec7784a0 0x561fec903620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x561fec9135d0 0x561fec915a10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec8f9ab0 0x561fec9040a0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.083+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3352902663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec862160 0x561fec903b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.084+0000 7f51340c2b80 1 -- 172.21.15.115:0/3352902663 >> 172.21.15.115:0/3352902663 conn(0x561fec7d86a0 msgr2=0x561fec8f01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.084+0000 7f5102ff5700 0 client.18057 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.084+0000 7f51340c2b80 1 -- 172.21.15.115:0/3352902663 shutdown_connections 2024-05-09T05:47:47.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.084+0000 7f51340c2b80 1 -- 172.21.15.115:0/3352902663 wait complete. 2024-05-09T05:47:47.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.084+0000 7f51340c2b80 20 client.18057 trim_cache size 0 max 16384 2024-05-09T05:47:47.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.098+0000 7f51340c2b80 1 Processor -- start 2024-05-09T05:47:47.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.098+0000 7f51340c2b80 1 -- start start 2024-05-09T05:47:47.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec9011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec903ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec8fd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f51340c2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f51201ae040 con 0x561fec7784a0 2024-05-09T05:47:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f51340c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f51201afc40 con 0x561fec862160 2024-05-09T05:47:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f51340c2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f51201ae710 con 0x561fec8f9ab0 2024-05-09T05:47:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec9011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec9011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43846/0 (socket says 172.21.15.115:43846) 2024-05-09T05:47:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511effd700 1 -- 172.21.15.115:0/1927229237 learned_addr learned my addr 172.21.15.115:0/1927229237 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511e7fc700 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec903ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511f7fe700 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec8fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511effd700 1 -- 172.21.15.115:0/1927229237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 msgr2=0x561fec8fd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511effd700 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec8fd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511effd700 1 -- 172.21.15.115:0/1927229237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 msgr2=0x561fec903ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511effd700 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec903ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511effd700 1 -- 172.21.15.115:0/1927229237 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fec905ec0 con 0x561fec7784a0 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511e7fc700 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec903ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f511effd700 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec9011c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f51140df990 tx=0x7f5114091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2f95133f0de6f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.099+0000 7f5104ff9700 1 -- 172.21.15.115:0/1927229237 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511401d040 con 0x561fec7784a0 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f5104ff9700 1 -- 172.21.15.115:0/1927229237 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51140a94c0 con 0x561fec7784a0 2024-05-09T05:47:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f5104ff9700 1 -- 172.21.15.115:0/1927229237 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140dfd00 con 0x561fec7784a0 2024-05-09T05:47:47.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f51340c2b80 1 -- 172.21.15.115:0/1927229237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 msgr2=0x561fec9011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:47.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f51340c2b80 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec9011c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f51140df990 tx=0x7f5114091040 comp rx=0 tx=0).stop 2024-05-09T05:47:47.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f51340c2b80 1 -- 172.21.15.115:0/1927229237 shutdown_connections 2024-05-09T05:47:47.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f51340c2b80 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec8fd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f51340c2b80 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec9011c0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f51340c2b80 1 --2- 172.21.15.115:0/1927229237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec903ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f51340c2b80 1 -- 172.21.15.115:0/1927229237 >> 172.21.15.115:0/1927229237 conn(0x561fec7d86a0 msgr2=0x561fec8fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:47.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f51340c2b80 1 -- 172.21.15.115:0/1927229237 shutdown_connections 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.100+0000 7f51340c2b80 1 -- 172.21.15.115:0/1927229237 wait complete. 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f51340c2b80 1 -- start start 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f51340c2b80 1 Processor -- start 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec90b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec90bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f51340c2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec91a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f51340c2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f51201af570 con 0x561fec7784a0 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f51340c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f51201aee10 con 0x561fec862160 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f51340c2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f51201abfe0 con 0x561fec8f9ab0 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f51340c2b80 10 client.0 _ll_register_callbacks cb 0x561fec6896c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f511effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec90b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f511effd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec90b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43848/0 (socket says 172.21.15.115:43848) 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f511effd700 1 -- 172.21.15.115:0/3990348956 learned_addr learned my addr 172.21.15.115:0/3990348956 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f511f7fe700 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec91a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f511e7fc700 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec90bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f511effd700 1 -- 172.21.15.115:0/3990348956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 msgr2=0x561fec91a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f511effd700 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec91a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f511effd700 1 -- 172.21.15.115:0/3990348956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 msgr2=0x561fec90bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.101+0000 7f511effd700 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec90bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f511effd700 1 -- 172.21.15.115:0/3990348956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fec91a700 con 0x561fec7784a0 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f511e7fc700 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec90bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f511effd700 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec90b760 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f51140a5540 tx=0x7f5114007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf1af6a80a0f512 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f5102ff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f5102ff5700 1 -- 172.21.15.115:0/3990348956 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511406d070 con 0x561fec7784a0 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f5102ff5700 1 -- 172.21.15.115:0/3990348956 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f51140a5750 con 0x561fec7784a0 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f51340c2b80 1 -- 172.21.15.115:0/3990348956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51140a94c0 con 0x561fec7784a0 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f5102ff5700 1 -- 172.21.15.115:0/3990348956 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f511401d070 con 0x561fec7784a0 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f51340c2b80 10 client.18063 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f51340c2b80 1 -- 172.21.15.115:0/3990348956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561fec97d370 con 0x561fec7784a0 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f5102ff5700 1 -- 172.21.15.115:0/3990348956 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51140a94c0 con 0x561fec7784a0 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f5102ff5700 1 -- 172.21.15.115:0/3990348956 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f51140e2100 con 0x561fec7784a0 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f51340c2b80 10 client.18063 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f51340c2b80 10 client.18063 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f51340c2b80 20 client.18063 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x561fec97da60 0x561fec97fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f51340c2b80 4 client.18063 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.102+0000 7f51340c2b80 1 -- 172.21.15.115:0/3990348956 --> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fec81fa70 con 0x561fec97da60 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.103+0000 7f511e7fc700 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x561fec97da60 0x561fec97fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.103+0000 7f5102ff5700 10 client.18063 ms_handle_connect on v2:172.21.15.161:6838/234220396 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.103+0000 7f511e7fc700 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x561fec97da60 0x561fec97fe80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab91584cf835273 server_cookie=6247a02f8fae4949 in_seq=0 out_seq=0 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.103+0000 7f5102ff5700 1 -- 172.21.15.115:0/3990348956 <== mds.3 v2:172.21.15.161:6838/234220396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fec81fa70 con 0x561fec97da60 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.103+0000 7f5102ff5700 10 client.18063 handle_command_reply: tid=0 2024-05-09T05:47:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.103+0000 7f51340c2b80 1 client.18063 shutdown 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.103+0000 7f51340c2b80 1 -- 172.21.15.115:0/3990348956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 msgr2=0x561fec90b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.103+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec90b760 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f51140a5540 tx=0x7f5114007eb0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51340c2b80 1 -- 172.21.15.115:0/3990348956 shutdown_connections 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.161:6838/234220396,v1:172.21.15.161:6839/234220396] conn(0x561fec97da60 0x561fec97fe80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561fec8f9ab0 0x561fec91a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561fec7784a0 0x561fec90b760 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51340c2b80 1 --2- 172.21.15.115:0/3990348956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fec862160 0x561fec90bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51340c2b80 1 -- 172.21.15.115:0/3990348956 >> 172.21.15.115:0/3990348956 conn(0x561fec7d86a0 msgr2=0x561fec901e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f5102ff5700 0 client.18063 ms_handle_reset on v2:172.21.15.161:6838/234220396 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51340c2b80 1 -- 172.21.15.115:0/3990348956 shutdown_connections 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51340c2b80 1 -- 172.21.15.115:0/3990348956 wait complete. 2024-05-09T05:47:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51340c2b80 20 client.18063 trim_cache size 0 max 16384 2024-05-09T05:47:47.102 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:47:47.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 -- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f51080b29f0 msgr2=0x7f51080b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:47.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f51080b29f0 0x7f51080b4f00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f51201024b0 tx=0x7f511000b410 comp rx=0 tx=0).stop 2024-05-09T05:47:47.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 -- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51201034c0 msgr2=0x7f512019bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:47.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51201034c0 0x7f512019bae0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f511400c920 tx=0x7f511400cc30 comp rx=0 tx=0).stop 2024-05-09T05:47:47.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 -- 172.21.15.115:0/1288223247 shutdown_connections 2024-05-09T05:47:47.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f51080b29f0 0x7f51080b4f00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5120108740 0x7f512019c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51201034c0 0x7f512019bae0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 --2- 172.21.15.115:0/1288223247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5120107df0 0x7f512019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:47.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 -- 172.21.15.115:0/1288223247 >> 172.21.15.115:0/1288223247 conn(0x7f51200fedc0 msgr2=0x7f5120107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:47.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 -- 172.21.15.115:0/1288223247 shutdown_connections 2024-05-09T05:47:47.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:47.104+0000 7f51256ac700 1 -- 172.21.15.115:0/1288223247 wait complete. 2024-05-09T05:47:47.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:47 smithi115 ceph-mon[30349]: pgmap v298: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-09T05:47:47.460 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:47 smithi161 ceph-mon[33805]: pgmap v298: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-09T05:47:47.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:47 smithi195 ceph-mon[33844]: pgmap v298: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-09T05:47:47.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:47:48.049 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:49.542 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:49 smithi115 ceph-mon[30349]: pgmap v299: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-05-09T05:47:49.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:49 smithi195 ceph-mon[33844]: pgmap v299: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-05-09T05:47:49.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:49 smithi161 ceph-mon[33805]: pgmap v299: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-05-09T05:47:49.818 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:47:49.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.823+0000 7ff591841700 1 -- 172.21.15.115:0/468257076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 msgr2=0x7ff58c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:49.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.823+0000 7ff591841700 1 --2- 172.21.15.115:0/468257076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 0x7ff58c109350 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff5740099c0 tx=0x7ff574009cd0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 -- 172.21.15.115:0/468257076 shutdown_connections 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 --2- 172.21.15.115:0/468257076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c109890 0x7ff58c1111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 --2- 172.21.15.115:0/468257076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58c102b20 0x7ff58c102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 --2- 172.21.15.115:0/468257076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 0x7ff58c109350 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 -- 172.21.15.115:0/468257076 >> 172.21.15.115:0/468257076 conn(0x7ff58c0fe520 msgr2=0x7ff58c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 -- 172.21.15.115:0/468257076 shutdown_connections 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 -- 172.21.15.115:0/468257076 wait complete. 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 Processor -- start 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 -- start start 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.824+0000 7ff591841700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58c102b20 0x7ff58c103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff591841700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 0x7ff58c103fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff591841700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c109890 0x7ff58c1044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff591841700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c114340 con 0x7ff58c108f40 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff591841700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1141c0 con 0x7ff58c109890 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff591841700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1144c0 con 0x7ff58c102b20 2024-05-09T05:47:49.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff583fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 0x7ff58c103fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff583fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 0x7ff58c103fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43862/0 (socket says 172.21.15.115:43862) 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff583fff700 1 -- 172.21.15.115:0/531399613 learned_addr learned my addr 172.21.15.115:0/531399613 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff58affd700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58c102b20 0x7ff58c103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff58b7fe700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c109890 0x7ff58c1044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff583fff700 1 -- 172.21.15.115:0/531399613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58c102b20 msgr2=0x7ff58c103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff583fff700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58c102b20 0x7ff58c103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff583fff700 1 -- 172.21.15.115:0/531399613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c109890 msgr2=0x7ff58c1044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff583fff700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c109890 0x7ff58c1044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff583fff700 1 -- 172.21.15.115:0/531399613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c1a8fe0 con 0x7ff58c108f40 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff58b7fe700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c109890 0x7ff58c1044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff583fff700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 0x7ff58c103fb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff57400b580 tx=0x7ff57400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff59083f700 1 -- 172.21.15.115:0/531399613 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574027070 con 0x7ff58c108f40 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff591841700 1 -- 172.21.15.115:0/531399613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c1a9270 con 0x7ff58c108f40 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.825+0000 7ff59083f700 1 -- 172.21.15.115:0/531399613 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5740056d0 con 0x7ff58c108f40 2024-05-09T05:47:49.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.826+0000 7ff591841700 1 -- 172.21.15.115:0/531399613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1a9720 con 0x7ff58c108f40 2024-05-09T05:47:49.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.826+0000 7ff59083f700 1 -- 172.21.15.115:0/531399613 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5740173f0 con 0x7ff58c108f40 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.828+0000 7ff59083f700 1 -- 172.21.15.115:0/531399613 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff57402a070 con 0x7ff58c108f40 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.829+0000 7ff59083f700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5780b29f0 0x7ff5780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.829+0000 7ff5a0257b80 1 Processor -- start 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.829+0000 7ff58affd700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5780b29f0 0x7ff5780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.829+0000 7ff59083f700 1 -- 172.21.15.115:0/531399613 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7ff5740e3860 con 0x7ff58c108f40 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.829+0000 7ff5a0257b80 1 -- start start 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.829+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085d4bd00 0x559085e54240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085cd14a0 0x559085e54780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 0x559085e4e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559085d710a0 con 0x559085d4bd00 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559085d169a0 con 0x559085cd14a0 2024-05-09T05:47:49.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559085d6d060 con 0x559085e4e400 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085d4bd00 0x559085e54240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085d4bd00 0x559085e54240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43876/0 (socket says 172.21.15.115:43876) 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 -- 172.21.15.115:0/415443780 learned_addr learned my addr 172.21.15.115:0/415443780 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5780b29f0 0x7ff5780b4f00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff58c06bc30 tx=0x7ff57c00a3b0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff583fff700 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085cd14a0 0x559085e54780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58b7fe700 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 0x559085e4e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 -- 172.21.15.115:0/415443780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 msgr2=0x559085e4e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 0x559085e4e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 -- 172.21.15.115:0/415443780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085cd14a0 msgr2=0x559085e54780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085cd14a0 0x559085e54780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 -- 172.21.15.115:0/415443780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559085d07500 con 0x559085d4bd00 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff583fff700 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085cd14a0 0x559085e54780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff58affd700 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085d4bd00 0x559085e54240 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00fed0 tx=0x7ff57c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2521a2f6f5f8f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff581ffb700 1 -- 172.21.15.115:0/415443780 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c016070 con 0x559085d4bd00 2024-05-09T05:47:49.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff581ffb700 1 -- 172.21.15.115:0/415443780 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff57c01c420 con 0x559085d4bd00 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.830+0000 7ff581ffb700 1 -- 172.21.15.115:0/415443780 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c01b680 con 0x559085d4bd00 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.831+0000 7ff5a0257b80 1 -- 172.21.15.115:0/415443780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085d4bd00 msgr2=0x559085e54240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.831+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085d4bd00 0x559085e54240 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00fed0 tx=0x7ff57c012000 comp rx=0 tx=0).stop 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.831+0000 7ff5a0257b80 1 -- 172.21.15.115:0/415443780 shutdown_connections 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.831+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 0x559085e4e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.831+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085d4bd00 0x559085e54240 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.831+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/415443780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085cd14a0 0x559085e54780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.831+0000 7ff5a0257b80 1 -- 172.21.15.115:0/415443780 >> 172.21.15.115:0/415443780 conn(0x559085cd4f70 msgr2=0x559085e49220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.831+0000 7ff5a0257b80 1 -- 172.21.15.115:0/415443780 shutdown_connections 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.831+0000 7ff5a0257b80 1 -- 172.21.15.115:0/415443780 wait complete. 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff5a0257b80 1 -- start start 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff5a0257b80 1 Processor -- start 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e58040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085d4bd00 0x559085e58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 0x559085e58ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559085e4f160 con 0x559085cd14a0 2024-05-09T05:47:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559085d0bb20 con 0x559085d4bd00 2024-05-09T05:47:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559085e5b020 con 0x559085e4e400 2024-05-09T05:47:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff5a0257b80 10 client.0 _ll_register_callbacks cb 0x559085be26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff58affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e58040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff58affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e58040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43878/0 (socket says 172.21.15.115:43878) 2024-05-09T05:47:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.832+0000 7ff58affd700 1 -- 172.21.15.115:0/3666163843 learned_addr learned my addr 172.21.15.115:0/3666163843 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff58b7fe700 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 0x559085e58ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff583fff700 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085d4bd00 0x559085e58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff58affd700 1 -- 172.21.15.115:0/3666163843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 msgr2=0x559085e58ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff58affd700 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 0x559085e58ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff58affd700 1 -- 172.21.15.115:0/3666163843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085d4bd00 msgr2=0x559085e58580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff58affd700 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085d4bd00 0x559085e58580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff58affd700 1 -- 172.21.15.115:0/3666163843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559085bec490 con 0x559085cd14a0 2024-05-09T05:47:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff58affd700 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e58040 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff57c010710 tx=0x7ff57c011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da48868bb5d3f8dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff56affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff56affd700 1 -- 172.21.15.115:0/3666163843 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c016040 con 0x559085cd14a0 2024-05-09T05:47:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff56affd700 1 -- 172.21.15.115:0/3666163843 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7ff57c011d00 con 0x559085cd14a0 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff5a0257b80 1 -- 172.21.15.115:0/3666163843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff57c01c420 con 0x559085cd14a0 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff56affd700 1 -- 172.21.15.115:0/3666163843 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff57c03c050 con 0x559085cd14a0 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff5a0257b80 10 client.18075 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.833+0000 7ff5a0257b80 1 -- 172.21.15.115:0/3666163843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x559085e65980 con 0x559085cd14a0 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.834+0000 7ff56affd700 1 -- 172.21.15.115:0/3666163843 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff57c01c420 con 0x559085cd14a0 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.834+0000 7ff56affd700 1 -- 172.21.15.115:0/3666163843 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7ff57c037030 con 0x559085cd14a0 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.834+0000 7ff5a0257b80 10 client.18075 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.834+0000 7ff5a0257b80 10 client.18075 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.834+0000 7ff5a0257b80 20 client.18075 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.834+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x559085e67f60 0x559085e6a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.834+0000 7ff5a0257b80 4 client.18075 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:49.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.834+0000 7ff5a0257b80 1 -- 172.21.15.115:0/3666163843 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559085d7ea80 con 0x559085e67f60 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.834+0000 7ff583fff700 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x559085e67f60 0x559085e6a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.835+0000 7ff56affd700 10 client.18075 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.835+0000 7ff583fff700 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x559085e67f60 0x559085e6a380 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=233a80691ea294a server_cookie=4c0d93c8ee8de7b6 in_seq=0 out_seq=0 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.835+0000 7ff56affd700 1 -- 172.21.15.115:0/3666163843 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559085d7ea80 con 0x559085e67f60 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.835+0000 7ff56affd700 10 client.18075 handle_command_reply: tid=0 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.835+0000 7ff5a0257b80 1 client.18075 shutdown 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 -- 172.21.15.115:0/3666163843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 msgr2=0x559085e58040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e58040 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff57c010710 tx=0x7ff57c011850 comp rx=0 tx=0).stop 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 -- 172.21.15.115:0/3666163843 shutdown_connections 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x559085e67f60 0x559085e6a380 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085e4e400 0x559085e58ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff56affd700 0 client.18075 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e58040 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/3666163843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085d4bd00 0x559085e58580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 -- 172.21.15.115:0/3666163843 >> 172.21.15.115:0/3666163843 conn(0x559085cd4f70 msgr2=0x559085e42ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:49.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 -- 172.21.15.115:0/3666163843 shutdown_connections 2024-05-09T05:47:49.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 1 -- 172.21.15.115:0/3666163843 wait complete. 2024-05-09T05:47:49.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.836+0000 7ff5a0257b80 20 client.18075 trim_cache size 0 max 16384 2024-05-09T05:47:49.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.851+0000 7ff5a0257b80 1 Processor -- start 2024-05-09T05:47:49.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.851+0000 7ff5a0257b80 1 -- start start 2024-05-09T05:47:49.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e57360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 0x559085e5a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 0x559085e64190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b2430 con 0x559085cd14a0 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b4030 con 0x559085e4e400 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b2b00 con 0x559085d4bd00 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e57360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e57360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43886/0 (socket says 172.21.15.115:43886) 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58affd700 1 -- 172.21.15.115:0/2621318856 learned_addr learned my addr 172.21.15.115:0/2621318856 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff583fff700 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 0x559085e5a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58b7fe700 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 0x559085e64190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58affd700 1 -- 172.21.15.115:0/2621318856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 msgr2=0x559085e5a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58affd700 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 0x559085e5a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58affd700 1 -- 172.21.15.115:0/2621318856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 msgr2=0x559085e64190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58affd700 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 0x559085e64190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58affd700 1 -- 172.21.15.115:0/2621318856 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559085e57e10 con 0x559085cd14a0 2024-05-09T05:47:49.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58b7fe700 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 0x559085e64190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:49.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.852+0000 7ff58affd700 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e57360 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff57c004b70 tx=0x7ff57c0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe72292f78170d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:49.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.853+0000 7ff5817fa700 1 -- 172.21.15.115:0/2621318856 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c045070 con 0x559085cd14a0 2024-05-09T05:47:49.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.853+0000 7ff5817fa700 1 -- 172.21.15.115:0/2621318856 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff57c004d60 con 0x559085cd14a0 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.853+0000 7ff5817fa700 1 -- 172.21.15.115:0/2621318856 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c016070 con 0x559085cd14a0 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.853+0000 7ff5a0257b80 1 -- 172.21.15.115:0/2621318856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 msgr2=0x559085e57360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.853+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e57360 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff57c004b70 tx=0x7ff57c0119f0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.854+0000 7ff5a0257b80 1 -- 172.21.15.115:0/2621318856 shutdown_connections 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.854+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 0x559085e64190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.854+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e57360 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.854+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/2621318856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 0x559085e5a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.854+0000 7ff5a0257b80 1 -- 172.21.15.115:0/2621318856 >> 172.21.15.115:0/2621318856 conn(0x559085cd4f70 msgr2=0x559085e42ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.854+0000 7ff5a0257b80 1 -- 172.21.15.115:0/2621318856 shutdown_connections 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.854+0000 7ff5a0257b80 1 -- 172.21.15.115:0/2621318856 wait complete. 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.854+0000 7ff5a0257b80 1 -- start start 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.854+0000 7ff5a0257b80 1 Processor -- start 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e6c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 0x559085e6cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff5a0257b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 0x559085ed04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b3960 con 0x559085cd14a0 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b3200 con 0x559085e4e400 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff5a0257b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b03d0 con 0x559085d4bd00 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff5a0257b80 10 client.0 _ll_register_callbacks cb 0x559085be26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff58affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e6c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff58affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e6c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43894/0 (socket says 172.21.15.115:43894) 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff58affd700 1 -- 172.21.15.115:0/970957608 learned_addr learned my addr 172.21.15.115:0/970957608 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff583fff700 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 0x559085e6cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff58b7fe700 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 0x559085ed04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff583fff700 1 -- 172.21.15.115:0/970957608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 msgr2=0x559085ed04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff583fff700 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 0x559085ed04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff583fff700 1 -- 172.21.15.115:0/970957608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 msgr2=0x559085e6c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff583fff700 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e6c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff583fff700 1 -- 172.21.15.115:0/970957608 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559085e42d40 con 0x559085d4bd00 2024-05-09T05:47:49.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff58affd700 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e6c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff583fff700 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 0x559085e6cb20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5740e0c70 tx=0x7ff5740b3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9822693996062cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.855+0000 7ff56affd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff56affd700 1 -- 172.21.15.115:0/970957608 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5740aaf90 con 0x559085d4bd00 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff56affd700 1 -- 172.21.15.115:0/970957608 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7ff574022670 con 0x559085d4bd00 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff5a0257b80 1 -- 172.21.15.115:0/970957608 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff57c004d60 con 0x559085d4bd00 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff56affd700 1 -- 172.21.15.115:0/970957608 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff574022470 con 0x559085d4bd00 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff5a0257b80 10 client.27457 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff5a0257b80 1 -- 172.21.15.115:0/970957608 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x559085ed1970 con 0x559085d4bd00 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff56affd700 1 -- 172.21.15.115:0/970957608 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff57c004d60 con 0x559085d4bd00 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff56affd700 1 -- 172.21.15.115:0/970957608 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7ff5740b31d0 con 0x559085d4bd00 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff5a0257b80 10 client.27457 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff5a0257b80 10 client.27457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff5a0257b80 20 client.27457 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:49.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x559085e67f60 0x559085ed40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff5a0257b80 4 client.27457 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff5a0257b80 1 -- 172.21.15.115:0/970957608 --> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559085d78b20 con 0x559085e67f60 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.856+0000 7ff58affd700 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x559085e67f60 0x559085ed40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff56affd700 10 client.27457 ms_handle_connect on v2:172.21.15.195:6834/969435997 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff58affd700 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x559085e67f60 0x559085ed40c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b3becaa496f6f70 server_cookie=d662fc019f950954 in_seq=0 out_seq=0 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff56affd700 1 -- 172.21.15.115:0/970957608 <== mds.2 v2:172.21.15.195:6834/969435997 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4658 (crc 0 0 0) 0x559085d78b20 con 0x559085e67f60 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff56affd700 10 client.27457 handle_command_reply: tid=0 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff5a0257b80 1 client.27457 shutdown 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff5a0257b80 1 -- 172.21.15.115:0/970957608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 msgr2=0x559085e6cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 0x559085e6cb20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5740e0c70 tx=0x7ff5740b3040 comp rx=0 tx=0).stop 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff5a0257b80 1 -- 172.21.15.115:0/970957608 shutdown_connections 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559085e4e400 0x559085ed04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.195:6834/969435997,v1:172.21.15.195:6835/969435997] conn(0x559085e67f60 0x559085ed40c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559085cd14a0 0x559085e6c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff5a0257b80 1 --2- 172.21.15.115:0/970957608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559085d4bd00 0x559085e6cb20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff5a0257b80 1 -- 172.21.15.115:0/970957608 >> 172.21.15.115:0/970957608 conn(0x559085cd4f70 msgr2=0x559085e43780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.857+0000 7ff56affd700 0 client.27457 ms_handle_reset on v2:172.21.15.195:6834/969435997 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff5a0257b80 1 -- 172.21.15.115:0/970957608 shutdown_connections 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff5a0257b80 1 -- 172.21.15.115:0/970957608 wait complete. 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff5a0257b80 20 client.27457 trim_cache size 0 max 16384 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 191972113, 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256055, 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fh.Po" 2024-05-09T05:47:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 225548017, 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256069, 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/list.Po" 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 307945780, 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 1099511628464, 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.git/index" 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 308887306, 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256065, 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/util.Po" 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 374827588, 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 1099511628764, 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/requests" 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 410162317, 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256068, 2024-05-09T05:47:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_stats.Po" 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 684929066, 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256066, 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/parser.Po" 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 875733192, 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256061, 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_tg.Po" 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 884169366, 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256057, 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/metaops.Po" 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 2068608072, 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256058, 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rwlock.Po" 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 2465910697, 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023255552, 2024-05-09T05:47:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fileops.Po" 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 2655745164, 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256060, 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rbt.Po" 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 2707392242, 2024-05-09T05:47:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023255553, 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rand.Po" 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 3212203721, 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256067, 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fc.Po" 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 3219493471, 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256063, 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_thread.Po" 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 3238711390, 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256056, 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/filelist.Po" 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 3391654524, 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256054, 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/main.Po" 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 3526424197, 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256062, 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fs.Po" 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 3846390614, 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256064, 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_op.Po" 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 3902455919, 2024-05-09T05:47:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 1099511628765, 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/output.0" 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 4161297249, 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 2199023256059, 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/cirlist.Po" 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "damage_type": "backtrace", 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 4277993129, 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 1099511628766, 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "path": "/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/traces.0" 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:49.868 INFO:teuthology.orchestra.run.smithi115.stdout:] 2024-05-09T05:47:49.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 -- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5780b29f0 msgr2=0x7ff5780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:49.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5780b29f0 0x7ff5780b4f00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff58c06bc30 tx=0x7ff57c00a3b0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 -- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 msgr2=0x7ff58c103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:49.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 0x7ff58c103fb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff57400b580 tx=0x7ff57400f690 comp rx=0 tx=0).stop 2024-05-09T05:47:49.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 -- 172.21.15.115:0/531399613 shutdown_connections 2024-05-09T05:47:49.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5780b29f0 0x7ff5780b4f00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c109890 0x7ff58c1044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff58c102b20 0x7ff58c103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 --2- 172.21.15.115:0/531399613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff58c108f40 0x7ff58c103fb0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:49.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.858+0000 7ff591841700 1 -- 172.21.15.115:0/531399613 >> 172.21.15.115:0/531399613 conn(0x7ff58c0fe520 msgr2=0x7ff58c108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:49.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.859+0000 7ff591841700 1 -- 172.21.15.115:0/531399613 shutdown_connections 2024-05-09T05:47:49.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:49.859+0000 7ff591841700 1 -- 172.21.15.115:0/531399613 wait complete. 2024-05-09T05:47:50.650 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:47:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:51.439 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:51.548 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:51 smithi115 ceph-mon[30349]: pgmap v300: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-05-09T05:47:51.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:51 smithi195 ceph-mon[33844]: pgmap v300: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-05-09T05:47:51.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:51 smithi161 ceph-mon[33805]: pgmap v300: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-05-09T05:47:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.592+0000 7f85851e0700 1 -- 172.21.15.115:0/1185795519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85801039a0 msgr2=0x7f8580103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.592+0000 7f85851e0700 1 --2- 172.21.15.115:0/1185795519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85801039a0 0x7f8580103db0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f8574009990 tx=0x7f8574009ca0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.592+0000 7f85851e0700 1 -- 172.21.15.115:0/1185795519 shutdown_connections 2024-05-09T05:47:52.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.592+0000 7f85851e0700 1 --2- 172.21.15.115:0/1185795519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8580108200 0x7f8580108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.592+0000 7f85851e0700 1 --2- 172.21.15.115:0/1185795519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85801042f0 0x7f8580110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.592+0000 7f85851e0700 1 --2- 172.21.15.115:0/1185795519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85801039a0 0x7f8580103db0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.592+0000 7f85851e0700 1 -- 172.21.15.115:0/1185795519 >> 172.21.15.115:0/1185795519 conn(0x7f85800fe520 msgr2=0x7f8580100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:52.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.592+0000 7f85851e0700 1 -- 172.21.15.115:0/1185795519 shutdown_connections 2024-05-09T05:47:52.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.592+0000 7f85851e0700 1 -- 172.21.15.115:0/1185795519 wait complete. 2024-05-09T05:47:52.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f85851e0700 1 Processor -- start 2024-05-09T05:47:52.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f85851e0700 1 -- start start 2024-05-09T05:47:52.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f85851e0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85801039a0 0x7f85801a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:52.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f85851e0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85801042f0 0x7f85801a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:52.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f85851e0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108200 0x7f858019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:52.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f85851e0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8580114370 con 0x7f8580108200 2024-05-09T05:47:52.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f85851e0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85801141f0 con 0x7f85801042f0 2024-05-09T05:47:52.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f85851e0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85801144f0 con 0x7f85801039a0 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f857f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108200 0x7f858019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f857f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108200 0x7f858019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51568/0 (socket says 172.21.15.115:51568) 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f857f59e700 1 -- 172.21.15.115:0/3600923881 learned_addr learned my addr 172.21.15.115:0/3600923881 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f857ed9d700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85801039a0 0x7f85801a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.593+0000 7f857e59c700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85801042f0 0x7f85801a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f857f59e700 1 -- 172.21.15.115:0/3600923881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85801039a0 msgr2=0x7f85801a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f857f59e700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85801039a0 0x7f85801a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f857f59e700 1 -- 172.21.15.115:0/3600923881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85801042f0 msgr2=0x7f85801a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f857f59e700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85801042f0 0x7f85801a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f857f59e700 1 -- 172.21.15.115:0/3600923881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858019b280 con 0x7f8580108200 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f857f59e700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108200 0x7f858019ab50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f8578007ee0 tx=0x7f857800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f8567fff700 1 -- 172.21.15.115:0/3600923881 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578004bb0 con 0x7f8580108200 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858019b570 con 0x7f8580108200 2024-05-09T05:47:52.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f8567fff700 1 -- 172.21.15.115:0/3600923881 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8578004d50 con 0x7f8580108200 2024-05-09T05:47:52.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f8567fff700 1 -- 172.21.15.115:0/3600923881 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85780058b0 con 0x7f8580108200 2024-05-09T05:47:52.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.594+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801a7770 con 0x7f8580108200 2024-05-09T05:47:52.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.595+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85801049b0 con 0x7f8580108200 2024-05-09T05:47:52.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.597+0000 7f8567fff700 1 -- 172.21.15.115:0/3600923881 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f857801a020 con 0x7f8580108200 2024-05-09T05:47:52.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.597+0000 7f8567fff700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85680b29f0 0x7f85680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:52.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.598+0000 7f857ed9d700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85680b29f0 0x7f85680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:52.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.598+0000 7f8567fff700 1 -- 172.21.15.115:0/3600923881 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f85780e26a0 con 0x7f8580108200 2024-05-09T05:47:52.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.598+0000 7f857ed9d700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85680b29f0 0x7f85680b4f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f858006c2f0 tx=0x7f8570009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:52.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.601+0000 7f8567fff700 1 -- 172.21.15.115:0/3600923881 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8578010070 con 0x7f8580108200 2024-05-09T05:47:52.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.864+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f858002e5f0 con 0x7f8580108200 2024-05-09T05:47:52.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.865+0000 7f8567fff700 1 -- 172.21.15.115:0/3600923881 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11957 (secure 0 0 0) 0x7f85780a9d40 con 0x7f8580108200 2024-05-09T05:47:52.857 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:47:52.857 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:47:38.366444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27316":{"gid":27316,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/391681574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":391681574},{"type":"v1","addr":"172.21.15.195:6833","nonce":391681574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85680b29f0 msgr2=0x7f85680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85680b29f0 0x7f85680b4f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f858006c2f0 tx=0x7f8570009260 comp rx=0 tx=0).stop 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108200 msgr2=0x7f858019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108200 0x7f858019ab50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f8578007ee0 tx=0x7f857800d3b0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 shutdown_connections 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85680b29f0 0x7f85680b4f00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8580108200 0x7f858019ab50 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85801039a0 0x7f85801a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 --2- 172.21.15.115:0/3600923881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85801042f0 0x7f85801a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 >> 172.21.15.115:0/3600923881 conn(0x7f85800fe520 msgr2=0x7f8580100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 shutdown_connections 2024-05-09T05:47:52.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:52.868+0000 7f85851e0700 1 -- 172.21.15.115:0/3600923881 wait complete. 2024-05-09T05:47:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 48 2024-05-09T05:47:53.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:53 smithi115 ceph-mon[30349]: pgmap v301: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-05-09T05:47:53.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:53 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3600923881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:47:53.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:53 smithi195 ceph-mon[33844]: pgmap v301: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-05-09T05:47:53.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:53 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3600923881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:47:53.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:53 smithi161 ceph-mon[33805]: pgmap v301: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-05-09T05:47:53.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:53 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3600923881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:47:54.139 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 9919d1d8 --release --await' 2024-05-09T05:47:54.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id 9919d1d8 --release --await 2024-05-09T05:47:54.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.620+0000 7f94b1822700 1 -- 172.21.15.115:0/578639729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94ac10e8e0 msgr2=0x7f94ac110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:54.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.620+0000 7f94b1822700 1 --2- 172.21.15.115:0/578639729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94ac10e8e0 0x7f94ac110dc0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f94a400b0a0 tx=0x7f94a400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.620+0000 7f94b1822700 1 -- 172.21.15.115:0/578639729 shutdown_connections 2024-05-09T05:47:54.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.620+0000 7f94b1822700 1 --2- 172.21.15.115:0/578639729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94ac10e8e0 0x7f94ac110dc0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.620+0000 7f94b1822700 1 --2- 172.21.15.115:0/578639729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac105500 0x7f94ac105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.620+0000 7f94b1822700 1 --2- 172.21.15.115:0/578639729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac105e50 0x7f94ac10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.620+0000 7f94b1822700 1 -- 172.21.15.115:0/578639729 >> 172.21.15.115:0/578639729 conn(0x7f94ac0fedc0 msgr2=0x7f94ac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:54.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.620+0000 7f94b1822700 1 -- 172.21.15.115:0/578639729 shutdown_connections 2024-05-09T05:47:54.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.620+0000 7f94b1822700 1 -- 172.21.15.115:0/578639729 wait complete. 2024-05-09T05:47:54.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.621+0000 7f94b1822700 1 Processor -- start 2024-05-09T05:47:54.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.621+0000 7f94b1822700 1 -- start start 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94b1822700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac105500 0x7f94ac1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94b1822700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94ac105e50 0x7f94ac1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94b1822700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac10e8e0 0x7f94ac19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94b1822700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1142f0 con 0x7f94ac105e50 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94b1822700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac114170 con 0x7f94ac105500 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94b1822700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac114470 con 0x7f94ac10e8e0 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94ac105e50 0x7f94ac1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aaffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac105500 0x7f94ac1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aaffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac105500 0x7f94ac1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:56364/0 (socket says 172.21.15.115:56364) 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac10e8e0 0x7f94ac19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aaffd700 1 -- 172.21.15.115:0/617223753 learned_addr learned my addr 172.21.15.115:0/617223753 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aaffd700 1 -- 172.21.15.115:0/617223753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac10e8e0 msgr2=0x7f94ac19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aaffd700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac10e8e0 0x7f94ac19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aaffd700 1 -- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94ac105e50 msgr2=0x7f94ac1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aaffd700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94ac105e50 0x7f94ac1a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aaffd700 1 -- 172.21.15.115:0/617223753 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac19b260 con 0x7f94ac105500 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.622+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94ac105e50 0x7f94ac1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.623+0000 7f94aaffd700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac105500 0x7f94ac1a09a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f949c00d950 tx=0x7f949c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.623+0000 7f94b0820700 1 -- 172.21.15.115:0/617223753 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c016070 con 0x7f94ac105500 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.623+0000 7f94b1822700 1 -- 172.21.15.115:0/617223753 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac19b4f0 con 0x7f94ac105500 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.623+0000 7f94b0820700 1 -- 172.21.15.115:0/617223753 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f949c00fd20 con 0x7f94ac105500 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.623+0000 7f94b1822700 1 -- 172.21.15.115:0/617223753 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac1a7750 con 0x7f94ac105500 2024-05-09T05:47:54.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.624+0000 7f94b0820700 1 -- 172.21.15.115:0/617223753 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c00e3b0 con 0x7f94ac105500 2024-05-09T05:47:54.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.626+0000 7f94b0820700 1 -- 172.21.15.115:0/617223753 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f949c019070 con 0x7f94ac105500 2024-05-09T05:47:54.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94c0238b80 1 Processor -- start 2024-05-09T05:47:54.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94c0238b80 1 -- start start 2024-05-09T05:47:54.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe7c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 0x562eebe7c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe76810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94c0238b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562eebd990a0 con 0x562eebd73d00 2024-05-09T05:47:54.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94c0238b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562eebd3e9a0 con 0x562eebcf94a0 2024-05-09T05:47:54.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94c0238b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562eebd95060 con 0x562eebe76400 2024-05-09T05:47:54.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe7c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe7c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51590/0 (socket says 172.21.15.115:51590) 2024-05-09T05:47:54.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94ab7fe700 1 -- 172.21.15.115:0/2926948505 learned_addr learned my addr 172.21.15.115:0/2926948505 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94b0820700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94940b29f0 0x7f94940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe76810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94aaffd700 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 0x562eebe7c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94ab7fe700 1 -- 172.21.15.115:0/2926948505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 msgr2=0x562eebe76810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe76810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94ab7fe700 1 -- 172.21.15.115:0/2926948505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 msgr2=0x562eebe7c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 0x562eebe7c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.627+0000 7f94ab7fe700 1 -- 172.21.15.115:0/2926948505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eebd2f500 con 0x562eebd73d00 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.628+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe76810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.628+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94940b29f0 0x7f94940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.628+0000 7f94aaffd700 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 0x562eebe7c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:47:54.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.628+0000 7f94b0820700 1 -- 172.21.15.115:0/617223753 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f949c0e2820 con 0x7f94ac105500 2024-05-09T05:47:54.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.628+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe7c240 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009270 tx=0x7f94a40092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabe0c8ddb5fab80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:54.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.628+0000 7f9491ffb700 1 -- 172.21.15.115:0/2926948505 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a402e070 con 0x562eebd73d00 2024-05-09T05:47:54.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.628+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94940b29f0 0x7f94940b4f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94a402b000 tx=0x7f94a402c000 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:54.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.628+0000 7f9491ffb700 1 -- 172.21.15.115:0/2926948505 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94a4007ba0 con 0x562eebd73d00 2024-05-09T05:47:54.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.628+0000 7f9491ffb700 1 -- 172.21.15.115:0/2926948505 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4028e30 con 0x562eebd73d00 2024-05-09T05:47:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.629+0000 7f94c0238b80 1 -- 172.21.15.115:0/2926948505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 msgr2=0x562eebe7c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.629+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe7c240 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009270 tx=0x7f94a40092a0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.629+0000 7f94c0238b80 1 -- 172.21.15.115:0/2926948505 shutdown_connections 2024-05-09T05:47:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.629+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe76810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.629+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe7c240 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.629+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2926948505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 0x562eebe7c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.629+0000 7f94c0238b80 1 -- 172.21.15.115:0/2926948505 >> 172.21.15.115:0/2926948505 conn(0x562eebcfcf70 msgr2=0x562eebe71220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.629+0000 7f94c0238b80 1 -- 172.21.15.115:0/2926948505 shutdown_connections 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.629+0000 7f94c0238b80 1 -- 172.21.15.115:0/2926948505 wait complete. 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94c0238b80 1 -- start start 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94c0238b80 1 Processor -- start 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebcf94a0 0x562eebe7fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 0x562eebe80530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe80a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94c0238b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562eebe77160 con 0x562eebcf94a0 2024-05-09T05:47:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94c0238b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562eebd33b20 con 0x562eebd73d00 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94c0238b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562eebe83020 con 0x562eebe76400 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94c0238b80 10 client.0 _ll_register_callbacks cb 0x562eebc0a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94aaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebcf94a0 0x562eebe7fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94aaffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebcf94a0 0x562eebe7fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51598/0 (socket says 172.21.15.115:51598) 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94aaffd700 1 -- 172.21.15.115:0/3988171939 learned_addr learned my addr 172.21.15.115:0/3988171939 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.630+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe80a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 0x562eebe80530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f94aaffd700 1 -- 172.21.15.115:0/3988171939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 msgr2=0x562eebe80a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f94aaffd700 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe80a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f94aaffd700 1 -- 172.21.15.115:0/3988171939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 msgr2=0x562eebe80530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f94aaffd700 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 0x562eebe80530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f94aaffd700 1 -- 172.21.15.115:0/3988171939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eebc14490 con 0x562eebcf94a0 2024-05-09T05:47:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe80a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f94aaffd700 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebcf94a0 0x562eebe7fff0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f949c0a57e0 tx=0x7f949c0a91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8de3da449cc1bef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f948affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f948affd700 1 -- 172.21.15.115:0/3988171939 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c020030 con 0x562eebcf94a0 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f94c0238b80 1 -- 172.21.15.115:0/3988171939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a4007ba0 con 0x562eebcf94a0 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.631+0000 7f948affd700 1 -- 172.21.15.115:0/3988171939 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f949c091110 con 0x562eebcf94a0 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.632+0000 7f948affd700 1 -- 172.21.15.115:0/3988171939 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f949c0e0060 con 0x562eebcf94a0 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.632+0000 7f94c0238b80 10 client.18099 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.632+0000 7f94c0238b80 1 -- 172.21.15.115:0/3988171939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562eebe8d8f0 con 0x562eebcf94a0 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.633+0000 7f948affd700 1 -- 172.21.15.115:0/3988171939 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94a4007ba0 con 0x562eebcf94a0 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.633+0000 7f948affd700 1 -- 172.21.15.115:0/3988171939 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f949c0a99c0 con 0x562eebcf94a0 2024-05-09T05:47:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.633+0000 7f94c0238b80 10 client.18099 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.633+0000 7f94c0238b80 10 client.18099 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.633+0000 7f94c0238b80 20 client.18099 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.633+0000 7f94c0238b80 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x562eebe8fe80 0x562eebe922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.633+0000 7f94c0238b80 4 client.18099 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.633+0000 7f94c0238b80 1 -- 172.21.15.115:0/3988171939 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eebda6a80 con 0x562eebe8fe80 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.633+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x562eebe8fe80 0x562eebe922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.634+0000 7f948affd700 10 client.18099 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.634+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x562eebe8fe80 0x562eebe922e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b6cffd3b46dda41 server_cookie=49f696ec9083f854 in_seq=0 out_seq=0 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f948affd700 1 -- 172.21.15.115:0/3988171939 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562eebda6a80 con 0x562eebe8fe80 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f948affd700 10 client.18099 handle_command_reply: tid=0 2024-05-09T05:47:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 client.18099 shutdown 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 -- 172.21.15.115:0/3988171939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebcf94a0 msgr2=0x562eebe7fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebcf94a0 0x562eebe7fff0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f949c0a57e0 tx=0x7f949c0a91a0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 -- 172.21.15.115:0/3988171939 shutdown_connections 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x562eebe8fe80 0x562eebe922e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe80a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebcf94a0 0x562eebe7fff0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 --2- 172.21.15.115:0/3988171939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 0x562eebe80530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 -- 172.21.15.115:0/3988171939 >> 172.21.15.115:0/3988171939 conn(0x562eebcfcf70 msgr2=0x562eebe6aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f948affd700 0 client.18099 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 -- 172.21.15.115:0/3988171939 shutdown_connections 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 1 -- 172.21.15.115:0/3988171939 wait complete. 2024-05-09T05:47:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.635+0000 7f94c0238b80 20 client.18099 trim_cache size 0 max 16384 2024-05-09T05:47:54.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94c0238b80 1 Processor -- start 2024-05-09T05:47:54.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94c0238b80 1 -- start start 2024-05-09T05:47:54.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 0x562eebe7b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe8be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe97ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94c0238b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1b0390 con 0x562eebd73d00 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94c0238b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1b1f90 con 0x562eebcf94a0 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94c0238b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1b0a60 con 0x562eebe76400 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94aa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe8be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94aa7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe8be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51604/0 (socket says 172.21.15.115:51604) 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.651+0000 7f94aa7fc700 1 -- 172.21.15.115:0/1439822155 learned_addr learned my addr 172.21.15.115:0/1439822155 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94aaffd700 1 --2- 172.21.15.115:0/1439822155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 0x562eebe7b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/1439822155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe97ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94aa7fc700 1 -- 172.21.15.115:0/1439822155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 msgr2=0x562eebe97ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/1439822155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe97ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94aa7fc700 1 -- 172.21.15.115:0/1439822155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 msgr2=0x562eebe7b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/1439822155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 0x562eebe7b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94aa7fc700 1 -- 172.21.15.115:0/1439822155 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eebe7f310 con 0x562eebd73d00 2024-05-09T05:47:54.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/1439822155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe8be20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f94ac19c260 tx=0x7f94a0014a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21a8e1292634cba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94917fa700 1 -- 172.21.15.115:0/1439822155 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0014390 con 0x562eebd73d00 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94917fa700 1 -- 172.21.15.115:0/1439822155 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94a00044e0 con 0x562eebd73d00 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.652+0000 7f94917fa700 1 -- 172.21.15.115:0/1439822155 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0012590 con 0x562eebd73d00 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.653+0000 7f94c0238b80 1 -- 172.21.15.115:0/1439822155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 msgr2=0x562eebe8be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.653+0000 7f94c0238b80 1 --2- 172.21.15.115:0/1439822155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe8be20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f94ac19c260 tx=0x7f94a0014a80 comp rx=0 tx=0).stop 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.653+0000 7f94c0238b80 1 -- 172.21.15.115:0/1439822155 shutdown_connections 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.653+0000 7f94c0238b80 1 --2- 172.21.15.115:0/1439822155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebcf94a0 0x562eebe7b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.653+0000 7f94c0238b80 1 --2- 172.21.15.115:0/1439822155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebe76400 0x562eebe97ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.653+0000 7f94c0238b80 1 --2- 172.21.15.115:0/1439822155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebd73d00 0x562eebe8be20 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.653+0000 7f94c0238b80 1 -- 172.21.15.115:0/1439822155 >> 172.21.15.115:0/1439822155 conn(0x562eebcfcf70 msgr2=0x562eebe7b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.653+0000 7f94c0238b80 1 -- 172.21.15.115:0/1439822155 shutdown_connections 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.654+0000 7f94c0238b80 1 -- 172.21.15.115:0/1439822155 wait complete. 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.654+0000 7f94c0238b80 1 -- start start 2024-05-09T05:47:54.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.654+0000 7f94c0238b80 1 Processor -- start 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebcf94a0 0x562eebe97ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 0x562eebe881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94c0238b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebe76400 0x562eebe886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94c0238b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1b18c0 con 0x562eebe76400 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94c0238b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1b1160 con 0x562eebd73d00 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94c0238b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1ae330 con 0x562eebcf94a0 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebe76400 0x562eebe886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94c0238b80 10 client.0 _ll_register_callbacks cb 0x562eebc0a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebe76400 0x562eebe886e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51608/0 (socket says 172.21.15.115:51608) 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94ab7fe700 1 -- 172.21.15.115:0/2240007599 learned_addr learned my addr 172.21.15.115:0/2240007599 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94aa7fc700 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 0x562eebe881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94aaffd700 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebcf94a0 0x562eebe97ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94ab7fe700 1 -- 172.21.15.115:0/2240007599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebcf94a0 msgr2=0x562eebe97ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebcf94a0 0x562eebe97ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94ab7fe700 1 -- 172.21.15.115:0/2240007599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 msgr2=0x562eebe881a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 0x562eebe881a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94ab7fe700 1 -- 172.21.15.115:0/2240007599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eebe96ad0 con 0x562eebe76400 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f94ab7fe700 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebe76400 0x562eebe886e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f94a400b3f0 tx=0x7f94a4009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b10444105a5e88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.655+0000 7f948affd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f948affd700 1 -- 172.21.15.115:0/2240007599 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a403b060 con 0x562eebe76400 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f948affd700 1 -- 172.21.15.115:0/2240007599 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f94a4032440 con 0x562eebe76400 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f94c0238b80 1 -- 172.21.15.115:0/2240007599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a00044e0 con 0x562eebe76400 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f948affd700 1 -- 172.21.15.115:0/2240007599 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f94a4042050 con 0x562eebe76400 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f94c0238b80 10 client.18105 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f94c0238b80 1 -- 172.21.15.115:0/2240007599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562eebef9780 con 0x562eebe76400 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f948affd700 1 -- 172.21.15.115:0/2240007599 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94a00044e0 con 0x562eebe76400 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f948affd700 1 -- 172.21.15.115:0/2240007599 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f94a4025040 con 0x562eebe76400 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f94c0238b80 10 client.18105 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f94c0238b80 10 client.18105 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-05-09T05:47:54.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f94c0238b80 20 client.18105 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:54.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x562eebef9a00 0x562eebefbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:54.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f94c0238b80 4 client.18105 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:47:54.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.656+0000 7f94c0238b80 1 -- 172.21.15.115:0/2240007599 --> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562eebda0b20 con 0x562eebef9a00 2024-05-09T05:47:54.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.657+0000 7f94aaffd700 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x562eebef9a00 0x562eebefbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:54.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.657+0000 7f948affd700 10 client.18105 ms_handle_connect on v2:172.21.15.195:6836/4026198088 2024-05-09T05:47:54.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.657+0000 7f94aaffd700 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x562eebef9a00 0x562eebefbe20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e06fa2db009edca8 server_cookie=abcec1c4f6ee8f5e in_seq=0 out_seq=0 2024-05-09T05:47:54.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.657+0000 7f948affd700 1 -- 172.21.15.115:0/2240007599 <== mds.1 v2:172.21.15.195:6836/4026198088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562eebda0b20 con 0x562eebef9a00 2024-05-09T05:47:54.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.657+0000 7f948affd700 10 client.18105 handle_command_reply: tid=0 2024-05-09T05:47:54.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.657+0000 7f94c0238b80 1 client.18105 shutdown 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 -- 172.21.15.115:0/2240007599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebe76400 msgr2=0x562eebe886e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebe76400 0x562eebe886e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f94a400b3f0 tx=0x7f94a4009570 comp rx=0 tx=0).stop 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 -- 172.21.15.115:0/2240007599 shutdown_connections 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.195:6836/4026198088,v1:172.21.15.195:6837/4026198088] conn(0x562eebef9a00 0x562eebefbe20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562eebe76400 0x562eebe886e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562eebcf94a0 0x562eebe97ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 --2- 172.21.15.115:0/2240007599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562eebd73d00 0x562eebe881a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 -- 172.21.15.115:0/2240007599 >> 172.21.15.115:0/2240007599 conn(0x562eebcfcf70 msgr2=0x562eebe6aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f948affd700 0 client.18105 ms_handle_reset on v2:172.21.15.195:6836/4026198088 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 -- 172.21.15.115:0/2240007599 shutdown_connections 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 1 -- 172.21.15.115:0/2240007599 wait complete. 2024-05-09T05:47:54.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94c0238b80 20 client.18105 trim_cache size 0 max 16384 2024-05-09T05:47:54.659 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:47:54.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94b1822700 1 -- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94940b29f0 msgr2=0x7f94940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94b1822700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94940b29f0 0x7f94940b4f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94a402b000 tx=0x7f94a402c000 comp rx=0 tx=0).stop 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94b1822700 1 -- 172.21.15.115:0/617223753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac105500 msgr2=0x7f94ac1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94b1822700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac105500 0x7f94ac1a09a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f949c00d950 tx=0x7f949c00dc60 comp rx=0 tx=0).stop 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94b1822700 1 -- 172.21.15.115:0/617223753 shutdown_connections 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94b1822700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94940b29f0 0x7f94940b4f00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94b1822700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac105500 0x7f94ac1a09a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94b1822700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94ac10e8e0 0x7f94ac19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.658+0000 7f94b1822700 1 --2- 172.21.15.115:0/617223753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94ac105e50 0x7f94ac1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.659+0000 7f94b1822700 1 -- 172.21.15.115:0/617223753 >> 172.21.15.115:0/617223753 conn(0x7f94ac0fedc0 msgr2=0x7f94ac0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.659+0000 7f94b1822700 1 -- 172.21.15.115:0/617223753 shutdown_connections 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:54.659+0000 7f94b1822700 1 -- 172.21.15.115:0/617223753 wait complete. 2024-05-09T05:47:54.661 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:55.327 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:55 smithi115 ceph-mon[30349]: pgmap v302: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-05-09T05:47:55.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:47:55.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:55 smithi195 ceph-mon[33844]: pgmap v302: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-05-09T05:47:55.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:55 smithi161 ceph-mon[33805]: pgmap v302: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-05-09T05:47:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 -- 172.21.15.115:0/3294413958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8109030 msgr2=0x7fcaf8102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/3294413958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8109030 0x7fcaf8102c70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009900 tx=0x7fcae802a410 comp rx=0 tx=0).stop 2024-05-09T05:47:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 -- 172.21.15.115:0/3294413958 shutdown_connections 2024-05-09T05:47:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/3294413958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf81032e0 0x7fcaf810f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/3294413958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8109030 0x7fcaf8102c70 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/3294413958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf8108b90 0x7fcaf8102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 -- 172.21.15.115:0/3294413958 >> 172.21.15.115:0/3294413958 conn(0x7fcaf80fe390 msgr2=0x7fcaf81007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 -- 172.21.15.115:0/3294413958 shutdown_connections 2024-05-09T05:47:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 -- 172.21.15.115:0/3294413958 wait complete. 2024-05-09T05:47:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 Processor -- start 2024-05-09T05:47:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.848+0000 7fcaffe2b640 1 -- start start 2024-05-09T05:47:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcaffe2b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf81032e0 0x7fcaf8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcaffe2b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8108b90 0x7fcaf8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcaffe2b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8109030 0x7fcaf819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcaffe2b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112c30 con 0x7fcaf8108b90 2024-05-09T05:47:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcaffe2b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112ab0 con 0x7fcaf81032e0 2024-05-09T05:47:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcaffe2b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112db0 con 0x7fcaf8109030 2024-05-09T05:47:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafd3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8108b90 0x7fcaf8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafd3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8108b90 0x7fcaf8199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51626/0 (socket says 172.21.15.115:51626) 2024-05-09T05:47:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafd3c6640 1 -- 172.21.15.115:0/4085680478 learned_addr learned my addr 172.21.15.115:0/4085680478 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf81032e0 0x7fcaf8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8109030 0x7fcaf819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafd3c6640 1 -- 172.21.15.115:0/4085680478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8109030 msgr2=0x7fcaf819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8109030 0x7fcaf819df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafd3c6640 1 -- 172.21.15.115:0/4085680478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf81032e0 msgr2=0x7fcaf8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf81032e0 0x7fcaf8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafd3c6640 1 -- 172.21.15.115:0/4085680478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf819e5b0 con 0x7fcaf8108b90 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.849+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8108b90 0x7fcaf8199cb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005940 tx=0x7fcae80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.850+0000 7fcaeeffd640 1 -- 172.21.15.115:0/4085680478 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae803f030 con 0x7fcaf8108b90 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.850+0000 7fcaeeffd640 1 -- 172.21.15.115:0/4085680478 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcae8003a80 con 0x7fcaf8108b90 2024-05-09T05:47:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.850+0000 7fcaeeffd640 1 -- 172.21.15.115:0/4085680478 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8035070 con 0x7fcaf8108b90 2024-05-09T05:47:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.850+0000 7fcaffe2b640 1 -- 172.21.15.115:0/4085680478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf819e840 con 0x7fcaf8108b90 2024-05-09T05:47:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.850+0000 7fcaffe2b640 1 -- 172.21.15.115:0/4085680478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf819ece0 con 0x7fcaf8108b90 2024-05-09T05:47:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.852+0000 7fcb06dc3740 1 Processor -- start 2024-05-09T05:47:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.852+0000 7fcb06dc3740 1 -- start start 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.852+0000 7fcaeeffd640 1 -- 172.21.15.115:0/4085680478 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcae8042030 con 0x7fcaf8108b90 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 0x55a7fb64acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 0x55a7fb6494d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb649a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb5ae7c0 con 0x55a7fb2854d0 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb54a0e0 con 0x55a7fb2a61b0 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb5aa780 con 0x55a7fb27d2f0 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafd3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb649a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafd3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb649a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51638/0 (socket says 172.21.15.115:51638) 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafd3c6640 1 -- 172.21.15.115:0/850719268 learned_addr learned my addr 172.21.15.115:0/850719268 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/850719268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 0x55a7fb64acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/850719268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 0x55a7fb6494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafd3c6640 1 -- 172.21.15.115:0/850719268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 msgr2=0x55a7fb6494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/850719268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 0x55a7fb6494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafd3c6640 1 -- 172.21.15.115:0/850719268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 msgr2=0x55a7fb64acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/850719268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 0x55a7fb64acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafd3c6640 1 -- 172.21.15.115:0/850719268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fb5737c0 con 0x55a7fb2854d0 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcaeeffd640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcae40b29d0 0x7fcae40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.853+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/850719268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb649a70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcae8100190 tx=0x7fcae8106f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4075438ca3a4955 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.854+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcae40b29d0 0x7fcae40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.854+0000 7fcae3fff640 1 -- 172.21.15.115:0/850719268 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80d2e60 con 0x55a7fb2854d0 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.854+0000 7fcaeeffd640 1 -- 172.21.15.115:0/4085680478 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fcae8102e10 con 0x7fcaf8108b90 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.854+0000 7fcae3fff640 1 -- 172.21.15.115:0/850719268 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcae8110070 con 0x55a7fb2854d0 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.854+0000 7fcae3fff640 1 -- 172.21.15.115:0/850719268 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80c5c70 con 0x55a7fb2854d0 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 -- 172.21.15.115:0/850719268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 msgr2=0x55a7fb649a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/850719268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb649a70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcae8100190 tx=0x7fcae8106f90 comp rx=0 tx=0).stop 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcae40b29d0 0x7fcae40b4ee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55a7fb647090 tx=0x7fcaf0009740 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 -- 172.21.15.115:0/850719268 shutdown_connections 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/850719268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb649a70 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/850719268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 0x55a7fb6494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/850719268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 0x55a7fb64acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 -- 172.21.15.115:0/850719268 >> 172.21.15.115:0/850719268 conn(0x55a7fb29a300 msgr2=0x55a7fb645e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 -- 172.21.15.115:0/850719268 shutdown_connections 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 -- 172.21.15.115:0/850719268 wait complete. 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 -- start start 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.855+0000 7fcb06dc3740 1 Processor -- start 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 0x55a7fb658260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb6587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 0x55a7fb663cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb5359c0 con 0x55a7fb2854d0 2024-05-09T05:47:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb65ab10 con 0x55a7fb2a61b0 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb664230 con 0x55a7fb27d2f0 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcb06dc3740 10 client.0 _ll_register_callbacks cb 0x55a7fb562020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafd3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb6587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafd3c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb6587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51650/0 (socket says 172.21.15.115:51650) 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafd3c6640 1 -- 172.21.15.115:0/3804897862 learned_addr learned my addr 172.21.15.115:0/3804897862 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 0x55a7fb663cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 0x55a7fb658260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafd3c6640 1 -- 172.21.15.115:0/3804897862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 msgr2=0x55a7fb658260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 0x55a7fb658260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafd3c6640 1 -- 172.21.15.115:0/3804897862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 msgr2=0x55a7fb663cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 0x55a7fb663cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafd3c6640 1 -- 172.21.15.115:0/3804897862 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fb55c1f0 con 0x55a7fb2854d0 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 0x55a7fb663cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb6587a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fcae8006010 tx=0x7fcae8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7403c946f91942c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcae17fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcae17fa640 1 -- 172.21.15.115:0/3804897862 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8049070 con 0x55a7fb2854d0 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcae17fa640 1 -- 172.21.15.115:0/3804897862 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fcae8003c30 con 0x55a7fb2854d0 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.856+0000 7fcb06dc3740 1 -- 172.21.15.115:0/3804897862 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae8110070 con 0x55a7fb2854d0 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcae17fa640 1 -- 172.21.15.115:0/3804897862 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fcae8035210 con 0x55a7fb2854d0 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcb06dc3740 10 client.18117 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcb06dc3740 1 -- 172.21.15.115:0/3804897862 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a7fb311790 con 0x55a7fb2854d0 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcae17fa640 1 -- 172.21.15.115:0/3804897862 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcae8110070 con 0x55a7fb2854d0 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcae17fa640 1 -- 172.21.15.115:0/3804897862 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fcae81004a0 con 0x55a7fb2854d0 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcb06dc3740 10 client.18117 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcb06dc3740 10 client.18117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcb06dc3740 20 client.18117 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a7fb264d80 0x55a7fb669eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcb06dc3740 4 client.18117 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcb06dc3740 1 -- 172.21.15.115:0/3804897862 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7fb5bc3d0 con 0x55a7fb264d80 2024-05-09T05:47:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.857+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a7fb264d80 0x55a7fb669eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.858+0000 7fcae17fa640 10 client.18117 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.858+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a7fb264d80 0x55a7fb669eb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb63560f3fed06a server_cookie=7adfa85477cbac35 in_seq=0 out_seq=0 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcae17fa640 1 -- 172.21.15.115:0/3804897862 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7fb5bc3d0 con 0x55a7fb264d80 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcae17fa640 10 client.18117 handle_command_reply: tid=0 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 client.18117 shutdown 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 -- 172.21.15.115:0/3804897862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 msgr2=0x55a7fb6587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb6587a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fcae8006010 tx=0x7fcae8005ef0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 -- 172.21.15.115:0/3804897862 shutdown_connections 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb2a61b0 0x55a7fb663cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb2854d0 0x55a7fb6587a0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a7fb264d80 0x55a7fb669eb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/3804897862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb27d2f0 0x55a7fb658260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 -- 172.21.15.115:0/3804897862 >> 172.21.15.115:0/3804897862 conn(0x55a7fb29a300 msgr2=0x55a7fb644700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcae17fa640 0 client.18117 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 -- 172.21.15.115:0/3804897862 shutdown_connections 2024-05-09T05:47:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 1 -- 172.21.15.115:0/3804897862 wait complete. 2024-05-09T05:47:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.859+0000 7fcb06dc3740 20 client.18117 trim_cache size 0 max 16384 2024-05-09T05:47:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.876+0000 7fcb06dc3740 1 Processor -- start 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.876+0000 7fcb06dc3740 1 -- start start 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb64ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 0x55a7fb644e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 0x55a7fb663950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81aef10 con 0x55a7fb29a300 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81b0b10 con 0x55a7fb27d2f0 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81af5e0 con 0x55a7fb2854d0 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafdbc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb64ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafdbc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb64ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51652/0 (socket says 172.21.15.115:51652) 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafdbc7640 1 -- 172.21.15.115:0/4012442645 learned_addr learned my addr 172.21.15.115:0/4012442645 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/4012442645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 0x55a7fb663950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/4012442645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 0x55a7fb644e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafdbc7640 1 -- 172.21.15.115:0/4012442645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 msgr2=0x55a7fb663950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/4012442645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 0x55a7fb663950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafdbc7640 1 -- 172.21.15.115:0/4012442645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 msgr2=0x55a7fb644e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/4012442645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 0x55a7fb644e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafdbc7640 1 -- 172.21.15.115:0/4012442645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fb3885f0 con 0x55a7fb29a300 2024-05-09T05:47:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/4012442645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb64ac90 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x55a7fb66a5d0 tx=0x7fcaf400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9238e205e01cc944 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.877+0000 7fcae37fe640 1 -- 172.21.15.115:0/4012442645 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4021030 con 0x55a7fb29a300 2024-05-09T05:47:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcae37fe640 1 -- 172.21.15.115:0/4012442645 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcaf4005020 con 0x55a7fb29a300 2024-05-09T05:47:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcae37fe640 1 -- 172.21.15.115:0/4012442645 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf401d880 con 0x55a7fb29a300 2024-05-09T05:47:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcb06dc3740 1 -- 172.21.15.115:0/4012442645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 msgr2=0x55a7fb64ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/4012442645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb64ac90 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x55a7fb66a5d0 tx=0x7fcaf400bdf0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcb06dc3740 1 -- 172.21.15.115:0/4012442645 shutdown_connections 2024-05-09T05:47:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/4012442645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 0x55a7fb663950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/4012442645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 0x55a7fb644e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/4012442645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb64ac90 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcb06dc3740 1 -- 172.21.15.115:0/4012442645 >> 172.21.15.115:0/4012442645 conn(0x55a7fb6539f0 msgr2=0x55a7fb644700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcb06dc3740 1 -- 172.21.15.115:0/4012442645 shutdown_connections 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.878+0000 7fcb06dc3740 1 -- 172.21.15.115:0/4012442645 wait complete. 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcb06dc3740 1 -- start start 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcb06dc3740 1 Processor -- start 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 0x55a7fb65fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 0x55a7fb65b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcb06dc3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb65b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81b0440 con 0x55a7fb29a300 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81afce0 con 0x55a7fb27d2f0 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcb06dc3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81acef0 con 0x55a7fb2854d0 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcb06dc3740 10 client.0 _ll_register_callbacks cb 0x7fcaf00095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcafe3c8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb65b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcafe3c8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb65b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51666/0 (socket says 172.21.15.115:51666) 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.879+0000 7fcafe3c8640 1 -- 172.21.15.115:0/2786360820 learned_addr learned my addr 172.21.15.115:0/2786360820 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 0x55a7fb65fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcafd3c6640 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 0x55a7fb65b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcafe3c8640 1 -- 172.21.15.115:0/2786360820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 msgr2=0x55a7fb65b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 0x55a7fb65b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcafe3c8640 1 -- 172.21.15.115:0/2786360820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 msgr2=0x55a7fb65fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 0x55a7fb65fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcafe3c8640 1 -- 172.21.15.115:0/2786360820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fb572ed0 con 0x55a7fb29a300 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcafe3c8640 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb65b7b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcaf001fac0 tx=0x7fcaf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770c8b8e550c050d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcae17fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcae17fa640 1 -- 172.21.15.115:0/2786360820 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf002d060 con 0x55a7fb29a300 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcae17fa640 1 -- 172.21.15.115:0/2786360820 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fcaf001d7d0 con 0x55a7fb29a300 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcb06dc3740 1 -- 172.21.15.115:0/2786360820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf4005020 con 0x55a7fb29a300 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcae17fa640 1 -- 172.21.15.115:0/2786360820 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fcaf0034050 con 0x55a7fb29a300 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcb06dc3740 10 client.18123 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcb06dc3740 1 -- 172.21.15.115:0/2786360820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a7fb6d1bf0 con 0x55a7fb29a300 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.880+0000 7fcae17fa640 1 -- 172.21.15.115:0/2786360820 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcaf4005020 con 0x55a7fb29a300 2024-05-09T05:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcae17fa640 1 -- 172.21.15.115:0/2786360820 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fcaf0027070 con 0x55a7fb29a300 2024-05-09T05:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcb06dc3740 10 client.18123 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcb06dc3740 10 client.18123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcb06dc3740 20 client.18123 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a7fb2a61b0 0x55a7fb6d4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcb06dc3740 4 client.18123 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcb06dc3740 1 -- 172.21.15.115:0/2786360820 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7fb5b6240 con 0x55a7fb2a61b0 2024-05-09T05:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a7fb2a61b0 0x55a7fb6d4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcafdbc7640 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a7fb2a61b0 0x55a7fb6d4450 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2329f47522393030 server_cookie=7fb935acb202d644 in_seq=0 out_seq=0 2024-05-09T05:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.881+0000 7fcae17fa640 10 client.18123 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcae17fa640 1 -- 172.21.15.115:0/2786360820 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a7fb5b6240 con 0x55a7fb2a61b0 2024-05-09T05:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcae17fa640 10 client.18123 handle_command_reply: tid=0 2024-05-09T05:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 client.18123 shutdown 2024-05-09T05:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 -- 172.21.15.115:0/2786360820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 msgr2=0x55a7fb65b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb65b7b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcaf001fac0 tx=0x7fcaf0004a40 comp rx=0 tx=0).stop 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 -- 172.21.15.115:0/2786360820 shutdown_connections 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a7fb29a300 0x55a7fb65b7b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a7fb2a61b0 0x55a7fb6d4450 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a7fb2854d0 0x55a7fb65b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcae17fa640 0 client.18123 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 --2- 172.21.15.115:0/2786360820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7fb27d2f0 0x55a7fb65fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 -- 172.21.15.115:0/2786360820 >> 172.21.15.115:0/2786360820 conn(0x55a7fb6539f0 msgr2=0x55a7fb645680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 -- 172.21.15.115:0/2786360820 shutdown_connections 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.886+0000 7fcb06dc3740 1 -- 172.21.15.115:0/2786360820 wait complete. 2024-05-09T05:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcb06dc3740 20 client.18123 trim_cache size 0 max 16384 2024-05-09T05:47:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:47:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:47:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 -- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcae40b29d0 msgr2=0x7fcae40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcae40b29d0 0x7fcae40b4ee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55a7fb647090 tx=0x7fcaf0009740 comp rx=0 tx=0).stop 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 -- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8108b90 msgr2=0x7fcaf8199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8108b90 0x7fcaf8199cb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005940 tx=0x7fcae80040e0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 -- 172.21.15.115:0/4085680478 shutdown_connections 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcae40b29d0 0x7fcae40b4ee0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8109030 0x7fcaf819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaf8108b90 0x7fcaf8199cb0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 --2- 172.21.15.115:0/4085680478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf81032e0 0x7fcaf8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 -- 172.21.15.115:0/4085680478 >> 172.21.15.115:0/4085680478 conn(0x7fcaf80fe390 msgr2=0x7fcaf810e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 -- 172.21.15.115:0/4085680478 shutdown_connections 2024-05-09T05:47:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:47:56.887+0000 7fcaffe2b640 1 -- 172.21.15.115:0/4085680478 wait complete. 2024-05-09T05:47:57.257 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:57 smithi115 ceph-mon[30349]: pgmap v303: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-09T05:47:57.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.279+0000 7fade9f41700 1 -- 172.21.15.115:0/2071032904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade41034d0 msgr2=0x7fade41038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:57.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.279+0000 7fade9f41700 1 --2- 172.21.15.115:0/2071032904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade41034d0 0x7fade41038e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fadd40099c0 tx=0x7fadd4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.279+0000 7fade9f41700 1 -- 172.21.15.115:0/2071032904 shutdown_connections 2024-05-09T05:47:57.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.279+0000 7fade9f41700 1 --2- 172.21.15.115:0/2071032904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fade4108750 0x7fade41100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.279+0000 7fade9f41700 1 --2- 172.21.15.115:0/2071032904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade41034d0 0x7fade41038e0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.279+0000 7fade9f41700 1 --2- 172.21.15.115:0/2071032904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade4107e00 0x7fade4108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.279+0000 7fade9f41700 1 -- 172.21.15.115:0/2071032904 >> 172.21.15.115:0/2071032904 conn(0x7fade40fef20 msgr2=0x7fade4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.279+0000 7fade9f41700 1 -- 172.21.15.115:0/2071032904 shutdown_connections 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.279+0000 7fade9f41700 1 -- 172.21.15.115:0/2071032904 wait complete. 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade9f41700 1 Processor -- start 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade9f41700 1 -- start start 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade9f41700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fade41034d0 0x7fade419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade9f41700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade4107e00 0x7fade419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade9f41700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade4108750 0x7fade419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade9f41700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fade4113230 con 0x7fade4108750 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade9f41700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fade41130b0 con 0x7fade41034d0 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade9f41700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fade41133b0 con 0x7fade4107e00 2024-05-09T05:47:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade4108750 0x7fade419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade4108750 0x7fade419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51682/0 (socket says 172.21.15.115:51682) 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade3fff700 1 -- 172.21.15.115:0/2414655548 learned_addr learned my addr 172.21.15.115:0/2414655548 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade37fe700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fade41034d0 0x7fade419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.280+0000 7fade2ffd700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade4107e00 0x7fade419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade3fff700 1 -- 172.21.15.115:0/2414655548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade4107e00 msgr2=0x7fade419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade3fff700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade4107e00 0x7fade419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade3fff700 1 -- 172.21.15.115:0/2414655548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fade41034d0 msgr2=0x7fade419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade3fff700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fade41034d0 0x7fade419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade3fff700 1 -- 172.21.15.115:0/2414655548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade419fc20 con 0x7fade4108750 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade3fff700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade4108750 0x7fade419f580 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7faddc007ee0 tx=0x7faddc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade0ff9700 1 -- 172.21.15.115:0/2414655548 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc00dd20 con 0x7fade4108750 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade9f41700 1 -- 172.21.15.115:0/2414655548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade419ff10 con 0x7fade4108750 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade0ff9700 1 -- 172.21.15.115:0/2414655548 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faddc004020 con 0x7fade4108750 2024-05-09T05:47:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade0ff9700 1 -- 172.21.15.115:0/2414655548 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc015660 con 0x7fade4108750 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.281+0000 7fade9f41700 1 -- 172.21.15.115:0/2414655548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41a7830 con 0x7fade4108750 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.284+0000 7fade0ff9700 1 -- 172.21.15.115:0/2414655548 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7faddc015800 con 0x7fade4108750 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.284+0000 7fade0ff9700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fadcc0b2a70 0x7fadcc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.284+0000 7fade37fe700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fadcc0b2a70 0x7fadcc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade0ff9700 1 -- 172.21.15.115:0/2414655548 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7faddc0e2210 con 0x7fade4108750 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fadf8957b80 1 Processor -- start 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fadf8957b80 1 -- start start 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade37fe700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fadcc0b2a70 0x7fadcc0b4f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadd4000c00 tx=0x7fadd4005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 0x55f1eb915b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 0x55f1eb916090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb924d10 0x55f1eb927130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fadf8957b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f1eb8391a0 con 0x55f1eb924d10 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fadf8957b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f1eb7deaa0 con 0x55f1eb7995a0 2024-05-09T05:47:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fadf8957b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f1eb835160 con 0x55f1eb813e00 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb924d10 0x55f1eb927130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb924d10 0x55f1eb927130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51686/0 (socket says 172.21.15.115:51686) 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade3fff700 1 -- 172.21.15.115:0/4190557352 learned_addr learned my addr 172.21.15.115:0/4190557352 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade37fe700 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 0x55f1eb915b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade2ffd700 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 0x55f1eb916090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade3fff700 1 -- 172.21.15.115:0/4190557352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 msgr2=0x55f1eb915b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade3fff700 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 0x55f1eb915b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade3fff700 1 -- 172.21.15.115:0/4190557352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 msgr2=0x55f1eb916090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade3fff700 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 0x55f1eb916090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade3fff700 1 -- 172.21.15.115:0/4190557352 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1eb7cf600 con 0x55f1eb924d10 2024-05-09T05:47:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.285+0000 7fade37fe700 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 0x55f1eb915b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.286+0000 7fade3fff700 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb924d10 0x55f1eb927130 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7faddc0a5230 tx=0x7faddc00df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5ceb56534bb87d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.286+0000 7fadc9ffb700 1 -- 172.21.15.115:0/4190557352 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc0ad560 con 0x55f1eb924d10 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.286+0000 7fadc9ffb700 1 -- 172.21.15.115:0/4190557352 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faddc0b2010 con 0x55f1eb924d10 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.286+0000 7fadc9ffb700 1 -- 172.21.15.115:0/4190557352 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc0a9650 con 0x55f1eb924d10 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 -- 172.21.15.115:0/4190557352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb924d10 msgr2=0x55f1eb927130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb924d10 0x55f1eb927130 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7faddc0a5230 tx=0x7faddc00df40 comp rx=0 tx=0).stop 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 -- 172.21.15.115:0/4190557352 shutdown_connections 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb924d10 0x55f1eb927130 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 0x55f1eb915b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4190557352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 0x55f1eb916090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 -- 172.21.15.115:0/4190557352 >> 172.21.15.115:0/4190557352 conn(0x55f1eb79d070 msgr2=0x55f1eb911b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 -- 172.21.15.115:0/4190557352 shutdown_connections 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 -- 172.21.15.115:0/4190557352 wait complete. 2024-05-09T05:47:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 -- start start 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.287+0000 7fadf8957b80 1 Processor -- start 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 0x55f1eb921da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb813e00 0x55f1eb9222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb924d10 0x55f1eb922820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadf8957b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f1eb927790 con 0x55f1eb813e00 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadf8957b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f1eb7d3c20 con 0x55f1eb7995a0 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadf8957b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f1eb927910 con 0x55f1eb924d10 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadf8957b80 10 client.0 _ll_register_callbacks cb 0x55f1eb6aa7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb813e00 0x55f1eb9222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb813e00 0x55f1eb9222e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51698/0 (socket says 172.21.15.115:51698) 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade2ffd700 1 -- 172.21.15.115:0/3963155344 learned_addr learned my addr 172.21.15.115:0/3963155344 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade37fe700 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 0x55f1eb921da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade3fff700 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb924d10 0x55f1eb922820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade2ffd700 1 -- 172.21.15.115:0/3963155344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb924d10 msgr2=0x55f1eb922820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade2ffd700 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb924d10 0x55f1eb922820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade2ffd700 1 -- 172.21.15.115:0/3963155344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 msgr2=0x55f1eb921da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade2ffd700 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 0x55f1eb921da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade2ffd700 1 -- 172.21.15.115:0/3963155344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1eb6b4590 con 0x55f1eb813e00 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade3fff700 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb924d10 0x55f1eb922820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fade2ffd700 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb813e00 0x55f1eb9222e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fadd800ba90 tx=0x7fadd800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb45eb72c9caf49 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadc2ffd700 1 -- 172.21.15.115:0/3963155344 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8004520 con 0x55f1eb813e00 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadc2ffd700 1 -- 172.21.15.115:0/3963155344 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fadd8004830 con 0x55f1eb813e00 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.288+0000 7fadf8957b80 1 -- 172.21.15.115:0/3963155344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faddc0b2010 con 0x55f1eb813e00 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadc2ffd700 1 -- 172.21.15.115:0/3963155344 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fadd8023050 con 0x55f1eb813e00 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadf8957b80 10 client.18135 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:57.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadf8957b80 1 -- 172.21.15.115:0/3963155344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f1eb932540 con 0x55f1eb813e00 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadc2ffd700 1 -- 172.21.15.115:0/3963155344 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faddc0b2010 con 0x55f1eb813e00 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadc2ffd700 1 -- 172.21.15.115:0/3963155344 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fadd8019030 con 0x55f1eb813e00 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadf8957b80 10 client.18135 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadf8957b80 10 client.18135 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadf8957b80 20 client.18135 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadf8957b80 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f1eb934c20 0x55f1eb937080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadf8957b80 4 client.18135 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.289+0000 7fadf8957b80 1 -- 172.21.15.115:0/3963155344 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1eb846b80 con 0x55f1eb934c20 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.290+0000 7fade37fe700 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f1eb934c20 0x55f1eb937080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.290+0000 7fadc2ffd700 10 client.18135 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.290+0000 7fade37fe700 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f1eb934c20 0x55f1eb937080 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbebd9ac0292bdab server_cookie=85071b479705cd2a in_seq=0 out_seq=0 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadc2ffd700 1 -- 172.21.15.115:0/3963155344 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1eb846b80 con 0x55f1eb934c20 2024-05-09T05:47:57.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadc2ffd700 10 client.18135 handle_command_reply: tid=0 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadf8957b80 1 client.18135 shutdown 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadf8957b80 1 -- 172.21.15.115:0/3963155344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb813e00 msgr2=0x55f1eb9222e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadf8957b80 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb813e00 0x55f1eb9222e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fadd800ba90 tx=0x7fadd800bda0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadf8957b80 1 -- 172.21.15.115:0/3963155344 shutdown_connections 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadf8957b80 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb7995a0 0x55f1eb921da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadf8957b80 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f1eb934c20 0x55f1eb937080 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadf8957b80 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb924d10 0x55f1eb922820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadf8957b80 1 --2- 172.21.15.115:0/3963155344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb813e00 0x55f1eb9222e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadc2ffd700 0 client.18135 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.291+0000 7fadf8957b80 1 -- 172.21.15.115:0/3963155344 >> 172.21.15.115:0/3963155344 conn(0x55f1eb79d070 msgr2=0x55f1eb910de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.292+0000 7fadf8957b80 1 -- 172.21.15.115:0/3963155344 shutdown_connections 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.292+0000 7fadf8957b80 1 -- 172.21.15.115:0/3963155344 wait complete. 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.292+0000 7fadf8957b80 20 client.18135 trim_cache size 0 max 16384 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.311+0000 7fadf8957b80 1 Processor -- start 2024-05-09T05:47:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.311+0000 7fadf8957b80 1 -- start start 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb79d070 0x55f1eb911c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb922660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 0x55f1eb917de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fadf8957b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ae140 con 0x55f1eb7995a0 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fadf8957b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fade41afd40 con 0x55f1eb79d070 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fadf8957b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ae810 con 0x55f1eb813e00 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb922660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb922660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51708/0 (socket says 172.21.15.115:51708) 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade2ffd700 1 -- 172.21.15.115:0/1121851031 learned_addr learned my addr 172.21.15.115:0/1121851031 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade37fe700 1 --2- 172.21.15.115:0/1121851031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb79d070 0x55f1eb911c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade3fff700 1 --2- 172.21.15.115:0/1121851031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 0x55f1eb917de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade2ffd700 1 -- 172.21.15.115:0/1121851031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 msgr2=0x55f1eb917de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade2ffd700 1 --2- 172.21.15.115:0/1121851031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 0x55f1eb917de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade2ffd700 1 -- 172.21.15.115:0/1121851031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb79d070 msgr2=0x55f1eb911c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade2ffd700 1 --2- 172.21.15.115:0/1121851031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb79d070 0x55f1eb911c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade2ffd700 1 -- 172.21.15.115:0/1121851031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1eb920310 con 0x55f1eb7995a0 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fade2ffd700 1 --2- 172.21.15.115:0/1121851031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb922660 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fadd8010510 tx=0x7fadd8005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca97ef324f7c98a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fadc97fa700 1 -- 172.21.15.115:0/1121851031 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8003d20 con 0x55f1eb7995a0 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.312+0000 7fadc97fa700 1 -- 172.21.15.115:0/1121851031 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadd80054c0 con 0x55f1eb7995a0 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadc97fa700 1 -- 172.21.15.115:0/1121851031 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8010e30 con 0x55f1eb7995a0 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadf8957b80 1 -- 172.21.15.115:0/1121851031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 msgr2=0x55f1eb922660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadf8957b80 1 --2- 172.21.15.115:0/1121851031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb922660 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fadd8010510 tx=0x7fadd8005950 comp rx=0 tx=0).stop 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadf8957b80 1 -- 172.21.15.115:0/1121851031 shutdown_connections 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadf8957b80 1 --2- 172.21.15.115:0/1121851031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb79d070 0x55f1eb911c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadf8957b80 1 --2- 172.21.15.115:0/1121851031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb813e00 0x55f1eb917de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadf8957b80 1 --2- 172.21.15.115:0/1121851031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb922660 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadf8957b80 1 -- 172.21.15.115:0/1121851031 >> 172.21.15.115:0/1121851031 conn(0x55f1eb90ad50 msgr2=0x55f1eb9176d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadf8957b80 1 -- 172.21.15.115:0/1121851031 shutdown_connections 2024-05-09T05:47:57.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.313+0000 7fadf8957b80 1 -- 172.21.15.115:0/1121851031 wait complete. 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.314+0000 7fadf8957b80 1 -- start start 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.314+0000 7fadf8957b80 1 Processor -- start 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.314+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb92d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.314+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb79d070 0x55f1eb927b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.314+0000 7fadf8957b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb813e00 0x55f1eb9280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.314+0000 7fadf8957b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fade41af670 con 0x55f1eb7995a0 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.314+0000 7fadf8957b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fade41aef10 con 0x55f1eb813e00 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.314+0000 7fadf8957b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ac100 con 0x55f1eb79d070 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.314+0000 7fadf8957b80 10 client.0 _ll_register_callbacks cb 0x55f1eb6aa7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb92d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb92d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51724/0 (socket says 172.21.15.115:51724) 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade37fe700 1 -- 172.21.15.115:0/4103231420 learned_addr learned my addr 172.21.15.115:0/4103231420 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade2ffd700 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb79d070 0x55f1eb927b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade3fff700 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb813e00 0x55f1eb9280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade37fe700 1 -- 172.21.15.115:0/4103231420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb79d070 msgr2=0x55f1eb927b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade37fe700 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb79d070 0x55f1eb927b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade37fe700 1 -- 172.21.15.115:0/4103231420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb813e00 msgr2=0x55f1eb9280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade37fe700 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb813e00 0x55f1eb9280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade37fe700 1 -- 172.21.15.115:0/4103231420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1eb928810 con 0x55f1eb7995a0 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade3fff700 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb813e00 0x55f1eb9280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fade37fe700 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb92d1e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fadd40099c0 tx=0x7fadd4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae7995cea6ea1c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fadc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fadc2ffd700 1 -- 172.21.15.115:0/4103231420 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4039070 con 0x55f1eb7995a0 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fadf8957b80 1 -- 172.21.15.115:0/4103231420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd80054c0 con 0x55f1eb7995a0 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fadc2ffd700 1 -- 172.21.15.115:0/4103231420 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fadd4003d70 con 0x55f1eb7995a0 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fadc2ffd700 1 -- 172.21.15.115:0/4103231420 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fadd4042050 con 0x55f1eb7995a0 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fadf8957b80 10 client.18141 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:57.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.315+0000 7fadf8957b80 1 -- 172.21.15.115:0/4103231420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f1eb9a03b0 con 0x55f1eb7995a0 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.316+0000 7fadc2ffd700 1 -- 172.21.15.115:0/4103231420 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadd80054c0 con 0x55f1eb7995a0 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.316+0000 7fadc2ffd700 1 -- 172.21.15.115:0/4103231420 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7fadd403c020 con 0x55f1eb7995a0 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.316+0000 7fadf8957b80 10 client.18141 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.316+0000 7fadf8957b80 10 client.18141 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.316+0000 7fadf8957b80 20 client.18141 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.316+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f1eb924d10 0x55f1eb9a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.316+0000 7fadf8957b80 4 client.18141 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9919d1d8", "release": true, "await": true}] 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.316+0000 7fadf8957b80 1 -- 172.21.15.115:0/4103231420 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9919d1d8", "release": true, "await": true}) -- 0x55f1eb840c20 con 0x55f1eb924d10 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.316+0000 7fade2ffd700 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f1eb924d10 0x55f1eb9a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.317+0000 7fade2ffd700 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f1eb924d10 0x55f1eb9a2b90 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e21d9c9883dc024 server_cookie=d07dbed19d01c368 in_seq=0 out_seq=0 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.317+0000 7fadc2ffd700 10 client.18141 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:57.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.320+0000 7fadc2ffd700 1 -- 172.21.15.115:0/4103231420 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f1eb840c20 con 0x55f1eb924d10 2024-05-09T05:47:57.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.320+0000 7fadc2ffd700 10 client.18141 handle_command_reply: tid=0 2024-05-09T05:47:57.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.320+0000 7fadf8957b80 1 client.18141 shutdown 2024-05-09T05:47:57.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 -- 172.21.15.115:0/4103231420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 msgr2=0x55f1eb92d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:57.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb92d1e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fadd40099c0 tx=0x7fadd4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 -- 172.21.15.115:0/4103231420 shutdown_connections 2024-05-09T05:47:57.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1eb813e00 0x55f1eb9280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f1eb924d10 0x55f1eb9a2b90 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.503 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:47:57.503 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 48, 2024-05-09T05:47:57.503 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:47:57.504 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 28, 2024-05-09T05:47:57.504 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:47:57.504 INFO:teuthology.orchestra.run.smithi115.stdout: "9919d1d8": { 2024-05-09T05:47:57.504 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 28, 2024-05-09T05:47:57.504 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:47:57.505 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:47:57.505 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:47:57.505 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:47:57.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:47:57.505 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:47:57.505 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:47:57.506 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:47:57.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f1eb7995a0 0x55f1eb92d1e0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 --2- 172.21.15.115:0/4103231420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1eb79d070 0x55f1eb927b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadc2ffd700 0 client.18141 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:57.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 -- 172.21.15.115:0/4103231420 >> 172.21.15.115:0/4103231420 conn(0x55f1eb90ad50 msgr2=0x55f1eb91d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:57.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 -- 172.21.15.115:0/4103231420 shutdown_connections 2024-05-09T05:47:57.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 1 -- 172.21.15.115:0/4103231420 wait complete. 2024-05-09T05:47:57.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fadf8957b80 20 client.18141 trim_cache size 0 max 16384 2024-05-09T05:47:57.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fade9f41700 1 -- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fadcc0b2a70 msgr2=0x7fadcc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:57.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fade9f41700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fadcc0b2a70 0x7fadcc0b4f80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadd4000c00 tx=0x7fadd4005fb0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fade9f41700 1 -- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade4108750 msgr2=0x7fade419f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fade9f41700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade4108750 0x7fade419f580 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7faddc007ee0 tx=0x7faddc00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.321+0000 7fade9f41700 1 -- 172.21.15.115:0/2414655548 shutdown_connections 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.322+0000 7fade9f41700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fadcc0b2a70 0x7fadcc0b4f80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.322+0000 7fade9f41700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fade41034d0 0x7fade419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.322+0000 7fade9f41700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fade4108750 0x7fade419f580 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.322+0000 7fade9f41700 1 --2- 172.21.15.115:0/2414655548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fade4107e00 0x7fade419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.322+0000 7fade9f41700 1 -- 172.21.15.115:0/2414655548 >> 172.21.15.115:0/2414655548 conn(0x7fade40fef20 msgr2=0x7fade406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.322+0000 7fade9f41700 1 -- 172.21.15.115:0/2414655548 shutdown_connections 2024-05-09T05:47:57.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:57.322+0000 7fade9f41700 1 -- 172.21.15.115:0/2414655548 wait complete. 2024-05-09T05:47:57.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:57 smithi195 ceph-mon[33844]: pgmap v303: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-09T05:47:57.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:57 smithi161 ceph-mon[33805]: pgmap v303: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-09T05:47:58.240 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9919d1d8', total seconds elapsed: 38.4 2024-05-09T05:47:58.240 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-09T05:47:58.241 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:47:59.196 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:47:59.394 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:47:59 smithi115 ceph-mon[30349]: pgmap v304: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 958 KiB/s wr, 14 op/s 2024-05-09T05:47:59.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:47:59 smithi195 ceph-mon[33844]: pgmap v304: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 958 KiB/s wr, 14 op/s 2024-05-09T05:47:59.659 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:47:59 smithi161 ceph-mon[33805]: pgmap v304: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 958 KiB/s wr, 14 op/s 2024-05-09T05:47:59.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.804+0000 7f095b2ac700 1 -- 172.21.15.115:0/297060594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09541034e0 msgr2=0x7f09541038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:59.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.804+0000 7f095b2ac700 1 --2- 172.21.15.115:0/297060594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09541034e0 0x7f09541038f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f09500099c0 tx=0x7f0950009cd0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.804+0000 7f095b2ac700 1 -- 172.21.15.115:0/297060594 shutdown_connections 2024-05-09T05:47:59.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.804+0000 7f095b2ac700 1 --2- 172.21.15.115:0/297060594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0954108760 0x7f09541100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.804+0000 7f095b2ac700 1 --2- 172.21.15.115:0/297060594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09541034e0 0x7f09541038f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.804+0000 7f095b2ac700 1 --2- 172.21.15.115:0/297060594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0954107e10 0x7f0954108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.804+0000 7f095b2ac700 1 -- 172.21.15.115:0/297060594 >> 172.21.15.115:0/297060594 conn(0x7f09540fef30 msgr2=0x7f0954101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:59.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.804+0000 7f095b2ac700 1 -- 172.21.15.115:0/297060594 shutdown_connections 2024-05-09T05:47:59.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.804+0000 7f095b2ac700 1 -- 172.21.15.115:0/297060594 wait complete. 2024-05-09T05:47:59.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f095b2ac700 1 Processor -- start 2024-05-09T05:47:59.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f095b2ac700 1 -- start start 2024-05-09T05:47:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f095b2ac700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09541034e0 0x7f095419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f095b2ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0954107e10 0x7f095419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f095b2ac700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0954108760 0x7f095419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f095b2ac700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0954113260 con 0x7f0954107e10 2024-05-09T05:47:59.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f095b2ac700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09541130e0 con 0x7f09541034e0 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f095b2ac700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09541133e0 con 0x7f0954108760 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f0958847700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0954107e10 0x7f095419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f0958847700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0954107e10 0x7f095419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51734/0 (socket says 172.21.15.115:51734) 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f0958847700 1 -- 172.21.15.115:0/563378265 learned_addr learned my addr 172.21.15.115:0/563378265 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f0959048700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09541034e0 0x7f095419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.805+0000 7f0959849700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0954108760 0x7f095419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f0958847700 1 -- 172.21.15.115:0/563378265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0954108760 msgr2=0x7f095419f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f0958847700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0954108760 0x7f095419f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f0958847700 1 -- 172.21.15.115:0/563378265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09541034e0 msgr2=0x7f095419ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f0958847700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09541034e0 0x7f095419ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f0958847700 1 -- 172.21.15.115:0/563378265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095419fc70 con 0x7f0954107e10 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f0958847700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0954107e10 0x7f095419b340 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f094400c970 tx=0x7f094400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f094a7fc700 1 -- 172.21.15.115:0/563378265 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944004a30 con 0x7f0954107e10 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f095b2ac700 1 -- 172.21.15.115:0/563378265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095419ff60 con 0x7f0954107e10 2024-05-09T05:47:59.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f094a7fc700 1 -- 172.21.15.115:0/563378265 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09440054b0 con 0x7f0954107e10 2024-05-09T05:47:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f094a7fc700 1 -- 172.21.15.115:0/563378265 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944007af0 con 0x7f0954107e10 2024-05-09T05:47:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.806+0000 7f095b2ac700 1 -- 172.21.15.115:0/563378265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09540ff340 con 0x7f0954107e10 2024-05-09T05:47:59.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.809+0000 7f094a7fc700 1 -- 172.21.15.115:0/563378265 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0944007cf0 con 0x7f0954107e10 2024-05-09T05:47:59.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.810+0000 7f094a7fc700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f09400b2a70 0x7f09400b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.810+0000 7f0959048700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f09400b2a70 0x7f09400b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.810+0000 7f094a7fc700 1 -- 172.21.15.115:0/563378265 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f09440e23d0 con 0x7f0954107e10 2024-05-09T05:47:59.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.811+0000 7f0959048700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f09400b2a70 0x7f09400b4f80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0950000c00 tx=0x7f0950005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:59.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.811+0000 7f0969cc2b80 1 Processor -- start 2024-05-09T05:47:59.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.811+0000 7f0969cc2b80 1 -- start start 2024-05-09T05:47:59.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.811+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739519d00 0x55b739622240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73949f4a0 0x55b739622780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73961c400 0x55b73961c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b73953f0a0 con 0x55b73949f4a0 2024-05-09T05:47:59.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b7394e49a0 con 0x55b73961c400 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b73953b060 con 0x55b739519d00 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0958847700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73949f4a0 0x55b739622780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0958847700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73949f4a0 0x55b739622780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51738/0 (socket says 172.21.15.115:51738) 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0958847700 1 -- 172.21.15.115:0/2098873756 learned_addr learned my addr 172.21.15.115:0/2098873756 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0959048700 1 --2- 172.21.15.115:0/2098873756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739519d00 0x55b739622240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0959849700 1 --2- 172.21.15.115:0/2098873756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73961c400 0x55b73961c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0958847700 1 -- 172.21.15.115:0/2098873756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739519d00 msgr2=0x55b739622240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0958847700 1 --2- 172.21.15.115:0/2098873756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739519d00 0x55b739622240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0958847700 1 -- 172.21.15.115:0/2098873756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73961c400 msgr2=0x55b73961c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0958847700 1 --2- 172.21.15.115:0/2098873756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73961c400 0x55b73961c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0958847700 1 -- 172.21.15.115:0/2098873756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7394d5500 con 0x55b73949f4a0 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f0958847700 1 --2- 172.21.15.115:0/2098873756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73949f4a0 0x55b739622780 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f09440a9070 tx=0x7f09440a8df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687e908af88e9c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.812+0000 7f093f7fe700 1 -- 172.21.15.115:0/2098873756 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094401d040 con 0x55b73949f4a0 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.813+0000 7f093f7fe700 1 -- 172.21.15.115:0/2098873756 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09440ad720 con 0x55b73949f4a0 2024-05-09T05:47:59.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.813+0000 7f093f7fe700 1 -- 172.21.15.115:0/2098873756 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440a95d0 con 0x55b73949f4a0 2024-05-09T05:47:59.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.813+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2098873756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73949f4a0 msgr2=0x55b739622780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:59.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.813+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/2098873756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73949f4a0 0x55b739622780 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f09440a9070 tx=0x7f09440a8df0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.814+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2098873756 shutdown_connections 2024-05-09T05:47:59.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.814+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/2098873756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b73961c400 0x55b73961c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.814+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/2098873756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739519d00 0x55b739622240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.814+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/2098873756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73949f4a0 0x55b739622780 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.814+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2098873756 >> 172.21.15.115:0/2098873756 conn(0x55b7394a2f70 msgr2=0x55b739617220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.814+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2098873756 shutdown_connections 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.814+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2098873756 wait complete. 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.814+0000 7f0969cc2b80 1 -- start start 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.814+0000 7f0969cc2b80 1 Processor -- start 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739519d00 0x55b739626110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73961c400 0x55b739626650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739626b90 0x55b739632240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b73961d160 con 0x55b73961c400 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b7394d9b20 con 0x55b739519d00 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0958847700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73961c400 0x55b739626650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b739629020 con 0x55b739626b90 2024-05-09T05:47:59.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0958847700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73961c400 0x55b739626650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51750/0 (socket says 172.21.15.115:51750) 2024-05-09T05:47:59.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0958847700 1 -- 172.21.15.115:0/296071243 learned_addr learned my addr 172.21.15.115:0/296071243 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:59.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0969cc2b80 10 client.0 _ll_register_callbacks cb 0x55b7393b06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:59.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0959849700 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739626b90 0x55b739632240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.815+0000 7f0959048700 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739519d00 0x55b739626110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0958847700 1 -- 172.21.15.115:0/296071243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739626b90 msgr2=0x55b739632240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:59.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0958847700 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739626b90 0x55b739632240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0958847700 1 -- 172.21.15.115:0/296071243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739519d00 msgr2=0x55b739626110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:59.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0958847700 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739519d00 0x55b739626110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0958847700 1 -- 172.21.15.115:0/296071243 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7393ba490 con 0x55b73961c400 2024-05-09T05:47:59.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0959849700 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739626b90 0x55b739632240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:59.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0959048700 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739519d00 0x55b739626110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:59.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0958847700 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73961c400 0x55b739626650 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f09440a9020 tx=0x7f0944004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2bb5b3969e02a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:59.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f093cff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:59.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f093cff9700 1 -- 172.21.15.115:0/296071243 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440c9060 con 0x55b73961c400 2024-05-09T05:47:59.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f093cff9700 1 -- 172.21.15.115:0/296071243 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f09440a97e0 con 0x55b73961c400 2024-05-09T05:47:59.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0969cc2b80 1 -- 172.21.15.115:0/296071243 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09440ad720 con 0x55b73961c400 2024-05-09T05:47:59.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f093cff9700 1 -- 172.21.15.115:0/296071243 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f094401d070 con 0x55b73961c400 2024-05-09T05:47:59.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0969cc2b80 10 client.18153 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:59.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f0969cc2b80 1 -- 172.21.15.115:0/296071243 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b739633f90 con 0x55b73961c400 2024-05-09T05:47:59.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.816+0000 7f093cff9700 1 -- 172.21.15.115:0/296071243 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09440ad720 con 0x55b73961c400 2024-05-09T05:47:59.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f093cff9700 1 -- 172.21.15.115:0/296071243 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f09440a85d0 con 0x55b73961c400 2024-05-09T05:47:59.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f0969cc2b80 10 client.18153 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:59.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f0969cc2b80 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:59.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f0969cc2b80 20 client.18153 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:59.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b739636460 0x55b739638880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f0969cc2b80 4 client.18153 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:47:59.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f0969cc2b80 1 -- 172.21.15.115:0/296071243 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b73954ca80 con 0x55b739636460 2024-05-09T05:47:59.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f0959048700 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b739636460 0x55b739638880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f0959048700 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b739636460 0x55b739638880 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739fedfbc3047db1 server_cookie=6fadde42c1000185 in_seq=0 out_seq=0 2024-05-09T05:47:59.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.817+0000 7f093cff9700 10 client.18153 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:59.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f093cff9700 1 -- 172.21.15.115:0/296071243 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b73954ca80 con 0x55b739636460 2024-05-09T05:47:59.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f093cff9700 10 client.18153 handle_command_reply: tid=0 2024-05-09T05:47:59.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f0969cc2b80 1 client.18153 shutdown 2024-05-09T05:47:59.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f0969cc2b80 1 -- 172.21.15.115:0/296071243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73961c400 msgr2=0x55b739626650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:59.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73961c400 0x55b739626650 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f09440a9020 tx=0x7f0944004e00 comp rx=0 tx=0).stop 2024-05-09T05:47:59.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f0969cc2b80 1 -- 172.21.15.115:0/296071243 shutdown_connections 2024-05-09T05:47:59.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739519d00 0x55b739626110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b739636460 0x55b739638880 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b739626b90 0x55b739632240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/296071243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b73961c400 0x55b739626650 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f0969cc2b80 1 -- 172.21.15.115:0/296071243 >> 172.21.15.115:0/296071243 conn(0x55b7394a2f70 msgr2=0x55b739610ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:59.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.818+0000 7f093cff9700 0 client.18153 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:59.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.819+0000 7f0969cc2b80 1 -- 172.21.15.115:0/296071243 shutdown_connections 2024-05-09T05:47:59.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.819+0000 7f0969cc2b80 1 -- 172.21.15.115:0/296071243 wait complete. 2024-05-09T05:47:59.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.819+0000 7f0969cc2b80 20 client.18153 trim_cache size 0 max 16384 2024-05-09T05:47:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0969cc2b80 1 Processor -- start 2024-05-09T05:47:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0969cc2b80 1 -- start start 2024-05-09T05:47:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739625430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739623d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 0x55b739611700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f09541ae240 con 0x55b739519d00 2024-05-09T05:47:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09541afe40 con 0x55b739626b90 2024-05-09T05:47:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09541ae910 con 0x55b73961c400 2024-05-09T05:47:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0959048700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739625430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0959048700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739625430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51762/0 (socket says 172.21.15.115:51762) 2024-05-09T05:47:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0959048700 1 -- 172.21.15.115:0/2966156968 learned_addr learned my addr 172.21.15.115:0/2966156968 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0958847700 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739623d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0959849700 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 0x55b739611700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0959048700 1 -- 172.21.15.115:0/2966156968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 msgr2=0x55b739623d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0959048700 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739623d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0959048700 1 -- 172.21.15.115:0/2966156968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 msgr2=0x55b739611700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0959048700 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 0x55b739611700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.838+0000 7f0959048700 1 -- 172.21.15.115:0/2966156968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b739622af0 con 0x55b739519d00 2024-05-09T05:47:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.839+0000 7f0958847700 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739623d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:47:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.839+0000 7f0959048700 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739625430 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f095000b580 tx=0x7f0950022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca32ee0a828b8e6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.839+0000 7f093effd700 1 -- 172.21.15.115:0/2966156968 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500055c0 con 0x55b739519d00 2024-05-09T05:47:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.839+0000 7f093effd700 1 -- 172.21.15.115:0/2966156968 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0950039030 con 0x55b739519d00 2024-05-09T05:47:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.839+0000 7f093effd700 1 -- 172.21.15.115:0/2966156968 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095003a600 con 0x55b739519d00 2024-05-09T05:47:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2966156968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 msgr2=0x55b739625430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739625430 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f095000b580 tx=0x7f0950022000 comp rx=0 tx=0).stop 2024-05-09T05:47:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2966156968 shutdown_connections 2024-05-09T05:47:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 0x55b739611700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739625430 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/2966156968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739623d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2966156968 >> 172.21.15.115:0/2966156968 conn(0x55b7394a2f70 msgr2=0x55b739610ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2966156968 shutdown_connections 2024-05-09T05:47:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 -- 172.21.15.115:0/2966156968 wait complete. 2024-05-09T05:47:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.840+0000 7f0969cc2b80 1 -- start start 2024-05-09T05:47:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0969cc2b80 1 Processor -- start 2024-05-09T05:47:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739638880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739638dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0969cc2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 0x55b739639300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0969cc2b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f09541af770 con 0x55b739519d00 2024-05-09T05:47:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0959048700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739638880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0959048700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739638880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51778/0 (socket says 172.21.15.115:51778) 2024-05-09T05:47:59.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0959048700 1 -- 172.21.15.115:0/852546094 learned_addr learned my addr 172.21.15.115:0/852546094 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:47:59.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09541af010 con 0x55b739626b90 2024-05-09T05:47:59.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09541ac200 con 0x55b73961c400 2024-05-09T05:47:59.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0969cc2b80 10 client.0 _ll_register_callbacks cb 0x55b739617880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:47:59.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.841+0000 7f0958847700 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739638dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0959849700 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 0x55b739639300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0959048700 1 -- 172.21.15.115:0/852546094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 msgr2=0x55b739638dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:59.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0959048700 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739638dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0959048700 1 -- 172.21.15.115:0/852546094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 msgr2=0x55b739639300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:47:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0959048700 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 0x55b739639300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0959048700 1 -- 172.21.15.115:0/852546094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73969eb60 con 0x55b739519d00 2024-05-09T05:47:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0958847700 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739638dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:47:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0959048700 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739638880 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f09500098e0 tx=0x7f09500055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0845b48c12f4d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:47:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f093cff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:47:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f093cff9700 1 -- 172.21.15.115:0/852546094 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950003880 con 0x55b739519d00 2024-05-09T05:47:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f093cff9700 1 -- 172.21.15.115:0/852546094 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f0950003a20 con 0x55b739519d00 2024-05-09T05:47:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0950039030 con 0x55b739519d00 2024-05-09T05:47:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f093cff9700 1 -- 172.21.15.115:0/852546094 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0950051050 con 0x55b739519d00 2024-05-09T05:47:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0969cc2b80 10 client.18159 fetch_fsmap learned FSMap version 48 2024-05-09T05:47:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.842+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b73969fd80 con 0x55b739519d00 2024-05-09T05:47:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.843+0000 7f093cff9700 1 -- 172.21.15.115:0/852546094 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0950039030 con 0x55b739519d00 2024-05-09T05:47:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.843+0000 7f093cff9700 1 -- 172.21.15.115:0/852546094 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f0950026070 con 0x55b739519d00 2024-05-09T05:47:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.843+0000 7f0969cc2b80 10 client.18159 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:47:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.843+0000 7f0969cc2b80 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T05:47:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.843+0000 7f0969cc2b80 20 client.18159 populate_metadata read hostname 'smithi115' 2024-05-09T05:47:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.843+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b7396a0420 0x55b7396a2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:47:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.843+0000 7f0969cc2b80 4 client.18159 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:47:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.843+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b739546b20 con 0x55b7396a0420 2024-05-09T05:47:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.843+0000 7f0958847700 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b7396a0420 0x55b7396a2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:47:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f093cff9700 10 client.18159 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0958847700 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b7396a0420 0x55b7396a2840 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad51c53771912784 server_cookie=e697d0085759f8fa in_seq=0 out_seq=0 2024-05-09T05:47:59.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f093cff9700 1 -- 172.21.15.115:0/852546094 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b739546b20 con 0x55b7396a0420 2024-05-09T05:47:59.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f093cff9700 10 client.18159 handle_command_reply: tid=0 2024-05-09T05:47:59.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 client.18159 shutdown 2024-05-09T05:47:59.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 msgr2=0x55b739638880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:59.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739638880 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f09500098e0 tx=0x7f09500055c0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 shutdown_connections 2024-05-09T05:47:59.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b739626b90 0x55b739639300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b7396a0420 0x55b7396a2840 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b739519d00 0x55b739638880 secure :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f09500098e0 tx=0x7f09500055c0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 --2- 172.21.15.115:0/852546094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b73961c400 0x55b739638dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f093cff9700 0 client.18159 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:47:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 >> 172.21.15.115:0/852546094 conn(0x55b7394a2f70 msgr2=0x55b739625190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 shutdown_connections 2024-05-09T05:47:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.844+0000 7f0969cc2b80 1 -- 172.21.15.115:0/852546094 wait complete. 2024-05-09T05:47:59.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f0969cc2b80 20 client.18159 trim_cache size 0 max 16384 2024-05-09T05:47:59.850 INFO:teuthology.orchestra.run.smithi115.stdout:[] 2024-05-09T05:47:59.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 -- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f09400b2a70 msgr2=0x7f09400b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:59.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f09400b2a70 0x7f09400b4f80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0950000c00 tx=0x7f0950005fb0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 -- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0954107e10 msgr2=0x7f095419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:47:59.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0954107e10 0x7f095419b340 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f094400c970 tx=0x7f094400cc80 comp rx=0 tx=0).stop 2024-05-09T05:47:59.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 -- 172.21.15.115:0/563378265 shutdown_connections 2024-05-09T05:47:59.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f09400b2a70 0x7f09400b4f80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09541034e0 0x7f095419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0954108760 0x7f095419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 --2- 172.21.15.115:0/563378265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0954107e10 0x7f095419b340 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:47:59.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.845+0000 7f095b2ac700 1 -- 172.21.15.115:0/563378265 >> 172.21.15.115:0/563378265 conn(0x7f09540fef30 msgr2=0x7f09541096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:47:59.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.846+0000 7f095b2ac700 1 -- 172.21.15.115:0/563378265 shutdown_connections 2024-05-09T05:47:59.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:47:59.846+0000 7f095b2ac700 1 -- 172.21.15.115:0/563378265 wait complete. 2024-05-09T05:48:01.395 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:01 smithi115 ceph-mon[30349]: pgmap v305: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 955 KiB/s wr, 8 op/s 2024-05-09T05:48:01.615 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:01 smithi195 ceph-mon[33844]: pgmap v305: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 955 KiB/s wr, 8 op/s 2024-05-09T05:48:01.621 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 191972113, 'ino': 2199023256055, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fh.Po'}, {'damage_type': 'backtrace', 'id': 225548017, 'ino': 2199023256069, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/list.Po'}, {'damage_type': 'backtrace', 'id': 307945780, 'ino': 1099511628464, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.git/index'}, {'damage_type': 'backtrace', 'id': 308887306, 'ino': 2199023256065, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/util.Po'}, {'damage_type': 'backtrace', 'id': 374827588, 'ino': 1099511628764, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/requests'}, {'damage_type': 'backtrace', 'id': 410162317, 'ino': 2199023256068, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_stats.Po'}, {'damage_type': 'backtrace', 'id': 684929066, 'ino': 2199023256066, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/parser.Po'}, {'damage_type': 'backtrace', 'id': 875733192, 'ino': 2199023256061, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_tg.Po'}, {'damage_type': 'backtrace', 'id': 884169366, 'ino': 2199023256057, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/metaops.Po'}, {'damage_type': 'backtrace', 'id': 2068608072, 'ino': 2199023256058, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rwlock.Po'}, {'damage_type': 'backtrace', 'id': 2465910697, 'ino': 2199023255552, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/fileops.Po'}, {'damage_type': 'backtrace', 'id': 2655745164, 'ino': 2199023256060, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rbt.Po'}, {'damage_type': 'backtrace', 'id': 2707392242, 'ino': 2199023255553, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/rand.Po'}, {'damage_type': 'backtrace', 'id': 3212203721, 'ino': 2199023256067, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fc.Po'}, {'damage_type': 'backtrace', 'id': 3219493471, 'ino': 2199023256063, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_thread.Po'}, {'damage_type': 'backtrace', 'id': 3238711390, 'ino': 2199023256056, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/filelist.Po'}, {'damage_type': 'backtrace', 'id': 3391654524, 'ino': 2199023256054, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/main.Po'}, {'damage_type': 'backtrace', 'id': 3526424197, 'ino': 2199023256062, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_fs.Po'}, {'damage_type': 'backtrace', 'id': 3846390614, 'ino': 2199023256064, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/ffsb_op.Po'}, {'damage_type': 'backtrace', 'id': 3902455919, 'ino': 1099511628765, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/output.0'}, {'damage_type': 'backtrace', 'id': 4161297249, 'ino': 2199023256059, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps/cirlist.Po'}, {'damage_type': 'backtrace', 'id': 4277993129, 'ino': 1099511628766, 'path': '/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/autom4te.cache/traces.0'}] 2024-05-09T05:48:01.621 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-05-09T05:48:01.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:01 smithi161 ceph-mon[33805]: pgmap v305: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 955 KiB/s wr, 8 op/s 2024-05-09T05:48:02.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.153+0000 7f7882d4b700 1 -- 172.21.15.115:0/1873937519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 msgr2=0x7f787c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:02.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.153+0000 7f7882d4b700 1 --2- 172.21.15.115:0/1873937519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 0x7f787c105890 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7870009990 tx=0x7f7870009ca0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.154+0000 7f7882d4b700 1 -- 172.21.15.115:0/1873937519 shutdown_connections 2024-05-09T05:48:02.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.154+0000 7f7882d4b700 1 --2- 172.21.15.115:0/1873937519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f787c10e8f0 0x7f787c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.154+0000 7f7882d4b700 1 --2- 172.21.15.115:0/1873937519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 0x7f787c105890 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.154+0000 7f7882d4b700 1 --2- 172.21.15.115:0/1873937519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f787c105dd0 0x7f787c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.154+0000 7f7882d4b700 1 -- 172.21.15.115:0/1873937519 >> 172.21.15.115:0/1873937519 conn(0x7f787c0fef30 msgr2=0x7f787c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:02.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.154+0000 7f7882d4b700 1 -- 172.21.15.115:0/1873937519 shutdown_connections 2024-05-09T05:48:02.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.154+0000 7f7882d4b700 1 -- 172.21.15.115:0/1873937519 wait complete. 2024-05-09T05:48:02.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.154+0000 7f7882d4b700 1 Processor -- start 2024-05-09T05:48:02.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.154+0000 7f7882d4b700 1 -- start start 2024-05-09T05:48:02.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7882d4b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 0x7f787c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:02.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7882d4b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f787c105dd0 0x7f787c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:02.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7882d4b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f787c10e8f0 0x7f787c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:02.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7882d4b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f787c114360 con 0x7f787c105480 2024-05-09T05:48:02.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7882d4b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f787c1141e0 con 0x7f787c105dd0 2024-05-09T05:48:02.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7882d4b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f787c1144e0 con 0x7f787c10e8f0 2024-05-09T05:48:02.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7880ae7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 0x7f787c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:02.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7880ae7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 0x7f787c19ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51800/0 (socket says 172.21.15.115:51800) 2024-05-09T05:48:02.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7880ae7700 1 -- 172.21.15.115:0/2660478171 learned_addr learned my addr 172.21.15.115:0/2660478171 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f787bfff700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f787c105dd0 0x7f787c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f78812e8700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f787c10e8f0 0x7f787c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7880ae7700 1 -- 172.21.15.115:0/2660478171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f787c10e8f0 msgr2=0x7f787c19f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7880ae7700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f787c10e8f0 0x7f787c19f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7880ae7700 1 -- 172.21.15.115:0/2660478171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f787c105dd0 msgr2=0x7f787c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7880ae7700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f787c105dd0 0x7f787c19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.155+0000 7f7880ae7700 1 -- 172.21.15.115:0/2660478171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c19fc40 con 0x7f787c105480 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.156+0000 7f7880ae7700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 0x7f787c19ad70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7870005910 tx=0x7f78700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.156+0000 7f7879ffb700 1 -- 172.21.15.115:0/2660478171 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870027070 con 0x7f787c105480 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.156+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c19fed0 con 0x7f787c105480 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.156+0000 7f7879ffb700 1 -- 172.21.15.115:0/2660478171 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f787000fb00 con 0x7f787c105480 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.156+0000 7f7879ffb700 1 -- 172.21.15.115:0/2660478171 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870022b90 con 0x7f787c105480 2024-05-09T05:48:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.156+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c1a7850 con 0x7f787c105480 2024-05-09T05:48:02.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.157+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787c0512c0 con 0x7f787c105480 2024-05-09T05:48:02.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.160+0000 7f7879ffb700 1 -- 172.21.15.115:0/2660478171 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f787002a070 con 0x7f787c105480 2024-05-09T05:48:02.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.161+0000 7f7879ffb700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f786c0b29f0 0x7f786c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:02.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.161+0000 7f787bfff700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f786c0b29f0 0x7f786c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:02.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.161+0000 7f7879ffb700 1 -- 172.21.15.115:0/2660478171 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f78700e3f10 con 0x7f787c105480 2024-05-09T05:48:02.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.162+0000 7f787bfff700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f786c0b29f0 0x7f786c0b4f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f787c1023e0 tx=0x7f78680073d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:02.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.167+0000 7f7879ffb700 1 -- 172.21.15.115:0/2660478171 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7870031020 con 0x7f787c105480 2024-05-09T05:48:02.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.453+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f787c19c0d0 con 0x7f787c105480 2024-05-09T05:48:02.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.454+0000 7f7879ffb700 1 -- 172.21.15.115:0/2660478171 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11957 (secure 0 0 0) 0x7f7870013070 con 0x7f787c105480 2024-05-09T05:48:02.446 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:48:02.446 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:47:38.366444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27316":{"gid":27316,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/391681574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":391681574},{"type":"v1","addr":"172.21.15.195:6833","nonce":391681574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:48:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f786c0b29f0 msgr2=0x7f786c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f786c0b29f0 0x7f786c0b4f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f787c1023e0 tx=0x7f78680073d0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 msgr2=0x7f787c19ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 0x7f787c19ad70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7870005910 tx=0x7f78700047b0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 shutdown_connections 2024-05-09T05:48:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f786c0b29f0 0x7f786c0b4f00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f787c10e8f0 0x7f787c19f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f787c105480 0x7f787c19ad70 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 --2- 172.21.15.115:0/2660478171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f787c105dd0 0x7f787c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:02.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 >> 172.21.15.115:0/2660478171 conn(0x7f787c0fef30 msgr2=0x7f787c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:02.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 shutdown_connections 2024-05-09T05:48:02.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:02.457+0000 7f7882d4b700 1 -- 172.21.15.115:0/2660478171 wait complete. 2024-05-09T05:48:02.450 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 48 2024-05-09T05:48:03.223 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-09T05:48:03.223 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-09T05:48:03.491 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:48:03.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:03 smithi195 ceph-mon[33844]: pgmap v306: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 955 KiB/s wr, 8 op/s 2024-05-09T05:48:03.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:03 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2660478171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:48:03.616 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:03 smithi115 ceph-mon[30349]: pgmap v306: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 955 KiB/s wr, 8 op/s 2024-05-09T05:48:03.616 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:03 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2660478171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:48:03.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:03 smithi161 ceph-mon[33805]: pgmap v306: 129 pgs: 129 active+clean; 8.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 955 KiB/s wr, 8 op/s 2024-05-09T05:48:03.662 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:03 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2660478171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:48:05.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.445+0000 7f4250384700 1 -- 172.21.15.115:0/2354428771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 msgr2=0x7f42481100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.445+0000 7f4250384700 1 --2- 172.21.15.115:0/2354428771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 0x7f42481100a0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f424000b0d0 tx=0x7f424000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.445+0000 7f4250384700 1 -- 172.21.15.115:0/2354428771 shutdown_connections 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.445+0000 7f4250384700 1 --2- 172.21.15.115:0/2354428771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42481034c0 0x7f42481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.445+0000 7f4250384700 1 --2- 172.21.15.115:0/2354428771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 0x7f42481100a0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.445+0000 7f4250384700 1 --2- 172.21.15.115:0/2354428771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4248107df0 0x7f4248108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.445+0000 7f4250384700 1 -- 172.21.15.115:0/2354428771 >> 172.21.15.115:0/2354428771 conn(0x7f42480fedc0 msgr2=0x7f42481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.445+0000 7f4250384700 1 -- 172.21.15.115:0/2354428771 shutdown_connections 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.445+0000 7f4250384700 1 -- 172.21.15.115:0/2354428771 wait complete. 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.446+0000 7f4250384700 1 Processor -- start 2024-05-09T05:48:05.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.446+0000 7f4250384700 1 -- start start 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.446+0000 7f4250384700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42481034c0 0x7f424819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.446+0000 7f4250384700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248107df0 0x7f424819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.446+0000 7f4250384700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 0x7f424819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f4250384700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4248113220 con 0x7f4248108740 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f4250384700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42481130a0 con 0x7f4248107df0 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f4250384700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42481133a0 con 0x7f42481034c0 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e921700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 0x7f424819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e921700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 0x7f424819c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38996/0 (socket says 172.21.15.115:38996) 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e921700 1 -- 172.21.15.115:0/3785221836 learned_addr learned my addr 172.21.15.115:0/3785221836 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:05.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e120700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42481034c0 0x7f424819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424d91f700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248107df0 0x7f424819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e921700 1 -- 172.21.15.115:0/3785221836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42481034c0 msgr2=0x7f424819bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e921700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42481034c0 0x7f424819bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e921700 1 -- 172.21.15.115:0/3785221836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248107df0 msgr2=0x7f424819c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e921700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248107df0 0x7f424819c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e921700 1 -- 172.21.15.115:0/3785221836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42481a6e70 con 0x7f4248108740 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f424e921700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 0x7f424819c5c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4240009850 tx=0x7f4240007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f423f7fe700 1 -- 172.21.15.115:0/3785221836 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424001f070 con 0x7f4248108740 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f4250384700 1 -- 172.21.15.115:0/3785221836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42481a7100 con 0x7f4248108740 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.447+0000 7f423f7fe700 1 -- 172.21.15.115:0/3785221836 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4240004c50 con 0x7f4248108740 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.448+0000 7f4250384700 1 -- 172.21.15.115:0/3785221836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42481a7560 con 0x7f4248108740 2024-05-09T05:48:05.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.448+0000 7f423f7fe700 1 -- 172.21.15.115:0/3785221836 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240024450 con 0x7f4248108740 2024-05-09T05:48:05.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.450+0000 7f423f7fe700 1 -- 172.21.15.115:0/3785221836 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4240022070 con 0x7f4248108740 2024-05-09T05:48:05.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.451+0000 7f423f7fe700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42340b29f0 0x7f42340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.452+0000 7f424e120700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42340b29f0 0x7f42340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.452+0000 7f423f7fe700 1 -- 172.21.15.115:0/3785221836 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f42400e3820 con 0x7f4248108740 2024-05-09T05:48:05.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.452+0000 7f425ed9ab80 1 Processor -- start 2024-05-09T05:48:05.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.452+0000 7f424e120700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42340b29f0 0x7f42340b4f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4244005950 tx=0x7f424400b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:05.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.452+0000 7f425ed9ab80 1 -- start start 2024-05-09T05:48:05.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2ac5e90 0x558ee2bc8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bc9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 0x558ee2bd70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558ee2aeb230 con 0x558ee2a4b630 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558ee2a90b30 con 0x558ee2ac5e90 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558ee2ae71f0 con 0x558ee2bd6cb0 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424d91f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bc9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424d91f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bc9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39002/0 (socket says 172.21.15.115:39002) 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424d91f700 1 -- 172.21.15.115:0/764760046 learned_addr learned my addr 172.21.15.115:0/764760046 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424e921700 1 --2- 172.21.15.115:0/764760046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 0x558ee2bd70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424e120700 1 --2- 172.21.15.115:0/764760046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2ac5e90 0x558ee2bc8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424d91f700 1 -- 172.21.15.115:0/764760046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 msgr2=0x558ee2bd70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424d91f700 1 --2- 172.21.15.115:0/764760046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 0x558ee2bd70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424d91f700 1 -- 172.21.15.115:0/764760046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2ac5e90 msgr2=0x558ee2bc8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424d91f700 1 --2- 172.21.15.115:0/764760046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2ac5e90 0x558ee2bc8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424d91f700 1 -- 172.21.15.115:0/764760046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee2a81690 con 0x558ee2a4b630 2024-05-09T05:48:05.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f424d91f700 1 --2- 172.21.15.115:0/764760046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bc9040 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4238009cb0 tx=0x7f4238011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6264adadf79a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:05.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.453+0000 7f423cd47700 1 -- 172.21.15.115:0/764760046 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238011d20 con 0x558ee2a4b630 2024-05-09T05:48:05.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.454+0000 7f423cd47700 1 -- 172.21.15.115:0/764760046 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42380044e0 con 0x558ee2a4b630 2024-05-09T05:48:05.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.454+0000 7f423cd47700 1 -- 172.21.15.115:0/764760046 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238005280 con 0x558ee2a4b630 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.454+0000 7f425ed9ab80 1 -- 172.21.15.115:0/764760046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 msgr2=0x558ee2bc9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.454+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/764760046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bc9040 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4238009cb0 tx=0x7f4238011440 comp rx=0 tx=0).stop 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.454+0000 7f425ed9ab80 1 -- 172.21.15.115:0/764760046 shutdown_connections 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.454+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/764760046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2ac5e90 0x558ee2bc8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.455+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/764760046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 0x558ee2bd70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.455+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/764760046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bc9040 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.455+0000 7f425ed9ab80 1 -- 172.21.15.115:0/764760046 >> 172.21.15.115:0/764760046 conn(0x558ee2a4f100 msgr2=0x558ee2bc3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.455+0000 7f425ed9ab80 1 -- 172.21.15.115:0/764760046 shutdown_connections 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.455+0000 7f425ed9ab80 1 -- 172.21.15.115:0/764760046 wait complete. 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.455+0000 7f425ed9ab80 1 -- start start 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.455+0000 7f425ed9ab80 1 Processor -- start 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2a4b630 0x558ee2bdadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2bdb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 0x558ee2bdb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558ee2bd7a80 con 0x558ee2ac5e90 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558ee2a85cb0 con 0x558ee2a4b630 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558ee2bdde10 con 0x558ee2bd6cb0 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f425ed9ab80 10 client.0 _ll_register_callbacks cb 0x558ee295c850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424d91f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2bdb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424d91f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2bdb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39010/0 (socket says 172.21.15.115:39010) 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424d91f700 1 -- 172.21.15.115:0/963163154 learned_addr learned my addr 172.21.15.115:0/963163154 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424e120700 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2a4b630 0x558ee2bdadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424e921700 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 0x558ee2bdb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424d91f700 1 -- 172.21.15.115:0/963163154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 msgr2=0x558ee2bdb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424d91f700 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 0x558ee2bdb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424d91f700 1 -- 172.21.15.115:0/963163154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2a4b630 msgr2=0x558ee2bdadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424d91f700 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2a4b630 0x558ee2bdadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424d91f700 1 -- 172.21.15.115:0/963163154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee2966620 con 0x558ee2ac5e90 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424e120700 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2a4b630 0x558ee2bdadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f424d91f700 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2bdb310 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f4238009c80 tx=0x7f4238004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569d09c965685030 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.456+0000 7f4231ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f4231ffb700 1 -- 172.21.15.115:0/963163154 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238026020 con 0x558ee2ac5e90 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f4231ffb700 1 -- 172.21.15.115:0/963163154 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f4238011b20 con 0x558ee2ac5e90 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f425ed9ab80 1 -- 172.21.15.115:0/963163154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42380044e0 con 0x558ee2ac5e90 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f4231ffb700 1 -- 172.21.15.115:0/963163154 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4238011d60 con 0x558ee2ac5e90 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f425ed9ab80 10 client.18177 fetch_fsmap learned FSMap version 48 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f425ed9ab80 1 -- 172.21.15.115:0/963163154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558ee2be87f0 con 0x558ee2ac5e90 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f4231ffb700 1 -- 172.21.15.115:0/963163154 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42380044e0 con 0x558ee2ac5e90 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f4231ffb700 1 -- 172.21.15.115:0/963163154 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f4238018070 con 0x558ee2ac5e90 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f425ed9ab80 10 client.18177 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:48:05.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f425ed9ab80 10 client.18177 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.457+0000 7f425ed9ab80 20 client.18177 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.458+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ee2bead10 0x558ee2bed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.458+0000 7f425ed9ab80 4 client.18177 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.458+0000 7f425ed9ab80 1 -- 172.21.15.115:0/963163154 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ee2af8c10 con 0x558ee2bead10 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.458+0000 7f424e120700 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ee2bead10 0x558ee2bed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.458+0000 7f424e120700 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ee2bead10 0x558ee2bed1c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8876d5c4c1bfafd2 server_cookie=ae5bade5d07fb686 in_seq=0 out_seq=0 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.458+0000 7f4231ffb700 10 client.18177 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f4231ffb700 1 -- 172.21.15.115:0/963163154 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ee2af8c10 con 0x558ee2bead10 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f4231ffb700 10 client.18177 handle_command_reply: tid=0 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 client.18177 shutdown 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 -- 172.21.15.115:0/963163154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 msgr2=0x558ee2bdb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2bdb310 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f4238009c80 tx=0x7f4238004cc0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 -- 172.21.15.115:0/963163154 shutdown_connections 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2a4b630 0x558ee2bdadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ee2bead10 0x558ee2bed1c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2bd6cb0 0x558ee2bdb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/963163154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2bdb310 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 -- 172.21.15.115:0/963163154 >> 172.21.15.115:0/963163154 conn(0x558ee2a4f100 msgr2=0x558ee2bc2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:05.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f4231ffb700 0 client.18177 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:05.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 -- 172.21.15.115:0/963163154 shutdown_connections 2024-05-09T05:48:05.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.459+0000 7f425ed9ab80 1 -- 172.21.15.115:0/963163154 wait complete. 2024-05-09T05:48:05.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.460+0000 7f425ed9ab80 20 client.18177 trim_cache size 0 max 16384 2024-05-09T05:48:05.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.489+0000 7f425ed9ab80 1 Processor -- start 2024-05-09T05:48:05.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.489+0000 7f425ed9ab80 1 -- start start 2024-05-09T05:48:05.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.489+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2a4b630 0x558ee2bdd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.489+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2be6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.489+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 0x558ee2bdb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.482 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.489+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42481ae0c0 con 0x558ee2ac5e90 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.489+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42481afcc0 con 0x558ee2bd6cb0 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.489+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42481ae790 con 0x558ee2a4b630 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424d91f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2be6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424d91f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2be6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39012/0 (socket says 172.21.15.115:39012) 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424d91f700 1 -- 172.21.15.115:0/3741481613 learned_addr learned my addr 172.21.15.115:0/3741481613 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424e120700 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2a4b630 0x558ee2bdd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424e921700 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 0x558ee2bdb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424d91f700 1 -- 172.21.15.115:0/3741481613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2a4b630 msgr2=0x558ee2bdd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424d91f700 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2a4b630 0x558ee2bdd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424d91f700 1 -- 172.21.15.115:0/3741481613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 msgr2=0x558ee2bdb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424d91f700 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 0x558ee2bdb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424d91f700 1 -- 172.21.15.115:0/3741481613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee2bbcfb0 con 0x558ee2ac5e90 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424e921700 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 0x558ee2bdb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f424d91f700 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2be6ce0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f4238000c00 tx=0x7f4238004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c2ef8280ebc9d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f4233fff700 1 -- 172.21.15.115:0/3741481613 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238026030 con 0x558ee2ac5e90 2024-05-09T05:48:05.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f4233fff700 1 -- 172.21.15.115:0/3741481613 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4238011b20 con 0x558ee2ac5e90 2024-05-09T05:48:05.484 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.490+0000 7f4233fff700 1 -- 172.21.15.115:0/3741481613 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238031030 con 0x558ee2ac5e90 2024-05-09T05:48:05.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.491+0000 7f425ed9ab80 1 -- 172.21.15.115:0/3741481613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 msgr2=0x558ee2be6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.491+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2be6ce0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f4238000c00 tx=0x7f4238004af0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.491+0000 7f425ed9ab80 1 -- 172.21.15.115:0/3741481613 shutdown_connections 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.491+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 0x558ee2bdb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.491+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2a4b630 0x558ee2bdd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.491+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/3741481613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2ac5e90 0x558ee2be6ce0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.491+0000 7f425ed9ab80 1 -- 172.21.15.115:0/3741481613 >> 172.21.15.115:0/3741481613 conn(0x558ee2a4f100 msgr2=0x558ee2bc2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 -- 172.21.15.115:0/3741481613 shutdown_connections 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 -- 172.21.15.115:0/3741481613 wait complete. 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 -- start start 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 Processor -- start 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bf1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2ac5e90 0x558ee2bf2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 0x558ee2bf29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42481af5f0 con 0x558ee2a4b630 2024-05-09T05:48:05.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42481aee90 con 0x558ee2bd6cb0 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.492+0000 7f425ed9ab80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f42481ac080 con 0x558ee2ac5e90 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f425ed9ab80 10 client.0 _ll_register_callbacks cb 0x558ee295c850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e120700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bf1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e120700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bf1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39026/0 (socket says 172.21.15.115:39026) 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e120700 1 -- 172.21.15.115:0/1407109307 learned_addr learned my addr 172.21.15.115:0/1407109307 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424d91f700 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2ac5e90 0x558ee2bf2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e921700 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 0x558ee2bf29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e120700 1 -- 172.21.15.115:0/1407109307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2ac5e90 msgr2=0x558ee2bf2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e120700 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2ac5e90 0x558ee2bf2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e120700 1 -- 172.21.15.115:0/1407109307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 msgr2=0x558ee2bf29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e120700 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 0x558ee2bf29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e120700 1 -- 172.21.15.115:0/1407109307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee2be12e0 con 0x558ee2a4b630 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f424e120700 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bf1f40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4244005d90 tx=0x7f42440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e4c6f39df26dab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:05.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f4231ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f4231ffb700 1 -- 172.21.15.115:0/1407109307 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244021db0 con 0x558ee2a4b630 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f4231ffb700 1 -- 172.21.15.115:0/1407109307 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7f424402a380 con 0x558ee2a4b630 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.493+0000 7f425ed9ab80 1 -- 172.21.15.115:0/1407109307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4238011b20 con 0x558ee2a4b630 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f4231ffb700 1 -- 172.21.15.115:0/1407109307 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f424403a050 con 0x558ee2a4b630 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f425ed9ab80 10 client.18183 fetch_fsmap learned FSMap version 48 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f425ed9ab80 1 -- 172.21.15.115:0/1407109307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558ee2c554d0 con 0x558ee2a4b630 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f4231ffb700 1 -- 172.21.15.115:0/1407109307 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4238011b20 con 0x558ee2a4b630 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f4231ffb700 1 -- 172.21.15.115:0/1407109307 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) ==== 8378+0+0 (secure 0 0 0) 0x7f4244011860 con 0x558ee2a4b630 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f425ed9ab80 10 client.18183 fetch_fsmap finished waiting for FSMap version 48 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f425ed9ab80 10 client.18183 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f425ed9ab80 20 client.18183 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ee2c55d40 0x558ee2c58160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f425ed9ab80 4 client.18183 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-09T05:48:05.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f425ed9ab80 1 -- 172.21.15.115:0/1407109307 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x558ee2af2cb0 con 0x558ee2c55d40 2024-05-09T05:48:05.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.494+0000 7f424d91f700 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ee2c55d40 0x558ee2c58160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:05.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.495+0000 7f4231ffb700 10 client.18183 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:05.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.495+0000 7f424d91f700 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ee2c55d40 0x558ee2c58160 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e67d7052d24117 server_cookie=3a94984e78e4f194 in_seq=0 out_seq=0 2024-05-09T05:48:05.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f4231ffb700 1 -- 172.21.15.115:0/1407109307 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x558ee2af2cb0 con 0x558ee2c55d40 2024-05-09T05:48:05.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f4231ffb700 10 client.18183 handle_command_reply: tid=0 2024-05-09T05:48:05.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f425ed9ab80 1 client.18183 shutdown 2024-05-09T05:48:05.584 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:05 smithi115 ceph-mon[30349]: pgmap v307: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 34 op/s 2024-05-09T05:48:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f425ed9ab80 1 -- 172.21.15.115:0/1407109307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 msgr2=0x558ee2bf1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bf1f40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4244005d90 tx=0x7f42440047c0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f425ed9ab80 1 -- 172.21.15.115:0/1407109307 shutdown_connections 2024-05-09T05:48:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee2bd6cb0 0x558ee2bf29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ee2c55d40 0x558ee2c58160 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f4231ffb700 0 client.18183 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ee2a4b630 0x558ee2bf1f40 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f425ed9ab80 1 --2- 172.21.15.115:0/1407109307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ee2ac5e90 0x558ee2bf2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.592+0000 7f425ed9ab80 1 -- 172.21.15.115:0/1407109307 >> 172.21.15.115:0/1407109307 conn(0x558ee2a4f100 msgr2=0x558ee2bc38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f425ed9ab80 1 -- 172.21.15.115:0/1407109307 shutdown_connections 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f425ed9ab80 1 -- 172.21.15.115:0/1407109307 wait complete. 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f425ed9ab80 20 client.18183 trim_cache size 0 max 16384 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 48, 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 30, 2024-05-09T05:48:05.586 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "6f88124": { 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 30, 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:48:05.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:05.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:05.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:05.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:05.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:05.588 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:48:05.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 -- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42340b29f0 msgr2=0x7f42340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:05.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42340b29f0 0x7f42340b4f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4244005950 tx=0x7f424400b410 comp rx=0 tx=0).stop 2024-05-09T05:48:05.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 -- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 msgr2=0x7f424819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:05.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 0x7f424819c5c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4240009850 tx=0x7f4240007a20 comp rx=0 tx=0).stop 2024-05-09T05:48:05.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 -- 172.21.15.115:0/3785221836 shutdown_connections 2024-05-09T05:48:05.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f42340b29f0 0x7f42340b4f00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4248108740 0x7f424819c5c0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42481034c0 0x7f424819bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 --2- 172.21.15.115:0/3785221836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4248107df0 0x7f424819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:05.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.593+0000 7f4250384700 1 -- 172.21.15.115:0/3785221836 >> 172.21.15.115:0/3785221836 conn(0x7f42480fedc0 msgr2=0x7f4248107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:05.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.594+0000 7f4250384700 1 -- 172.21.15.115:0/3785221836 shutdown_connections 2024-05-09T05:48:05.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:05.594+0000 7f4250384700 1 -- 172.21.15.115:0/3785221836 wait complete. 2024-05-09T05:48:05.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:05 smithi195 ceph-mon[33844]: pgmap v307: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 34 op/s 2024-05-09T05:48:05.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:05 smithi161 ceph-mon[33805]: pgmap v307: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 34 op/s 2024-05-09T05:48:06.431 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6f88124', quiesce took 8.2 seconds. Will release after: 21.8 2024-05-09T05:48:07.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:07 smithi195 ceph-mon[33844]: pgmap v308: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 30 op/s 2024-05-09T05:48:07.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:07 smithi115 ceph-mon[30349]: pgmap v308: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 30 op/s 2024-05-09T05:48:07.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:07 smithi161 ceph-mon[33805]: pgmap v308: 129 pgs: 129 active+clean; 8.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 30 op/s 2024-05-09T05:48:09.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:09 smithi195 ceph-mon[33844]: pgmap v309: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 38 op/s 2024-05-09T05:48:09.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:09 smithi115 ceph-mon[30349]: pgmap v309: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 38 op/s 2024-05-09T05:48:09.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:09 smithi161 ceph-mon[33805]: pgmap v309: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 38 op/s 2024-05-09T05:48:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.998+0000 7fd90a0d1640 1 -- 172.21.15.115:0/305228872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 msgr2=0x7fd904101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.998+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/305228872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 0x7fd904101ca0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009ca0 tx=0x7fd8f402a8c0 comp rx=0 tx=0).stop 2024-05-09T05:48:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.998+0000 7fd90a0d1640 1 -- 172.21.15.115:0/305228872 shutdown_connections 2024-05-09T05:48:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.998+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/305228872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd90410e6e0 0x7fd904110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.998+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/305228872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9041021e0 0x7fd9041025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.998+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/305228872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 0x7fd904101ca0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.998+0000 7fd90a0d1640 1 -- 172.21.15.115:0/305228872 >> 172.21.15.115:0/305228872 conn(0x7fd9040fd9d0 msgr2=0x7fd9040ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.998+0000 7fd90a0d1640 1 -- 172.21.15.115:0/305228872 shutdown_connections 2024-05-09T05:48:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.998+0000 7fd90a0d1640 1 -- 172.21.15.115:0/305228872 wait complete. 2024-05-09T05:48:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd90a0d1640 1 Processor -- start 2024-05-09T05:48:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd90a0d1640 1 -- start start 2024-05-09T05:48:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd90a0d1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9041021e0 0x7fd904102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd90a0d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 0x7fd9041033c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd90a0d1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd90410e6e0 0x7fd904103900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd90a0d1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd904113d80 con 0x7fd904109d30 2024-05-09T05:48:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd90a0d1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd904113c00 con 0x7fd9041021e0 2024-05-09T05:48:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd90a0d1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd904113f00 con 0x7fd90410e6e0 2024-05-09T05:48:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd902ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 0x7fd9041033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd902ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 0x7fd9041033c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39054/0 (socket says 172.21.15.115:39054) 2024-05-09T05:48:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd902ffd640 1 -- 172.21.15.115:0/3419464082 learned_addr learned my addr 172.21.15.115:0/3419464082 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd903fff640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd90410e6e0 0x7fd904103900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:09.999+0000 7fd9037fe640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9041021e0 0x7fd904102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd902ffd640 1 -- 172.21.15.115:0/3419464082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd90410e6e0 msgr2=0x7fd904103900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd902ffd640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd90410e6e0 0x7fd904103900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd902ffd640 1 -- 172.21.15.115:0/3419464082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9041021e0 msgr2=0x7fd904102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:09.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd902ffd640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9041021e0 0x7fd904102e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd902ffd640 1 -- 172.21.15.115:0/3419464082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9041a8b40 con 0x7fd904109d30 2024-05-09T05:48:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd9037fe640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9041021e0 0x7fd904102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:48:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd902ffd640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 0x7fd9041033c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800d770 tx=0x7fd8f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd900ff9640 1 -- 172.21.15.115:0/3419464082 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8017070 con 0x7fd904109d30 2024-05-09T05:48:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd900ff9640 1 -- 172.21.15.115:0/3419464082 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8f8009c30 con 0x7fd904109d30 2024-05-09T05:48:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd90a0d1640 1 -- 172.21.15.115:0/3419464082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9041a8e30 con 0x7fd904109d30 2024-05-09T05:48:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd900ff9640 1 -- 172.21.15.115:0/3419464082 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800f620 con 0x7fd904109d30 2024-05-09T05:48:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.000+0000 7fd90a0d1640 1 -- 172.21.15.115:0/3419464082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9041a92a0 con 0x7fd904109d30 2024-05-09T05:48:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd910f86740 1 Processor -- start 2024-05-09T05:48:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd910f86740 1 -- start start 2024-05-09T05:48:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd900ff9640 1 -- 172.21.15.115:0/3419464082 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fd8f801a070 con 0x7fd904109d30 2024-05-09T05:48:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd910f86740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x562946134680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd910f86740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 0x562946134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd910f86740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946135100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd910f86740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629460997c0 con 0x562945d911b0 2024-05-09T05:48:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd910f86740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5629460350e0 con 0x562945d704d0 2024-05-09T05:48:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd910f86740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562946095780 con 0x562945d682f0 2024-05-09T05:48:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd903fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x562946134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd903fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x562946134680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39070/0 (socket says 172.21.15.115:39070) 2024-05-09T05:48:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd903fff640 1 -- 172.21.15.115:0/2143618079 learned_addr learned my addr 172.21.15.115:0/2143618079 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd902ffd640 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd9037fe640 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 0x562946134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd903fff640 1 -- 172.21.15.115:0/2143618079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 msgr2=0x562946134bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd903fff640 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 0x562946134bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd903fff640 1 -- 172.21.15.115:0/2143618079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 msgr2=0x562946135100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd903fff640 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946135100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd903fff640 1 -- 172.21.15.115:0/2143618079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56294605e7c0 con 0x562945d911b0 2024-05-09T05:48:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.003+0000 7fd9037fe640 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 0x562946134bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:48:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.004+0000 7fd903fff640 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x562946134680 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009ba0 tx=0x7fd8fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ac2043d6448b07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.004+0000 7fd900ff9640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd8ec0b2950 0x7fd8ec0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.004+0000 7fd8e9ffb640 1 -- 172.21.15.115:0/2143618079 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc004230 con 0x562945d911b0 2024-05-09T05:48:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.004+0000 7fd9037fe640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd8ec0b2950 0x7fd8ec0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.004+0000 7fd8e9ffb640 1 -- 172.21.15.115:0/2143618079 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8fc009e10 con 0x562945d911b0 2024-05-09T05:48:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.004+0000 7fd8e9ffb640 1 -- 172.21.15.115:0/2143618079 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc004b40 con 0x562945d911b0 2024-05-09T05:48:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.004+0000 7fd900ff9640 1 -- 172.21.15.115:0/3419464082 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fd8f80e2460 con 0x7fd904109d30 2024-05-09T05:48:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 -- 172.21.15.115:0/2143618079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 msgr2=0x562946134680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x562946134680 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009ba0 tx=0x7fd8fc011810 comp rx=0 tx=0).stop 2024-05-09T05:48:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd9037fe640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd8ec0b2950 0x7fd8ec0b4e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4006010 tx=0x7fd8f4005c30 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 -- 172.21.15.115:0/2143618079 shutdown_connections 2024-05-09T05:48:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946135100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 0x562946134bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 --2- 172.21.15.115:0/2143618079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x562946134680 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 -- 172.21.15.115:0/2143618079 >> 172.21.15.115:0/2143618079 conn(0x562945d85300 msgr2=0x562946130e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 -- 172.21.15.115:0/2143618079 shutdown_connections 2024-05-09T05:48:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 -- 172.21.15.115:0/2143618079 wait complete. 2024-05-09T05:48:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 -- start start 2024-05-09T05:48:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.005+0000 7fd910f86740 1 Processor -- start 2024-05-09T05:48:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd910f86740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 0x5629461406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd910f86740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946140be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd910f86740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x56294613eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd910f86740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5629461417b0 con 0x562945d911b0 2024-05-09T05:48:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd910f86740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5629460209c0 con 0x562945d704d0 2024-05-09T05:48:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd910f86740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562946141930 con 0x562945d682f0 2024-05-09T05:48:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd910f86740 10 client.0 _ll_register_callbacks cb 0x56294604d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd903fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x56294613eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd903fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x56294613eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39082/0 (socket says 172.21.15.115:39082) 2024-05-09T05:48:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd903fff640 1 -- 172.21.15.115:0/2054402977 learned_addr learned my addr 172.21.15.115:0/2054402977 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd902ffd640 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd9037fe640 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 0x5629461406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd903fff640 1 -- 172.21.15.115:0/2054402977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 msgr2=0x5629461406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd903fff640 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 0x5629461406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd903fff640 1 -- 172.21.15.115:0/2054402977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 msgr2=0x562946140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd903fff640 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd903fff640 1 -- 172.21.15.115:0/2054402977 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629460471f0 con 0x562945d911b0 2024-05-09T05:48:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd903fff640 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x56294613eb40 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0096c0 tx=0x7fd8fc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2765f2208e9a042 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.006+0000 7fd8e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:48:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/2054402977 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc027020 con 0x562945d911b0 2024-05-09T05:48:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/2054402977 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fd8fc003c10 con 0x562945d911b0 2024-05-09T05:48:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd910f86740 1 -- 172.21.15.115:0/2054402977 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8fc009e10 con 0x562945d911b0 2024-05-09T05:48:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/2054402977 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd8fc032050 con 0x562945d911b0 2024-05-09T05:48:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd910f86740 10 client.18195 fetch_fsmap learned FSMap version 49 2024-05-09T05:48:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd910f86740 1 -- 172.21.15.115:0/2054402977 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562945dfc790 con 0x562945d911b0 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/2054402977 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8fc009e10 con 0x562945d911b0 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/2054402977 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8374+0+0 (secure 0 0 0) 0x7fd8fc018070 con 0x562945d911b0 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd910f86740 10 client.18195 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd910f86740 10 client.18195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd910f86740 20 client.18195 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd910f86740 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562945d4fd80 0x562946150e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd910f86740 4 client.18195 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.007+0000 7fd910f86740 1 -- 172.21.15.115:0/2054402977 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629460a73d0 con 0x562945d4fd80 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.008+0000 7fd9037fe640 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562945d4fd80 0x562946150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.008+0000 7fd9037fe640 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562945d4fd80 0x562946150e70 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8990d670e2f48f server_cookie=9b62872755a6b2d9 in_seq=0 out_seq=0 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.008+0000 7fd8e2ffd640 10 client.18195 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/2054402977 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629460a73d0 con 0x562945d4fd80 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd8e2ffd640 10 client.18195 handle_command_reply: tid=0 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 client.18195 shutdown 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 -- 172.21.15.115:0/2054402977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 msgr2=0x56294613eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x56294613eb40 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0096c0 tx=0x7fd8fc003820 comp rx=0 tx=0).stop 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 -- 172.21.15.115:0/2054402977 shutdown_connections 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d911b0 0x56294613eb40 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562945d4fd80 0x562946150e70 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 --2- 172.21.15.115:0/2054402977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d682f0 0x5629461406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd8e2ffd640 0 client.18195 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 -- 172.21.15.115:0/2054402977 >> 172.21.15.115:0/2054402977 conn(0x562945d85300 msgr2=0x56294612f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 -- 172.21.15.115:0/2054402977 shutdown_connections 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 1 -- 172.21.15.115:0/2054402977 wait complete. 2024-05-09T05:48:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.009+0000 7fd910f86740 20 client.18195 trim_cache size 0 max 16384 2024-05-09T05:48:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.024+0000 7fd910f86740 1 Processor -- start 2024-05-09T05:48:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.024+0000 7fd910f86740 1 -- start start 2024-05-09T05:48:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd910f86740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d682f0 0x562946140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd910f86740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd910f86740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d911b0 0x562946147100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd910f86740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041b3350 con 0x562945d682f0 2024-05-09T05:48:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd910f86740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041b4f20 con 0x562945d704d0 2024-05-09T05:48:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd910f86740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041b3a20 con 0x562945d911b0 2024-05-09T05:48:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d682f0 0x562946140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d682f0 0x562946140db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39096/0 (socket says 172.21.15.115:39096) 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd9037fe640 1 -- 172.21.15.115:0/3958477646 learned_addr learned my addr 172.21.15.115:0/3958477646 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd9037fe640 1 -- 172.21.15.115:0/3958477646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d911b0 msgr2=0x562946147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd9037fe640 1 --2- 172.21.15.115:0/3958477646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d911b0 0x562946147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd902ffd640 1 --2- 172.21.15.115:0/3958477646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd9037fe640 1 -- 172.21.15.115:0/3958477646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 msgr2=0x562946146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd9037fe640 1 --2- 172.21.15.115:0/3958477646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd9037fe640 1 -- 172.21.15.115:0/3958477646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56294613a400 con 0x562945d682f0 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd9037fe640 1 --2- 172.21.15.115:0/3958477646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d682f0 0x562946140db0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd90406b5f0 tx=0x7fd8f4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d9a08b3611e21f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd8e97fa640 1 -- 172.21.15.115:0/3958477646 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4030880 con 0x562945d682f0 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd8e97fa640 1 -- 172.21.15.115:0/3958477646 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8f4058070 con 0x562945d682f0 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.025+0000 7fd8e97fa640 1 -- 172.21.15.115:0/3958477646 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4047c10 con 0x562945d682f0 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.026+0000 7fd910f86740 1 -- 172.21.15.115:0/3958477646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d682f0 msgr2=0x562946140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.026+0000 7fd910f86740 1 --2- 172.21.15.115:0/3958477646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d682f0 0x562946140db0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd90406b5f0 tx=0x7fd8f4040000 comp rx=0 tx=0).stop 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.026+0000 7fd910f86740 1 -- 172.21.15.115:0/3958477646 shutdown_connections 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.026+0000 7fd910f86740 1 --2- 172.21.15.115:0/3958477646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d911b0 0x562946147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.026+0000 7fd910f86740 1 --2- 172.21.15.115:0/3958477646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d704d0 0x562946146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.026+0000 7fd910f86740 1 --2- 172.21.15.115:0/3958477646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d682f0 0x562946140db0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.026+0000 7fd910f86740 1 -- 172.21.15.115:0/3958477646 >> 172.21.15.115:0/3958477646 conn(0x562945d85300 msgr2=0x5629461406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.026+0000 7fd910f86740 1 -- 172.21.15.115:0/3958477646 shutdown_connections 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.026+0000 7fd910f86740 1 -- 172.21.15.115:0/3958477646 wait complete. 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd910f86740 1 -- start start 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd910f86740 1 Processor -- start 2024-05-09T05:48:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd910f86740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d682f0 0x562946146f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd910f86740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d704d0 0x5629461442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd910f86740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d911b0 0x562946144800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd910f86740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041b4850 con 0x562945d704d0 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd910f86740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041b40f0 con 0x562945d682f0 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd910f86740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041b1330 con 0x562945d911b0 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd910f86740 10 client.0 _ll_register_callbacks cb 0x7fd8f80e5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd902ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d704d0 0x5629461442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd902ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d704d0 0x5629461442c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39112/0 (socket says 172.21.15.115:39112) 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd902ffd640 1 -- 172.21.15.115:0/1709414469 learned_addr learned my addr 172.21.15.115:0/1709414469 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd9037fe640 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d682f0 0x562946146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.027+0000 7fd903fff640 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d911b0 0x562946144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd902ffd640 1 -- 172.21.15.115:0/1709414469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d911b0 msgr2=0x562946144800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd902ffd640 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d911b0 0x562946144800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd902ffd640 1 -- 172.21.15.115:0/1709414469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d682f0 msgr2=0x562946146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd902ffd640 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d682f0 0x562946146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd902ffd640 1 -- 172.21.15.115:0/1709414469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562945e735f0 con 0x562945d704d0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd902ffd640 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d704d0 0x5629461442c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80a5480 tx=0x7fd8f80dfdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac77f6e73e0851df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd8e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/1709414469 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8021040 con 0x562945d704d0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/1709414469 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9350+0+0 (secure 0 0 0) 0x7fd8f80b0d40 con 0x562945d704d0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd910f86740 1 -- 172.21.15.115:0/1709414469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8f4058070 con 0x562945d704d0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/1709414469 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd8f80a91d0 con 0x562945d704d0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd910f86740 10 client.18201 fetch_fsmap learned FSMap version 49 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd910f86740 1 -- 172.21.15.115:0/1709414469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56294605ded0 con 0x562945d704d0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/1709414469 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8f4058070 con 0x562945d704d0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/1709414469 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8374+0+0 (secure 0 0 0) 0x7fd8f80e2120 con 0x562945d704d0 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd910f86740 10 client.18201 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd910f86740 10 client.18201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:48:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd910f86740 20 client.18201 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd910f86740 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562945d251c0 0x5629461bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd910f86740 4 client.18201 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:48:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.028+0000 7fd910f86740 1 -- 172.21.15.115:0/1709414469 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629460a1240 con 0x562945d251c0 2024-05-09T05:48:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.029+0000 7fd9037fe640 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562945d251c0 0x5629461bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.029+0000 7fd8e2ffd640 10 client.18201 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.029+0000 7fd9037fe640 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562945d251c0 0x5629461bb550 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc44c53d1cd3512e server_cookie=9e7b55806c69e6c0 in_seq=0 out_seq=0 2024-05-09T05:48:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.036+0000 7fd8e2ffd640 1 -- 172.21.15.115:0/1709414469 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5629460a1240 con 0x562945d251c0 2024-05-09T05:48:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.036+0000 7fd8e2ffd640 10 client.18201 handle_command_reply: tid=0 2024-05-09T05:48:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.036+0000 7fd910f86740 1 client.18201 shutdown 2024-05-09T05:48:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.036+0000 7fd910f86740 1 -- 172.21.15.115:0/1709414469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d704d0 msgr2=0x5629461442c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.036+0000 7fd910f86740 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d704d0 0x5629461442c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80a5480 tx=0x7fd8f80dfdb0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd910f86740 1 -- 172.21.15.115:0/1709414469 shutdown_connections 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd910f86740 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562945d251c0 0x5629461bb550 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd910f86740 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562945d911b0 0x562946144800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd910f86740 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562945d704d0 0x5629461442c0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd8e2ffd640 0 client.18201 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd910f86740 1 --2- 172.21.15.115:0/1709414469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562945d682f0 0x562946146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd910f86740 1 -- 172.21.15.115:0/1709414469 >> 172.21.15.115:0/1709414469 conn(0x562945d85300 msgr2=0x562946140680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd910f86740 1 -- 172.21.15.115:0/1709414469 shutdown_connections 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd910f86740 1 -- 172.21.15.115:0/1709414469 wait complete. 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd910f86740 20 client.18201 trim_cache size 0 max 16384 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:48:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:48:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:48:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 -- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd8ec0b2950 msgr2=0x7fd8ec0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd8ec0b2950 0x7fd8ec0b4e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4006010 tx=0x7fd8f4005c30 comp rx=0 tx=0).stop 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 -- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 msgr2=0x7fd9041033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 0x7fd9041033c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800d770 tx=0x7fd8f800dc30 comp rx=0 tx=0).stop 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 -- 172.21.15.115:0/3419464082 shutdown_connections 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd8ec0b2950 0x7fd8ec0b4e60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd90410e6e0 0x7fd904103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd904109d30 0x7fd9041033c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 --2- 172.21.15.115:0/3419464082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9041021e0 0x7fd904102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 -- 172.21.15.115:0/3419464082 >> 172.21.15.115:0/3419464082 conn(0x7fd9040fd9d0 msgr2=0x7fd9040fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 -- 172.21.15.115:0/3419464082 shutdown_connections 2024-05-09T05:48:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:10.037+0000 7fd90a0d1640 1 -- 172.21.15.115:0/3419464082 wait complete. 2024-05-09T05:48:10.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:10 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:48:10.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:10 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:48:10.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:10 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:48:11.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:11 smithi195 ceph-mon[33844]: pgmap v310: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-05-09T05:48:11.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:11 smithi195 ceph-mon[33844]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T05:48:11.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:11 smithi115 ceph-mon[30349]: pgmap v310: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-05-09T05:48:11.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:11 smithi115 ceph-mon[30349]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T05:48:11.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:11 smithi161 ceph-mon[33805]: pgmap v310: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 33 op/s 2024-05-09T05:48:11.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:11 smithi161 ceph-mon[33805]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T05:48:13.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:13 smithi195 ceph-mon[33844]: pgmap v312: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 21 MiB/s wr, 40 op/s 2024-05-09T05:48:13.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:13 smithi115 ceph-mon[30349]: pgmap v312: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 21 MiB/s wr, 40 op/s 2024-05-09T05:48:13.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:13 smithi161 ceph-mon[33805]: pgmap v312: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 21 MiB/s wr, 40 op/s 2024-05-09T05:48:15.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:15 smithi195 ceph-mon[33844]: pgmap v313: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-09T05:48:15.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:15 smithi115 ceph-mon[30349]: pgmap v313: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-09T05:48:15.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:15 smithi161 ceph-mon[33805]: pgmap v313: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-09T05:48:17.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:17 smithi195 ceph-mon[33844]: pgmap v314: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-09T05:48:17.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:17 smithi115 ceph-mon[30349]: pgmap v314: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-09T05:48:17.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:17 smithi161 ceph-mon[33805]: pgmap v314: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-09T05:48:19.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:19 smithi195 ceph-mon[33844]: pgmap v315: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 482 KiB/s wr, 11 op/s 2024-05-09T05:48:19.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:19 smithi195 ceph-mon[33844]: osdmap e98: 12 total, 12 up, 12 in 2024-05-09T05:48:19.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:19 smithi115 ceph-mon[30349]: pgmap v315: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 482 KiB/s wr, 11 op/s 2024-05-09T05:48:19.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:19 smithi115 ceph-mon[30349]: osdmap e98: 12 total, 12 up, 12 in 2024-05-09T05:48:19.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:19 smithi161 ceph-mon[33805]: pgmap v315: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 482 KiB/s wr, 11 op/s 2024-05-09T05:48:19.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:19 smithi161 ceph-mon[33805]: osdmap e98: 12 total, 12 up, 12 in 2024-05-09T05:48:21.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:21 smithi195 ceph-mon[33844]: pgmap v317: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 490 KiB/s wr, 11 op/s 2024-05-09T05:48:21.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:21 smithi115 ceph-mon[30349]: pgmap v317: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 490 KiB/s wr, 11 op/s 2024-05-09T05:48:21.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:21 smithi161 ceph-mon[33805]: pgmap v317: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 490 KiB/s wr, 11 op/s 2024-05-09T05:48:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 -- 172.21.15.115:0/2618205208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b41032e0 msgr2=0x7fb8b410f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 --2- 172.21.15.115:0/2618205208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b41032e0 0x7fb8b410f6b0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009ca0 tx=0x7fb8ac02a8c0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 -- 172.21.15.115:0/2618205208 shutdown_connections 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 --2- 172.21.15.115:0/2618205208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b41032e0 0x7fb8b410f6b0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 --2- 172.21.15.115:0/2618205208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8b4109030 0x7fb8b4102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 --2- 172.21.15.115:0/2618205208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b4108b90 0x7fb8b4102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 -- 172.21.15.115:0/2618205208 >> 172.21.15.115:0/2618205208 conn(0x7fb8b40fe390 msgr2=0x7fb8b41007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 -- 172.21.15.115:0/2618205208 shutdown_connections 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 -- 172.21.15.115:0/2618205208 wait complete. 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.146+0000 7fb8bad55640 1 Processor -- start 2024-05-09T05:48:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8bad55640 1 -- start start 2024-05-09T05:48:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8bad55640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b41032e0 0x7fb8b419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8bad55640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8b4108b90 0x7fb8b419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8bad55640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4109030 0x7fb8b419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8bad55640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b4112ca0 con 0x7fb8b4109030 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8bad55640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b4112b20 con 0x7fb8b4108b90 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8bad55640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b4112e20 con 0x7fb8b41032e0 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b92f2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4109030 0x7fb8b419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b92f2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4109030 0x7fb8b419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41946/0 (socket says 172.21.15.115:41946) 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b92f2640 1 -- 172.21.15.115:0/3214220434 learned_addr learned my addr 172.21.15.115:0/3214220434 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b41032e0 0x7fb8b419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8b4108b90 0x7fb8b419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b92f2640 1 -- 172.21.15.115:0/3214220434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b41032e0 msgr2=0x7fb8b419b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b92f2640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b41032e0 0x7fb8b419b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b92f2640 1 -- 172.21.15.115:0/3214220434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8b4108b90 msgr2=0x7fb8b419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b92f2640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8b4108b90 0x7fb8b419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b92f2640 1 -- 172.21.15.115:0/3214220434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b41a6920 con 0x7fb8b4109030 2024-05-09T05:48:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.147+0000 7fb8b92f2640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4109030 0x7fb8b419c080 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005cc0 tx=0x7fb8ac032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.148+0000 7fb8b1ffb640 1 -- 172.21.15.115:0/3214220434 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac032e00 con 0x7fb8b4109030 2024-05-09T05:48:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.148+0000 7fb8b1ffb640 1 -- 172.21.15.115:0/3214220434 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8ac0044d0 con 0x7fb8b4109030 2024-05-09T05:48:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.148+0000 7fb8bad55640 1 -- 172.21.15.115:0/3214220434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b41a6bb0 con 0x7fb8b4109030 2024-05-09T05:48:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.148+0000 7fb8b1ffb640 1 -- 172.21.15.115:0/3214220434 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac030b70 con 0x7fb8b4109030 2024-05-09T05:48:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.148+0000 7fb8bad55640 1 -- 172.21.15.115:0/3214220434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b41a7100 con 0x7fb8b4109030 2024-05-09T05:48:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.150+0000 7fb8c1c05740 1 Processor -- start 2024-05-09T05:48:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.150+0000 7fb8b1ffb640 1 -- 172.21.15.115:0/3214220434 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fb8ac004680 con 0x7fb8b4109030 2024-05-09T05:48:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8c1c05740 1 -- start start 2024-05-09T05:48:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740aa1b0 0x55647444d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740812f0 0x55647444dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 0x55647444dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5564743b27c0 con 0x5564740aa1b0 2024-05-09T05:48:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55647434e0e0 con 0x5564740812f0 2024-05-09T05:48:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5564743ae780 con 0x5564740894d0 2024-05-09T05:48:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740aa1b0 0x55647444d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740aa1b0 0x55647444d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41948/0 (socket says 172.21.15.115:41948) 2024-05-09T05:48:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b3fff640 1 -- 172.21.15.115:0/968548743 learned_addr learned my addr 172.21.15.115:0/968548743 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/968548743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740812f0 0x55647444dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/968548743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 0x55647444dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b1ffb640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb8a40b2950 0x7fb8a40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b3fff640 1 -- 172.21.15.115:0/968548743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 msgr2=0x55647444dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/968548743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 0x55647444dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b3fff640 1 -- 172.21.15.115:0/968548743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740812f0 msgr2=0x55647444dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/968548743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740812f0 0x55647444dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.151+0000 7fb8b3fff640 1 -- 172.21.15.115:0/968548743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564743777c0 con 0x5564740aa1b0 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.152+0000 7fb8b92f2640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb8a40b2950 0x7fb8a40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.152+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/968548743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740aa1b0 0x55647444d570 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0013450 tx=0x7fb8a000d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b4009a9a588627 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.152+0000 7fb8b1ffb640 1 -- 172.21.15.115:0/3214220434 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fb8ac103dc0 con 0x7fb8b4109030 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.152+0000 7fb89effd640 1 -- 172.21.15.115:0/968548743 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0019070 con 0x5564740aa1b0 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.152+0000 7fb89effd640 1 -- 172.21.15.115:0/968548743 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a00047a0 con 0x5564740aa1b0 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.152+0000 7fb89effd640 1 -- 172.21.15.115:0/968548743 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00116a0 con 0x5564740aa1b0 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 -- 172.21.15.115:0/968548743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740aa1b0 msgr2=0x55647444d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/968548743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740aa1b0 0x55647444d570 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0013450 tx=0x7fb8a000d5b0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8b92f2640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb8a40b2950 0x7fb8a40b4e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0c6ea0 tx=0x7fb8ac0c6d40 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 -- 172.21.15.115:0/968548743 shutdown_connections 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/968548743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 0x55647444dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/968548743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740812f0 0x55647444dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/968548743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740aa1b0 0x55647444d570 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 -- 172.21.15.115:0/968548743 >> 172.21.15.115:0/968548743 conn(0x55647409e300 msgr2=0x556474449e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 -- 172.21.15.115:0/968548743 shutdown_connections 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 -- 172.21.15.115:0/968548743 wait complete. 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 -- start start 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.153+0000 7fb8c1c05740 1 Processor -- start 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740812f0 0x55647445dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740894d0 0x55647445e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740aa1b0 0x55647445c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55647445ebc0 con 0x5564740894d0 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5564743399c0 con 0x5564740aa1b0 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55647445ed40 con 0x5564740812f0 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8c1c05740 10 client.0 _ll_register_callbacks cb 0x556474366020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b8af1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740894d0 0x55647445e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b8af1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740894d0 0x55647445e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41960/0 (socket says 172.21.15.115:41960) 2024-05-09T05:48:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b8af1640 1 -- 172.21.15.115:0/3183596194 learned_addr learned my addr 172.21.15.115:0/3183596194 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740812f0 0x55647445dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740aa1b0 0x55647445c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b8af1640 1 -- 172.21.15.115:0/3183596194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740812f0 msgr2=0x55647445dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740812f0 0x55647445dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b8af1640 1 -- 172.21.15.115:0/3183596194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740aa1b0 msgr2=0x55647445c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740aa1b0 0x55647445c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b8af1640 1 -- 172.21.15.115:0/3183596194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564743601f0 con 0x5564740894d0 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740894d0 0x55647445e160 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800b990 tx=0x7fb8a800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baefc6aabc12cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.154+0000 7fb893fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb893fff640 1 -- 172.21.15.115:0/3183596194 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8005020 con 0x5564740894d0 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb893fff640 1 -- 172.21.15.115:0/3183596194 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fb8a80051c0 con 0x5564740894d0 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb8c1c05740 1 -- 172.21.15.115:0/3183596194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a00047a0 con 0x5564740894d0 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb893fff640 1 -- 172.21.15.115:0/3183596194 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb8a80053c0 con 0x5564740894d0 2024-05-09T05:48:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb8c1c05740 10 client.18213 fetch_fsmap learned FSMap version 49 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb8c1c05740 1 -- 172.21.15.115:0/3183596194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556474115790 con 0x5564740894d0 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb893fff640 1 -- 172.21.15.115:0/3183596194 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a00047a0 con 0x5564740894d0 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb893fff640 1 -- 172.21.15.115:0/3183596194 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8374+0+0 (secure 0 0 0) 0x7fb8a801b500 con 0x5564740894d0 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb8c1c05740 10 client.18213 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb8c1c05740 10 client.18213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb8c1c05740 20 client.18213 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556474068d80 0x55647446e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb8c1c05740 4 client.18213 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.155+0000 7fb8c1c05740 1 -- 172.21.15.115:0/3183596194 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564743c03d0 con 0x556474068d80 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.156+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556474068d80 0x55647446e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.156+0000 7fb893fff640 10 client.18213 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.156+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556474068d80 0x55647446e520 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795d1d6a3af9c505 server_cookie=e775dcdbb819e981 in_seq=0 out_seq=0 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb893fff640 1 -- 172.21.15.115:0/3183596194 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564743c03d0 con 0x556474068d80 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb893fff640 10 client.18213 handle_command_reply: tid=0 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 client.18213 shutdown 2024-05-09T05:48:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 -- 172.21.15.115:0/3183596194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740894d0 msgr2=0x55647445e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740894d0 0x55647445e160 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800b990 tx=0x7fb8a800be50 comp rx=0 tx=0).stop 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 -- 172.21.15.115:0/3183596194 shutdown_connections 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740aa1b0 0x55647445c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740894d0 0x55647445e160 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556474068d80 0x55647446e520 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/3183596194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740812f0 0x55647445dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb893fff640 0 client.18213 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 -- 172.21.15.115:0/3183596194 >> 172.21.15.115:0/3183596194 conn(0x55647409e300 msgr2=0x55647444f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 -- 172.21.15.115:0/3183596194 shutdown_connections 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.157+0000 7fb8c1c05740 1 -- 172.21.15.115:0/3183596194 wait complete. 2024-05-09T05:48:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.158+0000 7fb8c1c05740 20 client.18213 trim_cache size 0 max 16384 2024-05-09T05:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.172+0000 7fb8c1c05740 1 Processor -- start 2024-05-09T05:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8c1c05740 1 -- start start 2024-05-09T05:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x55647444fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740894d0 0x5564744558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740aa1b0 0x556474455e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41aeef0 con 0x5564740812f0 2024-05-09T05:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41b0ac0 con 0x5564740894d0 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41af5c0 con 0x5564740aa1b0 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x55647444fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x55647444fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41970/0 (socket says 172.21.15.115:41970) 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b3fff640 1 -- 172.21.15.115:0/250553188 learned_addr learned my addr 172.21.15.115:0/250553188 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b3fff640 1 -- 172.21.15.115:0/250553188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740aa1b0 msgr2=0x556474455e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/250553188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740aa1b0 0x556474455e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b3fff640 1 -- 172.21.15.115:0/250553188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740894d0 msgr2=0x5564744558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/250553188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740894d0 0x5564744558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/250553188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740894d0 0x5564744558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b3fff640 1 -- 172.21.15.115:0/250553188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647418c5f0 con 0x5564740812f0 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/250553188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740894d0 0x5564744558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/250553188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x55647444fab0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0013da0 tx=0x7fb8a001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12eead651bb92344 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.173+0000 7fb89e7fc640 1 -- 172.21.15.115:0/250553188 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0019040 con 0x5564740812f0 2024-05-09T05:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb89e7fc640 1 -- 172.21.15.115:0/250553188 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a0031070 con 0x5564740812f0 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb89e7fc640 1 -- 172.21.15.115:0/250553188 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0005630 con 0x5564740812f0 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb8c1c05740 1 -- 172.21.15.115:0/250553188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 msgr2=0x55647444fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/250553188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x55647444fab0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0013da0 tx=0x7fb8a001f040 comp rx=0 tx=0).stop 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb8c1c05740 1 -- 172.21.15.115:0/250553188 shutdown_connections 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/250553188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740aa1b0 0x556474455e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/250553188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740894d0 0x5564744558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/250553188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x55647444fab0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb8c1c05740 1 -- 172.21.15.115:0/250553188 >> 172.21.15.115:0/250553188 conn(0x55647409e300 msgr2=0x55647444f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb8c1c05740 1 -- 172.21.15.115:0/250553188 shutdown_connections 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.174+0000 7fb8c1c05740 1 -- 172.21.15.115:0/250553188 wait complete. 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8c1c05740 1 -- start start 2024-05-09T05:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8c1c05740 1 Processor -- start 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x556474463fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 0x5564744644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8c1c05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740aa1b0 0x55647445f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41b03f0 con 0x5564740812f0 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41afc90 con 0x5564740aa1b0 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8c1c05740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41acef0 con 0x5564740894d0 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8c1c05740 10 client.0 _ll_register_callbacks cb 0x7fb8a80042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x556474463fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x556474463fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41984/0 (socket says 172.21.15.115:41984) 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8b3fff640 1 -- 172.21.15.115:0/1695823850 learned_addr learned my addr 172.21.15.115:0/1695823850 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.175+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 0x5564744644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb8b3fff640 1 -- 172.21.15.115:0/1695823850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 msgr2=0x5564744644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 0x5564744644f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb8b3fff640 1 -- 172.21.15.115:0/1695823850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740aa1b0 msgr2=0x55647445f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740aa1b0 0x55647445f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb8b3fff640 1 -- 172.21.15.115:0/1695823850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556474376ed0 con 0x5564740812f0 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb8b3fff640 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x556474463fb0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000db70 tx=0x7fb8a0005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8e592c4cca4c9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb893fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb893fff640 1 -- 172.21.15.115:0/1695823850 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a003a070 con 0x5564740812f0 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb893fff640 1 -- 172.21.15.115:0/1695823850 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fb8a001d9b0 con 0x5564740812f0 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb8c1c05740 1 -- 172.21.15.115:0/1695823850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a0031070 con 0x5564740812f0 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb893fff640 1 -- 172.21.15.115:0/1695823850 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb8a0019040 con 0x5564740812f0 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb8c1c05740 10 client.18219 fetch_fsmap learned FSMap version 49 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb8c1c05740 1 -- 172.21.15.115:0/1695823850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5564744d6070 con 0x5564740812f0 2024-05-09T05:48:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.176+0000 7fb893fff640 1 -- 172.21.15.115:0/1695823850 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a0031070 con 0x5564740812f0 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb893fff640 1 -- 172.21.15.115:0/1695823850 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8374+0+0 (secure 0 0 0) 0x7fb8a002c630 con 0x5564740812f0 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb8c1c05740 10 client.18219 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb8c1c05740 10 client.18219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb8c1c05740 20 client.18219 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55647403e1c0 0x5564744d8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb8c1c05740 4 client.18219 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb8c1c05740 1 -- 172.21.15.115:0/1695823850 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564743ba240 con 0x55647403e1c0 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55647403e1c0 0x5564744d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb893fff640 10 client.18219 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.177+0000 7fb8b8af1640 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55647403e1c0 0x5564744d8590 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc35f748852662b4 server_cookie=d70aef6cc1b16994 in_seq=0 out_seq=0 2024-05-09T05:48:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb893fff640 1 -- 172.21.15.115:0/1695823850 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5564743ba240 con 0x55647403e1c0 2024-05-09T05:48:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb893fff640 10 client.18219 handle_command_reply: tid=0 2024-05-09T05:48:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 client.18219 shutdown 2024-05-09T05:48:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 -- 172.21.15.115:0/1695823850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 msgr2=0x556474463fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x556474463fb0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000db70 tx=0x7fb8a0005d00 comp rx=0 tx=0).stop 2024-05-09T05:48:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 -- 172.21.15.115:0/1695823850 shutdown_connections 2024-05-09T05:48:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564740aa1b0 0x55647445f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55647403e1c0 0x5564744d8590 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5564740894d0 0x5564744644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb893fff640 0 client.18219 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 --2- 172.21.15.115:0/1695823850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564740812f0 0x556474463fb0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 -- 172.21.15.115:0/1695823850 >> 172.21.15.115:0/1695823850 conn(0x55647409e300 msgr2=0x55647445b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 -- 172.21.15.115:0/1695823850 shutdown_connections 2024-05-09T05:48:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 1 -- 172.21.15.115:0/1695823850 wait complete. 2024-05-09T05:48:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.186+0000 7fb8c1c05740 20 client.18219 trim_cache size 0 max 16384 2024-05-09T05:48:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:48:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:48:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:48:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 -- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb8a40b2950 msgr2=0x7fb8a40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb8a40b2950 0x7fb8a40b4e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0c6ea0 tx=0x7fb8ac0c6d40 comp rx=0 tx=0).stop 2024-05-09T05:48:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 -- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4109030 msgr2=0x7fb8b419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4109030 0x7fb8b419c080 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005cc0 tx=0x7fb8ac032a10 comp rx=0 tx=0).stop 2024-05-09T05:48:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 -- 172.21.15.115:0/3214220434 shutdown_connections 2024-05-09T05:48:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb8a40b2950 0x7fb8a40b4e60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8b4109030 0x7fb8b419c080 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8b4108b90 0x7fb8b419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 --2- 172.21.15.115:0/3214220434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8b41032e0 0x7fb8b419b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 -- 172.21.15.115:0/3214220434 >> 172.21.15.115:0/3214220434 conn(0x7fb8b40fe390 msgr2=0x7fb8b4105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 -- 172.21.15.115:0/3214220434 shutdown_connections 2024-05-09T05:48:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:23.187+0000 7fb8bad55640 1 -- 172.21.15.115:0/3214220434 wait complete. 2024-05-09T05:48:23.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:23 smithi195 ceph-mon[33844]: pgmap v318: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 482 KiB/s wr, 11 op/s 2024-05-09T05:48:23.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:23 smithi115 ceph-mon[30349]: pgmap v318: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 482 KiB/s wr, 11 op/s 2024-05-09T05:48:23.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:23 smithi161 ceph-mon[33805]: pgmap v318: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 482 KiB/s wr, 11 op/s 2024-05-09T05:48:25.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:25 smithi195 ceph-mon[33844]: pgmap v319: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-05-09T05:48:25.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:25 smithi115 ceph-mon[30349]: pgmap v319: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-05-09T05:48:25.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:25 smithi161 ceph-mon[33805]: pgmap v319: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-05-09T05:48:27.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:27 smithi195 ceph-mon[33844]: pgmap v320: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-05-09T05:48:27.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:27 smithi115 ceph-mon[30349]: pgmap v320: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-05-09T05:48:27.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:27 smithi161 ceph-mon[33805]: pgmap v320: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-05-09T05:48:28.233 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:48:28.488 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:48:29.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:29 smithi195 ceph-mon[33844]: pgmap v321: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-05-09T05:48:29.634 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:29 smithi115 ceph-mon[30349]: pgmap v321: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-05-09T05:48:29.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:29 smithi161 ceph-mon[33805]: pgmap v321: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-05-09T05:48:30.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.364+0000 7f32c85bd700 1 -- 172.21.15.115:0/60908840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c0108760 msgr2=0x7f32c01100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:30.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.364+0000 7f32c85bd700 1 --2- 172.21.15.115:0/60908840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c0108760 0x7f32c01100c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f32b80099c0 tx=0x7f32b8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.364+0000 7f32c85bd700 1 -- 172.21.15.115:0/60908840 shutdown_connections 2024-05-09T05:48:30.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.364+0000 7f32c85bd700 1 --2- 172.21.15.115:0/60908840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c0108760 0x7f32c01100c0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.364+0000 7f32c85bd700 1 --2- 172.21.15.115:0/60908840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32c01034e0 0x7f32c01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.364+0000 7f32c85bd700 1 --2- 172.21.15.115:0/60908840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c0107e10 0x7f32c0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.364+0000 7f32c85bd700 1 -- 172.21.15.115:0/60908840 >> 172.21.15.115:0/60908840 conn(0x7f32c00fef30 msgr2=0x7f32c0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:30.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.364+0000 7f32c85bd700 1 -- 172.21.15.115:0/60908840 shutdown_connections 2024-05-09T05:48:30.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.364+0000 7f32c85bd700 1 -- 172.21.15.115:0/60908840 wait complete. 2024-05-09T05:48:30.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c85bd700 1 Processor -- start 2024-05-09T05:48:30.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c85bd700 1 -- start start 2024-05-09T05:48:30.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c85bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c01034e0 0x7f32c019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c85bd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32c0107e10 0x7f32c019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c85bd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c0108760 0x7f32c019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c85bd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0113260 con 0x7f32c01034e0 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c85bd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01130e0 con 0x7f32c0108760 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c85bd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01133e0 con 0x7f32c0107e10 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c6359700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c01034e0 0x7f32c019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c6359700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c01034e0 0x7f32c019ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41998/0 (socket says 172.21.15.115:41998) 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.365+0000 7f32c6359700 1 -- 172.21.15.115:0/202402949 learned_addr learned my addr 172.21.15.115:0/202402949 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c5b58700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32c0107e10 0x7f32c019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c6b5a700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c0108760 0x7f32c019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c6359700 1 -- 172.21.15.115:0/202402949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32c0107e10 msgr2=0x7f32c019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c6359700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32c0107e10 0x7f32c019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c6359700 1 -- 172.21.15.115:0/202402949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c0108760 msgr2=0x7f32c019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:30.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c6359700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c0108760 0x7f32c019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c6359700 1 -- 172.21.15.115:0/202402949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c019fbf0 con 0x7f32c01034e0 2024-05-09T05:48:30.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c6b5a700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c0108760 0x7f32c019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:48:30.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c6359700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c01034e0 0x7f32c019ad20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f32bc007f10 tx=0x7f32bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:30.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32b77fe700 1 -- 172.21.15.115:0/202402949 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00dbd0 con 0x7f32c01034e0 2024-05-09T05:48:30.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c019fee0 con 0x7f32c01034e0 2024-05-09T05:48:30.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32b77fe700 1 -- 172.21.15.115:0/202402949 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32bc00dd70 con 0x7f32c01034e0 2024-05-09T05:48:30.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32b77fe700 1 -- 172.21.15.115:0/202402949 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc005950 con 0x7f32c01034e0 2024-05-09T05:48:30.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.366+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01a7800 con 0x7f32c01034e0 2024-05-09T05:48:30.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.367+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c00512c0 con 0x7f32c01034e0 2024-05-09T05:48:30.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.369+0000 7f32b77fe700 1 -- 172.21.15.115:0/202402949 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f32bc01a020 con 0x7f32c01034e0 2024-05-09T05:48:30.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.369+0000 7f32b77fe700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f32ac0b29f0 0x7f32ac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:30.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.369+0000 7f32c5b58700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f32ac0b29f0 0x7f32ac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:30.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.370+0000 7f32b77fe700 1 -- 172.21.15.115:0/202402949 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f32bc0e27e0 con 0x7f32c01034e0 2024-05-09T05:48:30.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.370+0000 7f32c5b58700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f32ac0b29f0 0x7f32ac0b4f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32c0109850 tx=0x7f32b000a300 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:30.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.373+0000 7f32b77fe700 1 -- 172.21.15.115:0/202402949 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32bc010070 con 0x7f32c01034e0 2024-05-09T05:48:30.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.622+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32c019c080 con 0x7f32c01034e0 2024-05-09T05:48:30.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.623+0000 7f32b77fe700 1 -- 172.21.15.115:0/202402949 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11956 (secure 0 0 0) 0x7f32bc0a9e30 con 0x7f32c01034e0 2024-05-09T05:48:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:48:30.615 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:48:08.367106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27316":{"gid":27316,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/391681574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":391681574},{"type":"v1","addr":"172.21.15.195:6833","nonce":391681574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:48:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f32ac0b29f0 msgr2=0x7f32ac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f32ac0b29f0 0x7f32ac0b4f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32c0109850 tx=0x7f32b000a300 comp rx=0 tx=0).stop 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c01034e0 msgr2=0x7f32c019ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c01034e0 0x7f32c019ad20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f32bc007f10 tx=0x7f32bc00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 shutdown_connections 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f32ac0b29f0 0x7f32ac0b4f00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c0108760 0x7f32c019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32c01034e0 0x7f32c019ad20 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 --2- 172.21.15.115:0/202402949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32c0107e10 0x7f32c019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 >> 172.21.15.115:0/202402949 conn(0x7f32c00fef30 msgr2=0x7f32c006c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 shutdown_connections 2024-05-09T05:48:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:30.625+0000 7f32c85bd700 1 -- 172.21.15.115:0/202402949 wait complete. 2024-05-09T05:48:30.619 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 49 2024-05-09T05:48:31.411 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 6f88124 --release --await' 2024-05-09T05:48:31.412 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id 6f88124 --release --await 2024-05-09T05:48:31.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:31 smithi195 ceph-mon[33844]: pgmap v322: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-05-09T05:48:31.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:31 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/202402949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:48:31.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:31 smithi115 ceph-mon[30349]: pgmap v322: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-05-09T05:48:31.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:31 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/202402949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:48:31.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:31 smithi161 ceph-mon[33805]: pgmap v322: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-05-09T05:48:31.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:31 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/202402949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:48:31.662 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:48:33.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:33 smithi115 ceph-mon[30349]: pgmap v323: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-09T05:48:33.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.516+0000 7f3a50f68700 1 -- 172.21.15.115:0/789030752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1042e0 msgr2=0x7f3a4c110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:33.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.516+0000 7f3a50f68700 1 --2- 172.21.15.115:0/789030752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1042e0 0x7f3a4c110270 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b0a0 tx=0x7f3a4400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.516+0000 7f3a50f68700 1 -- 172.21.15.115:0/789030752 shutdown_connections 2024-05-09T05:48:33.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.516+0000 7f3a50f68700 1 --2- 172.21.15.115:0/789030752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a4c1094d0 0x7f3a4c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.516+0000 7f3a50f68700 1 --2- 172.21.15.115:0/789030752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1042e0 0x7f3a4c110270 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.516+0000 7f3a50f68700 1 --2- 172.21.15.115:0/789030752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c103990 0x7f3a4c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.516+0000 7f3a50f68700 1 -- 172.21.15.115:0/789030752 >> 172.21.15.115:0/789030752 conn(0x7f3a4c0fe4f0 msgr2=0x7f3a4c100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:33.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.516+0000 7f3a50f68700 1 -- 172.21.15.115:0/789030752 shutdown_connections 2024-05-09T05:48:33.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.516+0000 7f3a50f68700 1 -- 172.21.15.115:0/789030752 wait complete. 2024-05-09T05:48:33.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.517+0000 7f3a50f68700 1 Processor -- start 2024-05-09T05:48:33.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.517+0000 7f3a50f68700 1 -- start start 2024-05-09T05:48:33.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.517+0000 7f3a50f68700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c103990 0x7f3a4c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.517+0000 7f3a50f68700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a4c1042e0 0x7f3a4c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.517+0000 7f3a50f68700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1094d0 0x7f3a4c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a50f68700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1134b0 con 0x7f3a4c1094d0 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a50f68700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c113330 con 0x7f3a4c1042e0 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a50f68700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c113630 con 0x7f3a4c103990 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1094d0 0x7f3a4c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1094d0 0x7f3a4c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59022/0 (socket says 172.21.15.115:59022) 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4ad9d700 1 -- 172.21.15.115:0/3821840129 learned_addr learned my addr 172.21.15.115:0/3821840129 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a4c1042e0 0x7f3a4c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c103990 0x7f3a4c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4ad9d700 1 -- 172.21.15.115:0/3821840129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c103990 msgr2=0x7f3a4c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c103990 0x7f3a4c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4ad9d700 1 -- 172.21.15.115:0/3821840129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a4c1042e0 msgr2=0x7f3a4c19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:33.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a4c1042e0 0x7f3a4c19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4ad9d700 1 -- 172.21.15.115:0/3821840129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c19fc70 con 0x7f3a4c1094d0 2024-05-09T05:48:33.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1094d0 0x7f3a4c19f680 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b830 tx=0x7f3a44004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:33.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a43fff700 1 -- 172.21.15.115:0/3821840129 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44021070 con 0x7f3a4c1094d0 2024-05-09T05:48:33.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a50f68700 1 -- 172.21.15.115:0/3821840129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c19ff00 con 0x7f3a4c1094d0 2024-05-09T05:48:33.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.518+0000 7f3a43fff700 1 -- 172.21.15.115:0/3821840129 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a44007b80 con 0x7f3a4c1094d0 2024-05-09T05:48:33.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.519+0000 7f3a43fff700 1 -- 172.21.15.115:0/3821840129 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a440052a0 con 0x7f3a4c1094d0 2024-05-09T05:48:33.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.519+0000 7f3a50f68700 1 -- 172.21.15.115:0/3821840129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c1a0430 con 0x7f3a4c1094d0 2024-05-09T05:48:33.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.521+0000 7f3a43fff700 1 -- 172.21.15.115:0/3821840129 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f3a44024070 con 0x7f3a4c1094d0 2024-05-09T05:48:33.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.522+0000 7f3a43fff700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3a380b29f0 0x7f3a380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.522+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3a380b29f0 0x7f3a380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.522+0000 7f3a43fff700 1 -- 172.21.15.115:0/3821840129 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f3a4400f020 con 0x7f3a4c1094d0 2024-05-09T05:48:33.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.522+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3a380b29f0 0x7f3a380b4f00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c06c060 tx=0x7f3a3c00a380 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:33.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.522+0000 7f3a5f97eb80 1 Processor -- start 2024-05-09T05:48:33.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a5f97eb80 1 -- start start 2024-05-09T05:48:33.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 0x559ed846ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 0x559ed846b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8475900 0x559ed8477d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559ed838e1a0 con 0x559ed8475900 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559ed8333aa0 con 0x559ed82ee5a0 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559ed838a160 con 0x559ed8368e00 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8475900 0x559ed8477d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8475900 0x559ed8477d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59038/0 (socket says 172.21.15.115:59038) 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a4ad9d700 1 -- 172.21.15.115:0/994861473 learned_addr learned my addr 172.21.15.115:0/994861473 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 0x559ed846b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 0x559ed846ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a41d9b700 1 -- 172.21.15.115:0/994861473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 msgr2=0x559ed846ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 0x559ed846ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a41d9b700 1 -- 172.21.15.115:0/994861473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8475900 msgr2=0x559ed8477d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:33.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8475900 0x559ed8477d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.523+0000 7f3a41d9b700 1 -- 172.21.15.115:0/994861473 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed8324600 con 0x559ed82ee5a0 2024-05-09T05:48:33.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.524+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8475900 0x559ed8477d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:48:33.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.524+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 0x559ed846b190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a34005950 tx=0x7f3a3400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb422e4dfed72069 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:33.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.524+0000 7f3a40d99700 1 -- 172.21.15.115:0/994861473 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34010c70 con 0x559ed82ee5a0 2024-05-09T05:48:33.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.524+0000 7f3a40d99700 1 -- 172.21.15.115:0/994861473 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a34010e10 con 0x559ed82ee5a0 2024-05-09T05:48:33.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.524+0000 7f3a40d99700 1 -- 172.21.15.115:0/994861473 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34005020 con 0x559ed82ee5a0 2024-05-09T05:48:33.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/994861473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 msgr2=0x559ed846b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:33.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 0x559ed846b190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a34005950 tx=0x7f3a3400bf60 comp rx=0 tx=0).stop 2024-05-09T05:48:33.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/994861473 shutdown_connections 2024-05-09T05:48:33.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8475900 0x559ed8477d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 0x559ed846ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/994861473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 0x559ed846b190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/994861473 >> 172.21.15.115:0/994861473 conn(0x559ed82f2070 msgr2=0x559ed8466b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/994861473 shutdown_connections 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/994861473 wait complete. 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.525+0000 7f3a5f97eb80 1 -- start start 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a5f97eb80 1 Processor -- start 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed8472820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 0x559ed8472d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8475900 0x559ed84732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559ed8478320 con 0x559ed82ee5a0 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559ed8328c20 con 0x559ed8475900 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559ed84784a0 con 0x559ed8368e00 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a5f97eb80 10 client.0 _ll_register_callbacks cb 0x559ed81ff7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a4a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed8472820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a4a59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed8472820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59042/0 (socket says 172.21.15.115:59042) 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a4a59c700 1 -- 172.21.15.115:0/1146609823 learned_addr learned my addr 172.21.15.115:0/1146609823 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8475900 0x559ed84732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 0x559ed8472d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a4a59c700 1 -- 172.21.15.115:0/1146609823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 msgr2=0x559ed8472d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 0x559ed8472d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a4a59c700 1 -- 172.21.15.115:0/1146609823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8475900 msgr2=0x559ed84732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8475900 0x559ed84732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a4a59c700 1 -- 172.21.15.115:0/1146609823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed8209590 con 0x559ed82ee5a0 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.526+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 0x559ed8472d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed8472820 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c003930 tx=0x7f3a3c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76235ef13fd0c7bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1146609823 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c016070 con 0x559ed82ee5a0 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1146609823 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f3a3c010ac0 con 0x559ed82ee5a0 2024-05-09T05:48:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1146609823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a34010e10 con 0x559ed82ee5a0 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1146609823 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3a3c030050 con 0x559ed82ee5a0 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a5f97eb80 10 client.18237 fetch_fsmap learned FSMap version 49 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1146609823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559ed8482f00 con 0x559ed82ee5a0 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1146609823 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a34010e10 con 0x559ed82ee5a0 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1146609823 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8374+0+0 (secure 0 0 0) 0x7f3a3c01ebd0 con 0x559ed82ee5a0 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a5f97eb80 10 client.18237 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a5f97eb80 10 client.18237 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a5f97eb80 20 client.18237 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.527+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559ed8485540 0x559ed84879c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.528+0000 7f3a5f97eb80 4 client.18237 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.528+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1146609823 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ed839bb80 con 0x559ed8485540 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.528+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559ed8485540 0x559ed84879c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.528+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559ed8485540 0x559ed84879c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc40d6bed8c9297 server_cookie=6e20573ae1fe2143 in_seq=0 out_seq=0 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.528+0000 7f3a31ffb700 10 client.18237 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1146609823 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ed839bb80 con 0x559ed8485540 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a31ffb700 10 client.18237 handle_command_reply: tid=0 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 client.18237 shutdown 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1146609823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 msgr2=0x559ed8472820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed8472820 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c003930 tx=0x7f3a3c0047c0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1146609823 shutdown_connections 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8475900 0x559ed84732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559ed8485540 0x559ed84879c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed8472820 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a31ffb700 0 client.18237 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1146609823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed8368e00 0x559ed8472d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1146609823 >> 172.21.15.115:0/1146609823 conn(0x559ed82f2070 msgr2=0x559ed8465de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1146609823 shutdown_connections 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1146609823 wait complete. 2024-05-09T05:48:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.529+0000 7f3a5f97eb80 20 client.18237 trim_cache size 0 max 16384 2024-05-09T05:48:33.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a5f97eb80 1 Processor -- start 2024-05-09T05:48:33.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a5f97eb80 1 -- start start 2024-05-09T05:48:33.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 0x559ed8474900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed84813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8368e00 0x559ed847d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b02b0 con 0x559ed82ee5a0 2024-05-09T05:48:33.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b1e80 con 0x559ed8368e00 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b0980 con 0x559ed82f2070 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a41d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed84813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a41d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed84813b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59048/0 (socket says 172.21.15.115:59048) 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.549+0000 7f3a41d9b700 1 -- 172.21.15.115:0/1554417074 learned_addr learned my addr 172.21.15.115:0/1554417074 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 0x559ed8474900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8368e00 0x559ed847d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a41d9b700 1 -- 172.21.15.115:0/1554417074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 msgr2=0x559ed8474900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 0x559ed8474900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a41d9b700 1 -- 172.21.15.115:0/1554417074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8368e00 msgr2=0x559ed847d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8368e00 0x559ed847d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a41d9b700 1 -- 172.21.15.115:0/1554417074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed8470d90 con 0x559ed82ee5a0 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8368e00 0x559ed847d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed84813b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400b6c0 tx=0x7f3a34010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d5c6390c3acb6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a33fff700 1 -- 172.21.15.115:0/1554417074 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3402c070 con 0x559ed82ee5a0 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a33fff700 1 -- 172.21.15.115:0/1554417074 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a340056d0 con 0x559ed82ee5a0 2024-05-09T05:48:33.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.550+0000 7f3a33fff700 1 -- 172.21.15.115:0/1554417074 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34004920 con 0x559ed82ee5a0 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1554417074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 msgr2=0x559ed84813b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed84813b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400b6c0 tx=0x7f3a34010c00 comp rx=0 tx=0).stop 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1554417074 shutdown_connections 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed8368e00 0x559ed847d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 0x559ed8474900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1554417074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed82ee5a0 0x559ed84813b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1554417074 >> 172.21.15.115:0/1554417074 conn(0x559ed845fd50 msgr2=0x559ed8465de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1554417074 shutdown_connections 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1554417074 wait complete. 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 -- start start 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.551+0000 7f3a5f97eb80 1 Processor -- start 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 0x559ed84786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 0x559ed8478c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a5f97eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8368e00 0x559ed84efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b17b0 con 0x559ed8368e00 2024-05-09T05:48:33.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b1050 con 0x559ed82ee5a0 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a5f97eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1ae270 con 0x559ed82f2070 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a5f97eb80 10 client.0 _ll_register_callbacks cb 0x559ed81ff7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8368e00 0x559ed84efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8368e00 0x559ed84efdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59050/0 (socket says 172.21.15.115:59050) 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4ad9d700 1 -- 172.21.15.115:0/1055700542 learned_addr learned my addr 172.21.15.115:0/1055700542 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a41d9b700 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 0x559ed8478c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 0x559ed84786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4ad9d700 1 -- 172.21.15.115:0/1055700542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 msgr2=0x559ed8478c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 0x559ed8478c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4ad9d700 1 -- 172.21.15.115:0/1055700542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 msgr2=0x559ed84786f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 0x559ed84786f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4ad9d700 1 -- 172.21.15.115:0/1055700542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed84f0300 con 0x559ed8368e00 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a4ad9d700 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8368e00 0x559ed84efdc0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009900 tx=0x7f3a440ab450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f585fe3989d6b71a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:33.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1055700542 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44007d30 con 0x559ed8368e00 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1055700542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a340056d0 con 0x559ed8368e00 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.552+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1055700542 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f3a440aa3e0 con 0x559ed8368e00 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1055700542 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3a440aaa80 con 0x559ed8368e00 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a5f97eb80 10 client.18243 fetch_fsmap learned FSMap version 49 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1055700542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559ed84f13c0 con 0x559ed8368e00 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1055700542 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a340056d0 con 0x559ed8368e00 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1055700542 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 49) ==== 8374+0+0 (secure 0 0 0) 0x7f3a44093040 con 0x559ed8368e00 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a5f97eb80 10 client.18243 fetch_fsmap finished waiting for FSMap version 49 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a5f97eb80 10 client.18243 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a5f97eb80 20 client.18243 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559ed8475900 0x559ed84f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a5f97eb80 4 client.18243 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6f88124", "release": true, "await": true}] 2024-05-09T05:48:33.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1055700542 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6f88124", "release": true, "await": true}) -- 0x559ed8395c20 con 0x559ed8475900 2024-05-09T05:48:33.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.553+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559ed8475900 0x559ed84f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:33.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.554+0000 7f3a4a59c700 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559ed8475900 0x559ed84f3b20 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1406f9915c632f server_cookie=54340d8079f6059d in_seq=0 out_seq=0 2024-05-09T05:48:33.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.554+0000 7f3a31ffb700 10 client.18243 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:33.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.557+0000 7f3a31ffb700 1 -- 172.21.15.115:0/1055700542 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559ed8395c20 con 0x559ed8475900 2024-05-09T05:48:33.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.557+0000 7f3a31ffb700 10 client.18243 handle_command_reply: tid=0 2024-05-09T05:48:33.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.557+0000 7f3a5f97eb80 1 client.18243 shutdown 2024-05-09T05:48:33.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1055700542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8368e00 msgr2=0x559ed84efdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:33.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8368e00 0x559ed84efdc0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009900 tx=0x7f3a440ab450 comp rx=0 tx=0).stop 2024-05-09T05:48:33.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1055700542 shutdown_connections 2024-05-09T05:48:33.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559ed82ee5a0 0x559ed84786f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559ed8475900 0x559ed84f3b20 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559ed8368e00 0x559ed84efdc0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 --2- 172.21.15.115:0/1055700542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ed82f2070 0x559ed8478c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1055700542 >> 172.21.15.115:0/1055700542 conn(0x559ed845fd50 msgr2=0x559ed8474550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:33.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a31ffb700 0 client.18243 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:33.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1055700542 shutdown_connections 2024-05-09T05:48:33.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 1 -- 172.21.15.115:0/1055700542 wait complete. 2024-05-09T05:48:33.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a5f97eb80 20 client.18243 trim_cache size 0 max 16384 2024-05-09T05:48:33.553 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:48:33.553 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 49, 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 32, 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "6f88124": { 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 32, 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:48:33.554 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:48:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:48:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:48:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:48:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:48:33.555 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:33.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:33.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:48:33.558 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:48:33.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a50f68700 1 -- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3a380b29f0 msgr2=0x7f3a380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:33.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a50f68700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3a380b29f0 0x7f3a380b4f00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c06c060 tx=0x7f3a3c00a380 comp rx=0 tx=0).stop 2024-05-09T05:48:33.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a50f68700 1 -- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1094d0 msgr2=0x7f3a4c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:33.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.558+0000 7f3a50f68700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1094d0 0x7f3a4c19f680 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b830 tx=0x7f3a44004d00 comp rx=0 tx=0).stop 2024-05-09T05:48:33.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.559+0000 7f3a50f68700 1 -- 172.21.15.115:0/3821840129 shutdown_connections 2024-05-09T05:48:33.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.559+0000 7f3a50f68700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3a380b29f0 0x7f3a380b4f00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.559+0000 7f3a50f68700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a4c1094d0 0x7f3a4c19f680 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.559+0000 7f3a50f68700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a4c103990 0x7f3a4c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.559+0000 7f3a50f68700 1 --2- 172.21.15.115:0/3821840129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a4c1042e0 0x7f3a4c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:33.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.559+0000 7f3a50f68700 1 -- 172.21.15.115:0/3821840129 >> 172.21.15.115:0/3821840129 conn(0x7f3a4c0fe4f0 msgr2=0x7f3a4c102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:33.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.559+0000 7f3a50f68700 1 -- 172.21.15.115:0/3821840129 shutdown_connections 2024-05-09T05:48:33.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:48:33.559+0000 7f3a50f68700 1 -- 172.21.15.115:0/3821840129 wait complete. 2024-05-09T05:48:33.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:33 smithi195 ceph-mon[33844]: pgmap v323: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-09T05:48:33.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:33 smithi161 ceph-mon[33805]: pgmap v323: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-09T05:48:34.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:34 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:48:34.383 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6f88124', total seconds elapsed: 36.1 2024-05-09T05:48:34.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:34 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:48:34.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:34 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:48:35.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:35 smithi195 ceph-mon[33844]: pgmap v324: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-05-09T05:48:35.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:35 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:48:35.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:35 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:35 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:35 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:35 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:35 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:35 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:35 smithi115 ceph-mon[30349]: pgmap v324: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-05-09T05:48:35.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:35 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:48:35.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:35 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:35 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:35 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:35 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:35 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:35 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:35 smithi161 ceph-mon[33805]: pgmap v324: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.6 KiB/s wr, 14 op/s 2024-05-09T05:48:35.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:35 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:48:35.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:35 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:35 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:35 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:35 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:35 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:35.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:35 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.309+0000 7f376396c640 1 -- 172.21.15.115:0/351118849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c108ba0 msgr2=0x7f375c102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.309+0000 7f376396c640 1 --2- 172.21.15.115:0/351118849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c108ba0 0x7f375c102740 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3758009900 tx=0x7f375802a410 comp rx=0 tx=0).stop 2024-05-09T05:48:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.309+0000 7f376396c640 1 -- 172.21.15.115:0/351118849 shutdown_connections 2024-05-09T05:48:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.309+0000 7f376396c640 1 --2- 172.21.15.115:0/351118849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f375c1032f0 0x7f375c10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.309+0000 7f376396c640 1 --2- 172.21.15.115:0/351118849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f375c109040 0x7f375c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.309+0000 7f376396c640 1 --2- 172.21.15.115:0/351118849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c108ba0 0x7f375c102740 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.309+0000 7f376396c640 1 -- 172.21.15.115:0/351118849 >> 172.21.15.115:0/351118849 conn(0x7f375c0782e0 msgr2=0x7f375c078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.309+0000 7f376396c640 1 -- 172.21.15.115:0/351118849 shutdown_connections 2024-05-09T05:48:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.309+0000 7f376396c640 1 -- 172.21.15.115:0/351118849 wait complete. 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.310+0000 7f376396c640 1 Processor -- start 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.310+0000 7f376396c640 1 -- start start 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.310+0000 7f376396c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c1032f0 0x7f375c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.310+0000 7f376396c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f375c108ba0 0x7f375c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.310+0000 7f376396c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f375c109040 0x7f375c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.310+0000 7f376396c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f375c112c40 con 0x7f375c1032f0 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.310+0000 7f376396c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f375c112ac0 con 0x7f375c108ba0 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.310+0000 7f376396c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f375c112dc0 con 0x7f375c109040 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.310+0000 7f3761708640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c1032f0 0x7f375c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3761708640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c1032f0 0x7f375c199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59064/0 (socket says 172.21.15.115:59064) 2024-05-09T05:48:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3761708640 1 -- 172.21.15.115:0/1108495791 learned_addr learned my addr 172.21.15.115:0/1108495791 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3760f07640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f375c108ba0 0x7f375c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3761f09640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f375c109040 0x7f375c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3761708640 1 -- 172.21.15.115:0/1108495791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f375c109040 msgr2=0x7f375c199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3761708640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f375c109040 0x7f375c199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3761708640 1 -- 172.21.15.115:0/1108495791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f375c108ba0 msgr2=0x7f375c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3761708640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f375c108ba0 0x7f375c199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3761708640 1 -- 172.21.15.115:0/1108495791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c1a8ae0 con 0x7f375c1032f0 2024-05-09T05:48:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3760f07640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f375c108ba0 0x7f375c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:48:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.311+0000 7f3761708640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c1032f0 0x7f375c199430 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3758005940 tx=0x7f3758005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.315+0000 7f37527fc640 1 -- 172.21.15.115:0/1108495791 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375803f030 con 0x7f375c1032f0 2024-05-09T05:48:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.315+0000 7f376396c640 1 -- 172.21.15.115:0/1108495791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c1a8d70 con 0x7f375c1032f0 2024-05-09T05:48:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.315+0000 7f37527fc640 1 -- 172.21.15.115:0/1108495791 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37580043d0 con 0x7f375c1032f0 2024-05-09T05:48:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.315+0000 7f37527fc640 1 -- 172.21.15.115:0/1108495791 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758035070 con 0x7f375c1032f0 2024-05-09T05:48:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.315+0000 7f376396c640 1 -- 172.21.15.115:0/1108495791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c1a9250 con 0x7f375c1032f0 2024-05-09T05:48:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.318+0000 7f376a81c740 1 Processor -- start 2024-05-09T05:48:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.318+0000 7f37527fc640 1 -- 172.21.15.115:0/1108495791 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f3758042030 con 0x7f375c1032f0 2024-05-09T05:48:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.318+0000 7f376a81c740 1 -- start start 2024-05-09T05:48:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f376a81c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f84195680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f376a81c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f84195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f376a81c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 0x558f84196100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f376a81c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558f840fa7c0 con 0x558f83df21b0 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f376a81c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f840960e0 con 0x558f83dc92f0 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f376a81c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558f840f6780 con 0x558f83dd14d0 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3761f09640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f84195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3761708640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f84195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3761f09640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f84195680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59072/0 (socket says 172.21.15.115:59072) 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3761f09640 1 -- 172.21.15.115:0/498107224 learned_addr learned my addr 172.21.15.115:0/498107224 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3760f07640 1 --2- 172.21.15.115:0/498107224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 0x558f84196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3761f09640 1 -- 172.21.15.115:0/498107224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 msgr2=0x558f84196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3761f09640 1 --2- 172.21.15.115:0/498107224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 0x558f84196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3761f09640 1 -- 172.21.15.115:0/498107224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 msgr2=0x558f84195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3761f09640 1 --2- 172.21.15.115:0/498107224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f84195bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f3761f09640 1 -- 172.21.15.115:0/498107224 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f840bf7c0 con 0x558f83df21b0 2024-05-09T05:48:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.319+0000 7f37527fc640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37480b29d0 0x7f37480b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.320+0000 7f3761708640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37480b29d0 0x7f37480b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.320+0000 7f3761f09640 1 --2- 172.21.15.115:0/498107224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f84195680 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f3754009fd0 tx=0x7f37540118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe26428d0b217c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.320+0000 7f37477fe640 1 -- 172.21.15.115:0/498107224 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754004d20 con 0x558f83df21b0 2024-05-09T05:48:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.320+0000 7f37527fc640 1 -- 172.21.15.115:0/1108495791 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f3758102c80 con 0x7f375c1032f0 2024-05-09T05:48:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.320+0000 7f37477fe640 1 -- 172.21.15.115:0/498107224 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3754004050 con 0x558f83df21b0 2024-05-09T05:48:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.320+0000 7f37477fe640 1 -- 172.21.15.115:0/498107224 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754005260 con 0x558f83df21b0 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f376a81c740 1 -- 172.21.15.115:0/498107224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 msgr2=0x558f84195680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f376a81c740 1 --2- 172.21.15.115:0/498107224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f84195680 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f3754009fd0 tx=0x7f37540118d0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f3761708640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37480b29d0 0x7f37480b4ee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37581000f0 tx=0x7f37581067d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f376a81c740 1 -- 172.21.15.115:0/498107224 shutdown_connections 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f376a81c740 1 --2- 172.21.15.115:0/498107224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 0x558f84196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f376a81c740 1 --2- 172.21.15.115:0/498107224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f84195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f376a81c740 1 --2- 172.21.15.115:0/498107224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f84195680 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f376a81c740 1 -- 172.21.15.115:0/498107224 >> 172.21.15.115:0/498107224 conn(0x558f83de6300 msgr2=0x558f84191e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f376a81c740 1 -- 172.21.15.115:0/498107224 shutdown_connections 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.321+0000 7f376a81c740 1 -- 172.21.15.115:0/498107224 wait complete. 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.322+0000 7f376a81c740 1 -- start start 2024-05-09T05:48:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.322+0000 7f376a81c740 1 Processor -- start 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.322+0000 7f376a81c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f8419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.322+0000 7f376a81c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 0x558f841a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.322+0000 7f376a81c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f841a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.322+0000 7f376a81c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558f841a27b0 con 0x558f83df21b0 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.322+0000 7f376a81c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f840819c0 con 0x558f83dc92f0 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.322+0000 7f376a81c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558f841a2930 con 0x558f83dd14d0 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f376a81c740 10 client.0 _ll_register_callbacks cb 0x558f840ae020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3761f09640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f841a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3761f09640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f841a5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59080/0 (socket says 172.21.15.115:59080) 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3761f09640 1 -- 172.21.15.115:0/2028430712 learned_addr learned my addr 172.21.15.115:0/2028430712 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3760f07640 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 0x558f841a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3761f09640 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f8419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3760f07640 1 -- 172.21.15.115:0/2028430712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 msgr2=0x558f8419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3760f07640 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f8419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3760f07640 1 -- 172.21.15.115:0/2028430712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 msgr2=0x558f841a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3760f07640 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f841a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3760f07640 1 -- 172.21.15.115:0/2028430712 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f840a81f0 con 0x558f83dd14d0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3761f09640 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f8419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3760f07640 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 0x558f841a0210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f375c19b590 tx=0x7f374c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e56b8e14dacf4ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.323+0000 7f3744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f3744ff9640 1 -- 172.21.15.115:0/2028430712 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c0120d0 con 0x558f83dd14d0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f3744ff9640 1 -- 172.21.15.115:0/2028430712 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f374c004030 con 0x558f83dd14d0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f376a81c740 1 -- 172.21.15.115:0/2028430712 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3754004050 con 0x558f83dd14d0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f3744ff9640 1 -- 172.21.15.115:0/2028430712 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f374c025050 con 0x558f83dd14d0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f376a81c740 10 client.27601 fetch_fsmap learned FSMap version 50 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f376a81c740 1 -- 172.21.15.115:0/2028430712 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558f83e5d790 con 0x558f83dd14d0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f3744ff9640 1 -- 172.21.15.115:0/2028430712 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3754004050 con 0x558f83dd14d0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f3744ff9640 1 -- 172.21.15.115:0/2028430712 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7f374c003c10 con 0x558f83dd14d0 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f376a81c740 10 client.27601 fetch_fsmap finished waiting for FSMap version 50 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f376a81c740 10 client.27601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f376a81c740 20 client.27601 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f376a81c740 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558f83db0d80 0x558f841b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f376a81c740 4 client.27601 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:48:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.324+0000 7f376a81c740 1 -- 172.21.15.115:0/2028430712 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f841083d0 con 0x558f83db0d80 2024-05-09T05:48:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.325+0000 7f3761f09640 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558f83db0d80 0x558f841b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.325+0000 7f3744ff9640 10 client.27601 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.325+0000 7f3761f09640 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558f83db0d80 0x558f841b2000 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e323a264e46d2d8 server_cookie=711c7fdd47ad0acc in_seq=0 out_seq=0 2024-05-09T05:48:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f3744ff9640 1 -- 172.21.15.115:0/2028430712 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f841083d0 con 0x558f83db0d80 2024-05-09T05:48:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f3744ff9640 10 client.27601 handle_command_reply: tid=0 2024-05-09T05:48:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f376a81c740 1 client.27601 shutdown 2024-05-09T05:48:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f376a81c740 1 -- 172.21.15.115:0/2028430712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 msgr2=0x558f841a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f376a81c740 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 0x558f841a0210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f375c19b590 tx=0x7f374c00f790 comp rx=0 tx=0).stop 2024-05-09T05:48:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f376a81c740 1 -- 172.21.15.115:0/2028430712 shutdown_connections 2024-05-09T05:48:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f376a81c740 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83df21b0 0x558f841a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f376a81c740 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558f83db0d80 0x558f841b2000 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f376a81c740 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dd14d0 0x558f841a0210 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f376a81c740 1 --2- 172.21.15.115:0/2028430712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f8419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f3744ff9640 0 client.27601 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.326+0000 7f376a81c740 1 -- 172.21.15.115:0/2028430712 >> 172.21.15.115:0/2028430712 conn(0x558f83de6300 msgr2=0x558f84190700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.327+0000 7f376a81c740 1 -- 172.21.15.115:0/2028430712 shutdown_connections 2024-05-09T05:48:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.327+0000 7f376a81c740 1 -- 172.21.15.115:0/2028430712 wait complete. 2024-05-09T05:48:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.327+0000 7f376a81c740 20 client.27601 trim_cache size 0 max 16384 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.351+0000 7f376a81c740 1 Processor -- start 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.351+0000 7f376a81c740 1 -- start start 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f376a81c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83db0d80 0x558f8419a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f376a81c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dc92f0 0x558f841a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f376a81c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dd14d0 0x558f8419e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f376a81c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1b3460 con 0x558f83db0d80 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f376a81c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1b4fd0 con 0x558f83dd14d0 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f376a81c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1b3b30 con 0x558f83dc92f0 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83db0d80 0x558f8419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83db0d80 0x558f8419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59084/0 (socket says 172.21.15.115:59084) 2024-05-09T05:48:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 -- 172.21.15.115:0/2485559261 learned_addr learned my addr 172.21.15.115:0/2485559261 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3760f07640 1 --2- 172.21.15.115:0/2485559261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dc92f0 0x558f841a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 --2- 172.21.15.115:0/2485559261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dd14d0 0x558f8419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 -- 172.21.15.115:0/2485559261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dc92f0 msgr2=0x558f841a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 --2- 172.21.15.115:0/2485559261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dc92f0 0x558f841a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 -- 172.21.15.115:0/2485559261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dd14d0 msgr2=0x558f8419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 --2- 172.21.15.115:0/2485559261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dd14d0 0x558f8419e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 -- 172.21.15.115:0/2485559261 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f83ed45f0 con 0x558f83db0d80 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.352+0000 7f3761f09640 1 --2- 172.21.15.115:0/2485559261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83db0d80 0x558f8419a380 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f37540048b0 tx=0x7f37540048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c863cac4ffc7ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.353+0000 7f3746ffd640 1 -- 172.21.15.115:0/2485559261 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754034020 con 0x558f83db0d80 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.353+0000 7f3746ffd640 1 -- 172.21.15.115:0/2485559261 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3754004050 con 0x558f83db0d80 2024-05-09T05:48:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.353+0000 7f3746ffd640 1 -- 172.21.15.115:0/2485559261 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375400fce0 con 0x558f83db0d80 2024-05-09T05:48:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 -- 172.21.15.115:0/2485559261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83db0d80 msgr2=0x558f8419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 --2- 172.21.15.115:0/2485559261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83db0d80 0x558f8419a380 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f37540048b0 tx=0x7f37540048e0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 -- 172.21.15.115:0/2485559261 shutdown_connections 2024-05-09T05:48:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 --2- 172.21.15.115:0/2485559261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dd14d0 0x558f8419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 --2- 172.21.15.115:0/2485559261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83dc92f0 0x558f841a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 --2- 172.21.15.115:0/2485559261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83db0d80 0x558f8419a380 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 -- 172.21.15.115:0/2485559261 >> 172.21.15.115:0/2485559261 conn(0x558f83de6300 msgr2=0x558f8419f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 -- 172.21.15.115:0/2485559261 shutdown_connections 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 -- 172.21.15.115:0/2485559261 wait complete. 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 -- start start 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.354+0000 7f376a81c740 1 Processor -- start 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f376a81c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83db0d80 0x558f841a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f376a81c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f841a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f376a81c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83dd14d0 0x558f84218360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f376a81c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1b4900 con 0x558f83dd14d0 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f376a81c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1b4200 con 0x558f83dc92f0 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f376a81c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1b1460 con 0x558f83db0d80 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f376a81c740 10 client.0 _ll_register_callbacks cb 0x7f374c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3761f09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83db0d80 0x558f841a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3760f07640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f841a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3760f07640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f841a7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:46304/0 (socket says 172.21.15.115:46304) 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3761f09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83db0d80 0x558f841a75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:53506/0 (socket says 172.21.15.115:53506) 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3760f07640 1 -- 172.21.15.115:0/1966438585 learned_addr learned my addr 172.21.15.115:0/1966438585 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3761f09640 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83dd14d0 0x558f84218360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3760f07640 1 -- 172.21.15.115:0/1966438585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83db0d80 msgr2=0x558f841a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3760f07640 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83db0d80 0x558f841a75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3760f07640 1 -- 172.21.15.115:0/1966438585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83dd14d0 msgr2=0x558f84218360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3760f07640 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83dd14d0 0x558f84218360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.355+0000 7f3760f07640 1 -- 172.21.15.115:0/1966438585 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f840beed0 con 0x558f83dc92f0 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.356+0000 7f3760f07640 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f841a7b30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x558f8418bee0 tx=0x7f374c0120d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c737bce51bee137 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.356+0000 7f3744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.356+0000 7f3744ff9640 1 -- 172.21.15.115:0/1966438585 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c022040 con 0x558f83dc92f0 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.356+0000 7f376a81c740 1 -- 172.21.15.115:0/1966438585 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3754004050 con 0x558f83dc92f0 2024-05-09T05:48:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.356+0000 7f3744ff9640 1 -- 172.21.15.115:0/1966438585 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f374c0052d0 con 0x558f83dc92f0 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.356+0000 7f3744ff9640 1 -- 172.21.15.115:0/1966438585 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f374c037050 con 0x558f83dc92f0 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.356+0000 7f376a81c740 10 client.27509 fetch_fsmap learned FSMap version 50 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.356+0000 7f376a81c740 1 -- 172.21.15.115:0/1966438585 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558f84219970 con 0x558f83dc92f0 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.357+0000 7f3744ff9640 1 -- 172.21.15.115:0/1966438585 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3754004050 con 0x558f83dc92f0 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.357+0000 7f3744ff9640 1 -- 172.21.15.115:0/1966438585 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7f374c01ccc0 con 0x558f83dc92f0 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.357+0000 7f376a81c740 10 client.27509 fetch_fsmap finished waiting for FSMap version 50 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.357+0000 7f376a81c740 10 client.27509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.357+0000 7f376a81c740 20 client.27509 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.357+0000 7f376a81c740 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558f83df21b0 0x558f8421c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.357+0000 7f376a81c740 4 client.27509 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.357+0000 7f376a81c740 1 -- 172.21.15.115:0/1966438585 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f84102240 con 0x558f83df21b0 2024-05-09T05:48:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.357+0000 7f3761f09640 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558f83df21b0 0x558f8421c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.358+0000 7f3744ff9640 10 client.27509 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.358+0000 7f3761f09640 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558f83df21b0 0x558f8421c1e0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2555047c4bdf696f server_cookie=85aa9440ed2e7271 in_seq=0 out_seq=0 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f3744ff9640 1 -- 172.21.15.115:0/1966438585 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558f84102240 con 0x558f83df21b0 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f3744ff9640 10 client.27509 handle_command_reply: tid=0 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 client.27509 shutdown 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 -- 172.21.15.115:0/1966438585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 msgr2=0x558f841a7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f841a7b30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x558f8418bee0 tx=0x7f374c0120d0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 -- 172.21.15.115:0/1966438585 shutdown_connections 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558f83dd14d0 0x558f84218360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f83dc92f0 0x558f841a7b30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558f83df21b0 0x558f8421c1e0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 --2- 172.21.15.115:0/1966438585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f83db0d80 0x558f841a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f3744ff9640 0 client.27509 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 -- 172.21.15.115:0/1966438585 >> 172.21.15.115:0/1966438585 conn(0x558f83de6300 msgr2=0x558f841a1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 -- 172.21.15.115:0/1966438585 shutdown_connections 2024-05-09T05:48:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 1 -- 172.21.15.115:0/1966438585 wait complete. 2024-05-09T05:48:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.362+0000 7f376a81c740 20 client.27509 trim_cache size 0 max 16384 2024-05-09T05:48:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:48:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:48:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:48:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 -- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37480b29d0 msgr2=0x7f37480b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37480b29d0 0x7f37480b4ee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37581000f0 tx=0x7f37581067d0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 -- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c1032f0 msgr2=0x7f375c199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c1032f0 0x7f375c199430 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3758005940 tx=0x7f3758005e30 comp rx=0 tx=0).stop 2024-05-09T05:48:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 -- 172.21.15.115:0/1108495791 shutdown_connections 2024-05-09T05:48:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37480b29d0 0x7f37480b4ee0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f375c109040 0x7f375c199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f375c108ba0 0x7f375c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 --2- 172.21.15.115:0/1108495791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f375c1032f0 0x7f375c199430 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 -- 172.21.15.115:0/1108495791 >> 172.21.15.115:0/1108495791 conn(0x7f375c0782e0 msgr2=0x7f375c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 -- 172.21.15.115:0/1108495791 shutdown_connections 2024-05-09T05:48:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:36.363+0000 7f376396c640 1 -- 172.21.15.115:0/1108495791 wait complete. 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: pgmap v325: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:48:37.865 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:37 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: pgmap v325: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:48:37.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:48:37.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:37 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: pgmap v325: 129 pgs: 129 active+clean; 8.3 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-05-09T05:48:37.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:48:37.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:48:37.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:48:37.910 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:37 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:48:38.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:38 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-09T05:48:38.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:38 smithi195 ceph-mon[33844]: Unable to set osd_memory_target on smithi195 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T05:48:38.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:38 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-05-09T05:48:38.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:38 smithi195 ceph-mon[33844]: Unable to set osd_memory_target on smithi161 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T05:48:38.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:38 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi115 to 176.1M 2024-05-09T05:48:38.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:38 smithi195 ceph-mon[33844]: Unable to set osd_memory_target on smithi115 to 184750899: error parsing value: Value '184750899' is below minimum 939524096 2024-05-09T05:48:38.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:38 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-09T05:48:38.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:38 smithi115 ceph-mon[30349]: Unable to set osd_memory_target on smithi195 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T05:48:38.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:38 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-05-09T05:48:38.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:38 smithi115 ceph-mon[30349]: Unable to set osd_memory_target on smithi161 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T05:48:38.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:38 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi115 to 176.1M 2024-05-09T05:48:38.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:38 smithi115 ceph-mon[30349]: Unable to set osd_memory_target on smithi115 to 184750899: error parsing value: Value '184750899' is below minimum 939524096 2024-05-09T05:48:38.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:38 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-09T05:48:38.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:38 smithi161 ceph-mon[33805]: Unable to set osd_memory_target on smithi195 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T05:48:38.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:38 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-05-09T05:48:38.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:38 smithi161 ceph-mon[33805]: Unable to set osd_memory_target on smithi161 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T05:48:38.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:38 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi115 to 176.1M 2024-05-09T05:48:38.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:38 smithi161 ceph-mon[33805]: Unable to set osd_memory_target on smithi115 to 184750899: error parsing value: Value '184750899' is below minimum 939524096 2024-05-09T05:48:39.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:39 smithi195 ceph-mon[33844]: pgmap v326: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 MiB/s wr, 18 op/s 2024-05-09T05:48:39.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:39 smithi115 ceph-mon[30349]: pgmap v326: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 MiB/s wr, 18 op/s 2024-05-09T05:48:39.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:39 smithi161 ceph-mon[33805]: pgmap v326: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 MiB/s wr, 18 op/s 2024-05-09T05:48:41.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:41 smithi115 ceph-mon[30349]: pgmap v327: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 12 op/s 2024-05-09T05:48:41.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:41 smithi195 ceph-mon[33844]: pgmap v327: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 12 op/s 2024-05-09T05:48:41.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:41 smithi161 ceph-mon[33805]: pgmap v327: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 12 op/s 2024-05-09T05:48:43.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:43 smithi195 ceph-mon[33844]: pgmap v328: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 12 op/s 2024-05-09T05:48:43.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:43 smithi115 ceph-mon[30349]: pgmap v328: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 12 op/s 2024-05-09T05:48:43.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:43 smithi161 ceph-mon[33805]: pgmap v328: 129 pgs: 129 active+clean; 8.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 12 op/s 2024-05-09T05:48:45.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:45 smithi195 ceph-mon[33844]: pgmap v329: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 23 op/s 2024-05-09T05:48:45.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:45 smithi115 ceph-mon[30349]: pgmap v329: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 23 op/s 2024-05-09T05:48:45.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:45 smithi161 ceph-mon[33805]: pgmap v329: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 23 op/s 2024-05-09T05:48:47.710 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:47 smithi161 ceph-mon[33805]: pgmap v330: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-05-09T05:48:47.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:47 smithi195 ceph-mon[33844]: pgmap v330: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-05-09T05:48:47.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:47 smithi115 ceph-mon[30349]: pgmap v330: 129 pgs: 129 active+clean; 8.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-05-09T05:48:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 -- 172.21.15.115:0/127937846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e0109040 msgr2=0x7f66e0102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 --2- 172.21.15.115:0/127937846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e0109040 0x7f66e0102c80 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009900 tx=0x7f66d402a410 comp rx=0 tx=0).stop 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 -- 172.21.15.115:0/127937846 shutdown_connections 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 --2- 172.21.15.115:0/127937846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66e01032f0 0x7f66e010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 --2- 172.21.15.115:0/127937846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e0109040 0x7f66e0102c80 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 --2- 172.21.15.115:0/127937846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e0108ba0 0x7f66e0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 -- 172.21.15.115:0/127937846 >> 172.21.15.115:0/127937846 conn(0x7f66e00782e0 msgr2=0x7f66e0078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 -- 172.21.15.115:0/127937846 shutdown_connections 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 -- 172.21.15.115:0/127937846 wait complete. 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.483+0000 7f66e6042640 1 Processor -- start 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66e6042640 1 -- start start 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66e6042640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e01032f0 0x7f66e01a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66e6042640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66e0108ba0 0x7f66e01a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66e6042640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e0109040 0x7f66e019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66e6042640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0112c40 con 0x7f66e01032f0 2024-05-09T05:48:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66e6042640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0112ac0 con 0x7f66e0109040 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66e6042640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0112dc0 con 0x7f66e0108ba0 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e01032f0 0x7f66e01a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e01032f0 0x7f66e01a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40018/0 (socket says 172.21.15.115:40018) 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66df7fe640 1 -- 172.21.15.115:0/517793340 learned_addr learned my addr 172.21.15.115:0/517793340 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66dffff640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e0109040 0x7f66e019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66deffd640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66e0108ba0 0x7f66e01a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66df7fe640 1 -- 172.21.15.115:0/517793340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66e0108ba0 msgr2=0x7f66e01a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66df7fe640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66e0108ba0 0x7f66e01a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66df7fe640 1 -- 172.21.15.115:0/517793340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e0109040 msgr2=0x7f66e019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66df7fe640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e0109040 0x7f66e019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.484+0000 7f66df7fe640 1 -- 172.21.15.115:0/517793340 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e019ae30 con 0x7f66e01032f0 2024-05-09T05:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.485+0000 7f66df7fe640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e01032f0 0x7f66e01a0580 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f66d0005fd0 tx=0x7f66d000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.485+0000 7f66dcff9640 1 -- 172.21.15.115:0/517793340 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0017070 con 0x7f66e01032f0 2024-05-09T05:48:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.485+0000 7f66dcff9640 1 -- 172.21.15.115:0/517793340 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66d0009c30 con 0x7f66e01032f0 2024-05-09T05:48:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.485+0000 7f66e6042640 1 -- 172.21.15.115:0/517793340 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e019b0c0 con 0x7f66e01032f0 2024-05-09T05:48:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.485+0000 7f66dcff9640 1 -- 172.21.15.115:0/517793340 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0003c70 con 0x7f66e01032f0 2024-05-09T05:48:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.485+0000 7f66e6042640 1 -- 172.21.15.115:0/517793340 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e01a7320 con 0x7f66e01032f0 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66ecfd0740 1 Processor -- start 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66ecfd0740 1 -- start start 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66dcff9640 1 -- 172.21.15.115:0/517793340 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f66d0003e10 con 0x7f66e01032f0 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x562014319680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x562014319bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 0x56201431a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56201427e7c0 con 0x562013f761b0 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56201421a0e0 con 0x562013f4d2f0 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56201427a780 con 0x562013f554d0 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66dffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x562014319680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66dffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x562014319680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40034/0 (socket says 172.21.15.115:40034) 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.488+0000 7f66dffff640 1 -- 172.21.15.115:0/3940310160 learned_addr learned my addr 172.21.15.115:0/3940310160 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66deffd640 1 --2- 172.21.15.115:0/3940310160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 0x56201431a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66df7fe640 1 --2- 172.21.15.115:0/3940310160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x562014319bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66dffff640 1 -- 172.21.15.115:0/3940310160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 msgr2=0x56201431a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66dffff640 1 --2- 172.21.15.115:0/3940310160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 0x56201431a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66dffff640 1 -- 172.21.15.115:0/3940310160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 msgr2=0x562014319bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66dffff640 1 --2- 172.21.15.115:0/3940310160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x562014319bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66dffff640 1 -- 172.21.15.115:0/3940310160 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620142437c0 con 0x562013f761b0 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66dcff9640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66c80b2950 0x7f66c80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66dffff640 1 --2- 172.21.15.115:0/3940310160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x562014319680 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f66d80039f0 tx=0x7f66d8003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b4edc0d1fa49ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66df7fe640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66c80b2950 0x7f66c80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66dcff9640 1 -- 172.21.15.115:0/517793340 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f66d00e2470 con 0x7f66e01032f0 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66c5ffb640 1 -- 172.21.15.115:0/3940310160 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d801a470 con 0x562013f761b0 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66c5ffb640 1 -- 172.21.15.115:0/3940310160 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66d8015070 con 0x562013f761b0 2024-05-09T05:48:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.489+0000 7f66c5ffb640 1 -- 172.21.15.115:0/3940310160 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8010770 con 0x562013f761b0 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.490+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3940310160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 msgr2=0x562014319680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.490+0000 7f66df7fe640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66c80b2950 0x7f66c80b4e60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f66d00df980 tx=0x7f66d000bf60 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.490+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3940310160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x562014319680 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f66d80039f0 tx=0x7f66d8003eb0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.490+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3940310160 shutdown_connections 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.490+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3940310160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 0x56201431a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.490+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3940310160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x562014319bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.490+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3940310160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x562014319680 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.490+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3940310160 >> 172.21.15.115:0/3940310160 conn(0x562013f6a300 msgr2=0x562014315e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.491+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3940310160 shutdown_connections 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.491+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3940310160 wait complete. 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.491+0000 7f66ecfd0740 1 -- start start 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.491+0000 7f66ecfd0740 1 Processor -- start 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f4d2f0 0x5620143256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f554d0 0x562014325c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 0x562014323bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5620143267b0 con 0x562013f4d2f0 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620142059c0 con 0x562013f554d0 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562014326930 con 0x562013f761b0 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66ecfd0740 10 client.0 _ll_register_callbacks cb 0x562014232020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66dffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f4d2f0 0x5620143256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66deffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f554d0 0x562014325c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66dffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f4d2f0 0x5620143256c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40036/0 (socket says 172.21.15.115:40036) 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66dffff640 1 -- 172.21.15.115:0/3042315209 learned_addr learned my addr 172.21.15.115:0/3042315209 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66dffff640 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 0x562014323bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66deffd640 1 -- 172.21.15.115:0/3042315209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 msgr2=0x562014323bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66deffd640 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 0x562014323bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66deffd640 1 -- 172.21.15.115:0/3042315209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f4d2f0 msgr2=0x5620143256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66deffd640 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f4d2f0 0x5620143256c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66deffd640 1 -- 172.21.15.115:0/3042315209 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56201422c1f0 con 0x562013f554d0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66dffff640 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f4d2f0 0x5620143256c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66deffd640 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f554d0 0x562014325c20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66d402a930 tx=0x7f66d4031f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94425cc4d0aa0391 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.492+0000 7f66beffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66beffd640 1 -- 172.21.15.115:0/3042315209 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4035040 con 0x562013f554d0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3042315209 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d8015070 con 0x562013f554d0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66beffd640 1 -- 172.21.15.115:0/3042315209 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f66d40317c0 con 0x562013f554d0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66beffd640 1 -- 172.21.15.115:0/3042315209 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f66d404f050 con 0x562013f554d0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66ecfd0740 10 client.27521 fetch_fsmap learned FSMap version 50 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3042315209 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562013fe1790 con 0x562013f554d0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66beffd640 1 -- 172.21.15.115:0/3042315209 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66d8015070 con 0x562013f554d0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66beffd640 1 -- 172.21.15.115:0/3042315209 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7f66d4003b90 con 0x562013f554d0 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66ecfd0740 10 client.27521 fetch_fsmap finished waiting for FSMap version 50 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66ecfd0740 10 client.27521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66ecfd0740 20 client.27521 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562013f34d80 0x562014335fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66ecfd0740 4 client.27521 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:48:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.493+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3042315209 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56201428c3d0 con 0x562013f34d80 2024-05-09T05:48:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.494+0000 7f66dffff640 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562013f34d80 0x562014335fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.494+0000 7f66beffd640 10 client.27521 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.494+0000 7f66dffff640 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562013f34d80 0x562014335fe0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e107ad1df9ec042 server_cookie=3b2cf1a29aa5021e in_seq=0 out_seq=0 2024-05-09T05:48:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66beffd640 1 -- 172.21.15.115:0/3042315209 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56201428c3d0 con 0x562013f34d80 2024-05-09T05:48:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66beffd640 10 client.27521 handle_command_reply: tid=0 2024-05-09T05:48:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 client.27521 shutdown 2024-05-09T05:48:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3042315209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f554d0 msgr2=0x562014325c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f554d0 0x562014325c20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66d402a930 tx=0x7f66d4031f40 comp rx=0 tx=0).stop 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3042315209 shutdown_connections 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562013f34d80 0x562014335fe0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 0x562014323bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66beffd640 0 client.27521 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f554d0 0x562014325c20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3042315209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f4d2f0 0x5620143256c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3042315209 >> 172.21.15.115:0/3042315209 conn(0x562013f6a300 msgr2=0x562014314700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3042315209 shutdown_connections 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.495+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3042315209 wait complete. 2024-05-09T05:48:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.496+0000 7f66ecfd0740 20 client.27521 trim_cache size 0 max 16384 2024-05-09T05:48:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66ecfd0740 1 Processor -- start 2024-05-09T05:48:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66ecfd0740 1 -- start start 2024-05-09T05:48:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x56201431da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 0x56201433b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x56201433bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01b1210 con 0x562013f761b0 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01b2d80 con 0x562013f4d2f0 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01b18e0 con 0x562013f554d0 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66dffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x56201433bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66dffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x56201433bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40050/0 (socket says 172.21.15.115:40050) 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66dffff640 1 -- 172.21.15.115:0/3110584728 learned_addr learned my addr 172.21.15.115:0/3110584728 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66dffff640 1 --2- 172.21.15.115:0/3110584728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x56201431da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.513+0000 7f66deffd640 1 --2- 172.21.15.115:0/3110584728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 0x56201433b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.514+0000 7f66dffff640 1 -- 172.21.15.115:0/3110584728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 msgr2=0x56201433b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.514+0000 7f66dffff640 1 --2- 172.21.15.115:0/3110584728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 0x56201433b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.514+0000 7f66dffff640 1 -- 172.21.15.115:0/3110584728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 msgr2=0x56201431da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.514+0000 7f66dffff640 1 --2- 172.21.15.115:0/3110584728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x56201431da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.514+0000 7f66dffff640 1 -- 172.21.15.115:0/3110584728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620140585f0 con 0x562013f761b0 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.514+0000 7f66dffff640 1 --2- 172.21.15.115:0/3110584728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x56201433bd00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f66d8005cc0 tx=0x7f66d8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15353d795fd94907 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.514+0000 7f66c57fa640 1 -- 172.21.15.115:0/3110584728 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8004980 con 0x562013f761b0 2024-05-09T05:48:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.514+0000 7f66c57fa640 1 -- 172.21.15.115:0/3110584728 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66d8015070 con 0x562013f761b0 2024-05-09T05:48:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.514+0000 7f66c57fa640 1 -- 172.21.15.115:0/3110584728 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8005320 con 0x562013f761b0 2024-05-09T05:48:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3110584728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 msgr2=0x56201433bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3110584728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x56201433bd00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f66d8005cc0 tx=0x7f66d8011ed0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3110584728 shutdown_connections 2024-05-09T05:48:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3110584728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f761b0 0x56201433bd00 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3110584728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f554d0 0x56201433b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/3110584728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x56201431da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3110584728 >> 172.21.15.115:0/3110584728 conn(0x562013f6a300 msgr2=0x56201431d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3110584728 shutdown_connections 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 -- 172.21.15.115:0/3110584728 wait complete. 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.515+0000 7f66ecfd0740 1 -- start start 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66ecfd0740 1 Processor -- start 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x562014329030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f554d0 0x562014329570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66ecfd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 0x5620143963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01b26b0 con 0x562013f554d0 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01b1fb0 con 0x562013f4d2f0 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66ecfd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01af210 con 0x562013f761b0 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66ecfd0740 10 client.0 _ll_register_callbacks cb 0x7f66d4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66deffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f554d0 0x562014329570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66deffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f554d0 0x562014329570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40052/0 (socket says 172.21.15.115:40052) 2024-05-09T05:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66deffd640 1 -- 172.21.15.115:0/2274907981 learned_addr learned my addr 172.21.15.115:0/2274907981 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66dffff640 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x562014329030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66dffff640 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 0x5620143963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66deffd640 1 -- 172.21.15.115:0/2274907981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 msgr2=0x5620143963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66deffd640 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 0x5620143963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66deffd640 1 -- 172.21.15.115:0/2274907981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 msgr2=0x562014329030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66deffd640 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x562014329030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.516+0000 7f66deffd640 1 -- 172.21.15.115:0/2274907981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562014242ed0 con 0x562013f554d0 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66deffd640 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f554d0 0x562014329570 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f66d4046570 tx=0x7f66d40465f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6edb046622010a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66beffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66beffd640 1 -- 172.21.15.115:0/2274907981 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4057020 con 0x562013f554d0 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66beffd640 1 -- 172.21.15.115:0/2274907981 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7f66d40315c0 con 0x562013f554d0 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66ecfd0740 1 -- 172.21.15.115:0/2274907981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d8015070 con 0x562013f554d0 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66beffd640 1 -- 172.21.15.115:0/2274907981 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f66d4035040 con 0x562013f554d0 2024-05-09T05:48:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66ecfd0740 10 client.18279 fetch_fsmap learned FSMap version 50 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66ecfd0740 1 -- 172.21.15.115:0/2274907981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56201439da90 con 0x562013f554d0 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66beffd640 1 -- 172.21.15.115:0/2274907981 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66d8015070 con 0x562013f554d0 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66beffd640 1 -- 172.21.15.115:0/2274907981 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7f66d40035f0 con 0x562013f554d0 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66ecfd0740 10 client.18279 fetch_fsmap finished waiting for FSMap version 50 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66ecfd0740 10 client.18279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66ecfd0740 20 client.18279 populate_metadata read hostname 'smithi115' 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562013f0a1c0 0x5620143a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66ecfd0740 4 client.18279 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.517+0000 7f66ecfd0740 1 -- 172.21.15.115:0/2274907981 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562014286240 con 0x562013f0a1c0 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.518+0000 7f66dffff640 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562013f0a1c0 0x5620143a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.518+0000 7f66beffd640 10 client.18279 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.518+0000 7f66dffff640 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562013f0a1c0 0x5620143a0300 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f86e8d1be22def0f server_cookie=c465a36348003d8f in_seq=0 out_seq=0 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66beffd640 1 -- 172.21.15.115:0/2274907981 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562014286240 con 0x562013f0a1c0 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66beffd640 10 client.18279 handle_command_reply: tid=0 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 client.18279 shutdown 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 -- 172.21.15.115:0/2274907981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f554d0 msgr2=0x562014329570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f554d0 0x562014329570 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f66d4046570 tx=0x7f66d40465f0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 -- 172.21.15.115:0/2274907981 shutdown_connections 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562013f0a1c0 0x5620143a0300 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562013f761b0 0x5620143963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562013f554d0 0x562014329570 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66beffd640 0 client.18279 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 --2- 172.21.15.115:0/2274907981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562013f4d2f0 0x562014329030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 -- 172.21.15.115:0/2274907981 >> 172.21.15.115:0/2274907981 conn(0x562013f6a300 msgr2=0x562014314700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 -- 172.21.15.115:0/2274907981 shutdown_connections 2024-05-09T05:48:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 1 -- 172.21.15.115:0/2274907981 wait complete. 2024-05-09T05:48:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.522+0000 7f66ecfd0740 20 client.18279 trim_cache size 0 max 16384 2024-05-09T05:48:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:48:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:48:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:48:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 -- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66c80b2950 msgr2=0x7f66c80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66c80b2950 0x7f66c80b4e60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f66d00df980 tx=0x7f66d000bf60 comp rx=0 tx=0).stop 2024-05-09T05:48:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 -- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e01032f0 msgr2=0x7f66e01a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:48:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e01032f0 0x7f66e01a0580 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f66d0005fd0 tx=0x7f66d000dde0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 -- 172.21.15.115:0/517793340 shutdown_connections 2024-05-09T05:48:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f66c80b2950 0x7f66c80b4e60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66e0109040 0x7f66e019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66e0108ba0 0x7f66e01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 --2- 172.21.15.115:0/517793340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66e01032f0 0x7f66e01a0580 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:48:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 -- 172.21.15.115:0/517793340 >> 172.21.15.115:0/517793340 conn(0x7f66e00782e0 msgr2=0x7f66e0106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:48:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 -- 172.21.15.115:0/517793340 shutdown_connections 2024-05-09T05:48:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:48:49.523+0000 7f66e6042640 1 -- 172.21.15.115:0/517793340 wait complete. 2024-05-09T05:48:49.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:49 smithi195 ceph-mon[33844]: pgmap v331: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 28 op/s 2024-05-09T05:48:49.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:49 smithi115 ceph-mon[30349]: pgmap v331: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 28 op/s 2024-05-09T05:48:49.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:49 smithi161 ceph-mon[33805]: pgmap v331: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 28 op/s 2024-05-09T05:48:51.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:51 smithi195 ceph-mon[33844]: pgmap v332: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:48:51.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:51 smithi115 ceph-mon[30349]: pgmap v332: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:48:51.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:51 smithi161 ceph-mon[33805]: pgmap v332: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:48:53.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:53 smithi195 ceph-mon[33844]: pgmap v333: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:48:53.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:53 smithi115 ceph-mon[30349]: pgmap v333: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:48:53.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:53 smithi161 ceph-mon[33805]: pgmap v333: 129 pgs: 129 active+clean; 8.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:48:54.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:54 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:48:54.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:54 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:48:54.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:54 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:48:55.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:55 smithi195 ceph-mon[33844]: pgmap v334: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:48:55.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:55 smithi115 ceph-mon[30349]: pgmap v334: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:48:55.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:55 smithi161 ceph-mon[33805]: pgmap v334: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:48:57.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:57 smithi195 ceph-mon[33844]: pgmap v335: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:48:57.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:57 smithi115 ceph-mon[30349]: pgmap v335: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:48:57.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:57 smithi161 ceph-mon[33805]: pgmap v335: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:48:59.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:48:59 smithi195 ceph-mon[33844]: pgmap v336: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:48:59.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:48:59 smithi115 ceph-mon[30349]: pgmap v336: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:48:59.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:48:59 smithi161 ceph-mon[33805]: pgmap v336: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 MiB/s wr, 29 op/s 2024-05-09T05:49:01.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:01 smithi195 ceph-mon[33844]: pgmap v337: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-05-09T05:49:01.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:01 smithi115 ceph-mon[30349]: pgmap v337: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-05-09T05:49:01.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:01 smithi161 ceph-mon[33805]: pgmap v337: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-05-09T05:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.641+0000 7fed0bc9f640 1 -- 172.21.15.115:0/1970389494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed0410e220 msgr2=0x7fed04110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.641+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/1970389494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed0410e220 0x7fed04110710 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009900 tx=0x7fecfc02a480 comp rx=0 tx=0).stop 2024-05-09T05:49:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.641+0000 7fed0bc9f640 1 -- 172.21.15.115:0/1970389494 shutdown_connections 2024-05-09T05:49:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.641+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/1970389494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed0410e220 0x7fed04110710 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.641+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/1970389494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed0410b8c0 0x7fed0410dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.641+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/1970389494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed04104e60 0x7fed04105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.641+0000 7fed0bc9f640 1 -- 172.21.15.115:0/1970389494 >> 172.21.15.115:0/1970389494 conn(0x7fed040781b0 msgr2=0x7fed04078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.642+0000 7fed0bc9f640 1 -- 172.21.15.115:0/1970389494 shutdown_connections 2024-05-09T05:49:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.642+0000 7fed0bc9f640 1 -- 172.21.15.115:0/1970389494 wait complete. 2024-05-09T05:49:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.642+0000 7fed0bc9f640 1 Processor -- start 2024-05-09T05:49:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed0bc9f640 1 -- start start 2024-05-09T05:49:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed0bc9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed04104e60 0x7fed0419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed0bc9f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0410b8c0 0x7fed0419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed0bc9f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed0410e220 0x7fed0419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed0bc9f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fed04113d30 con 0x7fed04104e60 2024-05-09T05:49:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed0bc9f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed04113bb0 con 0x7fed0410b8c0 2024-05-09T05:49:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed0bc9f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed04113eb0 con 0x7fed0410e220 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed09a3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed04104e60 0x7fed0419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed09a3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed04104e60 0x7fed0419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41822/0 (socket says 172.21.15.115:41822) 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed09a3b640 1 -- 172.21.15.115:0/4096379330 learned_addr learned my addr 172.21.15.115:0/4096379330 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed0923a640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0410b8c0 0x7fed0419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed0a23c640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed0410e220 0x7fed0419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed09a3b640 1 -- 172.21.15.115:0/4096379330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed0410e220 msgr2=0x7fed0419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed09a3b640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed0410e220 0x7fed0419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed09a3b640 1 -- 172.21.15.115:0/4096379330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0410b8c0 msgr2=0x7fed0419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed09a3b640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0410b8c0 0x7fed0419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.643+0000 7fed09a3b640 1 -- 172.21.15.115:0/4096379330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0419f5b0 con 0x7fed04104e60 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.644+0000 7fed09a3b640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed04104e60 0x7fed0419a750 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fed0000ebc0 tx=0x7fed0000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.644+0000 7fecfaffd640 1 -- 172.21.15.115:0/4096379330 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00010630 con 0x7fed04104e60 2024-05-09T05:49:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.644+0000 7fed0bc9f640 1 -- 172.21.15.115:0/4096379330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0419f8a0 con 0x7fed04104e60 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.644+0000 7fecfaffd640 1 -- 172.21.15.115:0/4096379330 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed00010c60 con 0x7fed04104e60 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.644+0000 7fecfaffd640 1 -- 172.21.15.115:0/4096379330 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00018b00 con 0x7fed04104e60 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.644+0000 7fed0bc9f640 1 -- 172.21.15.115:0/4096379330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041a7220 con 0x7fed04104e60 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.647+0000 7fed12c2f740 1 Processor -- start 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.647+0000 7fed12c2f740 1 -- start start 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.647+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 0x564f751ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.647+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.647+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751c9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.647+0000 7fed12c2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f751207c0 con 0x564f74df74d0 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.647+0000 7fed12c2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564f750bc0e0 con 0x564f74e181b0 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed12c2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f7511c780 con 0x564f74def2f0 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0923a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0923a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41836/0 (socket says 172.21.15.115:41836) 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0923a640 1 -- 172.21.15.115:0/2279032980 learned_addr learned my addr 172.21.15.115:0/2279032980 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fecfaffd640 1 -- 172.21.15.115:0/4096379330 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fed00018ca0 con 0x7fed04104e60 2024-05-09T05:49:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0a23c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 0x564f751ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed09a3b640 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0a23c640 1 -- 172.21.15.115:0/2279032980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 msgr2=0x564f751cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0a23c640 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0a23c640 1 -- 172.21.15.115:0/2279032980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 msgr2=0x564f751c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0a23c640 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0a23c640 1 -- 172.21.15.115:0/2279032980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f750e57c0 con 0x564f74e181b0 2024-05-09T05:49:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed09a3b640 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:49:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.648+0000 7fed0923a640 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.649+0000 7fed0a23c640 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 0x564f751ca6f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0057c0 tx=0x7fecfc032b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c75d3faa58a2d68 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.649+0000 7fecfaffd640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fecf00b2950 0x7fecf00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.649+0000 7fed09a3b640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fecf00b2950 0x7fecf00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.649+0000 7feceffff640 1 -- 172.21.15.115:0/2279032980 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc03f070 con 0x564f74e181b0 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.649+0000 7feceffff640 1 -- 172.21.15.115:0/2279032980 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecfc004030 con 0x564f74e181b0 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.649+0000 7feceffff640 1 -- 172.21.15.115:0/2279032980 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc035070 con 0x564f74e181b0 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.649+0000 7fecfaffd640 1 -- 172.21.15.115:0/4096379330 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fed000e3110 con 0x7fed04104e60 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.650+0000 7fed12c2f740 1 -- 172.21.15.115:0/2279032980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 msgr2=0x564f751ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.650+0000 7fed12c2f740 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 0x564f751ca6f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0057c0 tx=0x7fecfc032b00 comp rx=0 tx=0).stop 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.650+0000 7fed09a3b640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fecf00b2950 0x7fecf00b4e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fed000e0580 tx=0x7fed000a9e30 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.650+0000 7fed12c2f740 1 -- 172.21.15.115:0/2279032980 shutdown_connections 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.650+0000 7fed12c2f740 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.650+0000 7fed12c2f740 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.650+0000 7fed12c2f740 1 --2- 172.21.15.115:0/2279032980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 0x564f751ca6f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.650+0000 7fed12c2f740 1 -- 172.21.15.115:0/2279032980 >> 172.21.15.115:0/2279032980 conn(0x564f74e0c300 msgr2=0x564f751b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.650+0000 7fed12c2f740 1 -- 172.21.15.115:0/2279032980 shutdown_connections 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 1 -- 172.21.15.115:0/2279032980 wait complete. 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 1 -- start start 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 1 Processor -- start 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751d4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 0x564f751ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564f751c9d60 con 0x564f74df74d0 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564f750a79c0 con 0x564f74e181b0 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f751c9ee0 con 0x564f74def2f0 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.651+0000 7fed12c2f740 10 client.0 _ll_register_callbacks cb 0x564f750d4020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0923a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751d4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:44698/0 (socket says 172.21.15.115:44698) 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 -- 172.21.15.115:0/3414399145 learned_addr learned my addr 172.21.15.115:0/3414399145 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 0x564f751ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 -- 172.21.15.115:0/3414399145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 msgr2=0x564f751ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 0x564f751ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 -- 172.21.15.115:0/3414399145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 msgr2=0x564f751d4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751d4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 -- 172.21.15.115:0/3414399145 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f750ce1f0 con 0x564f74def2f0 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0923a640 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed0a23c640 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751d4710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x564f751b2860 tx=0x7fecfc009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c596285b7411d81d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7feced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7feced7fa640 1 -- 172.21.15.115:0/3414399145 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc03f070 con 0x564f74def2f0 2024-05-09T05:49:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.652+0000 7fed12c2f740 1 -- 172.21.15.115:0/3414399145 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecfc004030 con 0x564f74def2f0 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7feced7fa640 1 -- 172.21.15.115:0/3414399145 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fecfc0043f0 con 0x564f74def2f0 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7feced7fa640 1 -- 172.21.15.115:0/3414399145 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fecfc058050 con 0x564f74def2f0 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7fed12c2f740 10 client.27634 fetch_fsmap learned FSMap version 51 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7fed12c2f740 1 -- 172.21.15.115:0/3414399145 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564f74e83790 con 0x564f74def2f0 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7feced7fa640 1 -- 172.21.15.115:0/3414399145 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecfc004030 con 0x564f74def2f0 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7feced7fa640 1 -- 172.21.15.115:0/3414399145 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7fecfc050020 con 0x564f74def2f0 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7fed12c2f740 10 client.27634 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7fed12c2f740 10 client.27634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7fed12c2f740 20 client.27634 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7fed12c2f740 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564f74dd6d80 0x564f751e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7fed12c2f740 4 client.27634 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7fed12c2f740 1 -- 172.21.15.115:0/3414399145 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f7512e3d0 con 0x564f74dd6d80 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.653+0000 7fed0923a640 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564f74dd6d80 0x564f751e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.654+0000 7feced7fa640 10 client.27634 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.654+0000 7fed0923a640 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564f74dd6d80 0x564f751e0810 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44aca80db3e8c971 server_cookie=a7af70373f254e17 in_seq=0 out_seq=0 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.654+0000 7feced7fa640 1 -- 172.21.15.115:0/3414399145 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f7512e3d0 con 0x564f74dd6d80 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.654+0000 7feced7fa640 10 client.27634 handle_command_reply: tid=0 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 client.27634 shutdown 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 -- 172.21.15.115:0/3414399145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 msgr2=0x564f751d4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751d4710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x564f751b2860 tx=0x7fecfc009230 comp rx=0 tx=0).stop 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 -- 172.21.15.115:0/3414399145 shutdown_connections 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74e181b0 0x564f751ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564f74dd6d80 0x564f751e0810 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 --2- 172.21.15.115:0/3414399145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74def2f0 0x564f751d4710 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7feced7fa640 0 client.27634 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 -- 172.21.15.115:0/3414399145 >> 172.21.15.115:0/3414399145 conn(0x564f74e0c300 msgr2=0x564f751b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 -- 172.21.15.115:0/3414399145 shutdown_connections 2024-05-09T05:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 1 -- 172.21.15.115:0/3414399145 wait complete. 2024-05-09T05:49:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.655+0000 7fed12c2f740 20 client.27634 trim_cache size 0 max 16384 2024-05-09T05:49:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.671+0000 7fed12c2f740 1 Processor -- start 2024-05-09T05:49:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.671+0000 7fed12c2f740 1 -- start start 2024-05-09T05:49:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74def2f0 0x564f751bb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 0x564f751e6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed12c2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fed041aef00 con 0x564f74df74d0 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed12c2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed041b0aa0 con 0x564f74def2f0 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed12c2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed041af5d0 con 0x564f74e181b0 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0923a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0923a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41856/0 (socket says 172.21.15.115:41856) 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0923a640 1 -- 172.21.15.115:0/1559268862 learned_addr learned my addr 172.21.15.115:0/1559268862 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0a23c640 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74def2f0 0x564f751bb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0a23c640 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 0x564f751e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0923a640 1 -- 172.21.15.115:0/1559268862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 msgr2=0x564f751e6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0923a640 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 0x564f751e6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0923a640 1 -- 172.21.15.115:0/1559268862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74def2f0 msgr2=0x564f751bb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0923a640 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74def2f0 0x564f751bb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0923a640 1 -- 172.21.15.115:0/1559268862 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f751c88f0 con 0x564f74df74d0 2024-05-09T05:49:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0a23c640 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 0x564f751e6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:49:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.672+0000 7fed0923a640 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751cc280 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x564f751e0fb0 tx=0x7fecf4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb3ae077200e50f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.673+0000 7fecef7fe640 1 -- 172.21.15.115:0/1559268862 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4011890 con 0x564f74df74d0 2024-05-09T05:49:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.673+0000 7fecef7fe640 1 -- 172.21.15.115:0/1559268862 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecf4025020 con 0x564f74df74d0 2024-05-09T05:49:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.673+0000 7fecef7fe640 1 -- 172.21.15.115:0/1559268862 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf40219b0 con 0x564f74df74d0 2024-05-09T05:49:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 -- 172.21.15.115:0/1559268862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 msgr2=0x564f751cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751cc280 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x564f751e0fb0 tx=0x7fecf4005f90 comp rx=0 tx=0).stop 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 -- 172.21.15.115:0/1559268862 shutdown_connections 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 0x564f751e6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74df74d0 0x564f751cc280 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 --2- 172.21.15.115:0/1559268862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74def2f0 0x564f751bb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 -- 172.21.15.115:0/1559268862 >> 172.21.15.115:0/1559268862 conn(0x564f74e0c300 msgr2=0x564f751bad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 -- 172.21.15.115:0/1559268862 shutdown_connections 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 -- 172.21.15.115:0/1559268862 wait complete. 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.674+0000 7fed12c2f740 1 -- start start 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed12c2f740 1 Processor -- start 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74def2f0 0x564f751d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74df74d0 0x564f751d8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed12c2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 0x564f75246bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed12c2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fed041b03d0 con 0x564f74def2f0 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed12c2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed041afcd0 con 0x564f74df74d0 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed12c2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fed041acf00 con 0x564f74e181b0 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed12c2f740 10 client.0 _ll_register_callbacks cb 0x7fecf40095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed0a23c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74def2f0 0x564f751d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed0a23c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74def2f0 0x564f751d7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41870/0 (socket says 172.21.15.115:41870) 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed0a23c640 1 -- 172.21.15.115:0/653134788 learned_addr learned my addr 172.21.15.115:0/653134788 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed0923a640 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74df74d0 0x564f751d8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.675+0000 7fed0a23c640 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 0x564f75246bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7fed0a23c640 1 -- 172.21.15.115:0/653134788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 msgr2=0x564f75246bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7fed0a23c640 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 0x564f75246bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7fed0a23c640 1 -- 172.21.15.115:0/653134788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74df74d0 msgr2=0x564f751d8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7fed0a23c640 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74df74d0 0x564f751d8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7fed0a23c640 1 -- 172.21.15.115:0/653134788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f74efa5f0 con 0x564f74def2f0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7fed0a23c640 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74def2f0 0x564f751d7f50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fecfc005820 tx=0x7fecfc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73825f491aad57c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7feced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7feced7fa640 1 -- 172.21.15.115:0/653134788 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc03f030 con 0x564f74def2f0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7feced7fa640 1 -- 172.21.15.115:0/653134788 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9438+0+0 (secure 0 0 0) 0x7fecfc032c90 con 0x564f74def2f0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7fed12c2f740 1 -- 172.21.15.115:0/653134788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf4025020 con 0x564f74def2f0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7feced7fa640 1 -- 172.21.15.115:0/653134788 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fecfc035040 con 0x564f74def2f0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7fed12c2f740 10 client.18297 fetch_fsmap learned FSMap version 51 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7fed12c2f740 1 -- 172.21.15.115:0/653134788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564f750e4ed0 con 0x564f74def2f0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.676+0000 7feced7fa640 1 -- 172.21.15.115:0/653134788 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecf4025020 con 0x564f74def2f0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7feced7fa640 1 -- 172.21.15.115:0/653134788 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 51) ==== 8370+0+0 (secure 0 0 0) 0x7fecfc050040 con 0x564f74def2f0 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7fed12c2f740 10 client.18297 fetch_fsmap finished waiting for FSMap version 51 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7fed12c2f740 10 client.18297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7fed12c2f740 20 client.18297 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7fed12c2f740 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564f74dac1c0 0x564f7524a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7fed12c2f740 4 client.18297 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:49:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7fed12c2f740 1 -- 172.21.15.115:0/653134788 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f75128240 con 0x564f74dac1c0 2024-05-09T05:49:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7fed0923a640 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564f74dac1c0 0x564f7524a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7feced7fa640 10 client.18297 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:02.677+0000 7fed0923a640 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564f74dac1c0 0x564f7524a790 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b47eaf15003ec5b server_cookie=dbfef96b9159cf5f in_seq=0 out_seq=0 2024-05-09T05:49:03.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:03 smithi195 ceph-mon[33844]: pgmap v338: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-05-09T05:49:03.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:03 smithi115 ceph-mon[30349]: pgmap v338: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-05-09T05:49:03.909 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:03 smithi161 ceph-mon[33805]: pgmap v338: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-05-09T05:49:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7feced7fa640 1 -- 172.21.15.115:0/653134788 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564f75128240 con 0x564f74dac1c0 2024-05-09T05:49:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7feced7fa640 10 client.18297 handle_command_reply: tid=0 2024-05-09T05:49:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7fed12c2f740 1 client.18297 shutdown 2024-05-09T05:49:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7fed12c2f740 1 -- 172.21.15.115:0/653134788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74def2f0 msgr2=0x564f751d7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7fed12c2f740 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74def2f0 0x564f751d7f50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fecfc005820 tx=0x7fecfc005dd0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7fed12c2f740 1 -- 172.21.15.115:0/653134788 shutdown_connections 2024-05-09T05:49:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7fed12c2f740 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564f74dac1c0 0x564f7524a790 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7fed12c2f740 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f74e181b0 0x564f75246bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7fed12c2f740 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f74df74d0 0x564f751d8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7feced7fa640 0 client.18297 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7fed12c2f740 1 --2- 172.21.15.115:0/653134788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564f74def2f0 0x564f751d7f50 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.288+0000 7fed12c2f740 1 -- 172.21.15.115:0/653134788 >> 172.21.15.115:0/653134788 conn(0x564f74e0c300 msgr2=0x564f751d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed12c2f740 1 -- 172.21.15.115:0/653134788 shutdown_connections 2024-05-09T05:49:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed12c2f740 1 -- 172.21.15.115:0/653134788 wait complete. 2024-05-09T05:49:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed12c2f740 20 client.18297 trim_cache size 0 max 16384 2024-05-09T05:49:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:49:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:49:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:49:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 -- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fecf00b2950 msgr2=0x7fecf00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fecf00b2950 0x7fecf00b4e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fed000e0580 tx=0x7fed000a9e30 comp rx=0 tx=0).stop 2024-05-09T05:49:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 -- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed04104e60 msgr2=0x7fed0419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed04104e60 0x7fed0419a750 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fed0000ebc0 tx=0x7fed0000c6a0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 -- 172.21.15.115:0/4096379330 shutdown_connections 2024-05-09T05:49:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fecf00b2950 0x7fecf00b4e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed0410e220 0x7fed0419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed0410b8c0 0x7fed0419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 --2- 172.21.15.115:0/4096379330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed04104e60 0x7fed0419a750 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.289+0000 7fed0bc9f640 1 -- 172.21.15.115:0/4096379330 >> 172.21.15.115:0/4096379330 conn(0x7fed040781b0 msgr2=0x7fed041012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.290+0000 7fed0bc9f640 1 -- 172.21.15.115:0/4096379330 shutdown_connections 2024-05-09T05:49:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:04.290+0000 7fed0bc9f640 1 -- 172.21.15.115:0/4096379330 wait complete. 2024-05-09T05:49:04.614 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: *** Caught signal (Segmentation fault) ** 2024-05-09T05:49:04.614 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: in thread 7f5608609700 thread_name:md_log_replay 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f56199cbd20] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x1a) [0x55c4e3664d88] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x55c4e3665238] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55c4e365f8af] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55c4e3320cff] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 6: (MDLog::_replay_thread()+0x1f38) [0x55c4e36c592e] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55c4e36cee6d] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 8: (Thread::entry_wrapper()+0x43) [0x7f561ae1e9f5] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 9: (Thread::_entry_func(void*)+0xd) [0x7f561ae1ea11] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f56199c11ca] 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 11: clone() 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2024-05-09T05:49:04.355+0000 7f5608609700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:49:04.615 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: in thread 7f5608609700 thread_name:md_log_replay 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f56199cbd20] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x1a) [0x55c4e3664d88] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x55c4e3665238] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55c4e365f8af] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55c4e3320cff] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 6: (MDLog::_replay_thread()+0x1f38) [0x55c4e36c592e] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55c4e36cee6d] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 8: (Thread::entry_wrapper()+0x43) [0x7f561ae1e9f5] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 9: (Thread::_entry_func(void*)+0xd) [0x7f561ae1ea11] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f56199c11ca] 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 11: clone() 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:49:04.616 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 0> 2024-05-09T05:49:04.355+0000 7f5608609700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: in thread 7f5608609700 thread_name:md_log_replay 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f56199cbd20] 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x1a) [0x55c4e3664d88] 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x55c4e3665238] 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55c4e365f8af] 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55c4e3320cff] 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 6: (MDLog::_replay_thread()+0x1f38) [0x55c4e36c592e] 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55c4e36cee6d] 2024-05-09T05:49:04.617 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 8: (Thread::entry_wrapper()+0x43) [0x7f561ae1e9f5] 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 9: (Thread::_entry_func(void*)+0xd) [0x7f561ae1ea11] 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f56199c11ca] 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 11: clone() 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 0> 2024-05-09T05:49:04.355+0000 7f5608609700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: in thread 7f5608609700 thread_name:md_log_replay 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f56199cbd20] 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x1a) [0x55c4e3664d88] 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x55c4e3665238] 2024-05-09T05:49:04.618 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55c4e365f8af] 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55c4e3320cff] 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 6: (MDLog::_replay_thread()+0x1f38) [0x55c4e36c592e] 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55c4e36cee6d] 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 8: (Thread::entry_wrapper()+0x43) [0x7f561ae1e9f5] 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 9: (Thread::_entry_func(void*)+0xd) [0x7f561ae1ea11] 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7f56199c11ca] 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 11: clone() 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[69350]: 2024-05-09T05:49:04.619 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:04 smithi195 podman[70103]: 2024-05-09 05:49:04.40841971 +0000 UTC m=+0.020735748 container died 2e229c2e786c64be76f8874048ea44248897bdaf0f295db4deb6345e77e472c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:49:05.471 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:05 smithi195 podman[70103]: 2024-05-09 05:49:05.160449408 +0000 UTC m=+0.772765427 container remove 2e229c2e786c64be76f8874048ea44248897bdaf0f295db4deb6345e77e472c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:49:05.471 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:05 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-05-09T05:49:05.471 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:05 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-05-09T05:49:05.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:05 smithi195 ceph-mon[33844]: pgmap v339: 129 pgs: 129 active+clean; 9.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 35 MiB/s wr, 47 op/s 2024-05-09T05:49:05.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:05 smithi195 ceph-mon[33844]: osdmap e99: 12 total, 12 up, 12 in 2024-05-09T05:49:05.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:05 smithi115 ceph-mon[30349]: pgmap v339: 129 pgs: 129 active+clean; 9.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 35 MiB/s wr, 47 op/s 2024-05-09T05:49:05.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:05 smithi115 ceph-mon[30349]: osdmap e99: 12 total, 12 up, 12 in 2024-05-09T05:49:05.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:05 smithi161 ceph-mon[33805]: pgmap v339: 129 pgs: 129 active+clean; 9.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 35 MiB/s wr, 47 op/s 2024-05-09T05:49:05.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:05 smithi161 ceph-mon[33805]: osdmap e99: 12 total, 12 up, 12 in 2024-05-09T05:49:07.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:07 smithi195 ceph-mon[33844]: pgmap v341: 129 pgs: 129 active+clean; 9.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 45 op/s 2024-05-09T05:49:07.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:07 smithi115 ceph-mon[30349]: pgmap v341: 129 pgs: 129 active+clean; 9.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 45 op/s 2024-05-09T05:49:07.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:07 smithi161 ceph-mon[33805]: pgmap v341: 129 pgs: 129 active+clean; 9.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 45 op/s 2024-05-09T05:49:09.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:09 smithi195 ceph-mon[33844]: pgmap v342: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 45 op/s 2024-05-09T05:49:09.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:09 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:49:09.863 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:09 smithi195 ceph-mon[33844]: osdmap e100: 12 total, 12 up, 12 in 2024-05-09T05:49:09.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:09 smithi115 ceph-mon[30349]: pgmap v342: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 45 op/s 2024-05-09T05:49:09.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:09 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:49:09.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:09 smithi115 ceph-mon[30349]: osdmap e100: 12 total, 12 up, 12 in 2024-05-09T05:49:09.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:09 smithi161 ceph-mon[33805]: pgmap v342: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 45 op/s 2024-05-09T05:49:09.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:09 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:49:09.908 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:09 smithi161 ceph-mon[33805]: osdmap e100: 12 total, 12 up, 12 in 2024-05-09T05:49:11.113 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:10 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:49:11.113 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:10 smithi195 ceph-mon[33844]: pgmap v344: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 35 MiB/s wr, 56 op/s 2024-05-09T05:49:11.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:10 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:49:11.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:10 smithi115 ceph-mon[30349]: pgmap v344: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 35 MiB/s wr, 56 op/s 2024-05-09T05:49:11.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:10 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:49:11.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:10 smithi161 ceph-mon[33805]: pgmap v344: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 35 MiB/s wr, 56 op/s 2024-05-09T05:49:13.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:13 smithi195 ceph-mon[33844]: pgmap v345: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-05-09T05:49:13.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:13 smithi115 ceph-mon[30349]: pgmap v345: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-05-09T05:49:13.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:13 smithi161 ceph-mon[33805]: pgmap v345: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 14 op/s 2024-05-09T05:49:15.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:15 smithi195 ceph-mon[33844]: pgmap v346: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-05-09T05:49:15.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:15 smithi115 ceph-mon[30349]: pgmap v346: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-05-09T05:49:15.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:15 smithi161 ceph-mon[33805]: pgmap v346: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-05-09T05:49:15.881 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:15 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 3. 2024-05-09T05:49:15.882 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:15 smithi195 systemd[1]: Stopped Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:49:15.882 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:15 smithi195 systemd[1]: Starting Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:49:16.213 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:15 smithi195 podman[70201]: 2024-05-09 05:49:15.79446648 +0000 UTC m=+0.018336065 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:49:16.213 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:15 smithi195 podman[70201]: 2024-05-09 05:49:15.978290205 +0000 UTC m=+0.202159820 container create 0bf632ef37eef0d17b32414d3d309e119e8d7fc25bd231c2098f9e9c798ffdb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-09T05:49:16.591 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:16 smithi195 podman[70201]: 2024-05-09 05:49:16.223056179 +0000 UTC m=+0.446925746 container init 0bf632ef37eef0d17b32414d3d309e119e8d7fc25bd231c2098f9e9c798ffdb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-09T05:49:16.591 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:16 smithi195 podman[70201]: 2024-05-09 05:49:16.226391322 +0000 UTC m=+0.450260894 container start 0bf632ef37eef0d17b32414d3d309e119e8d7fc25bd231c2098f9e9c798ffdb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-09T05:49:16.592 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:16 smithi195 ceph-mds[70220]: -- 172.21.15.195:0/366032866 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d36bd43a40 con 0x55d36bd8b180 2024-05-09T05:49:16.592 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:16 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: starting mds.l at 2024-05-09T05:49:16.592 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:16 smithi195 bash[70201]: 0bf632ef37eef0d17b32414d3d309e119e8d7fc25bd231c2098f9e9c798ffdb9 2024-05-09T05:49:16.592 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:49:16 smithi195 systemd[1]: Started Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:49:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.397+0000 7f9cb56d9640 1 -- 172.21.15.115:0/3070315877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb01075d0 msgr2=0x7f9cb010ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.397+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/3070315877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb01075d0 0x7f9cb010ed70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009ca0 tx=0x7f9ca802a8c0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.397+0000 7f9cb56d9640 1 -- 172.21.15.115:0/3070315877 shutdown_connections 2024-05-09T05:49:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.397+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/3070315877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb01075d0 0x7f9cb010ed70 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.397+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/3070315877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb0106b10 0x7f9cb0106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.397+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/3070315877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb0102580 0x7f9cb0102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.397+0000 7f9cb56d9640 1 -- 172.21.15.115:0/3070315877 >> 172.21.15.115:0/3070315877 conn(0x7f9cb00779e0 msgr2=0x7f9cb0077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.397+0000 7f9cb56d9640 1 -- 172.21.15.115:0/3070315877 shutdown_connections 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.397+0000 7f9cb56d9640 1 -- 172.21.15.115:0/3070315877 wait complete. 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.398+0000 7f9cb56d9640 1 Processor -- start 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.398+0000 7f9cb56d9640 1 -- start start 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.398+0000 7f9cb56d9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb0102580 0x7f9cb0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.398+0000 7f9cb56d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb0106b10 0x7f9cb0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.398+0000 7f9cb56d9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb01075d0 0x7f9cb019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.398+0000 7f9cb56d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01123e0 con 0x7f9cb0106b10 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cb56d9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb0112260 con 0x7f9cb0102580 2024-05-09T05:49:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cb56d9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb0112560 con 0x7f9cb01075d0 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cae7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb0106b10 0x7f9cb0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cae7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb0106b10 0x7f9cb0199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39478/0 (socket says 172.21.15.115:39478) 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cae7fc640 1 -- 172.21.15.115:0/840580624 learned_addr learned my addr 172.21.15.115:0/840580624 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9caf7fe640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb01075d0 0x7f9cb019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9caeffd640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb0102580 0x7f9cb0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cae7fc640 1 -- 172.21.15.115:0/840580624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb01075d0 msgr2=0x7f9cb019dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb01075d0 0x7f9cb019dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cae7fc640 1 -- 172.21.15.115:0/840580624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb0102580 msgr2=0x7f9cb0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb0102580 0x7f9cb0199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cae7fc640 1 -- 172.21.15.115:0/840580624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb019e4e0 con 0x7f9cb0106b10 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb0106b10 0x7f9cb0199be0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4005fd0 tx=0x7f9ca4009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9c97fff640 1 -- 172.21.15.115:0/840580624 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400d020 con 0x7f9cb0106b10 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9cb56d9640 1 -- 172.21.15.115:0/840580624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb019e7d0 con 0x7f9cb0106b10 2024-05-09T05:49:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9c97fff640 1 -- 172.21.15.115:0/840580624 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ca400d1c0 con 0x7f9cb0106b10 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.399+0000 7f9c97fff640 1 -- 172.21.15.115:0/840580624 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4003850 con 0x7f9cb0106b10 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.400+0000 7f9cb56d9640 1 -- 172.21.15.115:0/840580624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb00ff150 con 0x7f9cb0106b10 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.402+0000 7f9cbc58e740 1 Processor -- start 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.402+0000 7f9cbc58e740 1 -- start start 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9c97fff640 1 -- 172.21.15.115:0/840580624 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f9ca401f020 con 0x7f9cb0106b10 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 0x55c935c5f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c5fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 0x55c935c5ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c935bc47c0 con 0x55c9358932f0 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c935b600e0 con 0x55c93589b4d0 2024-05-09T05:49:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c935bc0780 con 0x55c9358bc1b0 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9caeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c5fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9caeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c5fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39484/0 (socket says 172.21.15.115:39484) 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9caeffd640 1 -- 172.21.15.115:0/951477655 learned_addr learned my addr 172.21.15.115:0/951477655 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/951477655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 0x55c935c5ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9caf7fe640 1 --2- 172.21.15.115:0/951477655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 0x55c935c5f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9caeffd640 1 -- 172.21.15.115:0/951477655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 msgr2=0x55c935c5f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9caeffd640 1 --2- 172.21.15.115:0/951477655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 0x55c935c5f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9caeffd640 1 -- 172.21.15.115:0/951477655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 msgr2=0x55c935c5ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9caeffd640 1 --2- 172.21.15.115:0/951477655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 0x55c935c5ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.403+0000 7f9caeffd640 1 -- 172.21.15.115:0/951477655 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c935b897c0 con 0x55c9358932f0 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.404+0000 7f9c97fff640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c980b2950 0x7f9c980b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.404+0000 7f9caeffd640 1 --2- 172.21.15.115:0/951477655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c5fa50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009c90 tx=0x7f9ca00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233085ed808fe68c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.404+0000 7f9caf7fe640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c980b2950 0x7f9c980b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.404+0000 7f9c957fa640 1 -- 172.21.15.115:0/951477655 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0012450 con 0x55c9358932f0 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.404+0000 7f9c97fff640 1 -- 172.21.15.115:0/840580624 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7f9ca40e19d0 con 0x7f9cb0106b10 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.404+0000 7f9c957fa640 1 -- 172.21.15.115:0/951477655 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ca0012a80 con 0x55c9358932f0 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.404+0000 7f9c957fa640 1 -- 172.21.15.115:0/951477655 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000fa40 con 0x55c9358932f0 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 -- 172.21.15.115:0/951477655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 msgr2=0x55c935c5fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9caf7fe640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c980b2950 0x7f9c980b4e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006010 tx=0x7f9ca8009800 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/951477655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c5fa50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009c90 tx=0x7f9ca00117b0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 -- 172.21.15.115:0/951477655 shutdown_connections 2024-05-09T05:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/951477655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 0x55c935c5ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/951477655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c5fa50 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/951477655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 0x55c935c5f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 -- 172.21.15.115:0/951477655 >> 172.21.15.115:0/951477655 conn(0x55c9358b0300 msgr2=0x55c935c5be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 -- 172.21.15.115:0/951477655 shutdown_connections 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 -- 172.21.15.115:0/951477655 wait complete. 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 -- start start 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.405+0000 7f9cbc58e740 1 Processor -- start 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c6fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 0x55c935c70010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 0x55c935c6dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c935c70c20 con 0x55c9358932f0 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c935b4b9c0 con 0x55c93589b4d0 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c935c70da0 con 0x55c9358bc1b0 2024-05-09T05:49:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9cbc58e740 10 client.0 _ll_register_callbacks cb 0x55c935b78020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c6fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c6fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39486/0 (socket says 172.21.15.115:39486) 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9cae7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 0x55c935c70010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caeffd640 1 -- 172.21.15.115:0/1098645010 learned_addr learned my addr 172.21.15.115:0/1098645010 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caf7fe640 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 0x55c935c6dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caeffd640 1 -- 172.21.15.115:0/1098645010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 msgr2=0x55c935c6dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caeffd640 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 0x55c935c6dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caeffd640 1 -- 172.21.15.115:0/1098645010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 msgr2=0x55c935c70010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caeffd640 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 0x55c935c70010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caeffd640 1 -- 172.21.15.115:0/1098645010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c935b721f0 con 0x55c9358932f0 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caf7fe640 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 0x55c935c6dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9caeffd640 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c6fad0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9ca001c040 tx=0x7f9ca0012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91413b075e124735 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.406+0000 7f9c92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1098645010 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0028020 con 0x55c9358932f0 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1098645010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ca0012a80 con 0x55c9358932f0 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1098645010 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7f9ca000f990 con 0x55c9358932f0 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1098645010 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9ca0033050 con 0x55c9358932f0 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9cbc58e740 10 client.18315 fetch_fsmap learned FSMap version 55 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1098645010 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c935927790 con 0x55c9358932f0 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1098645010 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ca0012a80 con 0x55c9358932f0 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1098645010 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f9ca0005470 con 0x55c9358932f0 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9cbc58e740 10 client.18315 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9cbc58e740 10 client.18315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9cbc58e740 20 client.18315 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c93587ad80 0x55c935c802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9cbc58e740 4 client.18315 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:49:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.407+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1098645010 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c935bd23d0 con 0x55c93587ad80 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.408+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c93587ad80 0x55c935c802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.408+0000 7f9c92ff5640 10 client.18315 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.408+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c93587ad80 0x55c935c802d0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcdff3cbf2815005 server_cookie=60bb47784bb6bc27 in_seq=0 out_seq=0 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1098645010 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c935bd23d0 con 0x55c93587ad80 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9c92ff5640 10 client.18315 handle_command_reply: tid=0 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 client.18315 shutdown 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1098645010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 msgr2=0x55c935c6fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c6fad0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9ca001c040 tx=0x7f9ca0012700 comp rx=0 tx=0).stop 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1098645010 shutdown_connections 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c93587ad80 0x55c935c802d0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358bc1b0 0x55c935c6dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c93589b4d0 0x55c935c70010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1098645010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358932f0 0x55c935c6fad0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1098645010 >> 172.21.15.115:0/1098645010 conn(0x55c9358b0300 msgr2=0x55c935c613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9c92ff5640 0 client.18315 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1098645010 shutdown_connections 2024-05-09T05:49:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1098645010 wait complete. 2024-05-09T05:49:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.409+0000 7f9cbc58e740 20 client.18315 trim_cache size 0 max 16384 2024-05-09T05:49:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cbc58e740 1 Processor -- start 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cbc58e740 1 -- start start 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358932f0 0x55c935c6cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c93589b4d0 0x55c935c678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358bc1b0 0x55c935c67e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01aef10 con 0x55c9358bc1b0 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01b0ab0 con 0x55c9358932f0 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01af5e0 con 0x55c93589b4d0 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cae7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c93589b4d0 0x55c935c678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cae7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c93589b4d0 0x55c935c678c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:49694/0 (socket says 172.21.15.115:49694) 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9cae7fc640 1 -- 172.21.15.115:0/4205991908 learned_addr learned my addr 172.21.15.115:0/4205991908 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9caf7fe640 1 --2- 172.21.15.115:0/4205991908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358bc1b0 0x55c935c67e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.424+0000 7f9caeffd640 1 --2- 172.21.15.115:0/4205991908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358932f0 0x55c935c6cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.425+0000 7f9cae7fc640 1 -- 172.21.15.115:0/4205991908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358932f0 msgr2=0x55c935c6cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.425+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/4205991908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358932f0 0x55c935c6cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.425+0000 7f9cae7fc640 1 -- 172.21.15.115:0/4205991908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358bc1b0 msgr2=0x55c935c67e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.425+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/4205991908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358bc1b0 0x55c935c67e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.425+0000 7f9cae7fc640 1 -- 172.21.15.115:0/4205991908 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93599e5f0 con 0x55c93589b4d0 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.425+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/4205991908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c93589b4d0 0x55c935c678c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40dee40 tx=0x7f9ca40e3f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d9469f73f45f7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.425+0000 7f9c94ff9640 1 -- 172.21.15.115:0/4205991908 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca401c040 con 0x55c93589b4d0 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.425+0000 7f9c94ff9640 1 -- 172.21.15.115:0/4205991908 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ca40b3310 con 0x55c93589b4d0 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.425+0000 7f9c94ff9640 1 -- 172.21.15.115:0/4205991908 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40aab70 con 0x55c93589b4d0 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 -- 172.21.15.115:0/4205991908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c93589b4d0 msgr2=0x55c935c678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/4205991908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c93589b4d0 0x55c935c678c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40dee40 tx=0x7f9ca40e3f50 comp rx=0 tx=0).stop 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 -- 172.21.15.115:0/4205991908 shutdown_connections 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/4205991908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9358bc1b0 0x55c935c67e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/4205991908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c93589b4d0 0x55c935c678c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/4205991908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358932f0 0x55c935c6cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 -- 172.21.15.115:0/4205991908 >> 172.21.15.115:0/4205991908 conn(0x55c9358b0300 msgr2=0x55c935c6c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 -- 172.21.15.115:0/4205991908 shutdown_connections 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 -- 172.21.15.115:0/4205991908 wait complete. 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 -- start start 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.426+0000 7f9cbc58e740 1 Processor -- start 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358932f0 0x55c935c83490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c93589b4d0 0x55c935c839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cbc58e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358bc1b0 0x55c935c80910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01b03e0 con 0x55c93589b4d0 2024-05-09T05:49:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01afd10 con 0x55c9358bc1b0 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cbc58e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01acef0 con 0x55c9358932f0 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cbc58e740 10 client.0 _ll_register_callbacks cb 0x7f9ca40e5390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cae7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c93589b4d0 0x55c935c839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9caeffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358932f0 0x55c935c83490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cae7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c93589b4d0 0x55c935c839d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39504/0 (socket says 172.21.15.115:39504) 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cae7fc640 1 -- 172.21.15.115:0/1035911776 learned_addr learned my addr 172.21.15.115:0/1035911776 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9caf7fe640 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358bc1b0 0x55c935c80910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cae7fc640 1 -- 172.21.15.115:0/1035911776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358932f0 msgr2=0x55c935c83490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358932f0 0x55c935c83490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cae7fc640 1 -- 172.21.15.115:0/1035911776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358bc1b0 msgr2=0x55c935c80910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358bc1b0 0x55c935c80910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cae7fc640 1 -- 172.21.15.115:0/1035911776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c935b88ed0 con 0x55c93589b4d0 2024-05-09T05:49:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9caeffd640 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358932f0 0x55c935c83490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9cae7fc640 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c93589b4d0 0x55c935c839d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40e3010 tx=0x7f9ca40df310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c176cb27364f0535 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.427+0000 7f9c92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1035911776 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca401c040 con 0x55c93589b4d0 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1035911776 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7f9ca40e5030 con 0x55c93589b4d0 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1035911776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c935bcc240 con 0x55c93589b4d0 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1035911776 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9ca40ab270 con 0x55c93589b4d0 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9cbc58e740 10 client.18321 fetch_fsmap learned FSMap version 55 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1035911776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c935ce7d30 con 0x55c93589b4d0 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1035911776 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c935bcc240 con 0x55c93589b4d0 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1035911776 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f9ca40b3540 con 0x55c93589b4d0 2024-05-09T05:49:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9cbc58e740 10 client.18321 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:49:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9cbc58e740 10 client.18321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9cbc58e740 20 client.18321 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c9358501c0 0x55c935cea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9cbc58e740 4 client.18321 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:49:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1035911776 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c935b47350 con 0x55c9358501c0 2024-05-09T05:49:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.428+0000 7f9caeffd640 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c9358501c0 0x55c935cea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.429+0000 7f9c92ff5640 10 client.18321 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.429+0000 7f9caeffd640 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c9358501c0 0x55c935cea350 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4d8acfe6766616 server_cookie=ab2e6e08bdbce4bc in_seq=0 out_seq=0 2024-05-09T05:49:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9c92ff5640 1 -- 172.21.15.115:0/1035911776 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c935b47350 con 0x55c9358501c0 2024-05-09T05:49:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9c92ff5640 10 client.18321 handle_command_reply: tid=0 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 client.18321 shutdown 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1035911776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c93589b4d0 msgr2=0x55c935c839d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c93589b4d0 0x55c935c839d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40e3010 tx=0x7f9ca40df310 comp rx=0 tx=0).stop 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1035911776 shutdown_connections 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9358bc1b0 0x55c935c80910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c93589b4d0 0x55c935c839d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c9358501c0 0x55c935cea350 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 --2- 172.21.15.115:0/1035911776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9358932f0 0x55c935c83490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1035911776 >> 172.21.15.115:0/1035911776 conn(0x55c9358b0300 msgr2=0x55c935c61910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9c92ff5640 0 client.18321 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1035911776 shutdown_connections 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 1 -- 172.21.15.115:0/1035911776 wait complete. 2024-05-09T05:49:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.437+0000 7f9cbc58e740 20 client.18321 trim_cache size 0 max 16384 2024-05-09T05:49:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:49:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:49:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 -- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c980b2950 msgr2=0x7f9c980b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c980b2950 0x7f9c980b4e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006010 tx=0x7f9ca8009800 comp rx=0 tx=0).stop 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 -- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb0106b10 msgr2=0x7f9cb0199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb0106b10 0x7f9cb0199be0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4005fd0 tx=0x7f9ca4009e00 comp rx=0 tx=0).stop 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 -- 172.21.15.115:0/840580624 shutdown_connections 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f9c980b2950 0x7f9c980b4e60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9cb01075d0 0x7f9cb019dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb0106b10 0x7f9cb0199be0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 --2- 172.21.15.115:0/840580624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb0102580 0x7f9cb0199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 -- 172.21.15.115:0/840580624 >> 172.21.15.115:0/840580624 conn(0x7f9cb00779e0 msgr2=0x7f9cb0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 -- 172.21.15.115:0/840580624 shutdown_connections 2024-05-09T05:49:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:17.438+0000 7f9cb56d9640 1 -- 172.21.15.115:0/840580624 wait complete. 2024-05-09T05:49:17.460 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:17 smithi161 ceph-mon[33805]: pgmap v347: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-05-09T05:49:17.460 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:17 smithi161 ceph-mon[33805]: daemon mds.l restarted 2024-05-09T05:49:17.551 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:17 smithi195 ceph-mon[33844]: pgmap v347: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-05-09T05:49:17.551 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:17 smithi195 ceph-mon[33844]: daemon mds.l restarted 2024-05-09T05:49:17.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:17 smithi115 ceph-mon[30349]: pgmap v347: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-05-09T05:49:17.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:17 smithi115 ceph-mon[30349]: daemon mds.l restarted 2024-05-09T05:49:18.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:18 smithi195 ceph-mon[33844]: osdmap e101: 12 total, 12 up, 12 in 2024-05-09T05:49:18.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:18 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6832/197242519,v1:172.21.15.195:6833/197242519] up:boot 2024-05-09T05:49:18.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:18 smithi195 ceph-mon[33844]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:49:18.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:18 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:49:18.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:18 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:49:18.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:18 smithi115 ceph-mon[30349]: osdmap e101: 12 total, 12 up, 12 in 2024-05-09T05:49:18.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:18 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6832/197242519,v1:172.21.15.195:6833/197242519] up:boot 2024-05-09T05:49:18.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:18 smithi115 ceph-mon[30349]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:49:18.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:18 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:49:18.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:18 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:49:18.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:18 smithi161 ceph-mon[33805]: osdmap e101: 12 total, 12 up, 12 in 2024-05-09T05:49:18.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:18 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6832/197242519,v1:172.21.15.195:6833/197242519] up:boot 2024-05-09T05:49:18.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:18 smithi161 ceph-mon[33805]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:49:18.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:18 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:49:18.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:18 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:49:19.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:19 smithi195 ceph-mon[33844]: pgmap v349: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-05-09T05:49:19.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:19 smithi115 ceph-mon[30349]: pgmap v349: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-05-09T05:49:19.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:19 smithi161 ceph-mon[33805]: pgmap v349: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-05-09T05:49:21.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:21 smithi195 ceph-mon[33844]: pgmap v350: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:21.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:21 smithi115 ceph-mon[30349]: pgmap v350: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:21.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:21 smithi161 ceph-mon[33805]: pgmap v350: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:23.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:23 smithi195 ceph-mon[33844]: pgmap v351: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:23.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:23 smithi115 ceph-mon[30349]: pgmap v351: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:23.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:23 smithi161 ceph-mon[33805]: pgmap v351: 129 pgs: 129 active+clean; 9.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:25.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:25 smithi195 ceph-mon[33844]: pgmap v352: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:25.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:25 smithi115 ceph-mon[30349]: pgmap v352: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:25.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:25 smithi161 ceph-mon[33805]: pgmap v352: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:27.363 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:27 smithi195 ceph-mon[33844]: pgmap v353: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:27.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:27 smithi115 ceph-mon[30349]: pgmap v353: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:27.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:27 smithi161 ceph-mon[33805]: pgmap v353: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-05-09T05:49:29.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:29 smithi195 ceph-mon[33844]: pgmap v354: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 38 MiB/s wr, 33 op/s 2024-05-09T05:49:29.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:29 smithi115 ceph-mon[30349]: pgmap v354: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 38 MiB/s wr, 33 op/s 2024-05-09T05:49:29.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:29 smithi161 ceph-mon[33805]: pgmap v354: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 38 MiB/s wr, 33 op/s 2024-05-09T05:49:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.558+0000 7fcb162c6640 1 -- 172.21.15.115:0/1059853533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 msgr2=0x7fcb10102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.558+0000 7fcb162c6640 1 --2- 172.21.15.115:0/1059853533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 0x7fcb10102740 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009900 tx=0x7fcb0002a410 comp rx=0 tx=0).stop 2024-05-09T05:49:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.558+0000 7fcb162c6640 1 -- 172.21.15.115:0/1059853533 shutdown_connections 2024-05-09T05:49:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.558+0000 7fcb162c6640 1 --2- 172.21.15.115:0/1059853533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb101032f0 0x7fcb1010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.558+0000 7fcb162c6640 1 --2- 172.21.15.115:0/1059853533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb10109040 0x7fcb10102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.558+0000 7fcb162c6640 1 --2- 172.21.15.115:0/1059853533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 0x7fcb10102740 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.558+0000 7fcb162c6640 1 -- 172.21.15.115:0/1059853533 >> 172.21.15.115:0/1059853533 conn(0x7fcb100782e0 msgr2=0x7fcb10078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.558+0000 7fcb162c6640 1 -- 172.21.15.115:0/1059853533 shutdown_connections 2024-05-09T05:49:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.558+0000 7fcb162c6640 1 -- 172.21.15.115:0/1059853533 wait complete. 2024-05-09T05:49:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb162c6640 1 Processor -- start 2024-05-09T05:49:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb162c6640 1 -- start start 2024-05-09T05:49:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb162c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb101032f0 0x7fcb101a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb162c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 0x7fcb101a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb162c6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10109040 0x7fcb1019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb162c6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10112c90 con 0x7fcb10108ba0 2024-05-09T05:49:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb162c6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10112b10 con 0x7fcb10109040 2024-05-09T05:49:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb162c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10112e10 con 0x7fcb101032f0 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb0f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 0x7fcb101a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb0f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 0x7fcb101a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34870/0 (socket says 172.21.15.115:34870) 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/3229778203 learned_addr learned my addr 172.21.15.115:0/3229778203 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb101032f0 0x7fcb101a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.559+0000 7fcb14863640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10109040 0x7fcb1019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/3229778203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb101032f0 msgr2=0x7fcb101a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb101032f0 0x7fcb101a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/3229778203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10109040 msgr2=0x7fcb1019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10109040 0x7fcb1019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/3229778203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1019ad90 con 0x7fcb10108ba0 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 0x7fcb101a0a20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400e9f0 tx=0x7fcb0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb0d7fa640 1 -- 172.21.15.115:0/3229778203 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400ccc0 con 0x7fcb10108ba0 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb162c6640 1 -- 172.21.15.115:0/3229778203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1019b020 con 0x7fcb10108ba0 2024-05-09T05:49:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb0d7fa640 1 -- 172.21.15.115:0/3229778203 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb040035f0 con 0x7fcb10108ba0 2024-05-09T05:49:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb0d7fa640 1 -- 172.21.15.115:0/3229778203 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400ce60 con 0x7fcb10108ba0 2024-05-09T05:49:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.560+0000 7fcb162c6640 1 -- 172.21.15.115:0/3229778203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a7280 con 0x7fcb10108ba0 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.563+0000 7fcb0d7fa640 1 -- 172.21.15.115:0/3229778203 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcb04004000 con 0x7fcb10108ba0 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.563+0000 7fcb1d177740 1 Processor -- start 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.563+0000 7fcb1d177740 1 -- start start 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 0x562083744680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x562083744bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083745100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb1d177740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5620836a97c0 con 0x5620833804d0 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb1d177740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620836450e0 con 0x5620833782f0 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb1d177740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5620836a5780 con 0x5620833a11b0 2024-05-09T05:49:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0d7fa640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf80b2950 0x7fcaf80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083745100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083745100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34872/0 (socket says 172.21.15.115:34872) 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb14863640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 0x562083744680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/1288845209 learned_addr learned my addr 172.21.15.115:0/1288845209 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb14863640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf80b2950 0x7fcaf80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0d7fa640 1 -- 172.21.15.115:0/3229778203 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7fcb040e2c50 con 0x7fcb10108ba0 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/1288845209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x562083744bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/1288845209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 msgr2=0x562083744680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/1288845209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 0x562083744680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/1288845209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 msgr2=0x562083744bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/1288845209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x562083744bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/1288845209 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56208366e7c0 con 0x5620833804d0 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb14863640 1 --2- 172.21.15.115:0/1288845209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 0x562083744680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:49:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.564+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/1288845209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083745100 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fcb04015020 tx=0x7fcb040a5b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218196607f697f10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.565+0000 7fcb14863640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf80b2950 0x7fcaf80b4e60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5620837417f0 tx=0x7fcb08009500 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.565+0000 7fcaf67fc640 1 -- 172.21.15.115:0/1288845209 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb040a5380 con 0x5620833804d0 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.565+0000 7fcaf67fc640 1 -- 172.21.15.115:0/1288845209 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb040b23a0 con 0x5620833804d0 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.565+0000 7fcaf67fc640 1 -- 172.21.15.115:0/1288845209 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb040b2a50 con 0x5620833804d0 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 -- 172.21.15.115:0/1288845209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 msgr2=0x562083745100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 --2- 172.21.15.115:0/1288845209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083745100 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fcb04015020 tx=0x7fcb040a5b30 comp rx=0 tx=0).stop 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 -- 172.21.15.115:0/1288845209 shutdown_connections 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 --2- 172.21.15.115:0/1288845209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083745100 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 --2- 172.21.15.115:0/1288845209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x562083744bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 --2- 172.21.15.115:0/1288845209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 0x562083744680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 -- 172.21.15.115:0/1288845209 >> 172.21.15.115:0/1288845209 conn(0x562083395300 msgr2=0x562083740e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 -- 172.21.15.115:0/1288845209 shutdown_connections 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 -- 172.21.15.115:0/1288845209 wait complete. 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 -- start start 2024-05-09T05:49:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.566+0000 7fcb1d177740 1 Processor -- start 2024-05-09T05:49:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x5620837506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 0x562083750c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833a11b0 0x56208374eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb1d177740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5620837517b0 con 0x5620833a11b0 2024-05-09T05:49:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb1d177740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620836309c0 con 0x5620833782f0 2024-05-09T05:49:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb1d177740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562083751930 con 0x5620833804d0 2024-05-09T05:49:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb14863640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833a11b0 0x56208374eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb14863640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833a11b0 0x56208374eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34880/0 (socket says 172.21.15.115:34880) 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb14863640 1 -- 172.21.15.115:0/154382839 learned_addr learned my addr 172.21.15.115:0/154382839 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x5620837506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb1d177740 10 client.0 _ll_register_callbacks cb 0x56208365d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 0x562083750c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb14863640 1 -- 172.21.15.115:0/154382839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 msgr2=0x562083750c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb14863640 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 0x562083750c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb14863640 1 -- 172.21.15.115:0/154382839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 msgr2=0x5620837506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb14863640 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x5620837506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb14863640 1 -- 172.21.15.115:0/154382839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620836571f0 con 0x5620833a11b0 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x5620837506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcb14863640 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833a11b0 0x56208374eb90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fcb08019180 tx=0x7fcb08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7428d76e7a38228 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.567+0000 7fcaef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcaef7fe640 1 -- 172.21.15.115:0/154382839 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0802d060 con 0x5620833a11b0 2024-05-09T05:49:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcb1d177740 1 -- 172.21.15.115:0/154382839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb040b23a0 con 0x5620833a11b0 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcaef7fe640 1 -- 172.21.15.115:0/154382839 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7fcb080047b0 con 0x5620833a11b0 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcaef7fe640 1 -- 172.21.15.115:0/154382839 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcb08034050 con 0x5620833a11b0 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcb1d177740 10 client.18333 fetch_fsmap learned FSMap version 55 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcb1d177740 1 -- 172.21.15.115:0/154382839 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56208340c790 con 0x5620833a11b0 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcaef7fe640 1 -- 172.21.15.115:0/154382839 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb040b23a0 con 0x5620833a11b0 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcaef7fe640 1 -- 172.21.15.115:0/154382839 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fcb08027030 con 0x5620833a11b0 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcb1d177740 10 client.18333 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcb1d177740 10 client.18333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcb1d177740 20 client.18333 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcb1d177740 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56208335fd80 0x562083760f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcb1d177740 4 client.18333 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.568+0000 7fcb1d177740 1 -- 172.21.15.115:0/154382839 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620836b73d0 con 0x56208335fd80 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.569+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56208335fd80 0x562083760f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.569+0000 7fcaef7fe640 10 client.18333 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.569+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56208335fd80 0x562083760f50 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671025848ddf57c3 server_cookie=2893a4abc7f208fe in_seq=0 out_seq=0 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcaef7fe640 1 -- 172.21.15.115:0/154382839 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620836b73d0 con 0x56208335fd80 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcaef7fe640 10 client.18333 handle_command_reply: tid=0 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 client.18333 shutdown 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 -- 172.21.15.115:0/154382839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833a11b0 msgr2=0x56208374eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833a11b0 0x56208374eb90 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fcb08019180 tx=0x7fcb08004a40 comp rx=0 tx=0).stop 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 -- 172.21.15.115:0/154382839 shutdown_connections 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833a11b0 0x56208374eb90 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56208335fd80 0x562083760f50 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 0x562083750c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 --2- 172.21.15.115:0/154382839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x5620837506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 -- 172.21.15.115:0/154382839 >> 172.21.15.115:0/154382839 conn(0x562083395300 msgr2=0x56208373f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcaef7fe640 0 client.18333 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 -- 172.21.15.115:0/154382839 shutdown_connections 2024-05-09T05:49:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 1 -- 172.21.15.115:0/154382839 wait complete. 2024-05-09T05:49:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.570+0000 7fcb1d177740 20 client.18333 trim_cache size 0 max 16384 2024-05-09T05:49:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb1d177740 1 Processor -- start 2024-05-09T05:49:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb1d177740 1 -- start start 2024-05-09T05:49:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833782f0 0x562083748400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 0x562083748940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833a11b0 0x56208374d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb1d177740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b10f0 con 0x5620833782f0 2024-05-09T05:49:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb1d177740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b2c90 con 0x5620833a11b0 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb1d177740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b17c0 con 0x5620833804d0 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb0ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833782f0 0x562083748400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb0ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833782f0 0x562083748400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34884/0 (socket says 172.21.15.115:34884) 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb0ffff640 1 -- 172.21.15.115:0/2106917153 learned_addr learned my addr 172.21.15.115:0/2106917153 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 0x562083748940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb14863640 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833a11b0 0x56208374d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb0ffff640 1 -- 172.21.15.115:0/2106917153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 msgr2=0x562083748940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.587+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 0x562083748940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.588+0000 7fcb0ffff640 1 -- 172.21.15.115:0/2106917153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833a11b0 msgr2=0x56208374d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.588+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833a11b0 0x56208374d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.588+0000 7fcb0ffff640 1 -- 172.21.15.115:0/2106917153 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620834835f0 con 0x5620833782f0 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.588+0000 7fcb14863640 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833a11b0 0x56208374d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.588+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833782f0 0x562083748400 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fcb10103880 tx=0x7fcb00000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804d5493a8a43c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.588+0000 7fcaf5ffb640 1 -- 172.21.15.115:0/2106917153 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00035040 con 0x5620833782f0 2024-05-09T05:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.588+0000 7fcaf5ffb640 1 -- 172.21.15.115:0/2106917153 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb0004b070 con 0x5620833782f0 2024-05-09T05:49:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.588+0000 7fcaf5ffb640 1 -- 172.21.15.115:0/2106917153 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00046ad0 con 0x5620833782f0 2024-05-09T05:49:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 -- 172.21.15.115:0/2106917153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833782f0 msgr2=0x562083748400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833782f0 0x562083748400 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fcb10103880 tx=0x7fcb00000bc0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 -- 172.21.15.115:0/2106917153 shutdown_connections 2024-05-09T05:49:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833a11b0 0x56208374d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833804d0 0x562083748940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 --2- 172.21.15.115:0/2106917153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833782f0 0x562083748400 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 -- 172.21.15.115:0/2106917153 >> 172.21.15.115:0/2106917153 conn(0x562083395300 msgr2=0x562083749d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 -- 172.21.15.115:0/2106917153 shutdown_connections 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 -- 172.21.15.115:0/2106917153 wait complete. 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.589+0000 7fcb1d177740 1 -- start start 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb1d177740 1 Processor -- start 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x5620837570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083751b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb1d177740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 0x5620837520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb1d177740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b25c0 con 0x5620833804d0 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb1d177740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b1ef0 con 0x5620833782f0 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb1d177740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101af0d0 con 0x5620833a11b0 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb1d177740 10 client.0 _ll_register_callbacks cb 0x7fcb08009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb0f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083751b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb0f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083751b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34890/0 (socket says 172.21.15.115:34890) 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/3139236591 learned_addr learned my addr 172.21.15.115:0/3139236591 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb14863640 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 0x5620837520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x5620837570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/3139236591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 msgr2=0x5620837520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 0x5620837520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/3139236591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 msgr2=0x5620837570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x5620837570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.590+0000 7fcb0f7fe640 1 -- 172.21.15.115:0/3139236591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56208366ded0 con 0x5620833804d0 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcb0f7fe640 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083751b80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fcb040e0110 tx=0x7fcb040039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f5b3a731f76fd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcaef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcaef7fe640 1 -- 172.21.15.115:0/3139236591 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb040a9bf0 con 0x5620833804d0 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcaef7fe640 1 -- 172.21.15.115:0/3139236591 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7fcb040a5e80 con 0x5620833804d0 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcb1d177740 1 -- 172.21.15.115:0/3139236591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb0004b070 con 0x5620833804d0 2024-05-09T05:49:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcaef7fe640 1 -- 172.21.15.115:0/3139236591 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcb04084030 con 0x5620833804d0 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcb1d177740 10 client.18339 fetch_fsmap learned FSMap version 55 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcb1d177740 1 -- 172.21.15.115:0/3139236591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5620837c7ee0 con 0x5620833804d0 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcaef7fe640 1 -- 172.21.15.115:0/3139236591 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb0004b070 con 0x5620833804d0 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcaef7fe640 1 -- 172.21.15.115:0/3139236591 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fcb040042f0 con 0x5620833804d0 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcb1d177740 10 client.18339 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcb1d177740 10 client.18339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.591+0000 7fcb1d177740 20 client.18339 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.592+0000 7fcb1d177740 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5620833351c0 0x5620837ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.592+0000 7fcb1d177740 4 client.18339 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.592+0000 7fcb1d177740 1 -- 172.21.15.115:0/3139236591 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620836b1240 con 0x5620833351c0 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.592+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5620833351c0 0x5620837ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.592+0000 7fcaef7fe640 10 client.18339 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.592+0000 7fcb0ffff640 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5620833351c0 0x5620837ca1d0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80dfc60143040f06 server_cookie=9dc2f195dbfc04ba in_seq=0 out_seq=0 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.599+0000 7fcaef7fe640 1 -- 172.21.15.115:0/3139236591 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5620836b1240 con 0x5620833351c0 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.599+0000 7fcaef7fe640 10 client.18339 handle_command_reply: tid=0 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.599+0000 7fcb1d177740 1 client.18339 shutdown 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 -- 172.21.15.115:0/3139236591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 msgr2=0x562083751b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083751b80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fcb040e0110 tx=0x7fcb040039b0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 -- 172.21.15.115:0/3139236591 shutdown_connections 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5620833351c0 0x5620837ca1d0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5620833a11b0 0x5620837520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcaef7fe640 0 client.18339 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5620833804d0 0x562083751b80 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 --2- 172.21.15.115:0/3139236591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620833782f0 0x5620837570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 -- 172.21.15.115:0/3139236591 >> 172.21.15.115:0/3139236591 conn(0x562083395300 msgr2=0x56208374d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 -- 172.21.15.115:0/3139236591 shutdown_connections 2024-05-09T05:49:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 1 -- 172.21.15.115:0/3139236591 wait complete. 2024-05-09T05:49:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb1d177740 20 client.18339 trim_cache size 0 max 16384 2024-05-09T05:49:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:49:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:49:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:49:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb162c6640 1 -- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf80b2950 msgr2=0x7fcaf80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb162c6640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf80b2950 0x7fcaf80b4e60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5620837417f0 tx=0x7fcb08009500 comp rx=0 tx=0).stop 2024-05-09T05:49:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb162c6640 1 -- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 msgr2=0x7fcb101a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.600+0000 7fcb162c6640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 0x7fcb101a0a20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400e9f0 tx=0x7fcb0400eeb0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.601+0000 7fcb162c6640 1 -- 172.21.15.115:0/3229778203 shutdown_connections 2024-05-09T05:49:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.601+0000 7fcb162c6640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf80b2950 0x7fcaf80b4e60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.601+0000 7fcb162c6640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb10109040 0x7fcb1019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.601+0000 7fcb162c6640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb10108ba0 0x7fcb101a0a20 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.601+0000 7fcb162c6640 1 --2- 172.21.15.115:0/3229778203 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb101032f0 0x7fcb101a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.601+0000 7fcb162c6640 1 -- 172.21.15.115:0/3229778203 >> 172.21.15.115:0/3229778203 conn(0x7fcb100782e0 msgr2=0x7fcb1006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.601+0000 7fcb162c6640 1 -- 172.21.15.115:0/3229778203 shutdown_connections 2024-05-09T05:49:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:30.601+0000 7fcb162c6640 1 -- 172.21.15.115:0/3229778203 wait complete. 2024-05-09T05:49:31.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:31 smithi195 ceph-mon[33844]: pgmap v355: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:49:31.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:31 smithi115 ceph-mon[30349]: pgmap v355: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:49:31.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:31 smithi161 ceph-mon[33805]: pgmap v355: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:49:33.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:33 smithi195 ceph-mon[33844]: pgmap v356: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:49:33.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:33 smithi115 ceph-mon[30349]: pgmap v356: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:49:33.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:33 smithi161 ceph-mon[33805]: pgmap v356: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-05-09T05:49:35.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:35 smithi195 ceph-mon[33844]: pgmap v357: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:35.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:35 smithi115 ceph-mon[30349]: pgmap v357: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:35.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:35 smithi161 ceph-mon[33805]: pgmap v357: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:37.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:37 smithi195 ceph-mon[33844]: pgmap v358: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:37.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:37 smithi115 ceph-mon[30349]: pgmap v358: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:37.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:37 smithi161 ceph-mon[33805]: pgmap v358: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:38.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:38 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:49:38.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:38 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:49:38.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:38 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:49:39.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:39 smithi195 ceph-mon[33844]: pgmap v359: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:39.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:39 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:49:39.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:39 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:49:39.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:39 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:49:39.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:39 smithi115 ceph-mon[30349]: pgmap v359: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:39.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:39 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:49:39.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:39 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:49:39.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:39 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:49:39.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:39 smithi161 ceph-mon[33805]: pgmap v359: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:39.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:39 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:49:39.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:39 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:49:39.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:39 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:49:41.614 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:41 smithi195 ceph-mon[33844]: pgmap v360: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:41.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:41 smithi115 ceph-mon[30349]: pgmap v360: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:41.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:41 smithi161 ceph-mon[33805]: pgmap v360: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:43.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:43 smithi195 ceph-mon[33844]: pgmap v361: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:43.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:43 smithi115 ceph-mon[30349]: pgmap v361: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:43.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:43 smithi161 ceph-mon[33805]: pgmap v361: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.924+0000 7f3b38661640 1 -- 172.21.15.115:0/4016471450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 msgr2=0x7f3b30102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.924+0000 7f3b38661640 1 --2- 172.21.15.115:0/4016471450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 0x7f3b30102990 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009900 tx=0x7f3b2c02a410 comp rx=0 tx=0).stop 2024-05-09T05:49:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.924+0000 7f3b38661640 1 -- 172.21.15.115:0/4016471450 shutdown_connections 2024-05-09T05:49:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.924+0000 7f3b38661640 1 --2- 172.21.15.115:0/4016471450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b301075d0 0x7f3b3010ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.924+0000 7f3b38661640 1 --2- 172.21.15.115:0/4016471450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b30106b10 0x7f3b30106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.924+0000 7f3b38661640 1 --2- 172.21.15.115:0/4016471450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 0x7f3b30102990 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.924+0000 7f3b38661640 1 -- 172.21.15.115:0/4016471450 >> 172.21.15.115:0/4016471450 conn(0x7f3b300779e0 msgr2=0x7f3b30077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.924+0000 7f3b38661640 1 -- 172.21.15.115:0/4016471450 shutdown_connections 2024-05-09T05:49:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.924+0000 7f3b38661640 1 -- 172.21.15.115:0/4016471450 wait complete. 2024-05-09T05:49:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b38661640 1 Processor -- start 2024-05-09T05:49:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b38661640 1 -- start start 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b38661640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 0x7f3b30199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b38661640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b30106b10 0x7f3b30199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b38661640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b301075d0 0x7f3b3019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b38661640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301123b0 con 0x7f3b30102580 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b38661640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30112230 con 0x7f3b301075d0 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b38661640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30112530 con 0x7f3b30106b10 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b363fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 0x7f3b30199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b363fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 0x7f3b30199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53778/0 (socket says 172.21.15.115:53778) 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b363fd640 1 -- 172.21.15.115:0/3763988537 learned_addr learned my addr 172.21.15.115:0/3763988537 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b301075d0 0x7f3b3019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.925+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b30106b10 0x7f3b30199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b363fd640 1 -- 172.21.15.115:0/3763988537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b30106b10 msgr2=0x7f3b30199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b363fd640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b30106b10 0x7f3b30199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b363fd640 1 -- 172.21.15.115:0/3763988537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b301075d0 msgr2=0x7f3b3019df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b363fd640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b301075d0 0x7f3b3019df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b363fd640 1 -- 172.21.15.115:0/3763988537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3019e560 con 0x7f3b30102580 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b301075d0 0x7f3b3019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b30106b10 0x7f3b30199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b363fd640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 0x7f3b30199700 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009230 tx=0x7f3b2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b277fe640 1 -- 172.21.15.115:0/3763988537 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c03f030 con 0x7f3b30102580 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b277fe640 1 -- 172.21.15.115:0/3763988537 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b2c003a80 con 0x7f3b30102580 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b38661640 1 -- 172.21.15.115:0/3763988537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3019e7f0 con 0x7f3b30102580 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b277fe640 1 -- 172.21.15.115:0/3763988537 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c035070 con 0x7f3b30102580 2024-05-09T05:49:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.926+0000 7f3b38661640 1 -- 172.21.15.115:0/3763988537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3019ece0 con 0x7f3b30102580 2024-05-09T05:49:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.929+0000 7f3b277fe640 1 -- 172.21.15.115:0/3763988537 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f3b2c042030 con 0x7f3b30102580 2024-05-09T05:49:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.929+0000 7f3b277fe640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b1c0b2950 0x7f3b1c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.929+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b1c0b2950 0x7f3b1c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.930+0000 7f3b277fe640 1 -- 172.21.15.115:0/3763988537 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7f3b2c103270 con 0x7f3b30102580 2024-05-09T05:49:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.930+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b1c0b2950 0x7f3b1c0b4e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3b200058c0 tx=0x7f3b20005850 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.933+0000 7f3b3f5f9740 1 Processor -- start 2024-05-09T05:49:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.933+0000 7f3b3f5f9740 1 -- start start 2024-05-09T05:49:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.933+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c6824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.933+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c682a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.933+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c682f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.933+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561d2c5e77c0 con 0x561d2c2be4d0 2024-05-09T05:49:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.933+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561d2c5830e0 con 0x561d2c2df1b0 2024-05-09T05:49:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.933+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561d2c5e3780 con 0x561d2c2b62f0 2024-05-09T05:49:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b36bfe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c682f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b36bfe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c682f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53792/0 (socket says 172.21.15.115:53792) 2024-05-09T05:49:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b36bfe640 1 -- 172.21.15.115:0/4023611919 learned_addr learned my addr 172.21.15.115:0/4023611919 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b363fd640 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c6824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c682a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b36bfe640 1 -- 172.21.15.115:0/4023611919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 msgr2=0x561d2c682a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c682a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b36bfe640 1 -- 172.21.15.115:0/4023611919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 msgr2=0x561d2c6824d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c6824d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b36bfe640 1 -- 172.21.15.115:0/4023611919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2c5ac7c0 con 0x561d2c2be4d0 2024-05-09T05:49:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b363fd640 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c6824d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:49:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c682f50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009fd0 tx=0x7f3b28010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f784ba6a77ec265 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b24bfe640 1 -- 172.21.15.115:0/4023611919 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28011700 con 0x561d2c2be4d0 2024-05-09T05:49:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b24bfe640 1 -- 172.21.15.115:0/4023611919 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b28011d30 con 0x561d2c2be4d0 2024-05-09T05:49:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.934+0000 7f3b24bfe640 1 -- 172.21.15.115:0/4023611919 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2801be30 con 0x561d2c2be4d0 2024-05-09T05:49:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.935+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/4023611919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 msgr2=0x561d2c682f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.935+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c682f50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009fd0 tx=0x7f3b28010860 comp rx=0 tx=0).stop 2024-05-09T05:49:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.935+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/4023611919 shutdown_connections 2024-05-09T05:49:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.935+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c682f50 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.935+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c682a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.935+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/4023611919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c6824d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.935+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/4023611919 >> 172.21.15.115:0/4023611919 conn(0x561d2c2d3300 msgr2=0x561d2c67ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.935+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/4023611919 shutdown_connections 2024-05-09T05:49:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.935+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/4023611919 wait complete. 2024-05-09T05:49:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b3f5f9740 1 -- start start 2024-05-09T05:49:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b3f5f9740 1 Processor -- start 2024-05-09T05:49:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2b62f0 0x561d2c68ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2be4d0 0x561d2c68f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c694f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561d2c691a20 con 0x561d2c2b62f0 2024-05-09T05:49:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561d2c56e9c0 con 0x561d2c2df1b0 2024-05-09T05:49:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561d2c691ba0 con 0x561d2c2be4d0 2024-05-09T05:49:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b3f5f9740 10 client.0 _ll_register_callbacks cb 0x561d2c59b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b363fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2b62f0 0x561d2c68ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b363fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2b62f0 0x561d2c68ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53798/0 (socket says 172.21.15.115:53798) 2024-05-09T05:49:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b363fd640 1 -- 172.21.15.115:0/760393424 learned_addr learned my addr 172.21.15.115:0/760393424 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.936+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2be4d0 0x561d2c68f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c694f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b363fd640 1 -- 172.21.15.115:0/760393424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2be4d0 msgr2=0x561d2c68f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b363fd640 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2be4d0 0x561d2c68f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b363fd640 1 -- 172.21.15.115:0/760393424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 msgr2=0x561d2c694f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b363fd640 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c694f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b363fd640 1 -- 172.21.15.115:0/760393424 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2c5951f0 con 0x561d2c2b62f0 2024-05-09T05:49:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b363fd640 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2b62f0 0x561d2c68ef30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c003fd0 tx=0x7f3b2c0c9ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4242db095da5315 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:49:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b19ffb640 1 -- 172.21.15.115:0/760393424 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c049070 con 0x561d2c2b62f0 2024-05-09T05:49:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b19ffb640 1 -- 172.21.15.115:0/760393424 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7f3b2c0ce170 con 0x561d2c2b62f0 2024-05-09T05:49:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/760393424 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b28011d30 con 0x561d2c2b62f0 2024-05-09T05:49:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b19ffb640 1 -- 172.21.15.115:0/760393424 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3b2c08e020 con 0x561d2c2b62f0 2024-05-09T05:49:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b3f5f9740 10 client.18351 fetch_fsmap learned FSMap version 55 2024-05-09T05:49:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/760393424 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561d2c34a790 con 0x561d2c2b62f0 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.937+0000 7f3b19ffb640 1 -- 172.21.15.115:0/760393424 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b28011d30 con 0x561d2c2b62f0 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b19ffb640 1 -- 172.21.15.115:0/760393424 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f3b2c0ca470 con 0x561d2c2b62f0 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b3f5f9740 10 client.18351 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b3f5f9740 10 client.18351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b3f5f9740 20 client.18351 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561d2c29dd80 0x561d2c6a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b3f5f9740 4 client.18351 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/760393424 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d2c5f53d0 con 0x561d2c29dd80 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561d2c29dd80 0x561d2c6a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561d2c29dd80 0x561d2c6a11a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a015aae35cbde45 server_cookie=7081040d0a5f8e67 in_seq=0 out_seq=0 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.938+0000 7f3b19ffb640 10 client.18351 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b19ffb640 1 -- 172.21.15.115:0/760393424 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d2c5f53d0 con 0x561d2c29dd80 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b19ffb640 10 client.18351 handle_command_reply: tid=0 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b3f5f9740 1 client.18351 shutdown 2024-05-09T05:49:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/760393424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2b62f0 msgr2=0x561d2c68ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2b62f0 0x561d2c68ef30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c003fd0 tx=0x7f3b2c0c9ab0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/760393424 shutdown_connections 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c694f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561d2c29dd80 0x561d2c6a11a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2be4d0 0x561d2c68f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/760393424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2b62f0 0x561d2c68ef30 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b19ffb640 0 client.18351 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.939+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/760393424 >> 172.21.15.115:0/760393424 conn(0x561d2c2d3300 msgr2=0x561d2c684330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.940+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/760393424 shutdown_connections 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.940+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/760393424 wait complete. 2024-05-09T05:49:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.940+0000 7f3b3f5f9740 20 client.18351 trim_cache size 0 max 16384 2024-05-09T05:49:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b3f5f9740 1 Processor -- start 2024-05-09T05:49:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b3f5f9740 1 -- start start 2024-05-09T05:49:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c682e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2be4d0 0x561d2c6886c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2df1b0 0x561d2c688c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301aeee0 con 0x561d2c2df1b0 2024-05-09T05:49:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301b0a80 con 0x561d2c2be4d0 2024-05-09T05:49:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301af5b0 con 0x561d2c2b62f0 2024-05-09T05:49:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b36bfe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2df1b0 0x561d2c688c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b36bfe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2df1b0 0x561d2c688c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53814/0 (socket says 172.21.15.115:53814) 2024-05-09T05:49:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b36bfe640 1 -- 172.21.15.115:0/2965458442 learned_addr learned my addr 172.21.15.115:0/2965458442 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.959+0000 7f3b363fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c682e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/2965458442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2be4d0 0x561d2c6886c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b36bfe640 1 -- 172.21.15.115:0/2965458442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 msgr2=0x561d2c682e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/2965458442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c682e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b36bfe640 1 -- 172.21.15.115:0/2965458442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2be4d0 msgr2=0x561d2c6886c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/2965458442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2be4d0 0x561d2c6886c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b36bfe640 1 -- 172.21.15.115:0/2965458442 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2c3c15f0 con 0x561d2c2df1b0 2024-05-09T05:49:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/2965458442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2df1b0 0x561d2c688c00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009fd0 tx=0x7f3b28009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555428f43b4a524d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b1bfff640 1 -- 172.21.15.115:0/2965458442 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28004cc0 con 0x561d2c2df1b0 2024-05-09T05:49:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b1bfff640 1 -- 172.21.15.115:0/2965458442 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b2802a070 con 0x561d2c2df1b0 2024-05-09T05:49:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.960+0000 7f3b1bfff640 1 -- 172.21.15.115:0/2965458442 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b280052a0 con 0x561d2c2df1b0 2024-05-09T05:49:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.961+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/2965458442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2df1b0 msgr2=0x561d2c688c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.961+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/2965458442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2df1b0 0x561d2c688c00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009fd0 tx=0x7f3b28009a60 comp rx=0 tx=0).stop 2024-05-09T05:49:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.961+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/2965458442 shutdown_connections 2024-05-09T05:49:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.961+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/2965458442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2df1b0 0x561d2c688c00 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.961+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/2965458442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2be4d0 0x561d2c6886c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.961+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/2965458442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c682e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.961+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/2965458442 >> 172.21.15.115:0/2965458442 conn(0x561d2c2d3300 msgr2=0x561d2c6902f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.961+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/2965458442 shutdown_connections 2024-05-09T05:49:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.961+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/2965458442 wait complete. 2024-05-09T05:49:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b3f5f9740 1 -- start start 2024-05-09T05:49:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b3f5f9740 1 Processor -- start 2024-05-09T05:49:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c69b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c69bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b3f5f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c694bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301b03b0 con 0x561d2c2be4d0 2024-05-09T05:49:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301afce0 con 0x561d2c2df1b0 2024-05-09T05:49:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b3f5f9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301acec0 con 0x561d2c2b62f0 2024-05-09T05:49:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b3f5f9740 10 client.0 _ll_register_callbacks cb 0x7f3b2c0c60d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b35bfc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c69bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b35bfc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c69bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53824/0 (socket says 172.21.15.115:53824) 2024-05-09T05:49:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b35bfc640 1 -- 172.21.15.115:0/406543191 learned_addr learned my addr 172.21.15.115:0/406543191 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b363fd640 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c69b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b36bfe640 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c694bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b35bfc640 1 -- 172.21.15.115:0/406543191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 msgr2=0x561d2c69b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c69b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b35bfc640 1 -- 172.21.15.115:0/406543191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 msgr2=0x561d2c694bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c694bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.962+0000 7f3b35bfc640 1 -- 172.21.15.115:0/406543191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2c5abed0 con 0x561d2c2be4d0 2024-05-09T05:49:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b35bfc640 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c69bc30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x561d2c6a1940 tx=0x7f3b20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc82de34b5a7787 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:49:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b19ffb640 1 -- 172.21.15.115:0/406543191 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20020ab0 con 0x561d2c2be4d0 2024-05-09T05:49:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b19ffb640 1 -- 172.21.15.115:0/406543191 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7f3b20020c50 con 0x561d2c2be4d0 2024-05-09T05:49:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/406543191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b2802a070 con 0x561d2c2be4d0 2024-05-09T05:49:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b19ffb640 1 -- 172.21.15.115:0/406543191 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3b2003d050 con 0x561d2c2be4d0 2024-05-09T05:49:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b3f5f9740 10 client.18357 fetch_fsmap learned FSMap version 55 2024-05-09T05:49:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/406543191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561d2c699bc0 con 0x561d2c2be4d0 2024-05-09T05:49:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b19ffb640 1 -- 172.21.15.115:0/406543191 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b2802a070 con 0x561d2c2be4d0 2024-05-09T05:49:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.963+0000 7f3b19ffb640 1 -- 172.21.15.115:0/406543191 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f3b200323f0 con 0x561d2c2be4d0 2024-05-09T05:49:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.964+0000 7f3b3f5f9740 10 client.18357 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:49:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.964+0000 7f3b3f5f9740 10 client.18357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.964+0000 7f3b3f5f9740 20 client.18357 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.964+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561d2c2731c0 0x561d2c70b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.964+0000 7f3b3f5f9740 4 client.18357 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:49:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.964+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/406543191 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d2c5ef240 con 0x561d2c2731c0 2024-05-09T05:49:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.964+0000 7f3b363fd640 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561d2c2731c0 0x561d2c70b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.964+0000 7f3b363fd640 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561d2c2731c0 0x561d2c70b770 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00e0ad7f39e1a28 server_cookie=582d8e8ccd1ebcce in_seq=0 out_seq=0 2024-05-09T05:49:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.964+0000 7f3b19ffb640 10 client.18357 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b19ffb640 1 -- 172.21.15.115:0/406543191 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561d2c5ef240 con 0x561d2c2731c0 2024-05-09T05:49:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b19ffb640 10 client.18357 handle_command_reply: tid=0 2024-05-09T05:49:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 client.18357 shutdown 2024-05-09T05:49:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/406543191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 msgr2=0x561d2c69bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c69bc30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x561d2c6a1940 tx=0x7f3b20004730 comp rx=0 tx=0).stop 2024-05-09T05:49:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/406543191 shutdown_connections 2024-05-09T05:49:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561d2c2df1b0 0x561d2c694bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561d2c2be4d0 0x561d2c69bc30 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561d2c2731c0 0x561d2c70b770 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 --2- 172.21.15.115:0/406543191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561d2c2b62f0 0x561d2c69b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b19ffb640 0 client.18357 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/406543191 >> 172.21.15.115:0/406543191 conn(0x561d2c2d3300 msgr2=0x561d2c682ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/406543191 shutdown_connections 2024-05-09T05:49:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 1 -- 172.21.15.115:0/406543191 wait complete. 2024-05-09T05:49:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.975+0000 7f3b3f5f9740 20 client.18357 trim_cache size 0 max 16384 2024-05-09T05:49:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:49:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:49:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:49:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 -- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b1c0b2950 msgr2=0x7f3b1c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b1c0b2950 0x7f3b1c0b4e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3b200058c0 tx=0x7f3b20005850 comp rx=0 tx=0).stop 2024-05-09T05:49:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 -- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 msgr2=0x7f3b30199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 0x7f3b30199700 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009230 tx=0x7f3b2c004270 comp rx=0 tx=0).stop 2024-05-09T05:49:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 -- 172.21.15.115:0/3763988537 shutdown_connections 2024-05-09T05:49:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3b1c0b2950 0x7f3b1c0b4e60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b301075d0 0x7f3b3019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b30106b10 0x7f3b30199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 --2- 172.21.15.115:0/3763988537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b30102580 0x7f3b30199700 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 -- 172.21.15.115:0/3763988537 >> 172.21.15.115:0/3763988537 conn(0x7f3b300779e0 msgr2=0x7f3b3010cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 -- 172.21.15.115:0/3763988537 shutdown_connections 2024-05-09T05:49:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:43.976+0000 7f3b38661640 1 -- 172.21.15.115:0/3763988537 wait complete. 2024-05-09T05:49:45.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:45 smithi195 ceph-mon[33844]: pgmap v362: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:49:45.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:45 smithi115 ceph-mon[30349]: pgmap v362: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:49:45.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:45 smithi161 ceph-mon[33805]: pgmap v362: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:49:47.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:47 smithi195 ceph-mon[33844]: pgmap v363: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-05-09T05:49:47.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:47 smithi115 ceph-mon[30349]: pgmap v363: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-05-09T05:49:47.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:47 smithi161 ceph-mon[33805]: pgmap v363: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-05-09T05:49:49.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:49 smithi195 ceph-mon[33844]: pgmap v364: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:49.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:49 smithi115 ceph-mon[30349]: pgmap v364: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:49.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:49 smithi161 ceph-mon[33805]: pgmap v364: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:51.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:51 smithi195 ceph-mon[33844]: pgmap v365: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:51.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:51 smithi115 ceph-mon[30349]: pgmap v365: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:51.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:51 smithi161 ceph-mon[33805]: pgmap v365: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:53.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:53 smithi195 ceph-mon[33844]: pgmap v366: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:53.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:53 smithi115 ceph-mon[30349]: pgmap v366: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:53.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:53 smithi161 ceph-mon[33805]: pgmap v366: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:49:55.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:55 smithi195 ceph-mon[33844]: pgmap v367: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:55.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:55 smithi115 ceph-mon[30349]: pgmap v367: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:55.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:55 smithi161 ceph-mon[33805]: pgmap v367: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-09T05:49:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.596+0000 7f867a6b8640 1 -- 172.21.15.115:0/1699943607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8674106b00 msgr2=0x7f8674106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.596+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1699943607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8674106b00 0x7f8674106f30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f866800aa10 tx=0x7f866802b720 comp rx=0 tx=0).stop 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.596+0000 7f867a6b8640 1 -- 172.21.15.115:0/1699943607 shutdown_connections 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.596+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1699943607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f86741075c0 0x7f867410ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.596+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1699943607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8674106b00 0x7f8674106f30 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.596+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1699943607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8674102570 0x7f8674102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.596+0000 7f867a6b8640 1 -- 172.21.15.115:0/1699943607 >> 172.21.15.115:0/1699943607 conn(0x7f86740fd9d0 msgr2=0x7f86740ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.596+0000 7f867a6b8640 1 -- 172.21.15.115:0/1699943607 shutdown_connections 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.596+0000 7f867a6b8640 1 -- 172.21.15.115:0/1699943607 wait complete. 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.597+0000 7f867a6b8640 1 Processor -- start 2024-05-09T05:49:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.597+0000 7f867a6b8640 1 -- start start 2024-05-09T05:49:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.597+0000 7f867a6b8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8674102570 0x7f867419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.597+0000 7f867a6b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8674106b00 0x7f867419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.597+0000 7f867a6b8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86741075c0 0x7f867419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f867a6b8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8674111b00 con 0x7f86741075c0 2024-05-09T05:49:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f867a6b8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8674111980 con 0x7f8674102570 2024-05-09T05:49:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f867a6b8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8674111c80 con 0x7f8674106b00 2024-05-09T05:49:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8678c55640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86741075c0 0x7f867419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8678c55640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86741075c0 0x7f867419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55854/0 (socket says 172.21.15.115:55854) 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8678c55640 1 -- 172.21.15.115:0/1269988950 learned_addr learned my addr 172.21.15.115:0/1269988950 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8673fff640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8674102570 0x7f867419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f86737fe640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8674106b00 0x7f867419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8678c55640 1 -- 172.21.15.115:0/1269988950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8674106b00 msgr2=0x7f867419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8678c55640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8674106b00 0x7f867419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8678c55640 1 -- 172.21.15.115:0/1269988950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8674102570 msgr2=0x7f867419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8678c55640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8674102570 0x7f867419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8678c55640 1 -- 172.21.15.115:0/1269988950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867419f640 con 0x7f86741075c0 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f8678c55640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86741075c0 0x7f867419f000 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f866c009fd0 tx=0x7f866c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f86717fa640 1 -- 172.21.15.115:0/1269988950 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c017070 con 0x7f86741075c0 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f86717fa640 1 -- 172.21.15.115:0/1269988950 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f866c003e00 con 0x7f86741075c0 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.598+0000 7f867a6b8640 1 -- 172.21.15.115:0/1269988950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867419f930 con 0x7f86741075c0 2024-05-09T05:49:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.599+0000 7f86717fa640 1 -- 172.21.15.115:0/1269988950 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c010460 con 0x7f86741075c0 2024-05-09T05:49:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.599+0000 7f867a6b8640 1 -- 172.21.15.115:0/1269988950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867419fe00 con 0x7f86741075c0 2024-05-09T05:49:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.601+0000 7f86717fa640 1 -- 172.21.15.115:0/1269988950 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f866c003970 con 0x7f86741075c0 2024-05-09T05:49:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f868156a740 1 Processor -- start 2024-05-09T05:49:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f868156a740 1 -- start start 2024-05-09T05:49:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f868156a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f868156a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f868156a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f868156a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558ccb8157c0 con 0x558ccb4e42f0 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f868156a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558ccb7b10e0 con 0x558ccb50d1b0 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f868156a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558ccb811780 con 0x558ccb4ec4d0 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f8673fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f8673fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55866/0 (socket says 172.21.15.115:55866) 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f8673fff640 1 -- 172.21.15.115:0/888503731 learned_addr learned my addr 172.21.15.115:0/888503731 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f86717fa640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f865c0b2950 0x7f865c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f86737fe640 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f86737fe640 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f8673fff640 1 -- 172.21.15.115:0/888503731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 msgr2=0x558ccb8b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f8673fff640 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f8673fff640 1 -- 172.21.15.115:0/888503731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 msgr2=0x558ccb8b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f8673fff640 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f8673fff640 1 -- 172.21.15.115:0/888503731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccb7da7c0 con 0x558ccb4e42f0 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f8678c55640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f865c0b2950 0x7f865c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.602+0000 7f86737fe640 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.603+0000 7f86717fa640 1 -- 172.21.15.115:0/1269988950 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7f866c0e29c0 con 0x7f86741075c0 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.603+0000 7f8673fff640 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8b0bc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f866400bd10 tx=0x7f86640107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6f9fe0ab18312a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.603+0000 7f8678c55640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f865c0b2950 0x7f865c0b4e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f866c0a99a0 tx=0x7f866c0775e0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.603+0000 7f865a7fc640 1 -- 172.21.15.115:0/888503731 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664004030 con 0x558ccb4e42f0 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.603+0000 7f865a7fc640 1 -- 172.21.15.115:0/888503731 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86640035f0 con 0x558ccb4e42f0 2024-05-09T05:49:57.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.603+0000 7f865a7fc640 1 -- 172.21.15.115:0/888503731 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664011640 con 0x558ccb4e42f0 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 -- 172.21.15.115:0/888503731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 msgr2=0x558ccb8b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8b0bc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f866400bd10 tx=0x7f86640107a0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 -- 172.21.15.115:0/888503731 shutdown_connections 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8b0bc0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 --2- 172.21.15.115:0/888503731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 -- 172.21.15.115:0/888503731 >> 172.21.15.115:0/888503731 conn(0x558ccb501300 msgr2=0x558ccb8ace30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 -- 172.21.15.115:0/888503731 shutdown_connections 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 -- 172.21.15.115:0/888503731 wait complete. 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.604+0000 7f868156a740 1 -- start start 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f868156a740 1 Processor -- start 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f868156a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f868156a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8bcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f868156a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f868156a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558ccb8bd7b0 con 0x558ccb4e42f0 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f868156a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558ccb79c9c0 con 0x558ccb50d1b0 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f868156a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558ccb8bd930 con 0x558ccb4ec4d0 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f868156a740 10 client.0 _ll_register_callbacks cb 0x558ccb7c9020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f86737fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f86737fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55880/0 (socket says 172.21.15.115:55880) 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f86737fe640 1 -- 172.21.15.115:0/1141256469 learned_addr learned my addr 172.21.15.115:0/1141256469 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f8673fff640 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8bcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f86737fe640 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f86737fe640 1 -- 172.21.15.115:0/1141256469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 msgr2=0x558ccb8bcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f86737fe640 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8bcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f86737fe640 1 -- 172.21.15.115:0/1141256469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 msgr2=0x558ccb8baba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.605+0000 7f86737fe640 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8baba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f86737fe640 1 -- 172.21.15.115:0/1141256469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccb7c31f0 con 0x558ccb4e42f0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f86737fe640 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bc6e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f86680057c0 tx=0x7f866802b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53882698d151a32 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f86537fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f86537fe640 1 -- 172.21.15.115:0/1141256469 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668036040 con 0x558ccb4e42f0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f86537fe640 1 -- 172.21.15.115:0/1141256469 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7f86680037f0 con 0x558ccb4e42f0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f868156a740 1 -- 172.21.15.115:0/1141256469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86640035f0 con 0x558ccb4e42f0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f86537fe640 1 -- 172.21.15.115:0/1141256469 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8668051050 con 0x558ccb4e42f0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f868156a740 10 client.18369 fetch_fsmap learned FSMap version 55 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f868156a740 1 -- 172.21.15.115:0/1141256469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558ccb578790 con 0x558ccb4e42f0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f86537fe640 1 -- 172.21.15.115:0/1141256469 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86640035f0 con 0x558ccb4e42f0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.606+0000 7f86537fe640 1 -- 172.21.15.115:0/1141256469 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f866804a060 con 0x558ccb4e42f0 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.607+0000 7f868156a740 10 client.18369 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.607+0000 7f868156a740 10 client.18369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.607+0000 7f868156a740 20 client.18369 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.607+0000 7f868156a740 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ccb4cbd80 0x558ccb8ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.607+0000 7f868156a740 4 client.18369 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:49:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.607+0000 7f868156a740 1 -- 172.21.15.115:0/1141256469 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ccb8233d0 con 0x558ccb4cbd80 2024-05-09T05:49:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.607+0000 7f8673fff640 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ccb4cbd80 0x558ccb8ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.607+0000 7f8673fff640 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ccb4cbd80 0x558ccb8ccf50 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=827c4029ccc219e2 server_cookie=88bbb33c92fc1cc9 in_seq=0 out_seq=0 2024-05-09T05:49:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.607+0000 7f86537fe640 10 client.18369 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f86537fe640 1 -- 172.21.15.115:0/1141256469 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ccb8233d0 con 0x558ccb4cbd80 2024-05-09T05:49:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f86537fe640 10 client.18369 handle_command_reply: tid=0 2024-05-09T05:49:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f868156a740 1 client.18369 shutdown 2024-05-09T05:49:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f868156a740 1 -- 172.21.15.115:0/1141256469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 msgr2=0x558ccb8bc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f868156a740 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bc6e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f86680057c0 tx=0x7f866802b720 comp rx=0 tx=0).stop 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f868156a740 1 -- 172.21.15.115:0/1141256469 shutdown_connections 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f868156a740 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f868156a740 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ccb4cbd80 0x558ccb8ccf50 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f868156a740 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8bcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.608+0000 7f868156a740 1 --2- 172.21.15.115:0/1141256469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bc6e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.609+0000 7f868156a740 1 -- 172.21.15.115:0/1141256469 >> 172.21.15.115:0/1141256469 conn(0x558ccb501300 msgr2=0x558ccb8ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.609+0000 7f86537fe640 0 client.18369 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.609+0000 7f868156a740 1 -- 172.21.15.115:0/1141256469 shutdown_connections 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.609+0000 7f868156a740 1 -- 172.21.15.115:0/1141256469 wait complete. 2024-05-09T05:49:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.609+0000 7f868156a740 20 client.18369 trim_cache size 0 max 16384 2024-05-09T05:49:57.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:57 smithi195 ceph-mon[33844]: pgmap v368: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:49:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.626+0000 7f868156a740 1 Processor -- start 2024-05-09T05:49:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.626+0000 7f868156a740 1 -- start start 2024-05-09T05:49:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f868156a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f868156a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f868156a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f868156a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b1030 con 0x558ccb4e42f0 2024-05-09T05:49:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f868156a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b2bd0 con 0x558ccb50d1b0 2024-05-09T05:49:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f868156a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b1700 con 0x558ccb4ec4d0 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f86737fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f86737fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55884/0 (socket says 172.21.15.115:55884) 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f86737fe640 1 -- 172.21.15.115:0/2979497079 learned_addr learned my addr 172.21.15.115:0/2979497079 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f86737fe640 1 -- 172.21.15.115:0/2979497079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 msgr2=0x558ccb8d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f86737fe640 1 --2- 172.21.15.115:0/2979497079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f86737fe640 1 -- 172.21.15.115:0/2979497079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 msgr2=0x558ccb8d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f86737fe640 1 --2- 172.21.15.115:0/2979497079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f86737fe640 1 -- 172.21.15.115:0/2979497079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccb5ef5f0 con 0x558ccb4e42f0 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.627+0000 7f86737fe640 1 --2- 172.21.15.115:0/2979497079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bb270 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f866802bc40 tx=0x7f86680058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67ec6edc074653f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.628+0000 7f8659ffb640 1 -- 172.21.15.115:0/2979497079 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866805a070 con 0x558ccb4e42f0 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.628+0000 7f8659ffb640 1 -- 172.21.15.115:0/2979497079 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86680044b0 con 0x558ccb4e42f0 2024-05-09T05:49:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.628+0000 7f8659ffb640 1 -- 172.21.15.115:0/2979497079 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668036040 con 0x558ccb4e42f0 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 -- 172.21.15.115:0/2979497079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 msgr2=0x558ccb8bb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 --2- 172.21.15.115:0/2979497079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bb270 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f866802bc40 tx=0x7f86680058d0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 -- 172.21.15.115:0/2979497079 shutdown_connections 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 --2- 172.21.15.115:0/2979497079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb50d1b0 0x558ccb8d2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 --2- 172.21.15.115:0/2979497079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4ec4d0 0x558ccb8d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 --2- 172.21.15.115:0/2979497079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb4e42f0 0x558ccb8bb270 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 -- 172.21.15.115:0/2979497079 >> 172.21.15.115:0/2979497079 conn(0x558ccb501300 msgr2=0x558ccb8bab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 -- 172.21.15.115:0/2979497079 shutdown_connections 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 -- 172.21.15.115:0/2979497079 wait complete. 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 -- start start 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.629+0000 7f868156a740 1 Processor -- start 2024-05-09T05:49:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f868156a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4e42f0 0x558ccb8c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f868156a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb4ec4d0 0x558ccb8c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f868156a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb50d1b0 0x558ccb8bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f868156a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b2500 con 0x558ccb50d1b0 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f868156a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b1e30 con 0x558ccb4ec4d0 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f868156a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f86741af010 con 0x558ccb4e42f0 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f868156a740 10 client.0 _ll_register_callbacks cb 0x7f866800a750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb50d1b0 0x558ccb8bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb50d1b0 0x558ccb8bfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55892/0 (socket says 172.21.15.115:55892) 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 -- 172.21.15.115:0/1671774923 learned_addr learned my addr 172.21.15.115:0/1671774923 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4e42f0 0x558ccb8c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f8673fff640 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb4ec4d0 0x558ccb8c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 -- 172.21.15.115:0/1671774923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4e42f0 msgr2=0x558ccb8c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4e42f0 0x558ccb8c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 -- 172.21.15.115:0/1671774923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb4ec4d0 msgr2=0x558ccb8c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:49:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb4ec4d0 0x558ccb8c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 -- 172.21.15.115:0/1671774923 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccb7d9ed0 con 0x558ccb50d1b0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f8673fff640 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb4ec4d0 0x558ccb8c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86737fe640 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb50d1b0 0x558ccb8bfed0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8668030950 tx=0x7f8668042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fba6296ec56239f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86537fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86537fe640 1 -- 172.21.15.115:0/1671774923 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866805a030 con 0x558ccb50d1b0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f86537fe640 1 -- 172.21.15.115:0/1671774923 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9350+0+0 (secure 0 0 0) 0x7f8668055840 con 0x558ccb50d1b0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.630+0000 7f868156a740 1 -- 172.21.15.115:0/1671774923 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86680044b0 con 0x558ccb50d1b0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f86537fe640 1 -- 172.21.15.115:0/1671774923 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f866806c050 con 0x558ccb50d1b0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f868156a740 10 client.18375 fetch_fsmap learned FSMap version 55 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f868156a740 1 -- 172.21.15.115:0/1671774923 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558ccb934ad0 con 0x558ccb50d1b0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f86537fe640 1 -- 172.21.15.115:0/1671774923 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86680044b0 con 0x558ccb50d1b0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f86537fe640 1 -- 172.21.15.115:0/1671774923 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f866804a070 con 0x558ccb50d1b0 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f868156a740 10 client.18375 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f868156a740 10 client.18375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:49:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f868156a740 20 client.18375 populate_metadata read hostname 'smithi115' 2024-05-09T05:49:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f868156a740 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ccb4a11c0 0x558ccb9370d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:49:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f868156a740 4 client.18375 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:49:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f868156a740 1 -- 172.21.15.115:0/1671774923 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ccb81d240 con 0x558ccb4a11c0 2024-05-09T05:49:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.631+0000 7f8673fff640 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ccb4a11c0 0x558ccb9370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:49:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.632+0000 7f86537fe640 10 client.18375 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.632+0000 7f8673fff640 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ccb4a11c0 0x558ccb9370d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3934101dad4cc0a6 server_cookie=cd9aeca8b5e60598 in_seq=0 out_seq=0 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f86537fe640 1 -- 172.21.15.115:0/1671774923 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558ccb81d240 con 0x558ccb4a11c0 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f86537fe640 10 client.18375 handle_command_reply: tid=0 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 client.18375 shutdown 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 -- 172.21.15.115:0/1671774923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb50d1b0 msgr2=0x558ccb8bfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb50d1b0 0x558ccb8bfed0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8668030950 tx=0x7f8668042040 comp rx=0 tx=0).stop 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 -- 172.21.15.115:0/1671774923 shutdown_connections 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558ccb50d1b0 0x558ccb8bfed0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb4ec4d0 0x558ccb8c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x558ccb4a11c0 0x558ccb9370d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 --2- 172.21.15.115:0/1671774923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558ccb4e42f0 0x558ccb8c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f86537fe640 0 client.18375 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:49:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 -- 172.21.15.115:0/1671774923 >> 172.21.15.115:0/1671774923 conn(0x558ccb501300 msgr2=0x558ccb8ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 -- 172.21.15.115:0/1671774923 shutdown_connections 2024-05-09T05:49:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 1 -- 172.21.15.115:0/1671774923 wait complete. 2024-05-09T05:49:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.640+0000 7f868156a740 20 client.18375 trim_cache size 0 max 16384 2024-05-09T05:49:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:49:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:49:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:49:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 -- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f865c0b2950 msgr2=0x7f865c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f865c0b2950 0x7f865c0b4e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f866c0a99a0 tx=0x7f866c0775e0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 -- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86741075c0 msgr2=0x7f867419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:49:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86741075c0 0x7f867419f000 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f866c009fd0 tx=0x7f866c00b260 comp rx=0 tx=0).stop 2024-05-09T05:49:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 -- 172.21.15.115:0/1269988950 shutdown_connections 2024-05-09T05:49:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f865c0b2950 0x7f865c0b4e60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86741075c0 0x7f867419f000 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8674106b00 0x7f867419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 --2- 172.21.15.115:0/1269988950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8674102570 0x7f867419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:49:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 -- 172.21.15.115:0/1269988950 >> 172.21.15.115:0/1269988950 conn(0x7f86740fd9d0 msgr2=0x7f8674106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:49:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 -- 172.21.15.115:0/1269988950 shutdown_connections 2024-05-09T05:49:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:49:57.641+0000 7f867a6b8640 1 -- 172.21.15.115:0/1269988950 wait complete. 2024-05-09T05:49:57.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:57 smithi115 ceph-mon[30349]: pgmap v368: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:49:57.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:57 smithi161 ceph-mon[33805]: pgmap v368: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:49:59.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:49:59 smithi195 ceph-mon[33844]: pgmap v369: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:49:59.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:49:59 smithi115 ceph-mon[30349]: pgmap v369: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:49:59.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:49:59 smithi161 ceph-mon[33805]: pgmap v369: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 31 op/s 2024-05-09T05:50:00.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a[30345]: 2024-05-09T05:49:59.999+0000 7f5c15a39700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-09T05:50:00.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a[30345]: 2024-05-09T05:49:59.999+0000 7f5c15a39700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-09T05:50:00.250 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a[30345]: 2024-05-09T05:49:59.999+0000 7f5c15a39700 -1 log_channel(cluster) log [ERR] : mds.f(mds.2): Metadata damage detected 2024-05-09T05:50:00.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:00 smithi195 ceph-mon[33844]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-09T05:50:00.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:00 smithi195 ceph-mon[33844]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-09T05:50:00.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:00 smithi195 ceph-mon[33844]: mds.f(mds.2): Metadata damage detected 2024-05-09T05:50:00.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:00 smithi115 ceph-mon[30349]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-09T05:50:00.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:00 smithi115 ceph-mon[30349]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-09T05:50:00.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:00 smithi115 ceph-mon[30349]: mds.f(mds.2): Metadata damage detected 2024-05-09T05:50:00.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:00 smithi161 ceph-mon[33805]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-05-09T05:50:00.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:00 smithi161 ceph-mon[33805]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-05-09T05:50:00.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:00 smithi161 ceph-mon[33805]: mds.f(mds.2): Metadata damage detected 2024-05-09T05:50:01.544 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:01 smithi115 ceph-mon[30349]: pgmap v370: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:50:01.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:01 smithi195 ceph-mon[33844]: pgmap v370: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:50:01.658 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:01 smithi161 ceph-mon[33805]: pgmap v370: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:50:03.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:03 smithi195 ceph-mon[33844]: pgmap v371: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:50:03.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:03 smithi115 ceph-mon[30349]: pgmap v371: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:50:03.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:03 smithi161 ceph-mon[33805]: pgmap v371: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-05-09T05:50:05.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:05 smithi195 ceph-mon[33844]: pgmap v372: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 46 op/s 2024-05-09T05:50:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:05 smithi115 ceph-mon[30349]: pgmap v372: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 46 op/s 2024-05-09T05:50:05.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:05 smithi161 ceph-mon[33805]: pgmap v372: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 46 op/s 2024-05-09T05:50:07.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:07 smithi195 ceph-mon[33844]: pgmap v373: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 23 MiB/s wr, 37 op/s 2024-05-09T05:50:07.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:07 smithi115 ceph-mon[30349]: pgmap v373: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 23 MiB/s wr, 37 op/s 2024-05-09T05:50:07.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:07 smithi161 ceph-mon[33805]: pgmap v373: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 23 MiB/s wr, 37 op/s 2024-05-09T05:50:09.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:09 smithi195 ceph-mon[33844]: pgmap v374: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 35 MiB/s wr, 47 op/s 2024-05-09T05:50:09.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:09 smithi115 ceph-mon[30349]: pgmap v374: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 35 MiB/s wr, 47 op/s 2024-05-09T05:50:09.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:09 smithi161 ceph-mon[33805]: pgmap v374: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 35 MiB/s wr, 47 op/s 2024-05-09T05:50:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.777+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/579010547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc108ba0 msgr2=0x7f1ecc102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.777+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/579010547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc108ba0 0x7f1ecc102740 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009900 tx=0x7f1ec802a410 comp rx=0 tx=0).stop 2024-05-09T05:50:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.777+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/579010547 shutdown_connections 2024-05-09T05:50:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.777+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/579010547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc1032f0 0x7f1ecc10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.777+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/579010547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ecc109040 0x7f1ecc102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.777+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/579010547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc108ba0 0x7f1ecc102740 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.777+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/579010547 >> 172.21.15.115:0/579010547 conn(0x7f1ecc0782e0 msgr2=0x7f1ecc078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.777+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/579010547 shutdown_connections 2024-05-09T05:50:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.777+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/579010547 wait complete. 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed3c9f640 1 Processor -- start 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed3c9f640 1 -- start start 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed3c9f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ecc1032f0 0x7f1ecc1a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed3c9f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc108ba0 0x7f1ecc1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed3c9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc109040 0x7f1ecc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed3c9f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc112c40 con 0x7f1ecc109040 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed3c9f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc112ac0 con 0x7f1ecc108ba0 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed3c9f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc112dc0 con 0x7f1ecc1032f0 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed223c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc109040 0x7f1ecc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed223c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc109040 0x7f1ecc19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52368/0 (socket says 172.21.15.115:52368) 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed223c640 1 -- 172.21.15.115:0/353587710 learned_addr learned my addr 172.21.15.115:0/353587710 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed123a640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc108ba0 0x7f1ecc1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.778+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ecc1032f0 0x7f1ecc1a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ed223c640 1 -- 172.21.15.115:0/353587710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ecc1032f0 msgr2=0x7f1ecc1a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ed223c640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ecc1032f0 0x7f1ecc1a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ed223c640 1 -- 172.21.15.115:0/353587710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc108ba0 msgr2=0x7f1ecc1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ed223c640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc108ba0 0x7f1ecc1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ed223c640 1 -- 172.21.15.115:0/353587710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc19adc0 con 0x7f1ecc109040 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ecc1032f0 0x7f1ecc1a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ed223c640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc109040 0x7f1ecc19a690 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400eaa0 tx=0x7f1ec400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ec2ffd640 1 -- 172.21.15.115:0/353587710 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec400cdb0 con 0x7f1ecc109040 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/353587710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc19b050 con 0x7f1ecc109040 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ec2ffd640 1 -- 172.21.15.115:0/353587710 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ec4005020 con 0x7f1ecc109040 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ec2ffd640 1 -- 172.21.15.115:0/353587710 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40116d0 con 0x7f1ecc109040 2024-05-09T05:50:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.779+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/353587710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1a72b0 con 0x7f1ecc109040 2024-05-09T05:50:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.782+0000 7f1ec2ffd640 1 -- 172.21.15.115:0/353587710 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1ec40051d0 con 0x7f1ecc109040 2024-05-09T05:50:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.783+0000 7f1ec2ffd640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1eb80b2950 0x7f1eb80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.783+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1eb80b2950 0x7f1eb80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.783+0000 7f1ec2ffd640 1 -- 172.21.15.115:0/353587710 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f1ec40e2c00 con 0x7f1ecc109040 2024-05-09T05:50:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.783+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1eb80b2950 0x7f1eb80b4e60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8006010 tx=0x7f1ec8005de0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1edac2f740 1 Processor -- start 2024-05-09T05:50:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1edac2f740 1 -- start start 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 0x561a418b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 0x561a418b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1edac2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561a418177c0 con 0x561a414e62f0 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1edac2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561a417b30e0 con 0x561a4150f1b0 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1edac2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561a41813780 con 0x561a414ee4d0 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1ed123a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1ed123a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52374/0 (socket says 172.21.15.115:52374) 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1ed123a640 1 -- 172.21.15.115:0/3172943693 learned_addr learned my addr 172.21.15.115:0/3172943693 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 0x561a418b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.792+0000 7f1ed223c640 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 0x561a418b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1ed123a640 1 -- 172.21.15.115:0/3172943693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 msgr2=0x561a418b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1ed123a640 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 0x561a418b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1ed123a640 1 -- 172.21.15.115:0/3172943693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 msgr2=0x561a418b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1ed123a640 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 0x561a418b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1ed123a640 1 -- 172.21.15.115:0/3172943693 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a417dc7c0 con 0x561a414e62f0 2024-05-09T05:50:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1ed223c640 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 0x561a418b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1ed123a640 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418b2bc0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009c00 tx=0x7f1ebc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cf0b3af89b85b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1eb7fff640 1 -- 172.21.15.115:0/3172943693 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc01e070 con 0x561a414e62f0 2024-05-09T05:50:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1eb7fff640 1 -- 172.21.15.115:0/3172943693 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ebc0044c0 con 0x561a414e62f0 2024-05-09T05:50:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.793+0000 7f1eb7fff640 1 -- 172.21.15.115:0/3172943693 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003e50 con 0x561a414e62f0 2024-05-09T05:50:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 -- 172.21.15.115:0/3172943693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 msgr2=0x561a418b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418b2bc0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009c00 tx=0x7f1ebc011810 comp rx=0 tx=0).stop 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 -- 172.21.15.115:0/3172943693 shutdown_connections 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 0x561a418b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418b2bc0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 --2- 172.21.15.115:0/3172943693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 0x561a418b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 -- 172.21.15.115:0/3172943693 >> 172.21.15.115:0/3172943693 conn(0x561a41503300 msgr2=0x561a418aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 -- 172.21.15.115:0/3172943693 shutdown_connections 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 -- 172.21.15.115:0/3172943693 wait complete. 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.794+0000 7f1edac2f740 1 -- start start 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1edac2f740 1 Processor -- start 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414ee4d0 0x561a418becb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a4150f1b0 0x561a418bcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1edac2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x561a418bf7b0 con 0x561a414e62f0 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1edac2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561a4179e9c0 con 0x561a414ee4d0 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1edac2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561a418bf930 con 0x561a4150f1b0 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1edac2f740 10 client.0 _ll_register_callbacks cb 0x561a417cb020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1ed1a3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1ed1a3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418be770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52380/0 (socket says 172.21.15.115:52380) 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1ed1a3b640 1 -- 172.21.15.115:0/4031877112 learned_addr learned my addr 172.21.15.115:0/4031877112 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1ed223c640 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a4150f1b0 0x561a418bcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.795+0000 7f1ed123a640 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414ee4d0 0x561a418becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1ed1a3b640 1 -- 172.21.15.115:0/4031877112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a4150f1b0 msgr2=0x561a418bcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a4150f1b0 0x561a418bcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1ed1a3b640 1 -- 172.21.15.115:0/4031877112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414ee4d0 msgr2=0x561a418becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414ee4d0 0x561a418becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1ed1a3b640 1 -- 172.21.15.115:0/4031877112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a417c51f0 con 0x561a414e62f0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418be770 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc103880 tx=0x7f1ec8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e78df3f095af7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1eb57fa640 1 -- 172.21.15.115:0/4031877112 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8035050 con 0x561a414e62f0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1eb57fa640 1 -- 172.21.15.115:0/4031877112 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f1ec8005250 con 0x561a414e62f0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1edac2f740 1 -- 172.21.15.115:0/4031877112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a418253d0 con 0x561a414e62f0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1eb57fa640 1 -- 172.21.15.115:0/4031877112 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1ec8059050 con 0x561a414e62f0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1edac2f740 10 client.18387 fetch_fsmap learned FSMap version 55 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.796+0000 7f1edac2f740 1 -- 172.21.15.115:0/4031877112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561a4157a790 con 0x561a414e62f0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.797+0000 7f1eb57fa640 1 -- 172.21.15.115:0/4031877112 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561a418253d0 con 0x561a414e62f0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.797+0000 7f1eb57fa640 1 -- 172.21.15.115:0/4031877112 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f1ec80035f0 con 0x561a414e62f0 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.797+0000 7f1edac2f740 10 client.18387 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:50:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.797+0000 7f1edac2f740 10 client.18387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.797+0000 7f1edac2f740 20 client.18387 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.797+0000 7f1edac2f740 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a414cdd80 0x561a418cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.797+0000 7f1edac2f740 4 client.18387 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.797+0000 7f1edac2f740 1 -- 172.21.15.115:0/4031877112 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a4181f240 con 0x561a414cdd80 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.797+0000 7f1ed123a640 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a414cdd80 0x561a418cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.798+0000 7f1eb57fa640 10 client.18387 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.798+0000 7f1ed123a640 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a414cdd80 0x561a418cef90 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59418807558e8ad2 server_cookie=389be06920e5985d in_seq=0 out_seq=0 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.798+0000 7f1eb57fa640 1 -- 172.21.15.115:0/4031877112 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a4181f240 con 0x561a414cdd80 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.798+0000 7f1eb57fa640 10 client.18387 handle_command_reply: tid=0 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.798+0000 7f1edac2f740 1 client.18387 shutdown 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 -- 172.21.15.115:0/4031877112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 msgr2=0x561a418be770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418be770 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc103880 tx=0x7f1ec8009260 comp rx=0 tx=0).stop 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 -- 172.21.15.115:0/4031877112 shutdown_connections 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a414cdd80 0x561a418cef90 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a4150f1b0 0x561a418bcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414ee4d0 0x561a418becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 --2- 172.21.15.115:0/4031877112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414e62f0 0x561a418be770 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1eb57fa640 0 client.18387 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 -- 172.21.15.115:0/4031877112 >> 172.21.15.115:0/4031877112 conn(0x561a41503300 msgr2=0x561a418ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 -- 172.21.15.115:0/4031877112 shutdown_connections 2024-05-09T05:50:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 1 -- 172.21.15.115:0/4031877112 wait complete. 2024-05-09T05:50:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.799+0000 7f1edac2f740 20 client.18387 trim_cache size 0 max 16384 2024-05-09T05:50:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.816+0000 7f1edac2f740 1 Processor -- start 2024-05-09T05:50:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.816+0000 7f1edac2f740 1 -- start start 2024-05-09T05:50:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414e62f0 0x561a418aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 0x561a418d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4150f1b0 0x561a418d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1edac2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b12e0 con 0x561a4150f1b0 2024-05-09T05:50:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1edac2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b2e80 con 0x561a414e62f0 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1edac2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b19b0 con 0x561a414ee4d0 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed223c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4150f1b0 0x561a418d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed223c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4150f1b0 0x561a418d4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52382/0 (socket says 172.21.15.115:52382) 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed223c640 1 -- 172.21.15.115:0/2617132944 learned_addr learned my addr 172.21.15.115:0/2617132944 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed123a640 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 0x561a418d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414e62f0 0x561a418aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed223c640 1 -- 172.21.15.115:0/2617132944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 msgr2=0x561a418d4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed223c640 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 0x561a418d4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed223c640 1 -- 172.21.15.115:0/2617132944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414e62f0 msgr2=0x561a418aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed223c640 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414e62f0 0x561a418aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed223c640 1 -- 172.21.15.115:0/2617132944 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a415f15f0 con 0x561a4150f1b0 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414e62f0 0x561a418aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1ed223c640 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4150f1b0 0x561a418d4cd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40e0010 tx=0x7f1ec40b23b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c61decc494e71f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1eb77fe640 1 -- 172.21.15.115:0/2617132944 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec401f040 con 0x561a4150f1b0 2024-05-09T05:50:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.817+0000 7f1eb77fe640 1 -- 172.21.15.115:0/2617132944 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ec40a9af0 con 0x561a4150f1b0 2024-05-09T05:50:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.818+0000 7f1eb77fe640 1 -- 172.21.15.115:0/2617132944 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40a5460 con 0x561a4150f1b0 2024-05-09T05:50:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.818+0000 7f1edac2f740 1 -- 172.21.15.115:0/2617132944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4150f1b0 msgr2=0x561a418d4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.818+0000 7f1edac2f740 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4150f1b0 0x561a418d4cd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40e0010 tx=0x7f1ec40b23b0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.818+0000 7f1edac2f740 1 -- 172.21.15.115:0/2617132944 shutdown_connections 2024-05-09T05:50:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.819+0000 7f1edac2f740 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a4150f1b0 0x561a418d4cd0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.819+0000 7f1edac2f740 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414ee4d0 0x561a418d4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.819+0000 7f1edac2f740 1 --2- 172.21.15.115:0/2617132944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a414e62f0 0x561a418aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.819+0000 7f1edac2f740 1 -- 172.21.15.115:0/2617132944 >> 172.21.15.115:0/2617132944 conn(0x561a41503300 msgr2=0x561a418ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.819+0000 7f1edac2f740 1 -- 172.21.15.115:0/2617132944 shutdown_connections 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.819+0000 7f1edac2f740 1 -- 172.21.15.115:0/2617132944 wait complete. 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.819+0000 7f1edac2f740 1 -- start start 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.819+0000 7f1edac2f740 1 Processor -- start 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414e62f0 0x561a418c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414ee4d0 0x561a418c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1edac2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 0x561a418c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1edac2f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b27b0 con 0x561a414ee4d0 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1edac2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b20e0 con 0x561a4150f1b0 2024-05-09T05:50:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1edac2f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1af2e0 con 0x561a414e62f0 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1edac2f740 10 client.0 _ll_register_callbacks cb 0x7f1ebc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed123a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414ee4d0 0x561a418c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed123a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414ee4d0 0x561a418c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52386/0 (socket says 172.21.15.115:52386) 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed123a640 1 -- 172.21.15.115:0/143231380 learned_addr learned my addr 172.21.15.115:0/143231380 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414e62f0 0x561a418c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed223c640 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 0x561a418c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed123a640 1 -- 172.21.15.115:0/143231380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414e62f0 msgr2=0x561a418c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed123a640 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414e62f0 0x561a418c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed123a640 1 -- 172.21.15.115:0/143231380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 msgr2=0x561a418c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed123a640 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 0x561a418c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed123a640 1 -- 172.21.15.115:0/143231380 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a417dbed0 con 0x561a414ee4d0 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414e62f0 0x561a418c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1ed123a640 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414ee4d0 0x561a418c2570 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009250 tx=0x7f1ebc011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1005bb65908d6ded server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.820+0000 7f1eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:10.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1eb57fa640 1 -- 172.21.15.115:0/143231380 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc01e040 con 0x561a414ee4d0 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1eb57fa640 1 -- 172.21.15.115:0/143231380 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f1ebc005040 con 0x561a414ee4d0 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1edac2f740 1 -- 172.21.15.115:0/143231380 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ec40a9af0 con 0x561a414ee4d0 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1eb57fa640 1 -- 172.21.15.115:0/143231380 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc005240 con 0x561a414ee4d0 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1edac2f740 10 client.18393 fetch_fsmap learned FSMap version 55 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1edac2f740 1 -- 172.21.15.115:0/143231380 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561a41937130 con 0x561a414ee4d0 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1eb57fa640 1 -- 172.21.15.115:0/143231380 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ec40a9af0 con 0x561a414ee4d0 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1eb57fa640 1 -- 172.21.15.115:0/143231380 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f1ebc02b190 con 0x561a414ee4d0 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1edac2f740 10 client.18393 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1edac2f740 10 client.18393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1edac2f740 20 client.18393 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1edac2f740 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a414a31c0 0x561a41939750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1edac2f740 4 client.18393 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.821+0000 7f1edac2f740 1 -- 172.21.15.115:0/143231380 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a4179a350 con 0x561a414a31c0 2024-05-09T05:50:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.822+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a414a31c0 0x561a41939750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.822+0000 7f1eb57fa640 10 client.18393 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.822+0000 7f1ed1a3b640 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a414a31c0 0x561a41939750 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c57a8f60cccf2e72 server_cookie=bdd06bdd8574566b in_seq=0 out_seq=0 2024-05-09T05:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1eb57fa640 1 -- 172.21.15.115:0/143231380 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561a4179a350 con 0x561a414a31c0 2024-05-09T05:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1eb57fa640 10 client.18393 handle_command_reply: tid=0 2024-05-09T05:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1edac2f740 1 client.18393 shutdown 2024-05-09T05:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1edac2f740 1 -- 172.21.15.115:0/143231380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414ee4d0 msgr2=0x561a418c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1edac2f740 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414ee4d0 0x561a418c2570 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009250 tx=0x7f1ebc011e50 comp rx=0 tx=0).stop 2024-05-09T05:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1edac2f740 1 -- 172.21.15.115:0/143231380 shutdown_connections 2024-05-09T05:50:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1edac2f740 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a4150f1b0 0x561a418c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1edac2f740 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561a414ee4d0 0x561a418c2570 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1edac2f740 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561a414a31c0 0x561a41939750 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.830+0000 7f1edac2f740 1 --2- 172.21.15.115:0/143231380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a414e62f0 0x561a418c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1eb57fa640 0 client.18393 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1edac2f740 1 -- 172.21.15.115:0/143231380 >> 172.21.15.115:0/143231380 conn(0x561a41503300 msgr2=0x561a418be2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1edac2f740 1 -- 172.21.15.115:0/143231380 shutdown_connections 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1edac2f740 1 -- 172.21.15.115:0/143231380 wait complete. 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1edac2f740 20 client.18393 trim_cache size 0 max 16384 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:50:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:50:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1eb80b2950 msgr2=0x7f1eb80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1eb80b2950 0x7f1eb80b4e60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8006010 tx=0x7f1ec8005de0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc109040 msgr2=0x7f1ecc19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc109040 0x7f1ecc19a690 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400eaa0 tx=0x7f1ec400ef60 comp rx=0 tx=0).stop 2024-05-09T05:50:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/353587710 shutdown_connections 2024-05-09T05:50:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1eb80b2950 0x7f1eb80b4e60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ecc109040 0x7f1ecc19a690 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc108ba0 0x7f1ecc1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 --2- 172.21.15.115:0/353587710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ecc1032f0 0x7f1ecc1a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.831+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/353587710 >> 172.21.15.115:0/353587710 conn(0x7f1ecc0782e0 msgr2=0x7f1ecc06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.832+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/353587710 shutdown_connections 2024-05-09T05:50:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:10.832+0000 7f1ed3c9f640 1 -- 172.21.15.115:0/353587710 wait complete. 2024-05-09T05:50:11.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:11 smithi195 ceph-mon[33844]: pgmap v375: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 35 op/s 2024-05-09T05:50:11.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:11 smithi195 ceph-mon[33844]: osdmap e102: 12 total, 12 up, 12 in 2024-05-09T05:50:11.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:11 smithi115 ceph-mon[30349]: pgmap v375: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 35 op/s 2024-05-09T05:50:11.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:11 smithi115 ceph-mon[30349]: osdmap e102: 12 total, 12 up, 12 in 2024-05-09T05:50:11.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:11 smithi161 ceph-mon[33805]: pgmap v375: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 35 op/s 2024-05-09T05:50:11.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:11 smithi161 ceph-mon[33805]: osdmap e102: 12 total, 12 up, 12 in 2024-05-09T05:50:13.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:13 smithi195 ceph-mon[33844]: pgmap v377: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 28 MiB/s wr, 43 op/s 2024-05-09T05:50:13.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:13 smithi115 ceph-mon[30349]: pgmap v377: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 28 MiB/s wr, 43 op/s 2024-05-09T05:50:13.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:13 smithi161 ceph-mon[33805]: pgmap v377: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 28 MiB/s wr, 43 op/s 2024-05-09T05:50:15.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:15 smithi195 ceph-mon[33844]: pgmap v378: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:15.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:15 smithi115 ceph-mon[30349]: pgmap v378: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:15.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:15 smithi161 ceph-mon[33805]: pgmap v378: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:17.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:17 smithi195 ceph-mon[33844]: pgmap v379: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:17.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:17 smithi115 ceph-mon[30349]: pgmap v379: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:17.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:17 smithi161 ceph-mon[33805]: pgmap v379: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:19.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:19 smithi195 ceph-mon[33844]: pgmap v380: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:19.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:19 smithi195 ceph-mon[33844]: osdmap e103: 12 total, 12 up, 12 in 2024-05-09T05:50:19.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:19 smithi115 ceph-mon[30349]: pgmap v380: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:19.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:19 smithi115 ceph-mon[30349]: osdmap e103: 12 total, 12 up, 12 in 2024-05-09T05:50:19.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:19 smithi161 ceph-mon[33805]: pgmap v380: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:19.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:19 smithi161 ceph-mon[33805]: osdmap e103: 12 total, 12 up, 12 in 2024-05-09T05:50:21.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:21 smithi195 ceph-mon[33844]: pgmap v382: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:21.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:21 smithi115 ceph-mon[30349]: pgmap v382: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:21.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:21 smithi161 ceph-mon[33805]: pgmap v382: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:23.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:23 smithi195 ceph-mon[33844]: pgmap v383: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:23.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:23 smithi115 ceph-mon[30349]: pgmap v383: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:23.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:23 smithi161 ceph-mon[33805]: pgmap v383: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.987+0000 7fbb11c0c640 1 -- 172.21.15.115:0/2807193472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108f50 msgr2=0x7fbb0c1106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.987+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/2807193472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108f50 0x7fbb0c1106f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009ca0 tx=0x7fbb0402a8c0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.987+0000 7fbb11c0c640 1 -- 172.21.15.115:0/2807193472 shutdown_connections 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.987+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/2807193472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108f50 0x7fbb0c1106f0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.987+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/2807193472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb0c108490 0x7fbb0c1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.987+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/2807193472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb0c102410 0x7fbb0c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.987+0000 7fbb11c0c640 1 -- 172.21.15.115:0/2807193472 >> 172.21.15.115:0/2807193472 conn(0x7fbb0c0779e0 msgr2=0x7fbb0c077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.987+0000 7fbb11c0c640 1 -- 172.21.15.115:0/2807193472 shutdown_connections 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.987+0000 7fbb11c0c640 1 -- 172.21.15.115:0/2807193472 wait complete. 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb11c0c640 1 Processor -- start 2024-05-09T05:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb11c0c640 1 -- start start 2024-05-09T05:50:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb11c0c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb0c102410 0x7fbb0c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb11c0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108490 0x7fbb0c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb11c0c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb0c108f50 0x7fbb0c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb11c0c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c113d60 con 0x7fbb0c108490 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb11c0c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c113be0 con 0x7fbb0c102410 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb11c0c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c113ee0 con 0x7fbb0c108f50 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108490 0x7fbb0c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108490 0x7fbb0c19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32808/0 (socket says 172.21.15.115:32808) 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0affd640 1 -- 172.21.15.115:0/492944022 learned_addr learned my addr 172.21.15.115:0/492944022 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb0c102410 0x7fbb0c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0bfff640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb0c108f50 0x7fbb0c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0affd640 1 -- 172.21.15.115:0/492944022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb0c108f50 msgr2=0x7fbb0c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0affd640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb0c108f50 0x7fbb0c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0affd640 1 -- 172.21.15.115:0/492944022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb0c102410 msgr2=0x7fbb0c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0affd640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb0c102410 0x7fbb0c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0affd640 1 -- 172.21.15.115:0/492944022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c19f5e0 con 0x7fbb0c108490 2024-05-09T05:50:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.988+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb0c102410 0x7fbb0c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.989+0000 7fbb0affd640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108490 0x7fbb0c19ace0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000d740 tx=0x7fbb0000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.989+0000 7fbb08ff9640 1 -- 172.21.15.115:0/492944022 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00017070 con 0x7fbb0c108490 2024-05-09T05:50:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.989+0000 7fbb08ff9640 1 -- 172.21.15.115:0/492944022 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb0000de50 con 0x7fbb0c108490 2024-05-09T05:50:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.989+0000 7fbb11c0c640 1 -- 172.21.15.115:0/492944022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c19f8d0 con 0x7fbb0c108490 2024-05-09T05:50:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.989+0000 7fbb08ff9640 1 -- 172.21.15.115:0/492944022 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00004c60 con 0x7fbb0c108490 2024-05-09T05:50:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.989+0000 7fbb11c0c640 1 -- 172.21.15.115:0/492944022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c1a7250 con 0x7fbb0c108490 2024-05-09T05:50:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb18b99740 1 Processor -- start 2024-05-09T05:50:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb18b99740 1 -- start start 2024-05-09T05:50:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb08ff9640 1 -- 172.21.15.115:0/492944022 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fbb00004e00 con 0x7fbb0c108490 2024-05-09T05:50:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7e21b0 0x56035fb85680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7b92f0 0x56035fb85bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 0x56035fb86100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb18b99740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56035faea7c0 con 0x56035f7e21b0 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb18b99740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56035fa860e0 con 0x56035f7c14d0 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb18b99740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56035fae6780 con 0x56035f7b92f0 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7e21b0 0x56035fb85680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7e21b0 0x56035fb85680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32822/0 (socket says 172.21.15.115:32822) 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb0bfff640 1 -- 172.21.15.115:0/1302066677 learned_addr learned my addr 172.21.15.115:0/1302066677 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7b92f0 0x56035fb85bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.992+0000 7fbb0affd640 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 0x56035fb86100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb0bfff640 1 -- 172.21.15.115:0/1302066677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7b92f0 msgr2=0x56035fb85bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb0bfff640 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7b92f0 0x56035fb85bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb0bfff640 1 -- 172.21.15.115:0/1302066677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 msgr2=0x56035fb86100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb0bfff640 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 0x56035fb86100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb0bfff640 1 -- 172.21.15.115:0/1302066677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035faaf7c0 con 0x56035f7e21b0 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7b92f0 0x56035fb85bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb0bfff640 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7e21b0 0x56035fb85680 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbb04005cc0 tx=0x7fbb04009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11e2c3fa3611251 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb08ff9640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbaf40b29d0 0x7fbaf40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbaf40b29d0 0x7fbaf40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbaf1ffb640 1 -- 172.21.15.115:0/1302066677 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04031750 con 0x56035f7e21b0 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbb08ff9640 1 -- 172.21.15.115:0/492944022 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fbb000e2300 con 0x7fbb0c108490 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbaf1ffb640 1 -- 172.21.15.115:0/1302066677 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb04036040 con 0x56035f7e21b0 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.993+0000 7fbaf1ffb640 1 -- 172.21.15.115:0/1302066677 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040042a0 con 0x56035f7e21b0 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb18b99740 1 -- 172.21.15.115:0/1302066677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7e21b0 msgr2=0x56035fb85680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbaf40b29d0 0x7fbaf40b4ee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x56035fb86a30 tx=0x7fbafc005f90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb18b99740 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7e21b0 0x56035fb85680 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbb04005cc0 tx=0x7fbb04009a20 comp rx=0 tx=0).stop 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb18b99740 1 -- 172.21.15.115:0/1302066677 shutdown_connections 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb18b99740 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 0x56035fb86100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb18b99740 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7b92f0 0x56035fb85bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb18b99740 1 --2- 172.21.15.115:0/1302066677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7e21b0 0x56035fb85680 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb18b99740 1 -- 172.21.15.115:0/1302066677 >> 172.21.15.115:0/1302066677 conn(0x56035f7d6300 msgr2=0x56035fb81e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb18b99740 1 -- 172.21.15.115:0/1302066677 shutdown_connections 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.994+0000 7fbb18b99740 1 -- 172.21.15.115:0/1302066677 wait complete. 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb18b99740 1 -- start start 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb18b99740 1 Processor -- start 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb916e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7c14d0 0x56035fb91c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7e21b0 0x56035fb8fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb18b99740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56035fb92750 con 0x56035f7b92f0 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb18b99740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56035fa719c0 con 0x56035f7e21b0 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb18b99740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56035fb928d0 con 0x56035f7c14d0 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb18b99740 10 client.0 _ll_register_callbacks cb 0x56035fa9e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb0b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb916e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb0b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb916e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32832/0 (socket says 172.21.15.115:32832) 2024-05-09T05:50:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb0b7fe640 1 -- 172.21.15.115:0/3136119258 learned_addr learned my addr 172.21.15.115:0/3136119258 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb0bfff640 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7e21b0 0x56035fb8fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7c14d0 0x56035fb91c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb0b7fe640 1 -- 172.21.15.115:0/3136119258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7c14d0 msgr2=0x56035fb91c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.995+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7c14d0 0x56035fb91c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbb0b7fe640 1 -- 172.21.15.115:0/3136119258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7e21b0 msgr2=0x56035fb8fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7e21b0 0x56035fb8fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbb0b7fe640 1 -- 172.21.15.115:0/3136119258 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035fa981f0 con 0x56035f7b92f0 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7c14d0 0x56035fb91c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb916e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0097e0 tx=0x7fbafc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a423531985bec76d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbaeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3136119258 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc024070 con 0x56035f7b92f0 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3136119258 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fbafc004460 con 0x56035f7b92f0 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbb18b99740 1 -- 172.21.15.115:0/3136119258 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb04036040 con 0x56035f7b92f0 2024-05-09T05:50:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3136119258 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fbafc030050 con 0x56035f7b92f0 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbb18b99740 10 client.18405 fetch_fsmap learned FSMap version 55 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbb18b99740 1 -- 172.21.15.115:0/3136119258 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56035f84d790 con 0x56035f7b92f0 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.996+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3136119258 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb04036040 con 0x56035f7b92f0 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3136119258 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fbafc003c80 con 0x56035f7b92f0 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbb18b99740 10 client.18405 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbb18b99740 10 client.18405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbb18b99740 20 client.18405 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56035f7a0d80 0x56035fba2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbb18b99740 4 client.18405 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbb18b99740 1 -- 172.21.15.115:0/3136119258 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56035faf83d0 con 0x56035f7a0d80 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56035f7a0d80 0x56035fba2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbaeaffd640 10 client.18405 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.997+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56035f7a0d80 0x56035fba2060 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a13bdbcc3febd server_cookie=2bc6255a8c64cbdc in_seq=0 out_seq=0 2024-05-09T05:50:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3136119258 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56035faf83d0 con 0x56035f7a0d80 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbaeaffd640 10 client.18405 handle_command_reply: tid=0 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 client.18405 shutdown 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 -- 172.21.15.115:0/3136119258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 msgr2=0x56035fb916e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb916e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0097e0 tx=0x7fbafc0046c0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 -- 172.21.15.115:0/3136119258 shutdown_connections 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7e21b0 0x56035fb8fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56035f7a0d80 0x56035fba2060 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7c14d0 0x56035fb91c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3136119258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb916e0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 -- 172.21.15.115:0/3136119258 >> 172.21.15.115:0/3136119258 conn(0x56035f7d6300 msgr2=0x56035fb80700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbaeaffd640 0 client.18405 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.998+0000 7fbb18b99740 1 -- 172.21.15.115:0/3136119258 shutdown_connections 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.999+0000 7fbb18b99740 1 -- 172.21.15.115:0/3136119258 wait complete. 2024-05-09T05:50:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:23.999+0000 7fbb18b99740 20 client.18405 trim_cache size 0 max 16384 2024-05-09T05:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.017+0000 7fbb18b99740 1 Processor -- start 2024-05-09T05:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.017+0000 7fbb18b99740 1 -- start start 2024-05-09T05:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7b92f0 0x56035fb8e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7c14d0 0x56035fb8e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 0x56035fb97b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb18b99740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1b11a0 con 0x56035f7c14d0 2024-05-09T05:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb18b99740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1b2d40 con 0x56035f7b92f0 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb18b99740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1b1870 con 0x56035f7e21b0 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7c14d0 0x56035fb8e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7c14d0 0x56035fb8e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32836/0 (socket says 172.21.15.115:32836) 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0affd640 1 -- 172.21.15.115:0/3923635849 learned_addr learned my addr 172.21.15.115:0/3923635849 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0bfff640 1 --2- 172.21.15.115:0/3923635849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 0x56035fb97b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/3923635849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7b92f0 0x56035fb8e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0affd640 1 -- 172.21.15.115:0/3923635849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 msgr2=0x56035fb97b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3923635849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 0x56035fb97b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0affd640 1 -- 172.21.15.115:0/3923635849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7b92f0 msgr2=0x56035fb8e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3923635849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7b92f0 0x56035fb8e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0affd640 1 -- 172.21.15.115:0/3923635849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035f8c45f0 con 0x56035f7c14d0 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.018+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3923635849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7c14d0 0x56035fb8e9b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fbb000a8f50 tx=0x7fbb000e2600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e183400c59ba60ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.019+0000 7fbaf17fa640 1 -- 172.21.15.115:0/3923635849 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00020040 con 0x56035f7c14d0 2024-05-09T05:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.019+0000 7fbaf17fa640 1 -- 172.21.15.115:0/3923635849 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb000a8d60 con 0x56035f7c14d0 2024-05-09T05:50:24.012 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.019+0000 7fbaf17fa640 1 -- 172.21.15.115:0/3923635849 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000df940 con 0x56035f7c14d0 2024-05-09T05:50:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.019+0000 7fbb18b99740 1 -- 172.21.15.115:0/3923635849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7c14d0 msgr2=0x56035fb8e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.019+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3923635849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7c14d0 0x56035fb8e9b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fbb000a8f50 tx=0x7fbb000e2600 comp rx=0 tx=0).stop 2024-05-09T05:50:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 -- 172.21.15.115:0/3923635849 shutdown_connections 2024-05-09T05:50:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3923635849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 0x56035fb97b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3923635849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7c14d0 0x56035fb8e9b0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3923635849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7b92f0 0x56035fb8e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 -- 172.21.15.115:0/3923635849 >> 172.21.15.115:0/3923635849 conn(0x56035f7d6300 msgr2=0x56035fb8ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 -- 172.21.15.115:0/3923635849 shutdown_connections 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 -- 172.21.15.115:0/3923635849 wait complete. 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 -- start start 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 Processor -- start 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb97d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 0x56035fb98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 0x56035fb95200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1b2670 con 0x56035f7b92f0 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1b1fa0 con 0x56035f7c14d0 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1af1a0 con 0x56035f7e21b0 2024-05-09T05:50:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb18b99740 10 client.0 _ll_register_callbacks cb 0x7fbafc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb0b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb97d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.020+0000 7fbb0b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb97d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32848/0 (socket says 172.21.15.115:32848) 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0b7fe640 1 -- 172.21.15.115:0/3800922769 learned_addr learned my addr 172.21.15.115:0/3800922769 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 0x56035fb98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0bfff640 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 0x56035fb95200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0b7fe640 1 -- 172.21.15.115:0/3800922769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 msgr2=0x56035fb95200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 0x56035fb95200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0b7fe640 1 -- 172.21.15.115:0/3800922769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 msgr2=0x56035fb98270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 0x56035fb98270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0b7fe640 1 -- 172.21.15.115:0/3800922769 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035faaeed0 con 0x56035f7b92f0 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 0x56035fb98270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0bfff640 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 0x56035fb95200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb0b7fe640 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb97d30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0097e0 tx=0x7fbafc011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f013ebab97644ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbaeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3800922769 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc024040 con 0x56035f7b92f0 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3800922769 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fbafc020c20 con 0x56035f7b92f0 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb18b99740 1 -- 172.21.15.115:0/3800922769 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb000a8d60 con 0x56035f7b92f0 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3800922769 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fbafc044050 con 0x56035f7b92f0 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb18b99740 10 client.18411 fetch_fsmap learned FSMap version 55 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.021+0000 7fbb18b99740 1 -- 172.21.15.115:0/3800922769 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56035fc09db0 con 0x56035f7b92f0 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.022+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3800922769 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb000a8d60 con 0x56035f7b92f0 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.022+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3800922769 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fbafc0109a0 con 0x56035f7b92f0 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.022+0000 7fbb18b99740 10 client.18411 fetch_fsmap finished waiting for FSMap version 55 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.022+0000 7fbb18b99740 10 client.18411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.022+0000 7fbb18b99740 20 client.18411 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.022+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56035f7761c0 0x56035fc0c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.022+0000 7fbb18b99740 4 client.18411 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:50:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.022+0000 7fbb18b99740 1 -- 172.21.15.115:0/3800922769 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56035faf2240 con 0x56035f7761c0 2024-05-09T05:50:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.022+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56035f7761c0 0x56035fc0c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.023+0000 7fbaeaffd640 10 client.18411 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.023+0000 7fbb0affd640 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56035f7761c0 0x56035fc0c570 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e359b0d29d821a3 server_cookie=618b10f75be2f8b2 in_seq=0 out_seq=0 2024-05-09T05:50:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbaeaffd640 1 -- 172.21.15.115:0/3800922769 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56035faf2240 con 0x56035f7761c0 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbaeaffd640 10 client.18411 handle_command_reply: tid=0 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 client.18411 shutdown 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 -- 172.21.15.115:0/3800922769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 msgr2=0x56035fb97d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb97d30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0097e0 tx=0x7fbafc011860 comp rx=0 tx=0).stop 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 -- 172.21.15.115:0/3800922769 shutdown_connections 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56035f7761c0 0x56035fc0c570 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56035f7e21b0 0x56035fb95200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbaeaffd640 0 client.18411 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56035f7c14d0 0x56035fb98270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 --2- 172.21.15.115:0/3800922769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56035f7b92f0 0x56035fb97d30 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 -- 172.21.15.115:0/3800922769 >> 172.21.15.115:0/3800922769 conn(0x56035f7d6300 msgr2=0x56035fb8e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 -- 172.21.15.115:0/3800922769 shutdown_connections 2024-05-09T05:50:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 1 -- 172.21.15.115:0/3800922769 wait complete. 2024-05-09T05:50:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.030+0000 7fbb18b99740 20 client.18411 trim_cache size 0 max 16384 2024-05-09T05:50:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:50:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:50:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:50:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 -- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbaf40b29d0 msgr2=0x7fbaf40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbaf40b29d0 0x7fbaf40b4ee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x56035fb86a30 tx=0x7fbafc005f90 comp rx=0 tx=0).stop 2024-05-09T05:50:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 -- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108490 msgr2=0x7fbb0c19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108490 0x7fbb0c19ace0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000d740 tx=0x7fbb0000dc00 comp rx=0 tx=0).stop 2024-05-09T05:50:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 -- 172.21.15.115:0/492944022 shutdown_connections 2024-05-09T05:50:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbaf40b29d0 0x7fbaf40b4ee0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb0c108f50 0x7fbb0c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb0c108490 0x7fbb0c19ace0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 --2- 172.21.15.115:0/492944022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb0c102410 0x7fbb0c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 -- 172.21.15.115:0/492944022 >> 172.21.15.115:0/492944022 conn(0x7fbb0c0779e0 msgr2=0x7fbb0c0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 -- 172.21.15.115:0/492944022 shutdown_connections 2024-05-09T05:50:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:24.031+0000 7fbb11c0c640 1 -- 172.21.15.115:0/492944022 wait complete. 2024-05-09T05:50:25.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:25 smithi195 ceph-mon[33844]: pgmap v384: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-05-09T05:50:25.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:25 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:50:25.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:25 smithi115 ceph-mon[30349]: pgmap v384: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-05-09T05:50:25.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:25 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:50:25.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:25 smithi161 ceph-mon[33805]: pgmap v384: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-05-09T05:50:25.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:25 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:50:25.812 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-09T05:50:25.813 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:50:26.380 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:50:27.568 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:27 smithi115 ceph-mon[30349]: pgmap v385: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-05-09T05:50:27.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:27 smithi195 ceph-mon[33844]: pgmap v385: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-05-09T05:50:27.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:27 smithi161 ceph-mon[33805]: pgmap v385: 129 pgs: 129 active+clean; 11 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.778+0000 7fcff9d0b700 1 -- 172.21.15.115:0/3277316576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4108740 msgr2=0x7fcff41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.778+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/3277316576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4108740 0x7fcff41100a0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00b0d0 tx=0x7fcfec00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.779+0000 7fcff9d0b700 1 -- 172.21.15.115:0/3277316576 shutdown_connections 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.779+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/3277316576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4108740 0x7fcff41100a0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.779+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/3277316576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41034c0 0x7fcff41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.779+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/3277316576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcff4107df0 0x7fcff4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.779+0000 7fcff9d0b700 1 -- 172.21.15.115:0/3277316576 >> 172.21.15.115:0/3277316576 conn(0x7fcff40fedc0 msgr2=0x7fcff41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.779+0000 7fcff9d0b700 1 -- 172.21.15.115:0/3277316576 shutdown_connections 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.779+0000 7fcff9d0b700 1 -- 172.21.15.115:0/3277316576 wait complete. 2024-05-09T05:50:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.779+0000 7fcff9d0b700 1 Processor -- start 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff9d0b700 1 -- start start 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff9d0b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41034c0 0x7fcff419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff9d0b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4107df0 0x7fcff419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff9d0b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcff4108740 0x7fcff41a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff9d0b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4113240 con 0x7fcff4107df0 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff9d0b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcff41130c0 con 0x7fcff4108740 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff9d0b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcff41133c0 con 0x7fcff41034c0 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4107df0 0x7fcff419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4107df0 0x7fcff419c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32868/0 (socket says 172.21.15.115:32868) 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff2ffd700 1 -- 172.21.15.115:0/68600417 learned_addr learned my addr 172.21.15.115:0/68600417 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff3fff700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcff4108740 0x7fcff41a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff37fe700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41034c0 0x7fcff419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff2ffd700 1 -- 172.21.15.115:0/68600417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41034c0 msgr2=0x7fcff419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff2ffd700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41034c0 0x7fcff419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff2ffd700 1 -- 172.21.15.115:0/68600417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcff4108740 msgr2=0x7fcff41a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff2ffd700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcff4108740 0x7fcff41a6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.780+0000 7fcff2ffd700 1 -- 172.21.15.115:0/68600417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff419c8f0 con 0x7fcff4107df0 2024-05-09T05:50:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.781+0000 7fcff2ffd700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4107df0 0x7fcff419c040 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800d980 tx=0x7fcfe800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.781+0000 7fcff0ff9700 1 -- 172.21.15.115:0/68600417 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8017070 con 0x7fcff4107df0 2024-05-09T05:50:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.781+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff41a71e0 con 0x7fcff4107df0 2024-05-09T05:50:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.781+0000 7fcff0ff9700 1 -- 172.21.15.115:0/68600417 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfe800fd20 con 0x7fcff4107df0 2024-05-09T05:50:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.781+0000 7fcff0ff9700 1 -- 172.21.15.115:0/68600417 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800e3b0 con 0x7fcff4107df0 2024-05-09T05:50:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.781+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff41a76d0 con 0x7fcff4107df0 2024-05-09T05:50:28.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.782+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff40512c0 con 0x7fcff4107df0 2024-05-09T05:50:28.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.784+0000 7fcff0ff9700 1 -- 172.21.15.115:0/68600417 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcfe800f890 con 0x7fcff4107df0 2024-05-09T05:50:28.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.785+0000 7fcff0ff9700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfdc0b29f0 0x7fcfdc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:28.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.785+0000 7fcff37fe700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfdc0b29f0 0x7fcfdc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:28.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.785+0000 7fcff0ff9700 1 -- 172.21.15.115:0/68600417 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fcfe80e23b0 con 0x7fcff4107df0 2024-05-09T05:50:28.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.786+0000 7fcff37fe700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfdc0b29f0 0x7fcfdc0b4f00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009c60 tx=0x7fcfe4009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:28.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:28.789+0000 7fcff0ff9700 1 -- 172.21.15.115:0/68600417 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcfe80a9a00 con 0x7fcff4107df0 2024-05-09T05:50:29.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.120+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcff402e5f0 con 0x7fcff4107df0 2024-05-09T05:50:29.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.121+0000 7fcff0ff9700 1 -- 172.21.15.115:0/68600417 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+11956 (secure 0 0 0) 0x7fcfe80a9300 con 0x7fcff4107df0 2024-05-09T05:50:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:50:29.112 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:50:23.370056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:50:29.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfdc0b29f0 msgr2=0x7fcfdc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:29.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfdc0b29f0 0x7fcfdc0b4f00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009c60 tx=0x7fcfe4009260 comp rx=0 tx=0).stop 2024-05-09T05:50:29.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4107df0 msgr2=0x7fcff419c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:29.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4107df0 0x7fcff419c040 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800d980 tx=0x7fcfe800dc90 comp rx=0 tx=0).stop 2024-05-09T05:50:29.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 shutdown_connections 2024-05-09T05:50:29.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcfdc0b29f0 0x7fcfdc0b4f00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:29.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcff4108740 0x7fcff41a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:29.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff41034c0 0x7fcff419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:29.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 --2- 172.21.15.115:0/68600417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcff4107df0 0x7fcff419c040 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:29.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 >> 172.21.15.115:0/68600417 conn(0x7fcff40fedc0 msgr2=0x7fcff41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:29.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 shutdown_connections 2024-05-09T05:50:29.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:29.124+0000 7fcff9d0b700 1 -- 172.21.15.115:0/68600417 wait complete. 2024-05-09T05:50:29.116 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 56 2024-05-09T05:50:29.393 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:29 smithi115 ceph-mon[30349]: pgmap v386: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-05-09T05:50:29.393 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:29 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/68600417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:50:29.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:29 smithi195 ceph-mon[33844]: pgmap v386: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-05-09T05:50:29.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:29 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/68600417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:50:29.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:29 smithi161 ceph-mon[33805]: pgmap v386: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-05-09T05:50:29.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:29 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/68600417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:50:30.541 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-05-09T05:50:30.542 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-05-09T05:50:30.816 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:50:31.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:31 smithi195 ceph-mon[33844]: pgmap v387: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 23 op/s 2024-05-09T05:50:31.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:31 smithi115 ceph-mon[30349]: pgmap v387: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 23 op/s 2024-05-09T05:50:31.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:31 smithi161 ceph-mon[33805]: pgmap v387: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 23 op/s 2024-05-09T05:50:32.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.002+0000 7f87bb5a3700 1 -- 172.21.15.115:0/732617371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b4107df0 msgr2=0x7f87b4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:32.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.002+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/732617371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b4107df0 0x7f87b4108200 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f87a40099c0 tx=0x7f87a4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:50:32.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.002+0000 7f87bb5a3700 1 -- 172.21.15.115:0/732617371 shutdown_connections 2024-05-09T05:50:32.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.002+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/732617371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b4108740 0x7f87b41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:32.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.002+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/732617371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b41034c0 0x7f87b41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:32.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.002+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/732617371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b4107df0 0x7f87b4108200 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:32.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.002+0000 7f87bb5a3700 1 -- 172.21.15.115:0/732617371 >> 172.21.15.115:0/732617371 conn(0x7f87b40fedc0 msgr2=0x7f87b41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:32.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.002+0000 7f87bb5a3700 1 -- 172.21.15.115:0/732617371 shutdown_connections 2024-05-09T05:50:32.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.002+0000 7f87bb5a3700 1 -- 172.21.15.115:0/732617371 wait complete. 2024-05-09T05:50:32.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.003+0000 7f87bb5a3700 1 Processor -- start 2024-05-09T05:50:32.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.003+0000 7f87bb5a3700 1 -- start start 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87bb5a3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b41034c0 0x7f87b419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87bb5a3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b4107df0 0x7f87b419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87bb5a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b4108740 0x7f87b4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87bb5a3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4113200 con 0x7f87b41034c0 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87bb5a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4113080 con 0x7f87b4108740 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87bb5a3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4113380 con 0x7f87b4107df0 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b933f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b41034c0 0x7f87b419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b933f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b41034c0 0x7f87b419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45792/0 (socket says 172.21.15.115:45792) 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b933f700 1 -- 172.21.15.115:0/271659387 learned_addr learned my addr 172.21.15.115:0/271659387 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b8b3e700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b4107df0 0x7f87b419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b9b40700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b4108740 0x7f87b4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:32.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b933f700 1 -- 172.21.15.115:0/271659387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b4107df0 msgr2=0x7f87b419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b933f700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b4107df0 0x7f87b419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b933f700 1 -- 172.21.15.115:0/271659387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b4108740 msgr2=0x7f87b4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b933f700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b4108740 0x7f87b4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b933f700 1 -- 172.21.15.115:0/271659387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b419a130 con 0x7f87b41034c0 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b9b40700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b4108740 0x7f87b4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.004+0000 7f87b933f700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b41034c0 0x7f87b419f860 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e9a0 tx=0x7f87b000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.005+0000 7f87aa7fc700 1 -- 172.21.15.115:0/271659387 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b000cbd0 con 0x7f87b41034c0 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.005+0000 7f87bb5a3700 1 -- 172.21.15.115:0/271659387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b419a420 con 0x7f87b41034c0 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.005+0000 7f87aa7fc700 1 -- 172.21.15.115:0/271659387 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87b000cd70 con 0x7f87b41034c0 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.005+0000 7f87aa7fc700 1 -- 172.21.15.115:0/271659387 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0018680 con 0x7f87b41034c0 2024-05-09T05:50:32.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.005+0000 7f87bb5a3700 1 -- 172.21.15.115:0/271659387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b406b610 con 0x7f87b41034c0 2024-05-09T05:50:32.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.008+0000 7f87aa7fc700 1 -- 172.21.15.115:0/271659387 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f87b0010950 con 0x7f87b41034c0 2024-05-09T05:50:33.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.009+0000 7f87aa7fc700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f87a00b29f0 0x7f87a00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.009+0000 7f87b8b3e700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f87a00b29f0 0x7f87a00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.009+0000 7f87aa7fc700 1 -- 172.21.15.115:0/271659387 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f87b00e2ce0 con 0x7f87b41034c0 2024-05-09T05:50:33.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.009+0000 7f87b8b3e700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f87a00b29f0 0x7f87a00b4f00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f87a4000c00 tx=0x7f87a401a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:33.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.028+0000 7f87c9fb9b80 1 Processor -- start 2024-05-09T05:50:33.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.028+0000 7f87c9fb9b80 1 -- start start 2024-05-09T05:50:33.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664af820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 0x55d6664a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d6663c3180 con 0x55d66640d2f0 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d666361980 con 0x55d666323630 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d6663bf140 con 0x55d6664a93f0 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b933f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b8b3e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b933f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45800/0 (socket says 172.21.15.115:45800) 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b933f700 1 -- 172.21.15.115:0/3896057370 learned_addr learned my addr 172.21.15.115:0/3896057370 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b8b3e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:37748/0 (socket says 172.21.15.115:37748) 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b9b40700 1 --2- 172.21.15.115:0/3896057370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 0x55d6664a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b9b40700 1 -- 172.21.15.115:0/3896057370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 msgr2=0x55d6664af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b9b40700 1 --2- 172.21.15.115:0/3896057370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b9b40700 1 -- 172.21.15.115:0/3896057370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 msgr2=0x55d6664af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b9b40700 1 --2- 172.21.15.115:0/3896057370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.029+0000 7f87b9b40700 1 -- 172.21.15.115:0/3896057370 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d666359500 con 0x55d6664a93f0 2024-05-09T05:50:33.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.030+0000 7f87b9b40700 1 --2- 172.21.15.115:0/3896057370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 0x55d6664a9800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009d10 tx=0x7f87ac0077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75a8b3545689d4dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.030+0000 7f879f7fe700 1 -- 172.21.15.115:0/3896057370 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac009e60 con 0x55d6664a93f0 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.030+0000 7f879f7fe700 1 -- 172.21.15.115:0/3896057370 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87ac019070 con 0x55d6664a93f0 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.030+0000 7f879f7fe700 1 -- 172.21.15.115:0/3896057370 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac013740 con 0x55d6664a93f0 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.031+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/3896057370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 msgr2=0x55d6664a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.031+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/3896057370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 0x55d6664a9800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009d10 tx=0x7f87ac0077e0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.031+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/3896057370 shutdown_connections 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.031+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/3896057370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 0x55d6664a9800 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.031+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/3896057370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.031+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/3896057370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.031+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/3896057370 >> 172.21.15.115:0/3896057370 conn(0x55d666383830 msgr2=0x55d66649bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.031+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/3896057370 shutdown_connections 2024-05-09T05:50:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.031+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/3896057370 wait complete. 2024-05-09T05:50:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.032+0000 7f87c9fb9b80 1 -- start start 2024-05-09T05:50:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.032+0000 7f87c9fb9b80 1 Processor -- start 2024-05-09T05:50:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 0x55d6664b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d66623ed30 con 0x55d66640d2f0 2024-05-09T05:50:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b933f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b8b3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d66637d2e0 con 0x55d6664a93f0 2024-05-09T05:50:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b933f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:47612/0 (socket says 172.21.15.115:47612) 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d6664aa1e0 con 0x55d666323630 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b933f700 1 -- 172.21.15.115:0/557269059 learned_addr learned my addr 172.21.15.115:0/557269059 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b9b40700 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 0x55d6664b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87c9fb9b80 10 client.0 _ll_register_callbacks cb 0x55d666234850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b933f700 1 -- 172.21.15.115:0/557269059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 msgr2=0x55d6664b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b933f700 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 0x55d6664b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b933f700 1 -- 172.21.15.115:0/557269059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 msgr2=0x55d6664b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b933f700 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.033+0000 7f87b933f700 1 -- 172.21.15.115:0/557269059 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66623e620 con 0x55d666323630 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.034+0000 7f87b933f700 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b3020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f87b00e0150 tx=0x7f87b00b28e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7082c1fcef1245c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.034+0000 7f879cff9700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.034+0000 7f879cff9700 1 -- 172.21.15.115:0/557269059 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b00a53c0 con 0x55d666323630 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.034+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/557269059 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ac019070 con 0x55d666323630 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.034+0000 7f879cff9700 1 -- 172.21.15.115:0/557269059 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f87b0010640 con 0x55d666323630 2024-05-09T05:50:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.034+0000 7f879cff9700 1 -- 172.21.15.115:0/557269059 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f87b006e050 con 0x55d666323630 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.034+0000 7f87c9fb9b80 10 client.27769 fetch_fsmap learned FSMap version 56 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.034+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/557269059 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55d6664c0950 con 0x55d666323630 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.035+0000 7f879cff9700 1 -- 172.21.15.115:0/557269059 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87ac019070 con 0x55d666323630 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.035+0000 7f879cff9700 1 -- 172.21.15.115:0/557269059 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7f87b00a8900 con 0x55d666323630 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.035+0000 7f87c9fb9b80 10 client.27769 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.035+0000 7f87c9fb9b80 10 client.27769 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.035+0000 7f87c9fb9b80 20 client.27769 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.035+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6664c2ee0 0x55d6664c5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.035+0000 7f87c9fb9b80 4 client.27769 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.035+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/557269059 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6663d0b60 con 0x55d6664c2ee0 2024-05-09T05:50:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.035+0000 7f87b8b3e700 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6664c2ee0 0x55d6664c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.036+0000 7f879cff9700 10 client.27769 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.036+0000 7f87b8b3e700 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6664c2ee0 0x55d6664c5320 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fadae8b2085e2ce7 server_cookie=da57cabd0a22c216 in_seq=0 out_seq=0 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.036+0000 7f879cff9700 1 -- 172.21.15.115:0/557269059 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6663d0b60 con 0x55d6664c2ee0 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.036+0000 7f879cff9700 10 client.27769 handle_command_reply: tid=0 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.036+0000 7f87c9fb9b80 1 client.27769 shutdown 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/557269059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 msgr2=0x55d6664b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b3020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f87b00e0150 tx=0x7f87b00b28e0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/557269059 shutdown_connections 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 0x55d6664b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d6664c2ee0 0x55d6664c5320 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b3020 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/557269059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f879cff9700 0 client.27769 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/557269059 >> 172.21.15.115:0/557269059 conn(0x55d666383830 msgr2=0x55d66649b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/557269059 shutdown_connections 2024-05-09T05:50:33.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/557269059 wait complete. 2024-05-09T05:50:33.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.037+0000 7f87c9fb9b80 20 client.27769 trim_cache size 0 max 16384 2024-05-09T05:50:33.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87c9fb9b80 1 Processor -- start 2024-05-09T05:50:33.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87c9fb9b80 1 -- start start 2024-05-09T05:50:33.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 0x55d6664b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41ae300 con 0x55d66640d2f0 2024-05-09T05:50:33.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41afed0 con 0x55d6664a93f0 2024-05-09T05:50:33.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41ae9d0 con 0x55d666323630 2024-05-09T05:50:33.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87b8b3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87b8b3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45824/0 (socket says 172.21.15.115:45824) 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87b8b3e700 1 -- 172.21.15.115:0/256436746 learned_addr learned my addr 172.21.15.115:0/256436746 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87b9b40700 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 0x55d6664b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.059+0000 7f87b933f700 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f87b9b40700 1 -- 172.21.15.115:0/256436746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 msgr2=0x55d6664b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f87b9b40700 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f87b9b40700 1 -- 172.21.15.115:0/256436746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 msgr2=0x55d6664a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f87b9b40700 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f87b9b40700 1 -- 172.21.15.115:0/256436746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6664b3e20 con 0x55d6664a93f0 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f87b8b3e700 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f87b9b40700 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 0x55d6664b38e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007fd0 tx=0x7f87ac009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8995caf56c2aa423 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:33.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f879effd700 1 -- 172.21.15.115:0/256436746 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac003c00 con 0x55d6664a93f0 2024-05-09T05:50:33.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f879effd700 1 -- 172.21.15.115:0/256436746 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87ac019040 con 0x55d6664a93f0 2024-05-09T05:50:33.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.060+0000 7f879effd700 1 -- 172.21.15.115:0/256436746 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac01f670 con 0x55d6664a93f0 2024-05-09T05:50:33.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/256436746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 msgr2=0x55d6664b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:33.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 0x55d6664b38e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007fd0 tx=0x7f87ac009120 comp rx=0 tx=0).stop 2024-05-09T05:50:33.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/256436746 shutdown_connections 2024-05-09T05:50:33.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6664a93f0 0x55d6664b38e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d666323630 0x55d6664b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/256436746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/256436746 >> 172.21.15.115:0/256436746 conn(0x55d666383830 msgr2=0x55d6664acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:33.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/256436746 shutdown_connections 2024-05-09T05:50:33.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/256436746 wait complete. 2024-05-09T05:50:33.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.062+0000 7f87c9fb9b80 1 -- start start 2024-05-09T05:50:33.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87c9fb9b80 1 Processor -- start 2024-05-09T05:50:33.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87c9fb9b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 0x55d6664b8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41af800 con 0x55d66640d2f0 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41af130 con 0x55d666323630 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87c9fb9b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41ac2c0 con 0x55d6664a93f0 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87c9fb9b80 10 client.0 _ll_register_callbacks cb 0x55d666234850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87b8b3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87b8b3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664bb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45838/0 (socket says 172.21.15.115:45838) 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87b8b3e700 1 -- 172.21.15.115:0/1085555978 learned_addr learned my addr 172.21.15.115:0/1085555978 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87b933f700 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.063+0000 7f87b9b40700 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 0x55d6664b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87b8b3e700 1 -- 172.21.15.115:0/1085555978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 msgr2=0x55d6664b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87b8b3e700 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 0x55d6664b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87b8b3e700 1 -- 172.21.15.115:0/1085555978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 msgr2=0x55d6664bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:33.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87b8b3e700 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87b8b3e700 1 -- 172.21.15.115:0/1085555978 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6664b8660 con 0x55d66640d2f0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87b933f700 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87b8b3e700 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664bb690 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f87a4015000 tx=0x7f87a4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9ac5162198908f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f879cff9700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f879cff9700 1 -- 172.21.15.115:0/1085555978 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a400bba0 con 0x55d66640d2f0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f879cff9700 1 -- 172.21.15.115:0/1085555978 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f87a400bd40 con 0x55d66640d2f0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/1085555978 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ac019040 con 0x55d66640d2f0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f879cff9700 1 -- 172.21.15.115:0/1085555978 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f87a4043050 con 0x55d66640d2f0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87c9fb9b80 10 client.18435 fetch_fsmap learned FSMap version 56 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.064+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/1085555978 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55d66652dd20 con 0x55d66640d2f0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.065+0000 7f879cff9700 1 -- 172.21.15.115:0/1085555978 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87a40335a0 con 0x55d66640d2f0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.065+0000 7f879cff9700 1 -- 172.21.15.115:0/1085555978 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7f87a400f460 con 0x55d66640d2f0 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.065+0000 7f87c9fb9b80 10 client.18435 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:50:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.065+0000 7f87c9fb9b80 10 client.18435 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:50:33.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.065+0000 7f87c9fb9b80 20 client.18435 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:33.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.065+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d66652e4d0 0x55d6665308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:33.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.065+0000 7f87c9fb9b80 4 client.18435 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-05-09T05:50:33.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.065+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/1085555978 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x7f87ac019040 con 0x55d66652e4d0 2024-05-09T05:50:33.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.066+0000 7f87b933f700 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d66652e4d0 0x55d6665308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:33.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.066+0000 7f87b933f700 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d66652e4d0 0x55d6665308f0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257d240fdfa104bb server_cookie=e64e5ed9ad3475be in_seq=0 out_seq=0 2024-05-09T05:50:33.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.066+0000 7f879cff9700 10 client.18435 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:33.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.157+0000 7f879cff9700 1 -- 172.21.15.115:0/1085555978 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f87ac019040 con 0x55d66652e4d0 2024-05-09T05:50:33.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.157+0000 7f879cff9700 10 client.18435 handle_command_reply: tid=0 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.157+0000 7f87c9fb9b80 1 client.18435 shutdown 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/1085555978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 msgr2=0x55d6664bb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664bb690 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f87a4015000 tx=0x7f87a4009310 comp rx=0 tx=0).stop 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/1085555978 shutdown_connections 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d666323630 0x55d6664bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d66652e4d0 0x55d6665308f0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6664a93f0 0x55d6664b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 --2- 172.21.15.115:0/1085555978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d66640d2f0 0x55d6664bb690 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/1085555978 >> 172.21.15.115:0/1085555978 conn(0x55d666383830 msgr2=0x55d6664a0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f879cff9700 0 client.18435 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/1085555978 shutdown_connections 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 1 -- 172.21.15.115:0/1085555978 wait complete. 2024-05-09T05:50:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.158+0000 7f87c9fb9b80 20 client.18435 trim_cache size 0 max 16384 2024-05-09T05:50:33.151 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:50:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 56, 2024-05-09T05:50:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:50:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 34, 2024-05-09T05:50:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:50:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "c137a69d": { 2024-05-09T05:50:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 34, 2024-05-09T05:50:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:50:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 130.0, 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:50:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:33.153 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:50:33.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 -- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f87a00b29f0 msgr2=0x7f87a00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f87a00b29f0 0x7f87a00b4f00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f87a4000c00 tx=0x7f87a401a040 comp rx=0 tx=0).stop 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 -- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b41034c0 msgr2=0x7f87b419f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b41034c0 0x7f87b419f860 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e9a0 tx=0x7f87b000ed60 comp rx=0 tx=0).stop 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 -- 172.21.15.115:0/271659387 shutdown_connections 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f87a00b29f0 0x7f87a00b4f00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b4108740 0x7f87b4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b41034c0 0x7f87b419f860 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 --2- 172.21.15.115:0/271659387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87b4107df0 0x7f87b419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 -- 172.21.15.115:0/271659387 >> 172.21.15.115:0/271659387 conn(0x7f87b40fedc0 msgr2=0x7f87b41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 -- 172.21.15.115:0/271659387 shutdown_connections 2024-05-09T05:50:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:33.159+0000 7f87bb5a3700 1 -- 172.21.15.115:0/271659387 wait complete. 2024-05-09T05:50:33.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:33 smithi195 ceph-mon[33844]: pgmap v388: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:50:33.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:33 smithi115 ceph-mon[30349]: pgmap v388: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:50:33.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:33 smithi161 ceph-mon[33805]: pgmap v388: 129 pgs: 129 active+clean; 11 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-09T05:50:34.037 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c137a69d', quiesce took 8.2 seconds. Will release after: 1.8000000000000007 2024-05-09T05:50:35.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:35 smithi195 ceph-mon[33844]: pgmap v389: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 31 op/s 2024-05-09T05:50:35.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:35 smithi115 ceph-mon[30349]: pgmap v389: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 31 op/s 2024-05-09T05:50:35.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:35 smithi161 ceph-mon[33805]: pgmap v389: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 MiB/s wr, 31 op/s 2024-05-09T05:50:35.838 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:50:36.104 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:50:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.152+0000 7f6abaad8640 1 -- 172.21.15.115:0/1419846902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab41032e0 msgr2=0x7f6ab410f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.152+0000 7f6abaad8640 1 --2- 172.21.15.115:0/1419846902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab41032e0 0x7f6ab410f6b0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009ca0 tx=0x7f6aac02a8c0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.152+0000 7f6abaad8640 1 -- 172.21.15.115:0/1419846902 shutdown_connections 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.152+0000 7f6abaad8640 1 --2- 172.21.15.115:0/1419846902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab41032e0 0x7f6ab410f6b0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.152+0000 7f6abaad8640 1 --2- 172.21.15.115:0/1419846902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab4109030 0x7f6ab4102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.152+0000 7f6abaad8640 1 --2- 172.21.15.115:0/1419846902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab4108b90 0x7f6ab4102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.152+0000 7f6abaad8640 1 -- 172.21.15.115:0/1419846902 >> 172.21.15.115:0/1419846902 conn(0x7f6ab40fe390 msgr2=0x7f6ab41007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.153+0000 7f6abaad8640 1 -- 172.21.15.115:0/1419846902 shutdown_connections 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.153+0000 7f6abaad8640 1 -- 172.21.15.115:0/1419846902 wait complete. 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.153+0000 7f6abaad8640 1 Processor -- start 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.153+0000 7f6abaad8640 1 -- start start 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6abaad8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab41032e0 0x7f6ab419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6abaad8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab4108b90 0x7f6ab419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6abaad8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab4109030 0x7f6ab419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6abaad8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab4112c80 con 0x7f6ab4108b90 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6abaad8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab4112b00 con 0x7f6ab41032e0 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6abaad8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab4112e00 con 0x7f6ab4109030 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab4108b90 0x7f6ab419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab4108b90 0x7f6ab419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45856/0 (socket says 172.21.15.115:45856) 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab3fff640 1 -- 172.21.15.115:0/3358863099 learned_addr learned my addr 172.21.15.115:0/3358863099 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab8874640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab41032e0 0x7f6ab419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab9075640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab4109030 0x7f6ab419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab3fff640 1 -- 172.21.15.115:0/3358863099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab4109030 msgr2=0x7f6ab419c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab4109030 0x7f6ab419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab3fff640 1 -- 172.21.15.115:0/3358863099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab41032e0 msgr2=0x7f6ab419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab41032e0 0x7f6ab419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab3fff640 1 -- 172.21.15.115:0/3358863099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab41a6830 con 0x7f6ab4108b90 2024-05-09T05:50:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab4108b90 0x7f6ab419bb40 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400d740 tx=0x7f6aa400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab1ffb640 1 -- 172.21.15.115:0/3358863099 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4017070 con 0x7f6ab4108b90 2024-05-09T05:50:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab1ffb640 1 -- 172.21.15.115:0/3358863099 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6aa4009cb0 con 0x7f6ab4108b90 2024-05-09T05:50:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6abaad8640 1 -- 172.21.15.115:0/3358863099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab41a6b20 con 0x7f6ab4108b90 2024-05-09T05:50:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6ab1ffb640 1 -- 172.21.15.115:0/3358863099 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4003bb0 con 0x7f6ab4108b90 2024-05-09T05:50:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.154+0000 7f6abaad8640 1 -- 172.21.15.115:0/3358863099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab41a7070 con 0x7f6ab4108b90 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.157+0000 7f6ac1989740 1 Processor -- start 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.157+0000 7f6ab1ffb640 1 -- 172.21.15.115:0/3358863099 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f6aa4009820 con 0x7f6ab4108b90 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.157+0000 7f6ac1989740 1 -- start start 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 0x560dd0f71680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 0x560dd0f71bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0bad4d0 0x560dd0f72100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ac1989740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560dd0ed67c0 con 0x560dd0bad4d0 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ac1989740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560dd0e720e0 con 0x560dd0ba52f0 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ac1989740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560dd0ed2780 con 0x560dd0bce1b0 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0bad4d0 0x560dd0f72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0bad4d0 0x560dd0f72100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45866/0 (socket says 172.21.15.115:45866) 2024-05-09T05:50:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab3fff640 1 -- 172.21.15.115:0/128454386 learned_addr learned my addr 172.21.15.115:0/128454386 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab8874640 1 --2- 172.21.15.115:0/128454386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 0x560dd0f71bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab9075640 1 --2- 172.21.15.115:0/128454386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 0x560dd0f71680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab3fff640 1 -- 172.21.15.115:0/128454386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 msgr2=0x560dd0f71680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/128454386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 0x560dd0f71680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab3fff640 1 -- 172.21.15.115:0/128454386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 msgr2=0x560dd0f71bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/128454386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 0x560dd0f71bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab3fff640 1 -- 172.21.15.115:0/128454386 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd0e9b7c0 con 0x560dd0bad4d0 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.158+0000 7f6ab1ffb640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a9c0b2950 0x7f6a9c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.159+0000 7f6ab9075640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a9c0b2950 0x7f6a9c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.159+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/128454386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0bad4d0 0x560dd0f72100 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40df9b0 tx=0x7f6aa40e8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a127f5460836519 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.159+0000 7f6ab1ffb640 1 -- 172.21.15.115:0/3358863099 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f6aa40e2710 con 0x7f6ab4108b90 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.159+0000 7f6a9affd640 1 -- 172.21.15.115:0/128454386 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4020070 con 0x560dd0bad4d0 2024-05-09T05:50:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.159+0000 7f6a9affd640 1 -- 172.21.15.115:0/128454386 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6aa40a9110 con 0x560dd0bad4d0 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.159+0000 7f6a9affd640 1 -- 172.21.15.115:0/128454386 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40a93d0 con 0x560dd0bad4d0 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 -- 172.21.15.115:0/128454386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0bad4d0 msgr2=0x560dd0f72100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 --2- 172.21.15.115:0/128454386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0bad4d0 0x560dd0f72100 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40df9b0 tx=0x7f6aa40e8cb0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ab9075640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a9c0b2950 0x7f6a9c0b4e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6aac02aeb0 tx=0x7f6aac005c50 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 -- 172.21.15.115:0/128454386 shutdown_connections 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 --2- 172.21.15.115:0/128454386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0bad4d0 0x560dd0f72100 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 --2- 172.21.15.115:0/128454386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 0x560dd0f71bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 --2- 172.21.15.115:0/128454386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 0x560dd0f71680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 -- 172.21.15.115:0/128454386 >> 172.21.15.115:0/128454386 conn(0x560dd0bc2300 msgr2=0x560dd0f6de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 -- 172.21.15.115:0/128454386 shutdown_connections 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 -- 172.21.15.115:0/128454386 wait complete. 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 -- start start 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.160+0000 7f6ac1989740 1 Processor -- start 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bad4d0 0x560dd0f7dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 0x560dd0f7bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ac1989740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560dd0f7e7b0 con 0x560dd0ba52f0 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ac1989740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560dd0e5d9c0 con 0x560dd0bad4d0 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ac1989740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560dd0f7e930 con 0x560dd0bce1b0 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab8874640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ac1989740 10 client.0 _ll_register_callbacks cb 0x560dd0e8a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab8874640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45870/0 (socket says 172.21.15.115:45870) 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab8874640 1 -- 172.21.15.115:0/402144730 learned_addr learned my addr 172.21.15.115:0/402144730 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab9075640 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 0x560dd0f7bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bad4d0 0x560dd0f7dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab8874640 1 -- 172.21.15.115:0/402144730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 msgr2=0x560dd0f7bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab8874640 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 0x560dd0f7bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab8874640 1 -- 172.21.15.115:0/402144730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bad4d0 msgr2=0x560dd0f7dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab8874640 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bad4d0 0x560dd0f7dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab8874640 1 -- 172.21.15.115:0/402144730 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd0e841f0 con 0x560dd0ba52f0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab9075640 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 0x560dd0f7bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ab8874640 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7d770 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8010840 tx=0x7f6aa8010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b340b55ded864c3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6a93fff640 1 -- 172.21.15.115:0/402144730 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa801e060 con 0x560dd0ba52f0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6a93fff640 1 -- 172.21.15.115:0/402144730 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f6aa800fb90 con 0x560dd0ba52f0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.161+0000 7f6ac1989740 1 -- 172.21.15.115:0/402144730 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa40a9110 con 0x560dd0ba52f0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6a93fff640 1 -- 172.21.15.115:0/402144730 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f6aa8025050 con 0x560dd0ba52f0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6ac1989740 10 client.18447 fetch_fsmap learned FSMap version 56 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6ac1989740 1 -- 172.21.15.115:0/402144730 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x560dd0c39790 con 0x560dd0ba52f0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6a93fff640 1 -- 172.21.15.115:0/402144730 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6aa40a9110 con 0x560dd0ba52f0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6a93fff640 1 -- 172.21.15.115:0/402144730 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7f6aa8018370 con 0x560dd0ba52f0 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6ac1989740 10 client.18447 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6ac1989740 10 client.18447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6ac1989740 20 client.18447 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6ac1989740 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560dd0b8cd80 0x560dd0f8e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6ac1989740 4 client.18447 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6ac1989740 1 -- 172.21.15.115:0/402144730 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd0ee43d0 con 0x560dd0b8cd80 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.162+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560dd0b8cd80 0x560dd0f8e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.163+0000 7f6a93fff640 10 client.18447 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.163+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560dd0b8cd80 0x560dd0f8e060 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=992fdd73c62ac835 server_cookie=ff9a813372ca8bb5 in_seq=0 out_seq=0 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.163+0000 7f6a93fff640 1 -- 172.21.15.115:0/402144730 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560dd0ee43d0 con 0x560dd0b8cd80 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.163+0000 7f6a93fff640 10 client.18447 handle_command_reply: tid=0 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.163+0000 7f6ac1989740 1 client.18447 shutdown 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 -- 172.21.15.115:0/402144730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 msgr2=0x560dd0f7d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7d770 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8010840 tx=0x7f6aa8010d00 comp rx=0 tx=0).stop 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 -- 172.21.15.115:0/402144730 shutdown_connections 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560dd0b8cd80 0x560dd0f8e060 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bce1b0 0x560dd0f7bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bad4d0 0x560dd0f7dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 --2- 172.21.15.115:0/402144730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7d770 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6a93fff640 0 client.18447 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 -- 172.21.15.115:0/402144730 >> 172.21.15.115:0/402144730 conn(0x560dd0bc2300 msgr2=0x560dd0f6c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 -- 172.21.15.115:0/402144730 shutdown_connections 2024-05-09T05:50:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 1 -- 172.21.15.115:0/402144730 wait complete. 2024-05-09T05:50:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.164+0000 7f6ac1989740 20 client.18447 trim_cache size 0 max 16384 2024-05-09T05:50:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ac1989740 1 Processor -- start 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ac1989740 1 -- start start 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bc2300 0x560dd0f6ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0b8cd80 0x560dd0f753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 0x560dd0f75920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ac1989740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab41b1200 con 0x560dd0b8cd80 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ac1989740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab41b2dd0 con 0x560dd0ba52f0 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ac1989740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab41b18d0 con 0x560dd0bc2300 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0b8cd80 0x560dd0f753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0b8cd80 0x560dd0f753e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45874/0 (socket says 172.21.15.115:45874) 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab3fff640 1 -- 172.21.15.115:0/2811475828 learned_addr learned my addr 172.21.15.115:0/2811475828 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab9075640 1 --2- 172.21.15.115:0/2811475828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 0x560dd0f75920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab3fff640 1 -- 172.21.15.115:0/2811475828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bc2300 msgr2=0x560dd0f6ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/2811475828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bc2300 0x560dd0f6ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab3fff640 1 -- 172.21.15.115:0/2811475828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 msgr2=0x560dd0f75920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/2811475828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 0x560dd0f75920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab3fff640 1 -- 172.21.15.115:0/2811475828 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd0cb05f0 con 0x560dd0b8cd80 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.179+0000 7f6ab9075640 1 --2- 172.21.15.115:0/2811475828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 0x560dd0f75920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/2811475828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0b8cd80 0x560dd0f753e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40ea670 tx=0x7f6aa400d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf18454a487c8bf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6a9a7fc640 1 -- 172.21.15.115:0/2811475828 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40c3070 con 0x560dd0b8cd80 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6a9a7fc640 1 -- 172.21.15.115:0/2811475828 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6aa40a9110 con 0x560dd0b8cd80 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6a9a7fc640 1 -- 172.21.15.115:0/2811475828 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4020040 con 0x560dd0b8cd80 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ac1989740 1 -- 172.21.15.115:0/2811475828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0b8cd80 msgr2=0x560dd0f753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ac1989740 1 --2- 172.21.15.115:0/2811475828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0b8cd80 0x560dd0f753e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40ea670 tx=0x7f6aa400d6c0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ac1989740 1 -- 172.21.15.115:0/2811475828 shutdown_connections 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ac1989740 1 --2- 172.21.15.115:0/2811475828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0ba52f0 0x560dd0f75920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ac1989740 1 --2- 172.21.15.115:0/2811475828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0b8cd80 0x560dd0f753e0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ac1989740 1 --2- 172.21.15.115:0/2811475828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0bc2300 0x560dd0f6ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ac1989740 1 -- 172.21.15.115:0/2811475828 >> 172.21.15.115:0/2811475828 conn(0x560dd0f77570 msgr2=0x560dd0f6c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ac1989740 1 -- 172.21.15.115:0/2811475828 shutdown_connections 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.180+0000 7f6ac1989740 1 -- 172.21.15.115:0/2811475828 wait complete. 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ac1989740 1 -- start start 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ac1989740 1 Processor -- start 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0b8cd80 0x560dd0f83f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ac1989740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bc2300 0x560dd0f7f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ac1989740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab41b2700 con 0x560dd0ba52f0 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ac1989740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab41b2030 con 0x560dd0bc2300 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ac1989740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab41af200 con 0x560dd0b8cd80 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ac1989740 10 client.0 _ll_register_callbacks cb 0x7f6aa8010450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45884/0 (socket says 172.21.15.115:45884) 2024-05-09T05:50:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.181+0000 7f6ab3fff640 1 -- 172.21.15.115:0/1272753128 learned_addr learned my addr 172.21.15.115:0/1272753128 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab8874640 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0b8cd80 0x560dd0f83f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab9075640 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bc2300 0x560dd0f7f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab3fff640 1 -- 172.21.15.115:0/1272753128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0b8cd80 msgr2=0x560dd0f83f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0b8cd80 0x560dd0f83f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab3fff640 1 -- 172.21.15.115:0/1272753128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bc2300 msgr2=0x560dd0f7f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bc2300 0x560dd0f7f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab3fff640 1 -- 172.21.15.115:0/1272753128 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd0e9aed0 con 0x560dd0ba52f0 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab9075640 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bc2300 0x560dd0f7f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab8874640 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0b8cd80 0x560dd0f83f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ab3fff640 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7f2d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009f80 tx=0x7f6aa40ad9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0612a3e34b6b208 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6a93fff640 1 -- 172.21.15.115:0/1272753128 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4020070 con 0x560dd0ba52f0 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6a93fff640 1 -- 172.21.15.115:0/1272753128 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f6aa40e86d0 con 0x560dd0ba52f0 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ac1989740 1 -- 172.21.15.115:0/1272753128 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa40a9110 con 0x560dd0ba52f0 2024-05-09T05:50:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6a93fff640 1 -- 172.21.15.115:0/1272753128 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f6aa40c3040 con 0x560dd0ba52f0 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ac1989740 10 client.18453 fetch_fsmap learned FSMap version 56 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6ac1989740 1 -- 172.21.15.115:0/1272753128 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x560dd0ff5e40 con 0x560dd0ba52f0 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6a93fff640 1 -- 172.21.15.115:0/1272753128 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6aa40a9110 con 0x560dd0ba52f0 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.182+0000 7f6a93fff640 1 -- 172.21.15.115:0/1272753128 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7f6aa40e22a0 con 0x560dd0ba52f0 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.183+0000 7f6ac1989740 10 client.18453 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.183+0000 7f6ac1989740 10 client.18453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.183+0000 7f6ac1989740 20 client.18453 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.183+0000 7f6ac1989740 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560dd0bad4d0 0x560dd0ff80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.183+0000 7f6ac1989740 4 client.18453 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.183+0000 7f6ac1989740 1 -- 172.21.15.115:0/1272753128 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dd0ede240 con 0x560dd0bad4d0 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.183+0000 7f6ab8874640 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560dd0bad4d0 0x560dd0ff80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.183+0000 7f6a93fff640 10 client.18453 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.183+0000 7f6ab8874640 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560dd0bad4d0 0x560dd0ff80d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c3758adfa610ae4 server_cookie=79ab7e476d32dc37 in_seq=0 out_seq=0 2024-05-09T05:50:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6a93fff640 1 -- 172.21.15.115:0/1272753128 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560dd0ede240 con 0x560dd0bad4d0 2024-05-09T05:50:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6a93fff640 10 client.18453 handle_command_reply: tid=0 2024-05-09T05:50:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 client.18453 shutdown 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 -- 172.21.15.115:0/1272753128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 msgr2=0x560dd0f7f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7f2d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009f80 tx=0x7f6aa40ad9b0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 -- 172.21.15.115:0/1272753128 shutdown_connections 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dd0bc2300 0x560dd0f7f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560dd0ba52f0 0x560dd0f7f2d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560dd0bad4d0 0x560dd0ff80d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 --2- 172.21.15.115:0/1272753128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560dd0b8cd80 0x560dd0f83f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6a93fff640 0 client.18453 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 -- 172.21.15.115:0/1272753128 >> 172.21.15.115:0/1272753128 conn(0x560dd0f77570 msgr2=0x560dd0f6d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 -- 172.21.15.115:0/1272753128 shutdown_connections 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 1 -- 172.21.15.115:0/1272753128 wait complete. 2024-05-09T05:50:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.192+0000 7f6ac1989740 20 client.18453 trim_cache size 0 max 16384 2024-05-09T05:50:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:50:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:50:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 -- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a9c0b2950 msgr2=0x7f6a9c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a9c0b2950 0x7f6a9c0b4e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6aac02aeb0 tx=0x7f6aac005c50 comp rx=0 tx=0).stop 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 -- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab4108b90 msgr2=0x7f6ab419bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab4108b90 0x7f6ab419bb40 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400d740 tx=0x7f6aa400db40 comp rx=0 tx=0).stop 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 -- 172.21.15.115:0/3358863099 shutdown_connections 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6a9c0b2950 0x7f6a9c0b4e60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ab4109030 0x7f6ab419c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ab4108b90 0x7f6ab419bb40 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 --2- 172.21.15.115:0/3358863099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab41032e0 0x7f6ab419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 -- 172.21.15.115:0/3358863099 >> 172.21.15.115:0/3358863099 conn(0x7f6ab40fe390 msgr2=0x7f6ab41055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 -- 172.21.15.115:0/3358863099 shutdown_connections 2024-05-09T05:50:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:37.193+0000 7f6abaad8640 1 -- 172.21.15.115:0/3358863099 wait complete. 2024-05-09T05:50:37.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:37 smithi195 ceph-mon[33844]: pgmap v390: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-09T05:50:37.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:37 smithi115 ceph-mon[30349]: pgmap v390: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-09T05:50:37.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:37 smithi161 ceph-mon[33805]: pgmap v390: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-09T05:50:39.393 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:39 smithi115 ceph-mon[30349]: pgmap v391: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-05-09T05:50:39.393 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:39 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:50:39.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:39 smithi195 ceph-mon[33844]: pgmap v391: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-05-09T05:50:39.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:39 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:50:39.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:39 smithi161 ceph-mon[33805]: pgmap v391: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-05-09T05:50:39.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:39 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:50:41.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:41 smithi195 ceph-mon[33844]: pgmap v392: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-05-09T05:50:41.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:41 smithi115 ceph-mon[30349]: pgmap v392: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-05-09T05:50:41.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:41 smithi161 ceph-mon[33805]: pgmap v392: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-05-09T05:50:42.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:42 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:50:42.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:42 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:50:42.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:42 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:50:42.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:42 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:50:42.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:42 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:50:42.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:42 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:50:42.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:42 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:50:42.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:42 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:50:42.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:42 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:50:43.579 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:43 smithi115 ceph-mon[30349]: pgmap v393: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-05-09T05:50:43.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:43 smithi195 ceph-mon[33844]: pgmap v393: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-05-09T05:50:43.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:43 smithi161 ceph-mon[33805]: pgmap v393: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-05-09T05:50:43.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.701+0000 7f62afb71700 1 -- 172.21.15.115:0/509017147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81089f0 msgr2=0x7f62a8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:43.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.701+0000 7f62afb71700 1 --2- 172.21.15.115:0/509017147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81089f0 0x7f62a8108e00 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009990 tx=0x7f62a4009ca0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.701+0000 7f62afb71700 1 -- 172.21.15.115:0/509017147 shutdown_connections 2024-05-09T05:50:43.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.701+0000 7f62afb71700 1 --2- 172.21.15.115:0/509017147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a8103310 0x7f62a810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.701+0000 7f62afb71700 1 --2- 172.21.15.115:0/509017147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81089f0 0x7f62a8108e00 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.701+0000 7f62afb71700 1 --2- 172.21.15.115:0/509017147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a81029c0 0x7f62a8102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.701+0000 7f62afb71700 1 -- 172.21.15.115:0/509017147 >> 172.21.15.115:0/509017147 conn(0x7f62a80fe540 msgr2=0x7f62a8100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.701+0000 7f62afb71700 1 -- 172.21.15.115:0/509017147 shutdown_connections 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.701+0000 7f62afb71700 1 -- 172.21.15.115:0/509017147 wait complete. 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62afb71700 1 Processor -- start 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62afb71700 1 -- start start 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62afb71700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81029c0 0x7f62a819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62afb71700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a8103310 0x7f62a819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62afb71700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a81089f0 0x7f62a8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62afb71700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8113240 con 0x7f62a81029c0 2024-05-09T05:50:43.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62afb71700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81130c0 con 0x7f62a81089f0 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62afb71700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81133c0 con 0x7f62a8103310 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad90d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81029c0 0x7f62a819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad90d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81029c0 0x7f62a819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52160/0 (socket says 172.21.15.115:52160) 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad90d700 1 -- 172.21.15.115:0/1682031785 learned_addr learned my addr 172.21.15.115:0/1682031785 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ae10e700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a81089f0 0x7f62a8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad10c700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a8103310 0x7f62a819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad90d700 1 -- 172.21.15.115:0/1682031785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a8103310 msgr2=0x7f62a819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad90d700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a8103310 0x7f62a819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad90d700 1 -- 172.21.15.115:0/1682031785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a81089f0 msgr2=0x7f62a8199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad90d700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a81089f0 0x7f62a8199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad90d700 1 -- 172.21.15.115:0/1682031785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a819a130 con 0x7f62a81029c0 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.702+0000 7f62ad10c700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a8103310 0x7f62a819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.703+0000 7f62ad90d700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81029c0 0x7f62a819f860 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f62a4000c00 tx=0x7f62a40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.703+0000 7f629effd700 1 -- 172.21.15.115:0/1682031785 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4027070 con 0x7f62a81029c0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.703+0000 7f629effd700 1 -- 172.21.15.115:0/1682031785 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62a400fb80 con 0x7f62a81029c0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.703+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a819a3c0 con 0x7f62a81029c0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.703+0000 7f629effd700 1 -- 172.21.15.115:0/1682031785 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4022d00 con 0x7f62a81029c0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.703+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a819a8a0 con 0x7f62a81029c0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.704+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a80512c0 con 0x7f62a81029c0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.705+0000 7f629effd700 1 -- 172.21.15.115:0/1682031785 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f62a402a070 con 0x7f62a81029c0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.706+0000 7f629effd700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f62940b29f0 0x7f62940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.706+0000 7f62ad10c700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f62940b29f0 0x7f62940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.706+0000 7f629effd700 1 -- 172.21.15.115:0/1682031785 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f62a40e38f0 con 0x7f62a81029c0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.706+0000 7f62ad10c700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f62940b29f0 0x7f62940b4f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6298005950 tx=0x7f62980058e0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.711+0000 7f629effd700 1 -- 172.21.15.115:0/1682031785 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62a4013070 con 0x7f62a81029c0 2024-05-09T05:50:43.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.970+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62a802e5f0 con 0x7f62a81029c0 2024-05-09T05:50:43.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.971+0000 7f629effd700 1 -- 172.21.15.115:0/1682031785 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+11956 (secure 0 0 0) 0x7f62a40aafd0 con 0x7f62a81029c0 2024-05-09T05:50:43.961 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:50:43.962 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:50:23.370056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:50:43.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f62940b29f0 msgr2=0x7f62940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:43.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f62940b29f0 0x7f62940b4f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6298005950 tx=0x7f62980058e0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81029c0 msgr2=0x7f62a819f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:43.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81029c0 0x7f62a819f860 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f62a4000c00 tx=0x7f62a40047b0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 shutdown_connections 2024-05-09T05:50:43.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f62940b29f0 0x7f62940b4f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a81089f0 0x7f62a8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62a81029c0 0x7f62a819f860 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 --2- 172.21.15.115:0/1682031785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62a8103310 0x7f62a819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:43.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 >> 172.21.15.115:0/1682031785 conn(0x7f62a80fe540 msgr2=0x7f62a8100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:43.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 shutdown_connections 2024-05-09T05:50:43.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:43.973+0000 7f62afb71700 1 -- 172.21.15.115:0/1682031785 wait complete. 2024-05-09T05:50:43.964 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 56 2024-05-09T05:50:44.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:44 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1682031785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:50:44.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:44 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1682031785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:50:44.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:44 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1682031785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:50:44.833 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id c137a69d --release --await' 2024-05-09T05:50:44.833 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id c137a69d --release --await 2024-05-09T05:50:45.078 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:50:45.467 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:45 smithi115 ceph-mon[30349]: pgmap v394: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-05-09T05:50:45.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:45 smithi195 ceph-mon[33844]: pgmap v394: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-05-09T05:50:45.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:45 smithi161 ceph-mon[33805]: pgmap v394: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-05-09T05:50:46.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.918+0000 7fb008c57700 1 -- 172.21.15.115:0/1768306505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004107e50 msgr2=0x7fb004108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:46.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.918+0000 7fb008c57700 1 --2- 172.21.15.115:0/1768306505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004107e50 0x7fb004108260 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7faff80099c0 tx=0x7faff8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.918+0000 7fb008c57700 1 -- 172.21.15.115:0/1768306505 shutdown_connections 2024-05-09T05:50:46.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.918+0000 7fb008c57700 1 --2- 172.21.15.115:0/1768306505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb004103b30 0x7fb004103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.918+0000 7fb008c57700 1 --2- 172.21.15.115:0/1768306505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0041087a0 0x7fb004110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.918+0000 7fb008c57700 1 --2- 172.21.15.115:0/1768306505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004107e50 0x7fb004108260 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.918+0000 7fb008c57700 1 -- 172.21.15.115:0/1768306505 >> 172.21.15.115:0/1768306505 conn(0x7fb0040fe680 msgr2=0x7fb004100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:46.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.918+0000 7fb008c57700 1 -- 172.21.15.115:0/1768306505 shutdown_connections 2024-05-09T05:50:46.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.918+0000 7fb008c57700 1 -- 172.21.15.115:0/1768306505 wait complete. 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb008c57700 1 Processor -- start 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb008c57700 1 -- start start 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb008c57700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004103b30 0x7fb00419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb008c57700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb004107e50 0x7fb00419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb008c57700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0041087a0 0x7fb004199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb008c57700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb004113290 con 0x7fb004103b30 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb008c57700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb004113110 con 0x7fb004107e50 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb008c57700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb004113410 con 0x7fb0041087a0 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb00259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004103b30 0x7fb00419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb00259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004103b30 0x7fb00419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52166/0 (socket says 172.21.15.115:52166) 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.919+0000 7fb00259c700 1 -- 172.21.15.115:0/4067430484 learned_addr learned my addr 172.21.15.115:0/4067430484 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb002d9d700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0041087a0 0x7fb004199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb001d9b700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb004107e50 0x7fb00419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb00259c700 1 -- 172.21.15.115:0/4067430484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0041087a0 msgr2=0x7fb004199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb00259c700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0041087a0 0x7fb004199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb00259c700 1 -- 172.21.15.115:0/4067430484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb004107e50 msgr2=0x7fb00419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb00259c700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb004107e50 0x7fb00419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb00259c700 1 -- 172.21.15.115:0/4067430484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00419a0f0 con 0x7fb004103b30 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb00259c700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004103b30 0x7fb00419f890 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7faff400c9b0 tx=0x7faff400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7faff37fe700 1 -- 172.21.15.115:0/4067430484 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4007b00 con 0x7fb004103b30 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7faff37fe700 1 -- 172.21.15.115:0/4067430484 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faff4007ca0 con 0x7fb004103b30 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb008c57700 1 -- 172.21.15.115:0/4067430484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00419a3e0 con 0x7fb004103b30 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7faff37fe700 1 -- 172.21.15.115:0/4067430484 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4018660 con 0x7fb004103b30 2024-05-09T05:50:46.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.920+0000 7fb008c57700 1 -- 172.21.15.115:0/4067430484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0041a6720 con 0x7fb004103b30 2024-05-09T05:50:46.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.923+0000 7faff37fe700 1 -- 172.21.15.115:0/4067430484 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7faff400f450 con 0x7fb004103b30 2024-05-09T05:50:46.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.923+0000 7faff37fe700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafec0b2a70 0x7fafec0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.923+0000 7fb001d9b700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafec0b2a70 0x7fafec0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7faff37fe700 1 -- 172.21.15.115:0/4067430484 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7faff40e29f0 con 0x7fb004103b30 2024-05-09T05:50:46.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb001d9b700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafec0b2a70 0x7fafec0b4f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faff8009990 tx=0x7faff8005940 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:46.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb01766db80 1 Processor -- start 2024-05-09T05:50:46.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb01766db80 1 -- start start 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb01766db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d32fe00 0x562f5d431c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb01766db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d432190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb01766db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 0x562f5d43ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb01766db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562f5d3551a0 con 0x562f5d32fe00 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb01766db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562f5d2faaa0 con 0x562f5d2b55a0 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb01766db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562f5d351160 con 0x562f5d43c900 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb00259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d32fe00 0x562f5d431c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb00259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d32fe00 0x562f5d431c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52172/0 (socket says 172.21.15.115:52172) 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb00259c700 1 -- 172.21.15.115:0/2547360011 learned_addr learned my addr 172.21.15.115:0/2547360011 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.924+0000 7fb001d9b700 1 --2- 172.21.15.115:0/2547360011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d432190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7fb002d9d700 1 --2- 172.21.15.115:0/2547360011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 0x562f5d43ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7fb00259c700 1 -- 172.21.15.115:0/2547360011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 msgr2=0x562f5d43ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7fb00259c700 1 --2- 172.21.15.115:0/2547360011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 0x562f5d43ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7fb00259c700 1 -- 172.21.15.115:0/2547360011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 msgr2=0x562f5d432190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:46.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7fb00259c700 1 --2- 172.21.15.115:0/2547360011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d432190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7fb00259c700 1 -- 172.21.15.115:0/2547360011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5d2eb600 con 0x562f5d32fe00 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7fb00259c700 1 --2- 172.21.15.115:0/2547360011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d32fe00 0x562f5d431c50 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7faff40a98d0 tx=0x7faff40a9520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8719c2d93c29b3d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7faff0ff9700 1 -- 172.21.15.115:0/2547360011 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff408fe60 con 0x562f5d32fe00 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7faff0ff9700 1 -- 172.21.15.115:0/2547360011 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faff40adcf0 con 0x562f5d32fe00 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.925+0000 7faff0ff9700 1 -- 172.21.15.115:0/2547360011 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff40cc0d0 con 0x562f5d32fe00 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 -- 172.21.15.115:0/2547360011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d32fe00 msgr2=0x562f5d431c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 --2- 172.21.15.115:0/2547360011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d32fe00 0x562f5d431c50 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7faff40a98d0 tx=0x7faff40a9520 comp rx=0 tx=0).stop 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 -- 172.21.15.115:0/2547360011 shutdown_connections 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 --2- 172.21.15.115:0/2547360011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 0x562f5d43ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 --2- 172.21.15.115:0/2547360011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d32fe00 0x562f5d431c50 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 --2- 172.21.15.115:0/2547360011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d432190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 -- 172.21.15.115:0/2547360011 >> 172.21.15.115:0/2547360011 conn(0x562f5d2b9070 msgr2=0x562f5d42db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:46.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 -- 172.21.15.115:0/2547360011 shutdown_connections 2024-05-09T05:50:46.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 -- 172.21.15.115:0/2547360011 wait complete. 2024-05-09T05:50:46.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 -- start start 2024-05-09T05:50:46.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.926+0000 7fb01766db80 1 Processor -- start 2024-05-09T05:50:46.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb01766db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d439a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb01766db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 0x562f5d439f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb01766db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d43a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb01766db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562f5d4414d0 con 0x562f5d43c900 2024-05-09T05:50:46.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb01766db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562f5d2efc20 con 0x562f5d2b55a0 2024-05-09T05:50:46.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb01766db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562f5d441650 con 0x562f5d32fe00 2024-05-09T05:50:46.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb01766db80 10 client.0 _ll_register_callbacks cb 0x562f5d1c67c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d43a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d43a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52180/0 (socket says 172.21.15.115:52180) 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb002d9d700 1 -- 172.21.15.115:0/3273449168 learned_addr learned my addr 172.21.15.115:0/3273449168 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb001d9b700 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 0x562f5d439f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb00259c700 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d439a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb002d9d700 1 -- 172.21.15.115:0/3273449168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 msgr2=0x562f5d439f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb002d9d700 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 0x562f5d439f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb002d9d700 1 -- 172.21.15.115:0/3273449168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 msgr2=0x562f5d439a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb002d9d700 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d439a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb002d9d700 1 -- 172.21.15.115:0/3273449168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5d1d0590 con 0x562f5d43c900 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fb002d9d700 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d43a4d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7faffc011d00 tx=0x7faffc00f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193832b80c332cb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.927+0000 7fafe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:46.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3273449168 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc019070 con 0x562f5d43c900 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3273449168 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7faffc00fa10 con 0x562f5d43c900 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb01766db80 1 -- 172.21.15.115:0/3273449168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faff40adcf0 con 0x562f5d43c900 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3273449168 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7faffc025050 con 0x562f5d43c900 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb01766db80 10 client.18471 fetch_fsmap learned FSMap version 56 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb01766db80 1 -- 172.21.15.115:0/3273449168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x562f5d44c2f0 con 0x562f5d43c900 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3273449168 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faff40adcf0 con 0x562f5d43c900 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3273449168 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7faffc015c60 con 0x562f5d43c900 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb01766db80 10 client.18471 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb01766db80 10 client.18471 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb01766db80 20 client.18471 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb01766db80 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562f5d44e880 0x562f5d450cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb01766db80 4 client.18471 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:50:46.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb01766db80 1 -- 172.21.15.115:0/3273449168 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f5d362b80 con 0x562f5d44e880 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.928+0000 7fb00259c700 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562f5d44e880 0x562f5d450cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.929+0000 7fafe9ffb700 10 client.18471 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.929+0000 7fb00259c700 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562f5d44e880 0x562f5d450cc0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8cfdffdebb917e5 server_cookie=fe0cfae594280f37 in_seq=0 out_seq=0 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3273449168 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562f5d362b80 con 0x562f5d44e880 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fafe9ffb700 10 client.18471 handle_command_reply: tid=0 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 client.18471 shutdown 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 -- 172.21.15.115:0/3273449168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 msgr2=0x562f5d43a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d43a4d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7faffc011d00 tx=0x7faffc00f4b0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 -- 172.21.15.115:0/3273449168 shutdown_connections 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d439a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562f5d44e880 0x562f5d450cc0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d43a4d0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 --2- 172.21.15.115:0/3273449168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 0x562f5d439f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 -- 172.21.15.115:0/3273449168 >> 172.21.15.115:0/3273449168 conn(0x562f5d2b9070 msgr2=0x562f5d42cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:46.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fafe9ffb700 0 client.18471 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:46.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 -- 172.21.15.115:0/3273449168 shutdown_connections 2024-05-09T05:50:46.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 1 -- 172.21.15.115:0/3273449168 wait complete. 2024-05-09T05:50:46.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.930+0000 7fb01766db80 20 client.18471 trim_cache size 0 max 16384 2024-05-09T05:50:46.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.949+0000 7fb01766db80 1 Processor -- start 2024-05-09T05:50:46.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.949+0000 7fb01766db80 1 -- start start 2024-05-09T05:50:46.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb01766db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d456480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb01766db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 0x562f5d4569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb01766db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d4468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb01766db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041ae1f0 con 0x562f5d43c900 2024-05-09T05:50:46.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb01766db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041afdc0 con 0x562f5d2b55a0 2024-05-09T05:50:46.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb01766db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041ae8c0 con 0x562f5d32fe00 2024-05-09T05:50:46.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d4468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d4468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52194/0 (socket says 172.21.15.115:52194) 2024-05-09T05:50:46.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb002d9d700 1 -- 172.21.15.115:0/2166569381 learned_addr learned my addr 172.21.15.115:0/2166569381 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb001d9b700 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 0x562f5d4569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb00259c700 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d456480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb002d9d700 1 -- 172.21.15.115:0/2166569381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 msgr2=0x562f5d4569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb002d9d700 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 0x562f5d4569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb002d9d700 1 -- 172.21.15.115:0/2166569381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 msgr2=0x562f5d456480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb002d9d700 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d456480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb002d9d700 1 -- 172.21.15.115:0/2166569381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5d438d70 con 0x562f5d43c900 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.950+0000 7fb001d9b700 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 0x562f5d4569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.951+0000 7fb002d9d700 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d4468d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7faffc00f880 tx=0x7faffc00f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab432248dbfdb27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.951+0000 7fafebfff700 1 -- 172.21.15.115:0/2166569381 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc019070 con 0x562f5d43c900 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.951+0000 7fafebfff700 1 -- 172.21.15.115:0/2166569381 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faffc022070 con 0x562f5d43c900 2024-05-09T05:50:46.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.951+0000 7fafebfff700 1 -- 172.21.15.115:0/2166569381 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc003e30 con 0x562f5d43c900 2024-05-09T05:50:46.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 -- 172.21.15.115:0/2166569381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 msgr2=0x562f5d4468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:46.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d4468d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7faffc00f880 tx=0x7faffc00f8b0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 -- 172.21.15.115:0/2166569381 shutdown_connections 2024-05-09T05:50:46.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d2b55a0 0x562f5d456480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d43c900 0x562f5d4468d0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 --2- 172.21.15.115:0/2166569381 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d32fe00 0x562f5d4569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 -- 172.21.15.115:0/2166569381 >> 172.21.15.115:0/2166569381 conn(0x562f5d2b9070 msgr2=0x562f5d42cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 -- 172.21.15.115:0/2166569381 shutdown_connections 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 -- 172.21.15.115:0/2166569381 wait complete. 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 -- start start 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 Processor -- start 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d2b55a0 0x562f5d442b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d32fe00 0x562f5d443050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 0x562f5d4b8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041af6f0 con 0x562f5d2b55a0 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041af020 con 0x562f5d32fe00 2024-05-09T05:50:46.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.952+0000 7fb01766db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041ac1b0 con 0x562f5d43c900 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb01766db80 10 client.0 _ll_register_callbacks cb 0x562f5d1c67c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb00259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d2b55a0 0x562f5d442b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb00259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d2b55a0 0x562f5d442b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52208/0 (socket says 172.21.15.115:52208) 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb00259c700 1 -- 172.21.15.115:0/3553584583 learned_addr learned my addr 172.21.15.115:0/3553584583 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb002d9d700 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 0x562f5d4b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb001d9b700 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d32fe00 0x562f5d443050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb00259c700 1 -- 172.21.15.115:0/3553584583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 msgr2=0x562f5d4b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb00259c700 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 0x562f5d4b8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb00259c700 1 -- 172.21.15.115:0/3553584583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d32fe00 msgr2=0x562f5d443050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb00259c700 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d32fe00 0x562f5d443050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb00259c700 1 -- 172.21.15.115:0/3553584583 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f5d4b8570 con 0x562f5d2b55a0 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb001d9b700 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d32fe00 0x562f5d443050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb00259c700 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d2b55a0 0x562f5d442b10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7faff4000c00 tx=0x7faff4018800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e733ca4c55811d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fafe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:46.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3553584583 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff408fe60 con 0x562f5d2b55a0 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3553584583 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7faff40a5bd0 con 0x562f5d2b55a0 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.953+0000 7fb01766db80 1 -- 172.21.15.115:0/3553584583 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faffc022070 con 0x562f5d2b55a0 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3553584583 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7faff40adcf0 con 0x562f5d2b55a0 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fb01766db80 10 client.18477 fetch_fsmap learned FSMap version 56 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fb01766db80 1 -- 172.21.15.115:0/3553584583 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x562f5d4b96e0 con 0x562f5d2b55a0 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3553584583 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faffc022070 con 0x562f5d2b55a0 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3553584583 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7faff40b2310 con 0x562f5d2b55a0 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fb01766db80 10 client.18477 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fb01766db80 10 client.18477 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fb01766db80 20 client.18477 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fb01766db80 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562f5d4b9970 0x562f5d4bbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fb01766db80 4 client.18477 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c137a69d", "release": true, "await": true}] 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fb01766db80 1 -- 172.21.15.115:0/3553584583 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c137a69d", "release": true, "await": true}) -- 0x562f5d35cc20 con 0x562f5d4b9970 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.954+0000 7fb001d9b700 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562f5d4b9970 0x562f5d4bbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:46.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.955+0000 7fb001d9b700 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562f5d4b9970 0x562f5d4bbd90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9880fa9315840f77 server_cookie=2662d694ac60af19 in_seq=0 out_seq=0 2024-05-09T05:50:46.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.955+0000 7fafe9ffb700 10 client.18477 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fafe9ffb700 1 -- 172.21.15.115:0/3553584583 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562f5d35cc20 con 0x562f5d4b9970 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fafe9ffb700 10 client.18477 handle_command_reply: tid=0 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 client.18477 shutdown 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 -- 172.21.15.115:0/3553584583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d2b55a0 msgr2=0x562f5d442b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d2b55a0 0x562f5d442b10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7faff4000c00 tx=0x7faff4018800 comp rx=0 tx=0).stop 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 -- 172.21.15.115:0/3553584583 shutdown_connections 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562f5d4b9970 0x562f5d4bbd90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562f5d43c900 0x562f5d4b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f5d2b55a0 0x562f5d442b10 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fafe9ffb700 0 client.18477 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 --2- 172.21.15.115:0/3553584583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f5d32fe00 0x562f5d443050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 -- 172.21.15.115:0/3553584583 >> 172.21.15.115:0/3553584583 conn(0x562f5d2b9070 msgr2=0x562f5d42d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.958+0000 7fb01766db80 1 -- 172.21.15.115:0/3553584583 shutdown_connections 2024-05-09T05:50:46.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb01766db80 1 -- 172.21.15.115:0/3553584583 wait complete. 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb01766db80 20 client.18477 trim_cache size 0 max 16384 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 56, 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 36, 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout: "c137a69d": { 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 36, 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:50:46.951 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 130.0, 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:46.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:50:46.953 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:50:46.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 -- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafec0b2a70 msgr2=0x7fafec0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:46.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafec0b2a70 0x7fafec0b4f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faff8009990 tx=0x7faff8005940 comp rx=0 tx=0).stop 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 -- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004103b30 msgr2=0x7fb00419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004103b30 0x7fb00419f890 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7faff400c9b0 tx=0x7faff400cd70 comp rx=0 tx=0).stop 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 -- 172.21.15.115:0/4067430484 shutdown_connections 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafec0b2a70 0x7fafec0b4f80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0041087a0 0x7fb004199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb004103b30 0x7fb00419f890 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 --2- 172.21.15.115:0/4067430484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb004107e50 0x7fb00419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 -- 172.21.15.115:0/4067430484 >> 172.21.15.115:0/4067430484 conn(0x7fb0040fe680 msgr2=0x7fb0041096d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 -- 172.21.15.115:0/4067430484 shutdown_connections 2024-05-09T05:50:46.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:50:46.959+0000 7fb008c57700 1 -- 172.21.15.115:0/4067430484 wait complete. 2024-05-09T05:50:47.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:47 smithi195 ceph-mon[33844]: pgmap v395: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 854 KiB/s wr, 11 op/s 2024-05-09T05:50:47.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:47 smithi115 ceph-mon[30349]: pgmap v395: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 854 KiB/s wr, 11 op/s 2024-05-09T05:50:47.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:47 smithi161 ceph-mon[33805]: pgmap v395: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 854 KiB/s wr, 11 op/s 2024-05-09T05:50:47.807 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c137a69d', total seconds elapsed: 22.0 2024-05-09T05:50:49.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:49 smithi195 ceph-mon[33844]: pgmap v396: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.7 MiB/s wr, 17 op/s 2024-05-09T05:50:49.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:49 smithi115 ceph-mon[30349]: pgmap v396: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.7 MiB/s wr, 17 op/s 2024-05-09T05:50:49.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:49 smithi161 ceph-mon[33805]: pgmap v396: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.7 MiB/s wr, 17 op/s 2024-05-09T05:50:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.307+0000 7fdf2fd34640 1 -- 172.21.15.115:0/4175440618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28106b10 msgr2=0x7fdf28106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.307+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/4175440618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28106b10 0x7fdf28106f40 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009900 tx=0x7fdf1802a410 comp rx=0 tx=0).stop 2024-05-09T05:50:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.307+0000 7fdf2fd34640 1 -- 172.21.15.115:0/4175440618 shutdown_connections 2024-05-09T05:50:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.307+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/4175440618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf281075d0 0x7fdf2810ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.307+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/4175440618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28106b10 0x7fdf28106f40 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.307+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/4175440618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf28102580 0x7fdf28102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.307+0000 7fdf2fd34640 1 -- 172.21.15.115:0/4175440618 >> 172.21.15.115:0/4175440618 conn(0x7fdf280779e0 msgr2=0x7fdf28077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.307+0000 7fdf2fd34640 1 -- 172.21.15.115:0/4175440618 shutdown_connections 2024-05-09T05:50:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.307+0000 7fdf2fd34640 1 -- 172.21.15.115:0/4175440618 wait complete. 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2fd34640 1 Processor -- start 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2fd34640 1 -- start start 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2fd34640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28102580 0x7fdf28199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2fd34640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf28106b10 0x7fdf28199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2fd34640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf281075d0 0x7fdf2819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2fd34640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28112360 con 0x7fdf28102580 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2fd34640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281121e0 con 0x7fdf281075d0 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2fd34640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281124e0 con 0x7fdf28106b10 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2dad0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28102580 0x7fdf28199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.308+0000 7fdf2dad0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28102580 0x7fdf28199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52232/0 (socket says 172.21.15.115:52232) 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf2dad0640 1 -- 172.21.15.115:0/3537921142 learned_addr learned my addr 172.21.15.115:0/3537921142 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf28106b10 0x7fdf28199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf2dad0640 1 -- 172.21.15.115:0/3537921142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf28106b10 msgr2=0x7fdf28199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf2dad0640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf28106b10 0x7fdf28199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf2dad0640 1 -- 172.21.15.115:0/3537921142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf281075d0 msgr2=0x7fdf2819df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf2dad0640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf281075d0 0x7fdf2819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf2dad0640 1 -- 172.21.15.115:0/3537921142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2819e570 con 0x7fdf28102580 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf2dad0640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28102580 0x7fdf28199710 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400e9f0 tx=0x7fdf2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf1effd640 1 -- 172.21.15.115:0/3537921142 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400cd80 con 0x7fdf28102580 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf1effd640 1 -- 172.21.15.115:0/3537921142 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf240108c0 con 0x7fdf28102580 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.309+0000 7fdf1effd640 1 -- 172.21.15.115:0/3537921142 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf240187c0 con 0x7fdf28102580 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.310+0000 7fdf2fd34640 1 -- 172.21.15.115:0/3537921142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2819e860 con 0x7fdf28102580 2024-05-09T05:50:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.310+0000 7fdf2fd34640 1 -- 172.21.15.115:0/3537921142 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2819ed30 con 0x7fdf28102580 2024-05-09T05:50:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.313+0000 7fdf36be8740 1 Processor -- start 2024-05-09T05:50:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.313+0000 7fdf36be8740 1 -- start start 2024-05-09T05:50:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc9f1b0 0x56151c042680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc762f0 0x56151c042bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf1effd640 1 -- 172.21.15.115:0/3537921142 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fdf24010430 con 0x7fdf28102580 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 0x56151c043100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf36be8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56151bfa77c0 con 0x56151bc9f1b0 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf36be8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56151bf430e0 con 0x56151bc7e4d0 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf36be8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56151bfa3780 con 0x56151bc762f0 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2e2d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc9f1b0 0x56151c042680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2e2d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc9f1b0 0x56151c042680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52248/0 (socket says 172.21.15.115:52248) 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/1085248350 learned_addr learned my addr 172.21.15.115:0/1085248350 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/1085248350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 0x56151c043100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2dad0640 1 --2- 172.21.15.115:0/1085248350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc762f0 0x56151c042bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/1085248350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc762f0 msgr2=0x56151c042bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/1085248350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc762f0 0x56151c042bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/1085248350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 msgr2=0x56151c043100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/1085248350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 0x56151c043100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.314+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/1085248350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151bf6c7c0 con 0x56151bc9f1b0 2024-05-09T05:50:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.315+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/1085248350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc9f1b0 0x56151c042680 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009d10 tx=0x7fdf2000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70bf0bce5a80cc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.315+0000 7fdf13fff640 1 -- 172.21.15.115:0/1085248350 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20016070 con 0x56151bc9f1b0 2024-05-09T05:50:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.315+0000 7fdf13fff640 1 -- 172.21.15.115:0/1085248350 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf20005020 con 0x56151bc9f1b0 2024-05-09T05:50:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.315+0000 7fdf13fff640 1 -- 172.21.15.115:0/1085248350 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf200116d0 con 0x56151bc9f1b0 2024-05-09T05:50:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.315+0000 7fdf1effd640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fdf140b29d0 0x7fdf140b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.315+0000 7fdf2dad0640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fdf140b29d0 0x7fdf140b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.315+0000 7fdf1effd640 1 -- 172.21.15.115:0/3537921142 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fdf240e2d60 con 0x7fdf28102580 2024-05-09T05:50:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf2dad0640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fdf140b29d0 0x7fdf140b4ee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdf240e01d0 tx=0x7fdf240e68b0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf36be8740 1 -- 172.21.15.115:0/1085248350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc9f1b0 msgr2=0x56151c042680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf36be8740 1 --2- 172.21.15.115:0/1085248350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc9f1b0 0x56151c042680 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009d10 tx=0x7fdf2000e6c0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf36be8740 1 -- 172.21.15.115:0/1085248350 shutdown_connections 2024-05-09T05:50:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf36be8740 1 --2- 172.21.15.115:0/1085248350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 0x56151c043100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf36be8740 1 --2- 172.21.15.115:0/1085248350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc762f0 0x56151c042bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf36be8740 1 --2- 172.21.15.115:0/1085248350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc9f1b0 0x56151c042680 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf36be8740 1 -- 172.21.15.115:0/1085248350 >> 172.21.15.115:0/1085248350 conn(0x56151bc93300 msgr2=0x56151c03ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf36be8740 1 -- 172.21.15.115:0/1085248350 shutdown_connections 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.316+0000 7fdf36be8740 1 -- 172.21.15.115:0/1085248350 wait complete. 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.317+0000 7fdf36be8740 1 -- start start 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.317+0000 7fdf36be8740 1 Processor -- start 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.317+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.317+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc7e4d0 0x56151c04ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.317+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc9f1b0 0x56151c04cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.317+0000 7fdf36be8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56151c04f7b0 con 0x56151bc762f0 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.317+0000 7fdf36be8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56151bf2e9c0 con 0x56151bc9f1b0 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.317+0000 7fdf36be8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56151c04f930 con 0x56151bc7e4d0 2024-05-09T05:50:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52258/0 (socket says 172.21.15.115:52258) 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/3451704091 learned_addr learned my addr 172.21.15.115:0/3451704091 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf36be8740 10 client.0 _ll_register_callbacks cb 0x56151bf5b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc7e4d0 0x56151c04ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc9f1b0 0x56151c04cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/3451704091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc7e4d0 msgr2=0x56151c04ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc7e4d0 0x56151c04ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/3451704091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc9f1b0 msgr2=0x56151c04cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc9f1b0 0x56151c04cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/3451704091 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151bf551f0 con 0x56151bc762f0 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04e770 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005b40 tx=0x7fdf200059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb25aa9f5f35bdc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf117fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.318+0000 7fdf117fa640 1 -- 172.21.15.115:0/3451704091 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20016070 con 0x56151bc762f0 2024-05-09T05:50:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf36be8740 1 -- 172.21.15.115:0/3451704091 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf20005020 con 0x56151bc762f0 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf117fa640 1 -- 172.21.15.115:0/3451704091 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fdf200053d0 con 0x56151bc762f0 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf117fa640 1 -- 172.21.15.115:0/3451704091 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fdf20031050 con 0x56151bc762f0 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf36be8740 10 client.18489 fetch_fsmap learned FSMap version 56 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf36be8740 1 -- 172.21.15.115:0/3451704091 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56151bd0a790 con 0x56151bc762f0 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf117fa640 1 -- 172.21.15.115:0/3451704091 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf20005020 con 0x56151bc762f0 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf117fa640 1 -- 172.21.15.115:0/3451704091 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7fdf2000f040 con 0x56151bc762f0 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf36be8740 10 client.18489 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf36be8740 10 client.18489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf36be8740 20 client.18489 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56151bc5dd80 0x56151c05ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf36be8740 4 client.18489 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.319+0000 7fdf36be8740 1 -- 172.21.15.115:0/3451704091 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56151bfb53d0 con 0x56151bc5dd80 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.320+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56151bc5dd80 0x56151c05ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.320+0000 7fdf117fa640 10 client.18489 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.320+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56151bc5dd80 0x56151c05ee80 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15d687d317b81f7 server_cookie=3d40cce1dd96320 in_seq=0 out_seq=0 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf117fa640 1 -- 172.21.15.115:0/3451704091 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56151bfb53d0 con 0x56151bc5dd80 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf117fa640 10 client.18489 handle_command_reply: tid=0 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 client.18489 shutdown 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 -- 172.21.15.115:0/3451704091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 msgr2=0x56151c04e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04e770 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005b40 tx=0x7fdf200059f0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 -- 172.21.15.115:0/3451704091 shutdown_connections 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc9f1b0 0x56151c04cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56151bc5dd80 0x56151c05ee80 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc7e4d0 0x56151c04ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf117fa640 0 client.18489 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3451704091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04e770 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 -- 172.21.15.115:0/3451704091 >> 172.21.15.115:0/3451704091 conn(0x56151bc93300 msgr2=0x56151c03d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 -- 172.21.15.115:0/3451704091 shutdown_connections 2024-05-09T05:50:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 1 -- 172.21.15.115:0/3451704091 wait complete. 2024-05-09T05:50:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.321+0000 7fdf36be8740 20 client.18489 trim_cache size 0 max 16384 2024-05-09T05:50:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf36be8740 1 Processor -- start 2024-05-09T05:50:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf36be8740 1 -- start start 2024-05-09T05:50:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 0x56151c054bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 0x56151c055100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf36be8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281b34a0 con 0x56151bc762f0 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf36be8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281b5070 con 0x56151bc7e4d0 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf36be8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281b3b70 con 0x56151bc9f1b0 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf2e2d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf2e2d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52272/0 (socket says 172.21.15.115:52272) 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.344+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/1052934315 learned_addr learned my addr 172.21.15.115:0/1052934315 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 0x56151c054bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 0x56151c055100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/1052934315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 msgr2=0x56151c055100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 0x56151c055100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/1052934315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 msgr2=0x56151c054bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 0x56151c054bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf2e2d1640 1 -- 172.21.15.115:0/1052934315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151bd815f0 con 0x56151bc762f0 2024-05-09T05:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 0x56151c054bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04cb90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009790 tx=0x7fdf20003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38bcdae30ace7c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf137fe640 1 -- 172.21.15.115:0/1052934315 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20016070 con 0x56151bc762f0 2024-05-09T05:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf137fe640 1 -- 172.21.15.115:0/1052934315 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf2001e050 con 0x56151bc762f0 2024-05-09T05:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.345+0000 7fdf137fe640 1 -- 172.21.15.115:0/1052934315 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2002e040 con 0x56151bc762f0 2024-05-09T05:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 -- 172.21.15.115:0/1052934315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 msgr2=0x56151c04cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04cb90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009790 tx=0x7fdf20003b80 comp rx=0 tx=0).stop 2024-05-09T05:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 -- 172.21.15.115:0/1052934315 shutdown_connections 2024-05-09T05:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 0x56151c055100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc7e4d0 0x56151c054bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 --2- 172.21.15.115:0/1052934315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc762f0 0x56151c04cb90 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 -- 172.21.15.115:0/1052934315 >> 172.21.15.115:0/1052934315 conn(0x56151bc93300 msgr2=0x56151c04c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 -- 172.21.15.115:0/1052934315 shutdown_connections 2024-05-09T05:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 -- 172.21.15.115:0/1052934315 wait complete. 2024-05-09T05:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.346+0000 7fdf36be8740 1 -- start start 2024-05-09T05:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf36be8740 1 Processor -- start 2024-05-09T05:50:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc762f0 0x56151c054a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc7e4d0 0x56151c054fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf36be8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 0x56151c051f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf36be8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281b49a0 con 0x56151bc7e4d0 2024-05-09T05:50:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf36be8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281b42d0 con 0x56151bc762f0 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf36be8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281b14c0 con 0x56151bc9f1b0 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf36be8740 10 client.0 _ll_register_callbacks cb 0x7fdf18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf2d2cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc7e4d0 0x56151c054fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf2d2cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc7e4d0 0x56151c054fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52280/0 (socket says 172.21.15.115:52280) 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf2d2cf640 1 -- 172.21.15.115:0/3870086965 learned_addr learned my addr 172.21.15.115:0/3870086965 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 0x56151c051f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.347+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc762f0 0x56151c054a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf2d2cf640 1 -- 172.21.15.115:0/3870086965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 msgr2=0x56151c051f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 0x56151c051f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf2d2cf640 1 -- 172.21.15.115:0/3870086965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc762f0 msgr2=0x56151c054a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc762f0 0x56151c054a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf2d2cf640 1 -- 172.21.15.115:0/3870086965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151bf6bed0 con 0x56151bc7e4d0 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf2d2cf640 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc7e4d0 0x56151c054fc0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fdf280755d0 tx=0x7fdf180042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15b2347676992e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf117fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:50:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf117fa640 1 -- 172.21.15.115:0/3870086965 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18035040 con 0x56151bc7e4d0 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf117fa640 1 -- 172.21.15.115:0/3870086965 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fdf18031600 con 0x56151bc7e4d0 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf36be8740 1 -- 172.21.15.115:0/3870086965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf2001e050 con 0x56151bc7e4d0 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf117fa640 1 -- 172.21.15.115:0/3870086965 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fdf18057050 con 0x56151bc7e4d0 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf36be8740 10 client.18495 fetch_fsmap learned FSMap version 56 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf36be8740 1 -- 172.21.15.115:0/3870086965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56151c0c67e0 con 0x56151bc7e4d0 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.348+0000 7fdf117fa640 1 -- 172.21.15.115:0/3870086965 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf2001e050 con 0x56151bc7e4d0 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf117fa640 1 -- 172.21.15.115:0/3870086965 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7fdf18003710 con 0x56151bc7e4d0 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf36be8740 10 client.18495 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf36be8740 10 client.18495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf36be8740 20 client.18495 populate_metadata read hostname 'smithi115' 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56151bc331c0 0x56151c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf36be8740 4 client.18495 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf36be8740 1 -- 172.21.15.115:0/3870086965 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56151bfaf240 con 0x56151bc331c0 2024-05-09T05:50:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56151bc331c0 0x56151c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:50:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf117fa640 10 client.18495 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.349+0000 7fdf2e2d1640 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56151bc331c0 0x56151c0c8fd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1acd970ef3baf63 server_cookie=11dba0fa8cdd0be1 in_seq=0 out_seq=0 2024-05-09T05:50:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf117fa640 1 -- 172.21.15.115:0/3870086965 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56151bfaf240 con 0x56151bc331c0 2024-05-09T05:50:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf117fa640 10 client.18495 handle_command_reply: tid=0 2024-05-09T05:50:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 client.18495 shutdown 2024-05-09T05:50:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 -- 172.21.15.115:0/3870086965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc7e4d0 msgr2=0x56151c054fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc7e4d0 0x56151c054fc0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fdf280755d0 tx=0x7fdf180042e0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 -- 172.21.15.115:0/3870086965 shutdown_connections 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56151bc331c0 0x56151c0c8fd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56151bc9f1b0 0x56151c051f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf117fa640 0 client.18495 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56151bc7e4d0 0x56151c054fc0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 --2- 172.21.15.115:0/3870086965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56151bc762f0 0x56151c054a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 -- 172.21.15.115:0/3870086965 >> 172.21.15.115:0/3870086965 conn(0x56151bc93300 msgr2=0x56151c03d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 -- 172.21.15.115:0/3870086965 shutdown_connections 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 1 -- 172.21.15.115:0/3870086965 wait complete. 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.357+0000 7fdf36be8740 20 client.18495 trim_cache size 0 max 16384 2024-05-09T05:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 -- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fdf140b29d0 msgr2=0x7fdf140b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fdf140b29d0 0x7fdf140b4ee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdf240e01d0 tx=0x7fdf240e68b0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 -- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28102580 msgr2=0x7fdf28199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28102580 0x7fdf28199710 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400e9f0 tx=0x7fdf2400eeb0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 -- 172.21.15.115:0/3537921142 shutdown_connections 2024-05-09T05:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fdf140b29d0 0x7fdf140b4ee0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf281075d0 0x7fdf2819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf28106b10 0x7fdf28199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 --2- 172.21.15.115:0/3537921142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf28102580 0x7fdf28199710 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 -- 172.21.15.115:0/3537921142 >> 172.21.15.115:0/3537921142 conn(0x7fdf280779e0 msgr2=0x7fdf28103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 -- 172.21.15.115:0/3537921142 shutdown_connections 2024-05-09T05:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:50:50.358+0000 7fdf2fd34640 1 -- 172.21.15.115:0/3537921142 wait complete. 2024-05-09T05:50:51.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:51 smithi195 ceph-mon[33844]: pgmap v397: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.3 MiB/s wr, 10 op/s 2024-05-09T05:50:51.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:51 smithi115 ceph-mon[30349]: pgmap v397: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.3 MiB/s wr, 10 op/s 2024-05-09T05:50:51.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:51 smithi161 ceph-mon[33805]: pgmap v397: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.3 MiB/s wr, 10 op/s 2024-05-09T05:50:53.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:53 smithi195 ceph-mon[33844]: pgmap v398: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.3 MiB/s wr, 10 op/s 2024-05-09T05:50:53.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:53 smithi115 ceph-mon[30349]: pgmap v398: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.3 MiB/s wr, 10 op/s 2024-05-09T05:50:53.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:53 smithi161 ceph-mon[33805]: pgmap v398: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.3 MiB/s wr, 10 op/s 2024-05-09T05:50:55.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:55 smithi195 ceph-mon[33844]: pgmap v399: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 20 op/s 2024-05-09T05:50:55.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:55 smithi115 ceph-mon[30349]: pgmap v399: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 20 op/s 2024-05-09T05:50:55.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:55 smithi161 ceph-mon[33805]: pgmap v399: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 20 op/s 2024-05-09T05:50:57.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:57 smithi195 ceph-mon[33844]: pgmap v400: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-09T05:50:57.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:57 smithi115 ceph-mon[30349]: pgmap v400: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-09T05:50:57.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:57 smithi161 ceph-mon[33805]: pgmap v400: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-09T05:50:59.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:50:59 smithi195 ceph-mon[33844]: pgmap v401: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 26 op/s 2024-05-09T05:50:59.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:50:59 smithi115 ceph-mon[30349]: pgmap v401: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 26 op/s 2024-05-09T05:50:59.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:50:59 smithi161 ceph-mon[33805]: pgmap v401: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 26 op/s 2024-05-09T05:51:01.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:01 smithi195 ceph-mon[33844]: pgmap v402: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:51:01.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:01 smithi115 ceph-mon[30349]: pgmap v402: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:51:01.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:01 smithi161 ceph-mon[33805]: pgmap v402: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:51:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.473+0000 7fcc41cc8640 1 -- 172.21.15.115:0/1136259406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 msgr2=0x7fcc3c102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.473+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/1136259406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 0x7fcc3c102740 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009900 tx=0x7fcc2c02a410 comp rx=0 tx=0).stop 2024-05-09T05:51:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.473+0000 7fcc41cc8640 1 -- 172.21.15.115:0/1136259406 shutdown_connections 2024-05-09T05:51:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.473+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/1136259406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3c1032f0 0x7fcc3c10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.473+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/1136259406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3c109040 0x7fcc3c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.473+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/1136259406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 0x7fcc3c102740 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.473+0000 7fcc41cc8640 1 -- 172.21.15.115:0/1136259406 >> 172.21.15.115:0/1136259406 conn(0x7fcc3c0782e0 msgr2=0x7fcc3c078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.473+0000 7fcc41cc8640 1 -- 172.21.15.115:0/1136259406 shutdown_connections 2024-05-09T05:51:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.473+0000 7fcc41cc8640 1 -- 172.21.15.115:0/1136259406 wait complete. 2024-05-09T05:51:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.474+0000 7fcc41cc8640 1 Processor -- start 2024-05-09T05:51:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.474+0000 7fcc41cc8640 1 -- start start 2024-05-09T05:51:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc41cc8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3c1032f0 0x7fcc3c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc41cc8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 0x7fcc3c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc41cc8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3c109040 0x7fcc3c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc41cc8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c112c40 con 0x7fcc3c108ba0 2024-05-09T05:51:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc41cc8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c112ac0 con 0x7fcc3c1032f0 2024-05-09T05:51:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc41cc8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c112dc0 con 0x7fcc3c109040 2024-05-09T05:51:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 0x7fcc3c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 0x7fcc3c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46712/0 (socket says 172.21.15.115:46712) 2024-05-09T05:51:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3affd640 1 -- 172.21.15.115:0/4130270852 learned_addr learned my addr 172.21.15.115:0/4130270852 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3c1032f0 0x7fcc3c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3bfff640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3c109040 0x7fcc3c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3affd640 1 -- 172.21.15.115:0/4130270852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3c109040 msgr2=0x7fcc3c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3affd640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3c109040 0x7fcc3c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3affd640 1 -- 172.21.15.115:0/4130270852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3c1032f0 msgr2=0x7fcc3c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3affd640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3c1032f0 0x7fcc3c19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3affd640 1 -- 172.21.15.115:0/4130270852 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c19f660 con 0x7fcc3c108ba0 2024-05-09T05:51:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc3affd640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 0x7fcc3c19adb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000e9f0 tx=0x7fcc3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.475+0000 7fcc38ff9640 1 -- 172.21.15.115:0/4130270852 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3000ccc0 con 0x7fcc3c108ba0 2024-05-09T05:51:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.476+0000 7fcc41cc8640 1 -- 172.21.15.115:0/4130270852 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c19f950 con 0x7fcc3c108ba0 2024-05-09T05:51:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.476+0000 7fcc38ff9640 1 -- 172.21.15.115:0/4130270852 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc300035f0 con 0x7fcc3c108ba0 2024-05-09T05:51:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.476+0000 7fcc38ff9640 1 -- 172.21.15.115:0/4130270852 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3000ce60 con 0x7fcc3c108ba0 2024-05-09T05:51:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.476+0000 7fcc41cc8640 1 -- 172.21.15.115:0/4130270852 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c19fe50 con 0x7fcc3c108ba0 2024-05-09T05:51:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.478+0000 7fcc48b80740 1 Processor -- start 2024-05-09T05:51:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc48b80740 1 -- start start 2024-05-09T05:51:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ad0e1b0 0x56083b0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083aced4d0 0x56083b0b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc48b80740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56083b0167c0 con 0x56083ace52f0 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc48b80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56083afb20e0 con 0x56083aced4d0 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc48b80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56083b012780 con 0x56083ad0e1b0 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc3b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc3b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46714/0 (socket says 172.21.15.115:46714) 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc3b7fe640 1 -- 172.21.15.115:0/1827082886 learned_addr learned my addr 172.21.15.115:0/1827082886 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc3bfff640 1 --2- 172.21.15.115:0/1827082886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ad0e1b0 0x56083b0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc38ff9640 1 -- 172.21.15.115:0/4130270852 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcc30004030 con 0x7fcc3c108ba0 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc3b7fe640 1 -- 172.21.15.115:0/1827082886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ad0e1b0 msgr2=0x56083b0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/1827082886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ad0e1b0 0x56083b0b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc3b7fe640 1 -- 172.21.15.115:0/1827082886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083aced4d0 msgr2=0x56083b0b2100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/1827082886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083aced4d0 0x56083b0b2100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.479+0000 7fcc3b7fe640 1 -- 172.21.15.115:0/1827082886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083afdb7c0 con 0x56083ace52f0 2024-05-09T05:51:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.480+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/1827082886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0b1bc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0057c0 tx=0x7fcc2c031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71be7daedb629e81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.480+0000 7fcc21ffb640 1 -- 172.21.15.115:0/1827082886 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c035030 con 0x56083ace52f0 2024-05-09T05:51:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.480+0000 7fcc21ffb640 1 -- 172.21.15.115:0/1827082886 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc2c040070 con 0x56083ace52f0 2024-05-09T05:51:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.480+0000 7fcc21ffb640 1 -- 172.21.15.115:0/1827082886 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c003850 con 0x56083ace52f0 2024-05-09T05:51:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.480+0000 7fcc38ff9640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc240b2b50 0x7fcc240b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.480+0000 7fcc3bfff640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc240b2b50 0x7fcc240b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc38ff9640 1 -- 172.21.15.115:0/4130270852 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fcc30010430 con 0x7fcc3c108ba0 2024-05-09T05:51:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc48b80740 1 -- 172.21.15.115:0/1827082886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 msgr2=0x56083b0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1827082886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0b1bc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0057c0 tx=0x7fcc2c031a70 comp rx=0 tx=0).stop 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc3bfff640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc240b2b50 0x7fcc240b5060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x56083b0ae7f0 tx=0x7fcc34009500 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc48b80740 1 -- 172.21.15.115:0/1827082886 shutdown_connections 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1827082886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083aced4d0 0x56083b0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1827082886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0b1bc0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1827082886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ad0e1b0 0x56083b0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc48b80740 1 -- 172.21.15.115:0/1827082886 >> 172.21.15.115:0/1827082886 conn(0x56083ad02300 msgr2=0x56083b0ade30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc48b80740 1 -- 172.21.15.115:0/1827082886 shutdown_connections 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.481+0000 7fcc48b80740 1 -- 172.21.15.115:0/1827082886 wait complete. 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.482+0000 7fcc48b80740 1 -- start start 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.482+0000 7fcc48b80740 1 Processor -- start 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.482+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ace52f0 0x56083b0bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.482+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083aced4d0 0x56083b0bddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.482+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 0x56083b0bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.482+0000 7fcc48b80740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56083b0be750 con 0x56083aced4d0 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.482+0000 7fcc48b80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56083af9d9c0 con 0x56083ad0e1b0 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.482+0000 7fcc48b80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56083b0be8d0 con 0x56083ace52f0 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083aced4d0 0x56083b0bddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc48b80740 10 client.0 _ll_register_callbacks cb 0x56083afca020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083aced4d0 0x56083b0bddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46718/0 (socket says 172.21.15.115:46718) 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3affd640 1 -- 172.21.15.115:0/1797086698 learned_addr learned my addr 172.21.15.115:0/1797086698 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ace52f0 0x56083b0bd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3bfff640 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 0x56083b0bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3affd640 1 -- 172.21.15.115:0/1797086698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ace52f0 msgr2=0x56083b0bd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3affd640 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ace52f0 0x56083b0bd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3affd640 1 -- 172.21.15.115:0/1797086698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 msgr2=0x56083b0bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3affd640 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 0x56083b0bbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3affd640 1 -- 172.21.15.115:0/1797086698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083afc41f0 con 0x56083aced4d0 2024-05-09T05:51:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ace52f0 0x56083b0bd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc3affd640 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083aced4d0 0x56083b0bddd0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fcc300a9990 tx=0x7fcc30003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa4b4f85c38cc7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.483+0000 7fcc1affd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc1affd640 1 -- 172.21.15.115:0/1797086698 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300c3060 con 0x56083aced4d0 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc1affd640 1 -- 172.21.15.115:0/1797086698 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fcc300a5c90 con 0x56083aced4d0 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc48b80740 1 -- 172.21.15.115:0/1797086698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2c040070 con 0x56083aced4d0 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc1affd640 1 -- 172.21.15.115:0/1797086698 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fcc300a9a00 con 0x56083aced4d0 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc48b80740 10 client.18507 fetch_fsmap learned FSMap version 56 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc48b80740 1 -- 172.21.15.115:0/1797086698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56083ad79790 con 0x56083aced4d0 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc1affd640 1 -- 172.21.15.115:0/1797086698 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc2c040070 con 0x56083aced4d0 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc1affd640 1 -- 172.21.15.115:0/1797086698 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7fcc300c5020 con 0x56083aced4d0 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc48b80740 10 client.18507 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc48b80740 10 client.18507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc48b80740 20 client.18507 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:03.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56083acccd80 0x56083b0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc48b80740 4 client.18507 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.484+0000 7fcc48b80740 1 -- 172.21.15.115:0/1797086698 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56083b0243d0 con 0x56083acccd80 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.485+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56083acccd80 0x56083b0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.485+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56083acccd80 0x56083b0ce040 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6af7bdc9b2451c server_cookie=743f7199917a225b in_seq=0 out_seq=0 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.485+0000 7fcc1affd640 10 client.18507 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc1affd640 1 -- 172.21.15.115:0/1797086698 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56083b0243d0 con 0x56083acccd80 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc1affd640 10 client.18507 handle_command_reply: tid=0 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 client.18507 shutdown 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 -- 172.21.15.115:0/1797086698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083aced4d0 msgr2=0x56083b0bddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083aced4d0 0x56083b0bddd0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fcc300a9990 tx=0x7fcc30003cd0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 -- 172.21.15.115:0/1797086698 shutdown_connections 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 0x56083b0bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083aced4d0 0x56083b0bddd0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56083acccd80 0x56083b0ce040 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 --2- 172.21.15.115:0/1797086698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083ace52f0 0x56083b0bd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc1affd640 0 client.18507 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 -- 172.21.15.115:0/1797086698 >> 172.21.15.115:0/1797086698 conn(0x56083ad02300 msgr2=0x56083b0ac700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.486+0000 7fcc48b80740 1 -- 172.21.15.115:0/1797086698 shutdown_connections 2024-05-09T05:51:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.487+0000 7fcc48b80740 1 -- 172.21.15.115:0/1797086698 wait complete. 2024-05-09T05:51:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.487+0000 7fcc48b80740 20 client.18507 trim_cache size 0 max 16384 2024-05-09T05:51:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.502+0000 7fcc48b80740 1 Processor -- start 2024-05-09T05:51:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.502+0000 7fcc48b80740 1 -- start start 2024-05-09T05:51:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.502+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ace52f0 0x56083b0ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.502+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0d3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.502+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ad0e1b0 0x56083b0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.502+0000 7fcc48b80740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b3360 con 0x56083ad0e1b0 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.502+0000 7fcc48b80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b4f30 con 0x56083ace52f0 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.502+0000 7fcc48b80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b3a30 con 0x56083aced4d0 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ad0e1b0 0x56083b0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ad0e1b0 0x56083b0d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46734/0 (socket says 172.21.15.115:46734) 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0d3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:58510/0 (socket says 172.21.15.115:58510) 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3bfff640 1 -- 172.21.15.115:0/3504326502 learned_addr learned my addr 172.21.15.115:0/3504326502 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/3504326502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ace52f0 0x56083b0ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3affd640 1 -- 172.21.15.115:0/3504326502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ace52f0 msgr2=0x56083b0ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3affd640 1 --2- 172.21.15.115:0/3504326502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ace52f0 0x56083b0ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3affd640 1 -- 172.21.15.115:0/3504326502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ad0e1b0 msgr2=0x56083b0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3affd640 1 --2- 172.21.15.115:0/3504326502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ad0e1b0 0x56083b0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3affd640 1 -- 172.21.15.115:0/3504326502 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083adf05f0 con 0x56083aced4d0 2024-05-09T05:51:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc3affd640 1 --2- 172.21.15.115:0/3504326502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0d3790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcc300a6c70 tx=0x7fcc30005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17891922757e4544 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc217fa640 1 -- 172.21.15.115:0/3504326502 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300a9170 con 0x56083aced4d0 2024-05-09T05:51:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc217fa640 1 -- 172.21.15.115:0/3504326502 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc300bee50 con 0x56083aced4d0 2024-05-09T05:51:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.503+0000 7fcc217fa640 1 -- 172.21.15.115:0/3504326502 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300b2860 con 0x56083aced4d0 2024-05-09T05:51:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 -- 172.21.15.115:0/3504326502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 msgr2=0x56083b0d3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 --2- 172.21.15.115:0/3504326502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0d3790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcc300a6c70 tx=0x7fcc30005e30 comp rx=0 tx=0).stop 2024-05-09T05:51:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 -- 172.21.15.115:0/3504326502 shutdown_connections 2024-05-09T05:51:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 --2- 172.21.15.115:0/3504326502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ad0e1b0 0x56083b0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 --2- 172.21.15.115:0/3504326502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0d3790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 --2- 172.21.15.115:0/3504326502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ace52f0 0x56083b0ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 -- 172.21.15.115:0/3504326502 >> 172.21.15.115:0/3504326502 conn(0x56083ad02300 msgr2=0x56083b0ac700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 -- 172.21.15.115:0/3504326502 shutdown_connections 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 -- 172.21.15.115:0/3504326502 wait complete. 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 -- start start 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.505+0000 7fcc48b80740 1 Processor -- start 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc48b80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 0x56083b0bf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc48b80740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b4860 con 0x56083ace52f0 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc48b80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b4190 con 0x56083ad0e1b0 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc48b80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1b1360 con 0x56083aced4d0 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46738/0 (socket says 172.21.15.115:46738) 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3b7fe640 1 -- 172.21.15.115:0/2810294192 learned_addr learned my addr 172.21.15.115:0/2810294192 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc48b80740 10 client.0 _ll_register_callbacks cb 0x7fcc300056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3bfff640 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 0x56083b0bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3affd640 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3b7fe640 1 -- 172.21.15.115:0/2810294192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 msgr2=0x56083b0bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3b7fe640 1 -- 172.21.15.115:0/2810294192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 msgr2=0x56083b0bf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 0x56083b0bf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3b7fe640 1 -- 172.21.15.115:0/2810294192 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083afdaed0 con 0x56083ace52f0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3affd640 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc3b7fe640 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0c3f70 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c006010 tx=0x7fcc2c031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ccabe7b32276a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.506+0000 7fcc1affd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.507+0000 7fcc1affd640 1 -- 172.21.15.115:0/2810294192 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c054070 con 0x56083ace52f0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.507+0000 7fcc1affd640 1 -- 172.21.15.115:0/2810294192 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fcc2c03c910 con 0x56083ace52f0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.507+0000 7fcc48b80740 1 -- 172.21.15.115:0/2810294192 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc300bee50 con 0x56083ace52f0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.507+0000 7fcc1affd640 1 -- 172.21.15.115:0/2810294192 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fcc2c060050 con 0x56083ace52f0 2024-05-09T05:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.507+0000 7fcc48b80740 10 client.18513 fetch_fsmap learned FSMap version 56 2024-05-09T05:51:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.507+0000 7fcc48b80740 1 -- 172.21.15.115:0/2810294192 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56083b135ed0 con 0x56083ace52f0 2024-05-09T05:51:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.507+0000 7fcc1affd640 1 -- 172.21.15.115:0/2810294192 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc300bee50 con 0x56083ace52f0 2024-05-09T05:51:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.507+0000 7fcc1affd640 1 -- 172.21.15.115:0/2810294192 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7fcc2c044b90 con 0x56083ace52f0 2024-05-09T05:51:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.507+0000 7fcc48b80740 10 client.18513 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:51:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.508+0000 7fcc48b80740 10 client.18513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.508+0000 7fcc48b80740 20 client.18513 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.508+0000 7fcc48b80740 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56083aca21c0 0x56083b138250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.508+0000 7fcc48b80740 4 client.18513 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:51:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.508+0000 7fcc48b80740 1 -- 172.21.15.115:0/2810294192 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56083b01e240 con 0x56083aca21c0 2024-05-09T05:51:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.508+0000 7fcc3affd640 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56083aca21c0 0x56083b138250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.508+0000 7fcc1affd640 10 client.18513 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:03.508+0000 7fcc3affd640 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56083aca21c0 0x56083b138250 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e76b47a8cead4a server_cookie=7876821cf6d83731 in_seq=0 out_seq=0 2024-05-09T05:51:03.603 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:03 smithi115 ceph-mon[30349]: pgmap v403: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:51:03.613 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:03 smithi195 ceph-mon[33844]: pgmap v403: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:51:03.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:03 smithi161 ceph-mon[33805]: pgmap v403: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-05-09T05:51:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc1affd640 1 -- 172.21.15.115:0/2810294192 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56083b01e240 con 0x56083aca21c0 2024-05-09T05:51:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc1affd640 10 client.18513 handle_command_reply: tid=0 2024-05-09T05:51:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc48b80740 1 client.18513 shutdown 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc48b80740 1 -- 172.21.15.115:0/2810294192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 msgr2=0x56083b0c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc48b80740 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0c3f70 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c006010 tx=0x7fcc2c031890 comp rx=0 tx=0).stop 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc48b80740 1 -- 172.21.15.115:0/2810294192 shutdown_connections 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc48b80740 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56083ad0e1b0 0x56083b0bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc48b80740 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56083aca21c0 0x56083b138250 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc48b80740 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56083aced4d0 0x56083b0bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc48b80740 1 --2- 172.21.15.115:0/2810294192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56083ace52f0 0x56083b0c3f70 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc1affd640 0 client.18513 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.290+0000 7fcc48b80740 1 -- 172.21.15.115:0/2810294192 >> 172.21.15.115:0/2810294192 conn(0x56083ad02300 msgr2=0x56083b0ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc48b80740 1 -- 172.21.15.115:0/2810294192 shutdown_connections 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc48b80740 1 -- 172.21.15.115:0/2810294192 wait complete. 2024-05-09T05:51:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc48b80740 20 client.18513 trim_cache size 0 max 16384 2024-05-09T05:51:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:51:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:51:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:51:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 -- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc240b2b50 msgr2=0x7fcc240b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc240b2b50 0x7fcc240b5060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x56083b0ae7f0 tx=0x7fcc34009500 comp rx=0 tx=0).stop 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 -- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 msgr2=0x7fcc3c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 0x7fcc3c19adb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000e9f0 tx=0x7fcc3000eeb0 comp rx=0 tx=0).stop 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 -- 172.21.15.115:0/4130270852 shutdown_connections 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc240b2b50 0x7fcc240b5060 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3c109040 0x7fcc3c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc3c108ba0 0x7fcc3c19adb0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 --2- 172.21.15.115:0/4130270852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3c1032f0 0x7fcc3c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 -- 172.21.15.115:0/4130270852 >> 172.21.15.115:0/4130270852 conn(0x7fcc3c0782e0 msgr2=0x7fcc3c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 -- 172.21.15.115:0/4130270852 shutdown_connections 2024-05-09T05:51:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:04.291+0000 7fcc41cc8640 1 -- 172.21.15.115:0/4130270852 wait complete. 2024-05-09T05:51:05.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:05 smithi195 ceph-mon[33844]: pgmap v404: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 47 op/s 2024-05-09T05:51:05.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:05 smithi195 ceph-mon[33844]: osdmap e104: 12 total, 12 up, 12 in 2024-05-09T05:51:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:05 smithi115 ceph-mon[30349]: pgmap v404: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 47 op/s 2024-05-09T05:51:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:05 smithi115 ceph-mon[30349]: osdmap e104: 12 total, 12 up, 12 in 2024-05-09T05:51:05.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:05 smithi161 ceph-mon[33805]: pgmap v404: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 47 op/s 2024-05-09T05:51:05.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:05 smithi161 ceph-mon[33805]: osdmap e104: 12 total, 12 up, 12 in 2024-05-09T05:51:06.379 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-09T05:51:06.380 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-09T05:51:06.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:51:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:51:07.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:07 smithi195 ceph-mon[33844]: pgmap v406: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 44 op/s 2024-05-09T05:51:07.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:07 smithi115 ceph-mon[30349]: pgmap v406: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 44 op/s 2024-05-09T05:51:07.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:07 smithi161 ceph-mon[33805]: pgmap v406: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 44 op/s 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.616+0000 7fcb112bd700 1 -- 172.21.15.115:0/1591035999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 msgr2=0x7fcb0c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.616+0000 7fcb112bd700 1 --2- 172.21.15.115:0/1591035999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 0x7fcb0c108200 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fcb000099c0 tx=0x7fcb00009cd0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.617+0000 7fcb112bd700 1 -- 172.21.15.115:0/1591035999 shutdown_connections 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.617+0000 7fcb112bd700 1 --2- 172.21.15.115:0/1591035999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0c108740 0x7fcb0c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.617+0000 7fcb112bd700 1 --2- 172.21.15.115:0/1591035999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb0c1034c0 0x7fcb0c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.617+0000 7fcb112bd700 1 --2- 172.21.15.115:0/1591035999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 0x7fcb0c108200 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.617+0000 7fcb112bd700 1 -- 172.21.15.115:0/1591035999 >> 172.21.15.115:0/1591035999 conn(0x7fcb0c0fedc0 msgr2=0x7fcb0c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.617+0000 7fcb112bd700 1 -- 172.21.15.115:0/1591035999 shutdown_connections 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.617+0000 7fcb112bd700 1 -- 172.21.15.115:0/1591035999 wait complete. 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.617+0000 7fcb112bd700 1 Processor -- start 2024-05-09T05:51:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb112bd700 1 -- start start 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb112bd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0c1034c0 0x7fcb0c0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb112bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 0x7fcb0c0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb112bd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb0c108740 0x7fcb0c0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb112bd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c113200 con 0x7fcb0c107df0 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb112bd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c113080 con 0x7fcb0c1034c0 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb112bd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c113380 con 0x7fcb0c108740 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb0a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 0x7fcb0c0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb0a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 0x7fcb0c0ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46766/0 (socket says 172.21.15.115:46766) 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb0a7fc700 1 -- 172.21.15.115:0/2297414629 learned_addr learned my addr 172.21.15.115:0/2297414629 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb0affd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0c1034c0 0x7fcb0c0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb0b7fe700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb0c108740 0x7fcb0c0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb0a7fc700 1 -- 172.21.15.115:0/2297414629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb0c108740 msgr2=0x7fcb0c0ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:08.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb0a7fc700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb0c108740 0x7fcb0c0ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb0a7fc700 1 -- 172.21.15.115:0/2297414629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0c1034c0 msgr2=0x7fcb0c0ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.618+0000 7fcb0a7fc700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0c1034c0 0x7fcb0c0ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.619+0000 7fcb0a7fc700 1 -- 172.21.15.115:0/2297414629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c1a90e0 con 0x7fcb0c107df0 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.619+0000 7fcb0affd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0c1034c0 0x7fcb0c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.619+0000 7fcb0b7fe700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb0c108740 0x7fcb0c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.619+0000 7fcb0a7fc700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 0x7fcb0c0ff7e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fcb00005f50 tx=0x7fcb00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.619+0000 7fcaf3fff700 1 -- 172.21.15.115:0/2297414629 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00026070 con 0x7fcb0c107df0 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.619+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c1a9370 con 0x7fcb0c107df0 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.619+0000 7fcaf3fff700 1 -- 172.21.15.115:0/2297414629 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb00003c50 con 0x7fcb0c107df0 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.619+0000 7fcaf3fff700 1 -- 172.21.15.115:0/2297414629 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000fa30 con 0x7fcb0c107df0 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.619+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c1a97c0 con 0x7fcb0c107df0 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.620+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c0512c0 con 0x7fcb0c107df0 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.622+0000 7fcaf3fff700 1 -- 172.21.15.115:0/2297414629 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcb00029070 con 0x7fcb0c107df0 2024-05-09T05:51:08.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.623+0000 7fcaf3fff700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf40b29f0 0x7fcaf40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:08.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.623+0000 7fcb0affd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf40b29f0 0x7fcaf40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:08.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.623+0000 7fcaf3fff700 1 -- 172.21.15.115:0/2297414629 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9526+0+0 (secure 0 0 0) 0x7fcb000e32b0 con 0x7fcb0c107df0 2024-05-09T05:51:08.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.624+0000 7fcb0affd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf40b29f0 0x7fcaf40b4f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcafc011490 tx=0x7fcafc009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:08.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.629+0000 7fcaf3fff700 1 -- 172.21.15.115:0/2297414629 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb0005edf0 con 0x7fcb0c107df0 2024-05-09T05:51:08.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.956+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb0c02e7b0 con 0x7fcb0c107df0 2024-05-09T05:51:08.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.957+0000 7fcaf3fff700 1 -- 172.21.15.115:0/2297414629 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+11956 (secure 0 0 0) 0x7fcb00013070 con 0x7fcb0c107df0 2024-05-09T05:51:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:51:08.947 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:50:23.370056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf40b29f0 msgr2=0x7fcaf40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf40b29f0 0x7fcaf40b4f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcafc011490 tx=0x7fcafc009260 comp rx=0 tx=0).stop 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 msgr2=0x7fcb0c0ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 0x7fcb0c0ff7e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fcb00005f50 tx=0x7fcb00004270 comp rx=0 tx=0).stop 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 shutdown_connections 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcaf40b29f0 0x7fcaf40b4f00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0c1034c0 0x7fcb0c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb0c108740 0x7fcb0c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 --2- 172.21.15.115:0/2297414629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c107df0 0x7fcb0c0ff7e0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 >> 172.21.15.115:0/2297414629 conn(0x7fcb0c0fedc0 msgr2=0x7fcb0c06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 shutdown_connections 2024-05-09T05:51:08.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:08.959+0000 7fcb112bd700 1 -- 172.21.15.115:0/2297414629 wait complete. 2024-05-09T05:51:08.950 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 56 2024-05-09T05:51:09.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:09 smithi195 ceph-mon[33844]: pgmap v407: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 45 op/s 2024-05-09T05:51:09.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:09 smithi195 ceph-mon[33844]: osdmap e105: 12 total, 12 up, 12 in 2024-05-09T05:51:09.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:09 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2297414629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:51:09.635 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:09 smithi115 ceph-mon[30349]: pgmap v407: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 45 op/s 2024-05-09T05:51:09.635 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:09 smithi115 ceph-mon[30349]: osdmap e105: 12 total, 12 up, 12 in 2024-05-09T05:51:09.635 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:09 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2297414629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:51:09.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:09 smithi161 ceph-mon[33805]: pgmap v407: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 45 op/s 2024-05-09T05:51:09.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:09 smithi161 ceph-mon[33805]: osdmap e105: 12 total, 12 up, 12 in 2024-05-09T05:51:09.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:09 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2297414629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:51:09.662 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-09T05:51:09.662 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-09T05:51:09.954 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:51:11.385 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:11 smithi115 ceph-mon[30349]: pgmap v409: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 57 op/s 2024-05-09T05:51:11.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:11 smithi195 ceph-mon[33844]: pgmap v409: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 57 op/s 2024-05-09T05:51:11.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:11 smithi161 ceph-mon[33805]: pgmap v409: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 57 op/s 2024-05-09T05:51:11.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.850+0000 7f64a9e1a700 1 -- 172.21.15.115:0/644692817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 msgr2=0x7f64a41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:11.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.850+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/644692817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 0x7f64a41038d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f64940099c0 tx=0x7f6494009cd0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.850+0000 7f64a9e1a700 1 -- 172.21.15.115:0/644692817 shutdown_connections 2024-05-09T05:51:11.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.850+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/644692817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a4108740 0x7f64a41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.850+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/644692817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 0x7f64a41038d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.850+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/644692817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a4107df0 0x7f64a4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.850+0000 7f64a9e1a700 1 -- 172.21.15.115:0/644692817 >> 172.21.15.115:0/644692817 conn(0x7f64a40fedc0 msgr2=0x7f64a41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:11.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.850+0000 7f64a9e1a700 1 -- 172.21.15.115:0/644692817 shutdown_connections 2024-05-09T05:51:11.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.850+0000 7f64a9e1a700 1 -- 172.21.15.115:0/644692817 wait complete. 2024-05-09T05:51:11.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a9e1a700 1 Processor -- start 2024-05-09T05:51:11.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a9e1a700 1 -- start start 2024-05-09T05:51:11.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a9e1a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 0x7f64a419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a9e1a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a4107df0 0x7f64a419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a9e1a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a4108740 0x7f64a419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a9e1a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f64a4113220 con 0x7f64a41034c0 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a9e1a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41130a0 con 0x7f64a4107df0 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a9e1a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41133a0 con 0x7f64a4108740 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 0x7f64a419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 0x7f64a419ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46792/0 (socket says 172.21.15.115:46792) 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a37fe700 1 -- 172.21.15.115:0/2421575763 learned_addr learned my addr 172.21.15.115:0/2421575763 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a4107df0 0x7f64a419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.851+0000 7f64a3fff700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a4108740 0x7f64a419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a37fe700 1 -- 172.21.15.115:0/2421575763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a4108740 msgr2=0x7f64a419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a37fe700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a4108740 0x7f64a419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a37fe700 1 -- 172.21.15.115:0/2421575763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a4107df0 msgr2=0x7f64a419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a37fe700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a4107df0 0x7f64a419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a37fe700 1 -- 172.21.15.115:0/2421575763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a419fbf0 con 0x7f64a41034c0 2024-05-09T05:51:11.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a4107df0 0x7f64a419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:11.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a37fe700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 0x7f64a419ad20 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6494000c00 tx=0x7f64940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:11.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a0ff9700 1 -- 172.21.15.115:0/2421575763 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494004830 con 0x7f64a41034c0 2024-05-09T05:51:11.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a9e1a700 1 -- 172.21.15.115:0/2421575763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a419fe80 con 0x7f64a41034c0 2024-05-09T05:51:11.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a0ff9700 1 -- 172.21.15.115:0/2421575763 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f649400f7f0 con 0x7f64a41034c0 2024-05-09T05:51:11.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a0ff9700 1 -- 172.21.15.115:0/2421575763 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940053a0 con 0x7f64a41034c0 2024-05-09T05:51:11.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.852+0000 7f64a9e1a700 1 -- 172.21.15.115:0/2421575763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a41a0300 con 0x7f64a41034c0 2024-05-09T05:51:11.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.855+0000 7f64a0ff9700 1 -- 172.21.15.115:0/2421575763 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f6494029060 con 0x7f64a41034c0 2024-05-09T05:51:11.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.856+0000 7f64a0ff9700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f648c0b29f0 0x7f648c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.856+0000 7f64b8830b80 1 Processor -- start 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.856+0000 7f64a0ff9700 1 -- 172.21.15.115:0/2421575763 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f64940e3370 con 0x7f64a41034c0 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.856+0000 7f64b8830b80 1 -- start start 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.856+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f648c0b29f0 0x7f648c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb3dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb3e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb4adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64b8830b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c9ea61230 con 0x564c9eb48990 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64b8830b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564c9ea06b30 con 0x564c9ea3be90 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64b8830b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564c9ea5d1f0 con 0x564c9e9c1630 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f648c0b29f0 0x7f648c0b4f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f64a4106450 tx=0x7f649800b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb4adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb3dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb4adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46808/0 (socket says 172.21.15.115:46808) 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb3dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:59854/0 (socket says 172.21.15.115:59854) 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a3fff700 1 -- 172.21.15.115:0/3838256847 learned_addr learned my addr 172.21.15.115:0/3838256847 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/3838256847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb3e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a3fff700 1 -- 172.21.15.115:0/3838256847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 msgr2=0x564c9eb3e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a3fff700 1 --2- 172.21.15.115:0/3838256847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb3e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a3fff700 1 -- 172.21.15.115:0/3838256847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 msgr2=0x564c9eb3dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a3fff700 1 --2- 172.21.15.115:0/3838256847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb3dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.857+0000 7f64a3fff700 1 -- 172.21.15.115:0/3838256847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c9e9f7690 con 0x564c9eb48990 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.858+0000 7f64a3fff700 1 --2- 172.21.15.115:0/3838256847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb4adb0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f649c003f20 tx=0x7f649c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37a1ea4deff1694 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.858+0000 7f6489ffb700 1 -- 172.21.15.115:0/3838256847 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c004670 con 0x564c9eb48990 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.858+0000 7f6489ffb700 1 -- 172.21.15.115:0/3838256847 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f649c01e070 con 0x564c9eb48990 2024-05-09T05:51:11.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.858+0000 7f6489ffb700 1 -- 172.21.15.115:0/3838256847 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c00e6b0 con 0x564c9eb48990 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 -- 172.21.15.115:0/3838256847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 msgr2=0x564c9eb4adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 --2- 172.21.15.115:0/3838256847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb4adb0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f649c003f20 tx=0x7f649c009c20 comp rx=0 tx=0).stop 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 -- 172.21.15.115:0/3838256847 shutdown_connections 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 --2- 172.21.15.115:0/3838256847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb3dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 --2- 172.21.15.115:0/3838256847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb4adb0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 --2- 172.21.15.115:0/3838256847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb3e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 -- 172.21.15.115:0/3838256847 >> 172.21.15.115:0/3838256847 conn(0x564c9e9c5100 msgr2=0x564c9eb39bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 -- 172.21.15.115:0/3838256847 shutdown_connections 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 -- 172.21.15.115:0/3838256847 wait complete. 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 -- start start 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.859+0000 7f64b8830b80 1 Processor -- start 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb45940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb45e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb463c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64b8830b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564c9eb4b3b0 con 0x564c9eb48990 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64b8830b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564c9e9fbcb0 con 0x564c9ea3be90 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64b8830b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564c9eb4b530 con 0x564c9e9c1630 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46816/0 (socket says 172.21.15.115:46816) 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a37fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb45940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a3fff700 1 -- 172.21.15.115:0/1025345045 learned_addr learned my addr 172.21.15.115:0/1025345045 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb45e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64b8830b80 10 client.0 _ll_register_callbacks cb 0x564c9e8d2850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a3fff700 1 -- 172.21.15.115:0/1025345045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 msgr2=0x564c9eb45940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a3fff700 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb45940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a3fff700 1 -- 172.21.15.115:0/1025345045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 msgr2=0x564c9eb45e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:11.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a3fff700 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb45e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a3fff700 1 -- 172.21.15.115:0/1025345045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c9e8dc620 con 0x564c9eb48990 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f64a3fff700 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb463c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f649c007d60 tx=0x7f649c005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d3e6c6fc5ae26b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.860+0000 7f6482ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f6482ffd700 1 -- 172.21.15.115:0/1025345045 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c007500 con 0x564c9eb48990 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f6482ffd700 1 -- 172.21.15.115:0/1025345045 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f649c0076a0 con 0x564c9eb48990 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f64b8830b80 1 -- 172.21.15.115:0/1025345045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f649c01e070 con 0x564c9eb48990 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f6482ffd700 1 -- 172.21.15.115:0/1025345045 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f649c0078a0 con 0x564c9eb48990 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f64b8830b80 10 client.18531 fetch_fsmap learned FSMap version 56 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f64b8830b80 1 -- 172.21.15.115:0/1025345045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x564c9eb55fb0 con 0x564c9eb48990 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f6482ffd700 1 -- 172.21.15.115:0/1025345045 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f649c01e070 con 0x564c9eb48990 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f6482ffd700 1 -- 172.21.15.115:0/1025345045 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7f649c00e450 con 0x564c9eb48990 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f64b8830b80 10 client.18531 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f64b8830b80 10 client.18531 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.861+0000 7f64b8830b80 20 client.18531 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:11.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.862+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c9eb585f0 0x564c9eb5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.862+0000 7f64b8830b80 4 client.18531 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.862+0000 7f64b8830b80 1 -- 172.21.15.115:0/1025345045 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c9ea6ec10 con 0x564c9eb585f0 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.862+0000 7f64a37fe700 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c9eb585f0 0x564c9eb5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.862+0000 7f64a37fe700 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c9eb585f0 0x564c9eb5aa10 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dae9eedafcf83f6 server_cookie=6b6bb4622a7200c2 in_seq=0 out_seq=0 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.862+0000 7f6482ffd700 10 client.18531 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f6482ffd700 1 -- 172.21.15.115:0/1025345045 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c9ea6ec10 con 0x564c9eb585f0 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f6482ffd700 10 client.18531 handle_command_reply: tid=0 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f64b8830b80 1 client.18531 shutdown 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f64b8830b80 1 -- 172.21.15.115:0/1025345045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 msgr2=0x564c9eb463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb463c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f649c007d60 tx=0x7f649c005b60 comp rx=0 tx=0).stop 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f64b8830b80 1 -- 172.21.15.115:0/1025345045 shutdown_connections 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c9eb585f0 0x564c9eb5aa10 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb463c0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb45940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f6482ffd700 0 client.18531 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:11.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1025345045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb45e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.863+0000 7f64b8830b80 1 -- 172.21.15.115:0/1025345045 >> 172.21.15.115:0/1025345045 conn(0x564c9e9c5100 msgr2=0x564c9eb38e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:11.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.864+0000 7f64b8830b80 1 -- 172.21.15.115:0/1025345045 shutdown_connections 2024-05-09T05:51:11.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.864+0000 7f64b8830b80 1 -- 172.21.15.115:0/1025345045 wait complete. 2024-05-09T05:51:11.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.864+0000 7f64b8830b80 20 client.18531 trim_cache size 0 max 16384 2024-05-09T05:51:11.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.885+0000 7f64b8830b80 1 Processor -- start 2024-05-09T05:51:11.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.885+0000 7f64b8830b80 1 -- start start 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.885+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb44c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.885+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb47700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.885+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb49db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.885+0000 7f64b8830b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41ae230 con 0x564c9eb48990 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.885+0000 7f64b8830b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41afe00 con 0x564c9ea3be90 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.885+0000 7f64b8830b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41ae900 con 0x564c9e9c1630 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb49db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb49db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46830/0 (socket says 172.21.15.115:46830) 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a3fff700 1 -- 172.21.15.115:0/1094854464 learned_addr learned my addr 172.21.15.115:0/1094854464 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a37fe700 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb44c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb47700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a37fe700 1 -- 172.21.15.115:0/1094854464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 msgr2=0x564c9eb47700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:11.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a37fe700 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb47700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a37fe700 1 -- 172.21.15.115:0/1094854464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 msgr2=0x564c9eb49db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:11.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a37fe700 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb49db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a37fe700 1 -- 172.21.15.115:0/1094854464 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c9eb547d0 con 0x564c9e9c1630 2024-05-09T05:51:11.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a3fff700 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb49db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:11.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb47700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:11.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.886+0000 7f64a37fe700 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb44c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f64940e0730 tx=0x7f6494009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e44cde1a53e33dd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:11.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.887+0000 7f64897fa700 1 -- 172.21.15.115:0/1094854464 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940b3120 con 0x564c9e9c1630 2024-05-09T05:51:11.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.887+0000 7f64897fa700 1 -- 172.21.15.115:0/1094854464 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64940e30c0 con 0x564c9e9c1630 2024-05-09T05:51:11.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.887+0000 7f64897fa700 1 -- 172.21.15.115:0/1094854464 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940aa5b0 con 0x564c9e9c1630 2024-05-09T05:51:11.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 -- 172.21.15.115:0/1094854464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 msgr2=0x564c9eb44c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:11.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb44c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f64940e0730 tx=0x7f6494009f90 comp rx=0 tx=0).stop 2024-05-09T05:51:11.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 -- 172.21.15.115:0/1094854464 shutdown_connections 2024-05-09T05:51:11.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9eb48990 0x564c9eb49db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb44c60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1094854464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9ea3be90 0x564c9eb47700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 -- 172.21.15.115:0/1094854464 >> 172.21.15.115:0/1094854464 conn(0x564c9e9c5100 msgr2=0x564c9eb46ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:11.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 -- 172.21.15.115:0/1094854464 shutdown_connections 2024-05-09T05:51:11.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 -- 172.21.15.115:0/1094854464 wait complete. 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.888+0000 7f64b8830b80 1 -- start start 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64b8830b80 1 Processor -- start 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb5d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9ea3be90 0x564c9eb5dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64b8830b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9eb48990 0x564c9eb5be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64b8830b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41af730 con 0x564c9ea3be90 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64b8830b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41af060 con 0x564c9eb48990 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64b8830b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64a41ac1f0 con 0x564c9e9c1630 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64a2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9ea3be90 0x564c9eb5dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64b8830b80 10 client.0 _ll_register_callbacks cb 0x564c9e8d2850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64a2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9ea3be90 0x564c9eb5dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46836/0 (socket says 172.21.15.115:46836) 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64a2ffd700 1 -- 172.21.15.115:0/1488873124 learned_addr learned my addr 172.21.15.115:0/1488873124 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64a3fff700 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9eb48990 0x564c9eb5be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64a37fe700 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb5d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64a2ffd700 1 -- 172.21.15.115:0/1488873124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 msgr2=0x564c9eb5d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb5d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64a2ffd700 1 -- 172.21.15.115:0/1488873124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9eb48990 msgr2=0x564c9eb5be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.889+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9eb48990 0x564c9eb5be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.890+0000 7f64a2ffd700 1 -- 172.21.15.115:0/1488873124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c9eb5c4c0 con 0x564c9ea3be90 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.890+0000 7f64a2ffd700 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9ea3be90 0x564c9eb5dde0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6498005950 tx=0x7f64980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f16cf80e8f2d74 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.890+0000 7f6482ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.890+0000 7f6482ffd700 1 -- 172.21.15.115:0/1488873124 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649801eb90 con 0x564c9ea3be90 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.890+0000 7f64b8830b80 1 -- 172.21.15.115:0/1488873124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64940e30c0 con 0x564c9ea3be90 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.890+0000 7f6482ffd700 1 -- 172.21.15.115:0/1488873124 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f649801ed30 con 0x564c9ea3be90 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.890+0000 7f6482ffd700 1 -- 172.21.15.115:0/1488873124 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f6498004020 con 0x564c9ea3be90 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.890+0000 7f64b8830b80 10 client.18537 fetch_fsmap learned FSMap version 56 2024-05-09T05:51:11.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.890+0000 7f64b8830b80 1 -- 172.21.15.115:0/1488873124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x564c9ebc3630 con 0x564c9ea3be90 2024-05-09T05:51:11.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f6482ffd700 1 -- 172.21.15.115:0/1488873124 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64940e30c0 con 0x564c9ea3be90 2024-05-09T05:51:11.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f6482ffd700 1 -- 172.21.15.115:0/1488873124 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) ==== 8374+0+0 (secure 0 0 0) 0x7f6498028070 con 0x564c9ea3be90 2024-05-09T05:51:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f64b8830b80 10 client.18537 fetch_fsmap finished waiting for FSMap version 56 2024-05-09T05:51:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f64b8830b80 10 client.18537 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:51:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f64b8830b80 20 client.18537 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c9ebc3e10 0x564c9ebc6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f64b8830b80 4 client.18537 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-09T05:51:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f64b8830b80 1 -- 172.21.15.115:0/1488873124 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x564c9ea68cb0 con 0x564c9ebc3e10 2024-05-09T05:51:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f64a37fe700 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c9ebc3e10 0x564c9ebc6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f64a37fe700 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c9ebc3e10 0x564c9ebc6230 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3d275260ceff1a server_cookie=e081db8058dfab6c in_seq=0 out_seq=0 2024-05-09T05:51:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:11.891+0000 7f6482ffd700 10 client.18537 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.000+0000 7f6482ffd700 1 -- 172.21.15.115:0/1488873124 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564c9ea68cb0 con 0x564c9ebc3e10 2024-05-09T05:51:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.000+0000 7f6482ffd700 10 client.18537 handle_command_reply: tid=0 2024-05-09T05:51:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.000+0000 7f64b8830b80 1 client.18537 shutdown 2024-05-09T05:51:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 -- 172.21.15.115:0/1488873124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9ea3be90 msgr2=0x564c9eb5dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9ea3be90 0x564c9eb5dde0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6498005950 tx=0x7f64980047c0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 -- 172.21.15.115:0/1488873124 shutdown_connections 2024-05-09T05:51:11.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c9eb48990 0x564c9eb5be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564c9ebc3e10 0x564c9ebc6230 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f6482ffd700 0 client.18537 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564c9e9c1630 0x564c9eb5d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 --2- 172.21.15.115:0/1488873124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564c9ea3be90 0x564c9eb5dde0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 -- 172.21.15.115:0/1488873124 >> 172.21.15.115:0/1488873124 conn(0x564c9e9c5100 msgr2=0x564c9eb449c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 -- 172.21.15.115:0/1488873124 shutdown_connections 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 1 -- 172.21.15.115:0/1488873124 wait complete. 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64b8830b80 20 client.18537 trim_cache size 0 max 16384 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:51:11.992 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 56, 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 38, 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "b9097790": { 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 38, 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:51:11.993 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:11.994 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64a9e1a700 1 -- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f648c0b29f0 msgr2=0x7f648c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.001+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f648c0b29f0 0x7f648c0b4f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f64a4106450 tx=0x7f649800b410 comp rx=0 tx=0).stop 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 -- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 msgr2=0x7f64a419ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 0x7f64a419ad20 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6494000c00 tx=0x7f64940047b0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 -- 172.21.15.115:0/2421575763 shutdown_connections 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f648c0b29f0 0x7f648c0b4f00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64a4108740 0x7f64a419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64a41034c0 0x7f64a419ad20 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 --2- 172.21.15.115:0/2421575763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a4107df0 0x7f64a419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 -- 172.21.15.115:0/2421575763 >> 172.21.15.115:0/2421575763 conn(0x7f64a40fedc0 msgr2=0x7f64a4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 -- 172.21.15.115:0/2421575763 shutdown_connections 2024-05-09T05:51:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:12.002+0000 7f64a9e1a700 1 -- 172.21.15.115:0/2421575763 wait complete. 2024-05-09T05:51:12.700 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b9097790', quiesce took 6.3 seconds. Will release after: 23.7 2024-05-09T05:51:13.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:13 smithi195 ceph-mon[33844]: pgmap v410: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-05-09T05:51:13.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:13 smithi115 ceph-mon[30349]: pgmap v410: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-05-09T05:51:13.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:13 smithi161 ceph-mon[33805]: pgmap v410: 129 pgs: 129 active+clean; 12 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-05-09T05:51:15.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:15 smithi115 ceph-mon[30349]: pgmap v411: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 25 op/s 2024-05-09T05:51:15.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:15 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:51:15.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:15 smithi161 ceph-mon[33805]: pgmap v411: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 25 op/s 2024-05-09T05:51:15.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:15 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:51:15.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:15 smithi195 ceph-mon[33844]: pgmap v411: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 25 op/s 2024-05-09T05:51:15.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:15 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:51:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 -- 172.21.15.115:0/3069090596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 msgr2=0x7f6988102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 --2- 172.21.15.115:0/3069090596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 0x7f6988102740 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f6984009900 tx=0x7f698402a410 comp rx=0 tx=0).stop 2024-05-09T05:51:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 -- 172.21.15.115:0/3069090596 shutdown_connections 2024-05-09T05:51:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 --2- 172.21.15.115:0/3069090596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69881032f0 0x7f698810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 --2- 172.21.15.115:0/3069090596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6988109040 0x7f6988102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 --2- 172.21.15.115:0/3069090596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 0x7f6988102740 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 -- 172.21.15.115:0/3069090596 >> 172.21.15.115:0/3069090596 conn(0x7f69880782e0 msgr2=0x7f6988078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 -- 172.21.15.115:0/3069090596 shutdown_connections 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 -- 172.21.15.115:0/3069090596 wait complete. 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 Processor -- start 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.410+0000 7f698ff5e640 1 -- start start 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698ff5e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69881032f0 0x7f698819b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698ff5e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 0x7f698819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698ff5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6988109040 0x7f698819c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698ff5e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112c40 con 0x7f6988108ba0 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698ff5e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112ac0 con 0x7f69881032f0 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698ff5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112dc0 con 0x7f6988109040 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698d4f9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 0x7f698819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698d4f9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 0x7f698819bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52916/0 (socket says 172.21.15.115:52916) 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698d4f9640 1 -- 172.21.15.115:0/2892852257 learned_addr learned my addr 172.21.15.115:0/2892852257 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698e4fb640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6988109040 0x7f698819c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698dcfa640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69881032f0 0x7f698819b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698d4f9640 1 -- 172.21.15.115:0/2892852257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6988109040 msgr2=0x7f698819c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698d4f9640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6988109040 0x7f698819c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698d4f9640 1 -- 172.21.15.115:0/2892852257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69881032f0 msgr2=0x7f698819b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698d4f9640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69881032f0 0x7f698819b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698d4f9640 1 -- 172.21.15.115:0/2892852257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69881a6970 con 0x7f6988108ba0 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f698d4f9640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 0x7f698819bba0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f697800e9f0 tx=0x7f697800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.411+0000 7f697effd640 1 -- 172.21.15.115:0/2892852257 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697800ccc0 con 0x7f6988108ba0 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.412+0000 7f697effd640 1 -- 172.21.15.115:0/2892852257 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69780044b0 con 0x7f6988108ba0 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.412+0000 7f698ff5e640 1 -- 172.21.15.115:0/2892852257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a6c60 con 0x7f6988108ba0 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.412+0000 7f697effd640 1 -- 172.21.15.115:0/2892852257 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978010430 con 0x7f6988108ba0 2024-05-09T05:51:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.412+0000 7f698ff5e640 1 -- 172.21.15.115:0/2892852257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a71b0 con 0x7f6988108ba0 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.414+0000 7f6996e12740 1 Processor -- start 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.414+0000 7f6996e12740 1 -- start start 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.414+0000 7f6996e12740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0af680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.414+0000 7f6996e12740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.414+0000 7f697effd640 1 -- 172.21.15.115:0/2892852257 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f6978004020 con 0x7f6988108ba0 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.414+0000 7f6996e12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.414+0000 7f6996e12740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fe3d0147c0 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.414+0000 7f6996e12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fe3cfb00e0 con 0x55fe3cceb4d0 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.414+0000 7f6996e12740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fe3d010780 con 0x55fe3cce32f0 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698e4fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698e4fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52928/0 (socket says 172.21.15.115:52928) 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698e4fb640 1 -- 172.21.15.115:0/1214352389 learned_addr learned my addr 172.21.15.115:0/1214352389 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698d4f9640 1 --2- 172.21.15.115:0/1214352389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698dcfa640 1 --2- 172.21.15.115:0/1214352389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698e4fb640 1 -- 172.21.15.115:0/1214352389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 msgr2=0x55fe3d0afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698e4fb640 1 --2- 172.21.15.115:0/1214352389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698e4fb640 1 -- 172.21.15.115:0/1214352389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 msgr2=0x55fe3d0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698e4fb640 1 --2- 172.21.15.115:0/1214352389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698e4fb640 1 -- 172.21.15.115:0/1214352389 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3cfd97c0 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f698e4fb640 1 --2- 172.21.15.115:0/1214352389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0af680 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6980009f90 tx=0x7f69800118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fa241308c6d8ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f6973fff640 1 -- 172.21.15.115:0/1214352389 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980004d20 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f6973fff640 1 -- 172.21.15.115:0/1214352389 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6980004050 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f6973fff640 1 -- 172.21.15.115:0/1214352389 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980005280 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.415+0000 7f697effd640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69740b2950 0x7f69740b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f698dcfa640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69740b2950 0x7f69740b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f697effd640 1 -- 172.21.15.115:0/2892852257 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f69780e2d00 con 0x7f6988108ba0 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f6996e12740 1 -- 172.21.15.115:0/1214352389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 msgr2=0x55fe3d0af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f698dcfa640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69740b2950 0x7f69740b4e60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6984006010 tx=0x7f6984005a40 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f6996e12740 1 --2- 172.21.15.115:0/1214352389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0af680 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6980009f90 tx=0x7f69800118d0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f6996e12740 1 -- 172.21.15.115:0/1214352389 shutdown_connections 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f6996e12740 1 --2- 172.21.15.115:0/1214352389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f6996e12740 1 --2- 172.21.15.115:0/1214352389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f6996e12740 1 --2- 172.21.15.115:0/1214352389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0af680 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f6996e12740 1 -- 172.21.15.115:0/1214352389 >> 172.21.15.115:0/1214352389 conn(0x55fe3cd00300 msgr2=0x55fe3d0abe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f6996e12740 1 -- 172.21.15.115:0/1214352389 shutdown_connections 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.416+0000 7f6996e12740 1 -- 172.21.15.115:0/1214352389 wait complete. 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f6996e12740 1 -- start start 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f6996e12740 1 Processor -- start 2024-05-09T05:51:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f6996e12740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f6996e12740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f6996e12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f6996e12740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55fe3d0bc7b0 con 0x55fe3cce32f0 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f6996e12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fe3cf9b9c0 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f6996e12740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fe3d0bc930 con 0x55fe3cceb4d0 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f6996e12740 10 client.0 _ll_register_callbacks cb 0x55fe3cfc8020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f698dcfa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f698dcfa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52934/0 (socket says 172.21.15.115:52934) 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f698dcfa640 1 -- 172.21.15.115:0/1272599028 learned_addr learned my addr 172.21.15.115:0/1272599028 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f698e4fb640 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.417+0000 7f698d4f9640 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f698dcfa640 1 -- 172.21.15.115:0/1272599028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cceb4d0 msgr2=0x55fe3d0bbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f698dcfa640 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0bbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f698dcfa640 1 -- 172.21.15.115:0/1272599028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 msgr2=0x55fe3d0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f698dcfa640 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f698dcfa640 1 -- 172.21.15.115:0/1272599028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3cfc21f0 con 0x55fe3cce32f0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f698e4fb640 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f698dcfa640 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bb770 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f6984009900 tx=0x7f6984009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b350ce542f0afe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f69717fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f69717fa640 1 -- 172.21.15.115:0/1272599028 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698404c030 con 0x55fe3cce32f0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f69717fa640 1 -- 172.21.15.115:0/1272599028 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f69840043d0 con 0x55fe3cce32f0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f6996e12740 1 -- 172.21.15.115:0/1272599028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6980004050 con 0x55fe3cce32f0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f69717fa640 1 -- 172.21.15.115:0/1272599028 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f6984059050 con 0x55fe3cce32f0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f6996e12740 10 client.18549 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f6996e12740 1 -- 172.21.15.115:0/1272599028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55fe3cd77790 con 0x55fe3cce32f0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.418+0000 7f69717fa640 1 -- 172.21.15.115:0/1272599028 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6980004050 con 0x55fe3cce32f0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f69717fa640 1 -- 172.21.15.115:0/1272599028 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7f6984048640 con 0x55fe3cce32f0 2024-05-09T05:51:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f6996e12740 10 client.18549 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f6996e12740 10 client.18549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f6996e12740 20 client.18549 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f6996e12740 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fe3cccad80 0x55fe3d0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f6996e12740 4 client.18549 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f6996e12740 1 -- 172.21.15.115:0/1272599028 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe3d0223d0 con 0x55fe3cccad80 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f698d4f9640 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fe3cccad80 0x55fe3d0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f69717fa640 10 client.18549 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.419+0000 7f698d4f9640 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fe3cccad80 0x55fe3d0cbfe0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf12be4072f2ae7 server_cookie=807987e353ad84f in_seq=0 out_seq=0 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f69717fa640 1 -- 172.21.15.115:0/1272599028 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe3d0223d0 con 0x55fe3cccad80 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f69717fa640 10 client.18549 handle_command_reply: tid=0 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 client.18549 shutdown 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 -- 172.21.15.115:0/1272599028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cce32f0 msgr2=0x55fe3d0bb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bb770 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f6984009900 tx=0x7f6984009260 comp rx=0 tx=0).stop 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 -- 172.21.15.115:0/1272599028 shutdown_connections 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fe3cccad80 0x55fe3d0cbfe0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f69717fa640 0 client.18549 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 --2- 172.21.15.115:0/1272599028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bb770 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 -- 172.21.15.115:0/1272599028 >> 172.21.15.115:0/1272599028 conn(0x55fe3cd00300 msgr2=0x55fe3d0aa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 -- 172.21.15.115:0/1272599028 shutdown_connections 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.420+0000 7f6996e12740 1 -- 172.21.15.115:0/1272599028 wait complete. 2024-05-09T05:51:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.421+0000 7f6996e12740 20 client.18549 trim_cache size 0 max 16384 2024-05-09T05:51:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.435+0000 7f6996e12740 1 Processor -- start 2024-05-09T05:51:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f6996e12740 1 -- start start 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f6996e12740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f6996e12740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f6996e12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f6996e12740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f69881b1230 con 0x55fe3cceb4d0 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f6996e12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69881b2dd0 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f6996e12740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69881b1900 con 0x55fe3cce32f0 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698d4f9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698d4f9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52936/0 (socket says 172.21.15.115:52936) 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698d4f9640 1 -- 172.21.15.115:0/3911471367 learned_addr learned my addr 172.21.15.115:0/3911471367 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698dcfa640 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698e4fb640 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698d4f9640 1 -- 172.21.15.115:0/3911471367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 msgr2=0x55fe3d0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698d4f9640 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698d4f9640 1 -- 172.21.15.115:0/3911471367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 msgr2=0x55fe3d0c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698d4f9640 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698d4f9640 1 -- 172.21.15.115:0/3911471367 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3cdee5f0 con 0x55fe3cceb4d0 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698dcfa640 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.436+0000 7f698d4f9640 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0c1bc0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f69780a5cd0 tx=0x7f69780eb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cbd1a5a1387c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f69737fe640 1 -- 172.21.15.115:0/3911471367 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978091040 con 0x55fe3cceb4d0 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f69737fe640 1 -- 172.21.15.115:0/3911471367 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69780b23f0 con 0x55fe3cceb4d0 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f69737fe640 1 -- 172.21.15.115:0/3911471367 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69780a53e0 con 0x55fe3cceb4d0 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f6996e12740 1 -- 172.21.15.115:0/3911471367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cceb4d0 msgr2=0x55fe3d0c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f6996e12740 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0c1bc0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f69780a5cd0 tx=0x7f69780eb040 comp rx=0 tx=0).stop 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f6996e12740 1 -- 172.21.15.115:0/3911471367 shutdown_connections 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f6996e12740 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f6996e12740 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0c1bc0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f6996e12740 1 --2- 172.21.15.115:0/3911471367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f6996e12740 1 -- 172.21.15.115:0/3911471367 >> 172.21.15.115:0/3911471367 conn(0x55fe3cd00300 msgr2=0x55fe3d0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f6996e12740 1 -- 172.21.15.115:0/3911471367 shutdown_connections 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.437+0000 7f6996e12740 1 -- 172.21.15.115:0/3911471367 wait complete. 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f6996e12740 1 -- start start 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f6996e12740 1 Processor -- start 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f6996e12740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f6996e12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f6996e12740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f6996e12740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f69881b2700 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f6996e12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69881b2030 con 0x55fe3cceb4d0 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f6996e12740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69881af230 con 0x55fe3cce32f0 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f6996e12740 10 client.0 _ll_register_callbacks cb 0x7f69800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f698e4fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f698e4fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52952/0 (socket says 172.21.15.115:52952) 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f698e4fb640 1 -- 172.21.15.115:0/1017562541 learned_addr learned my addr 172.21.15.115:0/1017562541 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.438+0000 7f698dcfa640 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f698d4f9640 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f698e4fb640 1 -- 172.21.15.115:0/1017562541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 msgr2=0x55fe3d0bf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f698e4fb640 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f698e4fb640 1 -- 172.21.15.115:0/1017562541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 msgr2=0x55fe3d0bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f698e4fb640 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0bfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f698e4fb640 1 -- 172.21.15.115:0/1017562541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3cfd8ed0 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f698d4f9640 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f698e4fb640 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c0180 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x55fe3d0ac7f0 tx=0x7f6980004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47748d874c1e4dea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f69717fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f69717fa640 1 -- 172.21.15.115:0/1017562541 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980004510 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f69717fa640 1 -- 172.21.15.115:0/1017562541 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f69800046b0 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f6996e12740 1 -- 172.21.15.115:0/1017562541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe3d01c240 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f69717fa640 1 -- 172.21.15.115:0/1017562541 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f69800048b0 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f6996e12740 10 client.18555 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f6996e12740 1 -- 172.21.15.115:0/1017562541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55fe3d1342f0 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.439+0000 7f69717fa640 1 -- 172.21.15.115:0/1017562541 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55fe3d01c240 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f69717fa640 1 -- 172.21.15.115:0/1017562541 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7f69800055f0 con 0x55fe3cd0c1b0 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f6996e12740 10 client.18555 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f6996e12740 10 client.18555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f6996e12740 20 client.18555 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f6996e12740 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fe3cca01c0 0x55fe3d1368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f6996e12740 4 client.18555 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f6996e12740 1 -- 172.21.15.115:0/1017562541 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe3cf97350 con 0x55fe3cca01c0 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f698dcfa640 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fe3cca01c0 0x55fe3d1368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f69717fa640 10 client.18555 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.440+0000 7f698dcfa640 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fe3cca01c0 0x55fe3d1368f0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928d66b73871f96a server_cookie=e163f9464f789990 in_seq=0 out_seq=0 2024-05-09T05:51:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f69717fa640 1 -- 172.21.15.115:0/1017562541 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fe3cf97350 con 0x55fe3cca01c0 2024-05-09T05:51:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f69717fa640 10 client.18555 handle_command_reply: tid=0 2024-05-09T05:51:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 client.18555 shutdown 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 -- 172.21.15.115:0/1017562541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 msgr2=0x55fe3d0c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c0180 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x55fe3d0ac7f0 tx=0x7f6980004190 comp rx=0 tx=0).stop 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 -- 172.21.15.115:0/1017562541 shutdown_connections 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55fe3cd0c1b0 0x55fe3d0c0180 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe3cceb4d0 0x55fe3d0bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55fe3cca01c0 0x55fe3d1368f0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 --2- 172.21.15.115:0/1017562541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fe3cce32f0 0x55fe3d0bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f69717fa640 0 client.18555 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 -- 172.21.15.115:0/1017562541 >> 172.21.15.115:0/1017562541 conn(0x55fe3cd00300 msgr2=0x55fe3d0ab680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 -- 172.21.15.115:0/1017562541 shutdown_connections 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 1 -- 172.21.15.115:0/1017562541 wait complete. 2024-05-09T05:51:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.449+0000 7f6996e12740 20 client.18555 trim_cache size 0 max 16384 2024-05-09T05:51:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:51:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:51:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 -- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69740b2950 msgr2=0x7f69740b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69740b2950 0x7f69740b4e60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6984006010 tx=0x7f6984005a40 comp rx=0 tx=0).stop 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 -- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 msgr2=0x7f698819bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 0x7f698819bba0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f697800e9f0 tx=0x7f697800eeb0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 -- 172.21.15.115:0/2892852257 shutdown_connections 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f69740b2950 0x7f69740b4e60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6988109040 0x7f698819c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6988108ba0 0x7f698819bba0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 --2- 172.21.15.115:0/2892852257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69881032f0 0x7f698819b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 -- 172.21.15.115:0/2892852257 >> 172.21.15.115:0/2892852257 conn(0x7f69880782e0 msgr2=0x7f6988100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 -- 172.21.15.115:0/2892852257 shutdown_connections 2024-05-09T05:51:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:17.450+0000 7f698ff5e640 1 -- 172.21.15.115:0/2892852257 wait complete. 2024-05-09T05:51:17.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:17 smithi115 ceph-mon[30349]: pgmap v412: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 MiB/s wr, 25 op/s 2024-05-09T05:51:17.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:17 smithi161 ceph-mon[33805]: pgmap v412: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 MiB/s wr, 25 op/s 2024-05-09T05:51:17.664 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:17 smithi195 ceph-mon[33844]: pgmap v412: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 MiB/s wr, 25 op/s 2024-05-09T05:51:19.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:19 smithi115 ceph-mon[30349]: pgmap v413: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-05-09T05:51:19.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:19 smithi161 ceph-mon[33805]: pgmap v413: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-05-09T05:51:19.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:19 smithi195 ceph-mon[33844]: pgmap v413: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-05-09T05:51:21.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:21 smithi115 ceph-mon[30349]: pgmap v414: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 16 op/s 2024-05-09T05:51:21.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:21 smithi161 ceph-mon[33805]: pgmap v414: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 16 op/s 2024-05-09T05:51:21.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:21 smithi195 ceph-mon[33844]: pgmap v414: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 16 op/s 2024-05-09T05:51:23.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:23 smithi115 ceph-mon[30349]: pgmap v415: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.4 MiB/s wr, 15 op/s 2024-05-09T05:51:23.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:23 smithi161 ceph-mon[33805]: pgmap v415: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.4 MiB/s wr, 15 op/s 2024-05-09T05:51:23.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:23 smithi195 ceph-mon[33844]: pgmap v415: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.4 MiB/s wr, 15 op/s 2024-05-09T05:51:25.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:25 smithi115 ceph-mon[30349]: pgmap v416: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.4 MiB/s wr, 20 op/s 2024-05-09T05:51:25.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:25 smithi161 ceph-mon[33805]: pgmap v416: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.4 MiB/s wr, 20 op/s 2024-05-09T05:51:25.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:25 smithi195 ceph-mon[33844]: pgmap v416: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.4 MiB/s wr, 20 op/s 2024-05-09T05:51:27.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:27 smithi115 ceph-mon[30349]: pgmap v417: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 392 KiB/s wr, 11 op/s 2024-05-09T05:51:27.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:27 smithi161 ceph-mon[33805]: pgmap v417: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 392 KiB/s wr, 11 op/s 2024-05-09T05:51:27.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:27 smithi195 ceph-mon[33844]: pgmap v417: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 392 KiB/s wr, 11 op/s 2024-05-09T05:51:29.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:29 smithi161 ceph-mon[33805]: pgmap v418: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 394 KiB/s wr, 15 op/s 2024-05-09T05:51:29.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:29 smithi195 ceph-mon[33844]: pgmap v418: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 394 KiB/s wr, 15 op/s 2024-05-09T05:51:29.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:29 smithi115 ceph-mon[30349]: pgmap v418: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 394 KiB/s wr, 15 op/s 2024-05-09T05:51:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.565+0000 7f8b8f429640 1 -- 172.21.15.115:0/1420324513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b8810b8a0 msgr2=0x7f8b8810dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.565+0000 7f8b8f429640 1 --2- 172.21.15.115:0/1420324513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b8810b8a0 0x7f8b8810dcc0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009900 tx=0x7f8b7802a480 comp rx=0 tx=0).stop 2024-05-09T05:51:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.565+0000 7f8b8f429640 1 -- 172.21.15.115:0/1420324513 shutdown_connections 2024-05-09T05:51:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.565+0000 7f8b8f429640 1 --2- 172.21.15.115:0/1420324513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8810e250 0x7f8b88110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.565+0000 7f8b8f429640 1 --2- 172.21.15.115:0/1420324513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b8810b8a0 0x7f8b8810dcc0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.565+0000 7f8b8f429640 1 --2- 172.21.15.115:0/1420324513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b88104e40 0x7f8b88105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.565+0000 7f8b8f429640 1 -- 172.21.15.115:0/1420324513 >> 172.21.15.115:0/1420324513 conn(0x7f8b880fe390 msgr2=0x7f8b881007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.565+0000 7f8b8f429640 1 -- 172.21.15.115:0/1420324513 shutdown_connections 2024-05-09T05:51:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.565+0000 7f8b8f429640 1 -- 172.21.15.115:0/1420324513 wait complete. 2024-05-09T05:51:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8f429640 1 Processor -- start 2024-05-09T05:51:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8f429640 1 -- start start 2024-05-09T05:51:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8f429640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b88104e40 0x7f8b8819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8f429640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8810b8a0 0x7f8b8819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8f429640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8810e250 0x7f8b8819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8f429640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88113d40 con 0x7f8b88104e40 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8f429640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88113bc0 con 0x7f8b8810e250 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8f429640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88113ec0 con 0x7f8b8810b8a0 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8d1c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b88104e40 0x7f8b8819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.566+0000 7f8b8d1c5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b88104e40 0x7f8b8819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40860/0 (socket says 172.21.15.115:40860) 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8d1c5640 1 -- 172.21.15.115:0/201478081 learned_addr learned my addr 172.21.15.115:0/201478081 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8810b8a0 0x7f8b8819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8810e250 0x7f8b8819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8d1c5640 1 -- 172.21.15.115:0/201478081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8810b8a0 msgr2=0x7f8b8819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8d1c5640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8810b8a0 0x7f8b8819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8d1c5640 1 -- 172.21.15.115:0/201478081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8810e250 msgr2=0x7f8b8819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8d1c5640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8810e250 0x7f8b8819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8d1c5640 1 -- 172.21.15.115:0/201478081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8819f5e0 con 0x7f8b88104e40 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8810e250 0x7f8b8819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8d1c5640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b88104e40 0x7f8b8819a800 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d980 tx=0x7f8b8400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b7e7fc640 1 -- 172.21.15.115:0/201478081 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84017070 con 0x7f8b88104e40 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b7e7fc640 1 -- 172.21.15.115:0/201478081 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b84009c30 con 0x7f8b88104e40 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8f429640 1 -- 172.21.15.115:0/201478081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8819f8d0 con 0x7f8b88104e40 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b7e7fc640 1 -- 172.21.15.115:0/201478081 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400f3c0 con 0x7f8b88104e40 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.567+0000 7f8b8f429640 1 -- 172.21.15.115:0/201478081 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8819fdf0 con 0x7f8b88104e40 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b963b7740 1 Processor -- start 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b963b7740 1 -- start start 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b7e7fc640 1 -- 172.21.15.115:0/201478081 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f8b84010460 con 0x7f8b88104e40 2024-05-09T05:51:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfe72f0 0x56230e3b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfef4d0 0x56230e3b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b963b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56230e3187c0 con 0x56230e0101b0 2024-05-09T05:51:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b963b7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56230e2b40e0 con 0x56230dfe72f0 2024-05-09T05:51:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b963b7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56230e314780 con 0x56230dfef4d0 2024-05-09T05:51:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b8d9c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b8d9c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40876/0 (socket says 172.21.15.115:40876) 2024-05-09T05:51:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2779895517 learned_addr learned my addr 172.21.15.115:0/2779895517 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfef4d0 0x56230e3b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.570+0000 7f8b8d1c5640 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfe72f0 0x56230e3b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2779895517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfef4d0 msgr2=0x56230e3b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfef4d0 0x56230e3b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2779895517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfe72f0 msgr2=0x56230e3b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfe72f0 0x56230e3b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2779895517 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56230e2dd7c0 con 0x56230e0101b0 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b8d1c5640 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfe72f0 0x56230e3b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b7e7fc640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8b740b2950 0x7f8b740b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3b3680 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f8b80008de0 tx=0x7f8b8000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176d6acbc3479ca7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b8d1c5640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8b740b2950 0x7f8b740b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b737fe640 1 -- 172.21.15.115:0/2779895517 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000f830 con 0x56230e0101b0 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b737fe640 1 -- 172.21.15.115:0/2779895517 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b8000de20 con 0x56230e0101b0 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b737fe640 1 -- 172.21.15.115:0/2779895517 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000e3b0 con 0x56230e0101b0 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.571+0000 7f8b7e7fc640 1 -- 172.21.15.115:0/201478081 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f8b840e25e0 con 0x7f8b88104e40 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 -- 172.21.15.115:0/2779895517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 msgr2=0x56230e3b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b8d1c5640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8b740b2950 0x7f8b740b4e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b840dfa50 tx=0x7f8b8400dfd0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3b3680 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f8b80008de0 tx=0x7f8b8000d710 comp rx=0 tx=0).stop 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 -- 172.21.15.115:0/2779895517 shutdown_connections 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfef4d0 0x56230e3b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfe72f0 0x56230e3b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2779895517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3b3680 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 -- 172.21.15.115:0/2779895517 >> 172.21.15.115:0/2779895517 conn(0x56230e004300 msgr2=0x56230e3afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 -- 172.21.15.115:0/2779895517 shutdown_connections 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 -- 172.21.15.115:0/2779895517 wait complete. 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.572+0000 7f8b963b7740 1 -- start start 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b963b7740 1 Processor -- start 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 0x56230e3c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b963b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56230e3c07b0 con 0x56230dfe72f0 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b963b7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56230e29f9c0 con 0x56230dfef4d0 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b963b7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56230e3c0930 con 0x56230e0101b0 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b963b7740 10 client.0 _ll_register_callbacks cb 0x56230e2cc020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3bdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40892/0 (socket says 172.21.15.115:40892) 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2700850541 learned_addr learned my addr 172.21.15.115:0/2700850541 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 0x56230e3c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2700850541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 msgr2=0x56230e3c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 0x56230e3c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2700850541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 msgr2=0x56230e3be2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3be2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2700850541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56230e2c61f0 con 0x56230dfe72f0 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3bdd80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8b8001a040 tx=0x7f8b80018b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de59c545b5e94ccd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.573+0000 7f8b70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2700850541 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80018690 con 0x56230dfe72f0 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2700850541 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f8b80020040 con 0x56230dfe72f0 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b963b7740 1 -- 172.21.15.115:0/2700850541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b8000de20 con 0x56230dfe72f0 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2700850541 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f8b80018830 con 0x56230dfe72f0 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b963b7740 10 client.18567 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b963b7740 1 -- 172.21.15.115:0/2700850541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56230e07b790 con 0x56230dfe72f0 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2700850541 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b8000de20 con 0x56230dfe72f0 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2700850541 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7f8b80023030 con 0x56230dfe72f0 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b963b7740 10 client.18567 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b963b7740 10 client.18567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b963b7740 20 client.18567 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56230dfced80 0x56230e3cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b963b7740 4 client.18567 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b963b7740 1 -- 172.21.15.115:0/2700850541 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56230e3263d0 con 0x56230dfced80 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.574+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56230dfced80 0x56230e3cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.575+0000 7f8b70ff9640 10 client.18567 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.575+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56230dfced80 0x56230e3cfe60 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5890a3399d91b61 server_cookie=6883f3413ba0473d in_seq=0 out_seq=0 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2700850541 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56230e3263d0 con 0x56230dfced80 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b70ff9640 10 client.18567 handle_command_reply: tid=0 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 client.18567 shutdown 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 -- 172.21.15.115:0/2700850541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 msgr2=0x56230e3bdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3bdd80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8b8001a040 tx=0x7f8b80018b10 comp rx=0 tx=0).stop 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 -- 172.21.15.115:0/2700850541 shutdown_connections 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56230dfced80 0x56230e3cfe60 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 0x56230e3c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b70ff9640 0 client.18567 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2700850541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3bdd80 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 -- 172.21.15.115:0/2700850541 >> 172.21.15.115:0/2700850541 conn(0x56230e004300 msgr2=0x56230e3ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 -- 172.21.15.115:0/2700850541 shutdown_connections 2024-05-09T05:51:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 1 -- 172.21.15.115:0/2700850541 wait complete. 2024-05-09T05:51:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.576+0000 7f8b963b7740 20 client.18567 trim_cache size 0 max 16384 2024-05-09T05:51:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b963b7740 1 Processor -- start 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b963b7740 1 -- start start 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfe72f0 0x56230e3b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b963b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b881b1250 con 0x56230e0101b0 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b963b7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8b881b2df0 con 0x56230dfef4d0 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b963b7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b881b1920 con 0x56230dfe72f0 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b8d9c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b8d9c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3c40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40906/0 (socket says 172.21.15.115:40906) 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.591+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2503259269 learned_addr learned my addr 172.21.15.115:0/2503259269 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2503259269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfe72f0 0x56230e3b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/2503259269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2503259269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfe72f0 msgr2=0x56230e3b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2503259269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfe72f0 0x56230e3b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2503259269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 msgr2=0x56230e3c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2503259269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2503259269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56230e0f25f0 con 0x56230e0101b0 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2503259269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3c40e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8b800086f0 tx=0x7f8b800021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0a91edcfbc09b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b72ffd640 1 -- 172.21.15.115:0/2503259269 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80018690 con 0x56230e0101b0 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b72ffd640 1 -- 172.21.15.115:0/2503259269 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b800199a0 con 0x56230e0101b0 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.592+0000 7f8b72ffd640 1 -- 172.21.15.115:0/2503259269 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000fac0 con 0x56230e0101b0 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 -- 172.21.15.115:0/2503259269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 msgr2=0x56230e3c40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2503259269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3c40e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8b800086f0 tx=0x7f8b800021d0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 -- 172.21.15.115:0/2503259269 shutdown_connections 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2503259269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230e0101b0 0x56230e3c40e0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2503259269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2503259269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230dfe72f0 0x56230e3b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 -- 172.21.15.115:0/2503259269 >> 172.21.15.115:0/2503259269 conn(0x56230e004300 msgr2=0x56230e3b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 -- 172.21.15.115:0/2503259269 shutdown_connections 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 -- 172.21.15.115:0/2503259269 wait complete. 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 -- start start 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 Processor -- start 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.593+0000 7f8b963b7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 0x56230e43a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b963b7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8b881b2720 con 0x56230dfe72f0 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b963b7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8b881b2050 con 0x56230dfef4d0 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b963b7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b881af250 con 0x56230e0101b0 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b963b7740 10 client.0 _ll_register_callbacks cb 0x7f8b78005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40910/0 (socket says 172.21.15.115:40910) 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2627730548 learned_addr learned my addr 172.21.15.115:0/2627730548 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 0x56230e43a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2627730548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 msgr2=0x56230e43a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 0x56230e43a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2627730548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 msgr2=0x56230e3c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 -- 172.21.15.115:0/2627730548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56230e2dced0 con 0x56230dfe72f0 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b8d9c6640 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3c91d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f8b80008860 tx=0x7f8b800198b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865b3d1086a16da1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2627730548 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000de20 con 0x56230dfe72f0 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b963b7740 1 -- 172.21.15.115:0/2627730548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b800199a0 con 0x56230dfe72f0 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2627730548 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f8b8003e040 con 0x56230dfe72f0 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2627730548 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f8b8000f630 con 0x56230dfe72f0 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b963b7740 10 client.18573 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.594+0000 7f8b963b7740 1 -- 172.21.15.115:0/2627730548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56230e43b3a0 con 0x56230dfe72f0 2024-05-09T05:51:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.595+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2627730548 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b800199a0 con 0x56230dfe72f0 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.595+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2627730548 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7f8b80035040 con 0x56230dfe72f0 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.595+0000 7f8b963b7740 10 client.18573 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.595+0000 7f8b963b7740 10 client.18573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.595+0000 7f8b963b7740 20 client.18573 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.595+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56230dfa41c0 0x56230e43da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.595+0000 7f8b963b7740 4 client.18573 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.595+0000 7f8b963b7740 1 -- 172.21.15.115:0/2627730548 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56230e320240 con 0x56230dfa41c0 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.595+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56230dfa41c0 0x56230e43da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.596+0000 7f8b8c9c4640 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56230dfa41c0 0x56230e43da80 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4e137322892c5b server_cookie=c383fe6c8f5c69e7 in_seq=0 out_seq=0 2024-05-09T05:51:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.596+0000 7f8b70ff9640 10 client.18573 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b70ff9640 1 -- 172.21.15.115:0/2627730548 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56230e320240 con 0x56230dfa41c0 2024-05-09T05:51:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b70ff9640 10 client.18573 handle_command_reply: tid=0 2024-05-09T05:51:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b963b7740 1 client.18573 shutdown 2024-05-09T05:51:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b963b7740 1 -- 172.21.15.115:0/2627730548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 msgr2=0x56230e3c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3c91d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f8b80008860 tx=0x7f8b800198b0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b963b7740 1 -- 172.21.15.115:0/2627730548 shutdown_connections 2024-05-09T05:51:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56230dfa41c0 0x56230e43da80 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56230e0101b0 0x56230e43a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b70ff9640 0 client.18573 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56230dfef4d0 0x56230e3c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b963b7740 1 --2- 172.21.15.115:0/2627730548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56230dfe72f0 0x56230e3c91d0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.603+0000 7f8b963b7740 1 -- 172.21.15.115:0/2627730548 >> 172.21.15.115:0/2627730548 conn(0x56230e004300 msgr2=0x56230e3bf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b963b7740 1 -- 172.21.15.115:0/2627730548 shutdown_connections 2024-05-09T05:51:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b963b7740 1 -- 172.21.15.115:0/2627730548 wait complete. 2024-05-09T05:51:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b963b7740 20 client.18573 trim_cache size 0 max 16384 2024-05-09T05:51:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:51:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:51:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:51:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 -- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8b740b2950 msgr2=0x7f8b740b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8b740b2950 0x7f8b740b4e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b840dfa50 tx=0x7f8b8400dfd0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 -- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b88104e40 msgr2=0x7f8b8819a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b88104e40 0x7f8b8819a800 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d980 tx=0x7f8b8400de40 comp rx=0 tx=0).stop 2024-05-09T05:51:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 -- 172.21.15.115:0/201478081 shutdown_connections 2024-05-09T05:51:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8b740b2950 0x7f8b740b4e60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b8810e250 0x7f8b8819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b8810b8a0 0x7f8b8819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 --2- 172.21.15.115:0/201478081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b88104e40 0x7f8b8819a800 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 -- 172.21.15.115:0/201478081 >> 172.21.15.115:0/201478081 conn(0x7f8b880fe390 msgr2=0x7f8b8810e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 -- 172.21.15.115:0/201478081 shutdown_connections 2024-05-09T05:51:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:30.604+0000 7f8b8f429640 1 -- 172.21.15.115:0/201478081 wait complete. 2024-05-09T05:51:31.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:31 smithi161 ceph-mon[33805]: pgmap v419: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-09T05:51:31.695 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:31 smithi115 ceph-mon[30349]: pgmap v419: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-09T05:51:31.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:31 smithi195 ceph-mon[33844]: pgmap v419: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-09T05:51:33.657 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:33 smithi161 ceph-mon[33805]: pgmap v420: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-09T05:51:33.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:33 smithi195 ceph-mon[33844]: pgmap v420: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-09T05:51:33.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:33 smithi115 ceph-mon[30349]: pgmap v420: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-09T05:51:35.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:35 smithi161 ceph-mon[33805]: pgmap v421: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 15 op/s 2024-05-09T05:51:35.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:35 smithi195 ceph-mon[33844]: pgmap v421: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 15 op/s 2024-05-09T05:51:35.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:35 smithi115 ceph-mon[30349]: pgmap v421: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 15 op/s 2024-05-09T05:51:36.402 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:51:36.655 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:51:37.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:37 smithi195 ceph-mon[33844]: pgmap v422: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-05-09T05:51:37.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:37 smithi115 ceph-mon[30349]: pgmap v422: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-05-09T05:51:37.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:37 smithi161 ceph-mon[33805]: pgmap v422: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-05-09T05:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.515+0000 7fc24001d700 1 -- 172.21.15.115:0/3766716196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 msgr2=0x7fc2381111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.515+0000 7fc24001d700 1 --2- 172.21.15.115:0/3766716196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 0x7fc2381111b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc23000b0d0 tx=0x7fc23000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.515+0000 7fc24001d700 1 -- 172.21.15.115:0/3766716196 shutdown_connections 2024-05-09T05:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.515+0000 7fc24001d700 1 --2- 172.21.15.115:0/3766716196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 0x7fc2381111b0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.515+0000 7fc24001d700 1 --2- 172.21.15.115:0/3766716196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2381045d0 0x7fc2381049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.515+0000 7fc24001d700 1 --2- 172.21.15.115:0/3766716196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc238108f00 0x7fc238109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.515+0000 7fc24001d700 1 -- 172.21.15.115:0/3766716196 >> 172.21.15.115:0/3766716196 conn(0x7fc2380ffe80 msgr2=0x7fc2381022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.515+0000 7fc24001d700 1 -- 172.21.15.115:0/3766716196 shutdown_connections 2024-05-09T05:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.515+0000 7fc24001d700 1 -- 172.21.15.115:0/3766716196 wait complete. 2024-05-09T05:51:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc24001d700 1 Processor -- start 2024-05-09T05:51:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc24001d700 1 -- start start 2024-05-09T05:51:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc24001d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2381045d0 0x7fc2381a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc24001d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc238108f00 0x7fc2381a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc24001d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 0x7fc23819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc24001d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc238114330 con 0x7fc238109850 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc24001d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc2381141b0 con 0x7fc2381045d0 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc24001d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2381144b0 con 0x7fc238108f00 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23e5ba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 0x7fc23819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23e5ba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 0x7fc23819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52168/0 (socket says 172.21.15.115:52168) 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23e5ba700 1 -- 172.21.15.115:0/1731182717 learned_addr learned my addr 172.21.15.115:0/1731182717 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23ddb9700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2381045d0 0x7fc2381a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23d5b8700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc238108f00 0x7fc2381a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23e5ba700 1 -- 172.21.15.115:0/1731182717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc238108f00 msgr2=0x7fc2381a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23e5ba700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc238108f00 0x7fc2381a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23e5ba700 1 -- 172.21.15.115:0/1731182717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2381045d0 msgr2=0x7fc2381a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23e5ba700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2381045d0 0x7fc2381a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.516+0000 7fc23e5ba700 1 -- 172.21.15.115:0/1731182717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23819b1b0 con 0x7fc238109850 2024-05-09T05:51:38.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.517+0000 7fc23e5ba700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 0x7fc23819aad0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc230009850 tx=0x7fc230007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:38.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.517+0000 7fc22effd700 1 -- 172.21.15.115:0/1731182717 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23001f070 con 0x7fc238109850 2024-05-09T05:51:38.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.517+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23819b440 con 0x7fc238109850 2024-05-09T05:51:38.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.517+0000 7fc22effd700 1 -- 172.21.15.115:0/1731182717 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc230005500 con 0x7fc238109850 2024-05-09T05:51:38.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.517+0000 7fc22effd700 1 -- 172.21.15.115:0/1731182717 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23001b8f0 con 0x7fc238109850 2024-05-09T05:51:38.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.517+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2381a7740 con 0x7fc238109850 2024-05-09T05:51:38.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.518+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc238105a80 con 0x7fc238109850 2024-05-09T05:51:38.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.520+0000 7fc22effd700 1 -- 172.21.15.115:0/1731182717 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc230022070 con 0x7fc238109850 2024-05-09T05:51:38.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.521+0000 7fc22effd700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc2240b29f0 0x7fc2240b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:38.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.521+0000 7fc23ddb9700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc2240b29f0 0x7fc2240b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:38.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.521+0000 7fc22effd700 1 -- 172.21.15.115:0/1731182717 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fc23000f020 con 0x7fc238109850 2024-05-09T05:51:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.521+0000 7fc23ddb9700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc2240b29f0 0x7fc2240b4f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc23819bc10 tx=0x7fc23400b410 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.525+0000 7fc22effd700 1 -- 172.21.15.115:0/1731182717 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc230029020 con 0x7fc238109850 2024-05-09T05:51:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.777+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc23819bfc0 con 0x7fc238109850 2024-05-09T05:51:38.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.778+0000 7fc22effd700 1 -- 172.21.15.115:0/1731182717 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11954 (secure 0 0 0) 0x7fc2300ab890 con 0x7fc238109850 2024-05-09T05:51:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:51:38.769 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:51:13.374288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:51:38.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc2240b29f0 msgr2=0x7fc2240b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:38.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc2240b29f0 0x7fc2240b4f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc23819bc10 tx=0x7fc23400b410 comp rx=0 tx=0).stop 2024-05-09T05:51:38.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 msgr2=0x7fc23819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:38.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 0x7fc23819aad0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc230009850 tx=0x7fc230007d90 comp rx=0 tx=0).stop 2024-05-09T05:51:38.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 shutdown_connections 2024-05-09T05:51:38.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc2240b29f0 0x7fc2240b4f00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2381045d0 0x7fc2381a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc238109850 0x7fc23819aad0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 --2- 172.21.15.115:0/1731182717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc238108f00 0x7fc2381a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:38.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.780+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 >> 172.21.15.115:0/1731182717 conn(0x7fc2380ffe80 msgr2=0x7fc2381086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:38.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.781+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 shutdown_connections 2024-05-09T05:51:38.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:38.781+0000 7fc24001d700 1 -- 172.21.15.115:0/1731182717 wait complete. 2024-05-09T05:51:38.772 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 57 2024-05-09T05:51:39.606 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id b9097790 --release --await' 2024-05-09T05:51:39.607 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id b9097790 --release --await 2024-05-09T05:51:39.857 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:51:39.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:39 smithi195 ceph-mon[33844]: pgmap v423: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-05-09T05:51:39.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:39 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1731182717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:51:39.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:39 smithi115 ceph-mon[30349]: pgmap v423: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-05-09T05:51:39.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:39 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1731182717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:51:39.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:39 smithi161 ceph-mon[33805]: pgmap v423: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-05-09T05:51:39.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:39 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1731182717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:51:41.745 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:41 smithi115 ceph-mon[30349]: pgmap v424: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-09T05:51:41.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 -- 172.21.15.115:0/2916998230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 msgr2=0x7fc8bc1038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:41.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/2916998230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 0x7fc8bc1038e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00099c0 tx=0x7fc8b0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 -- 172.21.15.115:0/2916998230 shutdown_connections 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/2916998230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8bc108750 0x7fc8bc1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/2916998230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 0x7fc8bc1038e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/2916998230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8bc107e00 0x7fc8bc108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 -- 172.21.15.115:0/2916998230 >> 172.21.15.115:0/2916998230 conn(0x7fc8bc0fef20 msgr2=0x7fc8bc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 -- 172.21.15.115:0/2916998230 shutdown_connections 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 -- 172.21.15.115:0/2916998230 wait complete. 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.761+0000 7fc8c30ec700 1 Processor -- start 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c30ec700 1 -- start start 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c30ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 0x7fc8bc199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c30ec700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8bc107e00 0x7fc8bc19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c30ec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8bc108750 0x7fc8bc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c30ec700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc113320 con 0x7fc8bc1034d0 2024-05-09T05:51:41.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c30ec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1131a0 con 0x7fc8bc108750 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c30ec700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1134a0 con 0x7fc8bc107e00 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c0e88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 0x7fc8bc199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c0e88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 0x7fc8bc199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52176/0 (socket says 172.21.15.115:52176) 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c0e88700 1 -- 172.21.15.115:0/4054147853 learned_addr learned my addr 172.21.15.115:0/4054147853 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c1689700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8bc108750 0x7fc8bc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8bc107e00 0x7fc8bc19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c0e88700 1 -- 172.21.15.115:0/4054147853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8bc107e00 msgr2=0x7fc8bc19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8bc107e00 0x7fc8bc19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c0e88700 1 -- 172.21.15.115:0/4054147853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8bc108750 msgr2=0x7fc8bc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8bc108750 0x7fc8bc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c0e88700 1 -- 172.21.15.115:0/4054147853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc19ec60 con 0x7fc8bc1034d0 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8bc107e00 0x7fc8bc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.762+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 0x7fc8bc199d90 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0000c00 tx=0x7fc8b001a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:41.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.763+0000 7fc8b9ffb700 1 -- 172.21.15.115:0/4054147853 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b001ab10 con 0x7fc8bc1034d0 2024-05-09T05:51:41.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.763+0000 7fc8c30ec700 1 -- 172.21.15.115:0/4054147853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc19eef0 con 0x7fc8bc1034d0 2024-05-09T05:51:41.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.763+0000 7fc8c30ec700 1 -- 172.21.15.115:0/4054147853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc19f3c0 con 0x7fc8bc1034d0 2024-05-09T05:51:41.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.763+0000 7fc8b9ffb700 1 -- 172.21.15.115:0/4054147853 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8b001acb0 con 0x7fc8bc1034d0 2024-05-09T05:51:41.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.763+0000 7fc8b9ffb700 1 -- 172.21.15.115:0/4054147853 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0004d50 con 0x7fc8bc1034d0 2024-05-09T05:51:41.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.765+0000 7fc8b9ffb700 1 -- 172.21.15.115:0/4054147853 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc8b000f460 con 0x7fc8bc1034d0 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.766+0000 7fc8b9ffb700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc8ac0b2a70 0x7fc8ac0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc8ac0b2a70 0x7fc8ac0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8b9ffb700 1 -- 172.21.15.115:0/4054147853 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fc8b00e5d90 con 0x7fc8bc1034d0 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8d1b02b80 1 Processor -- start 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8d1b02b80 1 -- start start 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc8ac0b2a70 0x7fc8ac0b4f80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc106460 tx=0x7fc8a8008040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf473c260 0x556cf47d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf47cfb50 0x556cf47cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556cf46f20f0 con 0x556cf473c260 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556cf46908f0 con 0x556cf47cfb50 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556cf46ee0b0 con 0x556cf46525a0 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8c0e88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf473c260 0x556cf47d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8c0e88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf473c260 0x556cf47d5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52178/0 (socket says 172.21.15.115:52178) 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8c0e88700 1 -- 172.21.15.115:0/774802517 learned_addr learned my addr 172.21.15.115:0/774802517 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.767+0000 7fc8c1689700 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf47cfb50 0x556cf47cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8c0e88700 1 -- 172.21.15.115:0/774802517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 msgr2=0x556cf47d5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47d5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8c0e88700 1 -- 172.21.15.115:0/774802517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf47cfb50 msgr2=0x556cf47cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:41.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf47cfb50 0x556cf47cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8c0e88700 1 -- 172.21.15.115:0/774802517 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf4688470 con 0x556cf473c260 2024-05-09T05:51:41.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8c1689700 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf47cfb50 0x556cf47cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:51:41.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf473c260 0x556cf47d5a90 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00e31b0 tx=0x7fc8b00ac7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b6ffa6ffbbc908 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:41.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8a6ffd700 1 -- 172.21.15.115:0/774802517 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00046e0 con 0x556cf473c260 2024-05-09T05:51:41.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8a6ffd700 1 -- 172.21.15.115:0/774802517 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8b00ad470 con 0x556cf473c260 2024-05-09T05:51:41.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.768+0000 7fc8a6ffd700 1 -- 172.21.15.115:0/774802517 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00b56b0 con 0x556cf473c260 2024-05-09T05:51:41.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/774802517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf473c260 msgr2=0x556cf47d5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:41.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf473c260 0x556cf47d5a90 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00e31b0 tx=0x7fc8b00ac7f0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/774802517 shutdown_connections 2024-05-09T05:51:41.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf47cfb50 0x556cf47cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf473c260 0x556cf47d5a90 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/774802517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/774802517 >> 172.21.15.115:0/774802517 conn(0x556cf46b27a0 msgr2=0x556cf47caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/774802517 shutdown_connections 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/774802517 wait complete. 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 -- start start 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.769+0000 7fc8d1b02b80 1 Processor -- start 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47df260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47df7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47d9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556cf456dca0 con 0x556cf47cfb50 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556cf46ac250 con 0x556cf473c260 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556cf47d0790 con 0x556cf46525a0 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8d1b02b80 10 client.0 _ll_register_callbacks cb 0x556cf45637c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c1689700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47d9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c1689700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47d9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52186/0 (socket says 172.21.15.115:52186) 2024-05-09T05:51:41.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c1689700 1 -- 172.21.15.115:0/4116367952 learned_addr learned my addr 172.21.15.115:0/4116367952 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47df7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47df260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c1689700 1 -- 172.21.15.115:0/4116367952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 msgr2=0x556cf47df260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c1689700 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47df260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c1689700 1 -- 172.21.15.115:0/4116367952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 msgr2=0x556cf47df7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c1689700 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47df7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c1689700 1 -- 172.21.15.115:0/4116367952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf456d590 con 0x556cf47cfb50 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47df7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8c1689700 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47d9660 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400aee0 tx=0x7fc8b4009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9462d38176cecbaf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc89bfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc89bfff700 1 -- 172.21.15.115:0/4116367952 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4007b30 con 0x556cf47cfb50 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc89bfff700 1 -- 172.21.15.115:0/4116367952 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fc8b4007cd0 con 0x556cf47cfb50 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.770+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/4116367952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b00ad470 con 0x556cf47cfb50 2024-05-09T05:51:41.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc89bfff700 1 -- 172.21.15.115:0/4116367952 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fc8b40153f0 con 0x556cf47cfb50 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc8d1b02b80 10 client.18591 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/4116367952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x556cf47e6e50 con 0x556cf47cfb50 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc89bfff700 1 -- 172.21.15.115:0/4116367952 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8b00ad470 con 0x556cf47cfb50 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc89bfff700 1 -- 172.21.15.115:0/4116367952 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7fc8b401e4d0 con 0x556cf47cfb50 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc8d1b02b80 10 client.18591 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc8d1b02b80 10 client.18591 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc8d1b02b80 20 client.18591 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556cf47e9530 0x556cf47eb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc8d1b02b80 4 client.18591 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/4116367952 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cf46ffad0 con 0x556cf47e9530 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.771+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556cf47e9530 0x556cf47eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.772+0000 7fc89bfff700 10 client.18591 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.772+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556cf47e9530 0x556cf47eb950 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d91375dbb7c8ce server_cookie=b0b4fdc1322ddf3c in_seq=0 out_seq=0 2024-05-09T05:51:41.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc89bfff700 1 -- 172.21.15.115:0/4116367952 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556cf46ffad0 con 0x556cf47e9530 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc89bfff700 10 client.18591 handle_command_reply: tid=0 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 client.18591 shutdown 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/4116367952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 msgr2=0x556cf47d9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47d9660 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400aee0 tx=0x7fc8b4009d30 comp rx=0 tx=0).stop 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/4116367952 shutdown_connections 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556cf47e9530 0x556cf47eb950 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47d9660 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47df260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/4116367952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47df7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/4116367952 >> 172.21.15.115:0/4116367952 conn(0x556cf46b27a0 msgr2=0x556cf47ca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc89bfff700 0 client.18591 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/4116367952 shutdown_connections 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/4116367952 wait complete. 2024-05-09T05:51:41.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.773+0000 7fc8d1b02b80 20 client.18591 trim_cache size 0 max 16384 2024-05-09T05:51:41.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8d1b02b80 1 Processor -- start 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8d1b02b80 1 -- start start 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47f1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47f1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a000ecc0 con 0x556cf47cfb50 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a0010860 con 0x556cf473c260 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a000f390 con 0x556cf46525a0 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8c1689700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8c1689700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47f1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52200/0 (socket says 172.21.15.115:52200) 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8c1689700 1 -- 172.21.15.115:0/1335407955 learned_addr learned my addr 172.21.15.115:0/1335407955 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/1335407955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/1335407955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8c1689700 1 -- 172.21.15.115:0/1335407955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 msgr2=0x556cf47d4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:41.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8c1689700 1 --2- 172.21.15.115:0/1335407955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47d4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8c1689700 1 -- 172.21.15.115:0/1335407955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 msgr2=0x556cf47f1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:41.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8c1689700 1 --2- 172.21.15.115:0/1335407955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47f1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.793+0000 7fc8c1689700 1 -- 172.21.15.115:0/1335407955 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf47cc620 con 0x556cf47cfb50 2024-05-09T05:51:41.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.794+0000 7fc8c1689700 1 --2- 172.21.15.115:0/1335407955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47f1690 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4002230 tx=0x7fc8b400aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f01269bad4597b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:41.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.794+0000 7fc8a67fc700 1 -- 172.21.15.115:0/1335407955 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b401ebf0 con 0x556cf47cfb50 2024-05-09T05:51:41.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.794+0000 7fc8a67fc700 1 -- 172.21.15.115:0/1335407955 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8b401a070 con 0x556cf47cfb50 2024-05-09T05:51:41.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.794+0000 7fc8a67fc700 1 -- 172.21.15.115:0/1335407955 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40157f0 con 0x556cf47cfb50 2024-05-09T05:51:41.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.794+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/1335407955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 msgr2=0x556cf47f1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:41.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.794+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/1335407955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47f1690 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4002230 tx=0x7fc8b400aba0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/1335407955 shutdown_connections 2024-05-09T05:51:41.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/1335407955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf47cfb50 0x556cf47f1690 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/1335407955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf46525a0 0x556cf47d4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/1335407955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47f1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/1335407955 >> 172.21.15.115:0/1335407955 conn(0x556cf46b27a0 msgr2=0x556cf47d4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/1335407955 shutdown_connections 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/1335407955 wait complete. 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 -- start start 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 Processor -- start 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf46525a0 0x556cf47e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf47cfb50 0x556cf47e3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a0010190 con 0x556cf46525a0 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a000fac0 con 0x556cf473c260 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a000cc80 con 0x556cf47cfb50 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8d1b02b80 10 client.0 _ll_register_callbacks cb 0x556cf45637c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:41.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.795+0000 7fc8c0e88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf46525a0 0x556cf47e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8c0e88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf46525a0 0x556cf47e54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52206/0 (socket says 172.21.15.115:52206) 2024-05-09T05:51:41.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8c0e88700 1 -- 172.21.15.115:0/3266560833 learned_addr learned my addr 172.21.15.115:0/3266560833 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:41.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8c1689700 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf47cfb50 0x556cf47e3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8c0e88700 1 -- 172.21.15.115:0/3266560833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf47cfb50 msgr2=0x556cf47e3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:41.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf47cfb50 0x556cf47e3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8c0e88700 1 -- 172.21.15.115:0/3266560833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 msgr2=0x556cf47e5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47e5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8c0e88700 1 -- 172.21.15.115:0/3266560833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf47e40b0 con 0x556cf46525a0 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8c0e88700 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf46525a0 0x556cf47e54d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00a9fe0 tx=0x7fc8b00ac770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a246db292b28df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc89bfff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc89bfff700 1 -- 172.21.15.115:0/3266560833 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00acd70 con 0x556cf46525a0 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc89bfff700 1 -- 172.21.15.115:0/3266560833 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fc8b00c4070 con 0x556cf46525a0 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/3266560833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b401a070 con 0x556cf46525a0 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc89bfff700 1 -- 172.21.15.115:0/3266560833 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fc8b00b1120 con 0x556cf46525a0 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8d1b02b80 10 client.18597 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/3266560833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x556cf48546e0 con 0x556cf46525a0 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.796+0000 7fc89bfff700 1 -- 172.21.15.115:0/3266560833 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8b401a070 con 0x556cf46525a0 2024-05-09T05:51:41.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc89bfff700 1 -- 172.21.15.115:0/3266560833 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7fc8b00acfd0 con 0x556cf46525a0 2024-05-09T05:51:41.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc8d1b02b80 10 client.18597 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:41.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc8d1b02b80 10 client.18597 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:51:41.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc8d1b02b80 20 client.18597 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:41.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556cf4854a70 0x556cf4856e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:41.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc8d1b02b80 4 client.18597 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b9097790", "release": true, "await": true}] 2024-05-09T05:51:41.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/3266560833 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b9097790", "release": true, "await": true}) -- 0x556cf46f9b70 con 0x556cf4854a70 2024-05-09T05:51:41.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556cf4854a70 0x556cf4856e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:41.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc89bfff700 10 client.18597 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:41.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.797+0000 7fc8bbfff700 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556cf4854a70 0x556cf4856e90 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec6244c49db0b6d server_cookie=b143c7adecc88551 in_seq=0 out_seq=0 2024-05-09T05:51:41.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc89bfff700 1 -- 172.21.15.115:0/3266560833 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556cf46f9b70 con 0x556cf4854a70 2024-05-09T05:51:41.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc89bfff700 10 client.18597 handle_command_reply: tid=0 2024-05-09T05:51:41.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 client.18597 shutdown 2024-05-09T05:51:41.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/3266560833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf46525a0 msgr2=0x556cf47e54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:41.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf46525a0 0x556cf47e54d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00a9fe0 tx=0x7fc8b00ac770 comp rx=0 tx=0).stop 2024-05-09T05:51:41.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/3266560833 shutdown_connections 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556cf4854a70 0x556cf4856e90 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556cf47cfb50 0x556cf47e3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556cf46525a0 0x556cf47e54d0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc89bfff700 0 client.18597 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 --2- 172.21.15.115:0/3266560833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cf473c260 0x556cf47e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/3266560833 >> 172.21.15.115:0/3266560833 conn(0x556cf46b27a0 msgr2=0x556cf47d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/3266560833 shutdown_connections 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 1 -- 172.21.15.115:0/3266560833 wait complete. 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.801+0000 7fc8d1b02b80 20 client.18597 trim_cache size 0 max 16384 2024-05-09T05:51:41.794 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 57, 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 40, 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "b9097790": { 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 40, 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:51:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:41.796 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 -- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc8ac0b2a70 msgr2=0x7fc8ac0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc8ac0b2a70 0x7fc8ac0b4f80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc106460 tx=0x7fc8a8008040 comp rx=0 tx=0).stop 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 -- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 msgr2=0x7fc8bc199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 0x7fc8bc199d90 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0000c00 tx=0x7fc8b001a840 comp rx=0 tx=0).stop 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 -- 172.21.15.115:0/4054147853 shutdown_connections 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc8ac0b2a70 0x7fc8ac0b4f80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8bc108750 0x7fc8bc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8bc1034d0 0x7fc8bc199d90 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 --2- 172.21.15.115:0/4054147853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8bc107e00 0x7fc8bc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 -- 172.21.15.115:0/4054147853 >> 172.21.15.115:0/4054147853 conn(0x7fc8bc0fef20 msgr2=0x7fc8bc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 -- 172.21.15.115:0/4054147853 shutdown_connections 2024-05-09T05:51:41.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:41.802+0000 7fc8c30ec700 1 -- 172.21.15.115:0/4054147853 wait complete. 2024-05-09T05:51:41.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:41 smithi195 ceph-mon[33844]: pgmap v424: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-09T05:51:41.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:41 smithi161 ceph-mon[33805]: pgmap v424: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-09T05:51:42.619 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:42 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:51:42.657 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b9097790', total seconds elapsed: 36.3 2024-05-09T05:51:42.658 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-09T05:51:42.658 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:51:42.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:42 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:51:42.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:42 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:51:42.963 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:51:43.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:43 smithi115 ceph-mon[30349]: pgmap v425: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-09T05:51:43.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:43 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:51:43.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:43 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:51:43.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:43 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:51:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.730+0000 7fddc69ce640 1 -- 172.21.15.115:0/1735533810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc0109040 msgr2=0x7fddc0102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.730+0000 7fddc69ce640 1 --2- 172.21.15.115:0/1735533810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc0109040 0x7fddc0102c80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009900 tx=0x7fdda802a410 comp rx=0 tx=0).stop 2024-05-09T05:51:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.730+0000 7fddc69ce640 1 -- 172.21.15.115:0/1735533810 shutdown_connections 2024-05-09T05:51:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.730+0000 7fddc69ce640 1 --2- 172.21.15.115:0/1735533810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddc01032f0 0x7fddc010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.730+0000 7fddc69ce640 1 --2- 172.21.15.115:0/1735533810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc0109040 0x7fddc0102c80 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.730+0000 7fddc69ce640 1 --2- 172.21.15.115:0/1735533810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0108ba0 0x7fddc0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.730+0000 7fddc69ce640 1 -- 172.21.15.115:0/1735533810 >> 172.21.15.115:0/1735533810 conn(0x7fddc00782e0 msgr2=0x7fddc0078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.730+0000 7fddc69ce640 1 -- 172.21.15.115:0/1735533810 shutdown_connections 2024-05-09T05:51:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.730+0000 7fddc69ce640 1 -- 172.21.15.115:0/1735533810 wait complete. 2024-05-09T05:51:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddc69ce640 1 Processor -- start 2024-05-09T05:51:43.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddc69ce640 1 -- start start 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddc69ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc01032f0 0x7fddc0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddc69ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0108ba0 0x7fddc0199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddc69ce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddc0109040 0x7fddc019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddc69ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0112c90 con 0x7fddc01032f0 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddc69ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0112b10 con 0x7fddc0108ba0 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddc69ce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0112e10 con 0x7fddc0109040 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddbffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc01032f0 0x7fddc0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddbffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc01032f0 0x7fddc0199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43400/0 (socket says 172.21.15.115:43400) 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddbffff640 1 -- 172.21.15.115:0/2950137957 learned_addr learned my addr 172.21.15.115:0/2950137957 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.731+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddc0109040 0x7fddc019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.726 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddb7fff640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0108ba0 0x7fddc0199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddbffff640 1 -- 172.21.15.115:0/2950137957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddc0109040 msgr2=0x7fddc019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddbffff640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddc0109040 0x7fddc019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddbffff640 1 -- 172.21.15.115:0/2950137957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0108ba0 msgr2=0x7fddc0199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddbffff640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0108ba0 0x7fddc0199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddbffff640 1 -- 172.21.15.115:0/2950137957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc019e5d0 con 0x7fddc01032f0 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddc0109040 0x7fddc019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddbffff640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc01032f0 0x7fddc0199770 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fddb000d770 tx=0x7fddb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddbdffb640 1 -- 172.21.15.115:0/2950137957 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0017070 con 0x7fddc01032f0 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddc69ce640 1 -- 172.21.15.115:0/2950137957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc019e8c0 con 0x7fddc01032f0 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddbdffb640 1 -- 172.21.15.115:0/2950137957 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddb0009c30 con 0x7fddc01032f0 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddbdffb640 1 -- 172.21.15.115:0/2950137957 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb000f620 con 0x7fddc01032f0 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.732+0000 7fddc69ce640 1 -- 172.21.15.115:0/2950137957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc01a6240 con 0x7fddc01032f0 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.735+0000 7fddcd881740 1 Processor -- start 2024-05-09T05:51:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.735+0000 7fddcd881740 1 -- start start 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddcd881740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d781b0 0x55a99411b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddcd881740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a99411bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddcd881740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d574d0 0x55a99411c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddcd881740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a9940807c0 con 0x55a993d4f2f0 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddcd881740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a99401c0e0 con 0x55a993d781b0 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddcd881740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a99407c780 con 0x55a993d574d0 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddbdffb640 1 -- 172.21.15.115:0/2950137957 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fddb001a070 con 0x7fddc01032f0 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddc4f6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d781b0 0x55a99411b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddc4f6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d781b0 0x55a99411b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:58864/0 (socket says 172.21.15.115:58864) 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddc4f6b640 1 -- 172.21.15.115:0/757850370 learned_addr learned my addr 172.21.15.115:0/757850370 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddbffff640 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a99411bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddb7fff640 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d574d0 0x55a99411c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddb7fff640 1 -- 172.21.15.115:0/757850370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d781b0 msgr2=0x55a99411b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddb7fff640 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d781b0 0x55a99411b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddb7fff640 1 -- 172.21.15.115:0/757850370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 msgr2=0x55a99411bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddb7fff640 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a99411bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddb7fff640 1 -- 172.21.15.115:0/757850370 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9940457c0 con 0x55a993d574d0 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.736+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d781b0 0x55a99411b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.737+0000 7fddb7fff640 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d574d0 0x55a99411c100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdda802aa00 tx=0x7fdda80040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29242e60683176e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.737+0000 7fddbdffb640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fddac0b2950 0x7fddac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.737+0000 7fddb67fc640 1 -- 172.21.15.115:0/757850370 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8035030 con 0x55a993d574d0 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.737+0000 7fddb67fc640 1 -- 172.21.15.115:0/757850370 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdda80054d0 con 0x55a993d574d0 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.737+0000 7fddb67fc640 1 -- 172.21.15.115:0/757850370 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda80035f0 con 0x55a993d574d0 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.737+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fddac0b2950 0x7fddac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.737+0000 7fddbdffb640 1 -- 172.21.15.115:0/2950137957 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fddb00e2450 con 0x7fddc01032f0 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddcd881740 1 -- 172.21.15.115:0/757850370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d574d0 msgr2=0x55a99411c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fddac0b2950 0x7fddac0b4e60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55a9941187f0 tx=0x7fddb8009500 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddcd881740 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d574d0 0x55a99411c100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdda802aa00 tx=0x7fdda80040f0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddcd881740 1 -- 172.21.15.115:0/757850370 shutdown_connections 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddcd881740 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d574d0 0x55a99411c100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddcd881740 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a99411bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddcd881740 1 --2- 172.21.15.115:0/757850370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d781b0 0x55a99411b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddcd881740 1 -- 172.21.15.115:0/757850370 >> 172.21.15.115:0/757850370 conn(0x55a993d6c300 msgr2=0x55a994117e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddcd881740 1 -- 172.21.15.115:0/757850370 shutdown_connections 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.738+0000 7fddcd881740 1 -- 172.21.15.115:0/757850370 wait complete. 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.739+0000 7fddcd881740 1 -- start start 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.739+0000 7fddcd881740 1 Processor -- start 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.739+0000 7fddcd881740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 0x55a99412bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.739+0000 7fddcd881740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d574d0 0x55a99412c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.739+0000 7fddcd881740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d781b0 0x55a99412a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.739+0000 7fddcd881740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a99412cb60 con 0x55a993d781b0 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.739+0000 7fddcd881740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a9940079c0 con 0x55a993d574d0 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.739+0000 7fddcd881740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a99412cce0 con 0x55a993d4f2f0 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.739+0000 7fddcd881740 10 client.0 _ll_register_callbacks cb 0x55a994034020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddc4f6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d781b0 0x55a99412a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddc4f6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d781b0 0x55a99412a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43418/0 (socket says 172.21.15.115:43418) 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddc4f6b640 1 -- 172.21.15.115:0/714798745 learned_addr learned my addr 172.21.15.115:0/714798745 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddb7fff640 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d574d0 0x55a99412c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddbffff640 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 0x55a99412bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddc4f6b640 1 -- 172.21.15.115:0/714798745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 msgr2=0x55a99412bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 0x55a99412bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddc4f6b640 1 -- 172.21.15.115:0/714798745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d574d0 msgr2=0x55a99412c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d574d0 0x55a99412c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddc4f6b640 1 -- 172.21.15.115:0/714798745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99402e1f0 con 0x55a993d781b0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddb7fff640 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d574d0 0x55a99412c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d781b0 0x55a99412a000 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fddb80047a0 tx=0x7fddb8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264492b6a6e676ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fdd9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/714798745 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb802d060 con 0x55a993d781b0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/714798745 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fddb8017030 con 0x55a993d781b0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.740+0000 7fddcd881740 1 -- 172.21.15.115:0/714798745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda80054d0 con 0x55a993d781b0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/714798745 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fddb8034050 con 0x55a993d781b0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fddcd881740 10 client.18609 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fddcd881740 1 -- 172.21.15.115:0/714798745 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55a993de3790 con 0x55a993d781b0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/714798745 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdda80054d0 con 0x55a993d781b0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/714798745 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7fddb8028770 con 0x55a993d781b0 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fddcd881740 10 client.18609 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fddcd881740 10 client.18609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fddcd881740 20 client.18609 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fddcd881740 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a993d36d80 0x55a99413c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fddcd881740 4 client.18609 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.741+0000 7fddcd881740 1 -- 172.21.15.115:0/714798745 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a99408e3d0 con 0x55a993d36d80 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.742+0000 7fddbffff640 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a993d36d80 0x55a99413c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.742+0000 7fddbffff640 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a993d36d80 0x55a99413c3a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2b9b38e90a6042 server_cookie=92cf11eae608d110 in_seq=0 out_seq=0 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.742+0000 7fdd9f7fe640 10 client.18609 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/714798745 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a99408e3d0 con 0x55a993d36d80 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fdd9f7fe640 10 client.18609 handle_command_reply: tid=0 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 client.18609 shutdown 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 -- 172.21.15.115:0/714798745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d781b0 msgr2=0x55a99412a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d781b0 0x55a99412a000 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fddb80047a0 tx=0x7fddb8004ab0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 -- 172.21.15.115:0/714798745 shutdown_connections 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d781b0 0x55a99412a000 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d574d0 0x55a99412c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a993d36d80 0x55a99413c3a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 --2- 172.21.15.115:0/714798745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 0x55a99412bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fdd9f7fe640 0 client.18609 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 -- 172.21.15.115:0/714798745 >> 172.21.15.115:0/714798745 conn(0x55a993d6c300 msgr2=0x55a994116700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 -- 172.21.15.115:0/714798745 shutdown_connections 2024-05-09T05:51:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 1 -- 172.21.15.115:0/714798745 wait complete. 2024-05-09T05:51:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.743+0000 7fddcd881740 20 client.18609 trim_cache size 0 max 16384 2024-05-09T05:51:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.759+0000 7fddcd881740 1 Processor -- start 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.759+0000 7fddcd881740 1 -- start start 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddcd881740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d6c300 0x55a99411fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddcd881740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a994127900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddcd881740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a994141b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddcd881740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fddc01aef70 con 0x55a993d4f2f0 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddcd881740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fddc01b0b10 con 0x55a993d36d80 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddcd881740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fddc01af640 con 0x55a993d6c300 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddc4f6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a994141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddc4f6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a994141b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43422/0 (socket says 172.21.15.115:43422) 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddc4f6b640 1 -- 172.21.15.115:0/2902674690 learned_addr learned my addr 172.21.15.115:0/2902674690 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddbffff640 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d6c300 0x55a99411fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddb7fff640 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a994127900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddbffff640 1 -- 172.21.15.115:0/2902674690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 msgr2=0x55a994127900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddbffff640 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a994127900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddbffff640 1 -- 172.21.15.115:0/2902674690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 msgr2=0x55a994141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddbffff640 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a994141b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddbffff640 1 -- 172.21.15.115:0/2902674690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a993e5a5f0 con 0x55a993d6c300 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddb7fff640 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a994127900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a994141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddbffff640 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d6c300 0x55a99411fa50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fddb00df810 tx=0x7fddb00e8060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a9143f043c37afd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.760+0000 7fddb5ffb640 1 -- 172.21.15.115:0/2902674690 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0021040 con 0x55a993d6c300 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.761+0000 7fddb5ffb640 1 -- 172.21.15.115:0/2902674690 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddb00a8eb0 con 0x55a993d6c300 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.761+0000 7fddb5ffb640 1 -- 172.21.15.115:0/2902674690 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb00b2200 con 0x55a993d6c300 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.761+0000 7fddcd881740 1 -- 172.21.15.115:0/2902674690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d6c300 msgr2=0x55a99411fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.761+0000 7fddcd881740 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d6c300 0x55a99411fa50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fddb00df810 tx=0x7fddb00e8060 comp rx=0 tx=0).stop 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.762+0000 7fddcd881740 1 -- 172.21.15.115:0/2902674690 shutdown_connections 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.762+0000 7fddcd881740 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d4f2f0 0x55a994141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.762+0000 7fddcd881740 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a994127900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.762+0000 7fddcd881740 1 --2- 172.21.15.115:0/2902674690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d6c300 0x55a99411fa50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.762+0000 7fddcd881740 1 -- 172.21.15.115:0/2902674690 >> 172.21.15.115:0/2902674690 conn(0x55a99413c710 msgr2=0x55a99411f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.762+0000 7fddcd881740 1 -- 172.21.15.115:0/2902674690 shutdown_connections 2024-05-09T05:51:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.762+0000 7fddcd881740 1 -- 172.21.15.115:0/2902674690 wait complete. 2024-05-09T05:51:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.762+0000 7fddcd881740 1 -- start start 2024-05-09T05:51:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.762+0000 7fddcd881740 1 Processor -- start 2024-05-09T05:51:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddcd881740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a99412fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddcd881740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 0x55a994130200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddcd881740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d6c300 0x55a994130740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddcd881740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fddc01b0440 con 0x55a993d6c300 2024-05-09T05:51:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddcd881740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fddc01afd70 con 0x55a993d36d80 2024-05-09T05:51:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddcd881740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fddc01acf70 con 0x55a993d4f2f0 2024-05-09T05:51:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddbffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a99412fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddcd881740 10 client.0 _ll_register_callbacks cb 0x7fddb8003c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddbffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a99412fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:58892/0 (socket says 172.21.15.115:58892) 2024-05-09T05:51:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddbffff640 1 -- 172.21.15.115:0/1833108252 learned_addr learned my addr 172.21.15.115:0/1833108252 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddc4f6b640 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d6c300 0x55a994130740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddb7fff640 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 0x55a994130200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddbffff640 1 -- 172.21.15.115:0/1833108252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 msgr2=0x55a994130200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddbffff640 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 0x55a994130200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddbffff640 1 -- 172.21.15.115:0/1833108252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d6c300 msgr2=0x55a994130740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddbffff640 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d6c300 0x55a994130740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddbffff640 1 -- 172.21.15.115:0/1833108252 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a994044ed0 con 0x55a993d36d80 2024-05-09T05:51:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fddbffff640 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a99412fcc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fddb00a9330 tx=0x7fddb00e23e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbe25afda4011556 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fdd9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:51:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.763+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/1833108252 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0014020 con 0x55a993d36d80 2024-05-09T05:51:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/1833108252 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fddb00a5630 con 0x55a993d36d80 2024-05-09T05:51:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fddcd881740 1 -- 172.21.15.115:0/1833108252 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb00a8eb0 con 0x55a993d36d80 2024-05-09T05:51:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/1833108252 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fddb0021030 con 0x55a993d36d80 2024-05-09T05:51:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fddcd881740 10 client.27821 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fddcd881740 1 -- 172.21.15.115:0/1833108252 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55a9941a4830 con 0x55a993d36d80 2024-05-09T05:51:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/1833108252 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fddb00c7070 con 0x55a993d36d80 2024-05-09T05:51:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/1833108252 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7fddb00ca030 con 0x55a993d36d80 2024-05-09T05:51:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fddcd881740 10 client.27821 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fddcd881740 10 client.27821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fddcd881740 20 client.27821 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fddcd881740 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a993d574d0 0x55a9941a7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fddcd881740 4 client.27821 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:51:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.764+0000 7fddcd881740 1 -- 172.21.15.115:0/1833108252 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fddb00a8eb0 con 0x55a993d574d0 2024-05-09T05:51:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.765+0000 7fddb7fff640 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a993d574d0 0x55a9941a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.765+0000 7fdd9f7fe640 10 client.27821 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.765+0000 7fddb7fff640 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a993d574d0 0x55a9941a7000 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=937ad6b67d63d713 server_cookie=41fde8e68194906b in_seq=0 out_seq=0 2024-05-09T05:51:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fdd9f7fe640 1 -- 172.21.15.115:0/1833108252 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fddb00a8eb0 con 0x55a993d574d0 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fdd9f7fe640 10 client.27821 handle_command_reply: tid=0 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 client.27821 shutdown 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 -- 172.21.15.115:0/1833108252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 msgr2=0x55a99412fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a99412fcc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fddb00a9330 tx=0x7fddb00e23e0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 -- 172.21.15.115:0/1833108252 shutdown_connections 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a993d6c300 0x55a994130740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a993d574d0 0x55a9941a7000 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a993d4f2f0 0x55a994130200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 --2- 172.21.15.115:0/1833108252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a993d36d80 0x55a99412fcc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fdd9f7fe640 0 client.27821 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 -- 172.21.15.115:0/1833108252 >> 172.21.15.115:0/1833108252 conn(0x55a99413c710 msgr2=0x55a994116700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 -- 172.21.15.115:0/1833108252 shutdown_connections 2024-05-09T05:51:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 1 -- 172.21.15.115:0/1833108252 wait complete. 2024-05-09T05:51:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.769+0000 7fddcd881740 20 client.27821 trim_cache size 0 max 16384 2024-05-09T05:51:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:51:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:51:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:51:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 -- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fddac0b2950 msgr2=0x7fddac0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fddac0b2950 0x7fddac0b4e60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55a9941187f0 tx=0x7fddb8009500 comp rx=0 tx=0).stop 2024-05-09T05:51:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 -- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc01032f0 msgr2=0x7fddc0199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc01032f0 0x7fddc0199770 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fddb000d770 tx=0x7fddb000dc30 comp rx=0 tx=0).stop 2024-05-09T05:51:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 -- 172.21.15.115:0/2950137957 shutdown_connections 2024-05-09T05:51:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fddac0b2950 0x7fddac0b4e60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddc0109040 0x7fddc019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0108ba0 0x7fddc0199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 --2- 172.21.15.115:0/2950137957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fddc01032f0 0x7fddc0199770 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 -- 172.21.15.115:0/2950137957 >> 172.21.15.115:0/2950137957 conn(0x7fddc00782e0 msgr2=0x7fddc01012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 -- 172.21.15.115:0/2950137957 shutdown_connections 2024-05-09T05:51:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:43.770+0000 7fddc69ce640 1 -- 172.21.15.115:0/2950137957 wait complete. 2024-05-09T05:51:43.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:43 smithi195 ceph-mon[33844]: pgmap v425: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-09T05:51:43.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:43 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:51:43.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:43 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:51:43.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:43 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:51:43.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:43 smithi161 ceph-mon[33805]: pgmap v425: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-05-09T05:51:43.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:43 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:51:43.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:43 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:51:43.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:43 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:51:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.699+0000 7fc5d34b7700 1 -- 172.21.15.115:0/2510768289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc108760 msgr2=0x7fc5cc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.699+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/2510768289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc108760 0x7fc5cc1100c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400b0d0 tx=0x7fc5c400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.699+0000 7fc5d34b7700 1 -- 172.21.15.115:0/2510768289 shutdown_connections 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.699+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/2510768289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc108760 0x7fc5cc1100c0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.699+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/2510768289 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc1034e0 0x7fc5cc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.699+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/2510768289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5cc107e10 0x7fc5cc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.699+0000 7fc5d34b7700 1 -- 172.21.15.115:0/2510768289 >> 172.21.15.115:0/2510768289 conn(0x7fc5cc0fef30 msgr2=0x7fc5cc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.699+0000 7fc5d34b7700 1 -- 172.21.15.115:0/2510768289 shutdown_connections 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.699+0000 7fc5d34b7700 1 -- 172.21.15.115:0/2510768289 wait complete. 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.700+0000 7fc5d34b7700 1 Processor -- start 2024-05-09T05:51:44.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.700+0000 7fc5d34b7700 1 -- start start 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.700+0000 7fc5d34b7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc1034e0 0x7fc5cc199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.700+0000 7fc5d34b7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5cc107e10 0x7fc5cc19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.700+0000 7fc5d34b7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc108760 0x7fc5cc19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.700+0000 7fc5d34b7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc113240 con 0x7fc5cc1034e0 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.700+0000 7fc5d34b7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc1130c0 con 0x7fc5cc107e10 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.700+0000 7fc5d34b7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc1133c0 con 0x7fc5cc108760 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1253700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc1034e0 0x7fc5cc199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1253700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc1034e0 0x7fc5cc199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43442/0 (socket says 172.21.15.115:43442) 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1253700 1 -- 172.21.15.115:0/3077956890 learned_addr learned my addr 172.21.15.115:0/3077956890 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1a54700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc108760 0x7fc5cc19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d0a52700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5cc107e10 0x7fc5cc19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1253700 1 -- 172.21.15.115:0/3077956890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc108760 msgr2=0x7fc5cc19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1253700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc108760 0x7fc5cc19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1253700 1 -- 172.21.15.115:0/3077956890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5cc107e10 msgr2=0x7fc5cc19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1253700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5cc107e10 0x7fc5cc19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1253700 1 -- 172.21.15.115:0/3077956890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc19daa0 con 0x7fc5cc1034e0 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d1253700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc1034e0 0x7fc5cc199cb0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8010bb0 tx=0x7fc5c8010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5c27fc700 1 -- 172.21.15.115:0/3077956890 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c801d070 con 0x7fc5cc1034e0 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc19dd90 con 0x7fc5cc1034e0 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5c27fc700 1 -- 172.21.15.115:0/3077956890 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5c8018410 con 0x7fc5cc1034e0 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5c27fc700 1 -- 172.21.15.115:0/3077956890 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8016570 con 0x7fc5cc1034e0 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.701+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc06c040 con 0x7fc5cc1034e0 2024-05-09T05:51:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.703+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc104990 con 0x7fc5cc1034e0 2024-05-09T05:51:44.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.705+0000 7fc5c27fc700 1 -- 172.21.15.115:0/3077956890 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fc5c8020070 con 0x7fc5cc1034e0 2024-05-09T05:51:44.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.705+0000 7fc5c27fc700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc5b80b29f0 0x7fc5b80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:44.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.706+0000 7fc5d0a52700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc5b80b29f0 0x7fc5b80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:44.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.706+0000 7fc5c27fc700 1 -- 172.21.15.115:0/3077956890 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fc5c80e42a0 con 0x7fc5cc1034e0 2024-05-09T05:51:44.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.706+0000 7fc5d0a52700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc5b80b29f0 0x7fc5b80b4f00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc106750 tx=0x7fc5bc009260 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:44.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.709+0000 7fc5c27fc700 1 -- 172.21.15.115:0/3077956890 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5c80ab980 con 0x7fc5cc1034e0 2024-05-09T05:51:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.985+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5cc19afd0 con 0x7fc5cc1034e0 2024-05-09T05:51:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.986+0000 7fc5c27fc700 1 -- 172.21.15.115:0/3077956890 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11954 (secure 0 0 0) 0x7fc5c80ab280 con 0x7fc5cc1034e0 2024-05-09T05:51:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:51:44.976 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:51:13.374288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:51:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.989+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc5b80b29f0 msgr2=0x7fc5b80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.989+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc5b80b29f0 0x7fc5b80b4f00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc106750 tx=0x7fc5bc009260 comp rx=0 tx=0).stop 2024-05-09T05:51:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.989+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc1034e0 msgr2=0x7fc5cc199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.989+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc1034e0 0x7fc5cc199cb0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8010bb0 tx=0x7fc5c8010ec0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.990+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 shutdown_connections 2024-05-09T05:51:44.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.990+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fc5b80b29f0 0x7fc5b80b4f00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.990+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc108760 0x7fc5cc19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.990+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5cc1034e0 0x7fc5cc199cb0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.990+0000 7fc5d34b7700 1 --2- 172.21.15.115:0/3077956890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5cc107e10 0x7fc5cc19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:44.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.990+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 >> 172.21.15.115:0/3077956890 conn(0x7fc5cc0fef30 msgr2=0x7fc5cc1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:44.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.990+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 shutdown_connections 2024-05-09T05:51:44.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:44.990+0000 7fc5d34b7700 1 -- 172.21.15.115:0/3077956890 wait complete. 2024-05-09T05:51:44.980 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 57 2024-05-09T05:51:45.723 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:45 smithi115 ceph-mon[30349]: pgmap v426: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.0 MiB/s wr, 15 op/s 2024-05-09T05:51:45.723 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:45 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3077956890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:51:45.752 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-09T05:51:45.753 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-09T05:51:45.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:45 smithi195 ceph-mon[33844]: pgmap v426: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.0 MiB/s wr, 15 op/s 2024-05-09T05:51:45.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:45 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3077956890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:51:45.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:45 smithi161 ceph-mon[33805]: pgmap v426: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.0 MiB/s wr, 15 op/s 2024-05-09T05:51:45.907 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:45 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3077956890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:51:46.023 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:51:47.631 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:47 smithi115 ceph-mon[30349]: pgmap v427: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 MiB/s wr, 9 op/s 2024-05-09T05:51:47.709 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:47 smithi161 ceph-mon[33805]: pgmap v427: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 MiB/s wr, 9 op/s 2024-05-09T05:51:47.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:47 smithi195 ceph-mon[33844]: pgmap v427: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 MiB/s wr, 9 op/s 2024-05-09T05:51:47.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.930+0000 7fa9bd57d700 1 -- 172.21.15.115:0/3497076764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b81034c0 msgr2=0x7fa9b81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:47.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.930+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/3497076764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b81034c0 0x7fa9b81038d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80099c0 tx=0x7fa9a8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.930+0000 7fa9bd57d700 1 -- 172.21.15.115:0/3497076764 shutdown_connections 2024-05-09T05:51:47.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.930+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/3497076764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b8108740 0x7fa9b81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.930+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/3497076764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b81034c0 0x7fa9b81038d0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.930+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/3497076764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b8107df0 0x7fa9b8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.930+0000 7fa9bd57d700 1 -- 172.21.15.115:0/3497076764 >> 172.21.15.115:0/3497076764 conn(0x7fa9b80fedc0 msgr2=0x7fa9b81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:47.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.930+0000 7fa9bd57d700 1 -- 172.21.15.115:0/3497076764 shutdown_connections 2024-05-09T05:51:47.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.930+0000 7fa9bd57d700 1 -- 172.21.15.115:0/3497076764 wait complete. 2024-05-09T05:51:47.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9bd57d700 1 Processor -- start 2024-05-09T05:51:47.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9bd57d700 1 -- start start 2024-05-09T05:51:47.922 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9bd57d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b81034c0 0x7fa9b819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9bd57d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b8107df0 0x7fa9b819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9bd57d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b8108740 0x7fa9b8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9bd57d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b8113220 con 0x7fa9b8107df0 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9bd57d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81130a0 con 0x7fa9b81034c0 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9bd57d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81133a0 con 0x7fa9b8108740 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9b67fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b8107df0 0x7fa9b819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9b67fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b8107df0 0x7fa9b819fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43460/0 (socket says 172.21.15.115:43460) 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9b67fc700 1 -- 172.21.15.115:0/2009444072 learned_addr learned my addr 172.21.15.115:0/2009444072 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b8108740 0x7fa9b8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.931+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b81034c0 0x7fa9b819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa9b67fc700 1 -- 172.21.15.115:0/2009444072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b8108740 msgr2=0x7fa9b8199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b8108740 0x7fa9b8199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa9b67fc700 1 -- 172.21.15.115:0/2009444072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b81034c0 msgr2=0x7fa9b819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:47.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b81034c0 0x7fa9b819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa9b67fc700 1 -- 172.21.15.115:0/2009444072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b819a0f0 con 0x7fa9b8107df0 2024-05-09T05:51:47.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b8108740 0x7fa9b8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:47.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b8107df0 0x7fa9b819fd60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00e970 tx=0x7fa9ac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:47.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa99ffff700 1 -- 172.21.15.115:0/2009444072 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac004a30 con 0x7fa9b8107df0 2024-05-09T05:51:47.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa9bd57d700 1 -- 172.21.15.115:0/2009444072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b819a3e0 con 0x7fa9b8107df0 2024-05-09T05:51:47.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa99ffff700 1 -- 172.21.15.115:0/2009444072 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9ac00cc70 con 0x7fa9b8107df0 2024-05-09T05:51:47.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa99ffff700 1 -- 172.21.15.115:0/2009444072 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac005520 con 0x7fa9b8107df0 2024-05-09T05:51:47.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.932+0000 7fa9bd57d700 1 -- 172.21.15.115:0/2009444072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b81a66b0 con 0x7fa9b8107df0 2024-05-09T05:51:47.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.935+0000 7fa99ffff700 1 -- 172.21.15.115:0/2009444072 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa9ac00c7e0 con 0x7fa9b8107df0 2024-05-09T05:51:47.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.936+0000 7fa99ffff700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9a00b29f0 0x7fa9a00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.936+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9a00b29f0 0x7fa9a00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.936+0000 7fa99ffff700 1 -- 172.21.15.115:0/2009444072 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fa9ac0e2b80 con 0x7fa9b8107df0 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.936+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9a00b29f0 0x7fa9a00b4f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8000c00 tx=0x7fa9a801a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9cbf93b80 1 Processor -- start 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9cbf93b80 1 -- start start 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a23fe4630 0x555a2416a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 0x555a24164200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555a24084180 con 0x555a23fe4630 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555a24022980 con 0x555a24163df0 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555a24080140 con 0x555a240ce2f0 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9b67fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a23fe4630 0x555a2416a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9b67fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a23fe4630 0x555a2416a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43464/0 (socket says 172.21.15.115:43464) 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9b67fc700 1 -- 172.21.15.115:0/885498044 learned_addr learned my addr 172.21.15.115:0/885498044 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 0x555a24164200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9b67fc700 1 -- 172.21.15.115:0/885498044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 msgr2=0x555a24169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9b67fc700 1 -- 172.21.15.115:0/885498044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 msgr2=0x555a24164200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.937+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 0x555a24164200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.938+0000 7fa9b67fc700 1 -- 172.21.15.115:0/885498044 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a2401a500 con 0x555a23fe4630 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.938+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.938+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a23fe4630 0x555a2416a210 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0a97d0 tx=0x7fa9ac0a92c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4722def2e82c1acf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.938+0000 7fa99d7fa700 1 -- 172.21.15.115:0/885498044 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0c3070 con 0x555a23fe4630 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.938+0000 7fa99d7fa700 1 -- 172.21.15.115:0/885498044 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9ac0a95e0 con 0x555a23fe4630 2024-05-09T05:51:47.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.938+0000 7fa99d7fa700 1 -- 172.21.15.115:0/885498044 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0ade80 con 0x555a23fe4630 2024-05-09T05:51:47.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.939+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/885498044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a23fe4630 msgr2=0x555a2416a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:47.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.939+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a23fe4630 0x555a2416a210 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0a97d0 tx=0x7fa9ac0a92c0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.939+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/885498044 shutdown_connections 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.939+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 0x555a24164200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.939+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.939+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/885498044 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a23fe4630 0x555a2416a210 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.939+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/885498044 >> 172.21.15.115:0/885498044 conn(0x555a24044830 msgr2=0x555a2415cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.939+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/885498044 shutdown_connections 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.939+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/885498044 wait complete. 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9cbf93b80 1 -- start start 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9cbf93b80 1 Processor -- start 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 0x555a2416dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a2416e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a24179be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555a23effd30 con 0x555a2416e670 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555a2403e2e0 con 0x555a240ce2f0 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555a24164be0 con 0x555a24163df0 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9cbf93b80 10 client.0 _ll_register_callbacks cb 0x555a23ef5850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9b77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a24179be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9b77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a24179be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43480/0 (socket says 172.21.15.115:43480) 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9b77fe700 1 -- 172.21.15.115:0/2804742017 learned_addr learned my addr 172.21.15.115:0/2804742017 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 0x555a2416dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.940+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a2416e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9b77fe700 1 -- 172.21.15.115:0/2804742017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 msgr2=0x555a2416e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a2416e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9b77fe700 1 -- 172.21.15.115:0/2804742017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 msgr2=0x555a2416dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 0x555a2416dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9b77fe700 1 -- 172.21.15.115:0/2804742017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a23eff620 con 0x555a2416e670 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a2416e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a24179be0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0004660 tx=0x7fa9b0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736f33f545bd641e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa99aff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa99aff5700 1 -- 172.21.15.115:0/2804742017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b001a070 con 0x555a2416e670 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa99aff5700 1 -- 172.21.15.115:0/2804742017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fa9b0004db0 con 0x555a2416e670 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/2804742017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ac0a95e0 con 0x555a2416e670 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa99aff5700 1 -- 172.21.15.115:0/2804742017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fa9b0025050 con 0x555a2416e670 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9cbf93b80 10 client.18633 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.941+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/2804742017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555a2417ba00 con 0x555a2416e670 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.942+0000 7fa99aff5700 1 -- 172.21.15.115:0/2804742017 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9ac0a95e0 con 0x555a2416e670 2024-05-09T05:51:47.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.942+0000 7fa99aff5700 1 -- 172.21.15.115:0/2804742017 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7fa9b0016a10 con 0x555a2416e670 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.942+0000 7fa9cbf93b80 10 client.18633 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.942+0000 7fa9cbf93b80 10 client.18633 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.942+0000 7fa9cbf93b80 20 client.18633 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.942+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555a2417dfe0 0x555a24180420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.942+0000 7fa9cbf93b80 4 client.18633 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.942+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/2804742017 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a24091b60 con 0x555a2417dfe0 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.942+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555a2417dfe0 0x555a24180420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.943+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555a2417dfe0 0x555a24180420 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac4d19dd42a89e9 server_cookie=e8c743dc4156f280 in_seq=0 out_seq=0 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.943+0000 7fa99aff5700 10 client.18633 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa99aff5700 1 -- 172.21.15.115:0/2804742017 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a24091b60 con 0x555a2417dfe0 2024-05-09T05:51:47.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa99aff5700 10 client.18633 handle_command_reply: tid=0 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 client.18633 shutdown 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/2804742017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 msgr2=0x555a24179be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a24179be0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0004660 tx=0x7fa9b0004740 comp rx=0 tx=0).stop 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/2804742017 shutdown_connections 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 0x555a2416dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555a2417dfe0 0x555a24180420 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa99aff5700 0 client.18633 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a24179be0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/2804742017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a2416e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/2804742017 >> 172.21.15.115:0/2804742017 conn(0x555a24044830 msgr2=0x555a2415c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/2804742017 shutdown_connections 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/2804742017 wait complete. 2024-05-09T05:51:47.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.944+0000 7fa9cbf93b80 20 client.18633 trim_cache size 0 max 16384 2024-05-09T05:51:47.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.970+0000 7fa9cbf93b80 1 Processor -- start 2024-05-09T05:51:47.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.970+0000 7fa9cbf93b80 1 -- start start 2024-05-09T05:51:47.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 0x555a24179800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a2416f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81ae2b0 con 0x555a2416e670 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81afe50 con 0x555a24163df0 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81ae980 con 0x555a240ce2f0 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a2416f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a2416f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43492/0 (socket says 172.21.15.115:43492) 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b77fe700 1 -- 172.21.15.115:0/4070770923 learned_addr learned my addr 172.21.15.115:0/4070770923 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 0x555a24179800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b77fe700 1 -- 172.21.15.115:0/4070770923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 msgr2=0x555a24161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b77fe700 1 -- 172.21.15.115:0/4070770923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 msgr2=0x555a24179800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 0x555a24179800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b77fe700 1 -- 172.21.15.115:0/4070770923 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a24170490 con 0x555a2416e670 2024-05-09T05:51:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.971+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 0x555a24179800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.972+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a2416f860 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00093f0 tx=0x7fa9b0009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2384b9c12fa1737 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.972+0000 7fa99cff9700 1 -- 172.21.15.115:0/4070770923 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b001a040 con 0x555a2416e670 2024-05-09T05:51:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.972+0000 7fa99cff9700 1 -- 172.21.15.115:0/4070770923 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b00059a0 con 0x555a2416e670 2024-05-09T05:51:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.972+0000 7fa99cff9700 1 -- 172.21.15.115:0/4070770923 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0007b80 con 0x555a2416e670 2024-05-09T05:51:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.973+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/4070770923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 msgr2=0x555a2416f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.973+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a2416f860 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00093f0 tx=0x7fa9b0009380 comp rx=0 tx=0).stop 2024-05-09T05:51:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.973+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/4070770923 shutdown_connections 2024-05-09T05:51:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.973+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a2416f860 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.973+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a240ce2f0 0x555a24161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.973+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/4070770923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a24163df0 0x555a24179800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.973+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/4070770923 >> 172.21.15.115:0/4070770923 conn(0x555a24044830 msgr2=0x555a2416f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.973+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/4070770923 shutdown_connections 2024-05-09T05:51:47.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.973+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/4070770923 wait complete. 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9cbf93b80 1 -- start start 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9cbf93b80 1 Processor -- start 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 0x555a24176050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a241713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9cbf93b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a241718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81af780 con 0x555a2416e670 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81af0b0 con 0x555a240ce2f0 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9cbf93b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81ac270 con 0x555a24163df0 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9cbf93b80 10 client.0 _ll_register_callbacks cb 0x555a23ef5850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9b77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a241718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9b77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a241718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43500/0 (socket says 172.21.15.115:43500) 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9b77fe700 1 -- 172.21.15.115:0/3411861530 learned_addr learned my addr 172.21.15.115:0/3411861530 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 0x555a24176050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.974+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a241713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9b77fe700 1 -- 172.21.15.115:0/3411861530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 msgr2=0x555a241713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a241713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9b77fe700 1 -- 172.21.15.115:0/3411861530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 msgr2=0x555a24176050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 0x555a24176050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9b77fe700 1 -- 172.21.15.115:0/3411861530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a24171fd0 con 0x555a2416e670 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9b67fc700 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a241713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9b77fe700 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a241718f0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009200 tx=0x7fa9b0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2fc23fd638bb51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa99aff5700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa99aff5700 1 -- 172.21.15.115:0/3411861530 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b001a040 con 0x555a2416e670 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/3411861530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b00059a0 con 0x555a2416e670 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa99aff5700 1 -- 172.21.15.115:0/3411861530 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fa9b00077e0 con 0x555a2416e670 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa99aff5700 1 -- 172.21.15.115:0/3411861530 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fa9b0045050 con 0x555a2416e670 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9cbf93b80 10 client.18639 fetch_fsmap learned FSMap version 57 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/3411861530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555a241e65d0 con 0x555a2416e670 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.975+0000 7fa99aff5700 1 -- 172.21.15.115:0/3411861530 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b00059a0 con 0x555a2416e670 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.976+0000 7fa99aff5700 1 -- 172.21.15.115:0/3411861530 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 57) ==== 8370+0+0 (secure 0 0 0) 0x7fa9b002ca40 con 0x555a2416e670 2024-05-09T05:51:47.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.976+0000 7fa9cbf93b80 10 client.18639 fetch_fsmap finished waiting for FSMap version 57 2024-05-09T05:51:47.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.976+0000 7fa9cbf93b80 10 client.18639 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:51:47.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.976+0000 7fa9cbf93b80 20 client.18639 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:47.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.976+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555a2417dfe0 0x555a241e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:47.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.976+0000 7fa9cbf93b80 4 client.18639 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-09T05:51:47.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.976+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/3411861530 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x555a2408bc00 con 0x555a2417dfe0 2024-05-09T05:51:47.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.976+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555a2417dfe0 0x555a241e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:47.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.977+0000 7fa9b6ffd700 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555a2417dfe0 0x555a241e8c90 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64164ddbd84b82db server_cookie=6e47114e3e67625 in_seq=0 out_seq=0 2024-05-09T05:51:47.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:47.977+0000 7fa99aff5700 10 client.18639 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:48.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa99aff5700 1 -- 172.21.15.115:0/3411861530 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555a2408bc00 con 0x555a2417dfe0 2024-05-09T05:51:48.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa99aff5700 10 client.18639 handle_command_reply: tid=0 2024-05-09T05:51:48.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 client.18639 shutdown 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/3411861530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 msgr2=0x555a241718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a241718f0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009200 tx=0x7fa9b0005720 comp rx=0 tx=0).stop 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/3411861530 shutdown_connections 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a240ce2f0 0x555a24176050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555a2417dfe0 0x555a241e8c90 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555a2416e670 0x555a241718f0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa99aff5700 0 client.18639 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 --2- 172.21.15.115:0/3411861530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555a24163df0 0x555a241713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/3411861530 >> 172.21.15.115:0/3411861530 conn(0x555a24044830 msgr2=0x555a24161360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/3411861530 shutdown_connections 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 1 -- 172.21.15.115:0/3411861530 wait complete. 2024-05-09T05:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.087+0000 7fa9cbf93b80 20 client.18639 trim_cache size 0 max 16384 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 57, 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 42, 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "2c4ca6bd": { 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 42, 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:51:48.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:51:48.080 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 -- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9a00b29f0 msgr2=0x7fa9a00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9a00b29f0 0x7fa9a00b4f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8000c00 tx=0x7fa9a801a040 comp rx=0 tx=0).stop 2024-05-09T05:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 -- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b8107df0 msgr2=0x7fa9b819fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b8107df0 0x7fa9b819fd60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00e970 tx=0x7fa9ac00ec80 comp rx=0 tx=0).stop 2024-05-09T05:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 -- 172.21.15.115:0/2009444072 shutdown_connections 2024-05-09T05:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa9a00b29f0 0x7fa9a00b4f00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9b81034c0 0x7fa9b819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b8108740 0x7fa9b8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 --2- 172.21.15.115:0/2009444072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b8107df0 0x7fa9b819fd60 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 -- 172.21.15.115:0/2009444072 >> 172.21.15.115:0/2009444072 conn(0x7fa9b80fedc0 msgr2=0x7fa9b8101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 -- 172.21.15.115:0/2009444072 shutdown_connections 2024-05-09T05:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:51:48.088+0000 7fa9bd57d700 1 -- 172.21.15.115:0/2009444072 wait complete. 2024-05-09T05:51:48.889 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2c4ca6bd', quiesce took 6.2 seconds. Will release after: 23.8 2024-05-09T05:51:49.687 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:49 smithi115 ceph-mon[30349]: pgmap v428: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 19 op/s 2024-05-09T05:51:49.687 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:49 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:51:49.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:49 smithi195 ceph-mon[33844]: pgmap v428: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 19 op/s 2024-05-09T05:51:49.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:49 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:51:49.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:49 smithi161 ceph-mon[33805]: pgmap v428: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 19 op/s 2024-05-09T05:51:49.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:49 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:51:51.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:51 smithi195 ceph-mon[33844]: pgmap v429: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 15 op/s 2024-05-09T05:51:51.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:51 smithi115 ceph-mon[30349]: pgmap v429: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 15 op/s 2024-05-09T05:51:51.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:51 smithi161 ceph-mon[33805]: pgmap v429: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 15 op/s 2024-05-09T05:51:53.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:53 smithi195 ceph-mon[33844]: pgmap v430: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 15 op/s 2024-05-09T05:51:53.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:53 smithi115 ceph-mon[30349]: pgmap v430: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 15 op/s 2024-05-09T05:51:53.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:53 smithi161 ceph-mon[33805]: pgmap v430: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 15 op/s 2024-05-09T05:51:55.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:55 smithi195 ceph-mon[33844]: pgmap v431: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 20 op/s 2024-05-09T05:51:55.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:55 smithi115 ceph-mon[30349]: pgmap v431: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 20 op/s 2024-05-09T05:51:55.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:55 smithi161 ceph-mon[33805]: pgmap v431: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 20 op/s 2024-05-09T05:51:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.884+0000 7f95bc070640 1 -- 172.21.15.115:0/4266896729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 msgr2=0x7f95b4102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.884+0000 7f95bc070640 1 --2- 172.21.15.115:0/4266896729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 0x7f95b4102c80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009900 tx=0x7f95a402a410 comp rx=0 tx=0).stop 2024-05-09T05:51:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.884+0000 7f95bc070640 1 -- 172.21.15.115:0/4266896729 shutdown_connections 2024-05-09T05:51:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.884+0000 7f95bc070640 1 --2- 172.21.15.115:0/4266896729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b41032f0 0x7f95b410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.884+0000 7f95bc070640 1 --2- 172.21.15.115:0/4266896729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 0x7f95b4102c80 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.884+0000 7f95bc070640 1 --2- 172.21.15.115:0/4266896729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95b4108ba0 0x7f95b4102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.884+0000 7f95bc070640 1 -- 172.21.15.115:0/4266896729 >> 172.21.15.115:0/4266896729 conn(0x7f95b40782e0 msgr2=0x7f95b4078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.884+0000 7f95bc070640 1 -- 172.21.15.115:0/4266896729 shutdown_connections 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.884+0000 7f95bc070640 1 -- 172.21.15.115:0/4266896729 wait complete. 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95bc070640 1 Processor -- start 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95bc070640 1 -- start start 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95bc070640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95b41032f0 0x7f95b419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95bc070640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b4108ba0 0x7f95b419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95bc070640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 0x7f95b419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95bc070640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4112c90 con 0x7f95b4109040 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95bc070640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4112b10 con 0x7f95b41032f0 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95bc070640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4112e10 con 0x7f95b4108ba0 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95ba60d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 0x7f95b419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95ba60d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 0x7f95b419c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53834/0 (socket says 172.21.15.115:53834) 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95ba60d640 1 -- 172.21.15.115:0/2909622469 learned_addr learned my addr 172.21.15.115:0/2909622469 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.885+0000 7f95b960b640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b4108ba0 0x7f95b419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95b41032f0 0x7f95b419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95ba60d640 1 -- 172.21.15.115:0/2909622469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b4108ba0 msgr2=0x7f95b419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95ba60d640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b4108ba0 0x7f95b419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95ba60d640 1 -- 172.21.15.115:0/2909622469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95b41032f0 msgr2=0x7f95b419b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95ba60d640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95b41032f0 0x7f95b419b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95ba60d640 1 -- 172.21.15.115:0/2909622469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b41a69a0 con 0x7f95b4109040 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95b41032f0 0x7f95b419b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95ba60d640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 0x7f95b419c110 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00cad0 tx=0x7f95ac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95aaffd640 1 -- 172.21.15.115:0/2909622469 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00ae20 con 0x7f95b4109040 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95aaffd640 1 -- 172.21.15.115:0/2909622469 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95ac005020 con 0x7f95b4109040 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95aaffd640 1 -- 172.21.15.115:0/2909622469 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00f6b0 con 0x7f95b4109040 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95bc070640 1 -- 172.21.15.115:0/2909622469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b41a6c90 con 0x7f95b4109040 2024-05-09T05:51:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.886+0000 7f95bc070640 1 -- 172.21.15.115:0/2909622469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b41a71b0 con 0x7f95b4109040 2024-05-09T05:51:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.889+0000 7f95c2fff740 1 Processor -- start 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95aaffd640 1 -- 172.21.15.115:0/2909622469 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f95ac0051d0 con 0x7f95b4109040 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95c2fff740 1 -- start start 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26521b0 0x55dab29f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26292f0 0x55dab29f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 0x55dab29f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95c2fff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dab295a7c0 con 0x55dab26521b0 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95c2fff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dab28f60e0 con 0x55dab26292f0 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95c2fff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dab2956780 con 0x55dab26314d0 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b960b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26521b0 0x55dab29f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b960b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26521b0 0x55dab29f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53850/0 (socket says 172.21.15.115:53850) 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b960b640 1 -- 172.21.15.115:0/3371466818 learned_addr learned my addr 172.21.15.115:0/3371466818 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26292f0 0x55dab29f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b960b640 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 0x55dab29f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95aaffd640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95a00b29d0 0x7f95a00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b9e0c640 1 -- 172.21.15.115:0/3371466818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 msgr2=0x55dab29f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 0x55dab29f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b9e0c640 1 -- 172.21.15.115:0/3371466818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26521b0 msgr2=0x55dab29f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26521b0 0x55dab29f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b9e0c640 1 -- 172.21.15.115:0/3371466818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab291f7c0 con 0x55dab26292f0 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.890+0000 7f95b960b640 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26521b0 0x55dab29f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.891+0000 7f95ba60d640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95a00b29d0 0x7f95a00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.891+0000 7f95aaffd640 1 -- 172.21.15.115:0/2909622469 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f95ac0e2d10 con 0x7f95b4109040 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.891+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26292f0 0x55dab29f5bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009b80 tx=0x7f95b00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c3de0bd13e5756 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.891+0000 7f959ffff640 1 -- 172.21.15.115:0/3371466818 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0012450 con 0x55dab26292f0 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.891+0000 7f959ffff640 1 -- 172.21.15.115:0/3371466818 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95b0012a80 con 0x55dab26292f0 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.891+0000 7f959ffff640 1 -- 172.21.15.115:0/3371466818 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b000f8a0 con 0x55dab26292f0 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 -- 172.21.15.115:0/3371466818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26292f0 msgr2=0x55dab29f5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95ba60d640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95a00b29d0 0x7f95a00b4ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0e0180 tx=0x7f95ac076590 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26292f0 0x55dab29f5bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009b80 tx=0x7f95b00117b0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 -- 172.21.15.115:0/3371466818 shutdown_connections 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 0x55dab29f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26292f0 0x55dab29f5bc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 --2- 172.21.15.115:0/3371466818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26521b0 0x55dab29f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 -- 172.21.15.115:0/3371466818 >> 172.21.15.115:0/3371466818 conn(0x55dab2646300 msgr2=0x55dab29f1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 -- 172.21.15.115:0/3371466818 shutdown_connections 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 -- 172.21.15.115:0/3371466818 wait complete. 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 -- start start 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.892+0000 7f95c2fff740 1 Processor -- start 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab29ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 0x55dab2a00210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 0x55dab2a05c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95c2fff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dab2a027b0 con 0x55dab26292f0 2024-05-09T05:51:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95c2fff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dab28e19c0 con 0x55dab26521b0 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95c2fff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dab2a02930 con 0x55dab26314d0 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95c2fff740 10 client.0 _ll_register_callbacks cb 0x55dab290e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab29ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab29ffcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53864/0 (socket says 172.21.15.115:53864) 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 -- 172.21.15.115:0/4189039696 learned_addr learned my addr 172.21.15.115:0/4189039696 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 0x55dab2a05c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 0x55dab2a00210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 -- 172.21.15.115:0/4189039696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 msgr2=0x55dab2a00210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 0x55dab2a00210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 -- 172.21.15.115:0/4189039696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 msgr2=0x55dab2a05c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 0x55dab2a05c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 -- 172.21.15.115:0/4189039696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab29081f0 con 0x55dab26292f0 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95b960b640 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab29ffcd0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f95a40096c0 tx=0x7f95a4005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905f620c8ac11cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f959d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f959d7fa640 1 -- 172.21.15.115:0/4189039696 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4035070 con 0x55dab26292f0 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f959d7fa640 1 -- 172.21.15.115:0/4189039696 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f95a403d450 con 0x55dab26292f0 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.893+0000 7f95c2fff740 1 -- 172.21.15.115:0/4189039696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b0012a80 con 0x55dab26292f0 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f959d7fa640 1 -- 172.21.15.115:0/4189039696 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f95a4053050 con 0x55dab26292f0 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f95c2fff740 10 client.18651 fetch_fsmap learned FSMap version 58 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f95c2fff740 1 -- 172.21.15.115:0/4189039696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55dab26bd790 con 0x55dab26292f0 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f959d7fa640 1 -- 172.21.15.115:0/4189039696 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95b0012a80 con 0x55dab26292f0 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f959d7fa640 1 -- 172.21.15.115:0/4189039696 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 58) ==== 8366+0+0 (secure 0 0 0) 0x7f95a403b650 con 0x55dab26292f0 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f95c2fff740 10 client.18651 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f95c2fff740 10 client.18651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f95c2fff740 20 client.18651 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dab2610d80 0x55dab2a11e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f95c2fff740 4 client.18651 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f95c2fff740 1 -- 172.21.15.115:0/4189039696 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dab29683d0 con 0x55dab2610d80 2024-05-09T05:51:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.894+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dab2610d80 0x55dab2a11e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.895+0000 7f959d7fa640 10 client.18651 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.895+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dab2610d80 0x55dab2a11e40 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a395d16af3cc8c server_cookie=4ed62f82643adab0 in_seq=0 out_seq=0 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.895+0000 7f959d7fa640 1 -- 172.21.15.115:0/4189039696 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dab29683d0 con 0x55dab2610d80 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f959d7fa640 10 client.18651 handle_command_reply: tid=0 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 client.18651 shutdown 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 -- 172.21.15.115:0/4189039696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 msgr2=0x55dab29ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab29ffcd0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f95a40096c0 tx=0x7f95a4005a40 comp rx=0 tx=0).stop 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 -- 172.21.15.115:0/4189039696 shutdown_connections 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 0x55dab2a05c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dab2610d80 0x55dab2a11e40 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26314d0 0x55dab2a00210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4189039696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab29ffcd0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f959d7fa640 0 client.18651 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 -- 172.21.15.115:0/4189039696 >> 172.21.15.115:0/4189039696 conn(0x55dab2646300 msgr2=0x55dab29f0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 -- 172.21.15.115:0/4189039696 shutdown_connections 2024-05-09T05:51:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 1 -- 172.21.15.115:0/4189039696 wait complete. 2024-05-09T05:51:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.896+0000 7f95c2fff740 20 client.18651 trim_cache size 0 max 16384 2024-05-09T05:51:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95c2fff740 1 Processor -- start 2024-05-09T05:51:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95c2fff740 1 -- start start 2024-05-09T05:51:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26292f0 0x55dab29f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26314d0 0x55dab29f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 0x55dab2a018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95c2fff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41b10f0 con 0x55dab26314d0 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95c2fff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41b2c60 con 0x55dab26521b0 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95c2fff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41b17c0 con 0x55dab26292f0 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b9e0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26314d0 0x55dab29f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b9e0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26314d0 0x55dab29f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53872/0 (socket says 172.21.15.115:53872) 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b9e0c640 1 -- 172.21.15.115:0/4116829401 learned_addr learned my addr 172.21.15.115:0/4116829401 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b960b640 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 0x55dab2a018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b960b640 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26292f0 0x55dab29f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b9e0c640 1 -- 172.21.15.115:0/4116829401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26292f0 msgr2=0x55dab29f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26292f0 0x55dab29f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b9e0c640 1 -- 172.21.15.115:0/4116829401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 msgr2=0x55dab2a018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 0x55dab2a018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b9e0c640 1 -- 172.21.15.115:0/4116829401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab27345f0 con 0x55dab26314d0 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.911+0000 7f95b960b640 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26292f0 0x55dab29f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26314d0 0x55dab29f5f50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009b80 tx=0x7f95b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa118f09458404 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f959f7fe640 1 -- 172.21.15.115:0/4116829401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0009e40 con 0x55dab26314d0 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f959f7fe640 1 -- 172.21.15.115:0/4116829401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95b00054d0 con 0x55dab26314d0 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f959f7fe640 1 -- 172.21.15.115:0/4116829401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b00129b0 con 0x55dab26314d0 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f95c2fff740 1 -- 172.21.15.115:0/4116829401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26314d0 msgr2=0x55dab29f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26314d0 0x55dab29f5f50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009b80 tx=0x7f95b00057f0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f95c2fff740 1 -- 172.21.15.115:0/4116829401 shutdown_connections 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26521b0 0x55dab2a018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26314d0 0x55dab29f5f50 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f95c2fff740 1 --2- 172.21.15.115:0/4116829401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26292f0 0x55dab29f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.912+0000 7f95c2fff740 1 -- 172.21.15.115:0/4116829401 >> 172.21.15.115:0/4116829401 conn(0x55dab2646300 msgr2=0x55dab2a011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 -- 172.21.15.115:0/4116829401 shutdown_connections 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 -- 172.21.15.115:0/4116829401 wait complete. 2024-05-09T05:51:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 -- start start 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 Processor -- start 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab2a0c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26314d0 0x55dab2a079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26521b0 0x55dab2a07ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41b2590 con 0x55dab26292f0 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41b1ec0 con 0x55dab26314d0 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41af0f0 con 0x55dab26521b0 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95c2fff740 10 client.0 _ll_register_callbacks cb 0x7f95a4004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95b960b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab2a0c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95b960b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab2a0c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53876/0 (socket says 172.21.15.115:53876) 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.913+0000 7f95b960b640 1 -- 172.21.15.115:0/1450310376 learned_addr learned my addr 172.21.15.115:0/1450310376 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95b960b640 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26521b0 0x55dab2a07ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26314d0 0x55dab2a079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95b960b640 1 -- 172.21.15.115:0/1450310376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26521b0 msgr2=0x55dab2a07ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95b960b640 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26521b0 0x55dab2a07ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95b960b640 1 -- 172.21.15.115:0/1450310376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26314d0 msgr2=0x55dab2a079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95b960b640 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26314d0 0x55dab2a079b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95b960b640 1 -- 172.21.15.115:0/1450310376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab291eed0 con 0x55dab26292f0 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95b960b640 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab2a0c660 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f95a4005df0 tx=0x7f95a4005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7002d52cd3e96126 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f959d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f959d7fa640 1 -- 172.21.15.115:0/1450310376 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4035040 con 0x55dab26292f0 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f959d7fa640 1 -- 172.21.15.115:0/1450310376 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f95a403b620 con 0x55dab26292f0 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95c2fff740 1 -- 172.21.15.115:0/1450310376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b00054d0 con 0x55dab26292f0 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f959d7fa640 1 -- 172.21.15.115:0/1450310376 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f95a4046040 con 0x55dab26292f0 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95c2fff740 10 client.18657 fetch_fsmap learned FSMap version 58 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f95c2fff740 1 -- 172.21.15.115:0/1450310376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55dab2a79cf0 con 0x55dab26292f0 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.914+0000 7f959d7fa640 1 -- 172.21.15.115:0/1450310376 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95b00054d0 con 0x55dab26292f0 2024-05-09T05:51:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f959d7fa640 1 -- 172.21.15.115:0/1450310376 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 58) ==== 8366+0+0 (secure 0 0 0) 0x7f95a404b420 con 0x55dab26292f0 2024-05-09T05:51:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f95c2fff740 10 client.18657 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T05:51:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f95c2fff740 10 client.18657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:51:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f95c2fff740 20 client.18657 populate_metadata read hostname 'smithi115' 2024-05-09T05:51:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f95c2fff740 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dab25e61c0 0x55dab2a7c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:51:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f95c2fff740 4 client.18657 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:51:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f95c2fff740 1 -- 172.21.15.115:0/1450310376 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dab2962240 con 0x55dab25e61c0 2024-05-09T05:51:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dab25e61c0 0x55dab2a7c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:51:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f959d7fa640 10 client.18657 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.915+0000 7f95b9e0c640 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dab25e61c0 0x55dab2a7c370 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd82ff9350daea9 server_cookie=1ec491a0fb57dcc0 in_seq=0 out_seq=0 2024-05-09T05:51:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f959d7fa640 1 -- 172.21.15.115:0/1450310376 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dab2962240 con 0x55dab25e61c0 2024-05-09T05:51:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f959d7fa640 10 client.18657 handle_command_reply: tid=0 2024-05-09T05:51:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 client.18657 shutdown 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 -- 172.21.15.115:0/1450310376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 msgr2=0x55dab2a0c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab2a0c660 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f95a4005df0 tx=0x7f95a4005a40 comp rx=0 tx=0).stop 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 -- 172.21.15.115:0/1450310376 shutdown_connections 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dab25e61c0 0x55dab2a7c370 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dab26521b0 0x55dab2a07ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dab26314d0 0x55dab2a079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f959d7fa640 0 client.18657 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 --2- 172.21.15.115:0/1450310376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dab26292f0 0x55dab2a0c660 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 -- 172.21.15.115:0/1450310376 >> 172.21.15.115:0/1450310376 conn(0x55dab2646300 msgr2=0x55dab29f5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 -- 172.21.15.115:0/1450310376 shutdown_connections 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 1 -- 172.21.15.115:0/1450310376 wait complete. 2024-05-09T05:51:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.920+0000 7f95c2fff740 20 client.18657 trim_cache size 0 max 16384 2024-05-09T05:51:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:51:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:51:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:51:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 -- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95a00b29d0 msgr2=0x7f95a00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95a00b29d0 0x7f95a00b4ee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0e0180 tx=0x7f95ac076590 comp rx=0 tx=0).stop 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 -- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 msgr2=0x7f95b419c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 0x7f95b419c110 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00cad0 tx=0x7f95ac00cf90 comp rx=0 tx=0).stop 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 -- 172.21.15.115:0/2909622469 shutdown_connections 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95a00b29d0 0x7f95a00b4ee0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95b4109040 0x7f95b419c110 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b4108ba0 0x7f95b419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 --2- 172.21.15.115:0/2909622469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95b41032f0 0x7f95b419b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 -- 172.21.15.115:0/2909622469 >> 172.21.15.115:0/2909622469 conn(0x7f95b40782e0 msgr2=0x7f95b41012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 -- 172.21.15.115:0/2909622469 shutdown_connections 2024-05-09T05:51:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:51:56.921+0000 7f95bc070640 1 -- 172.21.15.115:0/2909622469 wait complete. 2024-05-09T05:51:57.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:57 smithi195 ceph-mon[33844]: pgmap v432: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 15 op/s 2024-05-09T05:51:57.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:57 smithi115 ceph-mon[30349]: pgmap v432: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 15 op/s 2024-05-09T05:51:57.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:57 smithi161 ceph-mon[33805]: pgmap v432: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 15 op/s 2024-05-09T05:51:59.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:51:59 smithi195 ceph-mon[33844]: pgmap v433: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-05-09T05:51:59.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:51:59 smithi115 ceph-mon[30349]: pgmap v433: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-05-09T05:51:59.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:51:59 smithi161 ceph-mon[33805]: pgmap v433: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-05-09T05:52:01.708 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:01 smithi115 ceph-mon[30349]: pgmap v434: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 575 KiB/s wr, 10 op/s 2024-05-09T05:52:01.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:01 smithi195 ceph-mon[33844]: pgmap v434: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 575 KiB/s wr, 10 op/s 2024-05-09T05:52:01.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:01 smithi161 ceph-mon[33805]: pgmap v434: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 575 KiB/s wr, 10 op/s 2024-05-09T05:52:03.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:03 smithi195 ceph-mon[33844]: pgmap v435: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 575 KiB/s wr, 10 op/s 2024-05-09T05:52:03.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:03 smithi115 ceph-mon[30349]: pgmap v435: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 575 KiB/s wr, 10 op/s 2024-05-09T05:52:03.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:03 smithi161 ceph-mon[33805]: pgmap v435: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 575 KiB/s wr, 10 op/s 2024-05-09T05:52:05.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:05 smithi195 ceph-mon[33844]: pgmap v436: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 622 KiB/s wr, 31 op/s 2024-05-09T05:52:05.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:05 smithi115 ceph-mon[30349]: pgmap v436: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 622 KiB/s wr, 31 op/s 2024-05-09T05:52:05.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:05 smithi161 ceph-mon[33805]: pgmap v436: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 622 KiB/s wr, 31 op/s 2024-05-09T05:52:07.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:07 smithi195 ceph-mon[33844]: pgmap v437: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 279 KiB/s wr, 26 op/s 2024-05-09T05:52:07.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:07 smithi115 ceph-mon[30349]: pgmap v437: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 279 KiB/s wr, 26 op/s 2024-05-09T05:52:07.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:07 smithi161 ceph-mon[33805]: pgmap v437: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 279 KiB/s wr, 26 op/s 2024-05-09T05:52:09.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:09 smithi195 ceph-mon[33844]: pgmap v438: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 279 KiB/s wr, 31 op/s 2024-05-09T05:52:09.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:09 smithi115 ceph-mon[30349]: pgmap v438: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 279 KiB/s wr, 31 op/s 2024-05-09T05:52:09.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:09 smithi161 ceph-mon[33805]: pgmap v438: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 279 KiB/s wr, 31 op/s 2024-05-09T05:52:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.034+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/2690318670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 msgr2=0x7f1e38102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.034+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/2690318670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 0x7f1e38102ca0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009900 tx=0x7f1e3402a410 comp rx=0 tx=0).stop 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.034+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/2690318670 shutdown_connections 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.034+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/2690318670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e38103310 0x7f1e3810f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.034+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/2690318670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 0x7f1e38102ca0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.034+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/2690318670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e38108bc0 0x7f1e38102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.034+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/2690318670 >> 172.21.15.115:0/2690318670 conn(0x7f1e380fe390 msgr2=0x7f1e381007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.034+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/2690318670 shutdown_connections 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.034+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/2690318670 wait complete. 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3f2bd640 1 Processor -- start 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3f2bd640 1 -- start start 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3f2bd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e38103310 0x7f1e38199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3f2bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e38108bc0 0x7f1e381999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3f2bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 0x7f1e38199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3f2bd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38112c60 con 0x7f1e38109060 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3f2bd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38112ae0 con 0x7f1e38103310 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3f2bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38112de0 con 0x7f1e38108bc0 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3d85a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 0x7f1e38199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3d85a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 0x7f1e38199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45120/0 (socket says 172.21.15.115:45120) 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.035+0000 7f1e3d85a640 1 -- 172.21.15.115:0/4169237777 learned_addr learned my addr 172.21.15.115:0/4169237777 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3c858640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e38108bc0 0x7f1e381999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3d059640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e38103310 0x7f1e38199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3d85a640 1 -- 172.21.15.115:0/4169237777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e38108bc0 msgr2=0x7f1e381999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3d85a640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e38108bc0 0x7f1e381999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3d85a640 1 -- 172.21.15.115:0/4169237777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e38103310 msgr2=0x7f1e38199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3d85a640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e38103310 0x7f1e38199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3d85a640 1 -- 172.21.15.115:0/4169237777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e381a8b40 con 0x7f1e38109060 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3d059640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e38103310 0x7f1e38199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3d85a640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 0x7f1e38199f10 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000eaa0 tx=0x7f1e3000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e2e7fc640 1 -- 172.21.15.115:0/4169237777 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000cdb0 con 0x7f1e38109060 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/4169237777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e381a8e30 con 0x7f1e38109060 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/4169237777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e381a92f0 con 0x7f1e38109060 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e2e7fc640 1 -- 172.21.15.115:0/4169237777 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e30005020 con 0x7f1e38109060 2024-05-09T05:52:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.036+0000 7f1e2e7fc640 1 -- 172.21.15.115:0/4169237777 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3001f060 con 0x7f1e38109060 2024-05-09T05:52:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e2e7fc640 1 -- 172.21.15.115:0/4169237777 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1e30004a40 con 0x7f1e38109060 2024-05-09T05:52:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e4617b740 1 Processor -- start 2024-05-09T05:52:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e4617b740 1 -- start start 2024-05-09T05:52:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa2e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa2e4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa2e5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e4617b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574fa2497c0 con 0x5574f9f204d0 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e4617b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5574fa1e50e0 con 0x5574f9f411b0 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e4617b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574fa245780 con 0x5574f9f182f0 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e2e7fc640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e240b2950 0x7f1e240b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e3c858640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa2e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.039+0000 7f1e3d059640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa2e4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3c858640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa2e5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45132/0 (socket says 172.21.15.115:45132) 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3d059640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa2e4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:38532/0 (socket says 172.21.15.115:38532) 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3c858640 1 -- 172.21.15.115:0/1513098786 learned_addr learned my addr 172.21.15.115:0/1513098786 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3c858640 1 --2- 172.21.15.115:0/1513098786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa2e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3d85a640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e240b2950 0x7f1e240b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e2e7fc640 1 -- 172.21.15.115:0/4169237777 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9526+0+0 (secure 0 0 0) 0x7f1e300e2a80 con 0x7f1e38109060 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3c858640 1 -- 172.21.15.115:0/1513098786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 msgr2=0x5574fa2e4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3c858640 1 --2- 172.21.15.115:0/1513098786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa2e4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3c858640 1 -- 172.21.15.115:0/1513098786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 msgr2=0x5574fa2e4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3c858640 1 --2- 172.21.15.115:0/1513098786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa2e4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3c858640 1 -- 172.21.15.115:0/1513098786 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fa20e7c0 con 0x5574f9f204d0 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3c858640 1 --2- 172.21.15.115:0/1513098786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa2e5100 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f1e340096c0 tx=0x7f1e34005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91072206127b480a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e3d85a640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e240b2950 0x7f1e240b4e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e300dfe90 tx=0x7f1e30073210 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e237fe640 1 -- 172.21.15.115:0/1513098786 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34035070 con 0x5574f9f204d0 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e237fe640 1 -- 172.21.15.115:0/1513098786 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e340044c0 con 0x5574f9f204d0 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.040+0000 7f1e237fe640 1 -- 172.21.15.115:0/1513098786 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34003d60 con 0x5574f9f204d0 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.041+0000 7f1e4617b740 1 -- 172.21.15.115:0/1513098786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 msgr2=0x5574fa2e5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.041+0000 7f1e4617b740 1 --2- 172.21.15.115:0/1513098786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa2e5100 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f1e340096c0 tx=0x7f1e34005fd0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.041+0000 7f1e4617b740 1 -- 172.21.15.115:0/1513098786 shutdown_connections 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.041+0000 7f1e4617b740 1 --2- 172.21.15.115:0/1513098786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa2e5100 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.041+0000 7f1e4617b740 1 --2- 172.21.15.115:0/1513098786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa2e4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.041+0000 7f1e4617b740 1 --2- 172.21.15.115:0/1513098786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa2e4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.041+0000 7f1e4617b740 1 -- 172.21.15.115:0/1513098786 >> 172.21.15.115:0/1513098786 conn(0x5574f9f35300 msgr2=0x5574fa2e0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.041+0000 7f1e4617b740 1 -- 172.21.15.115:0/1513098786 shutdown_connections 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.041+0000 7f1e4617b740 1 -- 172.21.15.115:0/1513098786 wait complete. 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e4617b740 1 -- start start 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e4617b740 1 Processor -- start 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f182f0 0x5574fa2f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f204d0 0x5574fa2f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa2eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e4617b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574fa2f1750 con 0x5574f9f182f0 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e4617b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5574fa1d09c0 con 0x5574f9f411b0 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e4617b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574fa2f18d0 con 0x5574f9f204d0 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e4617b740 10 client.0 _ll_register_callbacks cb 0x5574fa1fd020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3c858640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f182f0 0x5574fa2f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3c858640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f182f0 0x5574fa2f06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45138/0 (socket says 172.21.15.115:45138) 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3c858640 1 -- 172.21.15.115:0/948388190 learned_addr learned my addr 172.21.15.115:0/948388190 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3d059640 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f204d0 0x5574fa2f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3c858640 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa2eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3c858640 1 -- 172.21.15.115:0/948388190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f204d0 msgr2=0x5574fa2f0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3c858640 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f204d0 0x5574fa2f0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3c858640 1 -- 172.21.15.115:0/948388190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 msgr2=0x5574fa2eebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3c858640 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa2eebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.042+0000 7f1e3c858640 1 -- 172.21.15.115:0/948388190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fa1f71f0 con 0x5574f9f182f0 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e3c858640 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f182f0 0x5574fa2f06e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f1e34032f80 tx=0x7f1e34032cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568902141fac3524 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e20ff9640 1 -- 172.21.15.115:0/948388190 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34041070 con 0x5574f9f182f0 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e20ff9640 1 -- 172.21.15.115:0/948388190 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9526+0+0 (secure 0 0 0) 0x7f1e34003ae0 con 0x5574f9f182f0 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e4617b740 1 -- 172.21.15.115:0/948388190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5574fa2573d0 con 0x5574f9f182f0 2024-05-09T05:52:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e20ff9640 1 -- 172.21.15.115:0/948388190 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1e3405d050 con 0x5574f9f182f0 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e4617b740 10 client.18669 fetch_fsmap learned FSMap version 58 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e4617b740 1 -- 172.21.15.115:0/948388190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5574f9fac790 con 0x5574f9f182f0 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e20ff9640 1 -- 172.21.15.115:0/948388190 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5574fa2573d0 con 0x5574f9f182f0 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e20ff9640 1 -- 172.21.15.115:0/948388190 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 58) ==== 8366+0+0 (secure 0 0 0) 0x7f1e3403d730 con 0x5574f9f182f0 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e4617b740 10 client.18669 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e4617b740 10 client.18669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e4617b740 20 client.18669 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e4617b740 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574f9effd80 0x5574fa300e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e4617b740 4 client.18669 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.043+0000 7f1e4617b740 1 -- 172.21.15.115:0/948388190 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574fa251240 con 0x5574f9effd80 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.044+0000 7f1e3d059640 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574f9effd80 0x5574fa300e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.044+0000 7f1e20ff9640 10 client.18669 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.044+0000 7f1e3d059640 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574f9effd80 0x5574fa300e00 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea528815f7b6ecd server_cookie=682a07b01e16fbf7 in_seq=0 out_seq=0 2024-05-09T05:52:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e20ff9640 1 -- 172.21.15.115:0/948388190 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574fa251240 con 0x5574f9effd80 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e20ff9640 10 client.18669 handle_command_reply: tid=0 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 client.18669 shutdown 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 -- 172.21.15.115:0/948388190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f182f0 msgr2=0x5574fa2f06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f182f0 0x5574fa2f06e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f1e34032f80 tx=0x7f1e34032cf0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 -- 172.21.15.115:0/948388190 shutdown_connections 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa2eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574f9effd80 0x5574fa300e00 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f204d0 0x5574fa2f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 --2- 172.21.15.115:0/948388190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f182f0 0x5574fa2f06e0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 -- 172.21.15.115:0/948388190 >> 172.21.15.115:0/948388190 conn(0x5574f9f35300 msgr2=0x5574fa2df700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e20ff9640 0 client.18669 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 -- 172.21.15.115:0/948388190 shutdown_connections 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 1 -- 172.21.15.115:0/948388190 wait complete. 2024-05-09T05:52:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.045+0000 7f1e4617b740 20 client.18669 trim_cache size 0 max 16384 2024-05-09T05:52:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.060+0000 7f1e4617b740 1 Processor -- start 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.060+0000 7f1e4617b740 1 -- start start 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa2e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f204d0 0x5574fa2e8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f411b0 0x5574fa2e8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e4617b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c00ffb0 con 0x5574f9f411b0 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e4617b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c011b20 con 0x5574f9f204d0 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e4617b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c010680 con 0x5574f9f182f0 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f411b0 0x5574fa2e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f411b0 0x5574fa2e8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45154/0 (socket says 172.21.15.115:45154) 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 -- 172.21.15.115:0/2919941571 learned_addr learned my addr 172.21.15.115:0/2919941571 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3d059640 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f204d0 0x5574fa2e8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa2e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 -- 172.21.15.115:0/2919941571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 msgr2=0x5574fa2e0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa2e0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 -- 172.21.15.115:0/2919941571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f204d0 msgr2=0x5574fa2e8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f204d0 0x5574fa2e8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 -- 172.21.15.115:0/2919941571 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fa0235f0 con 0x5574f9f411b0 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3d059640 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f204d0 0x5574fa2e8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e3c858640 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f411b0 0x5574fa2e8940 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1e34000c00 tx=0x7f1e34005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e46f7f6f168694a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e22ffd640 1 -- 172.21.15.115:0/2919941571 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34041070 con 0x5574f9f411b0 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e22ffd640 1 -- 172.21.15.115:0/2919941571 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e3403dcf0 con 0x5574f9f411b0 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.061+0000 7f1e22ffd640 1 -- 172.21.15.115:0/2919941571 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34035040 con 0x5574f9f411b0 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 -- 172.21.15.115:0/2919941571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f411b0 msgr2=0x5574fa2e8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f411b0 0x5574fa2e8940 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1e34000c00 tx=0x7f1e34005fd0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 -- 172.21.15.115:0/2919941571 shutdown_connections 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f411b0 0x5574fa2e8940 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f204d0 0x5574fa2e8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2919941571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa2e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 -- 172.21.15.115:0/2919941571 >> 172.21.15.115:0/2919941571 conn(0x5574f9f35300 msgr2=0x5574fa2e0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 -- 172.21.15.115:0/2919941571 shutdown_connections 2024-05-09T05:52:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 -- 172.21.15.115:0/2919941571 wait complete. 2024-05-09T05:52:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.062+0000 7f1e4617b740 1 -- start start 2024-05-09T05:52:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e4617b740 1 Processor -- start 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa300ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa3011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e4617b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa301720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e4617b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c011450 con 0x5574f9f204d0 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e4617b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c010d80 con 0x5574f9f411b0 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e4617b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c00dfb0 con 0x5574f9f182f0 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e4617b740 10 client.0 _ll_register_callbacks cb 0x7f1e34032d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3d059640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa3011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3d059640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa3011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45166/0 (socket says 172.21.15.115:45166) 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3d059640 1 -- 172.21.15.115:0/2078016815 learned_addr learned my addr 172.21.15.115:0/2078016815 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3c858640 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa301720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3c858640 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa300ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3d059640 1 -- 172.21.15.115:0/2078016815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 msgr2=0x5574fa300ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3d059640 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa300ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3d059640 1 -- 172.21.15.115:0/2078016815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 msgr2=0x5574fa301720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3d059640 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa301720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3d059640 1 -- 172.21.15.115:0/2078016815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fa20ded0 con 0x5574f9f204d0 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e3d059640 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa3011e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x5574fa2e1dc0 tx=0x7f1e28009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f713f39aa186dad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.063+0000 7f1e20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e20ff9640 1 -- 172.21.15.115:0/2078016815 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28005220 con 0x5574f9f204d0 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e20ff9640 1 -- 172.21.15.115:0/2078016815 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9526+0+0 (secure 0 0 0) 0x7f1e280053c0 con 0x5574f9f204d0 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e4617b740 1 -- 172.21.15.115:0/2078016815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3403dcf0 con 0x5574f9f204d0 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e20ff9640 1 -- 172.21.15.115:0/2078016815 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1e2802c050 con 0x5574f9f204d0 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e4617b740 10 client.18675 fetch_fsmap learned FSMap version 58 2024-05-09T05:52:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e4617b740 1 -- 172.21.15.115:0/2078016815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5574fa368240 con 0x5574f9f204d0 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e20ff9640 1 -- 172.21.15.115:0/2078016815 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e3403dcf0 con 0x5574f9f204d0 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e20ff9640 1 -- 172.21.15.115:0/2078016815 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 58) ==== 8366+0+0 (secure 0 0 0) 0x7f1e2800f040 con 0x5574f9f204d0 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e4617b740 10 client.18675 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e4617b740 10 client.18675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e4617b740 20 client.18675 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574f9ed51c0 0x5574fa36a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e4617b740 4 client.18675 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.064+0000 7f1e4617b740 1 -- 172.21.15.115:0/2078016815 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574fa1cc350 con 0x5574f9ed51c0 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.065+0000 7f1e3c858640 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574f9ed51c0 0x5574fa36a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.065+0000 7f1e3c858640 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574f9ed51c0 0x5574fa36a980 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960289b8a937dfe5 server_cookie=8b81cba4bebc65b1 in_seq=0 out_seq=0 2024-05-09T05:52:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.065+0000 7f1e20ff9640 10 client.18675 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.074+0000 7f1e20ff9640 1 -- 172.21.15.115:0/2078016815 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5574fa1cc350 con 0x5574f9ed51c0 2024-05-09T05:52:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.074+0000 7f1e20ff9640 10 client.18675 handle_command_reply: tid=0 2024-05-09T05:52:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.074+0000 7f1e4617b740 1 client.18675 shutdown 2024-05-09T05:52:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.074+0000 7f1e4617b740 1 -- 172.21.15.115:0/2078016815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 msgr2=0x5574fa3011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.074+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa3011e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x5574fa2e1dc0 tx=0x7f1e28009b90 comp rx=0 tx=0).stop 2024-05-09T05:52:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e4617b740 1 -- 172.21.15.115:0/2078016815 shutdown_connections 2024-05-09T05:52:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574f9f411b0 0x5574fa301720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574f9f204d0 0x5574fa3011e0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574f9ed51c0 0x5574fa36a980 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e4617b740 1 --2- 172.21.15.115:0/2078016815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574f9f182f0 0x5574fa300ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e20ff9640 0 client.18675 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e4617b740 1 -- 172.21.15.115:0/2078016815 >> 172.21.15.115:0/2078016815 conn(0x5574f9f35300 msgr2=0x5574fa2ed4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e4617b740 1 -- 172.21.15.115:0/2078016815 shutdown_connections 2024-05-09T05:52:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e4617b740 1 -- 172.21.15.115:0/2078016815 wait complete. 2024-05-09T05:52:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e4617b740 20 client.18675 trim_cache size 0 max 16384 2024-05-09T05:52:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:52:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:52:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:52:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e240b2950 msgr2=0x7f1e240b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e240b2950 0x7f1e240b4e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e300dfe90 tx=0x7f1e30073210 comp rx=0 tx=0).stop 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 msgr2=0x7f1e38199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 0x7f1e38199f10 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000eaa0 tx=0x7f1e3000ef60 comp rx=0 tx=0).stop 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/4169237777 shutdown_connections 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e240b2950 0x7f1e240b4e60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e38109060 0x7f1e38199f10 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e38108bc0 0x7f1e381999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 --2- 172.21.15.115:0/4169237777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e38103310 0x7f1e38199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/4169237777 >> 172.21.15.115:0/4169237777 conn(0x7f1e380fe390 msgr2=0x7f1e3810e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/4169237777 shutdown_connections 2024-05-09T05:52:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:10.075+0000 7f1e3f2bd640 1 -- 172.21.15.115:0/4169237777 wait complete. 2024-05-09T05:52:10.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:10 smithi195 ceph-mon[33844]: osdmap e106: 12 total, 12 up, 12 in 2024-05-09T05:52:10.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:10 smithi115 ceph-mon[30349]: osdmap e106: 12 total, 12 up, 12 in 2024-05-09T05:52:10.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:10 smithi161 ceph-mon[33805]: osdmap e106: 12 total, 12 up, 12 in 2024-05-09T05:52:11.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:11 smithi195 ceph-mon[33844]: pgmap v440: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-05-09T05:52:11.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:11 smithi115 ceph-mon[30349]: pgmap v440: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-05-09T05:52:11.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:11 smithi161 ceph-mon[33805]: pgmap v440: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-05-09T05:52:12.691 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:52:12.948 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:52:13.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:13 smithi115 ceph-mon[30349]: pgmap v441: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-05-09T05:52:13.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:13 smithi195 ceph-mon[33844]: pgmap v441: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-05-09T05:52:13.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:13 smithi161 ceph-mon[33805]: pgmap v441: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-05-09T05:52:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.779+0000 7f5e78d8f700 1 -- 172.21.15.115:0/2492805971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e741039d0 msgr2=0x7f5e741106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.779+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/2492805971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e741039d0 0x7f5e741106f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009990 tx=0x7f5e68009ca0 comp rx=0 tx=0).stop 2024-05-09T05:52:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.779+0000 7f5e78d8f700 1 -- 172.21.15.115:0/2492805971 shutdown_connections 2024-05-09T05:52:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.779+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/2492805971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103080 0x7f5e74103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.779+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/2492805971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e74110c30 0x7f5e74113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.779+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/2492805971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e741039d0 0x7f5e741106f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.779+0000 7f5e78d8f700 1 -- 172.21.15.115:0/2492805971 >> 172.21.15.115:0/2492805971 conn(0x7f5e740fedf0 msgr2=0x7f5e74101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.779+0000 7f5e78d8f700 1 -- 172.21.15.115:0/2492805971 shutdown_connections 2024-05-09T05:52:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.779+0000 7f5e78d8f700 1 -- 172.21.15.115:0/2492805971 wait complete. 2024-05-09T05:52:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e78d8f700 1 Processor -- start 2024-05-09T05:52:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e78d8f700 1 -- start start 2024-05-09T05:52:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e78d8f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103080 0x7f5e7419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e78d8f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e741039d0 0x7f5e7419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e78d8f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74110c30 0x7f5e741a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e78d8f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74116510 con 0x7f5e74110c30 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e78d8f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74116390 con 0x7f5e74103080 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e78d8f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74116690 con 0x7f5e741039d0 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e72d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74110c30 0x7f5e741a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e72d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74110c30 0x7f5e741a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39732/0 (socket says 172.21.15.115:39732) 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e71d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e741039d0 0x7f5e7419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e72d9d700 1 -- 172.21.15.115:0/711041366 learned_addr learned my addr 172.21.15.115:0/711041366 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.780+0000 7f5e7259c700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103080 0x7f5e7419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e72d9d700 1 -- 172.21.15.115:0/711041366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e741039d0 msgr2=0x7f5e7419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e72d9d700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e741039d0 0x7f5e7419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e72d9d700 1 -- 172.21.15.115:0/711041366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103080 msgr2=0x7f5e7419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e72d9d700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103080 0x7f5e7419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e72d9d700 1 -- 172.21.15.115:0/711041366 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e741a1df0 con 0x7f5e74110c30 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e72d9d700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74110c30 0x7f5e741a1750 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c007ee0 tx=0x7f5e6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e637fe700 1 -- 172.21.15.115:0/711041366 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c004bb0 con 0x7f5e74110c30 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e741a20e0 con 0x7f5e74110c30 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e637fe700 1 -- 172.21.15.115:0/711041366 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e6c004d50 con 0x7f5e74110c30 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e637fe700 1 -- 172.21.15.115:0/711041366 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0058b0 con 0x7f5e74110c30 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.781+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e741a9a00 con 0x7f5e74110c30 2024-05-09T05:52:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.782+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e740512c0 con 0x7f5e74110c30 2024-05-09T05:52:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.784+0000 7f5e637fe700 1 -- 172.21.15.115:0/711041366 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f5e6c01a020 con 0x7f5e74110c30 2024-05-09T05:52:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.784+0000 7f5e637fe700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f5e5c0b29f0 0x7f5e5c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.784+0000 7f5e7259c700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f5e5c0b29f0 0x7f5e5c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.784+0000 7f5e637fe700 1 -- 172.21.15.115:0/711041366 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9526+0+0 (secure 0 0 0) 0x7f5e6c0e2700 con 0x7f5e74110c30 2024-05-09T05:52:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.785+0000 7f5e7259c700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f5e5c0b29f0 0x7f5e5c0b4f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5e64005950 tx=0x7f5e6400a300 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:14.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:14.788+0000 7f5e637fe700 1 -- 172.21.15.115:0/711041366 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e6c010070 con 0x7f5e74110c30 2024-05-09T05:52:15.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.039+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e7402e5f0 con 0x7f5e74110c30 2024-05-09T05:52:15.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.040+0000 7f5e637fe700 1 -- 172.21.15.115:0/711041366 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11952 (secure 0 0 0) 0x7f5e6c0a9cf0 con 0x7f5e74110c30 2024-05-09T05:52:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:52:15.031 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:51:48.372201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f5e5c0b29f0 msgr2=0x7f5e5c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f5e5c0b29f0 0x7f5e5c0b4f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5e64005950 tx=0x7f5e6400a300 comp rx=0 tx=0).stop 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74110c30 msgr2=0x7f5e741a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74110c30 0x7f5e741a1750 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c007ee0 tx=0x7f5e6c00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 shutdown_connections 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f5e5c0b29f0 0x7f5e5c0b4f00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74103080 0x7f5e7419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e74110c30 0x7f5e741a1750 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 --2- 172.21.15.115:0/711041366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e741039d0 0x7f5e7419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 >> 172.21.15.115:0/711041366 conn(0x7f5e740fedf0 msgr2=0x7f5e741128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 shutdown_connections 2024-05-09T05:52:15.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:15.042+0000 7f5e78d8f700 1 -- 172.21.15.115:0/711041366 wait complete. 2024-05-09T05:52:15.034 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 58 2024-05-09T05:52:15.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:15 smithi115 ceph-mon[30349]: pgmap v442: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-05-09T05:52:15.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:15 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/711041366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:52:15.701 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 2c4ca6bd --release --await' 2024-05-09T05:52:15.702 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id 2c4ca6bd --release --await 2024-05-09T05:52:15.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:15 smithi195 ceph-mon[33844]: pgmap v442: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-05-09T05:52:15.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:15 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/711041366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:52:15.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:15 smithi161 ceph-mon[33805]: pgmap v442: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-05-09T05:52:15.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:15 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/711041366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:52:15.952 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:52:17.751 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:17 smithi115 ceph-mon[30349]: pgmap v443: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-05-09T05:52:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.761+0000 7fabd2942700 1 -- 172.21.15.115:0/3055542161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc1086b0 msgr2=0x7fabcc108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.761+0000 7fabd2942700 1 --2- 172.21.15.115:0/3055542161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc1086b0 0x7fabcc108b80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b0a0 tx=0x7fabc400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.761+0000 7fabd2942700 1 -- 172.21.15.115:0/3055542161 shutdown_connections 2024-05-09T05:52:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.761+0000 7fabd2942700 1 --2- 172.21.15.115:0/3055542161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc1086b0 0x7fabcc108b80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.761+0000 7fabd2942700 1 --2- 172.21.15.115:0/3055542161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabcc103260 0x7fabcc103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.761+0000 7fabd2942700 1 --2- 172.21.15.115:0/3055542161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabcc103bb0 0x7fabcc1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.761+0000 7fabd2942700 1 -- 172.21.15.115:0/3055542161 >> 172.21.15.115:0/3055542161 conn(0x7fabcc0fef20 msgr2=0x7fabcc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.761+0000 7fabd2942700 1 -- 172.21.15.115:0/3055542161 shutdown_connections 2024-05-09T05:52:17.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.761+0000 7fabd2942700 1 -- 172.21.15.115:0/3055542161 wait complete. 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd2942700 1 Processor -- start 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd2942700 1 -- start start 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd2942700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc103260 0x7fabcc1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd2942700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabcc103bb0 0x7fabcc1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd2942700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabcc1086b0 0x7fabcc1a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd2942700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc11a930 con 0x7fabcc103260 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd2942700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc11a7b0 con 0x7fabcc1086b0 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd2942700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc11aab0 con 0x7fabcc103bb0 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabcbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc103260 0x7fabcc1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabcbfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc103260 0x7fabcc1a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39752/0 (socket says 172.21.15.115:39752) 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabcbfff700 1 -- 172.21.15.115:0/44005815 learned_addr learned my addr 172.21.15.115:0/44005815 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd0edf700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabcc1086b0 0x7fabcc1a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabc3fff700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabcc103bb0 0x7fabcc1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabcbfff700 1 -- 172.21.15.115:0/44005815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabcc103bb0 msgr2=0x7fabcc1a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabcbfff700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabcc103bb0 0x7fabcc1a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabcbfff700 1 -- 172.21.15.115:0/44005815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabcc1086b0 msgr2=0x7fabcc1a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabcbfff700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabcc1086b0 0x7fabcc1a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabcbfff700 1 -- 172.21.15.115:0/44005815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc1a6040 con 0x7fabcc103260 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.762+0000 7fabd0edf700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabcc1086b0 0x7fabcc1a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.763+0000 7fabcbfff700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc103260 0x7fabcc1a1220 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c9b0 tx=0x7fabbc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.763+0000 7fabc9ffb700 1 -- 172.21.15.115:0/44005815 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc007b00 con 0x7fabcc103260 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.763+0000 7fabd2942700 1 -- 172.21.15.115:0/44005815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabcc1a6330 con 0x7fabcc103260 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.763+0000 7fabc9ffb700 1 -- 172.21.15.115:0/44005815 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabbc007ca0 con 0x7fabcc103260 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.763+0000 7fabc9ffb700 1 -- 172.21.15.115:0/44005815 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc018660 con 0x7fabcc103260 2024-05-09T05:52:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.763+0000 7fabd2942700 1 -- 172.21.15.115:0/44005815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabcc1add00 con 0x7fabcc103260 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.765+0000 7fabc9ffb700 1 -- 172.21.15.115:0/44005815 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fabbc0188c0 con 0x7fabcc103260 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.766+0000 7fabc9ffb700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabb80b29f0 0x7fabb80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.766+0000 7fabc3fff700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabb80b29f0 0x7fabb80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.766+0000 7fabc9ffb700 1 -- 172.21.15.115:0/44005815 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9526+0+0 (secure 0 0 0) 0x7fabbc0e2ae0 con 0x7fabcc103260 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabc3fff700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabb80b29f0 0x7fabb80b4f00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fabb4005fd0 tx=0x7fabb400af20 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabe1358b80 1 Processor -- start 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabe1358b80 1 -- start start 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed0ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed0cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed06f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabe1358b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f72ec290f0 con 0x55f72ed06b50 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabe1358b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f72ebc78f0 con 0x55f72ec73260 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabe1358b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f72ec250b0 con 0x55f72eb895a0 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabd0edf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed06f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabd0edf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed06f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39760/0 (socket says 172.21.15.115:39760) 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.767+0000 7fabd0edf700 1 -- 172.21.15.115:0/120570968 learned_addr learned my addr 172.21.15.115:0/120570968 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabc3fff700 1 --2- 172.21.15.115:0/120570968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed0cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabcbfff700 1 --2- 172.21.15.115:0/120570968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed0ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabd0edf700 1 -- 172.21.15.115:0/120570968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 msgr2=0x55f72ed0cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabd0edf700 1 --2- 172.21.15.115:0/120570968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed0cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabd0edf700 1 -- 172.21.15.115:0/120570968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 msgr2=0x55f72ed0ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabd0edf700 1 --2- 172.21.15.115:0/120570968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed0ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabd0edf700 1 -- 172.21.15.115:0/120570968 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72ebbf470 con 0x55f72ed06b50 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabd0edf700 1 --2- 172.21.15.115:0/120570968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed06f60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b830 tx=0x7fabc40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59716ac4e4aa16a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabc27fc700 1 -- 172.21.15.115:0/120570968 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4003a00 con 0x55f72ed06b50 2024-05-09T05:52:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabc27fc700 1 -- 172.21.15.115:0/120570968 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabc4003ba0 con 0x55f72ed06b50 2024-05-09T05:52:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.768+0000 7fabc27fc700 1 -- 172.21.15.115:0/120570968 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc401c6c0 con 0x55f72ed06b50 2024-05-09T05:52:17.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.769+0000 7fabe1358b80 1 -- 172.21.15.115:0/120570968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 msgr2=0x55f72ed06f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:17.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.769+0000 7fabe1358b80 1 --2- 172.21.15.115:0/120570968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed06f60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b830 tx=0x7fabc40092e0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.769+0000 7fabe1358b80 1 -- 172.21.15.115:0/120570968 shutdown_connections 2024-05-09T05:52:17.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.769+0000 7fabe1358b80 1 --2- 172.21.15.115:0/120570968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed0ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.769+0000 7fabe1358b80 1 --2- 172.21.15.115:0/120570968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed06f60 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.769+0000 7fabe1358b80 1 --2- 172.21.15.115:0/120570968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed0cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.769+0000 7fabe1358b80 1 -- 172.21.15.115:0/120570968 >> 172.21.15.115:0/120570968 conn(0x55f72ebe97a0 msgr2=0x55f72ed01a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.769+0000 7fabe1358b80 1 -- 172.21.15.115:0/120570968 shutdown_connections 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.769+0000 7fabe1358b80 1 -- 172.21.15.115:0/120570968 wait complete. 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabe1358b80 1 -- start start 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabe1358b80 1 Processor -- start 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed10710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed10c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed11190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabe1358b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f72eaa4ca0 con 0x55f72ed06b50 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabe1358b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f72ebe3250 con 0x55f72ec73260 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabe1358b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f72ed07970 con 0x55f72eb895a0 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabe1358b80 10 client.0 _ll_register_callbacks cb 0x55f72ea9a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabd0edf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed11190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabd0edf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed11190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39764/0 (socket says 172.21.15.115:39764) 2024-05-09T05:52:17.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabd0edf700 1 -- 172.21.15.115:0/1304589827 learned_addr learned my addr 172.21.15.115:0/1304589827 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabc3fff700 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed10c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.770+0000 7fabcbfff700 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed10710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabd0edf700 1 -- 172.21.15.115:0/1304589827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 msgr2=0x55f72ed10710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabd0edf700 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed10710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabd0edf700 1 -- 172.21.15.115:0/1304589827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 msgr2=0x55f72ed10c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabd0edf700 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed10c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabd0edf700 1 -- 172.21.15.115:0/1304589827 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72eaa4590 con 0x55f72ed06b50 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabd0edf700 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed11190 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fabc4007ef0 tx=0x7fabc4007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4f82dbe8f536f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabab7fe700 1 -- 172.21.15.115:0/1304589827 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc40054d0 con 0x55f72ed06b50 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabab7fe700 1 -- 172.21.15.115:0/1304589827 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9526+0+0 (secure 0 0 0) 0x7fabc4005670 con 0x55f72ed06b50 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabe1358b80 1 -- 172.21.15.115:0/1304589827 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc4003ba0 con 0x55f72ed06b50 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabab7fe700 1 -- 172.21.15.115:0/1304589827 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fabc4035050 con 0x55f72ed06b50 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabe1358b80 10 client.18693 fetch_fsmap learned FSMap version 58 2024-05-09T05:52:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabe1358b80 1 -- 172.21.15.115:0/1304589827 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f72ed1e030 con 0x55f72ed06b50 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.771+0000 7fabab7fe700 1 -- 172.21.15.115:0/1304589827 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabc4003ba0 con 0x55f72ed06b50 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.772+0000 7fabab7fe700 1 -- 172.21.15.115:0/1304589827 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 58) ==== 8366+0+0 (secure 0 0 0) 0x7fabc4011070 con 0x55f72ed06b50 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.772+0000 7fabe1358b80 10 client.18693 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.772+0000 7fabe1358b80 10 client.18693 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.772+0000 7fabe1358b80 20 client.18693 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.772+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f72ed20500 0x55f72ed22920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.772+0000 7fabe1358b80 4 client.18693 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.772+0000 7fabe1358b80 1 -- 172.21.15.115:0/1304589827 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72ec36ad0 con 0x55f72ed20500 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.772+0000 7fabcbfff700 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f72ed20500 0x55f72ed22920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.773+0000 7fabab7fe700 10 client.18693 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.773+0000 7fabcbfff700 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f72ed20500 0x55f72ed22920 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a162d3963a8c8a server_cookie=745c0a173bc6b0e5 in_seq=0 out_seq=0 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.773+0000 7fabab7fe700 1 -- 172.21.15.115:0/1304589827 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f72ec36ad0 con 0x55f72ed20500 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.773+0000 7fabab7fe700 10 client.18693 handle_command_reply: tid=0 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.773+0000 7fabe1358b80 1 client.18693 shutdown 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 -- 172.21.15.115:0/1304589827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 msgr2=0x55f72ed11190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed11190 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fabc4007ef0 tx=0x7fabc4007a60 comp rx=0 tx=0).stop 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 -- 172.21.15.115:0/1304589827 shutdown_connections 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f72ed20500 0x55f72ed22920 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed11190 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed10710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabab7fe700 0 client.18693 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1304589827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed10c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 -- 172.21.15.115:0/1304589827 >> 172.21.15.115:0/1304589827 conn(0x55f72ebe97a0 msgr2=0x55f72ed01750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 -- 172.21.15.115:0/1304589827 shutdown_connections 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 1 -- 172.21.15.115:0/1304589827 wait complete. 2024-05-09T05:52:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.774+0000 7fabe1358b80 20 client.18693 trim_cache size 0 max 16384 2024-05-09T05:52:17.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.793+0000 7fabe1358b80 1 Processor -- start 2024-05-09T05:52:17.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabe1358b80 1 -- start start 2024-05-09T05:52:17.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72eb895a0 0x55f72ed06570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72ec73260 0x55f72ed124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed0b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabe1358b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1b6920 con 0x55f72ed06b50 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabe1358b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1b8490 con 0x55f72eb895a0 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabe1358b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1b6ff0 con 0x55f72ec73260 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabd0edf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed0b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabd0edf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed0b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39768/0 (socket says 172.21.15.115:39768) 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabcbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72eb895a0 0x55f72ed06570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabd0edf700 1 -- 172.21.15.115:0/490953050 learned_addr learned my addr 172.21.15.115:0/490953050 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabc3fff700 1 --2- 172.21.15.115:0/490953050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72ec73260 0x55f72ed124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabd0edf700 1 -- 172.21.15.115:0/490953050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72ec73260 msgr2=0x55f72ed124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabd0edf700 1 --2- 172.21.15.115:0/490953050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72ec73260 0x55f72ed124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabd0edf700 1 -- 172.21.15.115:0/490953050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72eb895a0 msgr2=0x55f72ed06570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabd0edf700 1 --2- 172.21.15.115:0/490953050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72eb895a0 0x55f72ed06570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabd0edf700 1 -- 172.21.15.115:0/490953050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72ed0fc90 con 0x55f72ed06b50 2024-05-09T05:52:17.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.794+0000 7fabd0edf700 1 --2- 172.21.15.115:0/490953050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed0b3b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009a20 tx=0x7fabc40079f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f361e52f72975a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabc1ffb700 1 -- 172.21.15.115:0/490953050 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4005c60 con 0x55f72ed06b50 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabc1ffb700 1 -- 172.21.15.115:0/490953050 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabc4005e00 con 0x55f72ed06b50 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabc1ffb700 1 -- 172.21.15.115:0/490953050 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4005530 con 0x55f72ed06b50 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabe1358b80 1 -- 172.21.15.115:0/490953050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 msgr2=0x55f72ed0b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabe1358b80 1 --2- 172.21.15.115:0/490953050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed0b3b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009a20 tx=0x7fabc40079f0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabe1358b80 1 -- 172.21.15.115:0/490953050 shutdown_connections 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabe1358b80 1 --2- 172.21.15.115:0/490953050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72eb895a0 0x55f72ed06570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabe1358b80 1 --2- 172.21.15.115:0/490953050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed0b3b0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabe1358b80 1 --2- 172.21.15.115:0/490953050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72ec73260 0x55f72ed124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.795+0000 7fabe1358b80 1 -- 172.21.15.115:0/490953050 >> 172.21.15.115:0/490953050 conn(0x55f72ebe97a0 msgr2=0x55f72ed11dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 -- 172.21.15.115:0/490953050 shutdown_connections 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 -- 172.21.15.115:0/490953050 wait complete. 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 -- start start 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 Processor -- start 2024-05-09T05:52:17.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed25aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed26040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed23f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1b7dc0 con 0x55f72ed06b50 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1b76f0 con 0x55f72ec73260 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.796+0000 7fabe1358b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1b48e0 con 0x55f72eb895a0 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabe1358b80 10 client.0 _ll_register_callbacks cb 0x55f72ea9a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabd0edf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed23f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabd0edf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed23f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39780/0 (socket says 172.21.15.115:39780) 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabd0edf700 1 -- 172.21.15.115:0/1562157689 learned_addr learned my addr 172.21.15.115:0/1562157689 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabcbfff700 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed25aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabc3fff700 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed26040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabd0edf700 1 -- 172.21.15.115:0/1562157689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 msgr2=0x55f72ed25aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabd0edf700 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed25aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabd0edf700 1 -- 172.21.15.115:0/1562157689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 msgr2=0x55f72ed26040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabd0edf700 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed26040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabd0edf700 1 -- 172.21.15.115:0/1562157689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72ed24730 con 0x55f72ed06b50 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabd0edf700 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed23f90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fabc4002230 tx=0x7fabc4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2232a620cda74d45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabab7fe700 1 -- 172.21.15.115:0/1562157689 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4048060 con 0x55f72ed06b50 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabab7fe700 1 -- 172.21.15.115:0/1562157689 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9526+0+0 (secure 0 0 0) 0x7fabc401c650 con 0x55f72ed06b50 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabe1358b80 1 -- 172.21.15.115:0/1562157689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc4005e00 con 0x55f72ed06b50 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.797+0000 7fabab7fe700 1 -- 172.21.15.115:0/1562157689 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fabc404f050 con 0x55f72ed06b50 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabe1358b80 10 client.18699 fetch_fsmap learned FSMap version 58 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabe1358b80 1 -- 172.21.15.115:0/1562157689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f72ed8b600 con 0x55f72ed06b50 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabab7fe700 1 -- 172.21.15.115:0/1562157689 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabc4005e00 con 0x55f72ed06b50 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabab7fe700 1 -- 172.21.15.115:0/1562157689 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 58) ==== 8366+0+0 (secure 0 0 0) 0x7fabc4011030 con 0x55f72ed06b50 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabe1358b80 10 client.18699 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabe1358b80 10 client.18699 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabe1358b80 20 client.18699 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:17.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f72ed8bba0 0x55f72ed8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:17.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabe1358b80 4 client.18699 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2c4ca6bd", "release": true, "await": true}] 2024-05-09T05:52:17.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabe1358b80 1 -- 172.21.15.115:0/1562157689 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2c4ca6bd", "release": true, "await": true}) -- 0x55f72ec30b70 con 0x55f72ed8bba0 2024-05-09T05:52:17.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.798+0000 7fabcbfff700 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f72ed8bba0 0x55f72ed8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:17.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.799+0000 7fabcbfff700 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f72ed8bba0 0x55f72ed8dfc0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c024ada24f98c2c server_cookie=9500855256632c6 in_seq=0 out_seq=0 2024-05-09T05:52:17.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.799+0000 7fabab7fe700 10 client.18699 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:17.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabab7fe700 1 -- 172.21.15.115:0/1562157689 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f72ec30b70 con 0x55f72ed8bba0 2024-05-09T05:52:17.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabab7fe700 10 client.18699 handle_command_reply: tid=0 2024-05-09T05:52:17.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 client.18699 shutdown 2024-05-09T05:52:17.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 -- 172.21.15.115:0/1562157689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 msgr2=0x55f72ed23f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:17.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed23f90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fabc4002230 tx=0x7fabc4005fb0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 -- 172.21.15.115:0/1562157689 shutdown_connections 2024-05-09T05:52:17.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f72ed8bba0 0x55f72ed8dfc0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f72ed06b50 0x55f72ed23f90 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f72eb895a0 0x55f72ed25aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabab7fe700 0 client.18699 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 --2- 172.21.15.115:0/1562157689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f72ec73260 0x55f72ed26040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 -- 172.21.15.115:0/1562157689 >> 172.21.15.115:0/1562157689 conn(0x55f72ebe97a0 msgr2=0x55f72ed062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 -- 172.21.15.115:0/1562157689 shutdown_connections 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 1 -- 172.21.15.115:0/1562157689 wait complete. 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.802+0000 7fabe1358b80 20 client.18699 trim_cache size 0 max 16384 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 58, 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:52:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 44, 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "2c4ca6bd": { 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 44, 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 150.0, 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:52:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:52:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:52:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:52:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:52:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:52:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:52:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:52:17.799 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:52:17.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 -- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabb80b29f0 msgr2=0x7fabb80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabb80b29f0 0x7fabb80b4f00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fabb4005fd0 tx=0x7fabb400af20 comp rx=0 tx=0).stop 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 -- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc103260 msgr2=0x7fabcc1a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc103260 0x7fabcc1a1220 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c9b0 tx=0x7fabbc00cd70 comp rx=0 tx=0).stop 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 -- 172.21.15.115:0/44005815 shutdown_connections 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fabb80b29f0 0x7fabb80b4f00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabcc1086b0 0x7fabcc1a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabcc103260 0x7fabcc1a1220 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 --2- 172.21.15.115:0/44005815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabcc103bb0 0x7fabcc1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 -- 172.21.15.115:0/44005815 >> 172.21.15.115:0/44005815 conn(0x7fabcc0fef20 msgr2=0x7fabcc106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 -- 172.21.15.115:0/44005815 shutdown_connections 2024-05-09T05:52:17.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:52:17.803+0000 7fabd2942700 1 -- 172.21.15.115:0/44005815 wait complete. 2024-05-09T05:52:17.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:17 smithi195 ceph-mon[33844]: pgmap v443: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-05-09T05:52:17.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:17 smithi161 ceph-mon[33805]: pgmap v443: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-05-09T05:52:18.533 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2c4ca6bd', total seconds elapsed: 35.9 2024-05-09T05:52:19.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:19 smithi195 ceph-mon[33844]: pgmap v444: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T05:52:19.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:19 smithi195 ceph-mon[33844]: osdmap e107: 12 total, 12 up, 12 in 2024-05-09T05:52:19.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:19 smithi115 ceph-mon[30349]: pgmap v444: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T05:52:19.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:19 smithi115 ceph-mon[30349]: osdmap e107: 12 total, 12 up, 12 in 2024-05-09T05:52:19.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:19 smithi161 ceph-mon[33805]: pgmap v444: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T05:52:19.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:19 smithi161 ceph-mon[33805]: osdmap e107: 12 total, 12 up, 12 in 2024-05-09T05:52:21.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:21 smithi195 ceph-mon[33844]: pgmap v446: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T05:52:21.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:21 smithi115 ceph-mon[30349]: pgmap v446: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T05:52:21.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:21 smithi161 ceph-mon[33805]: pgmap v446: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T05:52:23.117 INFO:tasks.workunit.client.0.smithi115.stdout:fs setup took 687 secs 2024-05-09T05:52:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.189+0000 7f9468176640 1 -- 172.21.15.115:0/69641706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 msgr2=0x7f9460102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.189+0000 7f9468176640 1 --2- 172.21.15.115:0/69641706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 0x7f9460102c80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9450009900 tx=0x7f945002a410 comp rx=0 tx=0).stop 2024-05-09T05:52:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.189+0000 7f9468176640 1 -- 172.21.15.115:0/69641706 shutdown_connections 2024-05-09T05:52:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.189+0000 7f9468176640 1 --2- 172.21.15.115:0/69641706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94601032f0 0x7f946010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.189+0000 7f9468176640 1 --2- 172.21.15.115:0/69641706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 0x7f9460102c80 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.189+0000 7f9468176640 1 --2- 172.21.15.115:0/69641706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9460108ba0 0x7f9460102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.189+0000 7f9468176640 1 -- 172.21.15.115:0/69641706 >> 172.21.15.115:0/69641706 conn(0x7f94600782e0 msgr2=0x7f9460078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.189+0000 7f9468176640 1 -- 172.21.15.115:0/69641706 shutdown_connections 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.189+0000 7f9468176640 1 -- 172.21.15.115:0/69641706 wait complete. 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9468176640 1 Processor -- start 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9468176640 1 -- start start 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9468176640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94601032f0 0x7f946019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9468176640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9460108ba0 0x7f946019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9468176640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 0x7f946019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9468176640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9460112c40 con 0x7f9460109040 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9468176640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9460112ac0 con 0x7f9460108ba0 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9468176640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9460112dc0 con 0x7f94601032f0 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9466713640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 0x7f946019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9466713640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 0x7f946019c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49482/0 (socket says 172.21.15.115:49482) 2024-05-09T05:52:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.190+0000 7f9466713640 1 -- 172.21.15.115:0/2505490317 learned_addr learned my addr 172.21.15.115:0/2505490317 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9465f12640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94601032f0 0x7f946019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9465711640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9460108ba0 0x7f946019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9466713640 1 -- 172.21.15.115:0/2505490317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94601032f0 msgr2=0x7f946019b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9466713640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94601032f0 0x7f946019b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9466713640 1 -- 172.21.15.115:0/2505490317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9460108ba0 msgr2=0x7f946019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9466713640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9460108ba0 0x7f946019bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9466713640 1 -- 172.21.15.115:0/2505490317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601a69c0 con 0x7f9460109040 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9466713640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 0x7f946019c120 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f945800eaa0 tx=0x7f945800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9456ffd640 1 -- 172.21.15.115:0/2505490317 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945800cdb0 con 0x7f9460109040 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9456ffd640 1 -- 172.21.15.115:0/2505490317 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9458005020 con 0x7f9460109040 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9468176640 1 -- 172.21.15.115:0/2505490317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94601a6cb0 con 0x7f9460109040 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9456ffd640 1 -- 172.21.15.115:0/2505490317 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94580116d0 con 0x7f9460109040 2024-05-09T05:52:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.191+0000 7f9468176640 1 -- 172.21.15.115:0/2505490317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601a7180 con 0x7f9460109040 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f946f02c740 1 Processor -- start 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f9456ffd640 1 -- 172.21.15.115:0/2505490317 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f94580051d0 con 0x7f9460109040 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f946f02c740 1 -- start start 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f946f02c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0b031b0 0x560ce0ea6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f946f02c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ada2f0 0x560ce0ea6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f946f02c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0ea6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f946f02c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560ce0e0b7c0 con 0x560ce0ae24d0 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f946f02c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ce0da70e0 con 0x560ce0ada2f0 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f946f02c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560ce0e07780 con 0x560ce0b031b0 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f9465711640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0ea6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f9465711640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0ea6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49486/0 (socket says 172.21.15.115:49486) 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f9465711640 1 -- 172.21.15.115:0/2820327906 learned_addr learned my addr 172.21.15.115:0/2820327906 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.194+0000 7f9465f12640 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ada2f0 0x560ce0ea6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9465711640 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0b031b0 0x560ce0ea6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9456ffd640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f944c0b2950 0x7f944c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9465f12640 1 -- 172.21.15.115:0/2820327906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0b031b0 msgr2=0x560ce0ea6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9465f12640 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0b031b0 0x560ce0ea6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9465f12640 1 -- 172.21.15.115:0/2820327906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 msgr2=0x560ce0ea6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9465f12640 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0ea6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9466713640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f944c0b2950 0x7f944c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9465f12640 1 -- 172.21.15.115:0/2820327906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce0dd07c0 con 0x560ce0ada2f0 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9465711640 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0ea6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9456ffd640 1 -- 172.21.15.115:0/2505490317 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f94580e2ba0 con 0x7f9460109040 2024-05-09T05:52:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9465f12640 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ada2f0 0x560ce0ea6a50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f945c00bb40 tx=0x7f945c0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10779882759d2fc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f9466713640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f944c0b2950 0x7f944c0b4e60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94580e0010 tx=0x7f945809f040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f944bfff640 1 -- 172.21.15.115:0/2820327906 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c004030 con 0x560ce0ada2f0 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f944bfff640 1 -- 172.21.15.115:0/2820327906 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f945c0035f0 con 0x560ce0ada2f0 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.195+0000 7f944bfff640 1 -- 172.21.15.115:0/2820327906 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0116a0 con 0x560ce0ada2f0 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.196+0000 7f946f02c740 1 -- 172.21.15.115:0/2820327906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ada2f0 msgr2=0x560ce0ea6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.196+0000 7f946f02c740 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ada2f0 0x560ce0ea6a50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f945c00bb40 tx=0x7f945c0103d0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.196+0000 7f946f02c740 1 -- 172.21.15.115:0/2820327906 shutdown_connections 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.196+0000 7f946f02c740 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0ea6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.196+0000 7f946f02c740 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ada2f0 0x560ce0ea6a50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.196+0000 7f946f02c740 1 --2- 172.21.15.115:0/2820327906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0b031b0 0x560ce0ea6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.196+0000 7f946f02c740 1 -- 172.21.15.115:0/2820327906 >> 172.21.15.115:0/2820327906 conn(0x560ce0af7300 msgr2=0x560ce0ea2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.196+0000 7f946f02c740 1 -- 172.21.15.115:0/2820327906 shutdown_connections 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.196+0000 7f946f02c740 1 -- 172.21.15.115:0/2820327906 wait complete. 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f946f02c740 1 -- start start 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f946f02c740 1 Processor -- start 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f946f02c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0eb6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f946f02c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eb6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f946f02c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 0x560ce0eb4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f946f02c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x560ce0eb7c20 con 0x560ce0ae24d0 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f946f02c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ce0d929c0 con 0x560ce0b031b0 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f946f02c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560ce0eb7da0 con 0x560ce0ada2f0 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f946f02c740 10 client.0 _ll_register_callbacks cb 0x560ce0dbf020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465f12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eb6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465f12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eb6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49498/0 (socket says 172.21.15.115:49498) 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465f12640 1 -- 172.21.15.115:0/1663641741 learned_addr learned my addr 172.21.15.115:0/1663641741 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465711640 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0eb6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465711640 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 0x560ce0eb4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465f12640 1 -- 172.21.15.115:0/1663641741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 msgr2=0x560ce0eb6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465f12640 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0eb6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465f12640 1 -- 172.21.15.115:0/1663641741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 msgr2=0x560ce0eb4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465f12640 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 0x560ce0eb4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465f12640 1 -- 172.21.15.115:0/1663641741 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce0db91f0 con 0x560ce0ae24d0 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.197+0000 7f9465711640 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 0x560ce0eb4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.198+0000 7f94497fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.198+0000 7f9465f12640 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eb6fe0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f945c003f50 tx=0x7f945c010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffbbf7011be57ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.198+0000 7f94497fa640 1 -- 172.21.15.115:0/1663641741 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c003a30 con 0x560ce0ae24d0 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.198+0000 7f946f02c740 1 -- 172.21.15.115:0/1663641741 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945c0035f0 con 0x560ce0ae24d0 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.198+0000 7f94497fa640 1 -- 172.21.15.115:0/1663641741 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f945c003fb0 con 0x560ce0ae24d0 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.198+0000 7f94497fa640 1 -- 172.21.15.115:0/1663641741 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f945c0041b0 con 0x560ce0ae24d0 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.198+0000 7f946f02c740 10 client.18711 fetch_fsmap learned FSMap version 58 2024-05-09T05:52:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.198+0000 7f946f02c740 1 -- 172.21.15.115:0/1663641741 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x560ce0b6e790 con 0x560ce0ae24d0 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.198+0000 7f94497fa640 1 -- 172.21.15.115:0/1663641741 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f945c0035f0 con 0x560ce0ae24d0 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.199+0000 7f94497fa640 1 -- 172.21.15.115:0/1663641741 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 58) ==== 8366+0+0 (secure 0 0 0) 0x7f945c005040 con 0x560ce0ae24d0 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.199+0000 7f946f02c740 10 client.18711 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.199+0000 7f946f02c740 10 client.18711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.199+0000 7f946f02c740 20 client.18711 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.199+0000 7f946f02c740 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560ce0ac1d80 0x560ce0ec72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.199+0000 7f946f02c740 4 client.18711 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.199+0000 7f946f02c740 1 -- 172.21.15.115:0/1663641741 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ce0e193d0 con 0x560ce0ac1d80 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.199+0000 7f9465711640 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560ce0ac1d80 0x560ce0ec72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.200+0000 7f9465711640 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560ce0ac1d80 0x560ce0ec72a0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece387cc6846abbd server_cookie=43ecfc5367a0b543 in_seq=0 out_seq=0 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.200+0000 7f94497fa640 10 client.18711 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.200+0000 7f94497fa640 1 -- 172.21.15.115:0/1663641741 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ce0e193d0 con 0x560ce0ac1d80 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.200+0000 7f94497fa640 10 client.18711 handle_command_reply: tid=0 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.200+0000 7f946f02c740 1 client.18711 shutdown 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 -- 172.21.15.115:0/1663641741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 msgr2=0x560ce0eb6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eb6fe0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f945c003f50 tx=0x7f945c010ef0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 -- 172.21.15.115:0/1663641741 shutdown_connections 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 0x560ce0eb4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eb6fe0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560ce0ac1d80 0x560ce0ec72a0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 --2- 172.21.15.115:0/1663641741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0eb6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f94497fa640 0 client.18711 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 -- 172.21.15.115:0/1663641741 >> 172.21.15.115:0/1663641741 conn(0x560ce0af7300 msgr2=0x560ce0ea1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 -- 172.21.15.115:0/1663641741 shutdown_connections 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 1 -- 172.21.15.115:0/1663641741 wait complete. 2024-05-09T05:52:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.201+0000 7f946f02c740 20 client.18711 trim_cache size 0 max 16384 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f946f02c740 1 Processor -- start 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f946f02c740 1 -- start start 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f946f02c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0eb6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f946f02c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f946f02c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 0x560ce0eaee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f946f02c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94601aefc0 con 0x560ce0ae24d0 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f946f02c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94601b0b30 con 0x560ce0b031b0 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f946f02c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94601af690 con 0x560ce0ada2f0 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465f12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465f12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49510/0 (socket says 172.21.15.115:49510) 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465f12640 1 -- 172.21.15.115:0/428504847 learned_addr learned my addr 172.21.15.115:0/428504847 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465711640 1 --2- 172.21.15.115:0/428504847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0eb6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465711640 1 --2- 172.21.15.115:0/428504847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 0x560ce0eaee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465f12640 1 -- 172.21.15.115:0/428504847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 msgr2=0x560ce0eb6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465f12640 1 --2- 172.21.15.115:0/428504847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0eb6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465f12640 1 -- 172.21.15.115:0/428504847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 msgr2=0x560ce0eaee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465f12640 1 --2- 172.21.15.115:0/428504847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 0x560ce0eaee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465f12640 1 -- 172.21.15.115:0/428504847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce0be55f0 con 0x560ce0ae24d0 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.216+0000 7f9465f12640 1 --2- 172.21.15.115:0/428504847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eae8c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f945c0056d0 tx=0x7f945c00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce27296fa1b3396 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f944b7fe640 1 -- 172.21.15.115:0/428504847 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c003f80 con 0x560ce0ae24d0 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f944b7fe640 1 -- 172.21.15.115:0/428504847 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f945c0035f0 con 0x560ce0ae24d0 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f944b7fe640 1 -- 172.21.15.115:0/428504847 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0118d0 con 0x560ce0ae24d0 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f946f02c740 1 -- 172.21.15.115:0/428504847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 msgr2=0x560ce0eae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f946f02c740 1 --2- 172.21.15.115:0/428504847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eae8c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f945c0056d0 tx=0x7f945c00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f946f02c740 1 -- 172.21.15.115:0/428504847 shutdown_connections 2024-05-09T05:52:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f946f02c740 1 --2- 172.21.15.115:0/428504847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0b031b0 0x560ce0eaee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f946f02c740 1 --2- 172.21.15.115:0/428504847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0ae24d0 0x560ce0eae8c0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f946f02c740 1 --2- 172.21.15.115:0/428504847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0eb6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f946f02c740 1 -- 172.21.15.115:0/428504847 >> 172.21.15.115:0/428504847 conn(0x560ce0af7300 msgr2=0x560ce0eb66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f946f02c740 1 -- 172.21.15.115:0/428504847 shutdown_connections 2024-05-09T05:52:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.217+0000 7f946f02c740 1 -- 172.21.15.115:0/428504847 wait complete. 2024-05-09T05:52:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f946f02c740 1 -- start start 2024-05-09T05:52:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f946f02c740 1 Processor -- start 2024-05-09T05:52:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f946f02c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0ebceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f946f02c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ae24d0 0x560ce0ebd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f946f02c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0b031b0 0x560ce0eb8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f946f02c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94601b0460 con 0x560ce0b031b0 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f946f02c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94601afd90 con 0x560ce0ae24d0 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f946f02c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f94601acfc0 con 0x560ce0ada2f0 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f946f02c740 10 client.0 _ll_register_callbacks cb 0x7f9450009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f9465711640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0b031b0 0x560ce0eb8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f9465711640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0b031b0 0x560ce0eb8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49514/0 (socket says 172.21.15.115:49514) 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f9465711640 1 -- 172.21.15.115:0/3008826262 learned_addr learned my addr 172.21.15.115:0/3008826262 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f9465f12640 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ae24d0 0x560ce0ebd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f9465711640 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0ebceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f9465711640 1 -- 172.21.15.115:0/3008826262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 msgr2=0x560ce0ebceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f9465711640 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0ebceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f9465711640 1 -- 172.21.15.115:0/3008826262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ae24d0 msgr2=0x560ce0ebd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.218+0000 7f9465711640 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ae24d0 0x560ce0ebd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f9465711640 1 -- 172.21.15.115:0/3008826262 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce0dcfed0 con 0x560ce0b031b0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f9465711640 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0b031b0 0x560ce0eb8430 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f945002a930 tx=0x7f945c00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8235e83f3ba8bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f94497fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f94497fa640 1 -- 172.21.15.115:0/3008826262 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450035040 con 0x560ce0b031b0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f94497fa640 1 -- 172.21.15.115:0/3008826262 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f9450004100 con 0x560ce0b031b0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f946f02c740 1 -- 172.21.15.115:0/3008826262 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945c0035f0 con 0x560ce0b031b0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f94497fa640 1 -- 172.21.15.115:0/3008826262 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f945005e050 con 0x560ce0b031b0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f946f02c740 10 client.18717 fetch_fsmap learned FSMap version 58 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f946f02c740 1 -- 172.21.15.115:0/3008826262 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x560ce0f2ec50 con 0x560ce0b031b0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f94497fa640 1 -- 172.21.15.115:0/3008826262 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f945c0035f0 con 0x560ce0b031b0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f94497fa640 1 -- 172.21.15.115:0/3008826262 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 58) ==== 8366+0+0 (secure 0 0 0) 0x7f945003a040 con 0x560ce0b031b0 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f946f02c740 10 client.18717 fetch_fsmap finished waiting for FSMap version 58 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f946f02c740 10 client.18717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:52:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f946f02c740 20 client.18717 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f946f02c740 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560ce0a971c0 0x560ce0f313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f946f02c740 4 client.18717 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:52:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.219+0000 7f946f02c740 1 -- 172.21.15.115:0/3008826262 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ce0e13240 con 0x560ce0a971c0 2024-05-09T05:52:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.220+0000 7f9465f12640 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560ce0a971c0 0x560ce0f313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.220+0000 7f94497fa640 10 client.18717 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.220+0000 7f9465f12640 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560ce0a971c0 0x560ce0f313c0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59649a56523c1cb server_cookie=c18135d10de6d864 in_seq=0 out_seq=0 2024-05-09T05:52:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f94497fa640 1 -- 172.21.15.115:0/3008826262 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560ce0e13240 con 0x560ce0a971c0 2024-05-09T05:52:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f94497fa640 10 client.18717 handle_command_reply: tid=0 2024-05-09T05:52:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 client.18717 shutdown 2024-05-09T05:52:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 -- 172.21.15.115:0/3008826262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0b031b0 msgr2=0x560ce0eb8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0b031b0 0x560ce0eb8430 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f945002a930 tx=0x7f945c00b740 comp rx=0 tx=0).stop 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 -- 172.21.15.115:0/3008826262 shutdown_connections 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560ce0b031b0 0x560ce0eb8430 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ce0ae24d0 0x560ce0ebd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560ce0a971c0 0x560ce0f313c0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 --2- 172.21.15.115:0/3008826262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560ce0ada2f0 0x560ce0ebceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f94497fa640 0 client.18717 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 -- 172.21.15.115:0/3008826262 >> 172.21.15.115:0/3008826262 conn(0x560ce0af7300 msgr2=0x560ce0ea8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 -- 172.21.15.115:0/3008826262 shutdown_connections 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 1 -- 172.21.15.115:0/3008826262 wait complete. 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.224+0000 7f946f02c740 20 client.18717 trim_cache size 0 max 16384 2024-05-09T05:52:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:52:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:52:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:52:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 -- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f944c0b2950 msgr2=0x7f944c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f944c0b2950 0x7f944c0b4e60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94580e0010 tx=0x7f945809f040 comp rx=0 tx=0).stop 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 -- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 msgr2=0x7f946019c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 0x7f946019c120 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f945800eaa0 tx=0x7f945800ef60 comp rx=0 tx=0).stop 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 -- 172.21.15.115:0/2505490317 shutdown_connections 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f944c0b2950 0x7f944c0b4e60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9460109040 0x7f946019c120 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9460108ba0 0x7f946019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 --2- 172.21.15.115:0/2505490317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94601032f0 0x7f946019b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 -- 172.21.15.115:0/2505490317 >> 172.21.15.115:0/2505490317 conn(0x7f94600782e0 msgr2=0x7f94601012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 -- 172.21.15.115:0/2505490317 shutdown_connections 2024-05-09T05:52:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:23.225+0000 7f9468176640 1 -- 172.21.15.115:0/2505490317 wait complete. 2024-05-09T05:52:23.500 INFO:tasks.workunit.client.0.smithi115.stdout:Syncing()...0 sec 2024-05-09T05:52:23.501 INFO:tasks.workunit.client.0.smithi115.stdout:Starting Actual Benchmark At: Thu May 9 05:52:23 2024 2024-05-09T05:52:23.501 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:52:23.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:23 smithi195 ceph-mon[33844]: pgmap v447: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T05:52:23.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:23 smithi115 ceph-mon[30349]: pgmap v447: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T05:52:23.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:23 smithi161 ceph-mon[33805]: pgmap v447: 129 pgs: 129 active+clean; 12 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-05-09T05:52:25.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:24 smithi115 ceph-mon[30349]: pgmap v448: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-09T05:52:25.156 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:24 smithi161 ceph-mon[33805]: pgmap v448: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-09T05:52:25.362 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:24 smithi195 ceph-mon[33844]: pgmap v448: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-09T05:52:27.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:27 smithi195 ceph-mon[33844]: pgmap v449: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-09T05:52:27.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:27 smithi115 ceph-mon[30349]: pgmap v449: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-09T05:52:27.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:27 smithi161 ceph-mon[33805]: pgmap v449: 129 pgs: 129 active+clean; 13 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-09T05:52:29.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:29 smithi195 ceph-mon[33844]: pgmap v450: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 2.02k op/s 2024-05-09T05:52:29.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:29 smithi115 ceph-mon[30349]: pgmap v450: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 2.02k op/s 2024-05-09T05:52:29.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:29 smithi161 ceph-mon[33805]: pgmap v450: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 2.02k op/s 2024-05-09T05:52:31.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:31 smithi195 ceph-mon[33844]: pgmap v451: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-05-09T05:52:31.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:31 smithi115 ceph-mon[30349]: pgmap v451: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-05-09T05:52:31.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:31 smithi161 ceph-mon[33805]: pgmap v451: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-05-09T05:52:32.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:32 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:32.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:32 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:32.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:32 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:33.379 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:33 smithi115 ceph-mon[30349]: pgmap v452: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 MiB/s wr, 1.68k op/s 2024-05-09T05:52:33.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:33 smithi195 ceph-mon[33844]: pgmap v452: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 MiB/s wr, 1.68k op/s 2024-05-09T05:52:33.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:33 smithi161 ceph-mon[33805]: pgmap v452: 129 pgs: 129 active+clean; 13 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 MiB/s wr, 1.68k op/s 2024-05-09T05:52:35.265 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:35 smithi115 ceph-mon[30349]: pgmap v453: 129 pgs: 129 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 27 MiB/s wr, 3.65k op/s 2024-05-09T05:52:35.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:35 smithi195 ceph-mon[33844]: pgmap v453: 129 pgs: 129 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 27 MiB/s wr, 3.65k op/s 2024-05-09T05:52:35.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:35 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:35.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:35 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:35.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:35 smithi161 ceph-mon[33805]: pgmap v453: 129 pgs: 129 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 27 MiB/s wr, 3.65k op/s 2024-05-09T05:52:35.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:35 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.442+0000 7f3d25ffb640 1 -- 172.21.15.115:0/1296785069 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20032a30 con 0x7f3d2810f7b0 2024-05-09T05:52:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.442+0000 7f3d2d76b640 1 -- 172.21.15.115:0/1296785069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d2810f7b0 msgr2=0x7f3d28116f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.442+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/1296785069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d2810f7b0 0x7f3d28116f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3d20005fd0 tx=0x7f3d2002b410 comp rx=0 tx=0).stop 2024-05-09T05:52:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.443+0000 7f3d2d76b640 1 -- 172.21.15.115:0/1296785069 shutdown_connections 2024-05-09T05:52:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.443+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/1296785069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d2810f7b0 0x7f3d28116f50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.443+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/1296785069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d2810ecf0 0x7f3d2810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.443+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/1296785069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d2810a7e0 0x7f3d2810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.443+0000 7f3d2d76b640 1 -- 172.21.15.115:0/1296785069 >> 172.21.15.115:0/1296785069 conn(0x7f3d2806f780 msgr2=0x7f3d2806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.444+0000 7f3d2d76b640 1 -- 172.21.15.115:0/1296785069 shutdown_connections 2024-05-09T05:52:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.444+0000 7f3d2d76b640 1 -- 172.21.15.115:0/1296785069 wait complete. 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.445+0000 7f3d2d76b640 1 Processor -- start 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.445+0000 7f3d2d76b640 1 -- start start 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.445+0000 7f3d2d76b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d2810a7e0 0x7f3d281a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.445+0000 7f3d2d76b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d2810ecf0 0x7f3d281a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.445+0000 7f3d2d76b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d2810f7b0 0x7f3d281ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.445+0000 7f3d2d76b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2811a1d0 con 0x7f3d2810a7e0 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.445+0000 7f3d2d76b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2811a050 con 0x7f3d2810f7b0 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.445+0000 7f3d2d76b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2811a350 con 0x7f3d2810ecf0 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d277fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d2810f7b0 0x7f3d281ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d277fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d2810f7b0 0x7f3d281ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:57722/0 (socket says 172.21.15.115:57722) 2024-05-09T05:52:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d277fe640 1 -- 172.21.15.115:0/3366813812 learned_addr learned my addr 172.21.15.115:0/3366813812 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d267fc640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d2810ecf0 0x7f3d281a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d2810a7e0 0x7f3d281a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d277fe640 1 -- 172.21.15.115:0/3366813812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d2810ecf0 msgr2=0x7f3d281a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d277fe640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d2810ecf0 0x7f3d281a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d277fe640 1 -- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d2810a7e0 msgr2=0x7f3d281a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d277fe640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d2810a7e0 0x7f3d281a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d277fe640 1 -- 172.21.15.115:0/3366813812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d281abe90 con 0x7f3d2810f7b0 2024-05-09T05:52:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.446+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d2810a7e0 0x7f3d281a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:52:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.447+0000 7f3d277fe640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d2810f7b0 0x7f3d281ab690 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000d090 tx=0x7f3d2000a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.448+0000 7f3d0ffff640 1 -- 172.21.15.115:0/3366813812 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20037040 con 0x7f3d2810f7b0 2024-05-09T05:52:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.448+0000 7f3d2d76b640 1 -- 172.21.15.115:0/3366813812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281a0b30 con 0x7f3d2810f7b0 2024-05-09T05:52:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.448+0000 7f3d2d76b640 1 -- 172.21.15.115:0/3366813812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281a1020 con 0x7f3d2810f7b0 2024-05-09T05:52:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.451+0000 7f3d0ffff640 1 -- 172.21.15.115:0/3366813812 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d20042070 con 0x7f3d2810f7b0 2024-05-09T05:52:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.451+0000 7f3d0ffff640 1 -- 172.21.15.115:0/3366813812 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20008330 con 0x7f3d2810f7b0 2024-05-09T05:52:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.453+0000 7f3d3461d740 1 Processor -- start 2024-05-09T05:52:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.453+0000 7f3d3461d740 1 -- start start 2024-05-09T05:52:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.453+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1def840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.453+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1defd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.453+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a2b4d0 0x5594e1df02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.453+0000 7f3d3461d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5594e1d547c0 con 0x5594e1a4c1b0 2024-05-09T05:52:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.453+0000 7f3d3461d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5594e1cf00e0 con 0x5594e1a232f0 2024-05-09T05:52:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.453+0000 7f3d3461d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5594e1d50780 con 0x5594e1a2b4d0 2024-05-09T05:52:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.456+0000 7f3d267fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1def840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.456+0000 7f3d267fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1def840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49046/0 (socket says 172.21.15.115:49046) 2024-05-09T05:52:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.456+0000 7f3d267fc640 1 -- 172.21.15.115:0/4236619193 learned_addr learned my addr 172.21.15.115:0/4236619193 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.457+0000 7f3d267fc640 1 -- 172.21.15.115:0/4236619193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a2b4d0 msgr2=0x5594e1df02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.457+0000 7f3d267fc640 1 --2- 172.21.15.115:0/4236619193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a2b4d0 0x5594e1df02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.457+0000 7f3d267fc640 1 -- 172.21.15.115:0/4236619193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 msgr2=0x5594e1defd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:52:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.457+0000 7f3d267fc640 1 --2- 172.21.15.115:0/4236619193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1defd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.457+0000 7f3d267fc640 1 -- 172.21.15.115:0/4236619193 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594e1d197c0 con 0x5594e1a4c1b0 2024-05-09T05:52:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.457+0000 7f3d267fc640 1 --2- 172.21.15.115:0/4236619193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1def840 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800a890 tx=0x7f3d1800ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1228fd2639a13e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.459+0000 7f3d0d7fa640 1 -- 172.21.15.115:0/4236619193 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800f9c0 con 0x5594e1a4c1b0 2024-05-09T05:52:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.459+0000 7f3d0d7fa640 1 -- 172.21.15.115:0/4236619193 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d18018070 con 0x5594e1a4c1b0 2024-05-09T05:52:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.459+0000 7f3d0d7fa640 1 -- 172.21.15.115:0/4236619193 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18013dc0 con 0x5594e1a4c1b0 2024-05-09T05:52:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.464+0000 7f3d3461d740 1 -- 172.21.15.115:0/4236619193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 msgr2=0x5594e1def840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.464+0000 7f3d3461d740 1 --2- 172.21.15.115:0/4236619193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1def840 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800a890 tx=0x7f3d1800ad50 comp rx=0 tx=0).stop 2024-05-09T05:52:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.464+0000 7f3d0ffff640 1 -- 172.21.15.115:0/3366813812 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f3d20007c50 con 0x7f3d2810f7b0 2024-05-09T05:52:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.465+0000 7f3d0ffff640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3d100b2b60 0x7f3d100b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.467+0000 7f3d0ffff640 1 -- 172.21.15.115:0/3366813812 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f3d20102f50 con 0x7f3d2810f7b0 2024-05-09T05:52:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.469+0000 7f3d3461d740 1 -- 172.21.15.115:0/4236619193 shutdown_connections 2024-05-09T05:52:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.469+0000 7f3d3461d740 1 --2- 172.21.15.115:0/4236619193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a2b4d0 0x5594e1df02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.469+0000 7f3d3461d740 1 --2- 172.21.15.115:0/4236619193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1defd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.469+0000 7f3d3461d740 1 --2- 172.21.15.115:0/4236619193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1def840 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.469+0000 7f3d3461d740 1 -- 172.21.15.115:0/4236619193 >> 172.21.15.115:0/4236619193 conn(0x5594e1a40300 msgr2=0x5594e1debff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.473+0000 7f3d267fc640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3d100b2b60 0x7f3d100b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.474+0000 7f3d267fc640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3d100b2b60 0x7f3d100b5070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d18013a60 tx=0x7f3d18006fd0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.474+0000 7f3d3461d740 1 -- 172.21.15.115:0/4236619193 shutdown_connections 2024-05-09T05:52:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.477+0000 7f3d3461d740 1 -- 172.21.15.115:0/4236619193 wait complete. 2024-05-09T05:52:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.478+0000 7f3d3461d740 1 -- start start 2024-05-09T05:52:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.478+0000 7f3d3461d740 1 Processor -- start 2024-05-09T05:52:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.478+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.478+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a2b4d0 0x5594e1dfbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.478+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1df9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.478+0000 7f3d3461d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5594e1dfc970 con 0x5594e1a4c1b0 2024-05-09T05:52:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.478+0000 7f3d3461d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5594e1cdb9c0 con 0x5594e1a232f0 2024-05-09T05:52:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.478+0000 7f3d3461d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5594e1dfcaf0 con 0x5594e1a2b4d0 2024-05-09T05:52:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.478+0000 7f3d3461d740 10 client.0 _ll_register_callbacks cb 0x5594e1d08020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.479+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.479+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:57740/0 (socket says 172.21.15.115:57740) 2024-05-09T05:52:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.479+0000 7f3d26ffd640 1 -- 172.21.15.115:0/547053216 learned_addr learned my addr 172.21.15.115:0/547053216 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.479+0000 7f3d277fe640 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1df9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.480+0000 7f3d26ffd640 1 -- 172.21.15.115:0/547053216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a2b4d0 msgr2=0x5594e1dfbdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:52:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.480+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a2b4d0 0x5594e1dfbdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.480+0000 7f3d26ffd640 1 -- 172.21.15.115:0/547053216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 msgr2=0x5594e1df9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.480+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1df9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.480+0000 7f3d26ffd640 1 -- 172.21.15.115:0/547053216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594e1d021f0 con 0x5594e1a232f0 2024-05-09T05:52:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.480+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfb880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00e320 tx=0x7f3d1c00e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b2114e7055990d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.480+0000 7f3d0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:52:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.480+0000 7f3d0aff5640 1 -- 172.21.15.115:0/547053216 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c016070 con 0x5594e1a232f0 2024-05-09T05:52:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.481+0000 7f3d0aff5640 1 -- 172.21.15.115:0/547053216 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f3d1c0108a0 con 0x5594e1a232f0 2024-05-09T05:52:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.482+0000 7f3d3461d740 1 -- 172.21.15.115:0/547053216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d18018070 con 0x5594e1a232f0 2024-05-09T05:52:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.484+0000 7f3d0aff5640 1 -- 172.21.15.115:0/547053216 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f3d1c020050 con 0x5594e1a232f0 2024-05-09T05:52:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.484+0000 7f3d3461d740 10 client.27914 fetch_fsmap learned FSMap version 61 2024-05-09T05:52:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.484+0000 7f3d3461d740 1 -- 172.21.15.115:0/547053216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5594e1ab7790 con 0x5594e1a232f0 2024-05-09T05:52:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.486+0000 7f3d0aff5640 1 -- 172.21.15.115:0/547053216 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d18018070 con 0x5594e1a232f0 2024-05-09T05:52:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.486+0000 7f3d0aff5640 1 -- 172.21.15.115:0/547053216 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 61) ==== 8378+0+0 (secure 0 0 0) 0x7f3d1c00f040 con 0x5594e1a232f0 2024-05-09T05:52:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.487+0000 7f3d3461d740 10 client.27914 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T05:52:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.487+0000 7f3d3461d740 10 client.27914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:52:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.487+0000 7f3d3461d740 20 client.27914 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.487+0000 7f3d3461d740 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594e1a0ad80 0x5594e1e0c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.487+0000 7f3d3461d740 4 client.27914 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:52:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.487+0000 7f3d3461d740 1 -- 172.21.15.115:0/547053216 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594e1d623d0 con 0x5594e1a0ad80 2024-05-09T05:52:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.489+0000 7f3d267fc640 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594e1a0ad80 0x5594e1e0c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.494+0000 7f3d267fc640 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594e1a0ad80 0x5594e1e0c2f0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e288cfe8fdf40edb server_cookie=e4e2fec952023e04 in_seq=0 out_seq=0 2024-05-09T05:52:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d0aff5640 10 client.27914 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d0aff5640 1 -- 172.21.15.115:0/547053216 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594e1d623d0 con 0x5594e1a0ad80 2024-05-09T05:52:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d0aff5640 10 client.27914 handle_command_reply: tid=0 2024-05-09T05:52:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d3461d740 1 client.27914 shutdown 2024-05-09T05:52:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d3461d740 1 -- 172.21.15.115:0/547053216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 msgr2=0x5594e1dfb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d3461d740 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfb880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00e320 tx=0x7f3d1c00e7e0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d3461d740 1 -- 172.21.15.115:0/547053216 shutdown_connections 2024-05-09T05:52:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d3461d740 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a4c1b0 0x5594e1df9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d3461d740 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594e1a0ad80 0x5594e1e0c2f0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d3461d740 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a2b4d0 0x5594e1dfbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d3461d740 1 --2- 172.21.15.115:0/547053216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfb880 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.497+0000 7f3d3461d740 1 -- 172.21.15.115:0/547053216 >> 172.21.15.115:0/547053216 conn(0x5594e1a40300 msgr2=0x5594e1dea860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.498+0000 7f3d0aff5640 0 client.27914 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.498+0000 7f3d3461d740 1 -- 172.21.15.115:0/547053216 shutdown_connections 2024-05-09T05:52:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.498+0000 7f3d3461d740 1 -- 172.21.15.115:0/547053216 wait complete. 2024-05-09T05:52:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.498+0000 7f3d3461d740 20 client.27914 trim_cache size 0 max 16384 2024-05-09T05:52:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.530+0000 7f3d3461d740 1 Processor -- start 2024-05-09T05:52:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.531+0000 7f3d3461d740 1 -- start start 2024-05-09T05:52:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.531+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.531+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a2b4d0 0x5594e1e11b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.532+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 0x5594e1e12080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.532+0000 7f3d3461d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1400ffb0 con 0x5594e1a2b4d0 2024-05-09T05:52:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.532+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.532+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dec0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:57746/0 (socket says 172.21.15.115:57746) 2024-05-09T05:52:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.532+0000 7f3d26ffd640 1 -- 172.21.15.115:0/3818019305 learned_addr learned my addr 172.21.15.115:0/3818019305 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.532+0000 7f3d277fe640 1 --2- 172.21.15.115:0/3818019305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 0x5594e1e12080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.532+0000 7f3d3461d740 1 -- 172.21.15.115:0/3818019305 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14011af0 con 0x5594e1a232f0 2024-05-09T05:52:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.532+0000 7f3d3461d740 1 -- 172.21.15.115:0/3818019305 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14010680 con 0x5594e1a4c1b0 2024-05-09T05:52:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.532+0000 7f3d267fc640 1 --2- 172.21.15.115:0/3818019305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a2b4d0 0x5594e1e11b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.536+0000 7f3d26ffd640 1 -- 172.21.15.115:0/3818019305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 msgr2=0x5594e1e12080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.536+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/3818019305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 0x5594e1e12080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.536+0000 7f3d26ffd640 1 -- 172.21.15.115:0/3818019305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a2b4d0 msgr2=0x5594e1e11b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.536+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/3818019305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a2b4d0 0x5594e1e11b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.536+0000 7f3d26ffd640 1 -- 172.21.15.115:0/3818019305 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594e1b2e5f0 con 0x5594e1a232f0 2024-05-09T05:52:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.536+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/3818019305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dec0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00ece0 tx=0x7f3d1c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51935e6afdeb6634 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.537+0000 7f3d0cff9640 1 -- 172.21.15.115:0/3818019305 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c029030 con 0x5594e1a232f0 2024-05-09T05:52:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.538+0000 7f3d0cff9640 1 -- 172.21.15.115:0/3818019305 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d1c005040 con 0x5594e1a232f0 2024-05-09T05:52:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.538+0000 7f3d0cff9640 1 -- 172.21.15.115:0/3818019305 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c016070 con 0x5594e1a232f0 2024-05-09T05:52:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.539+0000 7f3d3461d740 1 -- 172.21.15.115:0/3818019305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 msgr2=0x5594e1dec0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.539+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3818019305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dec0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00ece0 tx=0x7f3d1c009c20 comp rx=0 tx=0).stop 2024-05-09T05:52:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.545+0000 7f3d3461d740 1 -- 172.21.15.115:0/3818019305 shutdown_connections 2024-05-09T05:52:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.545+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3818019305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 0x5594e1e12080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.545+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3818019305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a2b4d0 0x5594e1e11b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.545+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3818019305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dec0a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.545+0000 7f3d3461d740 1 -- 172.21.15.115:0/3818019305 >> 172.21.15.115:0/3818019305 conn(0x5594e1a40300 msgr2=0x5594e1deb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.547+0000 7f3d3461d740 1 -- 172.21.15.115:0/3818019305 shutdown_connections 2024-05-09T05:52:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.551+0000 7f3d3461d740 1 -- 172.21.15.115:0/3818019305 wait complete. 2024-05-09T05:52:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.552+0000 7f3d3461d740 1 -- start start 2024-05-09T05:52:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.552+0000 7f3d3461d740 1 Processor -- start 2024-05-09T05:52:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.554+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.554+0000 7f3d3461d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a2b4d0 0x5594e1dfefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.555+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.555+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:57750/0 (socket says 172.21.15.115:57750) 2024-05-09T05:52:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.555+0000 7f3d26ffd640 1 -- 172.21.15.115:0/3939700216 learned_addr learned my addr 172.21.15.115:0/3939700216 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.555+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 0x5594e1dff520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.555+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14011420 con 0x5594e1a2b4d0 2024-05-09T05:52:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.555+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14010d50 con 0x5594e1a232f0 2024-05-09T05:52:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.555+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1400dfb0 con 0x5594e1a4c1b0 2024-05-09T05:52:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.556+0000 7f3d3461d740 10 client.0 _ll_register_callbacks cb 0x7f3d1c0092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.556+0000 7f3d277fe640 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 0x5594e1dff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.558+0000 7f3d277fe640 1 -- 172.21.15.115:0/3939700216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 msgr2=0x5594e1dfeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.558+0000 7f3d277fe640 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.558+0000 7f3d277fe640 1 -- 172.21.15.115:0/3939700216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a2b4d0 msgr2=0x5594e1dfefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.558+0000 7f3d277fe640 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a2b4d0 0x5594e1dfefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.558+0000 7f3d277fe640 1 -- 172.21.15.115:0/3939700216 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594e1d18ed0 con 0x5594e1a4c1b0 2024-05-09T05:52:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.559+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:52:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.559+0000 7f3d277fe640 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 0x5594e1dff520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d201002b0 tx=0x7f3d20007520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de96a81f6fbacb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.559+0000 7f3d0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:52:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.560+0000 7f3d0aff5640 1 -- 172.21.15.115:0/3939700216 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200cc070 con 0x5594e1a4c1b0 2024-05-09T05:52:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.560+0000 7f3d0aff5640 1 -- 172.21.15.115:0/3939700216 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f3d200ca2b0 con 0x5594e1a4c1b0 2024-05-09T05:52:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.560+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d1c005040 con 0x5594e1a4c1b0 2024-05-09T05:52:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.561+0000 7f3d0aff5640 1 -- 172.21.15.115:0/3939700216 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f3d200ca540 con 0x5594e1a4c1b0 2024-05-09T05:52:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.564+0000 7f3d3461d740 10 client.28036 fetch_fsmap learned FSMap version 61 2024-05-09T05:52:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.564+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5594e1e73820 con 0x5594e1a4c1b0 2024-05-09T05:52:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.567+0000 7f3d0aff5640 1 -- 172.21.15.115:0/3939700216 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d200c9e40 con 0x5594e1a4c1b0 2024-05-09T05:52:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.569+0000 7f3d0aff5640 1 -- 172.21.15.115:0/3939700216 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 61) ==== 8378+0+0 (secure 0 0 0) 0x7f3d20004b40 con 0x5594e1a4c1b0 2024-05-09T05:52:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.569+0000 7f3d3461d740 10 client.28036 fetch_fsmap finished waiting for FSMap version 61 2024-05-09T05:52:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.569+0000 7f3d3461d740 10 client.28036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:52:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.569+0000 7f3d3461d740 20 client.28036 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.569+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594e19e01c0 0x5594e1e75ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.569+0000 7f3d3461d740 4 client.28036 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:52:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.569+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3d1c005040 con 0x5594e19e01c0 2024-05-09T05:52:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.570+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594e19e01c0 0x5594e1e75ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.575+0000 7f3d26ffd640 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594e19e01c0 0x5594e1e75ff0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a8d7dc7d4704d1 server_cookie=ca2cebbd6da17d26 in_seq=0 out_seq=0 2024-05-09T05:52:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.575+0000 7f3d0aff5640 10 client.28036 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d0aff5640 1 -- 172.21.15.115:0/3939700216 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f3d1c005040 con 0x5594e19e01c0 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d0aff5640 10 client.28036 handle_command_reply: tid=0 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d3461d740 1 client.28036 shutdown 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 msgr2=0x5594e1dff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 0x5594e1dff520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d201002b0 tx=0x7f3d20007520 comp rx=0 tx=0).stop 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 shutdown_connections 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5594e19e01c0 0x5594e1e75ff0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5594e1a4c1b0 0x5594e1dff520 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5594e1a2b4d0 0x5594e1dfefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d3461d740 1 --2- 172.21.15.115:0/3939700216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594e1a232f0 0x5594e1dfeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.597+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 >> 172.21.15.115:0/3939700216 conn(0x5594e1a40300 msgr2=0x5594e1df3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.599+0000 7f3d0aff5640 0 client.28036 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.599+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 shutdown_connections 2024-05-09T05:52:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.599+0000 7f3d3461d740 1 -- 172.21.15.115:0/3939700216 wait complete. 2024-05-09T05:52:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.600+0000 7f3d3461d740 20 client.28036 trim_cache size 0 max 16384 2024-05-09T05:52:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:52:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:52:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:52:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 -- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3d100b2b60 msgr2=0x7f3d100b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3d100b2b60 0x7f3d100b5070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d18013a60 tx=0x7f3d18006fd0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 -- 172.21.15.115:0/3366813812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d2810f7b0 msgr2=0x7f3d281ab690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d2810f7b0 0x7f3d281ab690 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000d090 tx=0x7f3d2000a440 comp rx=0 tx=0).stop 2024-05-09T05:52:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 -- 172.21.15.115:0/3366813812 shutdown_connections 2024-05-09T05:52:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3d100b2b60 0x7f3d100b5070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d2810f7b0 0x7f3d281ab690 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d2810ecf0 0x7f3d281a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 --2- 172.21.15.115:0/3366813812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d2810a7e0 0x7f3d281a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.602+0000 7f3d2d76b640 1 -- 172.21.15.115:0/3366813812 >> 172.21.15.115:0/3366813812 conn(0x7f3d2806f780 msgr2=0x7f3d2810c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.603+0000 7f3d2d76b640 1 -- 172.21.15.115:0/3366813812 shutdown_connections 2024-05-09T05:52:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:36.603+0000 7f3d2d76b640 1 -- 172.21.15.115:0/3366813812 wait complete. 2024-05-09T05:52:37.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:37 smithi195 ceph-mon[33844]: pgmap v454: 129 pgs: 129 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 3.64k op/s 2024-05-09T05:52:37.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:37 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:37.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:37 smithi115 ceph-mon[30349]: pgmap v454: 129 pgs: 129 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 3.64k op/s 2024-05-09T05:52:37.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:37 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:37.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:37 smithi161 ceph-mon[33805]: pgmap v454: 129 pgs: 129 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 3.64k op/s 2024-05-09T05:52:37.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:37 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:39.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:39 smithi195 ceph-mon[33844]: pgmap v455: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 22 MiB/s wr, 5.56k op/s 2024-05-09T05:52:39.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:39 smithi115 ceph-mon[30349]: pgmap v455: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 22 MiB/s wr, 5.56k op/s 2024-05-09T05:52:39.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:39 smithi161 ceph-mon[33805]: pgmap v455: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 22 MiB/s wr, 5.56k op/s 2024-05-09T05:52:41.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:41 smithi195 ceph-mon[33844]: pgmap v456: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-05-09T05:52:41.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:41 smithi115 ceph-mon[30349]: pgmap v456: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-05-09T05:52:41.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:41 smithi161 ceph-mon[33805]: pgmap v456: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-05-09T05:52:43.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:43 smithi195 ceph-mon[33844]: pgmap v457: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-05-09T05:52:43.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:43 smithi115 ceph-mon[30349]: pgmap v457: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-05-09T05:52:43.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:43 smithi161 ceph-mon[33805]: pgmap v457: 129 pgs: 129 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-05-09T05:52:44.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:44 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:52:44.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:44 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:52:44.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:44 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:52:45.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:45 smithi195 ceph-mon[33844]: pgmap v458: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 23 MiB/s wr, 5.82k op/s 2024-05-09T05:52:45.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:45 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:52:45.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:45 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:52:45.612 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:45 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:52:45.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:45 smithi115 ceph-mon[30349]: pgmap v458: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 23 MiB/s wr, 5.82k op/s 2024-05-09T05:52:45.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:45 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:52:45.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:45 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:52:45.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:45 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:52:45.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:45 smithi161 ceph-mon[33805]: pgmap v458: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 23 MiB/s wr, 5.82k op/s 2024-05-09T05:52:45.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:45 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:52:45.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:45 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:52:45.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:45 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:52:47.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:47 smithi195 ceph-mon[33844]: pgmap v459: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-05-09T05:52:47.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:47 smithi115 ceph-mon[30349]: pgmap v459: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-05-09T05:52:47.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:47 smithi161 ceph-mon[33805]: pgmap v459: 129 pgs: 129 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-05-09T05:52:49.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:49 smithi195 ceph-mon[33844]: pgmap v460: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 22 MiB/s wr, 5.59k op/s 2024-05-09T05:52:49.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:49 smithi115 ceph-mon[30349]: pgmap v460: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 22 MiB/s wr, 5.59k op/s 2024-05-09T05:52:49.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:49 smithi161 ceph-mon[33805]: pgmap v460: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 22 MiB/s wr, 5.59k op/s 2024-05-09T05:52:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.812+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3756575794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a5810a210 msgr2=0x7f8a5810a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.812+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3756575794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a5810a210 0x7f8a5810a6a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009ca0 tx=0x7f8a5002a8f0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.813+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3756575794 shutdown_connections 2024-05-09T05:52:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.813+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3756575794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a5810a210 0x7f8a5810a6a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.813+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3756575794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a581103f0 0x7f8a58109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.813+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3756575794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a5810fa10 0x7f8a5810fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.813+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3756575794 >> 172.21.15.115:0/3756575794 conn(0x7f8a5806f810 msgr2=0x7f8a5806fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.820+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3756575794 shutdown_connections 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.820+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3756575794 wait complete. 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.821+0000 7f8a5d76c640 1 Processor -- start 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.821+0000 7f8a5d76c640 1 -- start start 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.821+0000 7f8a5d76c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a5810a210 0x7f8a58120390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.821+0000 7f8a5d76c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a5810fa10 0x7f8a581194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.821+0000 7f8a5d76c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581103f0 0x7f8a58119b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.821+0000 7f8a5d76c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58122960 con 0x7f8a581103f0 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.821+0000 7f8a5d76c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581227e0 con 0x7f8a5810a210 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.821+0000 7f8a5d76c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58122ae0 con 0x7f8a5810fa10 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.822+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a5810fa10 0x7f8a581194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.822+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a5810fa10 0x7f8a581194b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48692/0 (socket says 172.21.15.115:48692) 2024-05-09T05:52:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.822+0000 7f8a577fe640 1 -- 172.21.15.115:0/3796186618 learned_addr learned my addr 172.21.15.115:0/3796186618 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.822+0000 7f8a5cf6b640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581103f0 0x7f8a58119b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.823+0000 7f8a577fe640 1 -- 172.21.15.115:0/3796186618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a5810a210 msgr2=0x7f8a58120390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:52:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.823+0000 7f8a577fe640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a5810a210 0x7f8a58120390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.823+0000 7f8a577fe640 1 -- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581103f0 msgr2=0x7f8a58119b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.823+0000 7f8a577fe640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581103f0 0x7f8a58119b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.823+0000 7f8a577fe640 1 -- 172.21.15.115:0/3796186618 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5811deb0 con 0x7f8a5810fa10 2024-05-09T05:52:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.823+0000 7f8a577fe640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a5810fa10 0x7f8a581194b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800b470 tx=0x7f8a4800b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.824+0000 7f8a557fa640 1 -- 172.21.15.115:0/3796186618 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48015020 con 0x7f8a5810fa10 2024-05-09T05:52:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.825+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3796186618 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5811e1a0 con 0x7f8a5810fa10 2024-05-09T05:52:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.825+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3796186618 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5811e6f0 con 0x7f8a5810fa10 2024-05-09T05:52:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.825+0000 7f8a557fa640 1 -- 172.21.15.115:0/3796186618 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a48011d20 con 0x7f8a5810fa10 2024-05-09T05:52:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.825+0000 7f8a557fa640 1 -- 172.21.15.115:0/3796186618 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48010e30 con 0x7f8a5810fa10 2024-05-09T05:52:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.832+0000 7f8a65882740 1 Processor -- start 2024-05-09T05:52:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.832+0000 7f8a65882740 1 -- start start 2024-05-09T05:52:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.836+0000 7f8a65882740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311baf1b0 0x55e311f52840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.836+0000 7f8a65882740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 0x55e311f52d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.836+0000 7f8a65882740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311b8e4d0 0x55e311f532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.836+0000 7f8a65882740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e311eb77c0 con 0x55e311b8e4d0 2024-05-09T05:52:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.836+0000 7f8a65882740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e311e530e0 con 0x55e311baf1b0 2024-05-09T05:52:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.836+0000 7f8a65882740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e311eb3780 con 0x55e311b862f0 2024-05-09T05:52:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.837+0000 7f8a5cf6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311baf1b0 0x55e311f52840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.837+0000 7f8a5cf6b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311baf1b0 0x55e311f52840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:46902/0 (socket says 172.21.15.115:46902) 2024-05-09T05:52:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.837+0000 7f8a5cf6b640 1 -- 172.21.15.115:0/275266157 learned_addr learned my addr 172.21.15.115:0/275266157 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.837+0000 7f8a5cf6b640 1 -- 172.21.15.115:0/275266157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 msgr2=0x55e311f52d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:52:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.837+0000 7f8a5cf6b640 1 --2- 172.21.15.115:0/275266157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 0x55e311f52d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.837+0000 7f8a5cf6b640 1 -- 172.21.15.115:0/275266157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311b8e4d0 msgr2=0x55e311f532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.837+0000 7f8a5cf6b640 1 --2- 172.21.15.115:0/275266157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311b8e4d0 0x55e311f532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.837+0000 7f8a5cf6b640 1 -- 172.21.15.115:0/275266157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e311e7c7c0 con 0x55e311baf1b0 2024-05-09T05:52:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.838+0000 7f8a557fa640 1 -- 172.21.15.115:0/3796186618 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f8a48018020 con 0x7f8a5810fa10 2024-05-09T05:52:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.838+0000 7f8a5cf6b640 1 --2- 172.21.15.115:0/275266157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311baf1b0 0x55e311f52840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a50031040 tx=0x7f8a50009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e01c00189c284899 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.840+0000 7f8a3e7fc640 1 -- 172.21.15.115:0/275266157 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50003bf0 con 0x55e311baf1b0 2024-05-09T05:52:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.842+0000 7f8a3e7fc640 1 -- 172.21.15.115:0/275266157 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a50036070 con 0x55e311baf1b0 2024-05-09T05:52:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.842+0000 7f8a3e7fc640 1 -- 172.21.15.115:0/275266157 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a500308e0 con 0x55e311baf1b0 2024-05-09T05:52:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.843+0000 7f8a557fa640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8a400b2b60 0x7f8a400b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.843+0000 7f8a57fff640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8a400b2b60 0x7f8a400b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.843+0000 7f8a65882740 1 -- 172.21.15.115:0/275266157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311baf1b0 msgr2=0x55e311f52840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.843+0000 7f8a557fa640 1 -- 172.21.15.115:0/3796186618 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f8a480e2380 con 0x7f8a5810fa10 2024-05-09T05:52:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.843+0000 7f8a65882740 1 --2- 172.21.15.115:0/275266157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311baf1b0 0x55e311f52840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a50031040 tx=0x7f8a50009a10 comp rx=0 tx=0).stop 2024-05-09T05:52:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.845+0000 7f8a65882740 1 -- 172.21.15.115:0/275266157 shutdown_connections 2024-05-09T05:52:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.845+0000 7f8a65882740 1 --2- 172.21.15.115:0/275266157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311b8e4d0 0x55e311f532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.845+0000 7f8a65882740 1 --2- 172.21.15.115:0/275266157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 0x55e311f52d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.845+0000 7f8a65882740 1 --2- 172.21.15.115:0/275266157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311baf1b0 0x55e311f52840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.845+0000 7f8a65882740 1 -- 172.21.15.115:0/275266157 >> 172.21.15.115:0/275266157 conn(0x55e311ba3300 msgr2=0x55e311f4eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.849+0000 7f8a65882740 1 -- 172.21.15.115:0/275266157 shutdown_connections 2024-05-09T05:52:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.849+0000 7f8a57fff640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8a400b2b60 0x7f8a400b5070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55e311f53bf0 tx=0x7f8a4c00b370 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.849+0000 7f8a65882740 1 -- 172.21.15.115:0/275266157 wait complete. 2024-05-09T05:52:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.850+0000 7f8a65882740 1 -- start start 2024-05-09T05:52:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.850+0000 7f8a65882740 1 Processor -- start 2024-05-09T05:52:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a65882740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b862f0 0x55e311f5ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a65882740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b8e4d0 0x55e311f5ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a65882740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311baf1b0 0x55e311f5ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a65882740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e311f5f910 con 0x55e311baf1b0 2024-05-09T05:52:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a65882740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e311e3e9c0 con 0x55e311b862f0 2024-05-09T05:52:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a65882740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e311f5fa90 con 0x55e311b8e4d0 2024-05-09T05:52:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a65882740 10 client.0 _ll_register_callbacks cb 0x55e311e6b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b8e4d0 0x55e311f5ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b8e4d0 0x55e311f5ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48708/0 (socket says 172.21.15.115:48708) 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a577fe640 1 -- 172.21.15.115:0/4161211408 learned_addr learned my addr 172.21.15.115:0/4161211408 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.851+0000 7f8a57fff640 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b862f0 0x55e311f5ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.852+0000 7f8a577fe640 1 -- 172.21.15.115:0/4161211408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b862f0 msgr2=0x55e311f5ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.852+0000 7f8a577fe640 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b862f0 0x55e311f5ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.852+0000 7f8a577fe640 1 -- 172.21.15.115:0/4161211408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311baf1b0 msgr2=0x55e311f5ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.852+0000 7f8a577fe640 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311baf1b0 0x55e311f5ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.852+0000 7f8a577fe640 1 -- 172.21.15.115:0/4161211408 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e311e651f0 con 0x55e311b8e4d0 2024-05-09T05:52:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.852+0000 7f8a577fe640 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b8e4d0 0x55e311f5ef90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006e40 tx=0x7f8a480dfc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd3f8c7bd94572de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.852+0000 7f8a377fe640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:52:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.852+0000 7f8a377fe640 1 -- 172.21.15.115:0/4161211408 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480e2170 con 0x55e311b8e4d0 2024-05-09T05:52:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.853+0000 7f8a65882740 1 -- 172.21.15.115:0/4161211408 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a50036070 con 0x55e311b8e4d0 2024-05-09T05:52:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.853+0000 7f8a377fe640 1 -- 172.21.15.115:0/4161211408 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f8a48011aa0 con 0x55e311b8e4d0 2024-05-09T05:52:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.853+0000 7f8a377fe640 1 -- 172.21.15.115:0/4161211408 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f8a480a9380 con 0x55e311b8e4d0 2024-05-09T05:52:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.853+0000 7f8a65882740 10 client.28045 fetch_fsmap learned FSMap version 62 2024-05-09T05:52:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.853+0000 7f8a65882740 1 -- 172.21.15.115:0/4161211408 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55e311c1a790 con 0x55e311b8e4d0 2024-05-09T05:52:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.854+0000 7f8a377fe640 1 -- 172.21.15.115:0/4161211408 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a50036070 con 0x55e311b8e4d0 2024-05-09T05:52:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.854+0000 7f8a377fe640 1 -- 172.21.15.115:0/4161211408 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f8a480103e0 con 0x55e311b8e4d0 2024-05-09T05:52:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.854+0000 7f8a65882740 10 client.28045 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:52:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.854+0000 7f8a65882740 10 client.28045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:52:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.854+0000 7f8a65882740 20 client.28045 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.854+0000 7f8a65882740 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e311b6dd80 0x55e311f6f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.854+0000 7f8a65882740 4 client.28045 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:52:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.854+0000 7f8a65882740 1 -- 172.21.15.115:0/4161211408 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e311ec53d0 con 0x55e311b6dd80 2024-05-09T05:52:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.854+0000 7f8a57fff640 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e311b6dd80 0x55e311f6f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.855+0000 7f8a377fe640 10 client.28045 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.855+0000 7f8a57fff640 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e311b6dd80 0x55e311f6f230 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f244da39c3968d69 server_cookie=5f47d3eacc35eb43 in_seq=0 out_seq=0 2024-05-09T05:52:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.861+0000 7f8a377fe640 1 -- 172.21.15.115:0/4161211408 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e311ec53d0 con 0x55e311b6dd80 2024-05-09T05:52:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.861+0000 7f8a377fe640 10 client.28045 handle_command_reply: tid=0 2024-05-09T05:52:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.861+0000 7f8a65882740 1 client.28045 shutdown 2024-05-09T05:52:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.867+0000 7f8a65882740 1 -- 172.21.15.115:0/4161211408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b8e4d0 msgr2=0x55e311f5ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.867+0000 7f8a65882740 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b8e4d0 0x55e311f5ef90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006e40 tx=0x7f8a480dfc60 comp rx=0 tx=0).stop 2024-05-09T05:52:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.868+0000 7f8a65882740 1 -- 172.21.15.115:0/4161211408 shutdown_connections 2024-05-09T05:52:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.868+0000 7f8a65882740 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311baf1b0 0x55e311f5ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.868+0000 7f8a65882740 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e311b6dd80 0x55e311f6f230 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.868+0000 7f8a65882740 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b8e4d0 0x55e311f5ef90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.868+0000 7f8a65882740 1 --2- 172.21.15.115:0/4161211408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b862f0 0x55e311f5ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.868+0000 7f8a65882740 1 -- 172.21.15.115:0/4161211408 >> 172.21.15.115:0/4161211408 conn(0x55e311ba3300 msgr2=0x55e311f54560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.871+0000 7f8a377fe640 0 client.28045 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.872+0000 7f8a65882740 1 -- 172.21.15.115:0/4161211408 shutdown_connections 2024-05-09T05:52:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.872+0000 7f8a65882740 1 -- 172.21.15.115:0/4161211408 wait complete. 2024-05-09T05:52:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.873+0000 7f8a65882740 20 client.28045 trim_cache size 0 max 16384 2024-05-09T05:52:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.914+0000 7f8a65882740 1 Processor -- start 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.915+0000 7f8a65882740 1 -- start start 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.916+0000 7f8a65882740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311b862f0 0x55e311f54b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.916+0000 7f8a65882740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f74a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.916+0000 7f8a65882740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311baf1b0 0x55e311f74f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.916+0000 7f8a65882740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4400ff60 con 0x55e311b862f0 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.916+0000 7f8a65882740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44011ad0 con 0x55e311b8e4d0 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.916+0000 7f8a65882740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44010630 con 0x55e311baf1b0 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.917+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f74a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.917+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f74a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:46920/0 (socket says 172.21.15.115:46920) 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.917+0000 7f8a577fe640 1 -- 172.21.15.115:0/2903031377 learned_addr learned my addr 172.21.15.115:0/2903031377 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.918+0000 7f8a577fe640 1 -- 172.21.15.115:0/2903031377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311baf1b0 msgr2=0x55e311f74f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.918+0000 7f8a577fe640 1 --2- 172.21.15.115:0/2903031377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311baf1b0 0x55e311f74f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.918+0000 7f8a577fe640 1 -- 172.21.15.115:0/2903031377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311b862f0 msgr2=0x55e311f54b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:52:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.918+0000 7f8a577fe640 1 --2- 172.21.15.115:0/2903031377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311b862f0 0x55e311f54b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.918+0000 7f8a577fe640 1 -- 172.21.15.115:0/2903031377 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e311c915f0 con 0x55e311b8e4d0 2024-05-09T05:52:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.918+0000 7f8a577fe640 1 --2- 172.21.15.115:0/2903031377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f74a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a480a5920 tx=0x7f8a48010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3cb5bba88bbb906 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.919+0000 7f8a3dffb640 1 -- 172.21.15.115:0/2903031377 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480b21e0 con 0x55e311b8e4d0 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.919+0000 7f8a3dffb640 1 -- 172.21.15.115:0/2903031377 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a480ad730 con 0x55e311b8e4d0 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.920+0000 7f8a65882740 1 -- 172.21.15.115:0/2903031377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 msgr2=0x55e311f74a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.920+0000 7f8a65882740 1 --2- 172.21.15.115:0/2903031377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f74a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a480a5920 tx=0x7f8a48010fd0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.923+0000 7f8a65882740 1 -- 172.21.15.115:0/2903031377 shutdown_connections 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.923+0000 7f8a65882740 1 --2- 172.21.15.115:0/2903031377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311baf1b0 0x55e311f74f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.923+0000 7f8a65882740 1 --2- 172.21.15.115:0/2903031377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f74a10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.923+0000 7f8a65882740 1 --2- 172.21.15.115:0/2903031377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311b862f0 0x55e311f54b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.923+0000 7f8a65882740 1 -- 172.21.15.115:0/2903031377 >> 172.21.15.115:0/2903031377 conn(0x55e311ba3300 msgr2=0x55e311f54480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.924+0000 7f8a65882740 1 -- 172.21.15.115:0/2903031377 shutdown_connections 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.924+0000 7f8a65882740 1 -- 172.21.15.115:0/2903031377 wait complete. 2024-05-09T05:52:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.924+0000 7f8a65882740 1 -- start start 2024-05-09T05:52:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.924+0000 7f8a65882740 1 Processor -- start 2024-05-09T05:52:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a65882740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 0x55e311f64ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a65882740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f65440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a65882740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311baf1b0 0x55e311f60460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a65882740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44011400 con 0x55e311baf1b0 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a65882740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44010d30 con 0x55e311b8e4d0 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a65882740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4400df60 con 0x55e311b862f0 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a65882740 10 client.0 _ll_register_callbacks cb 0x7f8a48012ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a57fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 0x55e311f64ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a57fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 0x55e311f64ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:48714/0 (socket says 172.21.15.115:48714) 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.925+0000 7f8a57fff640 1 -- 172.21.15.115:0/3980633617 learned_addr learned my addr 172.21.15.115:0/3980633617 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.926+0000 7f8a577fe640 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f65440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.926+0000 7f8a57fff640 1 -- 172.21.15.115:0/3980633617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 msgr2=0x55e311f65440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.926+0000 7f8a57fff640 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f65440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.926+0000 7f8a57fff640 1 -- 172.21.15.115:0/3980633617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311baf1b0 msgr2=0x55e311f60460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:52:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.926+0000 7f8a57fff640 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311baf1b0 0x55e311f60460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.926+0000 7f8a57fff640 1 -- 172.21.15.115:0/3980633617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e311e7bed0 con 0x55e311b862f0 2024-05-09T05:52:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.928+0000 7f8a57fff640 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 0x55e311f64ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55e311f50330 tx=0x7f8a4c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0420b36ded776f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:52:49.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.929+0000 7f8a377fe640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:52:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.929+0000 7f8a377fe640 1 -- 172.21.15.115:0/3980633617 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c0252d0 con 0x55e311b862f0 2024-05-09T05:52:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.929+0000 7f8a65882740 1 -- 172.21.15.115:0/3980633617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e311ebf240 con 0x55e311b862f0 2024-05-09T05:52:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.929+0000 7f8a377fe640 1 -- 172.21.15.115:0/3980633617 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f8a4c025470 con 0x55e311b862f0 2024-05-09T05:52:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.930+0000 7f8a377fe640 1 -- 172.21.15.115:0/3980633617 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c033050 con 0x55e311b862f0 2024-05-09T05:52:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.930+0000 7f8a65882740 10 client.28048 fetch_fsmap learned FSMap version 62 2024-05-09T05:52:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.930+0000 7f8a65882740 1 -- 172.21.15.115:0/3980633617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55e311fd6f80 con 0x55e311b862f0 2024-05-09T05:52:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.930+0000 7f8a377fe640 1 -- 172.21.15.115:0/3980633617 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55e311ebf240 con 0x55e311b862f0 2024-05-09T05:52:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.931+0000 7f8a377fe640 1 -- 172.21.15.115:0/3980633617 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f8a4c025ce0 con 0x55e311b862f0 2024-05-09T05:52:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.932+0000 7f8a65882740 10 client.28048 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:52:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.932+0000 7f8a65882740 10 client.28048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:52:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.932+0000 7f8a65882740 20 client.28048 populate_metadata read hostname 'smithi115' 2024-05-09T05:52:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.932+0000 7f8a65882740 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e311b431c0 0x55e311fd9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:52:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.932+0000 7f8a65882740 4 client.28048 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:52:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.932+0000 7f8a65882740 1 -- 172.21.15.115:0/3980633617 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e311e3a350 con 0x55e311b431c0 2024-05-09T05:52:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.932+0000 7f8a577fe640 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e311b431c0 0x55e311fd9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:52:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.932+0000 7f8a577fe640 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e311b431c0 0x55e311fd9770 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1475930294371cdc server_cookie=7244c70639bb98d6 in_seq=0 out_seq=0 2024-05-09T05:52:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:49.933+0000 7f8a377fe640 10 client.28048 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.068+0000 7f8a377fe640 1 -- 172.21.15.115:0/3980633617 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e311e3a350 con 0x55e311b431c0 2024-05-09T05:52:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.068+0000 7f8a377fe640 10 client.28048 handle_command_reply: tid=0 2024-05-09T05:52:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.068+0000 7f8a65882740 1 client.28048 shutdown 2024-05-09T05:52:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.072+0000 7f8a65882740 1 -- 172.21.15.115:0/3980633617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 msgr2=0x55e311f64ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.072+0000 7f8a65882740 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 0x55e311f64ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55e311f50330 tx=0x7f8a4c004730 comp rx=0 tx=0).stop 2024-05-09T05:52:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.072+0000 7f8a65882740 1 -- 172.21.15.115:0/3980633617 shutdown_connections 2024-05-09T05:52:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.072+0000 7f8a65882740 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e311baf1b0 0x55e311f60460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.072+0000 7f8a65882740 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e311b8e4d0 0x55e311f65440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.072+0000 7f8a65882740 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e311b431c0 0x55e311fd9770 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.072+0000 7f8a65882740 1 --2- 172.21.15.115:0/3980633617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e311b862f0 0x55e311f64ee0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.072+0000 7f8a65882740 1 -- 172.21.15.115:0/3980633617 >> 172.21.15.115:0/3980633617 conn(0x55e311ba3300 msgr2=0x55e311f5c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.073+0000 7f8a377fe640 0 client.28048 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:52:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.073+0000 7f8a65882740 1 -- 172.21.15.115:0/3980633617 shutdown_connections 2024-05-09T05:52:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.073+0000 7f8a65882740 1 -- 172.21.15.115:0/3980633617 wait complete. 2024-05-09T05:52:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.073+0000 7f8a65882740 20 client.28048 trim_cache size 0 max 16384 2024-05-09T05:52:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:52:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:52:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:52:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.076+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8a400b2b60 msgr2=0x7f8a400b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.076+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8a400b2b60 0x7f8a400b5070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55e311f53bf0 tx=0x7f8a4c00b370 comp rx=0 tx=0).stop 2024-05-09T05:52:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.076+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3796186618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a5810fa10 msgr2=0x7f8a581194b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:52:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.076+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a5810fa10 0x7f8a581194b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800b470 tx=0x7f8a4800b930 comp rx=0 tx=0).stop 2024-05-09T05:52:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.077+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3796186618 shutdown_connections 2024-05-09T05:52:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.077+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8a400b2b60 0x7f8a400b5070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.077+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581103f0 0x7f8a58119b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.077+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a5810fa10 0x7f8a581194b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.077+0000 7f8a5d76c640 1 --2- 172.21.15.115:0/3796186618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a5810a210 0x7f8a58120390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:52:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.077+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3796186618 >> 172.21.15.115:0/3796186618 conn(0x7f8a5806f810 msgr2=0x7f8a5810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:52:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.077+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3796186618 shutdown_connections 2024-05-09T05:52:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:52:50.077+0000 7f8a5d76c640 1 -- 172.21.15.115:0/3796186618 wait complete. 2024-05-09T05:52:50.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:50 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:50.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:50 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:50.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:50 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:52:51.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:51 smithi195 ceph-mon[33844]: pgmap v461: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-05-09T05:52:51.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:51 smithi115 ceph-mon[30349]: pgmap v461: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-05-09T05:52:51.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:51 smithi161 ceph-mon[33805]: pgmap v461: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-05-09T05:52:53.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:53 smithi195 ceph-mon[33844]: pgmap v462: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-05-09T05:52:53.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:53 smithi115 ceph-mon[30349]: pgmap v462: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-05-09T05:52:53.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:53 smithi161 ceph-mon[33805]: pgmap v462: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-05-09T05:52:55.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:55 smithi195 ceph-mon[33844]: pgmap v463: 129 pgs: 129 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 22 MiB/s wr, 5.55k op/s 2024-05-09T05:52:55.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:55 smithi115 ceph-mon[30349]: pgmap v463: 129 pgs: 129 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 22 MiB/s wr, 5.55k op/s 2024-05-09T05:52:55.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:55 smithi161 ceph-mon[33805]: pgmap v463: 129 pgs: 129 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 22 MiB/s wr, 5.55k op/s 2024-05-09T05:52:57.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:57 smithi115 ceph-mon[30349]: pgmap v464: 129 pgs: 129 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-05-09T05:52:57.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:57 smithi161 ceph-mon[33805]: pgmap v464: 129 pgs: 129 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-05-09T05:52:57.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:57 smithi195 ceph-mon[33844]: pgmap v464: 129 pgs: 129 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-05-09T05:52:59.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:52:59 smithi115 ceph-mon[30349]: pgmap v465: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 21 MiB/s wr, 5.29k op/s 2024-05-09T05:52:59.656 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:52:59 smithi161 ceph-mon[33805]: pgmap v465: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 21 MiB/s wr, 5.29k op/s 2024-05-09T05:52:59.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:52:59 smithi195 ceph-mon[33844]: pgmap v465: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 21 MiB/s wr, 5.29k op/s 2024-05-09T05:53:01.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:01 smithi115 ceph-mon[30349]: pgmap v466: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-05-09T05:53:01.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:01 smithi161 ceph-mon[33805]: pgmap v466: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-05-09T05:53:01.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:01 smithi195 ceph-mon[33844]: pgmap v466: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-05-09T05:53:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.331+0000 7f986d834640 1 -- 172.21.15.115:0/3489079213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868110ba0 msgr2=0x7f9868110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.331+0000 7f986d834640 1 --2- 172.21.15.115:0/3489079213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868110ba0 0x7f9868110fd0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f985c009900 tx=0x7f985c02a440 comp rx=0 tx=0).stop 2024-05-09T05:53:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.335+0000 7f986d834640 1 -- 172.21.15.115:0/3489079213 shutdown_connections 2024-05-09T05:53:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.335+0000 7f986d834640 1 --2- 172.21.15.115:0/3489079213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9868111660 0x7f9868117bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.335+0000 7f986d834640 1 --2- 172.21.15.115:0/3489079213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9868110ba0 0x7f9868110fd0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.335+0000 7f986d834640 1 --2- 172.21.15.115:0/3489079213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f986810c180 0x7f986810c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.335+0000 7f986d834640 1 -- 172.21.15.115:0/3489079213 >> 172.21.15.115:0/3489079213 conn(0x7f986806f810 msgr2=0x7f986806fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.339+0000 7f986d834640 1 -- 172.21.15.115:0/3489079213 shutdown_connections 2024-05-09T05:53:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.339+0000 7f986d834640 1 -- 172.21.15.115:0/3489079213 wait complete. 2024-05-09T05:53:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.340+0000 7f986d834640 1 Processor -- start 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.340+0000 7f986d834640 1 -- start start 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f986d834640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f986810c180 0x7f9868120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f986d834640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868110ba0 0x7f98681210e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f986d834640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9868111660 0x7f9868126890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f986d834640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f986811a190 con 0x7f986810c180 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f986d834640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f986811a010 con 0x7f9868111660 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f986d834640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f986811a310 con 0x7f9868110ba0 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f9867fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868110ba0 0x7f98681210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f9867fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868110ba0 0x7f98681210e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:44230/0 (socket says 172.21.15.115:44230) 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f9867fff640 1 -- 172.21.15.115:0/3417074742 learned_addr learned my addr 172.21.15.115:0/3417074742 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.341+0000 7f986d033640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9868111660 0x7f9868126890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.342+0000 7f9867fff640 1 -- 172.21.15.115:0/3417074742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9868111660 msgr2=0x7f9868126890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.342+0000 7f9867fff640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9868111660 0x7f9868126890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.342+0000 7f9867fff640 1 -- 172.21.15.115:0/3417074742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f986810c180 msgr2=0x7f9868120b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.342+0000 7f9867fff640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f986810c180 0x7f9868120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.342+0000 7f9867fff640 1 -- 172.21.15.115:0/3417074742 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986811bd80 con 0x7f9868110ba0 2024-05-09T05:53:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.343+0000 7f9867fff640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868110ba0 0x7f98681210e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f985c0058c0 tx=0x7f985c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.343+0000 7f9865ffb640 1 -- 172.21.15.115:0/3417074742 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c03f070 con 0x7f9868110ba0 2024-05-09T05:53:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.344+0000 7f986d834640 1 -- 172.21.15.115:0/3417074742 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986811bfb0 con 0x7f9868110ba0 2024-05-09T05:53:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.344+0000 7f986d834640 1 -- 172.21.15.115:0/3417074742 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986811c4f0 con 0x7f9868110ba0 2024-05-09T05:53:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.346+0000 7f9865ffb640 1 -- 172.21.15.115:0/3417074742 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f985c03baf0 con 0x7f9868110ba0 2024-05-09T05:53:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.346+0000 7f9865ffb640 1 -- 172.21.15.115:0/3417074742 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c035030 con 0x7f9868110ba0 2024-05-09T05:53:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.349+0000 7f9875a28740 1 Processor -- start 2024-05-09T05:53:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.349+0000 7f9875a28740 1 -- start start 2024-05-09T05:53:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.349+0000 7f9875a28740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f18840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.349+0000 7f9875a28740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b4c2f0 0x562a37f18d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.349+0000 7f9875a28740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b544d0 0x562a37f192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.349+0000 7f9875a28740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562a37e7d7c0 con 0x562a37b751b0 2024-05-09T05:53:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.349+0000 7f9875a28740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562a37e190e0 con 0x562a37b4c2f0 2024-05-09T05:53:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.349+0000 7f9875a28740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a37e79780 con 0x562a37b544d0 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986d033640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f18840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986d033640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f18840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46282/0 (socket says 172.21.15.115:46282) 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986d033640 1 -- 172.21.15.115:0/763999 learned_addr learned my addr 172.21.15.115:0/763999 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986c832640 1 --2- 172.21.15.115:0/763999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b4c2f0 0x562a37f18d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986c832640 1 -- 172.21.15.115:0/763999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b544d0 msgr2=0x562a37f192c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986c832640 1 --2- 172.21.15.115:0/763999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b544d0 0x562a37f192c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986c832640 1 -- 172.21.15.115:0/763999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 msgr2=0x562a37f18840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986c832640 1 --2- 172.21.15.115:0/763999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f18840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986c832640 1 -- 172.21.15.115:0/763999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a37e427c0 con 0x562a37b4c2f0 2024-05-09T05:53:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.350+0000 7f986c832640 1 --2- 172.21.15.115:0/763999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b4c2f0 0x562a37f18d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9860009d30 tx=0x7f986000b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00fef0d8bf55b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.353+0000 7f984effd640 1 -- 172.21.15.115:0/763999 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860018070 con 0x562a37b4c2f0 2024-05-09T05:53:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.353+0000 7f984effd640 1 -- 172.21.15.115:0/763999 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98600119b0 con 0x562a37b4c2f0 2024-05-09T05:53:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.353+0000 7f984effd640 1 -- 172.21.15.115:0/763999 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860013930 con 0x562a37b4c2f0 2024-05-09T05:53:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.353+0000 7f9865ffb640 1 -- 172.21.15.115:0/3417074742 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f985c042070 con 0x7f9868110ba0 2024-05-09T05:53:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.355+0000 7f9875a28740 1 -- 172.21.15.115:0/763999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b4c2f0 msgr2=0x562a37f18d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.355+0000 7f9875a28740 1 --2- 172.21.15.115:0/763999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b4c2f0 0x562a37f18d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9860009d30 tx=0x7f986000b7e0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.355+0000 7f9865ffb640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98500b2aa0 0x7f98500b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.356+0000 7f9865ffb640 1 -- 172.21.15.115:0/3417074742 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f985c1030b0 con 0x7f9868110ba0 2024-05-09T05:53:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.357+0000 7f986d033640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98500b2aa0 0x7f98500b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.360+0000 7f986d033640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98500b2aa0 0x7f98500b4fb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f985800b3a0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.361+0000 7f9875a28740 1 -- 172.21.15.115:0/763999 shutdown_connections 2024-05-09T05:53:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.361+0000 7f9875a28740 1 --2- 172.21.15.115:0/763999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b544d0 0x562a37f192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.361+0000 7f9875a28740 1 --2- 172.21.15.115:0/763999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b4c2f0 0x562a37f18d80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.361+0000 7f9875a28740 1 --2- 172.21.15.115:0/763999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f18840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.361+0000 7f9875a28740 1 -- 172.21.15.115:0/763999 >> 172.21.15.115:0/763999 conn(0x562a37b69300 msgr2=0x562a37f14ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.370+0000 7f9875a28740 1 -- 172.21.15.115:0/763999 shutdown_connections 2024-05-09T05:53:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.370+0000 7f9875a28740 1 -- 172.21.15.115:0/763999 wait complete. 2024-05-09T05:53:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.370+0000 7f9875a28740 1 -- start start 2024-05-09T05:53:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.370+0000 7f9875a28740 1 Processor -- start 2024-05-09T05:53:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.371+0000 7f9875a28740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f24930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.371+0000 7f9875a28740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 0x562a37f24e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.371+0000 7f9875a28740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f22dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.371+0000 7f9875a28740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562a37f25970 con 0x562a37b751b0 2024-05-09T05:53:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.371+0000 7f9875a28740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562a37e049c0 con 0x562a37b544d0 2024-05-09T05:53:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.371+0000 7f9875a28740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a37f25af0 con 0x562a37b4c2f0 2024-05-09T05:53:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.371+0000 7f9875a28740 10 client.0 _ll_register_callbacks cb 0x562a37e31020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f9867fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 0x562a37f24e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f986c832640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f24930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f986c832640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f24930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:44246/0 (socket says 172.21.15.115:44246) 2024-05-09T05:53:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f986c832640 1 -- 172.21.15.115:0/2050731705 learned_addr learned my addr 172.21.15.115:0/2050731705 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f986d033640 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f22dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f986c832640 1 -- 172.21.15.115:0/2050731705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 msgr2=0x562a37f24e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f986c832640 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 0x562a37f24e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f986c832640 1 -- 172.21.15.115:0/2050731705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 msgr2=0x562a37f22dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f986c832640 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f22dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.372+0000 7f986c832640 1 -- 172.21.15.115:0/2050731705 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a37e2b1f0 con 0x562a37b4c2f0 2024-05-09T05:53:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.373+0000 7f986c832640 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f24930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9860013590 tx=0x7f9860009aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78d2d91e520f227f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:03.363 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.373+0000 7f9847fff640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:53:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.375+0000 7f9847fff640 1 -- 172.21.15.115:0/2050731705 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860018040 con 0x562a37b4c2f0 2024-05-09T05:53:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.375+0000 7f9875a28740 1 -- 172.21.15.115:0/2050731705 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98600119b0 con 0x562a37b4c2f0 2024-05-09T05:53:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.375+0000 7f9847fff640 1 -- 172.21.15.115:0/2050731705 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f9860011b60 con 0x562a37b4c2f0 2024-05-09T05:53:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.375+0000 7f9847fff640 1 -- 172.21.15.115:0/2050731705 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f986002b050 con 0x562a37b4c2f0 2024-05-09T05:53:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.376+0000 7f9875a28740 10 client.28054 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.376+0000 7f9875a28740 1 -- 172.21.15.115:0/2050731705 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x562a37be0790 con 0x562a37b4c2f0 2024-05-09T05:53:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.378+0000 7f9847fff640 1 -- 172.21.15.115:0/2050731705 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98600119b0 con 0x562a37b4c2f0 2024-05-09T05:53:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.378+0000 7f9847fff640 1 -- 172.21.15.115:0/2050731705 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f986001b050 con 0x562a37b4c2f0 2024-05-09T05:53:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.378+0000 7f9875a28740 10 client.28054 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:03.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.378+0000 7f9875a28740 10 client.28054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.378+0000 7f9875a28740 20 client.28054 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.378+0000 7f9875a28740 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a37b33d80 0x562a37f355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.378+0000 7f9875a28740 4 client.28054 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:53:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.378+0000 7f9875a28740 1 -- 172.21.15.115:0/2050731705 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a37e8b3d0 con 0x562a37b33d80 2024-05-09T05:53:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.378+0000 7f9867fff640 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a37b33d80 0x562a37f355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.383+0000 7f9867fff640 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a37b33d80 0x562a37f355c0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d2650498829314c server_cookie=cfcd5084a1907971 in_seq=0 out_seq=0 2024-05-09T05:53:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.384+0000 7f9847fff640 10 client.28054 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.386+0000 7f9847fff640 1 -- 172.21.15.115:0/2050731705 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a37e8b3d0 con 0x562a37b33d80 2024-05-09T05:53:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.386+0000 7f9847fff640 10 client.28054 handle_command_reply: tid=0 2024-05-09T05:53:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.386+0000 7f9875a28740 1 client.28054 shutdown 2024-05-09T05:53:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.386+0000 7f9875a28740 1 -- 172.21.15.115:0/2050731705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 msgr2=0x562a37f24930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.386+0000 7f9875a28740 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f24930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9860013590 tx=0x7f9860009aa0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.388+0000 7f9875a28740 1 -- 172.21.15.115:0/2050731705 shutdown_connections 2024-05-09T05:53:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.388+0000 7f9875a28740 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f22dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.388+0000 7f9875a28740 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 0x562a37f24e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.388+0000 7f9875a28740 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a37b33d80 0x562a37f355c0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.388+0000 7f9875a28740 1 --2- 172.21.15.115:0/2050731705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f24930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.388+0000 7f9875a28740 1 -- 172.21.15.115:0/2050731705 >> 172.21.15.115:0/2050731705 conn(0x562a37b69300 msgr2=0x562a37f13860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.388+0000 7f9847fff640 0 client.28054 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.389+0000 7f9875a28740 1 -- 172.21.15.115:0/2050731705 shutdown_connections 2024-05-09T05:53:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.390+0000 7f9875a28740 1 -- 172.21.15.115:0/2050731705 wait complete. 2024-05-09T05:53:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.394+0000 7f9875a28740 20 client.28054 trim_cache size 0 max 16384 2024-05-09T05:53:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.442+0000 7f9875a28740 1 Processor -- start 2024-05-09T05:53:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.444+0000 7f9875a28740 1 -- start start 2024-05-09T05:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.444+0000 7f9875a28740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f13f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.444+0000 7f9875a28740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b544d0 0x562a37f3adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.445+0000 7f9875a28740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b751b0 0x562a37f3b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.445+0000 7f9875a28740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f985400ffb0 con 0x562a37b544d0 2024-05-09T05:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.445+0000 7f9875a28740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9854011b20 con 0x562a37b751b0 2024-05-09T05:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.445+0000 7f9875a28740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9854010680 con 0x562a37b4c2f0 2024-05-09T05:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.445+0000 7f986c832640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f13f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.445+0000 7f986c832640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f13f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:44250/0 (socket says 172.21.15.115:44250) 2024-05-09T05:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.445+0000 7f986c832640 1 -- 172.21.15.115:0/614090053 learned_addr learned my addr 172.21.15.115:0/614090053 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.446+0000 7f9867fff640 1 --2- 172.21.15.115:0/614090053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b544d0 0x562a37f3adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.447+0000 7f986c832640 1 -- 172.21.15.115:0/614090053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b751b0 msgr2=0x562a37f3b300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:53:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.447+0000 7f986c832640 1 --2- 172.21.15.115:0/614090053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b751b0 0x562a37f3b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.447+0000 7f986c832640 1 -- 172.21.15.115:0/614090053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b544d0 msgr2=0x562a37f3adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.447+0000 7f986c832640 1 --2- 172.21.15.115:0/614090053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b544d0 0x562a37f3adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.447+0000 7f986c832640 1 -- 172.21.15.115:0/614090053 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a37c575f0 con 0x562a37b4c2f0 2024-05-09T05:53:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.448+0000 7f986c832640 1 --2- 172.21.15.115:0/614090053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f13f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f986001c6f0 tx=0x7f986001ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feda53a5e3edf789 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.449+0000 7f984e7fc640 1 -- 172.21.15.115:0/614090053 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860018040 con 0x562a37b4c2f0 2024-05-09T05:53:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.449+0000 7f984e7fc640 1 -- 172.21.15.115:0/614090053 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9860011e40 con 0x562a37b4c2f0 2024-05-09T05:53:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.450+0000 7f9875a28740 1 -- 172.21.15.115:0/614090053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 msgr2=0x562a37f13f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.450+0000 7f9875a28740 1 --2- 172.21.15.115:0/614090053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f13f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f986001c6f0 tx=0x7f986001ca70 comp rx=0 tx=0).stop 2024-05-09T05:53:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.450+0000 7f9875a28740 1 -- 172.21.15.115:0/614090053 shutdown_connections 2024-05-09T05:53:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.451+0000 7f9875a28740 1 --2- 172.21.15.115:0/614090053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b751b0 0x562a37f3b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.451+0000 7f9875a28740 1 --2- 172.21.15.115:0/614090053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b544d0 0x562a37f3adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.451+0000 7f9875a28740 1 --2- 172.21.15.115:0/614090053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f13f70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.451+0000 7f9875a28740 1 -- 172.21.15.115:0/614090053 >> 172.21.15.115:0/614090053 conn(0x562a37b69300 msgr2=0x562a37f13860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.451+0000 7f9875a28740 1 -- 172.21.15.115:0/614090053 shutdown_connections 2024-05-09T05:53:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.452+0000 7f9875a28740 1 -- 172.21.15.115:0/614090053 wait complete. 2024-05-09T05:53:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.452+0000 7f9875a28740 1 -- start start 2024-05-09T05:53:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.453+0000 7f9875a28740 1 Processor -- start 2024-05-09T05:53:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.453+0000 7f9875a28740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.454+0000 7f9875a28740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 0x562a37f2b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.454+0000 7f9875a28740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f27180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.454+0000 7f9875a28740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9854011450 con 0x562a37b751b0 2024-05-09T05:53:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.454+0000 7f9875a28740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9854010d80 con 0x562a37b544d0 2024-05-09T05:53:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.454+0000 7f9875a28740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f985400dfb0 con 0x562a37b4c2f0 2024-05-09T05:53:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.454+0000 7f9875a28740 10 client.0 _ll_register_callbacks cb 0x7f9860016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.456+0000 7f986d033640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f27180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.456+0000 7f986d033640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f27180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46298/0 (socket says 172.21.15.115:46298) 2024-05-09T05:53:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.456+0000 7f986d033640 1 -- 172.21.15.115:0/1761362009 learned_addr learned my addr 172.21.15.115:0/1761362009 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.457+0000 7f9867fff640 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 0x562a37f2b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.457+0000 7f986c832640 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f2acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.458+0000 7f9867fff640 1 -- 172.21.15.115:0/1761362009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 msgr2=0x562a37f2acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.458+0000 7f9867fff640 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f2acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.458+0000 7f9867fff640 1 -- 172.21.15.115:0/1761362009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 msgr2=0x562a37f27180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.458+0000 7f9867fff640 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f27180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.458+0000 7f9867fff640 1 -- 172.21.15.115:0/1761362009 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a37e41ed0 con 0x562a37b544d0 2024-05-09T05:53:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.460+0000 7f9867fff640 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 0x562a37f2b210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f985c100410 tx=0x7f985c02a940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=266baefdb5acf87c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.462+0000 7f9847fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:53:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.462+0000 7f9847fff640 1 -- 172.21.15.115:0/1761362009 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c049040 con 0x562a37b544d0 2024-05-09T05:53:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.462+0000 7f9847fff640 1 -- 172.21.15.115:0/1761362009 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f985c03bca0 con 0x562a37b544d0 2024-05-09T05:53:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.463+0000 7f9875a28740 1 -- 172.21.15.115:0/1761362009 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a37e85240 con 0x562a37b544d0 2024-05-09T05:53:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.467+0000 7f9847fff640 1 -- 172.21.15.115:0/1761362009 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f985c0cf310 con 0x562a37b544d0 2024-05-09T05:53:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.469+0000 7f9875a28740 10 client.27944 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.469+0000 7f9875a28740 1 -- 172.21.15.115:0/1761362009 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x562a37f9c1d0 con 0x562a37b544d0 2024-05-09T05:53:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.470+0000 7f9847fff640 1 -- 172.21.15.115:0/1761362009 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562a37e85240 con 0x562a37b544d0 2024-05-09T05:53:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.471+0000 7f9847fff640 1 -- 172.21.15.115:0/1761362009 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f985c0d3930 con 0x562a37b544d0 2024-05-09T05:53:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.475+0000 7f9875a28740 10 client.27944 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.475+0000 7f9875a28740 10 client.27944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.475+0000 7f9875a28740 20 client.27944 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.475+0000 7f9875a28740 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a37b091c0 0x562a37f9e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.475+0000 7f9875a28740 4 client.27944 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:53:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.475+0000 7f9875a28740 1 -- 172.21.15.115:0/1761362009 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a37e00350 con 0x562a37b091c0 2024-05-09T05:53:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.475+0000 7f986c832640 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a37b091c0 0x562a37f9e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.478+0000 7f986c832640 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a37b091c0 0x562a37f9e990 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a3d11c555dc8a7 server_cookie=d67453148426ede4 in_seq=0 out_seq=0 2024-05-09T05:53:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.478+0000 7f9847fff640 10 client.27944 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.535+0000 7f9847fff640 1 -- 172.21.15.115:0/1761362009 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a37e00350 con 0x562a37b091c0 2024-05-09T05:53:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.535+0000 7f9847fff640 10 client.27944 handle_command_reply: tid=0 2024-05-09T05:53:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.538+0000 7f9875a28740 1 client.27944 shutdown 2024-05-09T05:53:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.539+0000 7f9875a28740 1 -- 172.21.15.115:0/1761362009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 msgr2=0x562a37f2b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.539+0000 7f9875a28740 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 0x562a37f2b210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f985c100410 tx=0x7f985c02a940 comp rx=0 tx=0).stop 2024-05-09T05:53:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.539+0000 7f9875a28740 1 -- 172.21.15.115:0/1761362009 shutdown_connections 2024-05-09T05:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.539+0000 7f9875a28740 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a37b751b0 0x562a37f27180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.539+0000 7f9875a28740 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a37b544d0 0x562a37f2b210 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.539+0000 7f9875a28740 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a37b091c0 0x562a37f9e990 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.539+0000 7f9875a28740 1 --2- 172.21.15.115:0/1761362009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a37b4c2f0 0x562a37f2acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.539+0000 7f9875a28740 1 -- 172.21.15.115:0/1761362009 >> 172.21.15.115:0/1761362009 conn(0x562a37b69300 msgr2=0x562a37f212a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.541+0000 7f9847fff640 0 client.27944 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.541+0000 7f9875a28740 1 -- 172.21.15.115:0/1761362009 shutdown_connections 2024-05-09T05:53:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.544+0000 7f9875a28740 1 -- 172.21.15.115:0/1761362009 wait complete. 2024-05-09T05:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.546+0000 7f9875a28740 20 client.27944 trim_cache size 0 max 16384 2024-05-09T05:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:53:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 -- 172.21.15.115:0/3417074742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98500b2aa0 msgr2=0x7f98500b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98500b2aa0 0x7f98500b4fb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f985800b3a0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 -- 172.21.15.115:0/3417074742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868110ba0 msgr2=0x7f98681210e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868110ba0 0x7f98681210e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f985c0058c0 tx=0x7f985c003b90 comp rx=0 tx=0).stop 2024-05-09T05:53:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 -- 172.21.15.115:0/3417074742 shutdown_connections 2024-05-09T05:53:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f98500b2aa0 0x7f98500b4fb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9868111660 0x7f9868126890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9868110ba0 0x7f98681210e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 --2- 172.21.15.115:0/3417074742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f986810c180 0x7f9868120b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:03.542 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.552+0000 7f986d834640 1 -- 172.21.15.115:0/3417074742 >> 172.21.15.115:0/3417074742 conn(0x7f986806f810 msgr2=0x7f9868113b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.562+0000 7f986d834640 1 -- 172.21.15.115:0/3417074742 shutdown_connections 2024-05-09T05:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:03.563+0000 7f986d834640 1 -- 172.21.15.115:0/3417074742 wait complete. 2024-05-09T05:53:03.643 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:03 smithi115 ceph-mon[30349]: pgmap v467: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-05-09T05:53:03.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:03 smithi161 ceph-mon[33805]: pgmap v467: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-05-09T05:53:03.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:03 smithi195 ceph-mon[33844]: pgmap v467: 129 pgs: 129 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-05-09T05:53:05.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:05 smithi195 ceph-mon[33844]: pgmap v468: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 21 MiB/s wr, 5.33k op/s 2024-05-09T05:53:05.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:05 smithi195 ceph-mon[33844]: osdmap e108: 12 total, 12 up, 12 in 2024-05-09T05:53:05.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:05 smithi115 ceph-mon[30349]: pgmap v468: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 21 MiB/s wr, 5.33k op/s 2024-05-09T05:53:05.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:05 smithi115 ceph-mon[30349]: osdmap e108: 12 total, 12 up, 12 in 2024-05-09T05:53:05.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:05 smithi161 ceph-mon[33805]: pgmap v468: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 21 MiB/s wr, 5.33k op/s 2024-05-09T05:53:05.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:05 smithi161 ceph-mon[33805]: osdmap e108: 12 total, 12 up, 12 in 2024-05-09T05:53:07.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:07 smithi195 ceph-mon[33844]: pgmap v470: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-05-09T05:53:07.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:07 smithi115 ceph-mon[30349]: pgmap v470: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-05-09T05:53:07.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:07 smithi161 ceph-mon[33805]: pgmap v470: 129 pgs: 129 active+clean; 13 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-05-09T05:53:09.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:09 smithi195 ceph-mon[33844]: pgmap v471: 129 pgs: 10 active+clean+snaptrim, 94 active+clean, 25 active+clean+snaptrim_wait; 13 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2024-05-09T05:53:09.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:09 smithi115 ceph-mon[30349]: pgmap v471: 129 pgs: 10 active+clean+snaptrim, 94 active+clean, 25 active+clean+snaptrim_wait; 13 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2024-05-09T05:53:09.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:09 smithi161 ceph-mon[33805]: pgmap v471: 129 pgs: 10 active+clean+snaptrim, 94 active+clean, 25 active+clean+snaptrim_wait; 13 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2024-05-09T05:53:11.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:11 smithi195 ceph-mon[33844]: pgmap v472: 129 pgs: 14 active+clean+snaptrim, 82 active+clean, 33 active+clean+snaptrim_wait; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 4.10k op/s 2024-05-09T05:53:11.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:11 smithi115 ceph-mon[30349]: pgmap v472: 129 pgs: 14 active+clean+snaptrim, 82 active+clean, 33 active+clean+snaptrim_wait; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 4.10k op/s 2024-05-09T05:53:11.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:11 smithi161 ceph-mon[33805]: pgmap v472: 129 pgs: 14 active+clean+snaptrim, 82 active+clean, 33 active+clean+snaptrim_wait; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 4.10k op/s 2024-05-09T05:53:13.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:13 smithi195 ceph-mon[33844]: pgmap v473: 129 pgs: 14 active+clean+snaptrim, 82 active+clean, 33 active+clean+snaptrim_wait; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 4.10k op/s 2024-05-09T05:53:13.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:13 smithi115 ceph-mon[30349]: pgmap v473: 129 pgs: 14 active+clean+snaptrim, 82 active+clean, 33 active+clean+snaptrim_wait; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 4.10k op/s 2024-05-09T05:53:13.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:13 smithi161 ceph-mon[33805]: pgmap v473: 129 pgs: 14 active+clean+snaptrim, 82 active+clean, 33 active+clean+snaptrim_wait; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 4.10k op/s 2024-05-09T05:53:15.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:15 smithi195 ceph-mon[33844]: pgmap v474: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2024-05-09T05:53:15.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:15 smithi115 ceph-mon[30349]: pgmap v474: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2024-05-09T05:53:15.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:15 smithi161 ceph-mon[33805]: pgmap v474: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2024-05-09T05:53:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.859+0000 7fcc96dc9640 1 -- 172.21.15.115:0/3188043888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc9010fa10 msgr2=0x7fcc9010fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.859+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/3188043888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc9010fa10 0x7fcc9010fe20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009ca0 tx=0x7fcc8802a8d0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.862+0000 7fcc96dc9640 1 -- 172.21.15.115:0/3188043888 shutdown_connections 2024-05-09T05:53:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.862+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/3188043888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc9010a210 0x7fcc9010a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.862+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/3188043888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc901103f0 0x7fcc90109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.862+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/3188043888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc9010fa10 0x7fcc9010fe20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.862+0000 7fcc96dc9640 1 -- 172.21.15.115:0/3188043888 >> 172.21.15.115:0/3188043888 conn(0x7fcc9006f810 msgr2=0x7fcc9006fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.866+0000 7fcc96dc9640 1 -- 172.21.15.115:0/3188043888 shutdown_connections 2024-05-09T05:53:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.869+0000 7fcc96dc9640 1 -- 172.21.15.115:0/3188043888 wait complete. 2024-05-09T05:53:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.870+0000 7fcc96dc9640 1 Processor -- start 2024-05-09T05:53:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.871+0000 7fcc96dc9640 1 -- start start 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.871+0000 7fcc96dc9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc9010a210 0x7fcc901a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.871+0000 7fcc96dc9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc9010fa10 0x7fcc901a9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.871+0000 7fcc96dc9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc901103f0 0x7fcc901adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.871+0000 7fcc96dc9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90122910 con 0x7fcc9010a210 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.871+0000 7fcc96dc9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90122790 con 0x7fcc9010fa10 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.871+0000 7fcc96dc9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90122a90 con 0x7fcc901103f0 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.872+0000 7fcc95dc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc9010a210 0x7fcc901a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.872+0000 7fcc95dc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc9010a210 0x7fcc901a93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40518/0 (socket says 172.21.15.115:40518) 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.872+0000 7fcc95dc7640 1 -- 172.21.15.115:0/2307388007 learned_addr learned my addr 172.21.15.115:0/2307388007 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.873+0000 7fcc95dc7640 1 -- 172.21.15.115:0/2307388007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc901103f0 msgr2=0x7fcc901adbf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.873+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc901103f0 0x7fcc901adbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.873+0000 7fcc95dc7640 1 -- 172.21.15.115:0/2307388007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc9010fa10 msgr2=0x7fcc901a9930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:53:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.873+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc9010fa10 0x7fcc901a9930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.873+0000 7fcc95dc7640 1 -- 172.21.15.115:0/2307388007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc901ae230 con 0x7fcc9010a210 2024-05-09T05:53:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.874+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc9010a210 0x7fcc901a93d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcc8802ae40 tx=0x7fcc88032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.875+0000 7fcc86ffd640 1 -- 172.21.15.115:0/2307388007 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88003c70 con 0x7fcc9010a210 2024-05-09T05:53:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.875+0000 7fcc96dc9640 1 -- 172.21.15.115:0/2307388007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc901ae4c0 con 0x7fcc9010a210 2024-05-09T05:53:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.875+0000 7fcc96dc9640 1 -- 172.21.15.115:0/2307388007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc901aea10 con 0x7fcc9010a210 2024-05-09T05:53:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.878+0000 7fcc9efc0740 1 Processor -- start 2024-05-09T05:53:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc9efc0740 1 -- start start 2024-05-09T05:53:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 0x55e775a88840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756bc2f0 0x55e775a88d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756c44d0 0x55e775a892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e7759ed7c0 con 0x55e7756bc2f0 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e7759890e0 con 0x55e7756e51b0 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e7759e9780 con 0x55e7756c44d0 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc965c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 0x55e775a88840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc965c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 0x55e775a88840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:60450/0 (socket says 172.21.15.115:60450) 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc965c8640 1 -- 172.21.15.115:0/3049650753 learned_addr learned my addr 172.21.15.115:0/3049650753 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc86ffd640 1 -- 172.21.15.115:0/2307388007 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc88003e10 con 0x7fcc9010a210 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.879+0000 7fcc86ffd640 1 -- 172.21.15.115:0/2307388007 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88030ac0 con 0x7fcc9010a210 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.881+0000 7fcc955c6640 1 --2- 172.21.15.115:0/3049650753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756c44d0 0x55e775a892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.881+0000 7fcc955c6640 1 -- 172.21.15.115:0/3049650753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 msgr2=0x55e775a88840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.881+0000 7fcc955c6640 1 --2- 172.21.15.115:0/3049650753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 0x55e775a88840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.881+0000 7fcc955c6640 1 -- 172.21.15.115:0/3049650753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756bc2f0 msgr2=0x55e775a88d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.881+0000 7fcc955c6640 1 --2- 172.21.15.115:0/3049650753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756bc2f0 0x55e775a88d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.881+0000 7fcc955c6640 1 -- 172.21.15.115:0/3049650753 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7759b27c0 con 0x55e7756c44d0 2024-05-09T05:53:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.881+0000 7fcc955c6640 1 --2- 172.21.15.115:0/3049650753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756c44d0 0x55e775a892c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc9006c6f0 tx=0x7fcc8c009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd4b97f8e6c8e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.884+0000 7fcc7bfff640 1 -- 172.21.15.115:0/3049650753 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00ecb0 con 0x55e7756c44d0 2024-05-09T05:53:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.884+0000 7fcc7bfff640 1 -- 172.21.15.115:0/3049650753 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc8c00ee50 con 0x55e7756c44d0 2024-05-09T05:53:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.885+0000 7fcc86ffd640 1 -- 172.21.15.115:0/2307388007 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcc88030c60 con 0x7fcc9010a210 2024-05-09T05:53:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.885+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3049650753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756c44d0 msgr2=0x55e775a892c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.885+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3049650753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756c44d0 0x55e775a892c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc9006c6f0 tx=0x7fcc8c009f30 comp rx=0 tx=0).stop 2024-05-09T05:53:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.887+0000 7fcc86ffd640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc7c0b2b60 0x7fcc7c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.888+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3049650753 shutdown_connections 2024-05-09T05:53:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.888+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3049650753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756c44d0 0x55e775a892c0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc9006c6f0 tx=0x7fcc8c009f30 comp rx=0 tx=0).stop 2024-05-09T05:53:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.888+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3049650753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756bc2f0 0x55e775a88d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.888+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3049650753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 0x55e775a88840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.888+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3049650753 >> 172.21.15.115:0/3049650753 conn(0x55e7756d9300 msgr2=0x55e775a84ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.888+0000 7fcc86ffd640 1 -- 172.21.15.115:0/2307388007 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9526+0+0 (secure 0 0 0) 0x7fcc88105f60 con 0x7fcc9010a210 2024-05-09T05:53:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.888+0000 7fcc965c8640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc7c0b2b60 0x7fcc7c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.890+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3049650753 shutdown_connections 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.890+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3049650753 wait complete. 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.891+0000 7fcc9efc0740 1 -- start start 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.891+0000 7fcc9efc0740 1 Processor -- start 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.891+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756bc2f0 0x55e775a948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.891+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 0x55e775a94e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.891+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756a3d80 0x55e775a92e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.891+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55e775a959c0 con 0x55e7756bc2f0 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.891+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e7759749c0 con 0x55e7756e51b0 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.891+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e775a95b40 con 0x55e7756a3d80 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.891+0000 7fcc9efc0740 10 client.0 _ll_register_callbacks cb 0x55e7759a1020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.892+0000 7fcc965c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756a3d80 0x55e775a92e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.892+0000 7fcc965c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756a3d80 0x55e775a92e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33790/0 (socket says 172.21.15.115:33790) 2024-05-09T05:53:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.892+0000 7fcc965c8640 1 -- 172.21.15.115:0/626378147 learned_addr learned my addr 172.21.15.115:0/626378147 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.893+0000 7fcc965c8640 1 -- 172.21.15.115:0/626378147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 msgr2=0x55e775a94e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:53:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.893+0000 7fcc965c8640 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 0x55e775a94e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.893+0000 7fcc965c8640 1 -- 172.21.15.115:0/626378147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756bc2f0 msgr2=0x55e775a948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.893+0000 7fcc965c8640 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756bc2f0 0x55e775a948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.893+0000 7fcc965c8640 1 -- 172.21.15.115:0/626378147 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e77599b1f0 con 0x55e7756a3d80 2024-05-09T05:53:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.894+0000 7fcc965c8640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc7c0b2b60 0x7fcc7c0b5070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcc80038510 tx=0x7fcc8002bbd0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.894+0000 7fcc965c8640 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756a3d80 0x55e775a92e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc80038fd0 tx=0x7fcc800047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81df9afef7403c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.898+0000 7fcc797fa640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:53:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.898+0000 7fcc797fa640 1 -- 172.21.15.115:0/626378147 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80030040 con 0x55e7756a3d80 2024-05-09T05:53:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.898+0000 7fcc797fa640 1 -- 172.21.15.115:0/626378147 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9526+0+0 (secure 0 0 0) 0x7fcc8002b380 con 0x55e7756a3d80 2024-05-09T05:53:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.898+0000 7fcc9efc0740 1 -- 172.21.15.115:0/626378147 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc8c00ee50 con 0x55e7756a3d80 2024-05-09T05:53:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.904+0000 7fcc797fa640 1 -- 172.21.15.115:0/626378147 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fcc8004f050 con 0x55e7756a3d80 2024-05-09T05:53:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.904+0000 7fcc9efc0740 10 client.28069 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.904+0000 7fcc9efc0740 1 -- 172.21.15.115:0/626378147 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55e775750790 con 0x55e7756a3d80 2024-05-09T05:53:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.906+0000 7fcc797fa640 1 -- 172.21.15.115:0/626378147 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc8c00ee50 con 0x55e7756a3d80 2024-05-09T05:53:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.906+0000 7fcc797fa640 1 -- 172.21.15.115:0/626378147 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7fcc80044ab0 con 0x55e7756a3d80 2024-05-09T05:53:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.907+0000 7fcc9efc0740 10 client.28069 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.907+0000 7fcc9efc0740 10 client.28069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.907+0000 7fcc9efc0740 20 client.28069 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.907+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7756791c0 0x55e775aa5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.907+0000 7fcc9efc0740 4 client.28069 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:53:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.907+0000 7fcc9efc0740 1 -- 172.21.15.115:0/626378147 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7759fb3d0 con 0x55e7756791c0 2024-05-09T05:53:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.907+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7756791c0 0x55e775aa5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.908+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7756791c0 0x55e775aa5440 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcdd862bf3b41ba3 server_cookie=c9b50cd1f3e88a18 in_seq=0 out_seq=0 2024-05-09T05:53:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.908+0000 7fcc797fa640 10 client.28069 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.910+0000 7fcc797fa640 1 -- 172.21.15.115:0/626378147 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e7759fb3d0 con 0x55e7756791c0 2024-05-09T05:53:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.910+0000 7fcc797fa640 10 client.28069 handle_command_reply: tid=0 2024-05-09T05:53:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.910+0000 7fcc9efc0740 1 client.28069 shutdown 2024-05-09T05:53:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.913+0000 7fcc9efc0740 1 -- 172.21.15.115:0/626378147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756a3d80 msgr2=0x55e775a92e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.913+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756a3d80 0x55e775a92e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc80038fd0 tx=0x7fcc800047d0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.917+0000 7fcc9efc0740 1 -- 172.21.15.115:0/626378147 shutdown_connections 2024-05-09T05:53:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.917+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e7756791c0 0x55e775aa5440 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.917+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756a3d80 0x55e775a92e00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.917+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756e51b0 0x55e775a94e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.917+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/626378147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756bc2f0 0x55e775a948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.917+0000 7fcc9efc0740 1 -- 172.21.15.115:0/626378147 >> 172.21.15.115:0/626378147 conn(0x55e7756d9300 msgr2=0x55e775a8a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.917+0000 7fcc797fa640 0 client.28069 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.922+0000 7fcc9efc0740 1 -- 172.21.15.115:0/626378147 shutdown_connections 2024-05-09T05:53:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.922+0000 7fcc9efc0740 1 -- 172.21.15.115:0/626378147 wait complete. 2024-05-09T05:53:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.922+0000 7fcc9efc0740 20 client.28069 trim_cache size 0 max 16384 2024-05-09T05:53:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.968+0000 7fcc9efc0740 1 Processor -- start 2024-05-09T05:53:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.969+0000 7fcc9efc0740 1 -- start start 2024-05-09T05:53:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.969+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 0x55e775aaac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.969+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 0x55e775aab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.969+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a8c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.969+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74010110 con 0x55e7756a3d80 2024-05-09T05:53:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.969+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74011c80 con 0x55e7756bc2f0 2024-05-09T05:53:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc965c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a8c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc965c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a8c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33802/0 (socket says 172.21.15.115:33802) 2024-05-09T05:53:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc965c8640 1 -- 172.21.15.115:0/3489384470 learned_addr learned my addr 172.21.15.115:0/3489384470 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/3489384470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 0x55e775aaac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3489384470 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc740107e0 con 0x55e7756e51b0 2024-05-09T05:53:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc965c8640 1 -- 172.21.15.115:0/3489384470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 msgr2=0x55e775aab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc965c8640 1 --2- 172.21.15.115:0/3489384470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 0x55e775aab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc965c8640 1 -- 172.21.15.115:0/3489384470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 msgr2=0x55e775aaac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc965c8640 1 --2- 172.21.15.115:0/3489384470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 0x55e775aaac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.970+0000 7fcc965c8640 1 -- 172.21.15.115:0/3489384470 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7757c75f0 con 0x55e7756e51b0 2024-05-09T05:53:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.971+0000 7fcc965c8640 1 --2- 172.21.15.115:0/3489384470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a8c4f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc80044f20 tx=0x7fcc80006f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f3e32b19c20905 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.972+0000 7fcc7b7fe640 1 -- 172.21.15.115:0/3489384470 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80030040 con 0x55e7756e51b0 2024-05-09T05:53:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.972+0000 7fcc7b7fe640 1 -- 172.21.15.115:0/3489384470 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc80040070 con 0x55e7756e51b0 2024-05-09T05:53:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.973+0000 7fcc7b7fe640 1 -- 172.21.15.115:0/3489384470 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80044b00 con 0x55e7756e51b0 2024-05-09T05:53:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.978+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3489384470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 msgr2=0x55e775a8c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.978+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3489384470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a8c4f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc80044f20 tx=0x7fcc80006f30 comp rx=0 tx=0).stop 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.978+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3489384470 shutdown_connections 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.978+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3489384470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a8c4f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.978+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3489384470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 0x55e775aab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.978+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3489384470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 0x55e775aaac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.978+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3489384470 >> 172.21.15.115:0/3489384470 conn(0x55e7756d9300 msgr2=0x55e775a916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.978+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3489384470 shutdown_connections 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.978+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3489384470 wait complete. 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.979+0000 7fcc9efc0740 1 -- start start 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.979+0000 7fcc9efc0740 1 Processor -- start 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 0x55e775a95e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 0x55e775a96350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc9efc0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a96890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc740115b0 con 0x55e7756a3d80 2024-05-09T05:53:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74010ee0 con 0x55e7756bc2f0 2024-05-09T05:53:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc9efc0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7400e110 con 0x55e7756e51b0 2024-05-09T05:53:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc955c6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 0x55e775a96350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc955c6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 0x55e775a96350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:60490/0 (socket says 172.21.15.115:60490) 2024-05-09T05:53:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc955c6640 1 -- 172.21.15.115:0/3493977588 learned_addr learned my addr 172.21.15.115:0/3493977588 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc965c8640 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a96890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.980+0000 7fcc9efc0740 10 client.0 _ll_register_callbacks cb 0x7fcc8002baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.981+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 0x55e775a95e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.981+0000 7fcc95dc7640 1 -- 172.21.15.115:0/3493977588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 msgr2=0x55e775a96890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.981+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a96890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.981+0000 7fcc95dc7640 1 -- 172.21.15.115:0/3493977588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 msgr2=0x55e775a96350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.981+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 0x55e775a96350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.981+0000 7fcc95dc7640 1 -- 172.21.15.115:0/3493977588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7759b1ed0 con 0x55e7756a3d80 2024-05-09T05:53:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.982+0000 7fcc95dc7640 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 0x55e775a95e10 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fcc88103370 tx=0x7fcc88031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd048098713bc3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.982+0000 7fcc797fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:53:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.982+0000 7fcc797fa640 1 -- 172.21.15.115:0/3493977588 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc881036f0 con 0x55e7756a3d80 2024-05-09T05:53:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.983+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3493977588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc80040070 con 0x55e7756a3d80 2024-05-09T05:53:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.983+0000 7fcc797fa640 1 -- 172.21.15.115:0/3493977588 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9526+0+0 (secure 0 0 0) 0x7fcc880d58d0 con 0x55e7756a3d80 2024-05-09T05:53:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.983+0000 7fcc797fa640 1 -- 172.21.15.115:0/3493977588 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fcc880ccf00 con 0x55e7756a3d80 2024-05-09T05:53:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.983+0000 7fcc9efc0740 10 client.18756 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.983+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3493977588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55e775b0c990 con 0x55e7756a3d80 2024-05-09T05:53:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.984+0000 7fcc797fa640 1 -- 172.21.15.115:0/3493977588 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc80040070 con 0x55e7756a3d80 2024-05-09T05:53:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.984+0000 7fcc797fa640 1 -- 172.21.15.115:0/3493977588 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7fcc880c8870 con 0x55e7756a3d80 2024-05-09T05:53:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.986+0000 7fcc9efc0740 10 client.18756 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.986+0000 7fcc9efc0740 10 client.18756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.986+0000 7fcc9efc0740 20 client.18756 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.986+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e775b0d040 0x55e775b0f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.986+0000 7fcc9efc0740 4 client.18756 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:53:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.986+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3493977588 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7759f5240 con 0x55e775b0d040 2024-05-09T05:53:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.987+0000 7fcc955c6640 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e775b0d040 0x55e775b0f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.989+0000 7fcc955c6640 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e775b0d040 0x55e775b0f490 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb81ba93961202d server_cookie=c5329114e0dec830 in_seq=0 out_seq=0 2024-05-09T05:53:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:16.989+0000 7fcc797fa640 10 client.18756 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.114+0000 7fcc797fa640 1 -- 172.21.15.115:0/3493977588 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e7759f5240 con 0x55e775b0d040 2024-05-09T05:53:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.114+0000 7fcc797fa640 10 client.18756 handle_command_reply: tid=0 2024-05-09T05:53:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.114+0000 7fcc9efc0740 1 client.18756 shutdown 2024-05-09T05:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.117+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3493977588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 msgr2=0x55e775a95e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.117+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 0x55e775a95e10 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fcc88103370 tx=0x7fcc88031000 comp rx=0 tx=0).stop 2024-05-09T05:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.117+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3493977588 shutdown_connections 2024-05-09T05:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.117+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55e775b0d040 0x55e775b0f490 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.117+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7756e51b0 0x55e775a96890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.117+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7756bc2f0 0x55e775a96350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.117+0000 7fcc9efc0740 1 --2- 172.21.15.115:0/3493977588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e7756a3d80 0x55e775a95e10 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.117+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3493977588 >> 172.21.15.115:0/3493977588 conn(0x55e7756d9300 msgr2=0x55e775a849b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.121+0000 7fcc797fa640 0 client.18756 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.121+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3493977588 shutdown_connections 2024-05-09T05:53:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.121+0000 7fcc9efc0740 1 -- 172.21.15.115:0/3493977588 wait complete. 2024-05-09T05:53:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.121+0000 7fcc9efc0740 20 client.18756 trim_cache size 0 max 16384 2024-05-09T05:53:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:53:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:53:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 -- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc7c0b2b60 msgr2=0x7fcc7c0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc7c0b2b60 0x7fcc7c0b5070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcc80038510 tx=0x7fcc8002bbd0 comp rx=0 tx=0).stop 2024-05-09T05:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 -- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc9010a210 msgr2=0x7fcc901a93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc9010a210 0x7fcc901a93d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcc8802ae40 tx=0x7fcc88032f80 comp rx=0 tx=0).stop 2024-05-09T05:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 -- 172.21.15.115:0/2307388007 shutdown_connections 2024-05-09T05:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc7c0b2b60 0x7fcc7c0b5070 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc901103f0 0x7fcc901adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc9010fa10 0x7fcc901a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 --2- 172.21.15.115:0/2307388007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc9010a210 0x7fcc901a93d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.124+0000 7fcc96dc9640 1 -- 172.21.15.115:0/2307388007 >> 172.21.15.115:0/2307388007 conn(0x7fcc9006f810 msgr2=0x7fcc9010d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.131+0000 7fcc96dc9640 1 -- 172.21.15.115:0/2307388007 shutdown_connections 2024-05-09T05:53:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:17.131+0000 7fcc96dc9640 1 -- 172.21.15.115:0/2307388007 wait complete. 2024-05-09T05:53:17.708 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:17 smithi161 ceph-mon[33805]: pgmap v475: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-05-09T05:53:17.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:17 smithi195 ceph-mon[33844]: pgmap v475: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-05-09T05:53:17.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:17 smithi115 ceph-mon[30349]: pgmap v475: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-05-09T05:53:19.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:19 smithi195 ceph-mon[33844]: pgmap v476: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 21 MiB/s wr, 5.39k op/s 2024-05-09T05:53:19.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:19 smithi115 ceph-mon[30349]: pgmap v476: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 21 MiB/s wr, 5.39k op/s 2024-05-09T05:53:19.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:19 smithi161 ceph-mon[33805]: pgmap v476: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 21 MiB/s wr, 5.39k op/s 2024-05-09T05:53:21.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:21 smithi195 ceph-mon[33844]: pgmap v477: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-05-09T05:53:21.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:21 smithi115 ceph-mon[30349]: pgmap v477: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-05-09T05:53:21.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:21 smithi161 ceph-mon[33805]: pgmap v477: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-05-09T05:53:23.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:23 smithi195 ceph-mon[33844]: pgmap v478: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.76k op/s 2024-05-09T05:53:23.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:23 smithi115 ceph-mon[30349]: pgmap v478: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.76k op/s 2024-05-09T05:53:23.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:23 smithi161 ceph-mon[33805]: pgmap v478: 129 pgs: 8 active+clean+snaptrim, 110 active+clean, 11 active+clean+snaptrim_wait; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 3.76k op/s 2024-05-09T05:53:25.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:25 smithi195 ceph-mon[33844]: pgmap v479: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 21 MiB/s wr, 5.38k op/s 2024-05-09T05:53:25.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:25 smithi115 ceph-mon[30349]: pgmap v479: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 21 MiB/s wr, 5.38k op/s 2024-05-09T05:53:25.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:25 smithi161 ceph-mon[33805]: pgmap v479: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 21 MiB/s wr, 5.38k op/s 2024-05-09T05:53:27.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:27 smithi195 ceph-mon[33844]: pgmap v480: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2024-05-09T05:53:27.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:27 smithi115 ceph-mon[30349]: pgmap v480: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2024-05-09T05:53:27.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:27 smithi161 ceph-mon[33805]: pgmap v480: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2024-05-09T05:53:29.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:29 smithi195 ceph-mon[33844]: pgmap v481: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 5.07k op/s 2024-05-09T05:53:29.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:29 smithi115 ceph-mon[30349]: pgmap v481: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 5.07k op/s 2024-05-09T05:53:29.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:29 smithi161 ceph-mon[33805]: pgmap v481: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 5.07k op/s 2024-05-09T05:53:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.384+0000 7f5234abd640 1 -- 172.21.15.115:0/3883319034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52301103f0 msgr2=0x7f5230109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.384+0000 7f5234abd640 1 --2- 172.21.15.115:0/3883319034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52301103f0 0x7f5230109cd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5220009900 tx=0x7f522002a420 comp rx=0 tx=0).stop 2024-05-09T05:53:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.385+0000 7f5234abd640 1 -- 172.21.15.115:0/3883319034 shutdown_connections 2024-05-09T05:53:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.385+0000 7f5234abd640 1 --2- 172.21.15.115:0/3883319034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f523010a210 0x7f523010a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.385+0000 7f5234abd640 1 --2- 172.21.15.115:0/3883319034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52301103f0 0x7f5230109cd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.385+0000 7f5234abd640 1 --2- 172.21.15.115:0/3883319034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f523010fa10 0x7f523010fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.385+0000 7f5234abd640 1 -- 172.21.15.115:0/3883319034 >> 172.21.15.115:0/3883319034 conn(0x7f523006f810 msgr2=0x7f523006fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.394+0000 7f5234abd640 1 -- 172.21.15.115:0/3883319034 shutdown_connections 2024-05-09T05:53:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.395+0000 7f5234abd640 1 -- 172.21.15.115:0/3883319034 wait complete. 2024-05-09T05:53:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.396+0000 7f5234abd640 1 Processor -- start 2024-05-09T05:53:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.397+0000 7f5234abd640 1 -- start start 2024-05-09T05:53:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f5234abd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f523010a210 0x7f52301a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f5234abd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f523010fa10 0x7f52301a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f5234abd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52301103f0 0x7f52301adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f5234abd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5230122990 con 0x7f523010fa10 2024-05-09T05:53:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f5234abd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5230122810 con 0x7f52301103f0 2024-05-09T05:53:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f5234abd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5230122b10 con 0x7f523010a210 2024-05-09T05:53:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f522f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f523010a210 0x7f52301a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f522f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f523010a210 0x7f52301a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:51822/0 (socket says 172.21.15.115:51822) 2024-05-09T05:53:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f522f7fe640 1 -- 172.21.15.115:0/3301960891 learned_addr learned my addr 172.21.15.115:0/3301960891 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.398+0000 7f522ffff640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52301103f0 0x7f52301adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.399+0000 7f522f7fe640 1 -- 172.21.15.115:0/3301960891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52301103f0 msgr2=0x7f52301adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.399+0000 7f522f7fe640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52301103f0 0x7f52301adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.399+0000 7f522f7fe640 1 -- 172.21.15.115:0/3301960891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f523010fa10 msgr2=0x7f52301a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.399+0000 7f522f7fe640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f523010fa10 0x7f52301a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.399+0000 7f522f7fe640 1 -- 172.21.15.115:0/3301960891 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52301ae290 con 0x7f523010a210 2024-05-09T05:53:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.399+0000 7f522f7fe640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f523010a210 0x7f52301a9430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f522400b4a0 tx=0x7f522400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.399+0000 7f522cff9640 1 -- 172.21.15.115:0/3301960891 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224014070 con 0x7f523010a210 2024-05-09T05:53:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.399+0000 7f5234abd640 1 -- 172.21.15.115:0/3301960891 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301ae580 con 0x7f523010a210 2024-05-09T05:53:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.400+0000 7f5234abd640 1 -- 172.21.15.115:0/3301960891 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301b5f00 con 0x7f523010a210 2024-05-09T05:53:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.403+0000 7f523cbd1740 1 Processor -- start 2024-05-09T05:53:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.403+0000 7f522cff9640 1 -- 172.21.15.115:0/3301960891 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5224010aa0 con 0x7f523010a210 2024-05-09T05:53:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.403+0000 7f522cff9640 1 -- 172.21.15.115:0/3301960891 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224017b80 con 0x7f523010a210 2024-05-09T05:53:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.403+0000 7f523cbd1740 1 -- start start 2024-05-09T05:53:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.405+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 0x55a87b6ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.405+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 0x55a87b6add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.405+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e94d0 0x55a87b6ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.405+0000 7f523cbd1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a87b6127c0 con 0x55a87b30a1b0 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.405+0000 7f523cbd1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a87b5ae0e0 con 0x55a87b2e94d0 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.405+0000 7f523cbd1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a87b60e780 con 0x55a87b2e12f0 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.405+0000 7f522effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e94d0 0x55a87b6ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.405+0000 7f522effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e94d0 0x55a87b6ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:55976/0 (socket says 172.21.15.115:55976) 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.405+0000 7f522effd640 1 -- 172.21.15.115:0/2067013128 learned_addr learned my addr 172.21.15.115:0/2067013128 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.406+0000 7f522effd640 1 -- 172.21.15.115:0/2067013128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 msgr2=0x55a87b6add80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.406+0000 7f522effd640 1 --2- 172.21.15.115:0/2067013128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 0x55a87b6add80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.406+0000 7f522effd640 1 -- 172.21.15.115:0/2067013128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 msgr2=0x55a87b6ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.406+0000 7f522effd640 1 --2- 172.21.15.115:0/2067013128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 0x55a87b6ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.406+0000 7f522effd640 1 -- 172.21.15.115:0/2067013128 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87b5d77c0 con 0x55a87b2e94d0 2024-05-09T05:53:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.406+0000 7f522effd640 1 --2- 172.21.15.115:0/2067013128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e94d0 0x55a87b6ae2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5220005ab0 tx=0x7f5220004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad48c012517b7b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.406+0000 7f5215ffb640 1 -- 172.21.15.115:0/2067013128 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522003e030 con 0x55a87b2e94d0 2024-05-09T05:53:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.407+0000 7f5215ffb640 1 -- 172.21.15.115:0/2067013128 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52200042f0 con 0x55a87b2e94d0 2024-05-09T05:53:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.407+0000 7f5215ffb640 1 -- 172.21.15.115:0/2067013128 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220035040 con 0x55a87b2e94d0 2024-05-09T05:53:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.409+0000 7f523cbd1740 1 -- 172.21.15.115:0/2067013128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e94d0 msgr2=0x55a87b6ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.409+0000 7f523cbd1740 1 --2- 172.21.15.115:0/2067013128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e94d0 0x55a87b6ae2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5220005ab0 tx=0x7f5220004180 comp rx=0 tx=0).stop 2024-05-09T05:53:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.409+0000 7f522cff9640 1 -- 172.21.15.115:0/3301960891 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f5224017070 con 0x7f523010a210 2024-05-09T05:53:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.410+0000 7f522cff9640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f52180b2aa0 0x7f52180b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.410+0000 7f522cff9640 1 -- 172.21.15.115:0/3301960891 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9526+0+0 (secure 0 0 0) 0x7f52240e1ae0 con 0x7f523010a210 2024-05-09T05:53:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.411+0000 7f522ffff640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f52180b2aa0 0x7f52180b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.412+0000 7f522ffff640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f52180b2aa0 0x7f52180b4fb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5228005c40 tx=0x7f5228005bd0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.413+0000 7f523cbd1740 1 -- 172.21.15.115:0/2067013128 shutdown_connections 2024-05-09T05:53:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.413+0000 7f523cbd1740 1 --2- 172.21.15.115:0/2067013128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e94d0 0x55a87b6ae2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.413+0000 7f523cbd1740 1 --2- 172.21.15.115:0/2067013128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 0x55a87b6add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.413+0000 7f523cbd1740 1 --2- 172.21.15.115:0/2067013128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 0x55a87b6ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.413+0000 7f523cbd1740 1 -- 172.21.15.115:0/2067013128 >> 172.21.15.115:0/2067013128 conn(0x55a87b2fe300 msgr2=0x55a87b6a9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.415+0000 7f523cbd1740 1 -- 172.21.15.115:0/2067013128 shutdown_connections 2024-05-09T05:53:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.416+0000 7f523cbd1740 1 -- 172.21.15.115:0/2067013128 wait complete. 2024-05-09T05:53:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.416+0000 7f523cbd1740 1 -- start start 2024-05-09T05:53:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.416+0000 7f523cbd1740 1 Processor -- start 2024-05-09T05:53:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 0x55a87b6b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e94d0 0x55a87b6b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f523cbd1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55a87b6ba910 con 0x55a87b2e94d0 2024-05-09T05:53:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f523cbd1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a87b5999c0 con 0x55a87b30a1b0 2024-05-09T05:53:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f523cbd1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a87b6baa90 con 0x55a87b2e12f0 2024-05-09T05:53:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f523cbd1740 10 client.0 _ll_register_callbacks cb 0x55a87b5c6020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f522ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f522ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:55980/0 (socket says 172.21.15.115:55980) 2024-05-09T05:53:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.417+0000 7f522ffff640 1 -- 172.21.15.115:0/16643506 learned_addr learned my addr 172.21.15.115:0/16643506 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.418+0000 7f522f7fe640 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 0x55a87b6b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.418+0000 7f522f7fe640 1 -- 172.21.15.115:0/16643506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 msgr2=0x55a87b6bde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.418+0000 7f522f7fe640 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.419+0000 7f522f7fe640 1 -- 172.21.15.115:0/16643506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e94d0 msgr2=0x55a87b6b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.419+0000 7f522f7fe640 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e94d0 0x55a87b6b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.419+0000 7f522f7fe640 1 -- 172.21.15.115:0/16643506 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87b5c01f0 con 0x55a87b2e12f0 2024-05-09T05:53:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.420+0000 7f522f7fe640 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 0x55a87b6b7ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52240deef0 tx=0x7f52240a4340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c973c57df989ef8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.420+0000 7f52137f6640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:53:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.421+0000 7f52137f6640 1 -- 172.21.15.115:0/16643506 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522401e070 con 0x55a87b2e12f0 2024-05-09T05:53:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.421+0000 7f523cbd1740 1 -- 172.21.15.115:0/16643506 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52200042f0 con 0x55a87b2e12f0 2024-05-09T05:53:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.421+0000 7f52137f6640 1 -- 172.21.15.115:0/16643506 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9526+0+0 (secure 0 0 0) 0x7f52240df060 con 0x55a87b2e12f0 2024-05-09T05:53:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.424+0000 7f52137f6640 1 -- 172.21.15.115:0/16643506 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f52240df260 con 0x55a87b2e12f0 2024-05-09T05:53:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.426+0000 7f523cbd1740 10 client.28081 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.426+0000 7f523cbd1740 1 -- 172.21.15.115:0/16643506 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55a87b375790 con 0x55a87b2e12f0 2024-05-09T05:53:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.427+0000 7f52137f6640 1 -- 172.21.15.115:0/16643506 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52200042f0 con 0x55a87b2e12f0 2024-05-09T05:53:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.427+0000 7f52137f6640 1 -- 172.21.15.115:0/16643506 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f52240cb0b0 con 0x55a87b2e12f0 2024-05-09T05:53:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.427+0000 7f523cbd1740 10 client.28081 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.427+0000 7f523cbd1740 10 client.28081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.427+0000 7f523cbd1740 20 client.28081 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.427+0000 7f523cbd1740 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a87b2c8d80 0x55a87b6ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.427+0000 7f523cbd1740 4 client.28081 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:53:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.427+0000 7f523cbd1740 1 -- 172.21.15.115:0/16643506 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a87b6203d0 con 0x55a87b2c8d80 2024-05-09T05:53:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.429+0000 7f522effd640 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a87b2c8d80 0x55a87b6ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.430+0000 7f522effd640 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a87b2c8d80 0x55a87b6ca620 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e597ca870deae7 server_cookie=7f958603e4ce9445 in_seq=0 out_seq=0 2024-05-09T05:53:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.430+0000 7f52137f6640 10 client.28081 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.431+0000 7f52137f6640 1 -- 172.21.15.115:0/16643506 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a87b6203d0 con 0x55a87b2c8d80 2024-05-09T05:53:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.431+0000 7f52137f6640 10 client.28081 handle_command_reply: tid=0 2024-05-09T05:53:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.431+0000 7f523cbd1740 1 client.28081 shutdown 2024-05-09T05:53:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.431+0000 7f523cbd1740 1 -- 172.21.15.115:0/16643506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 msgr2=0x55a87b6b7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.431+0000 7f523cbd1740 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 0x55a87b6b7ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52240deef0 tx=0x7f52240a4340 comp rx=0 tx=0).stop 2024-05-09T05:53:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.432+0000 7f523cbd1740 1 -- 172.21.15.115:0/16643506 shutdown_connections 2024-05-09T05:53:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.432+0000 7f523cbd1740 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.432+0000 7f523cbd1740 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e94d0 0x55a87b6b83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.432+0000 7f523cbd1740 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a87b2c8d80 0x55a87b6ca620 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.432+0000 7f523cbd1740 1 --2- 172.21.15.115:0/16643506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e12f0 0x55a87b6b7ea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.432+0000 7f523cbd1740 1 -- 172.21.15.115:0/16643506 >> 172.21.15.115:0/16643506 conn(0x55a87b2fe300 msgr2=0x55a87b6a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.432+0000 7f52137f6640 0 client.28081 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.432+0000 7f523cbd1740 1 -- 172.21.15.115:0/16643506 shutdown_connections 2024-05-09T05:53:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.432+0000 7f523cbd1740 1 -- 172.21.15.115:0/16643506 wait complete. 2024-05-09T05:53:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.436+0000 7f523cbd1740 20 client.28081 trim_cache size 0 max 16384 2024-05-09T05:53:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.486+0000 7f523cbd1740 1 Processor -- start 2024-05-09T05:53:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.486+0000 7f523cbd1740 1 -- start start 2024-05-09T05:53:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.486+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e12f0 0x55a87b6a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 0x55a87b6bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f522f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e12f0 0x55a87b6a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f522f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e12f0 0x55a87b6a8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:55984/0 (socket says 172.21.15.115:55984) 2024-05-09T05:53:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 0x55a87b6be2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f522effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 0x55a87b6bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f522effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 0x55a87b6bdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:51830/0 (socket says 172.21.15.115:51830) 2024-05-09T05:53:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f522f7fe640 1 -- 172.21.15.115:0/959429131 learned_addr learned my addr 172.21.15.115:0/959429131 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f523cbd1740 1 -- 172.21.15.115:0/959429131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f521c0100d0 con 0x55a87b30a1b0 2024-05-09T05:53:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f523cbd1740 1 -- 172.21.15.115:0/959429131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f521c011c40 con 0x55a87b2e12f0 2024-05-09T05:53:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.487+0000 7f523cbd1740 1 -- 172.21.15.115:0/959429131 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f521c0107a0 con 0x55a87b2e94d0 2024-05-09T05:53:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.488+0000 7f522ffff640 1 --2- 172.21.15.115:0/959429131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 0x55a87b6be2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.490+0000 7f522ffff640 1 -- 172.21.15.115:0/959429131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 msgr2=0x55a87b6bdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.490+0000 7f522ffff640 1 --2- 172.21.15.115:0/959429131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 0x55a87b6bdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.490+0000 7f522ffff640 1 -- 172.21.15.115:0/959429131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e12f0 msgr2=0x55a87b6a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.490+0000 7f522ffff640 1 --2- 172.21.15.115:0/959429131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e12f0 0x55a87b6a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.490+0000 7f522ffff640 1 -- 172.21.15.115:0/959429131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87b3ec5f0 con 0x55a87b30a1b0 2024-05-09T05:53:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.509+0000 7f522ffff640 1 --2- 172.21.15.115:0/959429131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 0x55a87b6be2d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f522800ac20 tx=0x7f5228012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b9740375fc0ecc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.509+0000 7f52157fa640 1 -- 172.21.15.115:0/959429131 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228016070 con 0x55a87b30a1b0 2024-05-09T05:53:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.510+0000 7f52157fa640 1 -- 172.21.15.115:0/959429131 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f522801ee20 con 0x55a87b30a1b0 2024-05-09T05:53:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.511+0000 7f52157fa640 1 -- 172.21.15.115:0/959429131 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228005740 con 0x55a87b30a1b0 2024-05-09T05:53:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.511+0000 7f523cbd1740 1 -- 172.21.15.115:0/959429131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 msgr2=0x55a87b6be2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.511+0000 7f523cbd1740 1 --2- 172.21.15.115:0/959429131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 0x55a87b6be2d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f522800ac20 tx=0x7f5228012000 comp rx=0 tx=0).stop 2024-05-09T05:53:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.511+0000 7f523cbd1740 1 -- 172.21.15.115:0/959429131 shutdown_connections 2024-05-09T05:53:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.511+0000 7f523cbd1740 1 --2- 172.21.15.115:0/959429131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b30a1b0 0x55a87b6be2d0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.511+0000 7f523cbd1740 1 --2- 172.21.15.115:0/959429131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 0x55a87b6bdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.511+0000 7f523cbd1740 1 --2- 172.21.15.115:0/959429131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b2e12f0 0x55a87b6a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.511+0000 7f523cbd1740 1 -- 172.21.15.115:0/959429131 >> 172.21.15.115:0/959429131 conn(0x55a87b2fe300 msgr2=0x55a87b6a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.517+0000 7f523cbd1740 1 -- 172.21.15.115:0/959429131 shutdown_connections 2024-05-09T05:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.518+0000 7f523cbd1740 1 -- 172.21.15.115:0/959429131 wait complete. 2024-05-09T05:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.518+0000 7f523cbd1740 1 -- start start 2024-05-09T05:53:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.518+0000 7f523cbd1740 1 Processor -- start 2024-05-09T05:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.519+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e12f0 0x55a87b6c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.519+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 0x55a87b6bede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.519+0000 7f523cbd1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.519+0000 7f523cbd1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f521c011570 con 0x55a87b2e12f0 2024-05-09T05:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.519+0000 7f523cbd1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f521c010ea0 con 0x55a87b30a1b0 2024-05-09T05:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.519+0000 7f523cbd1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f521c00e0d0 con 0x55a87b2e94d0 2024-05-09T05:53:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.519+0000 7f523cbd1740 10 client.0 _ll_register_callbacks cb 0x7f52240a4940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.521+0000 7f522ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.521+0000 7f522ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:55992/0 (socket says 172.21.15.115:55992) 2024-05-09T05:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.521+0000 7f522ffff640 1 -- 172.21.15.115:0/4025520657 learned_addr learned my addr 172.21.15.115:0/4025520657 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.521+0000 7f522effd640 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 0x55a87b6bede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.522+0000 7f522f7fe640 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e12f0 0x55a87b6c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.523+0000 7f522f7fe640 1 -- 172.21.15.115:0/4025520657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 msgr2=0x55a87b6bede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.523+0000 7f522f7fe640 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 0x55a87b6bede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.523+0000 7f522f7fe640 1 -- 172.21.15.115:0/4025520657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 msgr2=0x55a87b6bf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.523+0000 7f522f7fe640 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.523+0000 7f522f7fe640 1 -- 172.21.15.115:0/4025520657 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87b5d6ed0 con 0x55a87b2e12f0 2024-05-09T05:53:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.524+0000 7f522f7fe640 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e12f0 0x55a87b6c4aa0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f522400be60 tx=0x7f52240a88c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d1f94b6945150e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.524+0000 7f52137f6640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:53:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.525+0000 7f52137f6640 1 -- 172.21.15.115:0/4025520657 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52240c8070 con 0x55a87b2e12f0 2024-05-09T05:53:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.525+0000 7f523cbd1740 1 -- 172.21.15.115:0/4025520657 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522801ee20 con 0x55a87b2e12f0 2024-05-09T05:53:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.525+0000 7f52137f6640 1 -- 172.21.15.115:0/4025520657 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9526+0+0 (secure 0 0 0) 0x7f5224010610 con 0x55a87b2e12f0 2024-05-09T05:53:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.526+0000 7f52137f6640 1 -- 172.21.15.115:0/4025520657 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f522408d050 con 0x55a87b2e12f0 2024-05-09T05:53:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.528+0000 7f523cbd1740 10 client.18762 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.528+0000 7f523cbd1740 1 -- 172.21.15.115:0/4025520657 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55a87b731860 con 0x55a87b2e12f0 2024-05-09T05:53:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.530+0000 7f52137f6640 1 -- 172.21.15.115:0/4025520657 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f522801ee20 con 0x55a87b2e12f0 2024-05-09T05:53:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.530+0000 7f52137f6640 1 -- 172.21.15.115:0/4025520657 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f52240a43f0 con 0x55a87b2e12f0 2024-05-09T05:53:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.532+0000 7f523cbd1740 10 client.18762 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.532+0000 7f523cbd1740 10 client.18762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.532+0000 7f523cbd1740 20 client.18762 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.532+0000 7f523cbd1740 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a87b29e1c0 0x55a87b733e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.532+0000 7f523cbd1740 4 client.18762 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:53:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.532+0000 7f523cbd1740 1 -- 172.21.15.115:0/4025520657 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a87b61a240 con 0x55a87b29e1c0 2024-05-09T05:53:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.532+0000 7f522effd640 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a87b29e1c0 0x55a87b733e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.533+0000 7f522effd640 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a87b29e1c0 0x55a87b733e60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=861fecf41b7dafa7 server_cookie=d1f432212640c863 in_seq=0 out_seq=0 2024-05-09T05:53:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.534+0000 7f52137f6640 10 client.18762 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f52137f6640 1 -- 172.21.15.115:0/4025520657 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a87b61a240 con 0x55a87b29e1c0 2024-05-09T05:53:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f52137f6640 10 client.18762 handle_command_reply: tid=0 2024-05-09T05:53:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 client.18762 shutdown 2024-05-09T05:53:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 -- 172.21.15.115:0/4025520657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e12f0 msgr2=0x55a87b6c4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e12f0 0x55a87b6c4aa0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f522400be60 tx=0x7f52240a88c0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 -- 172.21.15.115:0/4025520657 shutdown_connections 2024-05-09T05:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a87b30a1b0 0x55a87b6bf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55a87b29e1c0 0x55a87b733e60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a87b2e94d0 0x55a87b6bede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 --2- 172.21.15.115:0/4025520657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a87b2e12f0 0x55a87b6c4aa0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 -- 172.21.15.115:0/4025520657 >> 172.21.15.115:0/4025520657 conn(0x55a87b2fe300 msgr2=0x55a87b6a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f52137f6640 0 client.18762 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 -- 172.21.15.115:0/4025520657 shutdown_connections 2024-05-09T05:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.689+0000 7f523cbd1740 1 -- 172.21.15.115:0/4025520657 wait complete. 2024-05-09T05:53:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.695+0000 7f523cbd1740 20 client.18762 trim_cache size 0 max 16384 2024-05-09T05:53:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:53:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:53:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:53:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.702+0000 7f5234abd640 1 -- 172.21.15.115:0/3301960891 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f52180b2aa0 msgr2=0x7f52180b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.702+0000 7f5234abd640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f52180b2aa0 0x7f52180b4fb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5228005c40 tx=0x7f5228005bd0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.702+0000 7f5234abd640 1 -- 172.21.15.115:0/3301960891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f523010a210 msgr2=0x7f52301a9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.702+0000 7f5234abd640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f523010a210 0x7f52301a9430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f522400b4a0 tx=0x7f522400b960 comp rx=0 tx=0).stop 2024-05-09T05:53:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.704+0000 7f5234abd640 1 -- 172.21.15.115:0/3301960891 shutdown_connections 2024-05-09T05:53:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.705+0000 7f5234abd640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f52180b2aa0 0x7f52180b4fb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.705+0000 7f5234abd640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52301103f0 0x7f52301adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.705+0000 7f5234abd640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f523010fa10 0x7f52301a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.705+0000 7f5234abd640 1 --2- 172.21.15.115:0/3301960891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f523010a210 0x7f52301a9430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.706+0000 7f5234abd640 1 -- 172.21.15.115:0/3301960891 >> 172.21.15.115:0/3301960891 conn(0x7f523006f810 msgr2=0x7f523010d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.709+0000 7f5234abd640 1 -- 172.21.15.115:0/3301960891 shutdown_connections 2024-05-09T05:53:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:30.709+0000 7f5234abd640 1 -- 172.21.15.115:0/3301960891 wait complete. 2024-05-09T05:53:31.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:31 smithi195 ceph-mon[33844]: pgmap v482: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-05-09T05:53:31.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:31 smithi115 ceph-mon[30349]: pgmap v482: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-05-09T05:53:31.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:31 smithi161 ceph-mon[33805]: pgmap v482: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-05-09T05:53:33.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:33 smithi195 ceph-mon[33844]: pgmap v483: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.81k op/s 2024-05-09T05:53:33.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:33 smithi115 ceph-mon[30349]: pgmap v483: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.81k op/s 2024-05-09T05:53:33.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:33 smithi161 ceph-mon[33805]: pgmap v483: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.81k op/s 2024-05-09T05:53:35.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:35 smithi195 ceph-mon[33844]: pgmap v484: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 21 MiB/s wr, 5.25k op/s 2024-05-09T05:53:35.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:35 smithi115 ceph-mon[30349]: pgmap v484: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 21 MiB/s wr, 5.25k op/s 2024-05-09T05:53:35.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:35 smithi161 ceph-mon[33805]: pgmap v484: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 21 MiB/s wr, 5.25k op/s 2024-05-09T05:53:37.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:37 smithi195 ceph-mon[33844]: pgmap v485: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-05-09T05:53:37.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:37 smithi115 ceph-mon[30349]: pgmap v485: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-05-09T05:53:37.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:37 smithi161 ceph-mon[33805]: pgmap v485: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2024-05-09T05:53:39.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:39 smithi195 ceph-mon[33844]: pgmap v486: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 20 MiB/s wr, 4.98k op/s 2024-05-09T05:53:39.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:39 smithi115 ceph-mon[30349]: pgmap v486: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 20 MiB/s wr, 4.98k op/s 2024-05-09T05:53:39.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:39 smithi161 ceph-mon[33805]: pgmap v486: 129 pgs: 6 active+clean+snaptrim, 119 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 20 MiB/s wr, 4.98k op/s 2024-05-09T05:53:41.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:41 smithi195 ceph-mon[33844]: pgmap v487: 129 pgs: 5 active+clean+snaptrim, 120 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2024-05-09T05:53:41.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:41 smithi115 ceph-mon[30349]: pgmap v487: 129 pgs: 5 active+clean+snaptrim, 120 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2024-05-09T05:53:41.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:41 smithi161 ceph-mon[33805]: pgmap v487: 129 pgs: 5 active+clean+snaptrim, 120 active+clean, 4 active+clean+snaptrim_wait; 14 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2024-05-09T05:53:43.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:43 smithi195 ceph-mon[33844]: pgmap v488: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-05-09T05:53:43.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:43 smithi115 ceph-mon[30349]: pgmap v488: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-05-09T05:53:43.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:43 smithi161 ceph-mon[33805]: pgmap v488: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-05-09T05:53:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.967+0000 7fb4fabce640 1 -- 172.21.15.115:0/970381049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f4108ba0 msgr2=0x7fb4f4102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.967+0000 7fb4fabce640 1 --2- 172.21.15.115:0/970381049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f4108ba0 0x7fb4f4102740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009900 tx=0x7fb4e802a440 comp rx=0 tx=0).stop 2024-05-09T05:53:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.968+0000 7fb4fabce640 1 -- 172.21.15.115:0/970381049 shutdown_connections 2024-05-09T05:53:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.968+0000 7fb4fabce640 1 --2- 172.21.15.115:0/970381049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f41032f0 0x7fb4f410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.968+0000 7fb4fabce640 1 --2- 172.21.15.115:0/970381049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f4109040 0x7fb4f4102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.968+0000 7fb4fabce640 1 --2- 172.21.15.115:0/970381049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f4108ba0 0x7fb4f4102740 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.968+0000 7fb4fabce640 1 -- 172.21.15.115:0/970381049 >> 172.21.15.115:0/970381049 conn(0x7fb4f40782e0 msgr2=0x7fb4f4078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.973+0000 7fb4fabce640 1 -- 172.21.15.115:0/970381049 shutdown_connections 2024-05-09T05:53:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.980+0000 7fb4fabce640 1 -- 172.21.15.115:0/970381049 wait complete. 2024-05-09T05:53:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.981+0000 7fb4fabce640 1 Processor -- start 2024-05-09T05:53:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.981+0000 7fb4fabce640 1 -- start start 2024-05-09T05:53:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.982+0000 7fb4fabce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f41032f0 0x7fb4f410f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.982+0000 7fb4fabce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f4108ba0 0x7fb4f410d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.982+0000 7fb4fabce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f4109040 0x7fb4f410dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.982+0000 7fb4fabce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4112d90 con 0x7fb4f4108ba0 2024-05-09T05:53:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.982+0000 7fb4fabce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4112c10 con 0x7fb4f4109040 2024-05-09T05:53:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.982+0000 7fb4fabce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4112f10 con 0x7fb4f41032f0 2024-05-09T05:53:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.983+0000 7fb4f916b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f4109040 0x7fb4f410dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.983+0000 7fb4f916b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f4109040 0x7fb4f410dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:39896/0 (socket says 172.21.15.115:39896) 2024-05-09T05:53:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.983+0000 7fb4f916b640 1 -- 172.21.15.115:0/3062298817 learned_addr learned my addr 172.21.15.115:0/3062298817 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.986+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f4108ba0 0x7fb4f410d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.986+0000 7fb4f916b640 1 -- 172.21.15.115:0/3062298817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f41032f0 msgr2=0x7fb4f410f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.986+0000 7fb4f916b640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f41032f0 0x7fb4f410f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.986+0000 7fb4f916b640 1 -- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f4108ba0 msgr2=0x7fb4f410d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.987+0000 7fb4f916b640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f4108ba0 0x7fb4f410d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.987+0000 7fb4f916b640 1 -- 172.21.15.115:0/3062298817 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f410e570 con 0x7fb4f4109040 2024-05-09T05:53:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.987+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f4108ba0 0x7fb4f410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:53:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.988+0000 7fb4f916b640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f4109040 0x7fb4f410dd80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00b3f0 tx=0x7fb4ec00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.991+0000 7fb4f1ffb640 1 -- 172.21.15.115:0/3062298817 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec015020 con 0x7fb4f4109040 2024-05-09T05:53:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.991+0000 7fb4f1ffb640 1 -- 172.21.15.115:0/3062298817 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4ec011cc0 con 0x7fb4f4109040 2024-05-09T05:53:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.991+0000 7fb4fabce640 1 -- 172.21.15.115:0/3062298817 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f419eba0 con 0x7fb4f4109040 2024-05-09T05:53:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.991+0000 7fb4f1ffb640 1 -- 172.21.15.115:0/3062298817 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec010ce0 con 0x7fb4f4109040 2024-05-09T05:53:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.991+0000 7fb4fabce640 1 -- 172.21.15.115:0/3062298817 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f419f0a0 con 0x7fb4f4109040 2024-05-09T05:53:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:43.999+0000 7fb4f1ffb640 1 -- 172.21.15.115:0/3062298817 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fb4ec018020 con 0x7fb4f4109040 2024-05-09T05:53:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.001+0000 7fb4f1ffb640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb4dc0b2b60 0x7fb4dc0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.001+0000 7fb4f1ffb640 1 -- 172.21.15.115:0/3062298817 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fb4ec0e2ff0 con 0x7fb4f4109040 2024-05-09T05:53:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.003+0000 7fb501a80740 1 Processor -- start 2024-05-09T05:53:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.005+0000 7fb4f896a640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb4dc0b2b60 0x7fb4dc0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.013+0000 7fb501a80740 1 -- start start 2024-05-09T05:53:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb501a80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 0x55903ed2d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb501a80740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9612f0 0x55903ed2dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb501a80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed2e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb501a80740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55903ec927c0 con 0x55903e9612f0 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb501a80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55903ec2e0e0 con 0x55903e9694d0 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb501a80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55903ec8e780 con 0x55903e98a1b0 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb4f3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9612f0 0x55903ed2dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb4f3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9612f0 0x55903ed2dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56238/0 (socket says 172.21.15.115:56238) 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb4f3fff640 1 -- 172.21.15.115:0/2460038778 learned_addr learned my addr 172.21.15.115:0/2460038778 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.014+0000 7fb4f916b640 1 --2- 172.21.15.115:0/2460038778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.015+0000 7fb4f3fff640 1 -- 172.21.15.115:0/2460038778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 msgr2=0x55903ed2d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.015+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/2460038778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 0x55903ed2d680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.015+0000 7fb4f3fff640 1 -- 172.21.15.115:0/2460038778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 msgr2=0x55903ed2e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.015+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/2460038778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed2e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.015+0000 7fb4f3fff640 1 -- 172.21.15.115:0/2460038778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55903ec577c0 con 0x55903e9612f0 2024-05-09T05:53:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.015+0000 7fb4f916b640 1 --2- 172.21.15.115:0/2460038778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed2e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:53:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.015+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/2460038778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9612f0 0x55903ed2dbc0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007ec0 tx=0x7fb4e400c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cdd4b5ff5b06e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.023+0000 7fb4f896a640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb4dc0b2b60 0x7fb4dc0b5070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80096c0 tx=0x7fb4e80325f0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.024+0000 7fb4daffd640 1 -- 172.21.15.115:0/2460038778 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4014070 con 0x55903e9612f0 2024-05-09T05:53:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.024+0000 7fb4daffd640 1 -- 172.21.15.115:0/2460038778 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4e400fce0 con 0x55903e9612f0 2024-05-09T05:53:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.024+0000 7fb4daffd640 1 -- 172.21.15.115:0/2460038778 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400ebc0 con 0x55903e9612f0 2024-05-09T05:53:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.027+0000 7fb501a80740 1 -- 172.21.15.115:0/2460038778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9612f0 msgr2=0x55903ed2dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.027+0000 7fb501a80740 1 --2- 172.21.15.115:0/2460038778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9612f0 0x55903ed2dbc0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007ec0 tx=0x7fb4e400c690 comp rx=0 tx=0).stop 2024-05-09T05:53:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.028+0000 7fb501a80740 1 -- 172.21.15.115:0/2460038778 shutdown_connections 2024-05-09T05:53:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.028+0000 7fb501a80740 1 --2- 172.21.15.115:0/2460038778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed2e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.028+0000 7fb501a80740 1 --2- 172.21.15.115:0/2460038778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9612f0 0x55903ed2dbc0 secure :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007ec0 tx=0x7fb4e400c690 comp rx=0 tx=0).stop 2024-05-09T05:53:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.028+0000 7fb501a80740 1 --2- 172.21.15.115:0/2460038778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 0x55903ed2d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.028+0000 7fb501a80740 1 -- 172.21.15.115:0/2460038778 >> 172.21.15.115:0/2460038778 conn(0x55903e97e300 msgr2=0x55903ed29e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.030+0000 7fb501a80740 1 -- 172.21.15.115:0/2460038778 shutdown_connections 2024-05-09T05:53:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.035+0000 7fb501a80740 1 -- 172.21.15.115:0/2460038778 wait complete. 2024-05-09T05:53:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.036+0000 7fb501a80740 1 -- start start 2024-05-09T05:53:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.037+0000 7fb501a80740 1 Processor -- start 2024-05-09T05:53:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb501a80740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9694d0 0x55903ed39700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb501a80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 0x55903ed39c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb501a80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e948d80 0x55903ed37cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb501a80740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55903ed3a7b0 con 0x55903e9694d0 2024-05-09T05:53:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb501a80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55903ec199c0 con 0x55903e948d80 2024-05-09T05:53:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb501a80740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55903ed3a930 con 0x55903e98a1b0 2024-05-09T05:53:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb501a80740 10 client.0 _ll_register_callbacks cb 0x55903ec46020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb4f3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 0x55903ed39c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb4f3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 0x55903ed39c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33954/0 (socket says 172.21.15.115:33954) 2024-05-09T05:53:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.039+0000 7fb4f3fff640 1 -- 172.21.15.115:0/4046914277 learned_addr learned my addr 172.21.15.115:0/4046914277 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.040+0000 7fb4f3fff640 1 -- 172.21.15.115:0/4046914277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e948d80 msgr2=0x55903ed37cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:53:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.040+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e948d80 0x55903ed37cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.040+0000 7fb4f3fff640 1 -- 172.21.15.115:0/4046914277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9694d0 msgr2=0x55903ed39700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.040+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9694d0 0x55903ed39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.040+0000 7fb4f3fff640 1 -- 172.21.15.115:0/4046914277 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55903ec401f0 con 0x55903e98a1b0 2024-05-09T05:53:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.042+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 0x55903ed39c40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e7e0 tx=0x7fb4e400ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b5ecca1ec2077be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.051+0000 7fb4d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:53:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.051+0000 7fb4d3fff640 1 -- 172.21.15.115:0/4046914277 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4014040 con 0x55903e98a1b0 2024-05-09T05:53:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.053+0000 7fb4d3fff640 1 -- 172.21.15.115:0/4046914277 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fb4e400eb50 con 0x55903e98a1b0 2024-05-09T05:53:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.054+0000 7fb501a80740 1 -- 172.21.15.115:0/4046914277 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e400fce0 con 0x55903e98a1b0 2024-05-09T05:53:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.060+0000 7fb4d3fff640 1 -- 172.21.15.115:0/4046914277 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fb4e4007ae0 con 0x55903e98a1b0 2024-05-09T05:53:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.061+0000 7fb501a80740 10 client.28093 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.061+0000 7fb501a80740 1 -- 172.21.15.115:0/4046914277 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55903e9f5790 con 0x55903e98a1b0 2024-05-09T05:53:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.065+0000 7fb4d3fff640 1 -- 172.21.15.115:0/4046914277 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4e400fce0 con 0x55903e98a1b0 2024-05-09T05:53:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.066+0000 7fb4d3fff640 1 -- 172.21.15.115:0/4046914277 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7fb4e40199f0 con 0x55903e98a1b0 2024-05-09T05:53:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.067+0000 7fb501a80740 10 client.28093 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.067+0000 7fb501a80740 10 client.28093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.067+0000 7fb501a80740 20 client.28093 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.067+0000 7fb501a80740 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55903e91e1c0 0x55903ed4a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.067+0000 7fb501a80740 4 client.28093 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:53:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.067+0000 7fb501a80740 1 -- 172.21.15.115:0/4046914277 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55903eca03d0 con 0x55903e91e1c0 2024-05-09T05:53:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.071+0000 7fb4f896a640 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55903e91e1c0 0x55903ed4a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.083+0000 7fb4f896a640 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55903e91e1c0 0x55903ed4a3e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e263a9fa70eedf5 server_cookie=817e7673252d669d in_seq=0 out_seq=0 2024-05-09T05:53:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.083+0000 7fb4d3fff640 10 client.28093 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.084+0000 7fb4d3fff640 1 -- 172.21.15.115:0/4046914277 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55903eca03d0 con 0x55903e91e1c0 2024-05-09T05:53:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.084+0000 7fb4d3fff640 10 client.28093 handle_command_reply: tid=0 2024-05-09T05:53:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.084+0000 7fb501a80740 1 client.28093 shutdown 2024-05-09T05:53:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.084+0000 7fb501a80740 1 -- 172.21.15.115:0/4046914277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 msgr2=0x55903ed39c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.084+0000 7fb501a80740 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 0x55903ed39c40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e7e0 tx=0x7fb4e400ff70 comp rx=0 tx=0).stop 2024-05-09T05:53:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.090+0000 7fb501a80740 1 -- 172.21.15.115:0/4046914277 shutdown_connections 2024-05-09T05:53:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.090+0000 7fb501a80740 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e948d80 0x55903ed37cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.090+0000 7fb501a80740 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55903e91e1c0 0x55903ed4a3e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.090+0000 7fb501a80740 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e98a1b0 0x55903ed39c40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.090+0000 7fb501a80740 1 --2- 172.21.15.115:0/4046914277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e9694d0 0x55903ed39700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.090+0000 7fb501a80740 1 -- 172.21.15.115:0/4046914277 >> 172.21.15.115:0/4046914277 conn(0x55903e97e300 msgr2=0x55903ed2f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.091+0000 7fb4d3fff640 0 client.28093 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.091+0000 7fb501a80740 1 -- 172.21.15.115:0/4046914277 shutdown_connections 2024-05-09T05:53:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.092+0000 7fb501a80740 1 -- 172.21.15.115:0/4046914277 wait complete. 2024-05-09T05:53:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.092+0000 7fb501a80740 20 client.28093 trim_cache size 0 max 16384 2024-05-09T05:53:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.156+0000 7fb501a80740 1 Processor -- start 2024-05-09T05:53:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.159+0000 7fb501a80740 1 -- start start 2024-05-09T05:53:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.160+0000 7fb501a80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed38240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.160+0000 7fb501a80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.160+0000 7fb4f896a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed38240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.160+0000 7fb4f896a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed38240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33958/0 (socket says 172.21.15.115:33958) 2024-05-09T05:53:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.161+0000 7fb4f896a640 1 -- 172.21.15.115:0/216898168 learned_addr learned my addr 172.21.15.115:0/216898168 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.161+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.161+0000 7fb501a80740 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e98a1b0 0x55903ed50120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.161+0000 7fb501a80740 1 -- 172.21.15.115:0/216898168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e000fdf0 con 0x55903e98a1b0 2024-05-09T05:53:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.161+0000 7fb501a80740 1 -- 172.21.15.115:0/216898168 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0011960 con 0x55903e9694d0 2024-05-09T05:53:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.161+0000 7fb501a80740 1 -- 172.21.15.115:0/216898168 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e00104c0 con 0x55903e948d80 2024-05-09T05:53:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.164+0000 7fb4f916b640 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e98a1b0 0x55903ed50120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.164+0000 7fb4f3fff640 1 -- 172.21.15.115:0/216898168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 msgr2=0x55903ed38240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.164+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed38240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.164+0000 7fb4f3fff640 1 -- 172.21.15.115:0/216898168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e98a1b0 msgr2=0x55903ed50120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.164+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e98a1b0 0x55903ed50120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.164+0000 7fb4f3fff640 1 -- 172.21.15.115:0/216898168 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55903ea6c5f0 con 0x55903e9694d0 2024-05-09T05:53:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.165+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed4fbe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4019df0 tx=0x7fb4e400c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc612ed6af25994 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.168+0000 7fb4da7fc640 1 -- 172.21.15.115:0/216898168 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e403a070 con 0x55903e9694d0 2024-05-09T05:53:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.168+0000 7fb4da7fc640 1 -- 172.21.15.115:0/216898168 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4e40073e0 con 0x55903e9694d0 2024-05-09T05:53:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.168+0000 7fb4da7fc640 1 -- 172.21.15.115:0/216898168 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4014040 con 0x55903e9694d0 2024-05-09T05:53:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.169+0000 7fb501a80740 1 -- 172.21.15.115:0/216898168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 msgr2=0x55903ed4fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.169+0000 7fb501a80740 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed4fbe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4019df0 tx=0x7fb4e400c690 comp rx=0 tx=0).stop 2024-05-09T05:53:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.170+0000 7fb501a80740 1 -- 172.21.15.115:0/216898168 shutdown_connections 2024-05-09T05:53:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.170+0000 7fb501a80740 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e98a1b0 0x55903ed50120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.170+0000 7fb501a80740 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed4fbe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.170+0000 7fb501a80740 1 --2- 172.21.15.115:0/216898168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed38240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.170+0000 7fb501a80740 1 -- 172.21.15.115:0/216898168 >> 172.21.15.115:0/216898168 conn(0x55903e97e300 msgr2=0x55903ed365c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.175+0000 7fb501a80740 1 -- 172.21.15.115:0/216898168 shutdown_connections 2024-05-09T05:53:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.175+0000 7fb501a80740 1 -- 172.21.15.115:0/216898168 wait complete. 2024-05-09T05:53:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.176+0000 7fb501a80740 1 -- start start 2024-05-09T05:53:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.176+0000 7fb501a80740 1 Processor -- start 2024-05-09T05:53:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.177+0000 7fb501a80740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed3ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.177+0000 7fb501a80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed3abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.177+0000 7fb4f896a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed3ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.177+0000 7fb4f896a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed3ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33962/0 (socket says 172.21.15.115:33962) 2024-05-09T05:53:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.178+0000 7fb4f896a640 1 -- 172.21.15.115:0/1502349690 learned_addr learned my addr 172.21.15.115:0/1502349690 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.178+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed3abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.178+0000 7fb501a80740 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e98a1b0 0x55903ed3b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.178+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0011290 con 0x55903e98a1b0 2024-05-09T05:53:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.178+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e0010bc0 con 0x55903e9694d0 2024-05-09T05:53:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.178+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e000de80 con 0x55903e948d80 2024-05-09T05:53:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.178+0000 7fb4f896a640 1 -- 172.21.15.115:0/1502349690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 msgr2=0x55903ed3abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.178+0000 7fb4f896a640 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed3abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.179+0000 7fb4f896a640 1 -- 172.21.15.115:0/1502349690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e98a1b0 msgr2=0x55903ed3b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.179+0000 7fb4f896a640 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e98a1b0 0x55903ed3b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.179+0000 7fb4f896a640 1 -- 172.21.15.115:0/1502349690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55903ec56ed0 con 0x55903e948d80 2024-05-09T05:53:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.179+0000 7fb501a80740 10 client.0 _ll_register_callbacks cb 0x7fb4e4007d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.179+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e40073e0 con 0x55903e948d80 2024-05-09T05:53:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.179+0000 7fb4f896a640 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed3ff00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009850 tx=0x7fb4e8032930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae5d8c0e45b86617 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.179+0000 7fb4d3fff640 10 client.28099 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:53:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.180+0000 7fb4d3fff640 1 -- 172.21.15.115:0/1502349690 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8035040 con 0x55903e948d80 2024-05-09T05:53:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.180+0000 7fb4d3fff640 1 -- 172.21.15.115:0/1502349690 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fb4e804c600 con 0x55903e948d80 2024-05-09T05:53:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.180+0000 7fb4d3fff640 1 -- 172.21.15.115:0/1502349690 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fb4e804c800 con 0x55903e948d80 2024-05-09T05:53:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.181+0000 7fb501a80740 10 client.28099 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.181+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({fsmap=62}) -- 0x55903edb10f0 con 0x55903e948d80 2024-05-09T05:53:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.182+0000 7fb4d3fff640 1 -- 172.21.15.115:0/1502349690 <== mon.1 v2:172.21.15.195:3300/0 4 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7fb4e805f420 con 0x55903e948d80 2024-05-09T05:53:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.182+0000 7fb501a80740 10 client.28099 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.182+0000 7fb501a80740 10 client.28099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.182+0000 7fb501a80740 20 client.28099 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.182+0000 7fb501a80740 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55903edb18c0 0x55903edb3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.182+0000 7fb501a80740 4 client.28099 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:53:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.182+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55903ec9a240 con 0x55903edb18c0 2024-05-09T05:53:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.183+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55903edb18c0 0x55903edb3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.188+0000 7fb4f3fff640 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55903edb18c0 0x55903edb3d60 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f357e20a50c5ea server_cookie=7c5832f313931cae in_seq=0 out_seq=0 2024-05-09T05:53:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.188+0000 7fb4d3fff640 10 client.28099 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb4d3fff640 1 -- 172.21.15.115:0/1502349690 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55903ec9a240 con 0x55903edb18c0 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb4d3fff640 10 client.28099 handle_command_reply: tid=0 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 client.28099 shutdown 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 msgr2=0x55903ed3ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed3ff00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009850 tx=0x7fb4e8032930 comp rx=0 tx=0).stop 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 shutdown_connections 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55903e98a1b0 0x55903ed3b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55903e9694d0 0x55903ed3abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55903edb18c0 0x55903edb3d60 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 --2- 172.21.15.115:0/1502349690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55903e948d80 0x55903ed3ff00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 >> 172.21.15.115:0/1502349690 conn(0x55903e97e300 msgr2=0x55903ed297f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb4d3fff640 0 client.28099 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 shutdown_connections 2024-05-09T05:53:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.326+0000 7fb501a80740 1 -- 172.21.15.115:0/1502349690 wait complete. 2024-05-09T05:53:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.331+0000 7fb501a80740 20 client.28099 trim_cache size 0 max 16384 2024-05-09T05:53:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:53:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:53:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:53:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.332+0000 7fb4fabce640 1 -- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb4dc0b2b60 msgr2=0x7fb4dc0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.333+0000 7fb4fabce640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb4dc0b2b60 0x7fb4dc0b5070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80096c0 tx=0x7fb4e80325f0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.333+0000 7fb4fabce640 1 -- 172.21.15.115:0/3062298817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f4109040 msgr2=0x7fb4f410dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.333+0000 7fb4fabce640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f4109040 0x7fb4f410dd80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00b3f0 tx=0x7fb4ec00b8b0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.333+0000 7fb4fabce640 1 -- 172.21.15.115:0/3062298817 shutdown_connections 2024-05-09T05:53:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.333+0000 7fb4fabce640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb4dc0b2b60 0x7fb4dc0b5070 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.333+0000 7fb4fabce640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f4109040 0x7fb4f410dd80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.333+0000 7fb4fabce640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4f4108ba0 0x7fb4f410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.333+0000 7fb4fabce640 1 --2- 172.21.15.115:0/3062298817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f41032f0 0x7fb4f410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.334+0000 7fb4fabce640 1 -- 172.21.15.115:0/3062298817 >> 172.21.15.115:0/3062298817 conn(0x7fb4f40782e0 msgr2=0x7fb4f40ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.341+0000 7fb4fabce640 1 -- 172.21.15.115:0/3062298817 shutdown_connections 2024-05-09T05:53:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:44.342+0000 7fb4fabce640 1 -- 172.21.15.115:0/3062298817 wait complete. 2024-05-09T05:53:45.142 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:44 smithi115 ceph-mon[30349]: osdmap e109: 12 total, 12 up, 12 in 2024-05-09T05:53:45.142 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:44 smithi115 ceph-mon[30349]: pgmap v490: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-05-09T05:53:45.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:44 smithi161 ceph-mon[33805]: osdmap e109: 12 total, 12 up, 12 in 2024-05-09T05:53:45.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:44 smithi161 ceph-mon[33805]: pgmap v490: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-05-09T05:53:45.361 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:44 smithi195 ceph-mon[33844]: osdmap e109: 12 total, 12 up, 12 in 2024-05-09T05:53:45.361 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:44 smithi195 ceph-mon[33844]: pgmap v490: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-05-09T05:53:46.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:45 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:53:46.361 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:45 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:53:46.392 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:45 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:53:47.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:46 smithi195 ceph-mon[33844]: pgmap v491: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-05-09T05:53:47.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:46 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:53:47.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:46 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:53:47.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:46 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:53:47.392 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:46 smithi115 ceph-mon[30349]: pgmap v491: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-05-09T05:53:47.392 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:46 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:53:47.392 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:46 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:53:47.392 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:46 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:53:47.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:46 smithi161 ceph-mon[33805]: pgmap v491: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-05-09T05:53:47.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:46 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:53:47.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:46 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:53:47.406 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:46 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:53:49.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:49 smithi195 ceph-mon[33844]: pgmap v492: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 4.05k op/s 2024-05-09T05:53:49.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:49 smithi115 ceph-mon[30349]: pgmap v492: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 4.05k op/s 2024-05-09T05:53:49.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:49 smithi161 ceph-mon[33805]: pgmap v492: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 4.05k op/s 2024-05-09T05:53:51.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:51 smithi195 ceph-mon[33844]: pgmap v493: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 4.26k op/s 2024-05-09T05:53:51.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:51 smithi115 ceph-mon[30349]: pgmap v493: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 4.26k op/s 2024-05-09T05:53:51.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:51 smithi161 ceph-mon[33805]: pgmap v493: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 4.26k op/s 2024-05-09T05:53:53.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:53 smithi195 ceph-mon[33844]: pgmap v494: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 4.03k op/s 2024-05-09T05:53:53.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:53 smithi115 ceph-mon[30349]: pgmap v494: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 4.03k op/s 2024-05-09T05:53:53.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:53 smithi161 ceph-mon[33805]: pgmap v494: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 4.03k op/s 2024-05-09T05:53:55.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:55 smithi195 ceph-mon[33844]: pgmap v495: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2024-05-09T05:53:55.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:55 smithi115 ceph-mon[30349]: pgmap v495: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2024-05-09T05:53:55.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:55 smithi161 ceph-mon[33805]: pgmap v495: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2024-05-09T05:53:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.592+0000 7f838890c640 1 -- 172.21.15.115:0/2214206146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410a210 msgr2=0x7f838410a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.592+0000 7f838890c640 1 --2- 172.21.15.115:0/2214206146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410a210 0x7f838410a6a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f837c009be0 tx=0x7f837c02a710 comp rx=0 tx=0).stop 2024-05-09T05:53:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.592+0000 7f838890c640 1 -- 172.21.15.115:0/2214206146 shutdown_connections 2024-05-09T05:53:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.592+0000 7f838890c640 1 --2- 172.21.15.115:0/2214206146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410a210 0x7f838410a6a0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.592+0000 7f838890c640 1 --2- 172.21.15.115:0/2214206146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83841103f0 0x7f8384109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.592+0000 7f838890c640 1 --2- 172.21.15.115:0/2214206146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f838410fa10 0x7f838410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.592+0000 7f838890c640 1 -- 172.21.15.115:0/2214206146 >> 172.21.15.115:0/2214206146 conn(0x7f838406f810 msgr2=0x7f838406fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.593+0000 7f838890c640 1 -- 172.21.15.115:0/2214206146 shutdown_connections 2024-05-09T05:53:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.593+0000 7f838890c640 1 -- 172.21.15.115:0/2214206146 wait complete. 2024-05-09T05:53:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.594+0000 7f838890c640 1 Processor -- start 2024-05-09T05:53:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.594+0000 7f838890c640 1 -- start start 2024-05-09T05:53:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.594+0000 7f838890c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f838410a210 0x7f83841a9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.595+0000 7f838890c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410fa10 0x7f83841a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.595+0000 7f838890c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83841103f0 0x7f83841adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.595+0000 7f838890c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8384122920 con 0x7f838410fa10 2024-05-09T05:53:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.595+0000 7f838890c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f83841227a0 con 0x7f838410a210 2024-05-09T05:53:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.595+0000 7f838890c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8384122aa0 con 0x7f83841103f0 2024-05-09T05:53:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.595+0000 7f838259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410fa10 0x7f83841a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.595+0000 7f838259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410fa10 0x7f83841a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39846/0 (socket says 172.21.15.115:39846) 2024-05-09T05:53:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.595+0000 7f838259c640 1 -- 172.21.15.115:0/1805680824 learned_addr learned my addr 172.21.15.115:0/1805680824 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.596+0000 7f838259c640 1 -- 172.21.15.115:0/1805680824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83841103f0 msgr2=0x7f83841adb80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T05:53:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.596+0000 7f838259c640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83841103f0 0x7f83841adb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.596+0000 7f838259c640 1 -- 172.21.15.115:0/1805680824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f838410a210 msgr2=0x7f83841a9360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:53:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.596+0000 7f838259c640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f838410a210 0x7f83841a9360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.596+0000 7f838259c640 1 -- 172.21.15.115:0/1805680824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83841ae1c0 con 0x7f838410fa10 2024-05-09T05:53:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.596+0000 7f838259c640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410fa10 0x7f83841a98c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f837800d770 tx=0x7f837800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.597+0000 7f836bfff640 1 -- 172.21.15.115:0/1805680824 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378017070 con 0x7f838410fa10 2024-05-09T05:53:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.597+0000 7f838890c640 1 -- 172.21.15.115:0/1805680824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83841ae4b0 con 0x7f838410fa10 2024-05-09T05:53:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.597+0000 7f838890c640 1 -- 172.21.15.115:0/1805680824 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838407af30 con 0x7f838410fa10 2024-05-09T05:53:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.599+0000 7f836bfff640 1 -- 172.21.15.115:0/1805680824 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8378009c30 con 0x7f838410fa10 2024-05-09T05:53:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.599+0000 7f836bfff640 1 -- 172.21.15.115:0/1805680824 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837800f3c0 con 0x7f838410fa10 2024-05-09T05:53:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.601+0000 7f838ffc3740 1 Processor -- start 2024-05-09T05:53:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.601+0000 7f838ffc3740 1 -- start start 2024-05-09T05:53:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.601+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d65d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.601+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2912f0 0x55df3d65dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.601+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 0x55df3d65e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.601+0000 7f838ffc3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55df3d5c27c0 con 0x55df3d2ba1b0 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.601+0000 7f838ffc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55df3d55e0e0 con 0x55df3d2912f0 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.601+0000 7f838ffc3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55df3d5be780 con 0x55df3d2994d0 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.602+0000 7f8382d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2912f0 0x55df3d65dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.602+0000 7f8382d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2912f0 0x55df3d65dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:35086/0 (socket says 172.21.15.115:35086) 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.602+0000 7f8382d9d640 1 -- 172.21.15.115:0/2441205875 learned_addr learned my addr 172.21.15.115:0/2441205875 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.602+0000 7f838359e640 1 --2- 172.21.15.115:0/2441205875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d65d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.602+0000 7f8382d9d640 1 -- 172.21.15.115:0/2441205875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 msgr2=0x55df3d65e2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.602+0000 7f8382d9d640 1 --2- 172.21.15.115:0/2441205875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 0x55df3d65e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.602+0000 7f8382d9d640 1 -- 172.21.15.115:0/2441205875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 msgr2=0x55df3d65d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.602+0000 7f8382d9d640 1 --2- 172.21.15.115:0/2441205875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d65d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.603+0000 7f836bfff640 1 -- 172.21.15.115:0/1805680824 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f837801a070 con 0x7f838410fa10 2024-05-09T05:53:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.604+0000 7f8382d9d640 1 -- 172.21.15.115:0/2441205875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3d5877c0 con 0x55df3d2912f0 2024-05-09T05:53:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.605+0000 7f836bfff640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f836c0b2b60 0x7f836c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.605+0000 7f836bfff640 1 -- 172.21.15.115:0/1805680824 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f83780e2410 con 0x7f838410fa10 2024-05-09T05:53:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.606+0000 7f8382d9d640 1 --2- 172.21.15.115:0/2441205875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2912f0 0x55df3d65dd80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8374009ac0 tx=0x7f8374009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e841035067987ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.606+0000 7f83697fa640 1 -- 172.21.15.115:0/2441205875 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837400ed10 con 0x55df3d2912f0 2024-05-09T05:53:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.606+0000 7f838359e640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f836c0b2b60 0x7f836c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.606+0000 7f83697fa640 1 -- 172.21.15.115:0/2441205875 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83740054b0 con 0x55df3d2912f0 2024-05-09T05:53:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.606+0000 7f83697fa640 1 -- 172.21.15.115:0/2441205875 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374010430 con 0x55df3d2912f0 2024-05-09T05:53:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.607+0000 7f838ffc3740 1 -- 172.21.15.115:0/2441205875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2912f0 msgr2=0x55df3d65dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.607+0000 7f838ffc3740 1 --2- 172.21.15.115:0/2441205875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2912f0 0x55df3d65dd80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8374009ac0 tx=0x7f8374009f80 comp rx=0 tx=0).stop 2024-05-09T05:53:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.609+0000 7f838359e640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f836c0b2b60 0x7f836c0b5070 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f837c006010 tx=0x7f837c0093b0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.616+0000 7f838ffc3740 1 -- 172.21.15.115:0/2441205875 shutdown_connections 2024-05-09T05:53:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.616+0000 7f838ffc3740 1 --2- 172.21.15.115:0/2441205875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 0x55df3d65e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.616+0000 7f838ffc3740 1 --2- 172.21.15.115:0/2441205875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2912f0 0x55df3d65dd80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.616+0000 7f838ffc3740 1 --2- 172.21.15.115:0/2441205875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d65d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.616+0000 7f838ffc3740 1 -- 172.21.15.115:0/2441205875 >> 172.21.15.115:0/2441205875 conn(0x55df3d2ae300 msgr2=0x55df3d659ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.621+0000 7f838ffc3740 1 -- 172.21.15.115:0/2441205875 shutdown_connections 2024-05-09T05:53:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.621+0000 7f838ffc3740 1 -- 172.21.15.115:0/2441205875 wait complete. 2024-05-09T05:53:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.621+0000 7f838ffc3740 1 -- start start 2024-05-09T05:53:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.621+0000 7f838ffc3740 1 Processor -- start 2024-05-09T05:53:57.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:57 smithi195 ceph-mon[33844]: pgmap v496: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-05-09T05:53:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d669930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 0x55df3d669e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d667dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f838ffc3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55df3d66a970 con 0x55df3d2ba1b0 2024-05-09T05:53:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f838ffc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55df3d5499c0 con 0x55df3d2994d0 2024-05-09T05:53:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f838ffc3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55df3d66aaf0 con 0x55df3d2912f0 2024-05-09T05:53:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f838ffc3740 10 client.0 _ll_register_callbacks cb 0x55df3d576020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f8382d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d669930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f8382d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d669930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:49718/0 (socket says 172.21.15.115:49718) 2024-05-09T05:53:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.622+0000 7f8382d9d640 1 -- 172.21.15.115:0/3080663560 learned_addr learned my addr 172.21.15.115:0/3080663560 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.623+0000 7f8382d9d640 1 -- 172.21.15.115:0/3080663560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 msgr2=0x55df3d669e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.623+0000 7f8382d9d640 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 0x55df3d669e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.623+0000 7f8382d9d640 1 -- 172.21.15.115:0/3080663560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 msgr2=0x55df3d667dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.623+0000 7f8382d9d640 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d667dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.623+0000 7f8382d9d640 1 -- 172.21.15.115:0/3080663560 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3d5701f0 con 0x55df3d2912f0 2024-05-09T05:53:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.623+0000 7f8382d9d640 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d669930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8374018000 tx=0x7f83740048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=987bbcf9e57af638 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.625+0000 7f8366ff5640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:53:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.625+0000 7f8366ff5640 1 -- 172.21.15.115:0/3080663560 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374028060 con 0x55df3d2912f0 2024-05-09T05:53:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.625+0000 7f838ffc3740 1 -- 172.21.15.115:0/3080663560 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83740054b0 con 0x55df3d2912f0 2024-05-09T05:53:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.625+0000 7f8366ff5640 1 -- 172.21.15.115:0/3080663560 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f8374005200 con 0x55df3d2912f0 2024-05-09T05:53:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.630+0000 7f8366ff5640 1 -- 172.21.15.115:0/3080663560 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f837402f050 con 0x55df3d2912f0 2024-05-09T05:53:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.632+0000 7f838ffc3740 10 client.28105 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.632+0000 7f838ffc3740 1 -- 172.21.15.115:0/3080663560 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55df3d325790 con 0x55df3d2912f0 2024-05-09T05:53:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.635+0000 7f8366ff5640 1 -- 172.21.15.115:0/3080663560 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83740054b0 con 0x55df3d2912f0 2024-05-09T05:53:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.635+0000 7f8366ff5640 1 -- 172.21.15.115:0/3080663560 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f83740036b0 con 0x55df3d2912f0 2024-05-09T05:53:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.635+0000 7f838ffc3740 10 client.28105 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.635+0000 7f838ffc3740 10 client.28105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.635+0000 7f838ffc3740 20 client.28105 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.635+0000 7f838ffc3740 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55df3d278d80 0x55df3d68b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.635+0000 7f838ffc3740 4 client.28105 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:53:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.635+0000 7f838ffc3740 1 -- 172.21.15.115:0/3080663560 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3d5d03d0 con 0x55df3d278d80 2024-05-09T05:53:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.641+0000 7f838259c640 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55df3d278d80 0x55df3d68b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.644+0000 7f838259c640 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55df3d278d80 0x55df3d68b0e0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d65ae7a27ad288d6 server_cookie=74cf0ebdb8673cfc in_seq=0 out_seq=0 2024-05-09T05:53:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.644+0000 7f8366ff5640 10 client.28105 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.645+0000 7f8366ff5640 1 -- 172.21.15.115:0/3080663560 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df3d5d03d0 con 0x55df3d278d80 2024-05-09T05:53:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.645+0000 7f8366ff5640 10 client.28105 handle_command_reply: tid=0 2024-05-09T05:53:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.645+0000 7f838ffc3740 1 client.28105 shutdown 2024-05-09T05:53:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.651+0000 7f838ffc3740 1 -- 172.21.15.115:0/3080663560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 msgr2=0x55df3d669930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.651+0000 7f838ffc3740 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d669930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8374018000 tx=0x7f83740048d0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.651+0000 7f838ffc3740 1 -- 172.21.15.115:0/3080663560 shutdown_connections 2024-05-09T05:53:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.651+0000 7f838ffc3740 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d667dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.651+0000 7f838ffc3740 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 0x55df3d669e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.651+0000 7f838ffc3740 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55df3d278d80 0x55df3d68b0e0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.651+0000 7f838ffc3740 1 --2- 172.21.15.115:0/3080663560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d669930 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.651+0000 7f838ffc3740 1 -- 172.21.15.115:0/3080663560 >> 172.21.15.115:0/3080663560 conn(0x55df3d2ae300 msgr2=0x55df3d65f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.651+0000 7f8366ff5640 0 client.28105 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:57.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:57 smithi115 ceph-mon[30349]: pgmap v496: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-05-09T05:53:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.654+0000 7f838ffc3740 1 -- 172.21.15.115:0/3080663560 shutdown_connections 2024-05-09T05:53:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.655+0000 7f838ffc3740 1 -- 172.21.15.115:0/3080663560 wait complete. 2024-05-09T05:53:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.656+0000 7f838ffc3740 20 client.28105 trim_cache size 0 max 16384 2024-05-09T05:53:57.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:57 smithi161 ceph-mon[33805]: pgmap v496: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-05-09T05:53:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.708+0000 7f838ffc3740 1 Processor -- start 2024-05-09T05:53:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.709+0000 7f838ffc3740 1 -- start start 2024-05-09T05:53:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.709+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2912f0 0x55df3d65a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.709+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 0x55df3d6906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.709+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2ba1b0 0x55df3d690bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.709+0000 7f838ffc3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f837000ffb0 con 0x55df3d2912f0 2024-05-09T05:53:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.709+0000 7f838ffc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8370011b20 con 0x55df3d2ba1b0 2024-05-09T05:53:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.709+0000 7f838ffc3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8370010680 con 0x55df3d2994d0 2024-05-09T05:53:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.709+0000 7f8382d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2912f0 0x55df3d65a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.710+0000 7f8382d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2912f0 0x55df3d65a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39878/0 (socket says 172.21.15.115:39878) 2024-05-09T05:53:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.710+0000 7f8382d9d640 1 -- 172.21.15.115:0/1109474948 learned_addr learned my addr 172.21.15.115:0/1109474948 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.710+0000 7f838259c640 1 --2- 172.21.15.115:0/1109474948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 0x55df3d6906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.710+0000 7f838259c640 1 -- 172.21.15.115:0/1109474948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2ba1b0 msgr2=0x55df3d690bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.710+0000 7f838259c640 1 --2- 172.21.15.115:0/1109474948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2ba1b0 0x55df3d690bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.710+0000 7f838259c640 1 -- 172.21.15.115:0/1109474948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2912f0 msgr2=0x55df3d65a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.710+0000 7f838259c640 1 --2- 172.21.15.115:0/1109474948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2912f0 0x55df3d65a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.710+0000 7f838259c640 1 -- 172.21.15.115:0/1109474948 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3d39c5f0 con 0x55df3d2994d0 2024-05-09T05:53:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.713+0000 7f838259c640 1 --2- 172.21.15.115:0/1109474948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 0x55df3d6906b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83780df770 tx=0x7f837800df10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f25fb28d082f33b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.713+0000 7f8368ff9640 1 -- 172.21.15.115:0/1109474948 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378021040 con 0x55df3d2994d0 2024-05-09T05:53:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.713+0000 7f8368ff9640 1 -- 172.21.15.115:0/1109474948 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83780aa0b0 con 0x55df3d2994d0 2024-05-09T05:53:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.714+0000 7f8368ff9640 1 -- 172.21.15.115:0/1109474948 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83780a6340 con 0x55df3d2994d0 2024-05-09T05:53:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.715+0000 7f838ffc3740 1 -- 172.21.15.115:0/1109474948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 msgr2=0x55df3d6906b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.715+0000 7f838ffc3740 1 --2- 172.21.15.115:0/1109474948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 0x55df3d6906b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83780df770 tx=0x7f837800df10 comp rx=0 tx=0).stop 2024-05-09T05:53:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.721+0000 7f838ffc3740 1 -- 172.21.15.115:0/1109474948 shutdown_connections 2024-05-09T05:53:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.721+0000 7f838ffc3740 1 --2- 172.21.15.115:0/1109474948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2ba1b0 0x55df3d690bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.721+0000 7f838ffc3740 1 --2- 172.21.15.115:0/1109474948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2994d0 0x55df3d6906b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.721+0000 7f838ffc3740 1 --2- 172.21.15.115:0/1109474948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2912f0 0x55df3d65a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.721+0000 7f838ffc3740 1 -- 172.21.15.115:0/1109474948 >> 172.21.15.115:0/1109474948 conn(0x55df3d2ae300 msgr2=0x55df3d6599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.721+0000 7f838ffc3740 1 -- 172.21.15.115:0/1109474948 shutdown_connections 2024-05-09T05:53:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.721+0000 7f838ffc3740 1 -- 172.21.15.115:0/1109474948 wait complete. 2024-05-09T05:53:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.721+0000 7f838ffc3740 1 -- start start 2024-05-09T05:53:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.722+0000 7f838ffc3740 1 Processor -- start 2024-05-09T05:53:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.722+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d670480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.722+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 0x55df3d66add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.722+0000 7f838ffc3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d66b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.722+0000 7f838ffc3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8370011450 con 0x55df3d2ba1b0 2024-05-09T05:53:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.722+0000 7f838ffc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8370010d80 con 0x55df3d2994d0 2024-05-09T05:53:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.722+0000 7f838ffc3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f837000dfb0 con 0x55df3d2912f0 2024-05-09T05:53:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.722+0000 7f838ffc3740 10 client.0 _ll_register_callbacks cb 0x7f8374009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:53:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.723+0000 7f8382d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d670480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.723+0000 7f8382d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d670480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:49736/0 (socket says 172.21.15.115:49736) 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.723+0000 7f8382d9d640 1 -- 172.21.15.115:0/85141485 learned_addr learned my addr 172.21.15.115:0/85141485 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.723+0000 7f838259c640 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 0x55df3d66add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.723+0000 7f838359e640 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d66b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f838259c640 1 -- 172.21.15.115:0/85141485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 msgr2=0x55df3d670480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f838259c640 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d670480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f838259c640 1 -- 172.21.15.115:0/85141485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 msgr2=0x55df3d66b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f838259c640 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d66b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f838259c640 1 -- 172.21.15.115:0/85141485 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3d586ed0 con 0x55df3d2994d0 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f838259c640 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 0x55df3d66add0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8378009f10 tx=0x7f83780a6090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df319296e25d275 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:53:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f8366ff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:53:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f8366ff5640 1 -- 172.21.15.115:0/85141485 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83780a66a0 con 0x55df3d2994d0 2024-05-09T05:53:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f838ffc3740 1 -- 172.21.15.115:0/85141485 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83780aa0b0 con 0x55df3d2994d0 2024-05-09T05:53:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.724+0000 7f8366ff5640 1 -- 172.21.15.115:0/85141485 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f83780df9f0 con 0x55df3d2994d0 2024-05-09T05:53:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.725+0000 7f8366ff5640 1 -- 172.21.15.115:0/85141485 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f83780dfbf0 con 0x55df3d2994d0 2024-05-09T05:53:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.726+0000 7f838ffc3740 10 client.27989 fetch_fsmap learned FSMap version 62 2024-05-09T05:53:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.726+0000 7f838ffc3740 1 -- 172.21.15.115:0/85141485 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55df3d6f20a0 con 0x55df3d2994d0 2024-05-09T05:53:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.727+0000 7f8366ff5640 1 -- 172.21.15.115:0/85141485 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83780aa0b0 con 0x55df3d2994d0 2024-05-09T05:53:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.728+0000 7f8366ff5640 1 -- 172.21.15.115:0/85141485 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f8378004040 con 0x55df3d2994d0 2024-05-09T05:53:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.728+0000 7f838ffc3740 10 client.27989 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:53:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.728+0000 7f838ffc3740 10 client.27989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:53:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.728+0000 7f838ffc3740 20 client.27989 populate_metadata read hostname 'smithi115' 2024-05-09T05:53:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.728+0000 7f838ffc3740 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55df3d24e1c0 0x55df3d6f4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:53:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.728+0000 7f838ffc3740 4 client.27989 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:53:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.728+0000 7f838ffc3740 1 -- 172.21.15.115:0/85141485 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df3d5ca240 con 0x55df3d24e1c0 2024-05-09T05:53:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.728+0000 7f8382d9d640 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55df3d24e1c0 0x55df3d6f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:53:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.730+0000 7f8366ff5640 10 client.27989 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.730+0000 7f8382d9d640 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55df3d24e1c0 0x55df3d6f4780 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c20621093a688c server_cookie=354b3fff6980f852 in_seq=0 out_seq=0 2024-05-09T05:53:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.813+0000 7f8366ff5640 1 -- 172.21.15.115:0/85141485 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55df3d5ca240 con 0x55df3d24e1c0 2024-05-09T05:53:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.814+0000 7f8366ff5640 10 client.27989 handle_command_reply: tid=0 2024-05-09T05:53:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.814+0000 7f838ffc3740 1 client.27989 shutdown 2024-05-09T05:53:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.818+0000 7f838ffc3740 1 -- 172.21.15.115:0/85141485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 msgr2=0x55df3d66add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.818+0000 7f838ffc3740 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 0x55df3d66add0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8378009f10 tx=0x7f83780a6090 comp rx=0 tx=0).stop 2024-05-09T05:53:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f838ffc3740 1 -- 172.21.15.115:0/85141485 shutdown_connections 2024-05-09T05:53:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f838ffc3740 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55df3d2ba1b0 0x55df3d66b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f838ffc3740 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df3d2994d0 0x55df3d66add0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f838ffc3740 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55df3d24e1c0 0x55df3d6f4780 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f838ffc3740 1 --2- 172.21.15.115:0/85141485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df3d2912f0 0x55df3d670480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f838ffc3740 1 -- 172.21.15.115:0/85141485 >> 172.21.15.115:0/85141485 conn(0x55df3d2ae300 msgr2=0x55df3d65f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f8366ff5640 0 client.27989 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:53:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f838ffc3740 1 -- 172.21.15.115:0/85141485 shutdown_connections 2024-05-09T05:53:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f838ffc3740 1 -- 172.21.15.115:0/85141485 wait complete. 2024-05-09T05:53:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.822+0000 7f838ffc3740 20 client.27989 trim_cache size 0 max 16384 2024-05-09T05:53:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:53:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:53:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:53:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.830+0000 7f838890c640 1 -- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f836c0b2b60 msgr2=0x7f836c0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.830+0000 7f838890c640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f836c0b2b60 0x7f836c0b5070 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f837c006010 tx=0x7f837c0093b0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.830+0000 7f838890c640 1 -- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410fa10 msgr2=0x7f83841a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:53:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.830+0000 7f838890c640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410fa10 0x7f83841a98c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f837800d770 tx=0x7f837800dc30 comp rx=0 tx=0).stop 2024-05-09T05:53:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.831+0000 7f838890c640 1 -- 172.21.15.115:0/1805680824 shutdown_connections 2024-05-09T05:53:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.831+0000 7f838890c640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f836c0b2b60 0x7f836c0b5070 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.831+0000 7f838890c640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83841103f0 0x7f83841adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.831+0000 7f838890c640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f838410fa10 0x7f83841a98c0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.831+0000 7f838890c640 1 --2- 172.21.15.115:0/1805680824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f838410a210 0x7f83841a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:53:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.831+0000 7f838890c640 1 -- 172.21.15.115:0/1805680824 >> 172.21.15.115:0/1805680824 conn(0x7f838406f810 msgr2=0x7f838410d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:53:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.835+0000 7f838890c640 1 -- 172.21.15.115:0/1805680824 shutdown_connections 2024-05-09T05:53:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:53:57.835+0000 7f838890c640 1 -- 172.21.15.115:0/1805680824 wait complete. 2024-05-09T05:53:59.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:53:59 smithi195 ceph-mon[33844]: pgmap v497: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 4.62k op/s 2024-05-09T05:53:59.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:53:59 smithi115 ceph-mon[30349]: pgmap v497: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 4.62k op/s 2024-05-09T05:53:59.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:53:59 smithi161 ceph-mon[33805]: pgmap v497: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 4.62k op/s 2024-05-09T05:54:01.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:01 smithi195 ceph-mon[33844]: pgmap v498: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.90k op/s 2024-05-09T05:54:01.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:01 smithi115 ceph-mon[30349]: pgmap v498: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.90k op/s 2024-05-09T05:54:01.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:01 smithi161 ceph-mon[33805]: pgmap v498: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.90k op/s 2024-05-09T05:54:03.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:03 smithi195 ceph-mon[33844]: pgmap v499: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-05-09T05:54:03.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:03 smithi115 ceph-mon[30349]: pgmap v499: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-05-09T05:54:03.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:03 smithi161 ceph-mon[33805]: pgmap v499: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-05-09T05:54:05.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:05 smithi195 ceph-mon[33844]: pgmap v500: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 20 MiB/s wr, 5.13k op/s 2024-05-09T05:54:05.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:05 smithi115 ceph-mon[30349]: pgmap v500: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 20 MiB/s wr, 5.13k op/s 2024-05-09T05:54:05.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:05 smithi161 ceph-mon[33805]: pgmap v500: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 20 MiB/s wr, 5.13k op/s 2024-05-09T05:54:07.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:07 smithi195 ceph-mon[33844]: pgmap v501: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-05-09T05:54:07.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:07 smithi115 ceph-mon[30349]: pgmap v501: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-05-09T05:54:07.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:07 smithi161 ceph-mon[33805]: pgmap v501: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-05-09T05:54:09.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:09 smithi195 ceph-mon[33844]: pgmap v502: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-05-09T05:54:09.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:09 smithi115 ceph-mon[30349]: pgmap v502: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-05-09T05:54:09.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:09 smithi161 ceph-mon[33805]: pgmap v502: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-05-09T05:54:09.962 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.1 2024-05-09T05:54:09.963 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:54:10.367 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:10 smithi115 ceph-mon[30349]: osdmap e110: 12 total, 12 up, 12 in 2024-05-09T05:54:10.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:10 smithi161 ceph-mon[33805]: osdmap e110: 12 total, 12 up, 12 in 2024-05-09T05:54:10.756 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:54:10.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:10 smithi195 ceph-mon[33844]: osdmap e110: 12 total, 12 up, 12 in 2024-05-09T05:54:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.105+0000 7f7dd99c7640 1 -- 172.21.15.115:0/2066499911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dd411d250 msgr2=0x7f7dd411f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:11.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.105+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/2066499911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dd411d250 0x7f7dd411f670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc01acb0 tx=0x7f7dcc03b990 comp rx=0 tx=0).stop 2024-05-09T05:54:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.111+0000 7f7dd99c7640 1 -- 172.21.15.115:0/2066499911 shutdown_connections 2024-05-09T05:54:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.111+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/2066499911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dd411d250 0x7f7dd411f670 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.111+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/2066499911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd411a8f0 0x7f7dd411cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.111+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/2066499911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd410a200 0x7f7dd410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.111+0000 7f7dd99c7640 1 -- 172.21.15.115:0/2066499911 >> 172.21.15.115:0/2066499911 conn(0x7f7dd406e900 msgr2=0x7f7dd406ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.112+0000 7f7dd99c7640 1 -- 172.21.15.115:0/2066499911 shutdown_connections 2024-05-09T05:54:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.114+0000 7f7dd99c7640 1 -- 172.21.15.115:0/2066499911 wait complete. 2024-05-09T05:54:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.114+0000 7f7dd99c7640 1 Processor -- start 2024-05-09T05:54:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.115+0000 7f7dd99c7640 1 -- start start 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.115+0000 7f7dd99c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd410a200 0x7f7dd411a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.115+0000 7f7dd99c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd411a8f0 0x7f7dd4116380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.115+0000 7f7dd99c7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dd411d250 0x7f7dd41168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.115+0000 7f7dd99c7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4114810 con 0x7f7dd411a8f0 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.115+0000 7f7dd99c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4114690 con 0x7f7dd410a200 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.115+0000 7f7dd99c7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4114990 con 0x7f7dd411d250 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd411a8f0 0x7f7dd4116380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd410a200 0x7f7dd411a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd411a8f0 0x7f7dd4116380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56188/0 (socket says 172.21.15.115:56188) 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd27fc640 1 -- 172.21.15.115:0/1145748146 learned_addr learned my addr 172.21.15.115:0/1145748146 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dd411d250 0x7f7dd41168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd27fc640 1 -- 172.21.15.115:0/1145748146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dd411d250 msgr2=0x7f7dd41168c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dd411d250 0x7f7dd41168c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd27fc640 1 -- 172.21.15.115:0/1145748146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd410a200 msgr2=0x7f7dd411a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd410a200 0x7f7dd411a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd27fc640 1 -- 172.21.15.115:0/1145748146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd4117030 con 0x7f7dd411a8f0 2024-05-09T05:54:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.116+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd410a200 0x7f7dd411a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:54:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.117+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd411a8f0 0x7f7dd4116380 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c8a0 tx=0x7f7dc800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.117+0000 7f7dd89c5640 1 -- 172.21.15.115:0/1145748146 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8007bb0 con 0x7f7dd411a8f0 2024-05-09T05:54:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.117+0000 7f7dd89c5640 1 -- 172.21.15.115:0/1145748146 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dc8007d50 con 0x7f7dd411a8f0 2024-05-09T05:54:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.117+0000 7f7dd89c5640 1 -- 172.21.15.115:0/1145748146 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8018730 con 0x7f7dd411a8f0 2024-05-09T05:54:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.117+0000 7f7dd99c7640 1 -- 172.21.15.115:0/1145748146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd41cf160 con 0x7f7dd411a8f0 2024-05-09T05:54:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.117+0000 7f7dd99c7640 1 -- 172.21.15.115:0/1145748146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd41cf6a0 con 0x7f7dd411a8f0 2024-05-09T05:54:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.120+0000 7f7dd89c5640 1 -- 172.21.15.115:0/1145748146 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f7dc800f450 con 0x7f7dd411a8f0 2024-05-09T05:54:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.120+0000 7f7de0875740 1 Processor -- start 2024-05-09T05:54:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.120+0000 7f7de0875740 1 -- start start 2024-05-09T05:54:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7de0875740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630bef0d1b0 0x5630bf2b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7de0875740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beee42f0 0x5630bf2b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7de0875740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7de0875740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5630bf2157c0 con 0x5630bef0d1b0 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7de0875740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5630bf1b10e0 con 0x5630beee42f0 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7de0875740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5630bf211780 con 0x5630beeec4d0 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd89c5640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7dbc0b2a20 0x7f7dbc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630bef0d1b0 0x5630bf2b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630bef0d1b0 0x5630bf2b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56196/0 (socket says 172.21.15.115:56196) 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beee42f0 0x5630bf2b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 -- 172.21.15.115:0/49483372 learned_addr learned my addr 172.21.15.115:0/49483372 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/49483372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7dbc0b2a20 0x7f7dbc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd89c5640 1 -- 172.21.15.115:0/1145748146 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f7dc80e2960 con 0x7f7dd411a8f0 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 -- 172.21.15.115:0/49483372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 msgr2=0x5630bf2b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/49483372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 -- 172.21.15.115:0/49483372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beee42f0 msgr2=0x5630bf2b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/49483372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beee42f0 0x5630bf2b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 -- 172.21.15.115:0/49483372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630bf1da7c0 con 0x5630bef0d1b0 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/49483372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/49483372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630bef0d1b0 0x5630bf2b0680 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc03bf80 tx=0x7f7dcc003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59622750907ef10d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.121+0000 7f7db9ffb640 1 -- 172.21.15.115:0/49483372 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc04f070 con 0x5630bef0d1b0 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.122+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7dbc0b2a20 0x7f7dbc0b4f30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc04c000 tx=0x7f7dcc019000 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.122+0000 7f7db9ffb640 1 -- 172.21.15.115:0/49483372 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dcc004040 con 0x5630bef0d1b0 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.122+0000 7f7db9ffb640 1 -- 172.21.15.115:0/49483372 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc045040 con 0x5630bef0d1b0 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.122+0000 7f7de0875740 1 -- 172.21.15.115:0/49483372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630bef0d1b0 msgr2=0x5630bf2b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.122+0000 7f7de0875740 1 --2- 172.21.15.115:0/49483372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630bef0d1b0 0x5630bf2b0680 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc03bf80 tx=0x7f7dcc003ac0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 -- 172.21.15.115:0/49483372 shutdown_connections 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 --2- 172.21.15.115:0/49483372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 --2- 172.21.15.115:0/49483372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beee42f0 0x5630bf2b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 --2- 172.21.15.115:0/49483372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630bef0d1b0 0x5630bf2b0680 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 -- 172.21.15.115:0/49483372 >> 172.21.15.115:0/49483372 conn(0x5630bef01300 msgr2=0x5630bf2ace30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 -- 172.21.15.115:0/49483372 shutdown_connections 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 -- 172.21.15.115:0/49483372 wait complete. 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 -- start start 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 Processor -- start 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2bcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 0x5630bf2bab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.123+0000 7f7de0875740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5630bf2bd7b0 con 0x5630beee42f0 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7de0875740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5630bf19c9c0 con 0x5630bef0d1b0 2024-05-09T05:54:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7de0875740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5630bf2bd930 con 0x5630beeec4d0 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56202/0 (socket says 172.21.15.115:56202) 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7de0875740 10 client.0 _ll_register_callbacks cb 0x5630bf1c9020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/3146772376 learned_addr learned my addr 172.21.15.115:0/3146772376 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 0x5630bf2bab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2bcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/3146772376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 msgr2=0x5630bf2bcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2bcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/3146772376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 msgr2=0x5630bf2bab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 0x5630bf2bab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/3146772376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630bf1c31f0 con 0x5630beee42f0 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 0x5630bf2bab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bc6a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40118e0 tx=0x7f7dc4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af371dc7da47530 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7db2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7db2ffd640 1 -- 172.21.15.115:0/3146772376 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400fa90 con 0x5630beee42f0 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7db2ffd640 1 -- 172.21.15.115:0/3146772376 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f7dc400fc30 con 0x5630beee42f0 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.124+0000 7f7de0875740 1 -- 172.21.15.115:0/3146772376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dcc004040 con 0x5630beee42f0 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7db2ffd640 1 -- 172.21.15.115:0/3146772376 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7dc4026050 con 0x5630beee42f0 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7de0875740 10 client.18792 fetch_fsmap learned FSMap version 62 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7de0875740 1 -- 172.21.15.115:0/3146772376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5630bef78790 con 0x5630beee42f0 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7db2ffd640 1 -- 172.21.15.115:0/3146772376 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dcc004040 con 0x5630beee42f0 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7db2ffd640 1 -- 172.21.15.115:0/3146772376 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f7dc401d400 con 0x5630beee42f0 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7de0875740 10 client.18792 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7de0875740 10 client.18792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7de0875740 20 client.18792 populate_metadata read hostname 'smithi115' 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7de0875740 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5630beecbd80 0x5630bf2ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7de0875740 4 client.18792 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:54:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7de0875740 1 -- 172.21.15.115:0/3146772376 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630bf2233d0 con 0x5630beecbd80 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.125+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5630beecbd80 0x5630bf2ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.126+0000 7f7db2ffd640 10 client.18792 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.126+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5630beecbd80 0x5630bf2ccd80 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87afa8509bea2f7e server_cookie=1469fff9a4a10136 in_seq=0 out_seq=0 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7db2ffd640 1 -- 172.21.15.115:0/3146772376 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630bf2233d0 con 0x5630beecbd80 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7db2ffd640 10 client.18792 handle_command_reply: tid=0 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 client.18792 shutdown 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 -- 172.21.15.115:0/3146772376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 msgr2=0x5630bf2bc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bc6a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40118e0 tx=0x7f7dc4011da0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 -- 172.21.15.115:0/3146772376 shutdown_connections 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 0x5630bf2bab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5630beecbd80 0x5630bf2ccd80 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7db2ffd640 0 client.18792 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2bcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 --2- 172.21.15.115:0/3146772376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bc6a0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 -- 172.21.15.115:0/3146772376 >> 172.21.15.115:0/3146772376 conn(0x5630bef01300 msgr2=0x5630bf2b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 -- 172.21.15.115:0/3146772376 shutdown_connections 2024-05-09T05:54:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 1 -- 172.21.15.115:0/3146772376 wait complete. 2024-05-09T05:54:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.127+0000 7f7de0875740 20 client.18792 trim_cache size 0 max 16384 2024-05-09T05:54:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.142+0000 7f7de0875740 1 Processor -- start 2024-05-09T05:54:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.142+0000 7f7de0875740 1 -- start start 2024-05-09T05:54:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.142+0000 7f7de0875740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2baa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.142+0000 7f7de0875740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.142+0000 7f7de0875740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 0x5630bf2d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.142+0000 7f7de0875740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41d97d0 con 0x5630beee42f0 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.142+0000 7f7de0875740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41db340 con 0x5630bef0d1b0 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.142+0000 7f7de0875740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41d9ea0 con 0x5630beeec4d0 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2baa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56206/0 (socket says 172.21.15.115:56206) 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/1664502296 learned_addr learned my addr 172.21.15.115:0/1664502296 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/1664502296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/1664502296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 0x5630bf2d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/1664502296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 msgr2=0x5630bf2acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/1664502296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/1664502296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 msgr2=0x5630bf2d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/1664502296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 0x5630bf2d2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/1664502296 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630befef5f0 con 0x5630beee42f0 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/1664502296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2baa40 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40118b0 tx=0x7f7dc4009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cdd755eeb24b5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7db97fa640 1 -- 172.21.15.115:0/1664502296 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc402e020 con 0x5630beee42f0 2024-05-09T05:54:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7db97fa640 1 -- 172.21.15.115:0/1664502296 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dc401da40 con 0x5630beee42f0 2024-05-09T05:54:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.143+0000 7f7db97fa640 1 -- 172.21.15.115:0/1664502296 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40045c0 con 0x5630beee42f0 2024-05-09T05:54:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.144+0000 7f7de0875740 1 -- 172.21.15.115:0/1664502296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 msgr2=0x5630bf2baa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.144+0000 7f7de0875740 1 --2- 172.21.15.115:0/1664502296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2baa40 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40118b0 tx=0x7f7dc4009f00 comp rx=0 tx=0).stop 2024-05-09T05:54:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.144+0000 7f7de0875740 1 -- 172.21.15.115:0/1664502296 shutdown_connections 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.144+0000 7f7de0875740 1 --2- 172.21.15.115:0/1664502296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630bef0d1b0 0x5630bf2d2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.144+0000 7f7de0875740 1 --2- 172.21.15.115:0/1664502296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630beeec4d0 0x5630bf2acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.144+0000 7f7de0875740 1 --2- 172.21.15.115:0/1664502296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2baa40 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.144+0000 7f7de0875740 1 -- 172.21.15.115:0/1664502296 >> 172.21.15.115:0/1664502296 conn(0x5630bef01300 msgr2=0x5630bf2ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.144+0000 7f7de0875740 1 -- 172.21.15.115:0/1664502296 shutdown_connections 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.144+0000 7f7de0875740 1 -- 172.21.15.115:0/1664502296 wait complete. 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7de0875740 1 -- start start 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7de0875740 1 Processor -- start 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7de0875740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7de0875740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beeec4d0 0x5630bf2be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7de0875740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630bef0d1b0 0x5630bf2be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7de0875740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41dac70 con 0x5630beee42f0 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7de0875740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41da5a0 con 0x5630beeec4d0 2024-05-09T05:54:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7de0875740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41d77d0 con 0x5630bef0d1b0 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7de0875740 10 client.0 _ll_register_callbacks cb 0x7f7dc40114f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7dd27fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beeec4d0 0x5630bf2be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56216/0 (socket says 172.21.15.115:56216) 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7dd37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630bef0d1b0 0x5630bf2be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.145+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/2245608960 learned_addr learned my addr 172.21.15.115:0/2245608960 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/2245608960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630bef0d1b0 msgr2=0x5630bf2be620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630bef0d1b0 0x5630bf2be620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/2245608960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beeec4d0 msgr2=0x5630bf2be0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beeec4d0 0x5630bf2be0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/2245608960 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630bf1d9ed0 con 0x5630beee42f0 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bdb80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4011de0 tx=0x7f7dc401d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c356386b95d9b15 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7db2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7db2ffd640 1 -- 172.21.15.115:0/2245608960 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc402e030 con 0x5630beee42f0 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7de0875740 1 -- 172.21.15.115:0/2245608960 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc401da40 con 0x5630beee42f0 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7db2ffd640 1 -- 172.21.15.115:0/2245608960 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f7dc400fc90 con 0x5630beee42f0 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7db2ffd640 1 -- 172.21.15.115:0/2245608960 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f7dc400fe90 con 0x5630beee42f0 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7de0875740 10 client.18798 fetch_fsmap learned FSMap version 62 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7de0875740 1 -- 172.21.15.115:0/2245608960 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5630bf3343e0 con 0x5630beee42f0 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.146+0000 7f7db2ffd640 1 -- 172.21.15.115:0/2245608960 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dc401da40 con 0x5630beee42f0 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7db2ffd640 1 -- 172.21.15.115:0/2245608960 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f7dc4014440 con 0x5630beee42f0 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7de0875740 10 client.18798 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7de0875740 10 client.18798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7de0875740 20 client.18798 populate_metadata read hostname 'smithi115' 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7de0875740 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5630beea11c0 0x5630bf336a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7de0875740 4 client.18798 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7de0875740 1 -- 172.21.15.115:0/2245608960 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630bf21d240 con 0x5630beea11c0 2024-05-09T05:54:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5630beea11c0 0x5630bf336a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7db2ffd640 10 client.18798 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.147+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5630beea11c0 0x5630bf336a50 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db7f043c71e5a0a0 server_cookie=50f8be00312245a7 in_seq=0 out_seq=0 2024-05-09T05:54:11.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:11 smithi115 ceph-mon[30349]: pgmap v504: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-05-09T05:54:11.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:11 smithi161 ceph-mon[33805]: pgmap v504: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-05-09T05:54:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7db2ffd640 1 -- 172.21.15.115:0/2245608960 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5630bf21d240 con 0x5630beea11c0 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7db2ffd640 10 client.18798 handle_command_reply: tid=0 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7de0875740 1 client.18798 shutdown 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7de0875740 1 -- 172.21.15.115:0/2245608960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 msgr2=0x5630bf2bdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7de0875740 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bdb80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4011de0 tx=0x7f7dc401d6b0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7de0875740 1 -- 172.21.15.115:0/2245608960 shutdown_connections 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7de0875740 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5630beea11c0 0x5630bf336a50 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7db2ffd640 0 client.18798 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7de0875740 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5630bef0d1b0 0x5630bf2be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7de0875740 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5630beeec4d0 0x5630bf2be0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7de0875740 1 --2- 172.21.15.115:0/2245608960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5630beee42f0 0x5630bf2bdb80 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.787+0000 7f7de0875740 1 -- 172.21.15.115:0/2245608960 >> 172.21.15.115:0/2245608960 conn(0x5630bef01300 msgr2=0x5630bf2b8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.788+0000 7f7de0875740 1 -- 172.21.15.115:0/2245608960 shutdown_connections 2024-05-09T05:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.788+0000 7f7de0875740 1 -- 172.21.15.115:0/2245608960 wait complete. 2024-05-09T05:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.788+0000 7f7de0875740 20 client.18798 trim_cache size 0 max 16384 2024-05-09T05:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 -- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7dbc0b2a20 msgr2=0x7f7dbc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7dbc0b2a20 0x7f7dbc0b4f30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc04c000 tx=0x7f7dcc019000 comp rx=0 tx=0).stop 2024-05-09T05:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 -- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd411a8f0 msgr2=0x7f7dd4116380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd411a8f0 0x7f7dd4116380 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c8a0 tx=0x7f7dc800cd60 comp rx=0 tx=0).stop 2024-05-09T05:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 -- 172.21.15.115:0/1145748146 shutdown_connections 2024-05-09T05:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f7dbc0b2a20 0x7f7dbc0b4f30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7dd411d250 0x7f7dd41168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd411a8f0 0x7f7dd4116380 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 --2- 172.21.15.115:0/1145748146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd410a200 0x7f7dd411a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.793+0000 7f7dd99c7640 1 -- 172.21.15.115:0/1145748146 >> 172.21.15.115:0/1145748146 conn(0x7f7dd406e900 msgr2=0x7f7dd411ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.796+0000 7f7dd99c7640 1 -- 172.21.15.115:0/1145748146 shutdown_connections 2024-05-09T05:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:11.796+0000 7f7dd99c7640 1 -- 172.21.15.115:0/1145748146 wait complete. 2024-05-09T05:54:11.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:11 smithi195 ceph-mon[33844]: pgmap v504: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-05-09T05:54:13.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:13 smithi115 ceph-mon[30349]: pgmap v505: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-05-09T05:54:13.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:13 smithi161 ceph-mon[33805]: pgmap v505: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-05-09T05:54:13.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:13 smithi195 ceph-mon[33844]: pgmap v505: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 3.59k op/s 2024-05-09T05:54:15.386 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:15 smithi115 ceph-mon[30349]: pgmap v506: 129 pgs: 36 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 78 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-05-09T05:54:15.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:15 smithi161 ceph-mon[33805]: pgmap v506: 129 pgs: 36 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 78 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-05-09T05:54:15.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:15 smithi195 ceph-mon[33844]: pgmap v506: 129 pgs: 36 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 78 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-05-09T05:54:17.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:17 smithi115 ceph-mon[30349]: pgmap v507: 129 pgs: 38 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 74 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-05-09T05:54:17.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:17 smithi161 ceph-mon[33805]: pgmap v507: 129 pgs: 38 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 74 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-05-09T05:54:17.662 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:17 smithi195 ceph-mon[33844]: pgmap v507: 129 pgs: 38 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 74 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-05-09T05:54:19.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:19 smithi161 ceph-mon[33805]: pgmap v508: 129 pgs: 38 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 76 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 3.07k op/s 2024-05-09T05:54:19.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:19 smithi195 ceph-mon[33844]: pgmap v508: 129 pgs: 38 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 76 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 3.07k op/s 2024-05-09T05:54:19.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:19 smithi115 ceph-mon[30349]: pgmap v508: 129 pgs: 38 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 76 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 3.07k op/s 2024-05-09T05:54:21.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:21 smithi161 ceph-mon[33805]: pgmap v509: 129 pgs: 24 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 92 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-09T05:54:21.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:21 smithi195 ceph-mon[33844]: pgmap v509: 129 pgs: 24 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 92 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-09T05:54:21.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:21 smithi115 ceph-mon[30349]: pgmap v509: 129 pgs: 24 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 92 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-09T05:54:23.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:23 smithi161 ceph-mon[33805]: pgmap v510: 129 pgs: 24 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 92 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-09T05:54:23.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:23 smithi195 ceph-mon[33844]: pgmap v510: 129 pgs: 24 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 92 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-09T05:54:23.896 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:23 smithi115 ceph-mon[30349]: pgmap v510: 129 pgs: 24 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 92 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-09T05:54:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.944+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/4035453374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f58104420 msgr2=0x7f0f581107f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.944+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/4035453374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f58104420 0x7f0f581107f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009ca0 tx=0x7f0f5002a8c0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/4035453374 shutdown_connections 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/4035453374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f58104420 0x7f0f581107f0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/4035453374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5810a170 0x7f0f58103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/4035453374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f58109cd0 0x7f0f58103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/4035453374 >> 172.21.15.115:0/4035453374 conn(0x7f0f580793d0 msgr2=0x7f0f58079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/4035453374 shutdown_connections 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/4035453374 wait complete. 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 Processor -- start 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 -- start start 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f58104420 0x7f0f5819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.945+0000 7f0f5d6d9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f58109cd0 0x7f0f5819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f5d6d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5810a170 0x7f0f5819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f5d6d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58113de0 con 0x7f0f5810a170 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f5d6d9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58113c60 con 0x7f0f58109cd0 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f5d6d9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58113f60 con 0x7f0f58104420 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f577fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5810a170 0x7f0f5819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f577fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5810a170 0x7f0f5819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44624/0 (socket says 172.21.15.115:44624) 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f577fe640 1 -- 172.21.15.115:0/1663481147 learned_addr learned my addr 172.21.15.115:0/1663481147 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f58104420 0x7f0f5819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f567fc640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f58109cd0 0x7f0f5819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f577fe640 1 -- 172.21.15.115:0/1663481147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f58104420 msgr2=0x7f0f5819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f577fe640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f58104420 0x7f0f5819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f577fe640 1 -- 172.21.15.115:0/1663481147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f58109cd0 msgr2=0x7f0f5819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f577fe640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f58109cd0 0x7f0f5819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f577fe640 1 -- 172.21.15.115:0/1663481147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5819f600 con 0x7f0f5810a170 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f58104420 0x7f0f5819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f567fc640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f58109cd0 0x7f0f5819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:54:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f577fe640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5810a170 0x7f0f5819efc0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005cc0 tx=0x7f0f50003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f4ffff640 1 -- 172.21.15.115:0/1663481147 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50032c10 con 0x7f0f5810a170 2024-05-09T05:54:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f4ffff640 1 -- 172.21.15.115:0/1663481147 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f50032db0 con 0x7f0f5810a170 2024-05-09T05:54:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/1663481147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5819f890 con 0x7f0f5810a170 2024-05-09T05:54:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f4ffff640 1 -- 172.21.15.115:0/1663481147 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50030a60 con 0x7f0f5810a170 2024-05-09T05:54:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.946+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/1663481147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5819fde0 con 0x7f0f5810a170 2024-05-09T05:54:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.949+0000 7f0f64594740 1 Processor -- start 2024-05-09T05:54:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.949+0000 7f0f4ffff640 1 -- 172.21.15.115:0/1663481147 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0f50030c20 con 0x7f0f5810a170 2024-05-09T05:54:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.949+0000 7f0f64594740 1 -- start start 2024-05-09T05:54:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.949+0000 7f0f64594740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926521b0 0x55d9929f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.949+0000 7f0f64594740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926292f0 0x55d9929f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.949+0000 7f0f64594740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.949+0000 7f0f64594740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d99295a7c0 con 0x55d9926292f0 2024-05-09T05:54:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f64594740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d9928f60e0 con 0x55d9926521b0 2024-05-09T05:54:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f64594740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d992956780 con 0x55d9926314d0 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926292f0 0x55d9929f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926292f0 0x55d9929f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44626/0 (socket says 172.21.15.115:44626) 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f56ffd640 1 -- 172.21.15.115:0/1273757309 learned_addr learned my addr 172.21.15.115:0/1273757309 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f567fc640 1 --2- 172.21.15.115:0/1273757309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926521b0 0x55d9929f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f56ffd640 1 -- 172.21.15.115:0/1273757309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 msgr2=0x55d9929f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/1273757309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f56ffd640 1 -- 172.21.15.115:0/1273757309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926521b0 msgr2=0x55d9929f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/1273757309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926521b0 0x55d9929f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f56ffd640 1 -- 172.21.15.115:0/1273757309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99291f7c0 con 0x55d9926292f0 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f567fc640 1 --2- 172.21.15.115:0/1273757309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f4ffff640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f440b2950 0x7f0f440b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/1273757309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926292f0 0x55d9929f5bc0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0f480114f0 tx=0x7f0f480119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783dfc04f9e32c07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f567fc640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f440b2950 0x7f0f440b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f4d7fa640 1 -- 172.21.15.115:0/1273757309 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48011250 con 0x55d9926292f0 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f4ffff640 1 -- 172.21.15.115:0/1663481147 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f0f50103c00 con 0x7f0f5810a170 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f4d7fa640 1 -- 172.21.15.115:0/1273757309 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f480035f0 con 0x55d9926292f0 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.950+0000 7f0f4d7fa640 1 -- 172.21.15.115:0/1273757309 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800f7c0 con 0x55d9926292f0 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f64594740 1 -- 172.21.15.115:0/1273757309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926292f0 msgr2=0x55d9929f5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f567fc640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f440b2950 0x7f0f440b4e60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55d9929f27f0 tx=0x7f0f40005f90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f64594740 1 --2- 172.21.15.115:0/1273757309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926292f0 0x55d9929f5bc0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0f480114f0 tx=0x7f0f480119b0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f64594740 1 -- 172.21.15.115:0/1273757309 shutdown_connections 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f64594740 1 --2- 172.21.15.115:0/1273757309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f64594740 1 --2- 172.21.15.115:0/1273757309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926292f0 0x55d9929f5bc0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f64594740 1 --2- 172.21.15.115:0/1273757309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926521b0 0x55d9929f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f64594740 1 -- 172.21.15.115:0/1273757309 >> 172.21.15.115:0/1273757309 conn(0x55d992646300 msgr2=0x55d9929f1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f64594740 1 -- 172.21.15.115:0/1273757309 shutdown_connections 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.951+0000 7f0f64594740 1 -- 172.21.15.115:0/1273757309 wait complete. 2024-05-09T05:54:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f64594740 1 -- start start 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f64594740 1 Processor -- start 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f64594740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926292f0 0x55d9929ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f64594740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926314d0 0x55d992a00280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f64594740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926521b0 0x55d992a05ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f64594740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55d992a027b0 con 0x55d9926314d0 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f64594740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d9928e19c0 con 0x55d9926292f0 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f64594740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d992a02930 con 0x55d9926521b0 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f64594740 10 client.0 _ll_register_callbacks cb 0x55d99290e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926314d0 0x55d992a00280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926314d0 0x55d992a00280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44642/0 (socket says 172.21.15.115:44642) 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f567fc640 1 -- 172.21.15.115:0/1285070400 learned_addr learned my addr 172.21.15.115:0/1285070400 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926292f0 0x55d9929ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f577fe640 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926521b0 0x55d992a05ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f567fc640 1 -- 172.21.15.115:0/1285070400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926521b0 msgr2=0x55d992a05ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f567fc640 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926521b0 0x55d992a05ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f567fc640 1 -- 172.21.15.115:0/1285070400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926292f0 msgr2=0x55d9929ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f567fc640 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926292f0 0x55d9929ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.952+0000 7f0f567fc640 1 -- 172.21.15.115:0/1285070400 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9929081f0 con 0x55d9926314d0 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f567fc640 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926314d0 0x55d992a00280 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f0f40003f80 tx=0x7f0f40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b91667a5fa1b4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/1285070400 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40030060 con 0x55d9926314d0 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/1285070400 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f0f4001a050 con 0x55d9926314d0 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f64594740 1 -- 172.21.15.115:0/1285070400 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f480035f0 con 0x55d9926314d0 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/1285070400 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f0f40037050 con 0x55d9926314d0 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f64594740 10 client.18810 fetch_fsmap learned FSMap version 62 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f64594740 1 -- 172.21.15.115:0/1285070400 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55d9926bd790 con 0x55d9926314d0 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/1285070400 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f480035f0 con 0x55d9926314d0 2024-05-09T05:54:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/1285070400 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f0f40020400 con 0x55d9926314d0 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f64594740 10 client.18810 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f64594740 10 client.18810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f64594740 20 client.18810 populate_metadata read hostname 'smithi115' 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f64594740 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d992610d80 0x55d992a12070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f64594740 4 client.18810 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.953+0000 7f0f64594740 1 -- 172.21.15.115:0/1285070400 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9929683d0 con 0x55d992610d80 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.954+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d992610d80 0x55d992a12070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.954+0000 7f0f3e7fc640 10 client.18810 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.954+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d992610d80 0x55d992a12070 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c446a000604d99d1 server_cookie=531f2728780c6711 in_seq=0 out_seq=0 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/1285070400 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d9929683d0 con 0x55d992610d80 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f3e7fc640 10 client.18810 handle_command_reply: tid=0 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 client.18810 shutdown 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 -- 172.21.15.115:0/1285070400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926314d0 msgr2=0x55d992a00280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926314d0 0x55d992a00280 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f0f40003f80 tx=0x7f0f40004730 comp rx=0 tx=0).stop 2024-05-09T05:54:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 -- 172.21.15.115:0/1285070400 shutdown_connections 2024-05-09T05:54:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d992610d80 0x55d992a12070 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926521b0 0x55d992a05ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f3e7fc640 0 client.18810 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d9926314d0 0x55d992a00280 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 --2- 172.21.15.115:0/1285070400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926292f0 0x55d9929ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 -- 172.21.15.115:0/1285070400 >> 172.21.15.115:0/1285070400 conn(0x55d992646300 msgr2=0x55d9929f0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 -- 172.21.15.115:0/1285070400 shutdown_connections 2024-05-09T05:54:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 1 -- 172.21.15.115:0/1285070400 wait complete. 2024-05-09T05:54:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.955+0000 7f0f64594740 20 client.18810 trim_cache size 0 max 16384 2024-05-09T05:54:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.982+0000 7f0f64594740 1 Processor -- start 2024-05-09T05:54:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.982+0000 7f0f64594740 1 -- start start 2024-05-09T05:54:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.983+0000 7f0f64594740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d9929f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.983+0000 7f0f64594740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926292f0 0x55d9929f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.983+0000 7f0f64594740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.983+0000 7f0f64594740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581b1210 con 0x55d992610d80 2024-05-09T05:54:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.983+0000 7f0f64594740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581b2d80 con 0x55d9926292f0 2024-05-09T05:54:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.983+0000 7f0f64594740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581b18e0 con 0x55d9926314d0 2024-05-09T05:54:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.984+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d9929f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.984+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d9929f0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44652/0 (socket says 172.21.15.115:44652) 2024-05-09T05:54:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.984+0000 7f0f56ffd640 1 -- 172.21.15.115:0/2794349525 learned_addr learned my addr 172.21.15.115:0/2794349525 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.985+0000 7f0f577fe640 1 --2- 172.21.15.115:0/2794349525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.986+0000 7f0f577fe640 1 -- 172.21.15.115:0/2794349525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926292f0 msgr2=0x55d9929f93e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:54:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.986+0000 7f0f577fe640 1 --2- 172.21.15.115:0/2794349525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926292f0 0x55d9929f93e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.986+0000 7f0f577fe640 1 -- 172.21.15.115:0/2794349525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 msgr2=0x55d9929f0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.986+0000 7f0f577fe640 1 --2- 172.21.15.115:0/2794349525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d9929f0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.986+0000 7f0f577fe640 1 -- 172.21.15.115:0/2794349525 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9927345f0 con 0x55d9926314d0 2024-05-09T05:54:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.987+0000 7f0f577fe640 1 --2- 172.21.15.115:0/2794349525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f9920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f50101020 tx=0x7f0f50030f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee99a32397241b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.987+0000 7f0f4cff9640 1 -- 172.21.15.115:0/2794349525 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f500cb2e0 con 0x55d9926314d0 2024-05-09T05:54:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.988+0000 7f0f4cff9640 1 -- 172.21.15.115:0/2794349525 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f50036070 con 0x55d9926314d0 2024-05-09T05:54:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.988+0000 7f0f4cff9640 1 -- 172.21.15.115:0/2794349525 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f500d3a40 con 0x55d9926314d0 2024-05-09T05:54:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.989+0000 7f0f64594740 1 -- 172.21.15.115:0/2794349525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 msgr2=0x55d9929f9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.989+0000 7f0f64594740 1 --2- 172.21.15.115:0/2794349525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f9920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f50101020 tx=0x7f0f50030f90 comp rx=0 tx=0).stop 2024-05-09T05:54:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.989+0000 7f0f64594740 1 -- 172.21.15.115:0/2794349525 shutdown_connections 2024-05-09T05:54:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.989+0000 7f0f64594740 1 --2- 172.21.15.115:0/2794349525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926314d0 0x55d9929f9920 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.989+0000 7f0f64594740 1 --2- 172.21.15.115:0/2794349525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926292f0 0x55d9929f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.989+0000 7f0f64594740 1 --2- 172.21.15.115:0/2794349525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d9929f0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.989+0000 7f0f64594740 1 -- 172.21.15.115:0/2794349525 >> 172.21.15.115:0/2794349525 conn(0x55d992646300 msgr2=0x55d9929f0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.989+0000 7f0f64594740 1 -- 172.21.15.115:0/2794349525 shutdown_connections 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.990+0000 7f0f64594740 1 -- 172.21.15.115:0/2794349525 wait complete. 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.990+0000 7f0f64594740 1 -- start start 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.990+0000 7f0f64594740 1 Processor -- start 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f64594740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d992a75090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f64594740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926292f0 0x55d992a755d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f64594740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926314d0 0x55d992a0bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f64594740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581b26b0 con 0x55d992610d80 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f64594740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581b1fe0 con 0x55d9926314d0 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f64594740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581af210 con 0x55d9926292f0 2024-05-09T05:54:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f64594740 10 client.0 _ll_register_callbacks cb 0x7f0f40011f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d992a75090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d992a75090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44660/0 (socket says 172.21.15.115:44660) 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f56ffd640 1 -- 172.21.15.115:0/2295416566 learned_addr learned my addr 172.21.15.115:0/2295416566 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f577fe640 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926314d0 0x55d992a0bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.991+0000 7f0f567fc640 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926292f0 0x55d992a755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f577fe640 1 -- 172.21.15.115:0/2295416566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926292f0 msgr2=0x55d992a755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f577fe640 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926292f0 0x55d992a755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f577fe640 1 -- 172.21.15.115:0/2295416566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 msgr2=0x55d992a75090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f577fe640 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d992a75090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f577fe640 1 -- 172.21.15.115:0/2295416566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99291eed0 con 0x55d9926314d0 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f577fe640 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926314d0 0x55d992a0bd00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f50101050 tx=0x7f0f500c6bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f4ec1f6ba2b7a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/2295416566 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f500e9020 con 0x55d9926314d0 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/2295416566 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f0f500cae40 con 0x55d9926314d0 2024-05-09T05:54:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f64594740 1 -- 172.21.15.115:0/2295416566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f50036070 con 0x55d9926314d0 2024-05-09T05:54:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/2295416566 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f0f500e3070 con 0x55d9926314d0 2024-05-09T05:54:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f64594740 10 client.28019 fetch_fsmap learned FSMap version 62 2024-05-09T05:54:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.992+0000 7f0f64594740 1 -- 172.21.15.115:0/2295416566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55d992a07940 con 0x55d9926314d0 2024-05-09T05:54:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.993+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/2295416566 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f50036070 con 0x55d9926314d0 2024-05-09T05:54:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.993+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/2295416566 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 62) ==== 8382+0+0 (secure 0 0 0) 0x7f0f500a7030 con 0x55d9926314d0 2024-05-09T05:54:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.993+0000 7f0f64594740 10 client.28019 fetch_fsmap finished waiting for FSMap version 62 2024-05-09T05:54:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.993+0000 7f0f64594740 10 client.28019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:54:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.993+0000 7f0f64594740 20 client.28019 populate_metadata read hostname 'smithi115' 2024-05-09T05:54:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.993+0000 7f0f64594740 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d9926521b0 0x55d992a85420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.993+0000 7f0f64594740 4 client.28019 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:54:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.993+0000 7f0f64594740 1 -- 172.21.15.115:0/2295416566 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d992962240 con 0x55d9926521b0 2024-05-09T05:54:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.993+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d9926521b0 0x55d992a85420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.994+0000 7f0f3e7fc640 10 client.28019 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:24.994+0000 7f0f56ffd640 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d9926521b0 0x55d992a85420 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c796f6cdf41b85 server_cookie=703c9af874d3b0b4 in_seq=0 out_seq=0 2024-05-09T05:54:25.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:25 smithi195 ceph-mon[33844]: pgmap v511: 129 pgs: 15 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 103 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-05-09T05:54:25.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:25 smithi115 ceph-mon[30349]: pgmap v511: 129 pgs: 15 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 103 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-05-09T05:54:25.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:25 smithi161 ceph-mon[33805]: pgmap v511: 129 pgs: 15 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 103 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2024-05-09T05:54:27.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:27 smithi195 ceph-mon[33844]: pgmap v512: 129 pgs: 15 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 103 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-05-09T05:54:27.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:27 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:54:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.877+0000 7f0f3e7fc640 1 -- 172.21.15.115:0/2295416566 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d992962240 con 0x55d9926521b0 2024-05-09T05:54:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.877+0000 7f0f3e7fc640 10 client.28019 handle_command_reply: tid=0 2024-05-09T05:54:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.877+0000 7f0f64594740 1 client.28019 shutdown 2024-05-09T05:54:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.878+0000 7f0f64594740 1 -- 172.21.15.115:0/2295416566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926314d0 msgr2=0x55d992a0bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.878+0000 7f0f64594740 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926314d0 0x55d992a0bd00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f50101050 tx=0x7f0f500c6bd0 comp rx=0 tx=0).stop 2024-05-09T05:54:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.879+0000 7f0f64594740 1 -- 172.21.15.115:0/2295416566 shutdown_connections 2024-05-09T05:54:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.879+0000 7f0f64594740 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9926314d0 0x55d992a0bd00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.879+0000 7f0f64594740 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55d9926521b0 0x55d992a85420 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.879+0000 7f0f64594740 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d9926292f0 0x55d992a755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.879+0000 7f0f64594740 1 --2- 172.21.15.115:0/2295416566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55d992610d80 0x55d992a75090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.879+0000 7f0f64594740 1 -- 172.21.15.115:0/2295416566 >> 172.21.15.115:0/2295416566 conn(0x55d992646300 msgr2=0x55d9929fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.879+0000 7f0f3e7fc640 0 client.28019 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.881+0000 7f0f64594740 1 -- 172.21.15.115:0/2295416566 shutdown_connections 2024-05-09T05:54:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.882+0000 7f0f64594740 1 -- 172.21.15.115:0/2295416566 wait complete. 2024-05-09T05:54:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.885+0000 7f0f64594740 20 client.28019 trim_cache size 0 max 16384 2024-05-09T05:54:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:54:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:54:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:54:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.887+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f440b2950 msgr2=0x7f0f440b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.887+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f440b2950 0x7f0f440b4e60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55d9929f27f0 tx=0x7f0f40005f90 comp rx=0 tx=0).stop 2024-05-09T05:54:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.887+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5810a170 msgr2=0x7f0f5819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.887+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5810a170 0x7f0f5819efc0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0f50005cc0 tx=0x7f0f50003f90 comp rx=0 tx=0).stop 2024-05-09T05:54:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.888+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/1663481147 shutdown_connections 2024-05-09T05:54:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.888+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f440b2950 0x7f0f440b4e60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.888+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5810a170 0x7f0f5819efc0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.888+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f58109cd0 0x7f0f5819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.888+0000 7f0f5d6d9640 1 --2- 172.21.15.115:0/1663481147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f58104420 0x7f0f5819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.888+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/1663481147 >> 172.21.15.115:0/1663481147 conn(0x7f0f580793d0 msgr2=0x7f0f58102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:27.878 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.889+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/1663481147 shutdown_connections 2024-05-09T05:54:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:27.889+0000 7f0f5d6d9640 1 -- 172.21.15.115:0/1663481147 wait complete. 2024-05-09T05:54:27.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:27 smithi115 ceph-mon[30349]: pgmap v512: 129 pgs: 15 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 103 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-05-09T05:54:27.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:27 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:54:27.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:27 smithi161 ceph-mon[33805]: pgmap v512: 129 pgs: 15 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 103 active+clean; 14 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-05-09T05:54:27.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:27 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:54:29.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:29 smithi195 ceph-mon[33844]: pgmap v513: 129 pgs: 9 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 110 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-05-09T05:54:29.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:29 smithi115 ceph-mon[30349]: pgmap v513: 129 pgs: 9 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 110 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-05-09T05:54:29.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:29 smithi161 ceph-mon[33805]: pgmap v513: 129 pgs: 9 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 110 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-05-09T05:54:31.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:31 smithi195 ceph-mon[33844]: pgmap v514: 129 pgs: 9 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 110 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:54:31.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:31 smithi115 ceph-mon[30349]: pgmap v514: 129 pgs: 9 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 110 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:54:31.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:31 smithi161 ceph-mon[33805]: pgmap v514: 129 pgs: 9 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 110 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:54:33.758 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:33 smithi115 ceph-mon[30349]: pgmap v515: 129 pgs: 9 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 110 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.5 MiB/s wr, 2.43k op/s 2024-05-09T05:54:33.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:33 smithi195 ceph-mon[33844]: pgmap v515: 129 pgs: 9 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 110 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.5 MiB/s wr, 2.43k op/s 2024-05-09T05:54:33.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:33 smithi161 ceph-mon[33805]: pgmap v515: 129 pgs: 9 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 110 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.5 MiB/s wr, 2.43k op/s 2024-05-09T05:54:35.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:35 smithi115 ceph-mon[30349]: pgmap v516: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2024-05-09T05:54:35.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:35 smithi195 ceph-mon[33844]: pgmap v516: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2024-05-09T05:54:35.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:35 smithi161 ceph-mon[33805]: pgmap v516: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2024-05-09T05:54:37.722 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:37 smithi115 ceph-mon[30349]: pgmap v517: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-05-09T05:54:37.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:37 smithi195 ceph-mon[33844]: pgmap v517: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-05-09T05:54:37.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:37 smithi161 ceph-mon[33805]: pgmap v517: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-05-09T05:54:38.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.330+0000 7f92acbdf700 1 -- 172.21.15.115:0/327365654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a810c840 msgr2=0x7f92a810cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:38.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.330+0000 7f92acbdf700 1 --2- 172.21.15.115:0/327365654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a810c840 0x7f92a810cc50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9298009990 tx=0x7f9298009ca0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.333+0000 7f92acbdf700 1 -- 172.21.15.115:0/327365654 shutdown_connections 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.333+0000 7f92acbdf700 1 --2- 172.21.15.115:0/327365654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a8111fc0 0x7f92a810b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.333+0000 7f92acbdf700 1 --2- 172.21.15.115:0/327365654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a810c840 0x7f92a810cc50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.333+0000 7f92acbdf700 1 --2- 172.21.15.115:0/327365654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8111670 0x7f92a8111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.333+0000 7f92acbdf700 1 -- 172.21.15.115:0/327365654 >> 172.21.15.115:0/327365654 conn(0x7f92a806f290 msgr2=0x7f92a806f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.333+0000 7f92acbdf700 1 -- 172.21.15.115:0/327365654 shutdown_connections 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.334+0000 7f92acbdf700 1 -- 172.21.15.115:0/327365654 wait complete. 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92acbdf700 1 Processor -- start 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92acbdf700 1 -- start start 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92acbdf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a810c840 0x7f92a81211a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92acbdf700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a8111670 0x7f92a8121700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92acbdf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8111fc0 0x7f92a8126ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92acbdf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81140e0 con 0x7f92a810c840 2024-05-09T05:54:38.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92acbdf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8113f60 con 0x7f92a8111fc0 2024-05-09T05:54:38.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92acbdf700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8114260 con 0x7f92a8111670 2024-05-09T05:54:38.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92a77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a810c840 0x7f92a81211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:38.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92a77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a810c840 0x7f92a81211a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39728/0 (socket says 172.21.15.115:39728) 2024-05-09T05:54:38.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92a77fe700 1 -- 172.21.15.115:0/283849955 learned_addr learned my addr 172.21.15.115:0/283849955 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:38.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92a6ffd700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a8111670 0x7f92a8121700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:38.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.335+0000 7f92a7fff700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8111fc0 0x7f92a8126ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:38.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.336+0000 7f92a6ffd700 1 -- 172.21.15.115:0/283849955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8111fc0 msgr2=0x7f92a8126ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:38.834 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:38 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/283849955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:54:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:54:38.835 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:54:25.586886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24440,24425,24451,24469,24457]},"id":1}]} 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.336+0000 7f92a6ffd700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8111fc0 0x7f92a8126ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.336+0000 7f92a6ffd700 1 -- 172.21.15.115:0/283849955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a810c840 msgr2=0x7f92a81211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.336+0000 7f92a6ffd700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a810c840 0x7f92a81211a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.336+0000 7f92a6ffd700 1 -- 172.21.15.115:0/283849955 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a81276f0 con 0x7f92a8111670 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.336+0000 7f92a6ffd700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a8111670 0x7f92a8121700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f929c00e960 tx=0x7f929c00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.337+0000 7f92a4ff9700 1 -- 172.21.15.115:0/283849955 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c00c960 con 0x7f92a8111670 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.337+0000 7f92a4ff9700 1 -- 172.21.15.115:0/283849955 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f929c011470 con 0x7f92a8111670 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.337+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a81279e0 con 0x7f92a8111670 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.337+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a811c6a0 con 0x7f92a8111670 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.337+0000 7f92a4ff9700 1 -- 172.21.15.115:0/283849955 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c01a660 con 0x7f92a8111670 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.339+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a810ae60 con 0x7f92a8111670 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.340+0000 7f92a4ff9700 1 -- 172.21.15.115:0/283849955 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f929c00cb00 con 0x7f92a8111670 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.341+0000 7f92a4ff9700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b29f0 0x7f92900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.341+0000 7f92a77fe700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b29f0 0x7f92900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.342+0000 7f92a4ff9700 1 -- 172.21.15.115:0/283849955 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f929c0e2a50 con 0x7f92a8111670 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.342+0000 7f92a77fe700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b29f0 0x7f92900b4f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9298009750 tx=0x7f9298005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.346+0000 7f92a4ff9700 1 -- 172.21.15.115:0/283849955 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f929c016030 con 0x7f92a8111670 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.733+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92a811dc80 con 0x7f92a8111670 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.735+0000 7f92a4ff9700 1 -- 172.21.15.115:0/283849955 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11955 (secure 0 0 0) 0x7f929c0aa020 con 0x7f92a8111670 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b29f0 msgr2=0x7f92900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b29f0 0x7f92900b4f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9298009750 tx=0x7f9298005fb0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a8111670 msgr2=0x7f92a8121700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a8111670 0x7f92a8121700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f929c00e960 tx=0x7f929c00ec70 comp rx=0 tx=0).stop 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 shutdown_connections 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f92900b29f0 0x7f92900b4f00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92a8111fc0 0x7f92a8126ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92a810c840 0x7f92a81211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 --2- 172.21.15.115:0/283849955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92a8111670 0x7f92a8121700 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f929c00e960 tx=0x7f929c00ec70 comp rx=0 tx=0).stop 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.740+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 >> 172.21.15.115:0/283849955 conn(0x7f92a806f290 msgr2=0x7f92a810f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.748+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 shutdown_connections 2024-05-09T05:54:38.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:54:37.748+0000 7f92acbdf700 1 -- 172.21.15.115:0/283849955 wait complete. 2024-05-09T05:54:38.838 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 63 2024-05-09T05:54:38.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:38 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/283849955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:54:38.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:38 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/283849955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:54:39.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:39 smithi195 ceph-mon[33844]: pgmap v518: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-05-09T05:54:39.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:39 smithi115 ceph-mon[30349]: pgmap v518: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-05-09T05:54:39.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:39 smithi161 ceph-mon[33805]: pgmap v518: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-05-09T05:54:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.134+0000 7f65de784640 1 -- 172.21.15.115:0/2655105252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65d8104260 msgr2=0x7f65d81105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.134+0000 7f65de784640 1 --2- 172.21.15.115:0/2655105252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65d8104260 0x7f65d81105e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f65d000acb0 tx=0x7f65d002a940 comp rx=0 tx=0).stop 2024-05-09T05:54:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.134+0000 7f65de784640 1 -- 172.21.15.115:0/2655105252 shutdown_connections 2024-05-09T05:54:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.134+0000 7f65de784640 1 --2- 172.21.15.115:0/2655105252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65d8104260 0x7f65d81105e0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.134+0000 7f65de784640 1 --2- 172.21.15.115:0/2655105252 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d8109f60 0x7f65d8103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.134+0000 7f65de784640 1 --2- 172.21.15.115:0/2655105252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d8109ac0 0x7f65d81036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.134+0000 7f65de784640 1 -- 172.21.15.115:0/2655105252 >> 172.21.15.115:0/2655105252 conn(0x7f65d8079250 msgr2=0x7f65d8079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.134+0000 7f65de784640 1 -- 172.21.15.115:0/2655105252 shutdown_connections 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.134+0000 7f65de784640 1 -- 172.21.15.115:0/2655105252 wait complete. 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65de784640 1 Processor -- start 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65de784640 1 -- start start 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65de784640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d8104260 0x7f65d819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65de784640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d8109ac0 0x7f65d819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65de784640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65d8109f60 0x7f65d819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65de784640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8113b80 con 0x7f65d8109f60 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65de784640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8113a00 con 0x7f65d8109ac0 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65de784640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8113d00 con 0x7f65d8104260 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65dcd21640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65d8109f60 0x7f65d819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d8104260 0x7f65d819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d8104260 0x7f65d819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46344/0 (socket says 172.21.15.115:46344) 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65d7fff640 1 -- 172.21.15.115:0/236110753 learned_addr learned my addr 172.21.15.115:0/236110753 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.135+0000 7f65d77fe640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d8109ac0 0x7f65d819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65dcd21640 1 -- 172.21.15.115:0/236110753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d8104260 msgr2=0x7f65d819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65dcd21640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d8104260 0x7f65d819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65dcd21640 1 -- 172.21.15.115:0/236110753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d8109ac0 msgr2=0x7f65d819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65dcd21640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d8109ac0 0x7f65d819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65dcd21640 1 -- 172.21.15.115:0/236110753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d819f440 con 0x7f65d8109f60 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65dcd21640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65d8109f60 0x7f65d819ee00 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f65d002ae60 tx=0x7f65d002f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65d57fa640 1 -- 172.21.15.115:0/236110753 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d002fc70 con 0x7f65d8109f60 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65d57fa640 1 -- 172.21.15.115:0/236110753 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65d0040030 con 0x7f65d8109f60 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65de784640 1 -- 172.21.15.115:0/236110753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d819f6d0 con 0x7f65d8109f60 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65d57fa640 1 -- 172.21.15.115:0/236110753 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d003c420 con 0x7f65d8109f60 2024-05-09T05:54:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.136+0000 7f65de784640 1 -- 172.21.15.115:0/236110753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d819fc20 con 0x7f65d8109f60 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.138+0000 7f65e5638740 1 Processor -- start 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.138+0000 7f65d57fa640 1 -- 172.21.15.115:0/236110753 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f65d003b440 con 0x7f65d8109f60 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65e5638740 1 -- start start 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65e5638740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365f021b0 0x55f3662a5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65e5638740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ed92f0 0x55f3662a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65e5638740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65e5638740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f36620a7c0 con 0x55f365ed92f0 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65e5638740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f3661a60e0 con 0x55f365ee14d0 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65e5638740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f366206780 con 0x55f365f021b0 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ed92f0 0x55f3662a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ed92f0 0x55f3662a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39774/0 (socket says 172.21.15.115:39774) 2024-05-09T05:54:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d7fff640 1 -- 172.21.15.115:0/2478512391 learned_addr learned my addr 172.21.15.115:0/2478512391 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d77fe640 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365f021b0 0x55f3662a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d77fe640 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d7fff640 1 -- 172.21.15.115:0/2478512391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365f021b0 msgr2=0x55f3662a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d7fff640 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365f021b0 0x55f3662a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d7fff640 1 -- 172.21.15.115:0/2478512391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 msgr2=0x55f3662a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d7fff640 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d7fff640 1 -- 172.21.15.115:0/2478512391 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3661cf7c0 con 0x55f365ed92f0 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d77fe640 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d57fa640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f65c80b2950 0x7f65c80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.139+0000 7f65d7fff640 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ed92f0 0x55f3662a5d80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009d60 tx=0x7f65cc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307178ba91950a66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.140+0000 7f65d77fe640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f65c80b2950 0x7f65c80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.140+0000 7f65c27fc640 1 -- 172.21.15.115:0/2478512391 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc004220 con 0x55f365ed92f0 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.140+0000 7f65d57fa640 1 -- 172.21.15.115:0/236110753 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f65d0103dd0 con 0x7f65d8109f60 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.140+0000 7f65c27fc640 1 -- 172.21.15.115:0/2478512391 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65cc0043c0 con 0x55f365ed92f0 2024-05-09T05:54:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.140+0000 7f65c27fc640 1 -- 172.21.15.115:0/2478512391 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc019830 con 0x55f365ed92f0 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 -- 172.21.15.115:0/2478512391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ed92f0 msgr2=0x55f3662a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ed92f0 0x55f3662a5d80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009d60 tx=0x7f65cc011810 comp rx=0 tx=0).stop 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65d77fe640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f65c80b2950 0x7f65c80b4e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55f3662a29b0 tx=0x7f65c4013040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 -- 172.21.15.115:0/2478512391 shutdown_connections 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ed92f0 0x55f3662a5d80 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 --2- 172.21.15.115:0/2478512391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365f021b0 0x55f3662a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 -- 172.21.15.115:0/2478512391 >> 172.21.15.115:0/2478512391 conn(0x55f365ef6300 msgr2=0x55f3662a1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 -- 172.21.15.115:0/2478512391 shutdown_connections 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 -- 172.21.15.115:0/2478512391 wait complete. 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 -- start start 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 Processor -- start 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ed92f0 0x55f3662aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365f021b0 0x55f3662b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f3662b29c0 con 0x55f365f021b0 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f3661919c0 con 0x55f365ee14d0 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.141+0000 7f65e5638740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f3662b2b40 con 0x55f365ed92f0 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65e5638740 10 client.0 _ll_register_callbacks cb 0x55f3661be020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ed92f0 0x55f3662aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ed92f0 0x55f3662aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46370/0 (socket says 172.21.15.115:46370) 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d7fff640 1 -- 172.21.15.115:0/975555931 learned_addr learned my addr 172.21.15.115:0/975555931 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65dcd21640 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365f021b0 0x55f3662b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d77fe640 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d7fff640 1 -- 172.21.15.115:0/975555931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 msgr2=0x55f3662b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d7fff640 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d7fff640 1 -- 172.21.15.115:0/975555931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365f021b0 msgr2=0x55f3662b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d7fff640 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365f021b0 0x55f3662b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d7fff640 1 -- 172.21.15.115:0/975555931 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3661b81f0 con 0x55f365ed92f0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d77fe640 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65dcd21640 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365f021b0 0x55f3662b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65d7fff640 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ed92f0 0x55f3662aff00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65cc019490 tx=0x7f65cc003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b70cd6e47bc623f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65b77fe640 1 -- 172.21.15.115:0/975555931 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc028020 con 0x55f365ed92f0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65b77fe640 1 -- 172.21.15.115:0/975555931 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f65cc003a80 con 0x55f365ed92f0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.142+0000 7f65e5638740 1 -- 172.21.15.115:0/975555931 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65cc0043c0 con 0x55f365ed92f0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65b77fe640 1 -- 172.21.15.115:0/975555931 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f65cc033050 con 0x55f365ed92f0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65e5638740 10 client.28165 fetch_fsmap learned FSMap version 63 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65e5638740 1 -- 172.21.15.115:0/975555931 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f365f6d790 con 0x55f365ed92f0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65b77fe640 1 -- 172.21.15.115:0/975555931 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65cc0043c0 con 0x55f365ed92f0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65b77fe640 1 -- 172.21.15.115:0/975555931 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 63) ==== 8374+0+0 (secure 0 0 0) 0x7f65cc004c50 con 0x55f365ed92f0 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65e5638740 10 client.28165 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T05:54:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65e5638740 10 client.28165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65e5638740 20 client.28165 populate_metadata read hostname 'smithi115' 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65e5638740 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f365ec0d80 0x55f3662c2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65e5638740 4 client.28165 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65e5638740 1 -- 172.21.15.115:0/975555931 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3662183d0 con 0x55f365ec0d80 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.143+0000 7f65d77fe640 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f365ec0d80 0x55f3662c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65b77fe640 10 client.28165 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65d77fe640 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f365ec0d80 0x55f3662c2130 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7d0b734f048975 server_cookie=d9bb852e655ebac5 in_seq=0 out_seq=0 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65b77fe640 1 -- 172.21.15.115:0/975555931 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f3662183d0 con 0x55f365ec0d80 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65b77fe640 10 client.28165 handle_command_reply: tid=0 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65e5638740 1 client.28165 shutdown 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65e5638740 1 -- 172.21.15.115:0/975555931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ed92f0 msgr2=0x55f3662aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65e5638740 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ed92f0 0x55f3662aff00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65cc019490 tx=0x7f65cc003d60 comp rx=0 tx=0).stop 2024-05-09T05:54:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65e5638740 1 -- 172.21.15.115:0/975555931 shutdown_connections 2024-05-09T05:54:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65e5638740 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365f021b0 0x55f3662b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65e5638740 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ee14d0 0x55f3662b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.144+0000 7f65e5638740 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f365ec0d80 0x55f3662c2130 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.145+0000 7f65e5638740 1 --2- 172.21.15.115:0/975555931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ed92f0 0x55f3662aff00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.145+0000 7f65b77fe640 0 client.28165 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.145+0000 7f65e5638740 1 -- 172.21.15.115:0/975555931 >> 172.21.15.115:0/975555931 conn(0x55f365ef6300 msgr2=0x55f3662a0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.145+0000 7f65e5638740 1 -- 172.21.15.115:0/975555931 shutdown_connections 2024-05-09T05:54:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.145+0000 7f65e5638740 1 -- 172.21.15.115:0/975555931 wait complete. 2024-05-09T05:54:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.145+0000 7f65e5638740 20 client.28165 trim_cache size 0 max 16384 2024-05-09T05:54:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65e5638740 1 Processor -- start 2024-05-09T05:54:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65e5638740 1 -- start start 2024-05-09T05:54:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65e5638740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 0x55f3662a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65e5638740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662a2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65e5638740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 0x55f3662b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65e5638740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81b1090 con 0x55f365ec0d80 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 0x55f3662a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65d77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65e5638740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81b2c00 con 0x55f365ed92f0 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65d77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39798/0 (socket says 172.21.15.115:39798) 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 0x55f3662a9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46374/0 (socket says 172.21.15.115:46374) 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65d77fe640 1 -- 172.21.15.115:0/2369368855 learned_addr learned my addr 172.21.15.115:0/2369368855 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65e5638740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81b1760 con 0x55f365ef6300 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.160+0000 7f65dcd21640 1 --2- 172.21.15.115:0/2369368855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 0x55f3662b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.161+0000 7f65d77fe640 1 -- 172.21.15.115:0/2369368855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 msgr2=0x55f3662a9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.161+0000 7f65d77fe640 1 --2- 172.21.15.115:0/2369368855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 0x55f3662a9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.161+0000 7f65d77fe640 1 -- 172.21.15.115:0/2369368855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 msgr2=0x55f3662b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.161+0000 7f65d77fe640 1 --2- 172.21.15.115:0/2369368855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 0x55f3662b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.161+0000 7f65d77fe640 1 -- 172.21.15.115:0/2369368855 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f365fe45f0 con 0x55f365ec0d80 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.161+0000 7f65d77fe640 1 --2- 172.21.15.115:0/2369368855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662a2020 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f65d81002d0 tx=0x7f65c40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f4ba59c81c63d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.161+0000 7f65c1ffb640 1 -- 172.21.15.115:0/2369368855 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c400fdf0 con 0x55f365ec0d80 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.161+0000 7f65c1ffb640 1 -- 172.21.15.115:0/2369368855 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65c401fd10 con 0x55f365ec0d80 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.161+0000 7f65c1ffb640 1 -- 172.21.15.115:0/2369368855 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c402bae0 con 0x55f365ec0d80 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 -- 172.21.15.115:0/2369368855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 msgr2=0x55f3662a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 --2- 172.21.15.115:0/2369368855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662a2020 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f65d81002d0 tx=0x7f65c40096a0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 -- 172.21.15.115:0/2369368855 shutdown_connections 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 --2- 172.21.15.115:0/2369368855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 0x55f3662b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 --2- 172.21.15.115:0/2369368855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662a2020 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 --2- 172.21.15.115:0/2369368855 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 0x55f3662a9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 -- 172.21.15.115:0/2369368855 >> 172.21.15.115:0/2369368855 conn(0x55f3662ab670 msgr2=0x55f3662a1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 -- 172.21.15.115:0/2369368855 shutdown_connections 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 -- 172.21.15.115:0/2369368855 wait complete. 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 -- start start 2024-05-09T05:54:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 Processor -- start 2024-05-09T05:54:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 0x55f3662b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 0x55f3662b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.162+0000 7f65e5638740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81b2530 con 0x55f365ec0d80 2024-05-09T05:54:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65e5638740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81b1e60 con 0x55f365ed92f0 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65e5638740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81af070 con 0x55f365ef6300 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65e5638740 10 client.0 _ll_register_callbacks cb 0x7f65c4009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662b6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39812/0 (socket says 172.21.15.115:39812) 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d7fff640 1 -- 172.21.15.115:0/3896256893 learned_addr learned my addr 172.21.15.115:0/3896256893 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d77fe640 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 0x55f3662b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65dcd21640 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 0x55f3662b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d7fff640 1 -- 172.21.15.115:0/3896256893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 msgr2=0x55f3662b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d7fff640 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 0x55f3662b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d7fff640 1 -- 172.21.15.115:0/3896256893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 msgr2=0x55f3662b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d7fff640 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 0x55f3662b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d7fff640 1 -- 172.21.15.115:0/3896256893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3661ceed0 con 0x55f365ec0d80 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65d7fff640 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662b6ea0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0058c0 tx=0x7f65cc009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea02d5c41838d8d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:54:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65b77fe640 1 -- 172.21.15.115:0/3896256893 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc028040 con 0x55f365ec0d80 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65e5638740 1 -- 172.21.15.115:0/3896256893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c401fd10 con 0x55f365ec0d80 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65b77fe640 1 -- 172.21.15.115:0/3896256893 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f65cc019d90 con 0x55f365ec0d80 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65b77fe640 1 -- 172.21.15.115:0/3896256893 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f65cc045050 con 0x55f365ec0d80 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65e5638740 10 client.18837 fetch_fsmap learned FSMap version 63 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.163+0000 7f65e5638740 1 -- 172.21.15.115:0/3896256893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f366329320 con 0x55f365ec0d80 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65b77fe640 1 -- 172.21.15.115:0/3896256893 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65c401fd10 con 0x55f365ec0d80 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65b77fe640 1 -- 172.21.15.115:0/3896256893 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 63) ==== 8374+0+0 (secure 0 0 0) 0x7f65cc0051c0 con 0x55f365ec0d80 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65e5638740 10 client.18837 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65e5638740 10 client.18837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65e5638740 20 client.18837 populate_metadata read hostname 'smithi115' 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65e5638740 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f365ee14d0 0x55f36632ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65e5638740 4 client.18837 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65e5638740 1 -- 172.21.15.115:0/3896256893 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f366212240 con 0x55f365ee14d0 2024-05-09T05:54:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65d77fe640 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f365ee14d0 0x55f36632ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65d77fe640 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f365ee14d0 0x55f36632ba40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5adcbf59b9bbc1d1 server_cookie=b9c816a797710eac in_seq=0 out_seq=0 2024-05-09T05:54:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:41.164+0000 7f65b77fe640 10 client.18837 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:41.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:41 smithi195 ceph-mon[33844]: pgmap v519: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-05-09T05:54:41.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:41 smithi115 ceph-mon[30349]: pgmap v519: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-05-09T05:54:41.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:41 smithi161 ceph-mon[33805]: pgmap v519: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-05-09T05:54:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.141+0000 7f65b77fe640 1 -- 172.21.15.115:0/3896256893 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f366212240 con 0x55f365ee14d0 2024-05-09T05:54:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.141+0000 7f65b77fe640 10 client.18837 handle_command_reply: tid=0 2024-05-09T05:54:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.141+0000 7f65e5638740 1 client.18837 shutdown 2024-05-09T05:54:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.141+0000 7f65e5638740 1 -- 172.21.15.115:0/3896256893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 msgr2=0x55f3662b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.141+0000 7f65e5638740 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662b6ea0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0058c0 tx=0x7f65cc009790 comp rx=0 tx=0).stop 2024-05-09T05:54:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65e5638740 1 -- 172.21.15.115:0/3896256893 shutdown_connections 2024-05-09T05:54:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65e5638740 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f365ee14d0 0x55f36632ba40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65b77fe640 0 client.18837 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65e5638740 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f365ef6300 0x55f3662b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65e5638740 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f365ed92f0 0x55f3662b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65e5638740 1 --2- 172.21.15.115:0/3896256893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f365ec0d80 0x55f3662b6ea0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65e5638740 1 -- 172.21.15.115:0/3896256893 >> 172.21.15.115:0/3896256893 conn(0x55f3662ab670 msgr2=0x55f3662a55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65e5638740 1 -- 172.21.15.115:0/3896256893 shutdown_connections 2024-05-09T05:54:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65e5638740 1 -- 172.21.15.115:0/3896256893 wait complete. 2024-05-09T05:54:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.144+0000 7f65e5638740 20 client.18837 trim_cache size 0 max 16384 2024-05-09T05:54:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:54:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:54:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:54:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.150+0000 7f65de784640 1 -- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f65c80b2950 msgr2=0x7f65c80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.151+0000 7f65de784640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f65c80b2950 0x7f65c80b4e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55f3662a29b0 tx=0x7f65c4013040 comp rx=0 tx=0).stop 2024-05-09T05:54:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.151+0000 7f65de784640 1 -- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65d8109f60 msgr2=0x7f65d819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.151+0000 7f65de784640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65d8109f60 0x7f65d819ee00 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f65d002ae60 tx=0x7f65d002f870 comp rx=0 tx=0).stop 2024-05-09T05:54:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.154+0000 7f65de784640 1 -- 172.21.15.115:0/236110753 shutdown_connections 2024-05-09T05:54:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.154+0000 7f65de784640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f65c80b2950 0x7f65c80b4e60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.154+0000 7f65de784640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65d8109f60 0x7f65d819ee00 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.154+0000 7f65de784640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d8109ac0 0x7f65d819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.154+0000 7f65de784640 1 --2- 172.21.15.115:0/236110753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65d8104260 0x7f65d819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.154+0000 7f65de784640 1 -- 172.21.15.115:0/236110753 >> 172.21.15.115:0/236110753 conn(0x7f65d8079250 msgr2=0x7f65d8100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.156+0000 7f65de784640 1 -- 172.21.15.115:0/236110753 shutdown_connections 2024-05-09T05:54:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:42.156+0000 7f65de784640 1 -- 172.21.15.115:0/236110753 wait complete. 2024-05-09T05:54:43.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:43 smithi195 ceph-mon[33844]: pgmap v520: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:54:43.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:43 smithi115 ceph-mon[30349]: pgmap v520: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:54:43.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:43 smithi161 ceph-mon[33805]: pgmap v520: 129 pgs: 9 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 111 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:54:45.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:45 smithi195 ceph-mon[33844]: pgmap v521: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 14 MiB/s wr, 3.50k op/s 2024-05-09T05:54:45.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:45 smithi115 ceph-mon[30349]: pgmap v521: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 14 MiB/s wr, 3.50k op/s 2024-05-09T05:54:45.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:45 smithi161 ceph-mon[33805]: pgmap v521: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 14 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 14 MiB/s wr, 3.50k op/s 2024-05-09T05:54:47.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:47 smithi115 ceph-mon[30349]: pgmap v522: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-09T05:54:47.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:47 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:54:47.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:47 smithi195 ceph-mon[33844]: pgmap v522: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-09T05:54:47.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:47 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:54:47.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:47 smithi161 ceph-mon[33805]: pgmap v522: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-09T05:54:47.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:47 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:54:49.397 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 147.1 --await-for 45.0' 2024-05-09T05:54:49.398 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 147.1 --await-for 45.0 2024-05-09T05:54:49.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:49 smithi115 ceph-mon[30349]: pgmap v523: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-05-09T05:54:49.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:49 smithi195 ceph-mon[33844]: pgmap v523: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-05-09T05:54:49.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:49 smithi161 ceph-mon[33805]: pgmap v523: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-05-09T05:54:50.055 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:54:50.532 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:50 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:54:50.532 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:50 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:54:50.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:50 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:54:50.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:50 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:54:50.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:50 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:54:50.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:50 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:54:50.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:50 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:54:50.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:50 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:54:50.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:50 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:54:51.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:51 smithi195 ceph-mon[33844]: pgmap v524: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-05-09T05:54:51.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:51 smithi115 ceph-mon[30349]: pgmap v524: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-05-09T05:54:51.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:51 smithi161 ceph-mon[33805]: pgmap v524: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-05-09T05:54:53.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:53 smithi195 ceph-mon[33844]: pgmap v525: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 2.48k op/s 2024-05-09T05:54:53.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:53 smithi115 ceph-mon[30349]: pgmap v525: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 2.48k op/s 2024-05-09T05:54:53.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:53 smithi161 ceph-mon[33805]: pgmap v525: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 2.48k op/s 2024-05-09T05:54:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 -- 172.21.15.115:0/3892926776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58109040 msgr2=0x7f1e58102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 --2- 172.21.15.115:0/3892926776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58109040 0x7f1e58102c80 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009900 tx=0x7f1e4002a410 comp rx=0 tx=0).stop 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 -- 172.21.15.115:0/3892926776 shutdown_connections 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 --2- 172.21.15.115:0/3892926776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e581032f0 0x7f1e5810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 --2- 172.21.15.115:0/3892926776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58109040 0x7f1e58102c80 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 --2- 172.21.15.115:0/3892926776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e58108ba0 0x7f1e58102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 -- 172.21.15.115:0/3892926776 >> 172.21.15.115:0/3892926776 conn(0x7f1e580782e0 msgr2=0x7f1e58078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 -- 172.21.15.115:0/3892926776 shutdown_connections 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 -- 172.21.15.115:0/3892926776 wait complete. 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 Processor -- start 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.319+0000 7f1e5df72640 1 -- start start 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e5df72640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e581032f0 0x7f1e581996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e5df72640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58108ba0 0x7f1e58199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e5df72640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e58109040 0x7f1e5819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e5df72640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58112c40 con 0x7f1e58108ba0 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e5df72640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58112ac0 con 0x7f1e581032f0 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e5df72640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58112dc0 con 0x7f1e58109040 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58108ba0 0x7f1e58199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58108ba0 0x7f1e58199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35698/0 (socket says 172.21.15.115:35698) 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e56ffd640 1 -- 172.21.15.115:0/1704861674 learned_addr learned my addr 172.21.15.115:0/1704861674 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e57fff640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e58109040 0x7f1e5819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e577fe640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e581032f0 0x7f1e581996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e56ffd640 1 -- 172.21.15.115:0/1704861674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e58109040 msgr2=0x7f1e5819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e58109040 0x7f1e5819df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e56ffd640 1 -- 172.21.15.115:0/1704861674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e581032f0 msgr2=0x7f1e581996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e581032f0 0x7f1e581996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e56ffd640 1 -- 172.21.15.115:0/1704861674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5819e4c0 con 0x7f1e58108ba0 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58108ba0 0x7f1e58199c40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005940 tx=0x7f1e40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e54ff9640 1 -- 172.21.15.115:0/1704861674 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4003f030 con 0x7f1e58108ba0 2024-05-09T05:54:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e54ff9640 1 -- 172.21.15.115:0/1704861674 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e40004440 con 0x7f1e58108ba0 2024-05-09T05:54:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e5df72640 1 -- 172.21.15.115:0/1704861674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5819e750 con 0x7f1e58108ba0 2024-05-09T05:54:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.320+0000 7f1e54ff9640 1 -- 172.21.15.115:0/1704861674 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40035070 con 0x7f1e58108ba0 2024-05-09T05:54:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.321+0000 7f1e5df72640 1 -- 172.21.15.115:0/1704861674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5819ec40 con 0x7f1e58108ba0 2024-05-09T05:54:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.323+0000 7f1e54ff9640 1 -- 172.21.15.115:0/1704861674 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1e40042030 con 0x7f1e58108ba0 2024-05-09T05:54:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.323+0000 7f1e64e21740 1 Processor -- start 2024-05-09T05:54:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.323+0000 7f1e64e21740 1 -- start start 2024-05-09T05:54:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.323+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 0x564e174f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171324d0 0x564e174f9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e64e21740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564e1745b7c0 con 0x564e171531b0 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e64e21740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564e173f70e0 con 0x564e1712a2f0 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e64e21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564e17457780 con 0x564e171324d0 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35710/0 (socket says 172.21.15.115:35710) 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e57fff640 1 -- 172.21.15.115:0/4283822691 learned_addr learned my addr 172.21.15.115:0/4283822691 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/4283822691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171324d0 0x564e174f9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e577fe640 1 --2- 172.21.15.115:0/4283822691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 0x564e174f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e57fff640 1 -- 172.21.15.115:0/4283822691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171324d0 msgr2=0x564e174f9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e57fff640 1 --2- 172.21.15.115:0/4283822691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171324d0 0x564e174f9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e57fff640 1 -- 172.21.15.115:0/4283822691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 msgr2=0x564e174f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e57fff640 1 --2- 172.21.15.115:0/4283822691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 0x564e174f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e57fff640 1 -- 172.21.15.115:0/4283822691 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e174207c0 con 0x564e171531b0 2024-05-09T05:54:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e54ff9640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e440b2950 0x7f1e440b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e577fe640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e440b2950 0x7f1e440b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e57fff640 1 --2- 172.21.15.115:0/4283822691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f8890 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009ca0 tx=0x7f1e48011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1476ca24b3f5182 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e54ff9640 1 -- 172.21.15.115:0/1704861674 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f1e40102da0 con 0x7f1e58108ba0 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e4dffb640 1 -- 172.21.15.115:0/4283822691 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4801e070 con 0x564e171531b0 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e4dffb640 1 -- 172.21.15.115:0/4283822691 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e48009e50 con 0x564e171531b0 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.324+0000 7f1e4dffb640 1 -- 172.21.15.115:0/4283822691 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800e870 con 0x564e171531b0 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e64e21740 1 -- 172.21.15.115:0/4283822691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 msgr2=0x564e174f8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e577fe640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e440b2950 0x7f1e440b4e60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x564e174f9c40 tx=0x7f1e500075b0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e64e21740 1 --2- 172.21.15.115:0/4283822691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f8890 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009ca0 tx=0x7f1e48011810 comp rx=0 tx=0).stop 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e64e21740 1 -- 172.21.15.115:0/4283822691 shutdown_connections 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e64e21740 1 --2- 172.21.15.115:0/4283822691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171324d0 0x564e174f9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e64e21740 1 --2- 172.21.15.115:0/4283822691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 0x564e174f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e64e21740 1 --2- 172.21.15.115:0/4283822691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f8890 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e64e21740 1 -- 172.21.15.115:0/4283822691 >> 172.21.15.115:0/4283822691 conn(0x564e17147300 msgr2=0x564e174f5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e64e21740 1 -- 172.21.15.115:0/4283822691 shutdown_connections 2024-05-09T05:54:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.325+0000 7f1e64e21740 1 -- 172.21.15.115:0/4283822691 wait complete. 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e64e21740 1 -- start start 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e64e21740 1 Processor -- start 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 0x564e175048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e17504e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171531b0 0x564e17502df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e64e21740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x564e175059c0 con 0x564e171324d0 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e64e21740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564e173e29c0 con 0x564e1712a2f0 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e64e21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564e17505b40 con 0x564e171531b0 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e64e21740 10 client.0 _ll_register_callbacks cb 0x564e1740f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e17504e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e17504e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35720/0 (socket says 172.21.15.115:35720) 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e56ffd640 1 -- 172.21.15.115:0/214082392 learned_addr learned my addr 172.21.15.115:0/214082392 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e577fe640 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 0x564e175048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e57fff640 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171531b0 0x564e17502df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e56ffd640 1 -- 172.21.15.115:0/214082392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171531b0 msgr2=0x564e17502df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171531b0 0x564e17502df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.326+0000 7f1e56ffd640 1 -- 172.21.15.115:0/214082392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 msgr2=0x564e175048e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 0x564e175048e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e56ffd640 1 -- 172.21.15.115:0/214082392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e174091f0 con 0x564e171324d0 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e17504e20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f1e400c6f90 tx=0x7f1e401064e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784dd8bb1c6a1ab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e36ffd640 1 -- 172.21.15.115:0/214082392 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40049040 con 0x564e171324d0 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e36ffd640 1 -- 172.21.15.115:0/214082392 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f1e400c9f80 con 0x564e171324d0 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e64e21740 1 -- 172.21.15.115:0/214082392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e48009e50 con 0x564e171324d0 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e36ffd640 1 -- 172.21.15.115:0/214082392 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f1e400c9670 con 0x564e171324d0 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e64e21740 10 client.18849 fetch_fsmap learned FSMap version 63 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e64e21740 1 -- 172.21.15.115:0/214082392 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564e171be790 con 0x564e171324d0 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e36ffd640 1 -- 172.21.15.115:0/214082392 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e48009e50 con 0x564e171324d0 2024-05-09T05:54:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e36ffd640 1 -- 172.21.15.115:0/214082392 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 63) ==== 8374+0+0 (secure 0 0 0) 0x7f1e400d2640 con 0x564e171324d0 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e64e21740 10 client.18849 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e64e21740 10 client.18849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e64e21740 20 client.18849 populate_metadata read hostname 'smithi115' 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e64e21740 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564e17111d80 0x564e17515170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e64e21740 4 client.18849 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.327+0000 7f1e64e21740 1 -- 172.21.15.115:0/214082392 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e174693d0 con 0x564e17111d80 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.328+0000 7f1e577fe640 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564e17111d80 0x564e17515170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.328+0000 7f1e36ffd640 10 client.18849 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.328+0000 7f1e577fe640 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564e17111d80 0x564e17515170 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da129ae00ea4d4ed server_cookie=679e1091562caf71 in_seq=0 out_seq=0 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e36ffd640 1 -- 172.21.15.115:0/214082392 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e174693d0 con 0x564e17111d80 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e36ffd640 10 client.18849 handle_command_reply: tid=0 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 client.18849 shutdown 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 -- 172.21.15.115:0/214082392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 msgr2=0x564e17504e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e17504e20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f1e400c6f90 tx=0x7f1e401064e0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 -- 172.21.15.115:0/214082392 shutdown_connections 2024-05-09T05:54:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564e17111d80 0x564e17515170 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e171531b0 0x564e17502df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e17504e20 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e36ffd640 0 client.18849 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 --2- 172.21.15.115:0/214082392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e1712a2f0 0x564e175048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 -- 172.21.15.115:0/214082392 >> 172.21.15.115:0/214082392 conn(0x564e17147300 msgr2=0x564e174fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 -- 172.21.15.115:0/214082392 shutdown_connections 2024-05-09T05:54:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 1 -- 172.21.15.115:0/214082392 wait complete. 2024-05-09T05:54:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.329+0000 7f1e64e21740 20 client.18849 trim_cache size 0 max 16384 2024-05-09T05:54:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e64e21740 1 Processor -- start 2024-05-09T05:54:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e64e21740 1 -- start start 2024-05-09T05:54:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e174fabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e1751a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171531b0 0x564e1751aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e64e21740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e581aefd0 con 0x564e171324d0 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e64e21740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e581b0b40 con 0x564e171531b0 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e64e21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e581af6a0 con 0x564e1712a2f0 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e1751a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e56ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e1751a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35730/0 (socket says 172.21.15.115:35730) 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e56ffd640 1 -- 172.21.15.115:0/440167348 learned_addr learned my addr 172.21.15.115:0/440167348 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e577fe640 1 --2- 172.21.15.115:0/440167348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e174fabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e56ffd640 1 -- 172.21.15.115:0/440167348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 msgr2=0x564e174fabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/440167348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e174fabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e56ffd640 1 -- 172.21.15.115:0/440167348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171531b0 msgr2=0x564e1751aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e57fff640 1 --2- 172.21.15.115:0/440167348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171531b0 0x564e1751aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/440167348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171531b0 0x564e1751aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e56ffd640 1 -- 172.21.15.115:0/440167348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e172355f0 con 0x564e171324d0 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.344+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/440167348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e1751a970 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f1e4003d040 tx=0x7f1e400c9dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60ac7c9a35ca4aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e4d7fa640 1 -- 172.21.15.115:0/440167348 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40049040 con 0x564e171324d0 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e4d7fa640 1 -- 172.21.15.115:0/440167348 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e400d2410 con 0x564e171324d0 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e4d7fa640 1 -- 172.21.15.115:0/440167348 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400c9490 con 0x564e171324d0 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e64e21740 1 -- 172.21.15.115:0/440167348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 msgr2=0x564e1751a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e64e21740 1 --2- 172.21.15.115:0/440167348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e1751a970 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f1e4003d040 tx=0x7f1e400c9dc0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e64e21740 1 -- 172.21.15.115:0/440167348 shutdown_connections 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e64e21740 1 --2- 172.21.15.115:0/440167348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171531b0 0x564e1751aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e64e21740 1 --2- 172.21.15.115:0/440167348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171324d0 0x564e1751a970 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e64e21740 1 --2- 172.21.15.115:0/440167348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e174fabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e64e21740 1 -- 172.21.15.115:0/440167348 >> 172.21.15.115:0/440167348 conn(0x564e17147300 msgr2=0x564e174fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e64e21740 1 -- 172.21.15.115:0/440167348 shutdown_connections 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.345+0000 7f1e64e21740 1 -- 172.21.15.115:0/440167348 wait complete. 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e64e21740 1 -- start start 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e64e21740 1 Processor -- start 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e1750b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171324d0 0x564e174f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e64e21740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e64e21740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e581b0470 con 0x564e171531b0 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e64e21740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e581afda0 con 0x564e171324d0 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e64e21740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1e581acfd0 con 0x564e1712a2f0 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e64e21740 10 client.0 _ll_register_callbacks cb 0x7f1e40005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e577fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e1750b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e577fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e1750b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:58496/0 (socket says 172.21.15.115:58496) 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35744/0 (socket says 172.21.15.115:35744) 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e577fe640 1 -- 172.21.15.115:0/2778978432 learned_addr learned my addr 172.21.15.115:0/2778978432 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171324d0 0x564e174f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e57fff640 1 -- 172.21.15.115:0/2778978432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 msgr2=0x564e1750b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e57fff640 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e1750b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e57fff640 1 -- 172.21.15.115:0/2778978432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171324d0 msgr2=0x564e174f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e57fff640 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171324d0 0x564e174f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e57fff640 1 -- 172.21.15.115:0/2778978432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e1741fed0 con 0x564e171531b0 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e56ffd640 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171324d0 0x564e174f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.346+0000 7f1e577fe640 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e1750b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e57fff640 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f4620 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1e4801c040 tx=0x7f1e480056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff95c51185328fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e36ffd640 1 -- 172.21.15.115:0/2778978432 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48029070 con 0x564e171531b0 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e36ffd640 1 -- 172.21.15.115:0/2778978432 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f1e48003a00 con 0x564e171531b0 2024-05-09T05:54:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e64e21740 1 -- 172.21.15.115:0/2778978432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e17463240 con 0x564e171531b0 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e36ffd640 1 -- 172.21.15.115:0/2778978432 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f1e48032050 con 0x564e171531b0 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e64e21740 10 client.18855 fetch_fsmap learned FSMap version 63 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e64e21740 1 -- 172.21.15.115:0/2778978432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564e17509c50 con 0x564e171531b0 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e36ffd640 1 -- 172.21.15.115:0/2778978432 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x564e17463240 con 0x564e171531b0 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e36ffd640 1 -- 172.21.15.115:0/2778978432 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 63) ==== 8374+0+0 (secure 0 0 0) 0x7f1e4802c020 con 0x564e171531b0 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e64e21740 10 client.18855 fetch_fsmap finished waiting for FSMap version 63 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e64e21740 10 client.18855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e64e21740 20 client.18855 populate_metadata read hostname 'smithi115' 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e64e21740 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564e170e71c0 0x564e17581bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e64e21740 4 client.18855 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.347+0000 7f1e64e21740 1 -- 172.21.15.115:0/2778978432 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e173de350 con 0x564e170e71c0 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.348+0000 7f1e577fe640 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564e170e71c0 0x564e17581bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.348+0000 7f1e577fe640 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564e170e71c0 0x564e17581bb0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16cb72973272ff54 server_cookie=e75136026c20a3a2 in_seq=0 out_seq=0 2024-05-09T05:54:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:55.348+0000 7f1e36ffd640 10 client.18855 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:55.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:55 smithi195 ceph-mon[33844]: pgmap v526: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-05-09T05:54:55.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:55 smithi115 ceph-mon[30349]: pgmap v526: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-05-09T05:54:55.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:55 smithi161 ceph-mon[33805]: pgmap v526: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-05-09T05:54:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e36ffd640 1 -- 172.21.15.115:0/2778978432 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564e173de350 con 0x564e170e71c0 2024-05-09T05:54:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e36ffd640 10 client.18855 handle_command_reply: tid=0 2024-05-09T05:54:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 client.18855 shutdown 2024-05-09T05:54:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 -- 172.21.15.115:0/2778978432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 msgr2=0x564e174f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f4620 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1e4801c040 tx=0x7f1e480056a0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 -- 172.21.15.115:0/2778978432 shutdown_connections 2024-05-09T05:54:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x564e171531b0 0x564e174f4620 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e171324d0 0x564e174f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x564e170e71c0 0x564e17581bb0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 --2- 172.21.15.115:0/2778978432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564e1712a2f0 0x564e1750b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 -- 172.21.15.115:0/2778978432 >> 172.21.15.115:0/2778978432 conn(0x564e17147300 msgr2=0x564e174f8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e36ffd640 0 client.18855 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 -- 172.21.15.115:0/2778978432 shutdown_connections 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.932+0000 7f1e64e21740 1 -- 172.21.15.115:0/2778978432 wait complete. 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.933+0000 7f1e64e21740 20 client.18855 trim_cache size 0 max 16384 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:54:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:54:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:54:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.939+0000 7f1e5df72640 1 -- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e440b2950 msgr2=0x7f1e440b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.939+0000 7f1e5df72640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e440b2950 0x7f1e440b4e60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x564e174f9c40 tx=0x7f1e500075b0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.939+0000 7f1e5df72640 1 -- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58108ba0 msgr2=0x7f1e58199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:54:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.939+0000 7f1e5df72640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58108ba0 0x7f1e58199c40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005940 tx=0x7f1e40004040 comp rx=0 tx=0).stop 2024-05-09T05:54:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.940+0000 7f1e5df72640 1 -- 172.21.15.115:0/1704861674 shutdown_connections 2024-05-09T05:54:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.940+0000 7f1e5df72640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1e440b2950 0x7f1e440b4e60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.940+0000 7f1e5df72640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e58109040 0x7f1e5819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.940+0000 7f1e5df72640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e58108ba0 0x7f1e58199c40 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.940+0000 7f1e5df72640 1 --2- 172.21.15.115:0/1704861674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e581032f0 0x7f1e581996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:54:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.940+0000 7f1e5df72640 1 -- 172.21.15.115:0/1704861674 >> 172.21.15.115:0/1704861674 conn(0x7f1e580782e0 msgr2=0x7f1e581012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:54:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.940+0000 7f1e5df72640 1 -- 172.21.15.115:0/1704861674 shutdown_connections 2024-05-09T05:54:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:54:56.941+0000 7f1e5df72640 1 -- 172.21.15.115:0/1704861674 wait complete. 2024-05-09T05:54:57.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:57 smithi195 ceph-mon[33844]: pgmap v527: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:54:57.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:57 smithi115 ceph-mon[30349]: pgmap v527: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:54:57.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:57 smithi161 ceph-mon[33805]: pgmap v527: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:54:59.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:59 smithi195 ceph-mon[33844]: pgmap v528: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 3.16k op/s 2024-05-09T05:54:59.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:54:59 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:54:59.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:59 smithi115 ceph-mon[30349]: pgmap v528: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 3.16k op/s 2024-05-09T05:54:59.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:54:59 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:54:59.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:59 smithi161 ceph-mon[33805]: pgmap v528: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 3.16k op/s 2024-05-09T05:54:59.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:54:59 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:01.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:01 smithi195 ceph-mon[33844]: pgmap v529: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-05-09T05:55:01.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:01 smithi115 ceph-mon[30349]: pgmap v529: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-05-09T05:55:01.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:01 smithi161 ceph-mon[33805]: pgmap v529: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-05-09T05:55:02.549 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:02 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:02.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:02 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:02.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:02 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:03.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:03 smithi195 ceph-mon[33844]: pgmap v530: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.6 MiB/s wr, 2.42k op/s 2024-05-09T05:55:03.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:03 smithi115 ceph-mon[30349]: pgmap v530: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.6 MiB/s wr, 2.42k op/s 2024-05-09T05:55:03.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:03 smithi161 ceph-mon[33805]: pgmap v530: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.6 MiB/s wr, 2.42k op/s 2024-05-09T05:55:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:05 smithi195 ceph-mon[33844]: pgmap v531: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 3.24k op/s 2024-05-09T05:55:05.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:05 smithi115 ceph-mon[30349]: pgmap v531: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 3.24k op/s 2024-05-09T05:55:05.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:05 smithi161 ceph-mon[33805]: pgmap v531: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 3.24k op/s 2024-05-09T05:55:07.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:07 smithi115 ceph-mon[30349]: pgmap v532: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-09T05:55:07.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:07 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:07.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:07 smithi195 ceph-mon[33844]: pgmap v532: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-09T05:55:07.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:07 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:07.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:07 smithi161 ceph-mon[33805]: pgmap v532: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-09T05:55:07.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:07 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:09.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:09 smithi195 ceph-mon[33844]: pgmap v533: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-05-09T05:55:09.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:09 smithi115 ceph-mon[30349]: pgmap v533: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-05-09T05:55:09.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:09 smithi161 ceph-mon[33805]: pgmap v533: 129 pgs: 9 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 112 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-05-09T05:55:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 -- 172.21.15.115:0/2039506814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13841021d0 msgr2=0x7f13841025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 --2- 172.21.15.115:0/2039506814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13841021d0 0x7f13841025e0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1374009900 tx=0x7f137402a410 comp rx=0 tx=0).stop 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 -- 172.21.15.115:0/2039506814 shutdown_connections 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 --2- 172.21.15.115:0/2039506814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f138410e6d0 0x7f1384110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 --2- 172.21.15.115:0/2039506814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13841021d0 0x7f13841025e0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 --2- 172.21.15.115:0/2039506814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1384109d20 0x7f1384101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 -- 172.21.15.115:0/2039506814 >> 172.21.15.115:0/2039506814 conn(0x7f13840779e0 msgr2=0x7f1384077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 -- 172.21.15.115:0/2039506814 shutdown_connections 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 -- 172.21.15.115:0/2039506814 wait complete. 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 Processor -- start 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.113+0000 7f138c229640 1 -- start start 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138c229640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13841021d0 0x7f138419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138c229640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1384109d20 0x7f138419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138c229640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f138410e6d0 0x7f138419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138c229640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1384113d40 con 0x7f138410e6d0 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138c229640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1384113bc0 con 0x7f1384109d20 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138c229640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1384113ec0 con 0x7f13841021d0 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138a7c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f138410e6d0 0x7f138419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138a7c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f138410e6d0 0x7f138419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57456/0 (socket says 172.21.15.115:57456) 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138a7c6640 1 -- 172.21.15.115:0/2098855752 learned_addr learned my addr 172.21.15.115:0/2098855752 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f13897c4640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1384109d20 0x7f138419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f1389fc5640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13841021d0 0x7f138419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138a7c6640 1 -- 172.21.15.115:0/2098855752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13841021d0 msgr2=0x7f138419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138a7c6640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13841021d0 0x7f138419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138a7c6640 1 -- 172.21.15.115:0/2098855752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1384109d20 msgr2=0x7f138419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138a7c6640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1384109d20 0x7f138419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138a7c6640 1 -- 172.21.15.115:0/2098855752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138419f600 con 0x7f138410e6d0 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138a7c6640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f138410e6d0 0x7f138419efc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f137c00dac0 tx=0x7f137c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f137affd640 1 -- 172.21.15.115:0/2098855752 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c017070 con 0x7f138410e6d0 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f137affd640 1 -- 172.21.15.115:0/2098855752 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f137c00be30 con 0x7f138410e6d0 2024-05-09T05:55:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f137affd640 1 -- 172.21.15.115:0/2098855752 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c010460 con 0x7f138410e6d0 2024-05-09T05:55:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138c229640 1 -- 172.21.15.115:0/2098855752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138419f8f0 con 0x7f138410e6d0 2024-05-09T05:55:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.114+0000 7f138c229640 1 -- 172.21.15.115:0/2098855752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13841a7270 con 0x7f138410e6d0 2024-05-09T05:55:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.117+0000 7f13931b1740 1 Processor -- start 2024-05-09T05:55:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.117+0000 7f137affd640 1 -- 172.21.15.115:0/2098855752 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f137c003970 con 0x7f138410e6d0 2024-05-09T05:55:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.117+0000 7f13931b1740 1 -- start start 2024-05-09T05:55:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13931b1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4b001b0 0x55dbf4ea3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13931b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ea3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13931b1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ea4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13931b1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dbf4e087c0 con 0x55dbf4adf4d0 2024-05-09T05:55:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13931b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dbf4da40e0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13931b1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dbf4e04780 con 0x55dbf4b001b0 2024-05-09T05:55:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13897c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4b001b0 0x55dbf4ea3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f1389fc5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ea3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13897c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ea4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f1389fc5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ea3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:53646/0 (socket says 172.21.15.115:53646) 2024-05-09T05:55:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f1389fc5640 1 -- 172.21.15.115:0/12094733 learned_addr learned my addr 172.21.15.115:0/12094733 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f1389fc5640 1 -- 172.21.15.115:0/12094733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4b001b0 msgr2=0x55dbf4ea3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f1389fc5640 1 --2- 172.21.15.115:0/12094733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4b001b0 0x55dbf4ea3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f1389fc5640 1 -- 172.21.15.115:0/12094733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 msgr2=0x55dbf4ea4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f1389fc5640 1 --2- 172.21.15.115:0/12094733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ea4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f1389fc5640 1 -- 172.21.15.115:0/12094733 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf4dcd7c0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f137affd640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f13700b29d0 0x7f13700b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13897c4640 1 --2- 172.21.15.115:0/12094733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ea4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:55:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f13897c4640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f13700b29d0 0x7f13700b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f1389fc5640 1 --2- 172.21.15.115:0/12094733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ea3bc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1380009c60 tx=0x7f1380011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1748fb3f13d693f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f137affd640 1 -- 172.21.15.115:0/2098855752 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f137c0e25f0 con 0x7f138410e6d0 2024-05-09T05:55:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.118+0000 7f136ffff640 1 -- 172.21.15.115:0/12094733 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380004220 con 0x55dbf4ad72f0 2024-05-09T05:55:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f136ffff640 1 -- 172.21.15.115:0/12094733 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13800043c0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f136ffff640 1 -- 172.21.15.115:0/12094733 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380004ba0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13931b1740 1 -- 172.21.15.115:0/12094733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 msgr2=0x55dbf4ea3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13897c4640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f13700b29d0 0x7f13700b4ee0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1374004770 tx=0x7f13740058d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13931b1740 1 --2- 172.21.15.115:0/12094733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ea3bc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1380009c60 tx=0x7f1380011810 comp rx=0 tx=0).stop 2024-05-09T05:55:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13931b1740 1 -- 172.21.15.115:0/12094733 shutdown_connections 2024-05-09T05:55:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13931b1740 1 --2- 172.21.15.115:0/12094733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ea4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13931b1740 1 --2- 172.21.15.115:0/12094733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ea3bc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13931b1740 1 --2- 172.21.15.115:0/12094733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4b001b0 0x55dbf4ea3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13931b1740 1 -- 172.21.15.115:0/12094733 >> 172.21.15.115:0/12094733 conn(0x55dbf4af4300 msgr2=0x55dbf4e9fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13931b1740 1 -- 172.21.15.115:0/12094733 shutdown_connections 2024-05-09T05:55:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.119+0000 7f13931b1740 1 -- 172.21.15.115:0/12094733 wait complete. 2024-05-09T05:55:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13931b1740 1 -- start start 2024-05-09T05:55:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13931b1740 1 Processor -- start 2024-05-09T05:55:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13931b1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eadd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13931b1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4eae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13931b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4b001b0 0x55dbf4eb3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13931b1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55dbf4eb07b0 con 0x55dbf4adf4d0 2024-05-09T05:55:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13931b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dbf4d8f9c0 con 0x55dbf4b001b0 2024-05-09T05:55:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13931b1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dbf4eb0930 con 0x55dbf4ad72f0 2024-05-09T05:55:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13931b1740 10 client.0 _ll_register_callbacks cb 0x55dbf4dbc020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f1389fc5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eadd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f1389fc5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eadd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:35810/0 (socket says 172.21.15.115:35810) 2024-05-09T05:55:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f1389fc5640 1 -- 172.21.15.115:0/2392548596 learned_addr learned my addr 172.21.15.115:0/2392548596 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f138a7c6640 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4b001b0 0x55dbf4eb3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.120+0000 7f13897c4640 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4eae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f1389fc5640 1 -- 172.21.15.115:0/2392548596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4b001b0 msgr2=0x55dbf4eb3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f1389fc5640 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4b001b0 0x55dbf4eb3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f1389fc5640 1 -- 172.21.15.115:0/2392548596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 msgr2=0x55dbf4eae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f1389fc5640 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4eae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f1389fc5640 1 -- 172.21.15.115:0/2392548596 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf4db61f0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f13897c4640 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4eae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:55:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f138a7c6640 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4b001b0 0x55dbf4eb3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:55:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f1389fc5640 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eadd70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1380000c00 tx=0x7f1380019b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9df7f0678d1d58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f136d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:55:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f136d7fa640 1 -- 172.21.15.115:0/2392548596 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380027030 con 0x55dbf4ad72f0 2024-05-09T05:55:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f13931b1740 1 -- 172.21.15.115:0/2392548596 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13800043c0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f136d7fa640 1 -- 172.21.15.115:0/2392548596 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f1380019c90 con 0x55dbf4ad72f0 2024-05-09T05:55:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f136d7fa640 1 -- 172.21.15.115:0/2392548596 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f1380034050 con 0x55dbf4ad72f0 2024-05-09T05:55:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f13931b1740 10 client.28201 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.121+0000 7f13931b1740 1 -- 172.21.15.115:0/2392548596 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55dbf4b6b790 con 0x55dbf4ad72f0 2024-05-09T05:55:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f136d7fa640 1 -- 172.21.15.115:0/2392548596 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13800043c0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f136d7fa640 1 -- 172.21.15.115:0/2392548596 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7f13800238a0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f13931b1740 10 client.28201 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f13931b1740 10 client.28201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:55:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f13931b1740 20 client.28201 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f13931b1740 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dbf4abed80 0x55dbf4ebfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f13931b1740 4 client.28201 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:55:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f13931b1740 1 -- 172.21.15.115:0/2392548596 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf4e163d0 con 0x55dbf4abed80 2024-05-09T05:55:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f13897c4640 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dbf4abed80 0x55dbf4ebfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f136d7fa640 10 client.28201 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.122+0000 7f13897c4640 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dbf4abed80 0x55dbf4ebfec0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a1bff80600d14f server_cookie=f2a503c094a2622c in_seq=0 out_seq=0 2024-05-09T05:55:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f136d7fa640 1 -- 172.21.15.115:0/2392548596 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbf4e163d0 con 0x55dbf4abed80 2024-05-09T05:55:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f136d7fa640 10 client.28201 handle_command_reply: tid=0 2024-05-09T05:55:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 client.28201 shutdown 2024-05-09T05:55:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 -- 172.21.15.115:0/2392548596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4ad72f0 msgr2=0x55dbf4eadd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eadd70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1380000c00 tx=0x7f1380019b20 comp rx=0 tx=0).stop 2024-05-09T05:55:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 -- 172.21.15.115:0/2392548596 shutdown_connections 2024-05-09T05:55:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4b001b0 0x55dbf4eb3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4eae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dbf4abed80 0x55dbf4ebfec0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 --2- 172.21.15.115:0/2392548596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eadd70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f136d7fa640 0 client.28201 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 -- 172.21.15.115:0/2392548596 >> 172.21.15.115:0/2392548596 conn(0x55dbf4af4300 msgr2=0x55dbf4ea34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 -- 172.21.15.115:0/2392548596 shutdown_connections 2024-05-09T05:55:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 1 -- 172.21.15.115:0/2392548596 wait complete. 2024-05-09T05:55:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.123+0000 7f13931b1740 20 client.28201 trim_cache size 0 max 16384 2024-05-09T05:55:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f13931b1740 1 Processor -- start 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f13931b1740 1 -- start start 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f13931b1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4abed80 0x55dbf4eacb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f13931b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ec5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f13931b1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ec5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f13931b1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13841b1250 con 0x55dbf4adf4d0 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f13931b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13841b2d90 con 0x55dbf4ad72f0 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f13931b1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f13841b1920 con 0x55dbf4abed80 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f138a7c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ec5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f1389fc5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4abed80 0x55dbf4eacb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f1389fc5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4abed80 0x55dbf4eacb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:35816/0 (socket says 172.21.15.115:35816) 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f138a7c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ec5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57468/0 (socket says 172.21.15.115:57468) 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.138+0000 7f1389fc5640 1 -- 172.21.15.115:0/3132356446 learned_addr learned my addr 172.21.15.115:0/3132356446 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f13897c4640 1 --2- 172.21.15.115:0/3132356446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ec5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f1389fc5640 1 -- 172.21.15.115:0/3132356446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 msgr2=0x55dbf4ec5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f1389fc5640 1 --2- 172.21.15.115:0/3132356446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ec5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f1389fc5640 1 -- 172.21.15.115:0/3132356446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 msgr2=0x55dbf4ec5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f1389fc5640 1 --2- 172.21.15.115:0/3132356446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ec5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f1389fc5640 1 -- 172.21.15.115:0/3132356446 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf4be25f0 con 0x55dbf4abed80 2024-05-09T05:55:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f138a7c6640 1 --2- 172.21.15.115:0/3132356446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ec5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:55:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f1389fc5640 1 --2- 172.21.15.115:0/3132356446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4abed80 0x55dbf4eacb40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13800058c0 tx=0x7f1380011e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6af2ebc5391751a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f136f7fe640 1 -- 172.21.15.115:0/3132356446 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380027070 con 0x55dbf4abed80 2024-05-09T05:55:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f136f7fe640 1 -- 172.21.15.115:0/3132356446 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f138002e580 con 0x55dbf4abed80 2024-05-09T05:55:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.139+0000 7f136f7fe640 1 -- 172.21.15.115:0/3132356446 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138001d040 con 0x55dbf4abed80 2024-05-09T05:55:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 -- 172.21.15.115:0/3132356446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4abed80 msgr2=0x55dbf4eacb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 --2- 172.21.15.115:0/3132356446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4abed80 0x55dbf4eacb40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13800058c0 tx=0x7f1380011e40 comp rx=0 tx=0).stop 2024-05-09T05:55:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 -- 172.21.15.115:0/3132356446 shutdown_connections 2024-05-09T05:55:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 --2- 172.21.15.115:0/3132356446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4adf4d0 0x55dbf4ec5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 --2- 172.21.15.115:0/3132356446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4ad72f0 0x55dbf4ec5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 --2- 172.21.15.115:0/3132356446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4abed80 0x55dbf4eacb40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 -- 172.21.15.115:0/3132356446 >> 172.21.15.115:0/3132356446 conn(0x55dbf4af4300 msgr2=0x55dbf4eac430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 -- 172.21.15.115:0/3132356446 shutdown_connections 2024-05-09T05:55:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 -- 172.21.15.115:0/3132356446 wait complete. 2024-05-09T05:55:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 -- start start 2024-05-09T05:55:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.140+0000 7f13931b1740 1 Processor -- start 2024-05-09T05:55:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13931b1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4abed80 0x55dbf4eb5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13931b1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eb5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13931b1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4adf4d0 0x55dbf4f26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13931b1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f13841b26c0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13931b1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13841b1ff0 con 0x55dbf4abed80 2024-05-09T05:55:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13931b1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f13841af250 con 0x55dbf4adf4d0 2024-05-09T05:55:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13931b1740 10 client.0 _ll_register_callbacks cb 0x7f1374009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13897c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eb5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13897c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eb5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57484/0 (socket says 172.21.15.115:57484) 2024-05-09T05:55:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13897c4640 1 -- 172.21.15.115:0/2787134546 learned_addr learned my addr 172.21.15.115:0/2787134546 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f1389fc5640 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4abed80 0x55dbf4eb5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f138a7c6640 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4adf4d0 0x55dbf4f26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13897c4640 1 -- 172.21.15.115:0/2787134546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4adf4d0 msgr2=0x55dbf4f26110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13897c4640 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4adf4d0 0x55dbf4f26110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13897c4640 1 -- 172.21.15.115:0/2787134546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4abed80 msgr2=0x55dbf4eb5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13897c4640 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4abed80 0x55dbf4eb5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13897c4640 1 -- 172.21.15.115:0/2787134546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf4dcced0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13897c4640 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eb5780 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f13740057c0 tx=0x7f137402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e90426067ab54de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f136d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:55:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f136d7fa640 1 -- 172.21.15.115:0/2787134546 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374035040 con 0x55dbf4ad72f0 2024-05-09T05:55:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f136d7fa640 1 -- 172.21.15.115:0/2787134546 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7f13740035f0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13931b1740 1 -- 172.21.15.115:0/2787134546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f138002e580 con 0x55dbf4ad72f0 2024-05-09T05:55:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f136d7fa640 1 -- 172.21.15.115:0/2787134546 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f1374063050 con 0x55dbf4ad72f0 2024-05-09T05:55:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.141+0000 7f13931b1740 10 client.18873 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f13931b1740 1 -- 172.21.15.115:0/2787134546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55dbf4f27500 con 0x55dbf4ad72f0 2024-05-09T05:55:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f136d7fa640 1 -- 172.21.15.115:0/2787134546 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f138002e580 con 0x55dbf4ad72f0 2024-05-09T05:55:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f136d7fa640 1 -- 172.21.15.115:0/2787134546 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7f1374054ce0 con 0x55dbf4ad72f0 2024-05-09T05:55:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f13931b1740 10 client.18873 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f13931b1740 10 client.18873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:55:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f13931b1740 20 client.18873 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f13931b1740 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dbf4b001b0 0x55dbf4f29cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f13931b1740 4 client.18873 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:55:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f13931b1740 1 -- 172.21.15.115:0/2787134546 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dbf4e10240 con 0x55dbf4b001b0 2024-05-09T05:55:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f1389fc5640 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dbf4b001b0 0x55dbf4f29cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f136d7fa640 10 client.18873 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:10.142+0000 7f1389fc5640 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dbf4b001b0 0x55dbf4f29cf0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b272c502501c757 server_cookie=3faa5d961523d59c in_seq=0 out_seq=0 2024-05-09T05:55:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.784+0000 7f136d7fa640 1 -- 172.21.15.115:0/2787134546 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dbf4e10240 con 0x55dbf4b001b0 2024-05-09T05:55:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.784+0000 7f136d7fa640 10 client.18873 handle_command_reply: tid=0 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f13931b1740 1 client.18873 shutdown 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f13931b1740 1 -- 172.21.15.115:0/2787134546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4ad72f0 msgr2=0x55dbf4eb5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f13931b1740 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eb5780 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f13740057c0 tx=0x7f137402a410 comp rx=0 tx=0).stop 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f13931b1740 1 -- 172.21.15.115:0/2787134546 shutdown_connections 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f13931b1740 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55dbf4b001b0 0x55dbf4f29cf0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f13931b1740 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dbf4adf4d0 0x55dbf4f26110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f13931b1740 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dbf4ad72f0 0x55dbf4eb5780 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f13931b1740 1 --2- 172.21.15.115:0/2787134546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbf4abed80 0x55dbf4eb5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f136d7fa640 0 client.18873 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.785+0000 7f13931b1740 1 -- 172.21.15.115:0/2787134546 >> 172.21.15.115:0/2787134546 conn(0x55dbf4af4300 msgr2=0x55dbf4ec4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.786+0000 7f13931b1740 1 -- 172.21.15.115:0/2787134546 shutdown_connections 2024-05-09T05:55:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.786+0000 7f13931b1740 1 -- 172.21.15.115:0/2787134546 wait complete. 2024-05-09T05:55:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.786+0000 7f13931b1740 20 client.18873 trim_cache size 0 max 16384 2024-05-09T05:55:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:55:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:55:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:55:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.788+0000 7f138c229640 1 -- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f13700b29d0 msgr2=0x7f13700b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.788+0000 7f138c229640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f13700b29d0 0x7f13700b4ee0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1374004770 tx=0x7f13740058d0 comp rx=0 tx=0).stop 2024-05-09T05:55:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.788+0000 7f138c229640 1 -- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f138410e6d0 msgr2=0x7f138419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.788+0000 7f138c229640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f138410e6d0 0x7f138419efc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f137c00dac0 tx=0x7f137c00df80 comp rx=0 tx=0).stop 2024-05-09T05:55:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.789+0000 7f138c229640 1 -- 172.21.15.115:0/2098855752 shutdown_connections 2024-05-09T05:55:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.789+0000 7f138c229640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f13700b29d0 0x7f13700b4ee0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.789+0000 7f138c229640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f138410e6d0 0x7f138419efc0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.789+0000 7f138c229640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1384109d20 0x7f138419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.789+0000 7f138c229640 1 --2- 172.21.15.115:0/2098855752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13841021d0 0x7f138419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.789+0000 7f138c229640 1 -- 172.21.15.115:0/2098855752 >> 172.21.15.115:0/2098855752 conn(0x7f13840779e0 msgr2=0x7f1384105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.790+0000 7f138c229640 1 -- 172.21.15.115:0/2098855752 shutdown_connections 2024-05-09T05:55:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:11.790+0000 7f138c229640 1 -- 172.21.15.115:0/2098855752 wait complete. 2024-05-09T05:55:11.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:11 smithi195 ceph-mon[33844]: pgmap v534: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-05-09T05:55:11.876 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:11 smithi115 ceph-mon[30349]: pgmap v534: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-05-09T05:55:11.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:11 smithi161 ceph-mon[33805]: pgmap v534: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-05-09T05:55:13.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.467+0000 7fafbbe59700 1 -- 172.21.15.115:0/1115174681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 msgr2=0x7fafb4108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:13.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.467+0000 7fafbbe59700 1 --2- 172.21.15.115:0/1115174681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 0x7fafb4108e00 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009990 tx=0x7fafa8009ca0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.467+0000 7fafbbe59700 1 -- 172.21.15.115:0/1115174681 shutdown_connections 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.467+0000 7fafbbe59700 1 --2- 172.21.15.115:0/1115174681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafb4103310 0x7fafb410f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.467+0000 7fafbbe59700 1 --2- 172.21.15.115:0/1115174681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 0x7fafb4108e00 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.467+0000 7fafbbe59700 1 --2- 172.21.15.115:0/1115174681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb41029c0 0x7fafb4102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.467+0000 7fafbbe59700 1 -- 172.21.15.115:0/1115174681 >> 172.21.15.115:0/1115174681 conn(0x7fafb40fe500 msgr2=0x7fafb4100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.467+0000 7fafbbe59700 1 -- 172.21.15.115:0/1115174681 shutdown_connections 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.467+0000 7fafbbe59700 1 -- 172.21.15.115:0/1115174681 wait complete. 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafbbe59700 1 Processor -- start 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafbbe59700 1 -- start start 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafbbe59700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafb41029c0 0x7fafb4199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafbbe59700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb4103310 0x7fafb419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafbbe59700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 0x7fafb419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafbbe59700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4113240 con 0x7fafb41089f0 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafbbe59700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41130c0 con 0x7fafb41029c0 2024-05-09T05:55:13.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafbbe59700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41133c0 con 0x7fafb4103310 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafba3f6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 0x7fafb419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafba3f6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 0x7fafb419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57502/0 (socket says 172.21.15.115:57502) 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafba3f6700 1 -- 172.21.15.115:0/682640413 learned_addr learned my addr 172.21.15.115:0/682640413 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafb41029c0 0x7fafb4199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.468+0000 7fafb93f4700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb4103310 0x7fafb419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafba3f6700 1 -- 172.21.15.115:0/682640413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb4103310 msgr2=0x7fafb419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafba3f6700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb4103310 0x7fafb419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafba3f6700 1 -- 172.21.15.115:0/682640413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafb41029c0 msgr2=0x7fafb4199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafba3f6700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafb41029c0 0x7fafb4199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafba3f6700 1 -- 172.21.15.115:0/682640413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb419ec20 con 0x7fafb41089f0 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafba3f6700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 0x7fafb419e580 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fafac007f10 tx=0x7fafac00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafa6ffd700 1 -- 172.21.15.115:0/682640413 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac016030 con 0x7fafb41089f0 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafbbe59700 1 -- 172.21.15.115:0/682640413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb419ef10 con 0x7fafb41089f0 2024-05-09T05:55:13.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafa6ffd700 1 -- 172.21.15.115:0/682640413 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafac00fd30 con 0x7fafb41089f0 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafa6ffd700 1 -- 172.21.15.115:0/682640413 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac012d90 con 0x7fafb41089f0 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.469+0000 7fafbbe59700 1 -- 172.21.15.115:0/682640413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb410fe00 con 0x7fafb41089f0 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.472+0000 7fafa6ffd700 1 -- 172.21.15.115:0/682640413 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fafac005000 con 0x7fafb41089f0 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.472+0000 7fafa6ffd700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafa00b29f0 0x7fafa00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafa00b29f0 0x7fafa00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafa6ffd700 1 -- 172.21.15.115:0/682640413 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7fafac0e4bb0 con 0x7fafb41089f0 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafca86fb80 1 Processor -- start 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafca86fb80 1 -- start start 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafa00b29f0 0x7fafa00b4f00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fafa800b5c0 tx=0x7fafa8005960 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x55765393ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x55765393f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x55765394bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafca86fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557653862230 con 0x55765383ce90 2024-05-09T05:55:13.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafca86fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557653807b30 con 0x557653949990 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafca86fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55765385e1f0 con 0x5576537c2630 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafb9bf5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x55765393ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafb9bf5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x55765393ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57512/0 (socket says 172.21.15.115:57512) 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafb9bf5700 1 -- 172.21.15.115:0/767466256 learned_addr learned my addr 172.21.15.115:0/767466256 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafb93f4700 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x55765393f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.473+0000 7fafba3f6700 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x55765394bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7fafb9bf5700 1 -- 172.21.15.115:0/767466256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 msgr2=0x55765393f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x55765393f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7fafb9bf5700 1 -- 172.21.15.115:0/767466256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 msgr2=0x55765394bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x55765394bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7fafb9bf5700 1 -- 172.21.15.115:0/767466256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576537f8690 con 0x55765383ce90 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7fafb93f4700 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x55765393f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7fafba3f6700 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x55765394bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:55:13.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x55765393ece0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009990 tx=0x7fafa801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bec9368c1d1c60f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7faf9ffff700 1 -- 172.21.15.115:0/767466256 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa800bbb0 con 0x55765383ce90 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7faf9ffff700 1 -- 172.21.15.115:0/767466256 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafa800bd50 con 0x55765383ce90 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.474+0000 7faf9ffff700 1 -- 172.21.15.115:0/767466256 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa800f9b0 con 0x55765383ce90 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 -- 172.21.15.115:0/767466256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 msgr2=0x55765393ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x55765393ece0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009990 tx=0x7fafa801e000 comp rx=0 tx=0).stop 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 -- 172.21.15.115:0/767466256 shutdown_connections 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x55765394bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x55765393ece0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 --2- 172.21.15.115:0/767466256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x55765393f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 -- 172.21.15.115:0/767466256 >> 172.21.15.115:0/767466256 conn(0x5576537c6100 msgr2=0x55765393abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 -- 172.21.15.115:0/767466256 shutdown_connections 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 -- 172.21.15.115:0/767466256 wait complete. 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 -- start start 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.475+0000 7fafca86fb80 1 Processor -- start 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653946a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x557653946fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x5576539474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafca86fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557653950700 con 0x55765383ce90 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafca86fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5576537fccb0 con 0x557653949990 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafca86fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557653950880 con 0x5576537c2630 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafca86fb80 10 client.0 _ll_register_callbacks cb 0x5576536d3850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb93f4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x557653946fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb9bf5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653946a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb93f4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x557653946fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:57528/0 (socket says 172.21.15.115:57528) 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb93f4700 1 -- 172.21.15.115:0/3995756784 learned_addr learned my addr 172.21.15.115:0/3995756784 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafba3f6700 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x5576539474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb93f4700 1 -- 172.21.15.115:0/3995756784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 msgr2=0x557653946a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb93f4700 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653946a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb93f4700 1 -- 172.21.15.115:0/3995756784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 msgr2=0x5576539474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:13.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb93f4700 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x5576539474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb93f4700 1 -- 172.21.15.115:0/3995756784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576536dd620 con 0x55765383ce90 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653946a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafb93f4700 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x557653946fa0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fafb0011470 tx=0x7fafb0011780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898f243602ea7241 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7faf9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7faf9d7fa700 1 -- 172.21.15.115:0/3995756784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0004030 con 0x55765383ce90 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7faf9d7fa700 1 -- 172.21.15.115:0/3995756784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7fafb00041d0 con 0x55765383ce90 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.476+0000 7fafca86fb80 1 -- 172.21.15.115:0/3995756784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa800bd50 con 0x55765383ce90 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7faf9d7fa700 1 -- 172.21.15.115:0/3995756784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fafb0004410 con 0x55765383ce90 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7fafca86fb80 10 client.18885 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7fafca86fb80 1 -- 172.21.15.115:0/3995756784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55765395b4e0 con 0x55765383ce90 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7faf9d7fa700 1 -- 172.21.15.115:0/3995756784 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafa800bd50 con 0x55765383ce90 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7faf9d7fa700 1 -- 172.21.15.115:0/3995756784 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7fafb0003940 con 0x55765383ce90 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7fafca86fb80 10 client.18885 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7fafca86fb80 10 client.18885 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:55:13.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7fafca86fb80 20 client.18885 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7fafca86fb80 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55765395db20 0x55765395ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7fafca86fb80 4 client.18885 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7fafca86fb80 1 -- 172.21.15.115:0/3995756784 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55765386fc10 con 0x55765395db20 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.477+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55765395db20 0x55765395ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.478+0000 7faf9d7fa700 10 client.18885 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.478+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55765395db20 0x55765395ff40 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e721bd5113a5b0a server_cookie=15b2c1273e8363d4 in_seq=0 out_seq=0 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.478+0000 7faf9d7fa700 1 -- 172.21.15.115:0/3995756784 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55765386fc10 con 0x55765395db20 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.478+0000 7faf9d7fa700 10 client.18885 handle_command_reply: tid=0 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.478+0000 7fafca86fb80 1 client.18885 shutdown 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 -- 172.21.15.115:0/3995756784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 msgr2=0x557653946fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x557653946fa0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fafb0011470 tx=0x7fafb0011780 comp rx=0 tx=0).stop 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 -- 172.21.15.115:0/3995756784 shutdown_connections 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x5576539474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55765395db20 0x55765395ff40 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7faf9d7fa700 0 client.18885 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653946a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 --2- 172.21.15.115:0/3995756784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x557653946fa0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 -- 172.21.15.115:0/3995756784 >> 172.21.15.115:0/3995756784 conn(0x5576537c6100 msgr2=0x557653939e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 -- 172.21.15.115:0/3995756784 shutdown_connections 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 1 -- 172.21.15.115:0/3995756784 wait complete. 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.479+0000 7fafca86fb80 20 client.18885 trim_cache size 0 max 16384 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.512+0000 7fafca86fb80 1 Processor -- start 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.513+0000 7fafca86fb80 1 -- start start 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.513+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653959790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.514+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x557653959cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.514+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x55765394adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.514+0000 7fafca86fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41ae390 con 0x55765383ce90 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.514+0000 7fafb9bf5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653959790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.514+0000 7fafb9bf5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653959790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:38540/0 (socket says 172.21.15.115:38540) 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.514+0000 7fafb9bf5700 1 -- 172.21.15.115:0/1832297392 learned_addr learned my addr 172.21.15.115:0/1832297392 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.514+0000 7fafca86fb80 1 -- 172.21.15.115:0/1832297392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41afed0 con 0x557653949990 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.514+0000 7fafca86fb80 1 -- 172.21.15.115:0/1832297392 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41aea60 con 0x5576537c2630 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.514+0000 7fafba3f6700 1 --2- 172.21.15.115:0/1832297392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x55765394adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.515+0000 7fafb9bf5700 1 -- 172.21.15.115:0/1832297392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 msgr2=0x55765394adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.515+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/1832297392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x55765394adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.515+0000 7fafb9bf5700 1 -- 172.21.15.115:0/1832297392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 msgr2=0x557653959cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.515+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/1832297392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x557653959cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.515+0000 7fafb9bf5700 1 -- 172.21.15.115:0/1832297392 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55765393b580 con 0x5576537c2630 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.515+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/1832297392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653959790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fafa80059d0 tx=0x7fafa801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc234ecef3a2da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.517+0000 7faf9f7fe700 1 -- 172.21.15.115:0/1832297392 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8041070 con 0x5576537c2630 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.517+0000 7faf9f7fe700 1 -- 172.21.15.115:0/1832297392 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafa800be90 con 0x5576537c2630 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.517+0000 7faf9f7fe700 1 -- 172.21.15.115:0/1832297392 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa803c070 con 0x5576537c2630 2024-05-09T05:55:13.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.518+0000 7fafca86fb80 1 -- 172.21.15.115:0/1832297392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 msgr2=0x557653959790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.518+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1832297392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653959790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fafa80059d0 tx=0x7fafa801a040 comp rx=0 tx=0).stop 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.520+0000 7fafca86fb80 1 -- 172.21.15.115:0/1832297392 shutdown_connections 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.520+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1832297392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557653949990 0x55765394adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.520+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1832297392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653959790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.520+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1832297392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55765383ce90 0x557653959cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.520+0000 7fafca86fb80 1 -- 172.21.15.115:0/1832297392 >> 172.21.15.115:0/1832297392 conn(0x5576537c6100 msgr2=0x557653946310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.530+0000 7fafca86fb80 1 -- 172.21.15.115:0/1832297392 shutdown_connections 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.530+0000 7fafca86fb80 1 -- 172.21.15.115:0/1832297392 wait complete. 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.530+0000 7fafca86fb80 1 -- start start 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.530+0000 7fafca86fb80 1 Processor -- start 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.531+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653955fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.531+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55765383ce90 0x55765394f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.531+0000 7fafca86fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557653949990 0x55765394fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.531+0000 7fafca86fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41af800 con 0x557653949990 2024-05-09T05:55:13.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.531+0000 7fafca86fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41af130 con 0x55765383ce90 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.531+0000 7fafca86fb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41ac350 con 0x5576537c2630 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.531+0000 7fafca86fb80 10 client.0 _ll_register_callbacks cb 0x5576536d3850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.533+0000 7fafb93f4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55765383ce90 0x55765394f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.533+0000 7fafb9bf5700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653955fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.533+0000 7fafb93f4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55765383ce90 0x55765394f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:59202/0 (socket says 172.21.15.115:59202) 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.533+0000 7fafb93f4700 1 -- 172.21.15.115:0/1943900625 learned_addr learned my addr 172.21.15.115:0/1943900625 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.533+0000 7fafb93f4700 1 -- 172.21.15.115:0/1943900625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 msgr2=0x557653955fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.533+0000 7fafb93f4700 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653955fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.533+0000 7fafb93f4700 1 -- 172.21.15.115:0/1943900625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557653949990 msgr2=0x55765394fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.533+0000 7fafb93f4700 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557653949990 0x55765394fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.533+0000 7fafb93f4700 1 -- 172.21.15.115:0/1943900625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576539502c0 con 0x55765383ce90 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.534+0000 7fafb93f4700 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55765383ce90 0x55765394f740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fafb0003f50 tx=0x7fafb0005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de24f1a61e5981fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.535+0000 7faf9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:55:13.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.535+0000 7faf9d7fa700 1 -- 172.21.15.115:0/1943900625 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb002d020 con 0x55765383ce90 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.535+0000 7faf9d7fa700 1 -- 172.21.15.115:0/1943900625 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9558+0+0 (secure 0 0 0) 0x7fafb0003950 con 0x55765383ce90 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.536+0000 7fafca86fb80 1 -- 172.21.15.115:0/1943900625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa800be90 con 0x55765383ce90 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.537+0000 7faf9d7fa700 1 -- 172.21.15.115:0/1943900625 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fafb0003b50 con 0x55765383ce90 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.538+0000 7fafca86fb80 10 client.28088 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.538+0000 7fafca86fb80 1 -- 172.21.15.115:0/1943900625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x557653952af0 con 0x55765383ce90 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.539+0000 7faf9d7fa700 1 -- 172.21.15.115:0/1943900625 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafa800be90 con 0x55765383ce90 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.539+0000 7faf9d7fa700 1 -- 172.21.15.115:0/1943900625 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7fafb0010af0 con 0x55765383ce90 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.539+0000 7fafca86fb80 10 client.28088 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.539+0000 7fafca86fb80 10 client.28088 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.539+0000 7fafca86fb80 20 client.28088 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.539+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55765395db20 0x557653953200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.539+0000 7fafca86fb80 4 client.28088 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.1, "await_for": 45.0}] 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.539+0000 7fafca86fb80 1 -- 172.21.15.115:0/1943900625 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.1, "await_for": 45.0}) -- 0x557653869cb0 con 0x55765395db20 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.540+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55765395db20 0x557653953200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:13.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.544+0000 7fafb9bf5700 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55765395db20 0x557653953200 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94eb86a791ff0737 server_cookie=e249ce9c2e134758 in_seq=0 out_seq=0 2024-05-09T05:55:13.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:12.544+0000 7faf9d7fa700 10 client.28088 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:13.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:13 smithi195 ceph-mon[33844]: pgmap v535: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.0 MiB/s wr, 2.29k op/s 2024-05-09T05:55:13.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:13 smithi115 ceph-mon[30349]: pgmap v535: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.0 MiB/s wr, 2.29k op/s 2024-05-09T05:55:13.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:13 smithi161 ceph-mon[33805]: pgmap v535: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.0 MiB/s wr, 2.29k op/s 2024-05-09T05:55:15.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:14 smithi195 ceph-mon[33844]: pgmap v536: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-05-09T05:55:15.142 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:14 smithi115 ceph-mon[30349]: pgmap v536: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-05-09T05:55:15.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:14 smithi161 ceph-mon[33805]: pgmap v536: 129 pgs: 7 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 115 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-05-09T05:55:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7faf9d7fa700 1 -- 172.21.15.115:0/1943900625 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557653869cb0 con 0x55765395db20 2024-05-09T05:55:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7faf9d7fa700 10 client.28088 handle_command_reply: tid=0 2024-05-09T05:55:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 client.28088 shutdown 2024-05-09T05:55:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 -- 172.21.15.115:0/1943900625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55765383ce90 msgr2=0x55765394f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55765383ce90 0x55765394f740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fafb0003f50 tx=0x7fafb0005ce0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 -- 172.21.15.115:0/1943900625 shutdown_connections 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55765395db20 0x557653953200 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557653949990 0x55765394fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576537c2630 0x557653955fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 --2- 172.21.15.115:0/1943900625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55765383ce90 0x55765394f740 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 -- 172.21.15.115:0/1943900625 >> 172.21.15.115:0/1943900625 conn(0x5576537c6100 msgr2=0x557653946cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7faf9d7fa700 0 client.28088 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 -- 172.21.15.115:0/1943900625 shutdown_connections 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 1 -- 172.21.15.115:0/1943900625 wait complete. 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.653+0000 7fafca86fb80 20 client.28088 trim_cache size 0 max 16384 2024-05-09T05:55:15.644 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 66, 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 46, 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "82569837": { 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 46, 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 147.1, 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:55:15.645 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:15.646 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:55:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.654+0000 7fafbbe59700 1 -- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafa00b29f0 msgr2=0x7fafa00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.654+0000 7fafbbe59700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafa00b29f0 0x7fafa00b4f00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fafa800b5c0 tx=0x7fafa8005960 comp rx=0 tx=0).stop 2024-05-09T05:55:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.654+0000 7fafbbe59700 1 -- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 msgr2=0x7fafb419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.654+0000 7fafbbe59700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 0x7fafb419e580 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fafac007f10 tx=0x7fafac00f5c0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.655+0000 7fafbbe59700 1 -- 172.21.15.115:0/682640413 shutdown_connections 2024-05-09T05:55:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.655+0000 7fafbbe59700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fafa00b29f0 0x7fafa00b4f00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.655+0000 7fafbbe59700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafb41029c0 0x7fafb4199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.655+0000 7fafbbe59700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb41089f0 0x7fafb419e580 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.655+0000 7fafbbe59700 1 --2- 172.21.15.115:0/682640413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb4103310 0x7fafb419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.655+0000 7fafbbe59700 1 -- 172.21.15.115:0/682640413 >> 172.21.15.115:0/682640413 conn(0x7fafb40fe500 msgr2=0x7fafb4100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.655+0000 7fafbbe59700 1 -- 172.21.15.115:0/682640413 shutdown_connections 2024-05-09T05:55:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:15.655+0000 7fafbbe59700 1 -- 172.21.15.115:0/682640413 wait complete. 2024-05-09T05:55:17.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:17 smithi195 ceph-mon[33844]: pgmap v537: 129 pgs: 7 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 117 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 2.59k op/s 2024-05-09T05:55:17.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:17 smithi115 ceph-mon[30349]: pgmap v537: 129 pgs: 7 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 117 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 2.59k op/s 2024-05-09T05:55:17.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:17 smithi161 ceph-mon[33805]: pgmap v537: 129 pgs: 7 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 117 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 2.59k op/s 2024-05-09T05:55:19.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:19 smithi195 ceph-mon[33844]: pgmap v538: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 MiB/s wr, 2.85k op/s 2024-05-09T05:55:19.611 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:19 smithi195 ceph-mon[33844]: osdmap e111: 12 total, 12 up, 12 in 2024-05-09T05:55:19.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:19 smithi115 ceph-mon[30349]: pgmap v538: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 MiB/s wr, 2.85k op/s 2024-05-09T05:55:19.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:19 smithi115 ceph-mon[30349]: osdmap e111: 12 total, 12 up, 12 in 2024-05-09T05:55:19.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:19 smithi161 ceph-mon[33805]: pgmap v538: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 MiB/s wr, 2.85k op/s 2024-05-09T05:55:19.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:19 smithi161 ceph-mon[33805]: osdmap e111: 12 total, 12 up, 12 in 2024-05-09T05:55:21.360 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:20 smithi195 ceph-mon[33844]: osdmap e112: 12 total, 12 up, 12 in 2024-05-09T05:55:21.360 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:20 smithi195 ceph-mon[33844]: pgmap v541: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-09T05:55:21.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:20 smithi115 ceph-mon[30349]: osdmap e112: 12 total, 12 up, 12 in 2024-05-09T05:55:21.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:20 smithi115 ceph-mon[30349]: pgmap v541: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-09T05:55:21.405 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:20 smithi161 ceph-mon[33805]: osdmap e112: 12 total, 12 up, 12 in 2024-05-09T05:55:21.405 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:20 smithi161 ceph-mon[33805]: pgmap v541: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-09T05:55:23.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:23 smithi195 ceph-mon[33844]: pgmap v542: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.2 MiB/s wr, 1.31k op/s 2024-05-09T05:55:23.614 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:23 smithi115 ceph-mon[30349]: pgmap v542: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.2 MiB/s wr, 1.31k op/s 2024-05-09T05:55:23.646 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '82569837', quiesce took 73.7 seconds. Will release after: -46.6 2024-05-09T05:55:23.647 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:55:23.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:23 smithi161 ceph-mon[33805]: pgmap v542: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.2 MiB/s wr, 1.31k op/s 2024-05-09T05:55:23.897 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.921+0000 7f1cb8538640 1 -- 172.21.15.115:0/3382373948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 msgr2=0x7f1cb0102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.921+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3382373948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 0x7f1cb0102c80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009900 tx=0x7f1ca002a410 comp rx=0 tx=0).stop 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.921+0000 7f1cb8538640 1 -- 172.21.15.115:0/3382373948 shutdown_connections 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.921+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3382373948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb01032f0 0x7f1cb010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.921+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3382373948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 0x7f1cb0102c80 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.921+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3382373948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cb0108ba0 0x7f1cb0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.921+0000 7f1cb8538640 1 -- 172.21.15.115:0/3382373948 >> 172.21.15.115:0/3382373948 conn(0x7f1cb00782e0 msgr2=0x7f1cb0078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.921+0000 7f1cb8538640 1 -- 172.21.15.115:0/3382373948 shutdown_connections 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.921+0000 7f1cb8538640 1 -- 172.21.15.115:0/3382373948 wait complete. 2024-05-09T05:55:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb8538640 1 Processor -- start 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb8538640 1 -- start start 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb8538640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cb01032f0 0x7f1cb0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb8538640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb0108ba0 0x7f1cb0199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb8538640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 0x7f1cb019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb8538640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0112c90 con 0x7f1cb0109040 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb8538640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0112b10 con 0x7f1cb01032f0 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb8538640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0112e10 con 0x7f1cb0108ba0 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb6ad5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 0x7f1cb019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb6ad5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 0x7f1cb019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53840/0 (socket says 172.21.15.115:53840) 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.922+0000 7f1cb6ad5640 1 -- 172.21.15.115:0/3164079455 learned_addr learned my addr 172.21.15.115:0/3164079455 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb0108ba0 0x7f1cb0199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cb01032f0 0x7f1cb0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb6ad5640 1 -- 172.21.15.115:0/3164079455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb0108ba0 msgr2=0x7f1cb0199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb6ad5640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb0108ba0 0x7f1cb0199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb6ad5640 1 -- 172.21.15.115:0/3164079455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cb01032f0 msgr2=0x7f1cb0199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb6ad5640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cb01032f0 0x7f1cb0199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb6ad5640 1 -- 172.21.15.115:0/3164079455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb019e5d0 con 0x7f1cb0109040 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cb01032f0 0x7f1cb0199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb6ad5640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 0x7f1cb019df90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800eb10 tx=0x7f1ca800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1ca77fe640 1 -- 172.21.15.115:0/3164079455 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8004b20 con 0x7f1cb0109040 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1ca77fe640 1 -- 172.21.15.115:0/3164079455 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ca8004cc0 con 0x7f1cb0109040 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb8538640 1 -- 172.21.15.115:0/3164079455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb019e8c0 con 0x7f1cb0109040 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1ca77fe640 1 -- 172.21.15.115:0/3164079455 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8010690 con 0x7f1cb0109040 2024-05-09T05:55:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.923+0000 7f1cb8538640 1 -- 172.21.15.115:0/3164079455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb01a6240 con 0x7f1cb0109040 2024-05-09T05:55:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.926+0000 7f1ca77fe640 1 -- 172.21.15.115:0/3164079455 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1ca8010830 con 0x7f1cb0109040 2024-05-09T05:55:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.926+0000 7f1cbf3eb740 1 Processor -- start 2024-05-09T05:55:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.926+0000 7f1cbf3eb740 1 -- start start 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d31680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 0x563815d31bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815d32100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563815c967c0 con 0x56381596d4d0 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563815c320e0 con 0x5638159652f0 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563815c92780 con 0x56381598e1b0 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1ca77fe640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1c9c0b2950 0x7f1c9c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb5ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815d32100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb5ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815d32100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53854/0 (socket says 172.21.15.115:53854) 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/1051258518 learned_addr learned my addr 172.21.15.115:0/1051258518 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/1051258518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d31680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/1051258518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 0x563815d31bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb6ad5640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1c9c0b2950 0x7f1c9c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1ca77fe640 1 -- 172.21.15.115:0/3164079455 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9482+0+0 (secure 0 0 0) 0x7f1ca80e2fe0 con 0x7f1cb0109040 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/1051258518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 msgr2=0x563815d31680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/1051258518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d31680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/1051258518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 msgr2=0x563815d31bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/1051258518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 0x563815d31bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.927+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/1051258518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563815c5b7c0 con 0x56381596d4d0 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.928+0000 7f1cb6ad5640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1c9c0b2950 0x7f1c9c0b4e60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80e0970 tx=0x7f1ca80737a0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.928+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/1051258518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815d32100 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0000c00 tx=0x7f1ca0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4294c1dd5ffd5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.928+0000 7f1ca4bfe640 1 -- 172.21.15.115:0/1051258518 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca003f070 con 0x56381596d4d0 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.928+0000 7f1ca4bfe640 1 -- 172.21.15.115:0/1051258518 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ca0003a80 con 0x56381596d4d0 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.928+0000 7f1ca4bfe640 1 -- 172.21.15.115:0/1051258518 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0035040 con 0x56381596d4d0 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/1051258518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 msgr2=0x563815d32100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/1051258518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815d32100 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0000c00 tx=0x7f1ca0009560 comp rx=0 tx=0).stop 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/1051258518 shutdown_connections 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/1051258518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815d32100 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/1051258518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 0x563815d31bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/1051258518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d31680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/1051258518 >> 172.21.15.115:0/1051258518 conn(0x563815982300 msgr2=0x563815d2de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/1051258518 shutdown_connections 2024-05-09T05:55:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/1051258518 wait complete. 2024-05-09T05:55:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 -- start start 2024-05-09T05:55:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.929+0000 7f1cbf3eb740 1 Processor -- start 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638159652f0 0x563815d3d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d3dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d3bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563815d3e750 con 0x5638159652f0 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563815c1d9c0 con 0x56381596d4d0 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563815d3e8d0 con 0x56381598e1b0 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cbf3eb740 10 client.0 _ll_register_callbacks cb 0x563815c4a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb5ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638159652f0 0x563815d3d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb5ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638159652f0 0x563815d3d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53870/0 (socket says 172.21.15.115:53870) 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/3538215357 learned_addr learned my addr 172.21.15.115:0/3538215357 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d3dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/3538215357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 msgr2=0x563815d3bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d3bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/3538215357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 msgr2=0x563815d3dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d3dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/3538215357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563815c441f0 con 0x5638159652f0 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d3dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638159652f0 0x563815d3d780 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00092a0 tx=0x7f1ca0003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618f76fa1d5bfe95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.930+0000 7f1c99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1c99ffb640 1 -- 172.21.15.115:0/3538215357 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca003f030 con 0x5638159652f0 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3538215357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca0003a80 con 0x5638159652f0 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1c99ffb640 1 -- 172.21.15.115:0/3538215357 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9482+0+0 (secure 0 0 0) 0x7f1ca003b630 con 0x5638159652f0 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1c99ffb640 1 -- 172.21.15.115:0/3538215357 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f1ca0035040 con 0x5638159652f0 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1cbf3eb740 10 client.18897 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3538215357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5638159f9790 con 0x5638159652f0 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1c99ffb640 1 -- 172.21.15.115:0/3538215357 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ca0003a80 con 0x5638159652f0 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1c99ffb640 1 -- 172.21.15.115:0/3538215357 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7f1ca004a6c0 con 0x5638159652f0 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1cbf3eb740 10 client.18897 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1cbf3eb740 10 client.18897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1cbf3eb740 20 client.18897 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56381594cd80 0x563815d4ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1cbf3eb740 4 client.18897 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.931+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3538215357 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563815ca43d0 con 0x56381594cd80 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.932+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56381594cd80 0x563815d4ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.932+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56381594cd80 0x563815d4ddc0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897e5e8d68ec15fb server_cookie=d7d33af17a3c0c2e in_seq=0 out_seq=0 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.932+0000 7f1c99ffb640 10 client.18897 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1c99ffb640 1 -- 172.21.15.115:0/3538215357 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563815ca43d0 con 0x56381594cd80 2024-05-09T05:55:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1c99ffb640 10 client.18897 handle_command_reply: tid=0 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 client.18897 shutdown 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3538215357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638159652f0 msgr2=0x563815d3d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638159652f0 0x563815d3d780 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00092a0 tx=0x7f1ca0003e60 comp rx=0 tx=0).stop 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3538215357 shutdown_connections 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56381594cd80 0x563815d4ddc0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d3bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1c99ffb640 0 client.18897 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d3dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3538215357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5638159652f0 0x563815d3d780 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3538215357 >> 172.21.15.115:0/3538215357 conn(0x563815982300 msgr2=0x563815d2c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3538215357 shutdown_connections 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3538215357 wait complete. 2024-05-09T05:55:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.933+0000 7f1cbf3eb740 20 client.18897 trim_cache size 0 max 16384 2024-05-09T05:55:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cbf3eb740 1 Processor -- start 2024-05-09T05:55:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cbf3eb740 1 -- start start 2024-05-09T05:55:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638159652f0 0x563815d3da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d50df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381598e1b0 0x563815d51330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01af020 con 0x56381598e1b0 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01b0b60 con 0x56381596d4d0 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01af6f0 con 0x5638159652f0 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cb5ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381598e1b0 0x563815d51330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cb5ad3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381598e1b0 0x563815d51330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53880/0 (socket says 172.21.15.115:53880) 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/3290293401 learned_addr learned my addr 172.21.15.115:0/3290293401 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638159652f0 0x563815d3da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.949+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d50df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/3290293401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638159652f0 msgr2=0x563815d3da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638159652f0 0x563815d3da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/3290293401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 msgr2=0x563815d50df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d50df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1cb5ad3640 1 -- 172.21.15.115:0/3290293401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563815a705f0 con 0x56381598e1b0 2024-05-09T05:55:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d50df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:55:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381598e1b0 0x563815d51330 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00044f0 tx=0x7f1ca0009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a657bde3fb8b47b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1c9bfff640 1 -- 172.21.15.115:0/3290293401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca003f040 con 0x56381598e1b0 2024-05-09T05:55:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1c9bfff640 1 -- 172.21.15.115:0/3290293401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ca004e030 con 0x56381598e1b0 2024-05-09T05:55:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.950+0000 7f1c9bfff640 1 -- 172.21.15.115:0/3290293401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00555d0 con 0x56381598e1b0 2024-05-09T05:55:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3290293401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381598e1b0 msgr2=0x563815d51330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381598e1b0 0x563815d51330 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00044f0 tx=0x7f1ca0009230 comp rx=0 tx=0).stop 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3290293401 shutdown_connections 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381598e1b0 0x563815d51330 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56381596d4d0 0x563815d50df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/3290293401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5638159652f0 0x563815d3da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3290293401 >> 172.21.15.115:0/3290293401 conn(0x563815982300 msgr2=0x563815d3d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3290293401 shutdown_connections 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/3290293401 wait complete. 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 -- start start 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.951+0000 7f1cbf3eb740 1 Processor -- start 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 0x563815dae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815daed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cbf3eb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d50590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01b0490 con 0x56381596d4d0 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01afdc0 con 0x5638159652f0 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cbf3eb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01ad020 con 0x56381598e1b0 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cbf3eb740 10 client.0 _ll_register_callbacks cb 0x7f1cac013790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb62d4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815daed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb62d4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815daed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53890/0 (socket says 172.21.15.115:53890) 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb62d4640 1 -- 172.21.15.115:0/2339811015 learned_addr learned my addr 172.21.15.115:0/2339811015 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 0x563815dae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d50590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb62d4640 1 -- 172.21.15.115:0/2339811015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 msgr2=0x563815d50590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d50590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb62d4640 1 -- 172.21.15.115:0/2339811015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 msgr2=0x563815dae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 0x563815dae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb62d4640 1 -- 172.21.15.115:0/2339811015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563815c5aed0 con 0x56381596d4d0 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cb62d4640 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815daed60 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1cac015eb0 tx=0x7f1cac013df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c25ce2ddb8c6bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1c99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:55:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1c99ffb640 1 -- 172.21.15.115:0/2339811015 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac015040 con 0x56381596d4d0 2024-05-09T05:55:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1c99ffb640 1 -- 172.21.15.115:0/2339811015 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9482+0+0 (secure 0 0 0) 0x7f1cac0151e0 con 0x56381596d4d0 2024-05-09T05:55:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.952+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/2339811015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca004e030 con 0x56381596d4d0 2024-05-09T05:55:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1c99ffb640 1 -- 172.21.15.115:0/2339811015 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f1cac031050 con 0x56381596d4d0 2024-05-09T05:55:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1cbf3eb740 10 client.18903 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/2339811015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x563815db5cf0 con 0x56381596d4d0 2024-05-09T05:55:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1c99ffb640 1 -- 172.21.15.115:0/2339811015 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ca004e030 con 0x56381596d4d0 2024-05-09T05:55:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1c99ffb640 1 -- 172.21.15.115:0/2339811015 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7f1cac0099f0 con 0x56381596d4d0 2024-05-09T05:55:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1cbf3eb740 10 client.18903 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1cbf3eb740 10 client.18903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:55:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1cbf3eb740 20 client.18903 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5638159221c0 0x563815db84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1cbf3eb740 4 client.18903 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:55:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/2339811015 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563815c9e240 con 0x5638159221c0 2024-05-09T05:55:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.953+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5638159221c0 0x563815db84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.954+0000 7f1c99ffb640 10 client.18903 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.954+0000 7f1cb5ad3640 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5638159221c0 0x563815db84b0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72805a24c73cbb64 server_cookie=56f4389a1b908a4e in_seq=0 out_seq=0 2024-05-09T05:55:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.965+0000 7f1c99ffb640 1 -- 172.21.15.115:0/2339811015 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563815c9e240 con 0x5638159221c0 2024-05-09T05:55:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1c99ffb640 10 client.18903 handle_command_reply: tid=0 2024-05-09T05:55:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 client.18903 shutdown 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/2339811015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 msgr2=0x563815daed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815daed60 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1cac015eb0 tx=0x7f1cac013df0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/2339811015 shutdown_connections 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5638159221c0 0x563815db84b0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56381598e1b0 0x563815d50590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1c99ffb640 0 client.18903 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56381596d4d0 0x563815daed60 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 --2- 172.21.15.115:0/2339811015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638159652f0 0x563815dae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/2339811015 >> 172.21.15.115:0/2339811015 conn(0x563815982300 msgr2=0x563815d3d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/2339811015 shutdown_connections 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 1 -- 172.21.15.115:0/2339811015 wait complete. 2024-05-09T05:55:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cbf3eb740 20 client.18903 trim_cache size 0 max 16384 2024-05-09T05:55:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:55:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:55:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:55:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cb8538640 1 -- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1c9c0b2950 msgr2=0x7f1c9c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.966+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1c9c0b2950 0x7f1c9c0b4e60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80e0970 tx=0x7f1ca80737a0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 -- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 msgr2=0x7f1cb019df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 0x7f1cb019df90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800eb10 tx=0x7f1ca800c2d0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 -- 172.21.15.115:0/3164079455 shutdown_connections 2024-05-09T05:55:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f1c9c0b2950 0x7f1c9c0b4e60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cb0109040 0x7f1cb019df90 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cb0108ba0 0x7f1cb0199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 --2- 172.21.15.115:0/3164079455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cb01032f0 0x7f1cb0199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 -- 172.21.15.115:0/3164079455 >> 172.21.15.115:0/3164079455 conn(0x7f1cb00782e0 msgr2=0x7f1cb01012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 -- 172.21.15.115:0/3164079455 shutdown_connections 2024-05-09T05:55:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:24.967+0000 7f1cb8538640 1 -- 172.21.15.115:0/3164079455 wait complete. 2024-05-09T05:55:25.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:24 smithi161 ceph-mon[33805]: osdmap e113: 12 total, 12 up, 12 in 2024-05-09T05:55:25.155 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:24 smithi161 ceph-mon[33805]: pgmap v544: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 KiB/s wr, 21 op/s 2024-05-09T05:55:25.360 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:24 smithi195 ceph-mon[33844]: osdmap e113: 12 total, 12 up, 12 in 2024-05-09T05:55:25.360 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:24 smithi195 ceph-mon[33844]: pgmap v544: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 KiB/s wr, 21 op/s 2024-05-09T05:55:25.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:24 smithi115 ceph-mon[30349]: osdmap e113: 12 total, 12 up, 12 in 2024-05-09T05:55:25.391 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:24 smithi115 ceph-mon[30349]: pgmap v544: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 KiB/s wr, 21 op/s 2024-05-09T05:55:27.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:27 smithi195 ceph-mon[33844]: pgmap v545: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 84 KiB/s wr, 24 op/s 2024-05-09T05:55:27.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:27 smithi115 ceph-mon[30349]: pgmap v545: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 84 KiB/s wr, 24 op/s 2024-05-09T05:55:27.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:27 smithi161 ceph-mon[33805]: pgmap v545: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 84 KiB/s wr, 24 op/s 2024-05-09T05:55:29.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:29 smithi195 ceph-mon[33844]: pgmap v546: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 300 KiB/s wr, 80 op/s 2024-05-09T05:55:29.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:29 smithi195 ceph-mon[33844]: osdmap e114: 12 total, 12 up, 12 in 2024-05-09T05:55:29.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:29 smithi115 ceph-mon[30349]: pgmap v546: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 300 KiB/s wr, 80 op/s 2024-05-09T05:55:29.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:29 smithi115 ceph-mon[30349]: osdmap e114: 12 total, 12 up, 12 in 2024-05-09T05:55:29.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:29 smithi161 ceph-mon[33805]: pgmap v546: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 300 KiB/s wr, 80 op/s 2024-05-09T05:55:29.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:29 smithi161 ceph-mon[33805]: osdmap e114: 12 total, 12 up, 12 in 2024-05-09T05:55:31.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.317+0000 7f851f22e700 1 -- 172.21.15.115:0/3510322035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518107e20 msgr2=0x7f8518108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:31.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.317+0000 7f851f22e700 1 --2- 172.21.15.115:0/3510322035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518107e20 0x7f8518108230 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f85000099c0 tx=0x7f8500009cd0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.317+0000 7f851f22e700 1 -- 172.21.15.115:0/3510322035 shutdown_connections 2024-05-09T05:55:31.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.317+0000 7f851f22e700 1 --2- 172.21.15.115:0/3510322035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518103540 0x7f8518103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.317+0000 7f851f22e700 1 --2- 172.21.15.115:0/3510322035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8518108770 0x7f85181100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.317+0000 7f851f22e700 1 --2- 172.21.15.115:0/3510322035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518107e20 0x7f8518108230 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.317+0000 7f851f22e700 1 -- 172.21.15.115:0/3510322035 >> 172.21.15.115:0/3510322035 conn(0x7f8518078e50 msgr2=0x7f8518079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.317+0000 7f851f22e700 1 -- 172.21.15.115:0/3510322035 shutdown_connections 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.318+0000 7f851f22e700 1 -- 172.21.15.115:0/3510322035 wait complete. 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.318+0000 7f851f22e700 1 Processor -- start 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.318+0000 7f851f22e700 1 -- start start 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.318+0000 7f851f22e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518103540 0x7f85181006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.318+0000 7f851f22e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518107e20 0x7f8518100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.318+0000 7f851f22e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8518108770 0x7f8518101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.318+0000 7f851f22e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85181132a0 con 0x7f8518103540 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.318+0000 7f851f22e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8518113120 con 0x7f8518107e20 2024-05-09T05:55:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.318+0000 7f851f22e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8518113420 con 0x7f8518108770 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851cfca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518103540 0x7f85181006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851cfca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518103540 0x7f85181006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53896/0 (socket says 172.21.15.115:53896) 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851cfca700 1 -- 172.21.15.115:0/3620711539 learned_addr learned my addr 172.21.15.115:0/3620711539 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851d7cb700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8518108770 0x7f8518101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f8517fff700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518107e20 0x7f8518100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851cfca700 1 -- 172.21.15.115:0/3620711539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8518108770 msgr2=0x7f8518101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851cfca700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8518108770 0x7f8518101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851cfca700 1 -- 172.21.15.115:0/3620711539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518107e20 msgr2=0x7f8518100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851cfca700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518107e20 0x7f8518100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851cfca700 1 -- 172.21.15.115:0/3620711539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85181a9220 con 0x7f8518103540 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851d7cb700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8518108770 0x7f8518101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f8517fff700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518107e20 0x7f8518100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851cfca700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518103540 0x7f85181006a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f850800ba50 tx=0x7f850800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f8515ffb700 1 -- 172.21.15.115:0/3620711539 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508019070 con 0x7f8518103540 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85181a9510 con 0x7f8518103540 2024-05-09T05:55:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f8515ffb700 1 -- 172.21.15.115:0/3620711539 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8508014440 con 0x7f8518103540 2024-05-09T05:55:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f8515ffb700 1 -- 172.21.15.115:0/3620711539 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85080075d0 con 0x7f8518103540 2024-05-09T05:55:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.319+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85181a9a50 con 0x7f8518103540 2024-05-09T05:55:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.320+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85180512c0 con 0x7f8518103540 2024-05-09T05:55:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.322+0000 7f8515ffb700 1 -- 172.21.15.115:0/3620711539 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f850801c070 con 0x7f8518103540 2024-05-09T05:55:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.323+0000 7f8515ffb700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85040b29f0 0x7f85040b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.323+0000 7f8517fff700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85040b29f0 0x7f85040b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.323+0000 7f8515ffb700 1 -- 172.21.15.115:0/3620711539 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f85080e2670 con 0x7f8518103540 2024-05-09T05:55:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.323+0000 7f8517fff700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85040b29f0 0x7f85040b4f00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8500009fd0 tx=0x7f8500005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:31.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.327+0000 7f8515ffb700 1 -- 172.21.15.115:0/3620711539 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8508023070 con 0x7f8518103540 2024-05-09T05:55:31.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.578+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8518101bd0 con 0x7f8518103540 2024-05-09T05:55:31.567 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:31 smithi115 ceph-mon[30349]: pgmap v548: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 486 KiB/s wr, 127 op/s 2024-05-09T05:55:31.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.578+0000 7f8515ffb700 1 -- 172.21.15.115:0/3620711539 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11951 (secure 0 0 0) 0x7f85080a9dd0 con 0x7f8518103540 2024-05-09T05:55:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:55:31.568 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:55:05.587682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:55:31.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85040b29f0 msgr2=0x7f85040b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:31.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85040b29f0 0x7f85040b4f00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8500009fd0 tx=0x7f8500005fb0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518103540 msgr2=0x7f85181006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:31.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518103540 0x7f85181006a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f850800ba50 tx=0x7f850800be10 comp rx=0 tx=0).stop 2024-05-09T05:55:31.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 shutdown_connections 2024-05-09T05:55:31.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85040b29f0 0x7f85040b4f00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8518108770 0x7f8518101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8518103540 0x7f85181006a0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 --2- 172.21.15.115:0/3620711539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518107e20 0x7f8518100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 >> 172.21.15.115:0/3620711539 conn(0x7f8518078e50 msgr2=0x7f8518075e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 shutdown_connections 2024-05-09T05:55:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:31.581+0000 7f851f22e700 1 -- 172.21.15.115:0/3620711539 wait complete. 2024-05-09T05:55:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 66 2024-05-09T05:55:31.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:31 smithi195 ceph-mon[33844]: pgmap v548: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 486 KiB/s wr, 127 op/s 2024-05-09T05:55:31.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:31 smithi161 ceph-mon[33805]: pgmap v548: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 486 KiB/s wr, 127 op/s 2024-05-09T05:55:32.357 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 82569837 --release --await' 2024-05-09T05:55:32.357 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id 82569837 --release --await 2024-05-09T05:55:32.601 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:55:32.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:32 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3620711539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:55:32.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:32 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3620711539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:55:32.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:32 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3620711539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:55:33.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:33 smithi195 ceph-mon[33844]: pgmap v549: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 467 KiB/s wr, 121 op/s 2024-05-09T05:55:33.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:33 smithi115 ceph-mon[30349]: pgmap v549: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 467 KiB/s wr, 121 op/s 2024-05-09T05:55:33.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:33 smithi161 ceph-mon[33805]: pgmap v549: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 467 KiB/s wr, 121 op/s 2024-05-09T05:55:35.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:35 smithi195 ceph-mon[33844]: pgmap v550: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 407 KiB/s wr, 108 op/s 2024-05-09T05:55:35.675 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:35 smithi161 ceph-mon[33805]: pgmap v550: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 407 KiB/s wr, 108 op/s 2024-05-09T05:55:35.675 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:35 smithi115 ceph-mon[30349]: pgmap v550: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 407 KiB/s wr, 108 op/s 2024-05-09T05:55:37.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:37 smithi195 ceph-mon[33844]: pgmap v551: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 415 KiB/s wr, 110 op/s 2024-05-09T05:55:37.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:37 smithi115 ceph-mon[30349]: pgmap v551: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 415 KiB/s wr, 110 op/s 2024-05-09T05:55:37.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:37 smithi161 ceph-mon[33805]: pgmap v551: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 415 KiB/s wr, 110 op/s 2024-05-09T05:55:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.088+0000 7ff300e89640 1 -- 172.21.15.115:0/2891727278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc108f70 msgr2=0x7ff2fc110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.088+0000 7ff300e89640 1 --2- 172.21.15.115:0/2891727278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc108f70 0x7ff2fc110710 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009ca0 tx=0x7ff2f402a8c0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.088+0000 7ff300e89640 1 -- 172.21.15.115:0/2891727278 shutdown_connections 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.088+0000 7ff300e89640 1 --2- 172.21.15.115:0/2891727278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc108f70 0x7ff2fc110710 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.088+0000 7ff300e89640 1 --2- 172.21.15.115:0/2891727278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc1084d0 0x7ff2fc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.088+0000 7ff300e89640 1 --2- 172.21.15.115:0/2891727278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc102450 0x7ff2fc102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.088+0000 7ff300e89640 1 -- 172.21.15.115:0/2891727278 >> 172.21.15.115:0/2891727278 conn(0x7ff2fc0779e0 msgr2=0x7ff2fc077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.088+0000 7ff300e89640 1 -- 172.21.15.115:0/2891727278 shutdown_connections 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.088+0000 7ff300e89640 1 -- 172.21.15.115:0/2891727278 wait complete. 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff300e89640 1 Processor -- start 2024-05-09T05:55:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff300e89640 1 -- start start 2024-05-09T05:55:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff300e89640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc102450 0x7ff2fc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff300e89640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc1084d0 0x7ff2fc19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff300e89640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108f70 0x7ff2fc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff300e89640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc113d80 con 0x7ff2fc1084d0 2024-05-09T05:55:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff300e89640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc113c00 con 0x7ff2fc108f70 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff300e89640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc113f00 con 0x7ff2fc102450 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc1084d0 0x7ff2fc19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc1084d0 0x7ff2fc19ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43436/0 (socket says 172.21.15.115:43436) 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/2323890361 learned_addr learned my addr 172.21.15.115:0/2323890361 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc102450 0x7ff2fc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2fad9d640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108f70 0x7ff2fc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/2323890361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc102450 msgr2=0x7ff2fc19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc102450 0x7ff2fc19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/2323890361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108f70 msgr2=0x7ff2fc19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108f70 0x7ff2fc19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.089+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/2323890361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc19f620 con 0x7ff2fc1084d0 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.090+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc1084d0 0x7ff2fc19ad20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400d950 tx=0x7ff2e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.090+0000 7ff2f3fff640 1 -- 172.21.15.115:0/2323890361 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4017070 con 0x7ff2fc1084d0 2024-05-09T05:55:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.090+0000 7ff300e89640 1 -- 172.21.15.115:0/2323890361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc19f910 con 0x7ff2fc1084d0 2024-05-09T05:55:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.090+0000 7ff2f3fff640 1 -- 172.21.15.115:0/2323890361 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2e4009c30 con 0x7ff2fc1084d0 2024-05-09T05:55:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.090+0000 7ff2f3fff640 1 -- 172.21.15.115:0/2323890361 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40047a0 con 0x7ff2fc1084d0 2024-05-09T05:55:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.090+0000 7ff300e89640 1 -- 172.21.15.115:0/2323890361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc1a7290 con 0x7ff2fc1084d0 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff307e19740 1 Processor -- start 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff307e19740 1 -- start start 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2f3fff640 1 -- 172.21.15.115:0/2323890361 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff2e4004020 con 0x7ff2fc1084d0 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff307e19740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597409680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff307e19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x560597409bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff307e19740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x56059740a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff307e19740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56059736e7c0 con 0x5605970454d0 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff307e19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56059730a0e0 con 0x56059703d2f0 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff307e19740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56059736a780 con 0x5605970661b0 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x56059740a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x56059740a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43452/0 (socket says 172.21.15.115:43452) 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/3251978347 learned_addr learned my addr 172.21.15.115:0/3251978347 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2fad9d640 1 --2- 172.21.15.115:0/3251978347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597409680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/3251978347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x560597409bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/3251978347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 msgr2=0x560597409680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/3251978347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597409680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/3251978347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 msgr2=0x560597409bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.093+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/3251978347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x560597409bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.094+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/3251978347 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605973337c0 con 0x5605970454d0 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.094+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/3251978347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x56059740a100 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40df9d0 tx=0x7ff2e40ea690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0646ca1c0b26eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.094+0000 7ff2f3fff640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2e80b2950 0x7ff2e80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.094+0000 7ff2fad9d640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2e80b2950 0x7ff2e80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.094+0000 7ff2f0d99640 1 -- 172.21.15.115:0/3251978347 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40f1070 con 0x5605970454d0 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.094+0000 7ff2f3fff640 1 -- 172.21.15.115:0/2323890361 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7ff2e40e26a0 con 0x7ff2fc1084d0 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.094+0000 7ff2f0d99640 1 -- 172.21.15.115:0/3251978347 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2e40a9100 con 0x5605970454d0 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.094+0000 7ff2f0d99640 1 -- 172.21.15.115:0/3251978347 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40a54b0 con 0x5605970454d0 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 -- 172.21.15.115:0/3251978347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 msgr2=0x56059740a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 --2- 172.21.15.115:0/3251978347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x56059740a100 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40df9d0 tx=0x7ff2e40ea690 comp rx=0 tx=0).stop 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff2fad9d640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2e80b2950 0x7ff2e80b4e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5605974067f0 tx=0x7ff2f4005c50 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 -- 172.21.15.115:0/3251978347 shutdown_connections 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 --2- 172.21.15.115:0/3251978347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x56059740a100 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 --2- 172.21.15.115:0/3251978347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x560597409bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 --2- 172.21.15.115:0/3251978347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597409680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 -- 172.21.15.115:0/3251978347 >> 172.21.15.115:0/3251978347 conn(0x56059705a300 msgr2=0x560597405e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 -- 172.21.15.115:0/3251978347 shutdown_connections 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 -- 172.21.15.115:0/3251978347 wait complete. 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 -- start start 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.095+0000 7ff307e19740 1 Processor -- start 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff307e19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x5605974156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff307e19740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597415c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff307e19740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597413be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff307e19740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5605974167b0 con 0x5605970454d0 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff307e19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5605972f59c0 con 0x56059703d2f0 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff307e19740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560597416930 con 0x5605970661b0 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff307e19740 10 client.0 _ll_register_callbacks cb 0x560597322020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597415c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597415c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43462/0 (socket says 172.21.15.115:43462) 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/3412228474 learned_addr learned my addr 172.21.15.115:0/3412228474 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x5605974156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2fad9d640 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597413be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/3412228474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 msgr2=0x560597413be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597413be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/3412228474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 msgr2=0x5605974156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x5605974156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/3412228474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059731c1f0 con 0x5605970454d0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597415c10 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40e2a50 tx=0x7ff2e4005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97416f603515087 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.096+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/3412228474 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40f1070 con 0x5605970454d0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/3412228474 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7ff2e40dfce0 con 0x5605970454d0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff307e19740 1 -- 172.21.15.115:0/3412228474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e40a9100 con 0x5605970454d0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/3412228474 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ff2e40a8d60 con 0x5605970454d0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff307e19740 10 client.18921 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff307e19740 1 -- 172.21.15.115:0/3412228474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5605970d1790 con 0x5605970454d0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/3412228474 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2e40a9100 con 0x5605970454d0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/3412228474 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7ff2e4004310 con 0x5605970454d0 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff307e19740 10 client.18921 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff307e19740 10 client.18921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff307e19740 20 client.18921 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff307e19740 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560597024d80 0x560597425e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff307e19740 4 client.18921 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff307e19740 1 -- 172.21.15.115:0/3412228474 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56059737c3d0 con 0x560597024d80 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.097+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560597024d80 0x560597425e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.098+0000 7ff2e1ffb640 10 client.18921 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.098+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560597024d80 0x560597425e10 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e159d30e9afb26fb server_cookie=4444bd43e8b98ca2 in_seq=0 out_seq=0 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/3412228474 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56059737c3d0 con 0x560597024d80 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff2e1ffb640 10 client.18921 handle_command_reply: tid=0 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 client.18921 shutdown 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 -- 172.21.15.115:0/3412228474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 msgr2=0x560597415c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597415c10 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40e2a50 tx=0x7ff2e4005130 comp rx=0 tx=0).stop 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 -- 172.21.15.115:0/3412228474 shutdown_connections 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560597024d80 0x560597425e10 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597413be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597415c10 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 --2- 172.21.15.115:0/3412228474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x5605974156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff2e1ffb640 0 client.18921 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 -- 172.21.15.115:0/3412228474 >> 172.21.15.115:0/3412228474 conn(0x56059705a300 msgr2=0x56059740b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 -- 172.21.15.115:0/3412228474 shutdown_connections 2024-05-09T05:55:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 1 -- 172.21.15.115:0/3412228474 wait complete. 2024-05-09T05:55:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.099+0000 7ff307e19740 20 client.18921 trim_cache size 0 max 16384 2024-05-09T05:55:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.114+0000 7ff307e19740 1 Processor -- start 2024-05-09T05:55:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.114+0000 7ff307e19740 1 -- start start 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.114+0000 7ff307e19740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56059703d2f0 0x560597414130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff307e19740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970454d0 0x560597405710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff307e19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605970661b0 0x560597405c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff307e19740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1b1270 con 0x56059703d2f0 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff307e19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1b2db0 con 0x5605970661b0 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff307e19740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1b1940 con 0x5605970454d0 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2fa59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56059703d2f0 0x560597414130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2fa59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56059703d2f0 0x560597414130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43472/0 (socket says 172.21.15.115:43472) 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2fa59c640 1 -- 172.21.15.115:0/2551592863 learned_addr learned my addr 172.21.15.115:0/2551592863 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/2551592863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970454d0 0x560597405710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2fa59c640 1 -- 172.21.15.115:0/2551592863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970454d0 msgr2=0x560597405710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/2551592863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970454d0 0x560597405710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2fa59c640 1 -- 172.21.15.115:0/2551592863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605970661b0 msgr2=0x560597405c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/2551592863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605970661b0 0x560597405c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2fa59c640 1 -- 172.21.15.115:0/2551592863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605971485f0 con 0x56059703d2f0 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/2551592863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56059703d2f0 0x560597414130 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00abe0 tx=0x7ff2ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbba72cb2cbc2ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2e3fff640 1 -- 172.21.15.115:0/2551592863 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec021020 con 0x56059703d2f0 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2e3fff640 1 -- 172.21.15.115:0/2551592863 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2ec01dab0 con 0x56059703d2f0 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.115+0000 7ff2e3fff640 1 -- 172.21.15.115:0/2551592863 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00fdf0 con 0x56059703d2f0 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 -- 172.21.15.115:0/2551592863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56059703d2f0 msgr2=0x560597414130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 --2- 172.21.15.115:0/2551592863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56059703d2f0 0x560597414130 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00abe0 tx=0x7ff2ec0046c0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 -- 172.21.15.115:0/2551592863 shutdown_connections 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 --2- 172.21.15.115:0/2551592863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605970661b0 0x560597405c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 --2- 172.21.15.115:0/2551592863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970454d0 0x560597405710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 --2- 172.21.15.115:0/2551592863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56059703d2f0 0x560597414130 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 -- 172.21.15.115:0/2551592863 >> 172.21.15.115:0/2551592863 conn(0x56059705a300 msgr2=0x560597409e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 -- 172.21.15.115:0/2551592863 shutdown_connections 2024-05-09T05:55:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 -- 172.21.15.115:0/2551592863 wait complete. 2024-05-09T05:55:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.116+0000 7ff307e19740 1 -- start start 2024-05-09T05:55:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff307e19740 1 Processor -- start 2024-05-09T05:55:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff307e19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x5605974294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff307e19740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597426860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff307e19740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597426e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff307e19740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1b26e0 con 0x5605970454d0 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff307e19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1b2010 con 0x56059703d2f0 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff307e19740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1af270 con 0x5605970661b0 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff307e19740 10 client.0 _ll_register_callbacks cb 0x7ff2f4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597426860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597426860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43488/0 (socket says 172.21.15.115:43488) 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/1769591700 learned_addr learned my addr 172.21.15.115:0/1769591700 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x5605974294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2fad9d640 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597426e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/1769591700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 msgr2=0x560597426e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597426e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/1769591700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 msgr2=0x5605974294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x5605974294f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.117+0000 7ff2f1d9b640 1 -- 172.21.15.115:0/1769591700 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560597332ed0 con 0x5605970454d0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff2f1d9b640 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597426860 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4000c00 tx=0x7ff2e40a95f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ba2abf49b27000 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/1769591700 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40f1050 con 0x5605970454d0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/1769591700 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7ff2e40a5880 con 0x5605970454d0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff307e19740 1 -- 172.21.15.115:0/1769591700 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560597376240 con 0x5605970454d0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/1769591700 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ff2e40a8f40 con 0x5605970454d0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff307e19740 10 client.18927 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff307e19740 1 -- 172.21.15.115:0/1769591700 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x56059748dd90 con 0x5605970454d0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/1769591700 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560597376240 con 0x5605970454d0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/1769591700 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7ff2e40e22b0 con 0x5605970454d0 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff307e19740 10 client.18927 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff307e19740 10 client.18927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:55:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff307e19740 20 client.18927 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff307e19740 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560596ffa1c0 0x5605974902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff307e19740 4 client.18927 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:55:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff307e19740 1 -- 172.21.15.115:0/1769591700 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605972f1350 con 0x560596ffa1c0 2024-05-09T05:55:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.118+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560596ffa1c0 0x5605974902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.119+0000 7ff2e1ffb640 10 client.18927 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.119+0000 7ff2fa59c640 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560596ffa1c0 0x5605974902c0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55367c4f83884b8 server_cookie=ddb3264dc2667072 in_seq=0 out_seq=0 2024-05-09T05:55:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff2e1ffb640 1 -- 172.21.15.115:0/1769591700 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5605972f1350 con 0x560596ffa1c0 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff2e1ffb640 10 client.18927 handle_command_reply: tid=0 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 client.18927 shutdown 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 -- 172.21.15.115:0/1769591700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 msgr2=0x560597426860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597426860 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4000c00 tx=0x7ff2e40a95f0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 -- 172.21.15.115:0/1769591700 shutdown_connections 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x560596ffa1c0 0x5605974902c0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605970661b0 0x560597426e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5605970454d0 0x560597426860 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 --2- 172.21.15.115:0/1769591700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56059703d2f0 0x5605974294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff2e1ffb640 0 client.18927 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 -- 172.21.15.115:0/1769591700 >> 172.21.15.115:0/1769591700 conn(0x56059705a300 msgr2=0x560597413120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 -- 172.21.15.115:0/1769591700 shutdown_connections 2024-05-09T05:55:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 1 -- 172.21.15.115:0/1769591700 wait complete. 2024-05-09T05:55:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.129+0000 7ff307e19740 20 client.18927 trim_cache size 0 max 16384 2024-05-09T05:55:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:55:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:55:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 -- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2e80b2950 msgr2=0x7ff2e80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2e80b2950 0x7ff2e80b4e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5605974067f0 tx=0x7ff2f4005c50 comp rx=0 tx=0).stop 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 -- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc1084d0 msgr2=0x7ff2fc19ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc1084d0 0x7ff2fc19ad20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400d950 tx=0x7ff2e400de10 comp rx=0 tx=0).stop 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 -- 172.21.15.115:0/2323890361 shutdown_connections 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff2e80b2950 0x7ff2e80b4e60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2fc108f70 0x7ff2fc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2fc1084d0 0x7ff2fc19ad20 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 --2- 172.21.15.115:0/2323890361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2fc102450 0x7ff2fc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 -- 172.21.15.115:0/2323890361 >> 172.21.15.115:0/2323890361 conn(0x7ff2fc0779e0 msgr2=0x7ff2fc0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 -- 172.21.15.115:0/2323890361 shutdown_connections 2024-05-09T05:55:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:38.130+0000 7ff300e89640 1 -- 172.21.15.115:0/2323890361 wait complete. 2024-05-09T05:55:39.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:39 smithi195 ceph-mon[33844]: pgmap v552: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 247 KiB/s wr, 70 op/s 2024-05-09T05:55:39.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:39 smithi115 ceph-mon[30349]: pgmap v552: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 247 KiB/s wr, 70 op/s 2024-05-09T05:55:39.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:39 smithi161 ceph-mon[33805]: pgmap v552: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 247 KiB/s wr, 70 op/s 2024-05-09T05:55:41.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:41 smithi195 ceph-mon[33844]: pgmap v553: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 233 KiB/s wr, 67 op/s 2024-05-09T05:55:41.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:41 smithi115 ceph-mon[30349]: pgmap v553: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 233 KiB/s wr, 67 op/s 2024-05-09T05:55:41.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:41 smithi161 ceph-mon[33805]: pgmap v553: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 233 KiB/s wr, 67 op/s 2024-05-09T05:55:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.829+0000 7f27c0e9a700 1 -- 172.21.15.115:0/1929270856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc10e900 msgr2=0x7f27bc110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.829+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/1929270856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc10e900 0x7f27bc110de0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f27b400b0a0 tx=0x7f27b400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.829+0000 7f27c0e9a700 1 -- 172.21.15.115:0/1929270856 shutdown_connections 2024-05-09T05:55:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.829+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/1929270856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27bc1054d0 0x7f27bc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.829+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/1929270856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc10e900 0x7f27bc110de0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.829+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/1929270856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27bc105e20 0x7f27bc10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.829+0000 7f27c0e9a700 1 -- 172.21.15.115:0/1929270856 >> 172.21.15.115:0/1929270856 conn(0x7f27bc0fef20 msgr2=0x7f27bc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.829+0000 7f27c0e9a700 1 -- 172.21.15.115:0/1929270856 shutdown_connections 2024-05-09T05:55:42.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.829+0000 7f27c0e9a700 1 -- 172.21.15.115:0/1929270856 wait complete. 2024-05-09T05:55:42.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.830+0000 7f27c0e9a700 1 Processor -- start 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.830+0000 7f27c0e9a700 1 -- start start 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27c0e9a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc1054d0 0x7f27bc1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27c0e9a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27bc105e20 0x7f27bc1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27c0e9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27bc10e900 0x7f27bc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27c0e9a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc114330 con 0x7f27bc1054d0 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27c0e9a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1141b0 con 0x7f27bc10e900 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27c0e9a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1144b0 con 0x7f27bc105e20 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27ba59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc1054d0 0x7f27bc1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27ba59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc1054d0 0x7f27bc1a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39896/0 (socket says 172.21.15.115:39896) 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27ba59c700 1 -- 172.21.15.115:0/2514473668 learned_addr learned my addr 172.21.15.115:0/2514473668 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27bad9d700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27bc10e900 0x7f27bc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27bc105e20 0x7f27bc1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27ba59c700 1 -- 172.21.15.115:0/2514473668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27bc105e20 msgr2=0x7f27bc1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27ba59c700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27bc105e20 0x7f27bc1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27ba59c700 1 -- 172.21.15.115:0/2514473668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27bc10e900 msgr2=0x7f27bc19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:42.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27ba59c700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27bc10e900 0x7f27bc19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27ba59c700 1 -- 172.21.15.115:0/2514473668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc19b1b0 con 0x7f27bc1054d0 2024-05-09T05:55:42.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27bc105e20 0x7f27bc1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:55:42.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.831+0000 7f27ba59c700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc1054d0 0x7f27bc1a0890 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c920 tx=0x7f27ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:42.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.832+0000 7f27b3fff700 1 -- 172.21.15.115:0/2514473668 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac007b00 con 0x7f27bc1054d0 2024-05-09T05:55:42.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.832+0000 7f27b3fff700 1 -- 172.21.15.115:0/2514473668 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27ac004cc0 con 0x7f27bc1054d0 2024-05-09T05:55:42.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.832+0000 7f27c0e9a700 1 -- 172.21.15.115:0/2514473668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc19b4a0 con 0x7f27bc1054d0 2024-05-09T05:55:42.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.832+0000 7f27b3fff700 1 -- 172.21.15.115:0/2514473668 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0054f0 con 0x7f27bc1054d0 2024-05-09T05:55:42.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.832+0000 7f27c0e9a700 1 -- 172.21.15.115:0/2514473668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1a7740 con 0x7f27bc1054d0 2024-05-09T05:55:42.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.835+0000 7f27b3fff700 1 -- 172.21.15.115:0/2514473668 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f27ac007ca0 con 0x7f27bc1054d0 2024-05-09T05:55:42.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.836+0000 7f27b3fff700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f27a80b2a70 0x7f27a80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.836+0000 7f27cf8b0b80 1 Processor -- start 2024-05-09T05:55:42.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.836+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f27a80b2a70 0x7f27a80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.836+0000 7f27cf8b0b80 1 -- start start 2024-05-09T05:55:42.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.836+0000 7f27b3fff700 1 -- 172.21.15.115:0/2514473668 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f27ac0e2580 con 0x7f27bc1054d0 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.836+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d26c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.836+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 0x563627d27190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.836+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d33d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563627c4a1a0 con 0x563627c24e00 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563627befaa0 con 0x563627d31900 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563627c46160 con 0x563627baa5a0 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27ba59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d26c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27ba59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d26c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39910/0 (socket says 172.21.15.115:39910) 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27ba59c700 1 -- 172.21.15.115:0/4006506 learned_addr learned my addr 172.21.15.115:0/4006506 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/4006506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 0x563627d27190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27ba59c700 1 -- 172.21.15.115:0/4006506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 msgr2=0x563627d27190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:42.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27ba59c700 1 --2- 172.21.15.115:0/4006506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 0x563627d27190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27ba59c700 1 -- 172.21.15.115:0/4006506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 msgr2=0x563627d33d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:42.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f27a80b2a70 0x7f27a80b4f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f27bc102450 tx=0x7f27a4009550 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:42.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27ba59c700 1 --2- 172.21.15.115:0/4006506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d33d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27ba59c700 1 -- 172.21.15.115:0/4006506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563627be0600 con 0x563627c24e00 2024-05-09T05:55:42.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27ba59c700 1 --2- 172.21.15.115:0/4006506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d26c50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0a9460 tx=0x7f27ac0a9060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83422c89cded8e6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:42.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.837+0000 7f27b0d99700 1 -- 172.21.15.115:0/4006506 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac01d040 con 0x563627c24e00 2024-05-09T05:55:42.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.838+0000 7f27b0d99700 1 -- 172.21.15.115:0/4006506 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27ac0ad880 con 0x563627c24e00 2024-05-09T05:55:42.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.838+0000 7f27b0d99700 1 -- 172.21.15.115:0/4006506 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0a9990 con 0x563627c24e00 2024-05-09T05:55:42.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/4006506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 msgr2=0x563627d26c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:42.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/4006506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d26c50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0a9460 tx=0x7f27ac0a9060 comp rx=0 tx=0).stop 2024-05-09T05:55:42.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/4006506 shutdown_connections 2024-05-09T05:55:42.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/4006506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d33d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/4006506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d26c50 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/4006506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 0x563627d27190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/4006506 >> 172.21.15.115:0/4006506 conn(0x563627bae070 msgr2=0x563627d22b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/4006506 shutdown_connections 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/4006506 wait complete. 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 -- start start 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.839+0000 7f27cf8b0b80 1 Processor -- start 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d2e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 0x563627d2ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d2f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x563627d38730 con 0x563627baa5a0 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563627be4c20 con 0x563627d31900 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563627d388b0 con 0x563627c24e00 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27cf8b0b80 10 client.0 _ll_register_callbacks cb 0x563627abb7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27ba59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d2e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27ba59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d2e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39922/0 (socket says 172.21.15.115:39922) 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27ba59c700 1 -- 172.21.15.115:0/3749067701 learned_addr learned my addr 172.21.15.115:0/3749067701 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 0x563627d2ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27bad9d700 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d2f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27ba59c700 1 -- 172.21.15.115:0/3749067701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 msgr2=0x563627d2ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27ba59c700 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 0x563627d2ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27ba59c700 1 -- 172.21.15.115:0/3749067701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 msgr2=0x563627d2f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27ba59c700 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d2f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27ba59c700 1 -- 172.21.15.115:0/3749067701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563627ac5590 con 0x563627baa5a0 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27ba59c700 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d2e9c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0df9f0 tx=0x7f27ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0757d71012ace0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27a1ffb700 1 -- 172.21.15.115:0/3749067701 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0c8060 con 0x563627baa5a0 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27a1ffb700 1 -- 172.21.15.115:0/3749067701 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f27ac0b20b0 con 0x563627baa5a0 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.840+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/3749067701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27ac0ad880 con 0x563627baa5a0 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27a1ffb700 1 -- 172.21.15.115:0/3749067701 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f27ac088050 con 0x563627baa5a0 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27cf8b0b80 10 client.18939 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:42.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/3749067701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x563627d43450 con 0x563627baa5a0 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27a1ffb700 1 -- 172.21.15.115:0/3749067701 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27ac0ad880 con 0x563627baa5a0 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27a1ffb700 1 -- 172.21.15.115:0/3749067701 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7f27ac0a4e40 con 0x563627baa5a0 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27cf8b0b80 10 client.18939 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27cf8b0b80 10 client.18939 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27cf8b0b80 20 client.18939 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563627d45920 0x563627d47d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27cf8b0b80 4 client.18939 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/3749067701 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563627c57b80 con 0x563627d45920 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.841+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563627d45920 0x563627d47d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.842+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563627d45920 0x563627d47d40 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=636cd529ee8b88b9 server_cookie=b774be964548e460 in_seq=0 out_seq=0 2024-05-09T05:55:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.842+0000 7f27a1ffb700 10 client.18939 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:42.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27a1ffb700 1 -- 172.21.15.115:0/3749067701 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563627c57b80 con 0x563627d45920 2024-05-09T05:55:42.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27a1ffb700 10 client.18939 handle_command_reply: tid=0 2024-05-09T05:55:42.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 client.18939 shutdown 2024-05-09T05:55:42.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/3749067701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 msgr2=0x563627d2e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d2e9c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0df9f0 tx=0x7f27ac000bc0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/3749067701 shutdown_connections 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d2f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563627d45920 0x563627d47d40 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d2e9c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/3749067701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 0x563627d2ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/3749067701 >> 172.21.15.115:0/3749067701 conn(0x563627bae070 msgr2=0x563627d21de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27a1ffb700 0 client.18939 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/3749067701 shutdown_connections 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/3749067701 wait complete. 2024-05-09T05:55:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.843+0000 7f27cf8b0b80 20 client.18939 trim_cache size 0 max 16384 2024-05-09T05:55:42.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27cf8b0b80 1 Processor -- start 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27cf8b0b80 1 -- start start 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d27320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 0x563627d29e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d41570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1ae240 con 0x563627baa5a0 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1afd80 con 0x563627d31900 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1ae910 con 0x563627c24e00 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27ba59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d27320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27ba59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d27320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39924/0 (socket says 172.21.15.115:39924) 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27ba59c700 1 -- 172.21.15.115:0/2584693689 learned_addr learned my addr 172.21.15.115:0/2584693689 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/2584693689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 0x563627d29e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27ba59c700 1 -- 172.21.15.115:0/2584693689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 msgr2=0x563627d29e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:42.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27ba59c700 1 --2- 172.21.15.115:0/2584693689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 0x563627d29e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27ba59c700 1 -- 172.21.15.115:0/2584693689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 msgr2=0x563627d41570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:42.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27ba59c700 1 --2- 172.21.15.115:0/2584693689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d41570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27ba59c700 1 -- 172.21.15.115:0/2584693689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563627d27f90 con 0x563627baa5a0 2024-05-09T05:55:42.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/2584693689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 0x563627d29e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:55:42.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.863+0000 7f27ba59c700 1 --2- 172.21.15.115:0/2584693689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d27320 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0a91d0 tx=0x7f27ac0a9060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08da628a4fc1735 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:42.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27a3fff700 1 -- 172.21.15.115:0/2584693689 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0c8030 con 0x563627baa5a0 2024-05-09T05:55:42.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27a3fff700 1 -- 172.21.15.115:0/2584693689 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27ac0c2030 con 0x563627baa5a0 2024-05-09T05:55:42.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27a3fff700 1 -- 172.21.15.115:0/2584693689 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac01d040 con 0x563627baa5a0 2024-05-09T05:55:42.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/2584693689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 msgr2=0x563627d27320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:42.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/2584693689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d27320 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0a91d0 tx=0x7f27ac0a9060 comp rx=0 tx=0).stop 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/2584693689 shutdown_connections 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/2584693689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d41570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/2584693689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627baa5a0 0x563627d27320 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/2584693689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627c24e00 0x563627d29e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/2584693689 >> 172.21.15.115:0/2584693689 conn(0x563627bae070 msgr2=0x563627d284d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.864+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/2584693689 shutdown_connections 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/2584693689 wait complete. 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 1 -- start start 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 1 Processor -- start 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 0x563627d46730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d46c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1af6b0 con 0x563627c24e00 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1aefe0 con 0x563627d31900 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1ac220 con 0x563627baa5a0 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27cf8b0b80 10 client.0 _ll_register_callbacks cb 0x563627abb7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27b1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d46c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27b1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d46c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39930/0 (socket says 172.21.15.115:39930) 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.865+0000 7f27b1d9b700 1 -- 172.21.15.115:0/1304534967 learned_addr learned my addr 172.21.15.115:0/1304534967 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27ba59c700 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 0x563627d46730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27bad9d700 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27b1d9b700 1 -- 172.21.15.115:0/1304534967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 msgr2=0x563627d46730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 0x563627d46730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27b1d9b700 1 -- 172.21.15.115:0/1304534967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 msgr2=0x563627d471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27b1d9b700 1 -- 172.21.15.115:0/1304534967 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563627daf020 con 0x563627c24e00 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27ba59c700 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 0x563627d46730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27b1d9b700 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d46c70 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f27a4000c00 tx=0x7f27a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2f632e2576452a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27a1ffb700 1 -- 172.21.15.115:0/1304534967 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4029500 con 0x563627c24e00 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27a1ffb700 1 -- 172.21.15.115:0/1304534967 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f27a40296a0 con 0x563627c24e00 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/1304534967 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27ac0c2030 con 0x563627c24e00 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27a1ffb700 1 -- 172.21.15.115:0/1304534967 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f27a40298a0 con 0x563627c24e00 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27cf8b0b80 10 client.18945 fetch_fsmap learned FSMap version 66 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.866+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/1304534967 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x563627db0300 con 0x563627c24e00 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.867+0000 7f27a1ffb700 1 -- 172.21.15.115:0/1304534967 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27ac0c2030 con 0x563627c24e00 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.867+0000 7f27a1ffb700 1 -- 172.21.15.115:0/1304534967 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 66) ==== 8362+0+0 (secure 0 0 0) 0x7f27a400f040 con 0x563627c24e00 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.867+0000 7f27cf8b0b80 10 client.18945 fetch_fsmap finished waiting for FSMap version 66 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.867+0000 7f27cf8b0b80 10 client.18945 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:55:42.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.867+0000 7f27cf8b0b80 20 client.18945 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:42.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.867+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563627db09f0 0x563627db2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:42.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.867+0000 7f27cf8b0b80 4 client.18945 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "82569837", "release": true, "await": true}] 2024-05-09T05:55:42.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.867+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/1304534967 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "82569837", "release": true, "await": true}) -- 0x563627c51c20 con 0x563627db09f0 2024-05-09T05:55:42.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.867+0000 7f27ba59c700 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563627db09f0 0x563627db2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:42.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.868+0000 7f27ba59c700 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563627db09f0 0x563627db2e10 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95047426dabbecac server_cookie=c4d902e57339a13f in_seq=0 out_seq=0 2024-05-09T05:55:42.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.868+0000 7f27a1ffb700 10 client.18945 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:42.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27a1ffb700 1 -- 172.21.15.115:0/1304534967 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563627c51c20 con 0x563627db09f0 2024-05-09T05:55:42.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27a1ffb700 10 client.18945 handle_command_reply: tid=0 2024-05-09T05:55:42.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 client.18945 shutdown 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/1304534967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 msgr2=0x563627d46c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d46c70 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f27a4000c00 tx=0x7f27a40047c0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/1304534967 shutdown_connections 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563627d31900 0x563627d471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x563627db09f0 0x563627db2e10 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563627baa5a0 0x563627d46730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 --2- 172.21.15.115:0/1304534967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x563627c24e00 0x563627d46c70 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27a1ffb700 0 client.18945 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/1304534967 >> 172.21.15.115:0/1304534967 conn(0x563627bae070 msgr2=0x563627d412d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/1304534967 shutdown_connections 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 1 -- 172.21.15.115:0/1304534967 wait complete. 2024-05-09T05:55:42.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.884+0000 7f27cf8b0b80 20 client.18945 trim_cache size 0 max 16384 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 66, 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 48, 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "82569837": { 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 48, 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:55:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 147.1, 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:55:42.875 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:55:42.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 -- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f27a80b2a70 msgr2=0x7f27a80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:42.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f27a80b2a70 0x7f27a80b4f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f27bc102450 tx=0x7f27a4009550 comp rx=0 tx=0).stop 2024-05-09T05:55:42.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 -- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc1054d0 msgr2=0x7f27bc1a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:42.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc1054d0 0x7f27bc1a0890 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c920 tx=0x7f27ac00cce0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 -- 172.21.15.115:0/2514473668 shutdown_connections 2024-05-09T05:55:42.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f27a80b2a70 0x7f27a80b4f80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27bc10e900 0x7f27bc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27bc1054d0 0x7f27bc1a0890 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 --2- 172.21.15.115:0/2514473668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27bc105e20 0x7f27bc1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:42.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 -- 172.21.15.115:0/2514473668 >> 172.21.15.115:0/2514473668 conn(0x7f27bc0fef20 msgr2=0x7f27bc100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:42.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 -- 172.21.15.115:0/2514473668 shutdown_connections 2024-05-09T05:55:42.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:55:42.885+0000 7f27c0e9a700 1 -- 172.21.15.115:0/2514473668 wait complete. 2024-05-09T05:55:43.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:43 smithi195 ceph-mon[33844]: pgmap v554: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-05-09T05:55:43.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:43 smithi115 ceph-mon[30349]: pgmap v554: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-05-09T05:55:43.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:43 smithi161 ceph-mon[33805]: pgmap v554: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-05-09T05:55:44.035 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '82569837', total seconds elapsed: 94.1 2024-05-09T05:55:45.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:45 smithi195 ceph-mon[33844]: pgmap v555: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 161 KiB/s wr, 51 op/s 2024-05-09T05:55:45.645 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:45 smithi115 ceph-mon[30349]: pgmap v555: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 161 KiB/s wr, 51 op/s 2024-05-09T05:55:45.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:45 smithi161 ceph-mon[33805]: pgmap v555: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 161 KiB/s wr, 51 op/s 2024-05-09T05:55:47.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:47 smithi115 ceph-mon[30349]: pgmap v556: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 426 KiB/s wr, 115 op/s 2024-05-09T05:55:47.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:47 smithi161 ceph-mon[33805]: pgmap v556: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 426 KiB/s wr, 115 op/s 2024-05-09T05:55:47.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:47 smithi195 ceph-mon[33844]: pgmap v556: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 426 KiB/s wr, 115 op/s 2024-05-09T05:55:49.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:49 smithi115 ceph-mon[30349]: pgmap v557: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.6 MiB/s wr, 674 op/s 2024-05-09T05:55:49.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:49 smithi161 ceph-mon[33805]: pgmap v557: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.6 MiB/s wr, 674 op/s 2024-05-09T05:55:49.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:49 smithi195 ceph-mon[33844]: pgmap v557: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.6 MiB/s wr, 674 op/s 2024-05-09T05:55:50.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:50 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:55:50.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:50 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:55:50.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:50 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:55:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.483+0000 7f4f84df1640 1 -- 172.21.15.115:0/1417151421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f8010a210 msgr2=0x7f4f8010a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.483+0000 7f4f84df1640 1 --2- 172.21.15.115:0/1417151421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f8010a210 0x7f4f8010a6a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009ca0 tx=0x7f4f7802a8f0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.489+0000 7f4f84df1640 1 -- 172.21.15.115:0/1417151421 shutdown_connections 2024-05-09T05:55:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.489+0000 7f4f84df1640 1 --2- 172.21.15.115:0/1417151421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f8010a210 0x7f4f8010a6a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.489+0000 7f4f84df1640 1 --2- 172.21.15.115:0/1417151421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f801103f0 0x7f4f80109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.489+0000 7f4f84df1640 1 --2- 172.21.15.115:0/1417151421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f8010fa10 0x7f4f8010fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.489+0000 7f4f84df1640 1 -- 172.21.15.115:0/1417151421 >> 172.21.15.115:0/1417151421 conn(0x7f4f8006f810 msgr2=0x7f4f8006fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.495+0000 7f4f84df1640 1 -- 172.21.15.115:0/1417151421 shutdown_connections 2024-05-09T05:55:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.495+0000 7f4f84df1640 1 -- 172.21.15.115:0/1417151421 wait complete. 2024-05-09T05:55:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.496+0000 7f4f84df1640 1 Processor -- start 2024-05-09T05:55:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.496+0000 7f4f84df1640 1 -- start start 2024-05-09T05:55:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.497+0000 7f4f84df1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f8010a210 0x7f4f80120390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.497+0000 7f4f84df1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f8010fa10 0x7f4f80119550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.497+0000 7f4f84df1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f801103f0 0x7f4f80119c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.497+0000 7f4f84df1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80122960 con 0x7f4f8010fa10 2024-05-09T05:55:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.497+0000 7f4f84df1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801227e0 con 0x7f4f801103f0 2024-05-09T05:55:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.497+0000 7f4f84df1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80122ae0 con 0x7f4f8010a210 2024-05-09T05:55:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.500+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f801103f0 0x7f4f80119c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.500+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f801103f0 0x7f4f80119c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49028/0 (socket says 172.21.15.115:49028) 2024-05-09T05:55:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.500+0000 7f4f7ffff640 1 -- 172.21.15.115:0/3189828551 learned_addr learned my addr 172.21.15.115:0/3189828551 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.500+0000 7f4f7effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f8010fa10 0x7f4f80119550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.500+0000 7f4f7f7fe640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f8010a210 0x7f4f80120390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.506+0000 7f4f7effd640 1 -- 172.21.15.115:0/3189828551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f8010a210 msgr2=0x7f4f80120390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.506+0000 7f4f7effd640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f8010a210 0x7f4f80120390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.506+0000 7f4f7effd640 1 -- 172.21.15.115:0/3189828551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f801103f0 msgr2=0x7f4f80119c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.506+0000 7f4f7effd640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f801103f0 0x7f4f80119c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.506+0000 7f4f7effd640 1 -- 172.21.15.115:0/3189828551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8011deb0 con 0x7f4f8010fa10 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.506+0000 7f4f7effd640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f8010fa10 0x7f4f80119550 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000b3c0 tx=0x7f4f7000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.506+0000 7f4f7cff9640 1 -- 172.21.15.115:0/3189828551 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70011830 con 0x7f4f8010fa10 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.507+0000 7f4f7cff9640 1 -- 172.21.15.115:0/3189828551 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f70016070 con 0x7f4f8010fa10 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.507+0000 7f4f7cff9640 1 -- 172.21.15.115:0/3189828551 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70010d40 con 0x7f4f8010fa10 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.507+0000 7f4f84df1640 1 -- 172.21.15.115:0/3189828551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8011e1a0 con 0x7f4f8010fa10 2024-05-09T05:55:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.507+0000 7f4f84df1640 1 -- 172.21.15.115:0/3189828551 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8011e6a0 con 0x7f4f8010fa10 2024-05-09T05:55:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.513+0000 7f4f7cff9640 1 -- 172.21.15.115:0/3189828551 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4f7001b460 con 0x7f4f8010fa10 2024-05-09T05:55:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.515+0000 7f4f8cfe7740 1 Processor -- start 2024-05-09T05:55:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.516+0000 7f4f7cff9640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4f680b2aa0 0x7f4f680b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.516+0000 7f4f7cff9640 1 -- 172.21.15.115:0/3189828551 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f4f700e2e10 con 0x7f4f8010fa10 2024-05-09T05:55:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.520+0000 7f4f8cfe7740 1 -- start start 2024-05-09T05:55:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.521+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae6d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.521+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa12f0 0x5574eae6dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.521+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa94d0 0x5574eae6e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.521+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574eadd27c0 con 0x5574eaaa94d0 2024-05-09T05:55:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.521+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5574ead6e0e0 con 0x5574eaaca1b0 2024-05-09T05:55:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.521+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574eadce780 con 0x5574eaaa12f0 2024-05-09T05:55:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.522+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4f680b2aa0 0x7f4f680b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.528+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa94d0 0x5574eae6e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.528+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa94d0 0x5574eae6e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39946/0 (socket says 172.21.15.115:39946) 2024-05-09T05:55:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.528+0000 7f4f7ffff640 1 -- 172.21.15.115:0/1059883131 learned_addr learned my addr 172.21.15.115:0/1059883131 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.528+0000 7f4f7f7fe640 1 --2- 172.21.15.115:0/1059883131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae6d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.531+0000 7f4f7effd640 1 --2- 172.21.15.115:0/1059883131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa12f0 0x5574eae6dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.531+0000 7f4f7ffff640 1 -- 172.21.15.115:0/1059883131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa12f0 msgr2=0x5574eae6dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.531+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/1059883131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa12f0 0x5574eae6dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.531+0000 7f4f7ffff640 1 -- 172.21.15.115:0/1059883131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 msgr2=0x5574eae6d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.531+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/1059883131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae6d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.531+0000 7f4f7ffff640 1 -- 172.21.15.115:0/1059883131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ead977c0 con 0x5574eaaa94d0 2024-05-09T05:55:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.534+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4f680b2aa0 0x7f4f680b4fb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4f7802aee0 tx=0x7f4f78003ca0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.537+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/1059883131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa94d0 0x5574eae6e2c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4f780044b0 tx=0x7f4f78047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd61ac36f95f6078 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.538+0000 7f4f65ffb640 1 -- 172.21.15.115:0/1059883131 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78004b40 con 0x5574eaaa94d0 2024-05-09T05:55:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.538+0000 7f4f65ffb640 1 -- 172.21.15.115:0/1059883131 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f78036040 con 0x5574eaaa94d0 2024-05-09T05:55:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.540+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/1059883131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa94d0 msgr2=0x5574eae6e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.540+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/1059883131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa94d0 0x5574eae6e2c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4f780044b0 tx=0x7f4f78047000 comp rx=0 tx=0).stop 2024-05-09T05:55:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.556+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/1059883131 shutdown_connections 2024-05-09T05:55:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.556+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/1059883131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa94d0 0x5574eae6e2c0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.556+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/1059883131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa12f0 0x5574eae6dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.556+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/1059883131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae6d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.556+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/1059883131 >> 172.21.15.115:0/1059883131 conn(0x5574eaabe300 msgr2=0x5574eae69ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.563+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/1059883131 shutdown_connections 2024-05-09T05:55:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.568+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/1059883131 wait complete. 2024-05-09T05:55:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.568+0000 7f4f8cfe7740 1 -- start start 2024-05-09T05:55:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.568+0000 7f4f8cfe7740 1 Processor -- start 2024-05-09T05:55:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa12f0 0x5574eae77e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae78390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae83eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5574eae7a9b0 con 0x5574eaaa12f0 2024-05-09T05:55:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5574ead599c0 con 0x5574eaaca1b0 2024-05-09T05:55:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574eae7ab30 con 0x5574eaaa94d0 2024-05-09T05:55:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f8cfe7740 10 client.0 _ll_register_callbacks cb 0x5574ead86020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae83eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae83eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49036/0 (socket says 172.21.15.115:49036) 2024-05-09T05:55:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.569+0000 7f4f7ffff640 1 -- 172.21.15.115:0/2527856619 learned_addr learned my addr 172.21.15.115:0/2527856619 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.570+0000 7f4f7ffff640 1 -- 172.21.15.115:0/2527856619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 msgr2=0x5574eae78390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:55:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.570+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae78390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.570+0000 7f4f7ffff640 1 -- 172.21.15.115:0/2527856619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa12f0 msgr2=0x5574eae77e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.570+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa12f0 0x5574eae77e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.570+0000 7f4f7ffff640 1 -- 172.21.15.115:0/2527856619 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ead801f0 con 0x5574eaaca1b0 2024-05-09T05:55:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.570+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae83eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f78031040 tx=0x7f4f780095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aadadf8063a61917 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.575+0000 7f4f5effd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:55:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.576+0000 7f4f5effd640 1 -- 172.21.15.115:0/2527856619 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78030bd0 con 0x5574eaaca1b0 2024-05-09T05:55:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.576+0000 7f4f5effd640 1 -- 172.21.15.115:0/2527856619 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f4f78030d70 con 0x5574eaaca1b0 2024-05-09T05:55:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.576+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/2527856619 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f78036040 con 0x5574eaaca1b0 2024-05-09T05:55:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.583+0000 7f4f5effd640 1 -- 172.21.15.115:0/2527856619 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f4f7804c450 con 0x5574eaaca1b0 2024-05-09T05:55:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.586+0000 7f4f8cfe7740 10 client.28136 fetch_fsmap learned FSMap version 67 2024-05-09T05:55:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.586+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/2527856619 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5574eab35790 con 0x5574eaaca1b0 2024-05-09T05:55:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.590+0000 7f4f5effd640 1 -- 172.21.15.115:0/2527856619 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f78036040 con 0x5574eaaca1b0 2024-05-09T05:55:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.590+0000 7f4f5effd640 1 -- 172.21.15.115:0/2527856619 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 67) ==== 8366+0+0 (secure 0 0 0) 0x7f4f7803e430 con 0x5574eaaca1b0 2024-05-09T05:55:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.593+0000 7f4f8cfe7740 10 client.28136 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T05:55:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.593+0000 7f4f8cfe7740 10 client.28136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:55:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.593+0000 7f4f8cfe7740 20 client.28136 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.593+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574eaa88d80 0x5574eae89f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.593+0000 7f4f8cfe7740 4 client.28136 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:55:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.593+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/2527856619 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574eade03d0 con 0x5574eaa88d80 2024-05-09T05:55:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.604+0000 7f4f7f7fe640 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574eaa88d80 0x5574eae89f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.625+0000 7f4f7f7fe640 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574eaa88d80 0x5574eae89f00 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5760ae7f3b3f0f95 server_cookie=c447c104748672a0 in_seq=0 out_seq=0 2024-05-09T05:55:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.627+0000 7f4f5effd640 10 client.28136 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.629+0000 7f4f5effd640 1 -- 172.21.15.115:0/2527856619 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574eade03d0 con 0x5574eaa88d80 2024-05-09T05:55:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.629+0000 7f4f5effd640 10 client.28136 handle_command_reply: tid=0 2024-05-09T05:55:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.630+0000 7f4f8cfe7740 1 client.28136 shutdown 2024-05-09T05:55:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.633+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/2527856619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 msgr2=0x5574eae83eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.633+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae83eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f78031040 tx=0x7f4f780095c0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.633+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/2527856619 shutdown_connections 2024-05-09T05:55:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.633+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae83eb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.633+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574eaa88d80 0x5574eae89f00 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.633+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae78390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.633+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/2527856619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa12f0 0x5574eae77e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.633+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/2527856619 >> 172.21.15.115:0/2527856619 conn(0x5574eaabe300 msgr2=0x5574eae6f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.638+0000 7f4f5effd640 0 client.28136 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.638+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/2527856619 shutdown_connections 2024-05-09T05:55:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.641+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/2527856619 wait complete. 2024-05-09T05:55:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.642+0000 7f4f8cfe7740 20 client.28136 trim_cache size 0 max 16384 2024-05-09T05:55:51.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:51 smithi161 ceph-mon[33805]: pgmap v558: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 1.04k op/s 2024-05-09T05:55:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.765+0000 7f4f8cfe7740 1 Processor -- start 2024-05-09T05:55:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.766+0000 7f4f8cfe7740 1 -- start start 2024-05-09T05:55:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.767+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaa12f0 0x5574eae76c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.767+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae83e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.767+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaca1b0 0x5574eae84360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.767+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b77b0 con 0x5574eaaca1b0 2024-05-09T05:55:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.767+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801db290 con 0x5574eaaa12f0 2024-05-09T05:55:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.767+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b7e80 con 0x5574eaaa94d0 2024-05-09T05:55:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.767+0000 7f4f7effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae83e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.767+0000 7f4f7effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae83e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:60040/0 (socket says 172.21.15.115:60040) 2024-05-09T05:55:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.767+0000 7f4f7effd640 1 -- 172.21.15.115:0/3282880637 learned_addr learned my addr 172.21.15.115:0/3282880637 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.768+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/3282880637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaca1b0 0x5574eae84360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.769+0000 7f4f7ffff640 1 -- 172.21.15.115:0/3282880637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 msgr2=0x5574eae83e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.769+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/3282880637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae83e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.769+0000 7f4f7ffff640 1 -- 172.21.15.115:0/3282880637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaa12f0 msgr2=0x5574eae76c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:55:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.769+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/3282880637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaa12f0 0x5574eae76c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.769+0000 7f4f7ffff640 1 -- 172.21.15.115:0/3282880637 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574eabac5f0 con 0x5574eaaca1b0 2024-05-09T05:55:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.769+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/3282880637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaca1b0 0x5574eae84360 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009a60 tx=0x7f4f780039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd97bafab2a9a36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.776+0000 7f4f657fa640 1 -- 172.21.15.115:0/3282880637 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f780314f0 con 0x5574eaaca1b0 2024-05-09T05:55:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.776+0000 7f4f657fa640 1 -- 172.21.15.115:0/3282880637 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f7802d070 con 0x5574eaaca1b0 2024-05-09T05:55:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.785+0000 7f4f657fa640 1 -- 172.21.15.115:0/3282880637 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78030bd0 con 0x5574eaaca1b0 2024-05-09T05:55:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.786+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/3282880637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaca1b0 msgr2=0x5574eae84360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.786+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/3282880637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaca1b0 0x5574eae84360 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009a60 tx=0x7f4f780039f0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.795+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/3282880637 shutdown_connections 2024-05-09T05:55:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.795+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/3282880637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaca1b0 0x5574eae84360 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.795+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/3282880637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae83e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.795+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/3282880637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaa12f0 0x5574eae76c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.795+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/3282880637 >> 172.21.15.115:0/3282880637 conn(0x5574eaabe300 msgr2=0x5574eae76510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.804+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/3282880637 shutdown_connections 2024-05-09T05:55:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.805+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/3282880637 wait complete. 2024-05-09T05:55:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.805+0000 7f4f8cfe7740 1 -- start start 2024-05-09T05:55:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.805+0000 7f4f8cfe7740 1 Processor -- start 2024-05-09T05:55:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa12f0 0x5574eae8d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae8a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f8cfe7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b8c20 con 0x5574eaaa12f0 2024-05-09T05:55:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b8550 con 0x5574eaaca1b0 2024-05-09T05:55:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f8cfe7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8010b360 con 0x5574eaaa94d0 2024-05-09T05:55:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f8cfe7740 10 client.0 _ll_register_callbacks cb 0x7f4f78004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:55:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae8aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49042/0 (socket says 172.21.15.115:49042) 2024-05-09T05:55:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.806+0000 7f4f7ffff640 1 -- 172.21.15.115:0/293265472 learned_addr learned my addr 172.21.15.115:0/293265472 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:55:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.808+0000 7f4f7f7fe640 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa12f0 0x5574eae8d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.810+0000 7f4f7ffff640 1 -- 172.21.15.115:0/293265472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 msgr2=0x5574eae8a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:55:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.810+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae8a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.810+0000 7f4f7ffff640 1 -- 172.21.15.115:0/293265472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa12f0 msgr2=0x5574eae8d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:55:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.810+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa12f0 0x5574eae8d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.810+0000 7f4f7ffff640 1 -- 172.21.15.115:0/293265472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ead96ed0 con 0x5574eaaca1b0 2024-05-09T05:55:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.811+0000 7f4f7ffff640 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae8aa20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4f78000f80 tx=0x7f4f7803e6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f63dd19a502474e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:55:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.811+0000 7f4f5effd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:55:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.818+0000 7f4f5effd640 1 -- 172.21.15.115:0/293265472 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f780314f0 con 0x5574eaaca1b0 2024-05-09T05:55:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.818+0000 7f4f5effd640 1 -- 172.21.15.115:0/293265472 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f4f7803e430 con 0x5574eaaca1b0 2024-05-09T05:55:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.821+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/293265472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f7802d070 con 0x5574eaaca1b0 2024-05-09T05:55:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.822+0000 7f4f5effd640 1 -- 172.21.15.115:0/293265472 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f4f78030bd0 con 0x5574eaaca1b0 2024-05-09T05:55:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.823+0000 7f4f8cfe7740 10 client.28139 fetch_fsmap learned FSMap version 67 2024-05-09T05:55:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.823+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/293265472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5574eaef1a20 con 0x5574eaaca1b0 2024-05-09T05:55:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.828+0000 7f4f5effd640 1 -- 172.21.15.115:0/293265472 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f7802d070 con 0x5574eaaca1b0 2024-05-09T05:55:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.828+0000 7f4f5effd640 1 -- 172.21.15.115:0/293265472 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 67) ==== 8366+0+0 (secure 0 0 0) 0x7f4f78056420 con 0x5574eaaca1b0 2024-05-09T05:55:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.828+0000 7f4f8cfe7740 10 client.28139 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T05:55:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.828+0000 7f4f8cfe7740 10 client.28139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:55:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.828+0000 7f4f8cfe7740 20 client.28139 populate_metadata read hostname 'smithi115' 2024-05-09T05:55:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.828+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574eaa5e1c0 0x5574eaef3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:55:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.828+0000 7f4f8cfe7740 4 client.28139 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:55:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.828+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/293265472 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574eadda240 con 0x5574eaa5e1c0 2024-05-09T05:55:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.829+0000 7f4f7f7fe640 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574eaa5e1c0 0x5574eaef3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:55:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.833+0000 7f4f7f7fe640 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574eaa5e1c0 0x5574eaef3dc0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e388103c65bced0 server_cookie=789efb7538a39470 in_seq=0 out_seq=0 2024-05-09T05:55:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:51.833+0000 7f4f5effd640 10 client.28139 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:51.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:51 smithi195 ceph-mon[33844]: pgmap v558: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 1.04k op/s 2024-05-09T05:55:51.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:51 smithi115 ceph-mon[30349]: pgmap v558: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 1.04k op/s 2024-05-09T05:55:52.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:52 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:52.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:52 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:52.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:52 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:55:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f5effd640 1 -- 172.21.15.115:0/293265472 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5574eadda240 con 0x5574eaa5e1c0 2024-05-09T05:55:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f5effd640 10 client.28139 handle_command_reply: tid=0 2024-05-09T05:55:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f8cfe7740 1 client.28139 shutdown 2024-05-09T05:55:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/293265472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 msgr2=0x5574eae8aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae8aa20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4f78000f80 tx=0x7f4f7803e6a0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/293265472 shutdown_connections 2024-05-09T05:55:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574eaaca1b0 0x5574eae8aa20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5574eaa5e1c0 0x5574eaef3dc0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574eaaa94d0 0x5574eae8a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f8cfe7740 1 --2- 172.21.15.115:0/293265472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa12f0 0x5574eae8d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.387+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/293265472 >> 172.21.15.115:0/293265472 conn(0x5574eaabe300 msgr2=0x5574eae6d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.388+0000 7f4f5effd640 0 client.28139 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:55:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.389+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/293265472 shutdown_connections 2024-05-09T05:55:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.389+0000 7f4f8cfe7740 1 -- 172.21.15.115:0/293265472 wait complete. 2024-05-09T05:55:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.389+0000 7f4f8cfe7740 20 client.28139 trim_cache size 0 max 16384 2024-05-09T05:55:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:55:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:55:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:55:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.397+0000 7f4f84df1640 1 -- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4f680b2aa0 msgr2=0x7f4f680b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.397+0000 7f4f84df1640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4f680b2aa0 0x7f4f680b4fb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4f7802aee0 tx=0x7f4f78003ca0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.397+0000 7f4f84df1640 1 -- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f8010fa10 msgr2=0x7f4f80119550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:55:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.397+0000 7f4f84df1640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f8010fa10 0x7f4f80119550 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000b3c0 tx=0x7f4f7000b880 comp rx=0 tx=0).stop 2024-05-09T05:55:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.401+0000 7f4f84df1640 1 -- 172.21.15.115:0/3189828551 shutdown_connections 2024-05-09T05:55:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.401+0000 7f4f84df1640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4f680b2aa0 0x7f4f680b4fb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.401+0000 7f4f84df1640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f801103f0 0x7f4f80119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.401+0000 7f4f84df1640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f8010fa10 0x7f4f80119550 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.401+0000 7f4f84df1640 1 --2- 172.21.15.115:0/3189828551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f8010a210 0x7f4f80120390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:55:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.401+0000 7f4f84df1640 1 -- 172.21.15.115:0/3189828551 >> 172.21.15.115:0/3189828551 conn(0x7f4f8006f810 msgr2=0x7f4f8010d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:55:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.406+0000 7f4f84df1640 1 -- 172.21.15.115:0/3189828551 shutdown_connections 2024-05-09T05:55:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:55:53.406+0000 7f4f84df1640 1 -- 172.21.15.115:0/3189828551 wait complete. 2024-05-09T05:55:53.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:53 smithi195 ceph-mon[33844]: pgmap v559: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 1.25k op/s 2024-05-09T05:55:53.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:53 smithi115 ceph-mon[30349]: pgmap v559: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 1.25k op/s 2024-05-09T05:55:53.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:53 smithi161 ceph-mon[33805]: pgmap v559: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 1.25k op/s 2024-05-09T05:55:55.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:55 smithi195 ceph-mon[33844]: pgmap v560: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.7 MiB/s wr, 2.22k op/s 2024-05-09T05:55:55.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:55 smithi115 ceph-mon[30349]: pgmap v560: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.7 MiB/s wr, 2.22k op/s 2024-05-09T05:55:55.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:55 smithi161 ceph-mon[33805]: pgmap v560: 129 pgs: 129 active+clean; 15 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.7 MiB/s wr, 2.22k op/s 2024-05-09T05:55:57.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:57 smithi195 ceph-mon[33844]: pgmap v561: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-05-09T05:55:57.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:57 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:55:57.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:57 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:55:57.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:57 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:55:57.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:57 smithi115 ceph-mon[30349]: pgmap v561: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-05-09T05:55:57.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:57 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:55:57.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:57 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:55:57.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:57 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:55:57.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:57 smithi161 ceph-mon[33805]: pgmap v561: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-05-09T05:55:57.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:57 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:55:57.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:57 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:55:57.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:57 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:55:59.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:55:59 smithi195 ceph-mon[33844]: pgmap v562: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 2.97k op/s 2024-05-09T05:55:59.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:55:59 smithi115 ceph-mon[30349]: pgmap v562: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 2.97k op/s 2024-05-09T05:55:59.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:55:59 smithi161 ceph-mon[33805]: pgmap v562: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 2.97k op/s 2024-05-09T05:56:01.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:01 smithi195 ceph-mon[33844]: pgmap v563: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-05-09T05:56:01.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:01 smithi115 ceph-mon[30349]: pgmap v563: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-05-09T05:56:01.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:01 smithi161 ceph-mon[33805]: pgmap v563: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-05-09T05:56:03.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:03 smithi195 ceph-mon[33844]: pgmap v564: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-05-09T05:56:03.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:03 smithi115 ceph-mon[30349]: pgmap v564: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-05-09T05:56:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:03 smithi161 ceph-mon[33805]: pgmap v564: 129 pgs: 129 active+clean; 15 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-05-09T05:56:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:05 smithi195 ceph-mon[33844]: pgmap v565: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-05-09T05:56:05.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:05 smithi115 ceph-mon[30349]: pgmap v565: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-05-09T05:56:05.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:05 smithi161 ceph-mon[33805]: pgmap v565: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 16 MiB/s wr, 3.96k op/s 2024-05-09T05:56:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.616+0000 7ff52139d640 1 -- 172.21.15.115:0/2622820878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff51c11d250 msgr2=0x7ff51c11f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.616+0000 7ff52139d640 1 --2- 172.21.15.115:0/2622820878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff51c11d250 0x7ff51c11f670 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009900 tx=0x7ff50c02a4d0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.616+0000 7ff52139d640 1 -- 172.21.15.115:0/2622820878 shutdown_connections 2024-05-09T05:56:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.616+0000 7ff52139d640 1 --2- 172.21.15.115:0/2622820878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff51c11d250 0x7ff51c11f670 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.617+0000 7ff52139d640 1 --2- 172.21.15.115:0/2622820878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff51c11a8f0 0x7ff51c11cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.617+0000 7ff52139d640 1 --2- 172.21.15.115:0/2622820878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff51c10a200 0x7ff51c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.617+0000 7ff52139d640 1 -- 172.21.15.115:0/2622820878 >> 172.21.15.115:0/2622820878 conn(0x7ff51c06e900 msgr2=0x7ff51c06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.617+0000 7ff52139d640 1 -- 172.21.15.115:0/2622820878 shutdown_connections 2024-05-09T05:56:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.617+0000 7ff52139d640 1 -- 172.21.15.115:0/2622820878 wait complete. 2024-05-09T05:56:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff52139d640 1 Processor -- start 2024-05-09T05:56:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff52139d640 1 -- start start 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff52139d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff51c10a200 0x7ff51c11a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff52139d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff51c11a8f0 0x7ff51c114120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff52139d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff51c11d250 0x7ff51c114660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff52139d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c112800 con 0x7ff51c11a8f0 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff52139d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c112680 con 0x7ff51c11d250 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff52139d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c112980 con 0x7ff51c10a200 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff51affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff51c10a200 0x7ff51c11a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff51affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff51c10a200 0x7ff51c11a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:33454/0 (socket says 172.21.15.115:33454) 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff51affd640 1 -- 172.21.15.115:0/3202681486 learned_addr learned my addr 172.21.15.115:0/3202681486 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.618+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff51c11d250 0x7ff51c114660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.619+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff51c11a8f0 0x7ff51c114120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.619+0000 7ff51a7fc640 1 -- 172.21.15.115:0/3202681486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff51c10a200 msgr2=0x7ff51c11a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.619+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff51c10a200 0x7ff51c11a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.619+0000 7ff51a7fc640 1 -- 172.21.15.115:0/3202681486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff51c11d250 msgr2=0x7ff51c114660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.619+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff51c11d250 0x7ff51c114660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.619+0000 7ff51a7fc640 1 -- 172.21.15.115:0/3202681486 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c114ca0 con 0x7ff51c11a8f0 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.620+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff51c11a8f0 0x7ff51c114120 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ff51000e9f0 tx=0x7ff51000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.620+0000 7ff503fff640 1 -- 172.21.15.115:0/3202681486 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51000ccc0 con 0x7ff51c11a8f0 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.620+0000 7ff503fff640 1 -- 172.21.15.115:0/3202681486 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5100108c0 con 0x7ff51c11a8f0 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.620+0000 7ff503fff640 1 -- 172.21.15.115:0/3202681486 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510018810 con 0x7ff51c11a8f0 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.620+0000 7ff52139d640 1 -- 172.21.15.115:0/3202681486 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c1be160 con 0x7ff51c11a8f0 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.620+0000 7ff52139d640 1 -- 172.21.15.115:0/3202681486 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c1be510 con 0x7ff51c11a8f0 2024-05-09T05:56:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.620+0000 7ff51affd640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff51c10a200 0x7ff51c11a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:56:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.625+0000 7ff528255740 1 Processor -- start 2024-05-09T05:56:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.626+0000 7ff503fff640 1 -- 172.21.15.115:0/3202681486 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff510010430 con 0x7ff51c11a8f0 2024-05-09T05:56:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.627+0000 7ff503fff640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5040b2aa0 0x7ff5040b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.627+0000 7ff503fff640 1 -- 172.21.15.115:0/3202681486 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7ff5100e2b50 con 0x7ff51c11a8f0 2024-05-09T05:56:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.628+0000 7ff528255740 1 -- start start 2024-05-09T05:56:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.628+0000 7ff528255740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 0x55c9576ef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.628+0000 7ff528255740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9573232f0 0x55c9576efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.628+0000 7ff528255740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95732b4d0 0x55c9576f0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.628+0000 7ff528255740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c9576547c0 con 0x55c95734c1b0 2024-05-09T05:56:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.628+0000 7ff528255740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c9575f00e0 con 0x55c95732b4d0 2024-05-09T05:56:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.628+0000 7ff528255740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c957650780 con 0x55c9573232f0 2024-05-09T05:56:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.628+0000 7ff51b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95732b4d0 0x55c9576f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.629+0000 7ff51b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95732b4d0 0x55c9576f0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33676/0 (socket says 172.21.15.115:33676) 2024-05-09T05:56:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.629+0000 7ff51b7fe640 1 -- 172.21.15.115:0/3159500345 learned_addr learned my addr 172.21.15.115:0/3159500345 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.629+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5040b2aa0 0x7ff5040b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.629+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/3159500345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9573232f0 0x55c9576efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.629+0000 7ff51b7fe640 1 -- 172.21.15.115:0/3159500345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9573232f0 msgr2=0x55c9576efbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.629+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/3159500345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9573232f0 0x55c9576efbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.629+0000 7ff51b7fe640 1 -- 172.21.15.115:0/3159500345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 msgr2=0x55c9576ef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.629+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/3159500345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 0x55c9576ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.629+0000 7ff51b7fe640 1 -- 172.21.15.115:0/3159500345 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9576197c0 con 0x55c95732b4d0 2024-05-09T05:56:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.630+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/3159500345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95732b4d0 0x55c9576f0100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff50c033040 tx=0x7ff50c031760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184cdf285c6a95c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.630+0000 7ff5017fa640 1 -- 172.21.15.115:0/3159500345 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c040440 con 0x55c95732b4d0 2024-05-09T05:56:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.630+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5040b2aa0 0x7ff5040b4fb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff50c004a70 tx=0x7ff50c047000 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.630+0000 7ff5017fa640 1 -- 172.21.15.115:0/3159500345 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff50c03f870 con 0x55c95732b4d0 2024-05-09T05:56:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.630+0000 7ff5017fa640 1 -- 172.21.15.115:0/3159500345 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c03d610 con 0x55c95732b4d0 2024-05-09T05:56:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.631+0000 7ff528255740 1 -- 172.21.15.115:0/3159500345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95732b4d0 msgr2=0x55c9576f0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.631+0000 7ff528255740 1 --2- 172.21.15.115:0/3159500345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95732b4d0 0x55c9576f0100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff50c033040 tx=0x7ff50c031760 comp rx=0 tx=0).stop 2024-05-09T05:56:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.634+0000 7ff528255740 1 -- 172.21.15.115:0/3159500345 shutdown_connections 2024-05-09T05:56:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.634+0000 7ff528255740 1 --2- 172.21.15.115:0/3159500345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95732b4d0 0x55c9576f0100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.634+0000 7ff528255740 1 --2- 172.21.15.115:0/3159500345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c9573232f0 0x55c9576efbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.634+0000 7ff528255740 1 --2- 172.21.15.115:0/3159500345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 0x55c9576ef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.634+0000 7ff528255740 1 -- 172.21.15.115:0/3159500345 >> 172.21.15.115:0/3159500345 conn(0x55c957340300 msgr2=0x55c9576ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.637+0000 7ff528255740 1 -- 172.21.15.115:0/3159500345 shutdown_connections 2024-05-09T05:56:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.637+0000 7ff528255740 1 -- 172.21.15.115:0/3159500345 wait complete. 2024-05-09T05:56:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.638+0000 7ff528255740 1 -- start start 2024-05-09T05:56:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.638+0000 7ff528255740 1 Processor -- start 2024-05-09T05:56:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff528255740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c9576fb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff528255740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 0x55c9576fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff528255740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 0x55c9576f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff528255740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c9576fc6f0 con 0x55c95734c1b0 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff528255740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c9575db9c0 con 0x55c9573232f0 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff528255740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c9576fc870 con 0x55c95732b4d0 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff528255740 10 client.0 _ll_register_callbacks cb 0x55c957608020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff51affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c9576fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff51b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 0x55c9576f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff51a7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 0x55c9576fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff51b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 0x55c9576f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60878/0 (socket says 172.21.15.115:60878) 2024-05-09T05:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff51affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c9576fb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33678/0 (socket says 172.21.15.115:33678) 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.639+0000 7ff51b7fe640 1 -- 172.21.15.115:0/148946218 learned_addr learned my addr 172.21.15.115:0/148946218 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff51b7fe640 1 -- 172.21.15.115:0/148946218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 msgr2=0x55c9576fbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 0x55c9576fbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff51b7fe640 1 -- 172.21.15.115:0/148946218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 msgr2=0x55c9576fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c9576fb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff51b7fe640 1 -- 172.21.15.115:0/148946218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9576021f0 con 0x55c95734c1b0 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff51affd640 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c9576fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 0x55c9576f9b80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02a9f0 tx=0x7ff50c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5f5ff535afe277 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff4feff5640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff4feff5640 1 -- 172.21.15.115:0/148946218 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c056020 con 0x55c95734c1b0 2024-05-09T05:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff528255740 1 -- 172.21.15.115:0/148946218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff50c03f870 con 0x55c95734c1b0 2024-05-09T05:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.640+0000 7ff4feff5640 1 -- 172.21.15.115:0/148946218 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7ff50c03fc30 con 0x55c95734c1b0 2024-05-09T05:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.641+0000 7ff4feff5640 1 -- 172.21.15.115:0/148946218 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff50c061050 con 0x55c95734c1b0 2024-05-09T05:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.641+0000 7ff528255740 10 client.18969 fetch_fsmap learned FSMap version 67 2024-05-09T05:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.641+0000 7ff528255740 1 -- 172.21.15.115:0/148946218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55c9573b7790 con 0x55c95734c1b0 2024-05-09T05:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.641+0000 7ff4feff5640 1 -- 172.21.15.115:0/148946218 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff50c03f870 con 0x55c95734c1b0 2024-05-09T05:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.642+0000 7ff4feff5640 1 -- 172.21.15.115:0/148946218 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 67) ==== 8366+0+0 (secure 0 0 0) 0x7ff50c052470 con 0x55c95734c1b0 2024-05-09T05:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.642+0000 7ff528255740 10 client.18969 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T05:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.642+0000 7ff528255740 10 client.18969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.642+0000 7ff528255740 20 client.18969 populate_metadata read hostname 'smithi115' 2024-05-09T05:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.642+0000 7ff528255740 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c95730ad80 0x55c95770bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.642+0000 7ff528255740 4 client.18969 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.642+0000 7ff528255740 1 -- 172.21.15.115:0/148946218 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9576623d0 con 0x55c95730ad80 2024-05-09T05:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.642+0000 7ff51affd640 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c95730ad80 0x55c95770bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.643+0000 7ff4feff5640 10 client.18969 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.643+0000 7ff51affd640 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c95730ad80 0x55c95770bb20 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b698a39ee6958 server_cookie=6da41b7d2b82a1d7 in_seq=0 out_seq=0 2024-05-09T05:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.646+0000 7ff4feff5640 1 -- 172.21.15.115:0/148946218 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9576623d0 con 0x55c95730ad80 2024-05-09T05:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.646+0000 7ff4feff5640 10 client.18969 handle_command_reply: tid=0 2024-05-09T05:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.646+0000 7ff528255740 1 client.18969 shutdown 2024-05-09T05:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.646+0000 7ff528255740 1 -- 172.21.15.115:0/148946218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 msgr2=0x55c9576f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.646+0000 7ff528255740 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 0x55c9576f9b80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02a9f0 tx=0x7ff50c005fd0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.647+0000 7ff528255740 1 -- 172.21.15.115:0/148946218 shutdown_connections 2024-05-09T05:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.647+0000 7ff528255740 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95734c1b0 0x55c9576f9b80 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.647+0000 7ff528255740 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c95730ad80 0x55c95770bb20 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.647+0000 7ff528255740 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 0x55c9576fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.647+0000 7ff528255740 1 --2- 172.21.15.115:0/148946218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c9576fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.647+0000 7ff528255740 1 -- 172.21.15.115:0/148946218 >> 172.21.15.115:0/148946218 conn(0x55c957340300 msgr2=0x55c9576ea700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.648+0000 7ff4feff5640 0 client.18969 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.648+0000 7ff528255740 1 -- 172.21.15.115:0/148946218 shutdown_connections 2024-05-09T05:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.649+0000 7ff528255740 1 -- 172.21.15.115:0/148946218 wait complete. 2024-05-09T05:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.649+0000 7ff528255740 20 client.18969 trim_cache size 0 max 16384 2024-05-09T05:56:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.676+0000 7ff528255740 1 Processor -- start 2024-05-09T05:56:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.676+0000 7ff528255740 1 -- start start 2024-05-09T05:56:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.677+0000 7ff528255740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9573232f0 0x55c9576f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.677+0000 7ff528255740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 0x55c9576efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.677+0000 7ff528255740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95734c1b0 0x55c9576f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.677+0000 7ff528255740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1c84a0 con 0x55c9573232f0 2024-05-09T05:56:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.677+0000 7ff528255740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1c9fe0 con 0x55c95734c1b0 2024-05-09T05:56:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.677+0000 7ff528255740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1c8b70 con 0x55c95732b4d0 2024-05-09T05:56:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.677+0000 7ff51b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95734c1b0 0x55c9576f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.677+0000 7ff51b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95734c1b0 0x55c9576f0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33694/0 (socket says 172.21.15.115:33694) 2024-05-09T05:56:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.677+0000 7ff51b7fe640 1 -- 172.21.15.115:0/2774432008 learned_addr learned my addr 172.21.15.115:0/2774432008 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.678+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/2774432008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 0x55c9576efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.679+0000 7ff51affd640 1 --2- 172.21.15.115:0/2774432008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9573232f0 0x55c9576f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.679+0000 7ff51b7fe640 1 -- 172.21.15.115:0/2774432008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 msgr2=0x55c9576efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.679+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/2774432008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 0x55c9576efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.679+0000 7ff51b7fe640 1 -- 172.21.15.115:0/2774432008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9573232f0 msgr2=0x55c9576f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.679+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/2774432008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9573232f0 0x55c9576f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.679+0000 7ff51b7fe640 1 -- 172.21.15.115:0/2774432008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c95742e5f0 con 0x55c95734c1b0 2024-05-09T05:56:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.683+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/2774432008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95734c1b0 0x55c9576f0060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff50c000c00 tx=0x7ff50c005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4eed95f266cad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.684+0000 7ff500ff9640 1 -- 172.21.15.115:0/2774432008 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c056040 con 0x55c95734c1b0 2024-05-09T05:56:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.684+0000 7ff500ff9640 1 -- 172.21.15.115:0/2774432008 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff50c03d5e0 con 0x55c95734c1b0 2024-05-09T05:56:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.684+0000 7ff500ff9640 1 -- 172.21.15.115:0/2774432008 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c03fc30 con 0x55c95734c1b0 2024-05-09T05:56:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.686+0000 7ff528255740 1 -- 172.21.15.115:0/2774432008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95734c1b0 msgr2=0x55c9576f0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.686+0000 7ff528255740 1 --2- 172.21.15.115:0/2774432008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95734c1b0 0x55c9576f0060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff50c000c00 tx=0x7ff50c005ee0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.687+0000 7ff528255740 1 -- 172.21.15.115:0/2774432008 shutdown_connections 2024-05-09T05:56:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.687+0000 7ff528255740 1 --2- 172.21.15.115:0/2774432008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c95734c1b0 0x55c9576f0060 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.687+0000 7ff528255740 1 --2- 172.21.15.115:0/2774432008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95732b4d0 0x55c9576efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.687+0000 7ff528255740 1 --2- 172.21.15.115:0/2774432008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c9573232f0 0x55c9576f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.687+0000 7ff528255740 1 -- 172.21.15.115:0/2774432008 >> 172.21.15.115:0/2774432008 conn(0x55c957340300 msgr2=0x55c9576ef410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.691+0000 7ff528255740 1 -- 172.21.15.115:0/2774432008 shutdown_connections 2024-05-09T05:56:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.692+0000 7ff528255740 1 -- 172.21.15.115:0/2774432008 wait complete. 2024-05-09T05:56:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.692+0000 7ff528255740 1 -- start start 2024-05-09T05:56:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.693+0000 7ff528255740 1 Processor -- start 2024-05-09T05:56:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.693+0000 7ff528255740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c957701d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.693+0000 7ff528255740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95732b4d0 0x55c957702260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.693+0000 7ff528255740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95734c1b0 0x55c9576ff290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.693+0000 7ff528255740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1c9910 con 0x55c95732b4d0 2024-05-09T05:56:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.693+0000 7ff528255740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1c9240 con 0x55c9573232f0 2024-05-09T05:56:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.693+0000 7ff528255740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1c64a0 con 0x55c95734c1b0 2024-05-09T05:56:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.693+0000 7ff528255740 10 client.0 _ll_register_callbacks cb 0x7ff50c031660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:56:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.694+0000 7ff51affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c957701d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.695+0000 7ff51affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c957701d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33708/0 (socket says 172.21.15.115:33708) 2024-05-09T05:56:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.695+0000 7ff51affd640 1 -- 172.21.15.115:0/2150478230 learned_addr learned my addr 172.21.15.115:0/2150478230 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95732b4d0 0x55c957702260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95734c1b0 0x55c9576ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51affd640 1 -- 172.21.15.115:0/2150478230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95734c1b0 msgr2=0x55c9576ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51affd640 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95734c1b0 0x55c9576ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51affd640 1 -- 172.21.15.115:0/2150478230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95732b4d0 msgr2=0x55c957702260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51affd640 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95732b4d0 0x55c957702260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51affd640 1 -- 172.21.15.115:0/2150478230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c957618ed0 con 0x55c9573232f0 2024-05-09T05:56:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95732b4d0 0x55c957702260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:56:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51b7fe640 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95734c1b0 0x55c9576ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:56:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff51affd640 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c957701d20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff514009fd0 tx=0x7ff514003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7f71d6f21826138 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.696+0000 7ff4feff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:56:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.697+0000 7ff4feff5640 1 -- 172.21.15.115:0/2150478230 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51401c710 con 0x55c9573232f0 2024-05-09T05:56:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.697+0000 7ff528255740 1 -- 172.21.15.115:0/2150478230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff50c03d5e0 con 0x55c9573232f0 2024-05-09T05:56:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.697+0000 7ff4feff5640 1 -- 172.21.15.115:0/2150478230 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7ff51401c8b0 con 0x55c9573232f0 2024-05-09T05:56:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.702+0000 7ff4feff5640 1 -- 172.21.15.115:0/2150478230 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff51401cab0 con 0x55c9573232f0 2024-05-09T05:56:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.702+0000 7ff528255740 10 client.28157 fetch_fsmap learned FSMap version 67 2024-05-09T05:56:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.702+0000 7ff528255740 1 -- 172.21.15.115:0/2150478230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55c957773760 con 0x55c9573232f0 2024-05-09T05:56:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.703+0000 7ff4feff5640 1 -- 172.21.15.115:0/2150478230 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff50c03d5e0 con 0x55c9573232f0 2024-05-09T05:56:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.704+0000 7ff4feff5640 1 -- 172.21.15.115:0/2150478230 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 67) ==== 8366+0+0 (secure 0 0 0) 0x7ff51400f040 con 0x55c9573232f0 2024-05-09T05:56:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.704+0000 7ff528255740 10 client.28157 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T05:56:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.704+0000 7ff528255740 10 client.28157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:56:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.704+0000 7ff528255740 20 client.28157 populate_metadata read hostname 'smithi115' 2024-05-09T05:56:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.704+0000 7ff528255740 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c9572e01c0 0x55c957775f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.704+0000 7ff528255740 4 client.28157 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:56:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.704+0000 7ff528255740 1 -- 172.21.15.115:0/2150478230 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c95765c240 con 0x55c9572e01c0 2024-05-09T05:56:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.704+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c9572e01c0 0x55c957775f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.705+0000 7ff51a7fc640 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c9572e01c0 0x55c957775f00 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a9586e8937eca4 server_cookie=fd91e259f3035240 in_seq=0 out_seq=0 2024-05-09T05:56:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.705+0000 7ff4feff5640 10 client.28157 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff4feff5640 1 -- 172.21.15.115:0/2150478230 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c95765c240 con 0x55c9572e01c0 2024-05-09T05:56:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff4feff5640 10 client.28157 handle_command_reply: tid=0 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 client.28157 shutdown 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 -- 172.21.15.115:0/2150478230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 msgr2=0x55c957701d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c957701d20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff514009fd0 tx=0x7ff514003e00 comp rx=0 tx=0).stop 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 -- 172.21.15.115:0/2150478230 shutdown_connections 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c9572e01c0 0x55c957775f00 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c95734c1b0 0x55c9576ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c95732b4d0 0x55c957702260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 --2- 172.21.15.115:0/2150478230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9573232f0 0x55c957701d20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 -- 172.21.15.115:0/2150478230 >> 172.21.15.115:0/2150478230 conn(0x55c957340300 msgr2=0x55c9576efe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff4feff5640 0 client.28157 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 -- 172.21.15.115:0/2150478230 shutdown_connections 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.919+0000 7ff528255740 1 -- 172.21.15.115:0/2150478230 wait complete. 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.920+0000 7ff528255740 20 client.28157 trim_cache size 0 max 16384 2024-05-09T05:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:56:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:56:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:56:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 -- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5040b2aa0 msgr2=0x7ff5040b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5040b2aa0 0x7ff5040b4fb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff50c004a70 tx=0x7ff50c047000 comp rx=0 tx=0).stop 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 -- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff51c11a8f0 msgr2=0x7ff51c114120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff51c11a8f0 0x7ff51c114120 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ff51000e9f0 tx=0x7ff51000eeb0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 -- 172.21.15.115:0/3202681486 shutdown_connections 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff5040b2aa0 0x7ff5040b4fb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff51c11d250 0x7ff51c114660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff51c11a8f0 0x7ff51c114120 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 --2- 172.21.15.115:0/3202681486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff51c10a200 0x7ff51c11a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 -- 172.21.15.115:0/3202681486 >> 172.21.15.115:0/3202681486 conn(0x7ff51c06e900 msgr2=0x7ff51c11c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 -- 172.21.15.115:0/3202681486 shutdown_connections 2024-05-09T05:56:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:06.922+0000 7ff52139d640 1 -- 172.21.15.115:0/3202681486 wait complete. 2024-05-09T05:56:07.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:07 smithi195 ceph-mon[33844]: pgmap v566: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-05-09T05:56:07.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:07 smithi115 ceph-mon[30349]: pgmap v566: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-05-09T05:56:07.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:07 smithi161 ceph-mon[33805]: pgmap v566: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-05-09T05:56:09.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:09 smithi195 ceph-mon[33844]: pgmap v567: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-05-09T05:56:09.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:09 smithi115 ceph-mon[30349]: pgmap v567: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-05-09T05:56:09.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:09 smithi161 ceph-mon[33805]: pgmap v567: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-05-09T05:56:11.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:11 smithi195 ceph-mon[33844]: pgmap v568: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-05-09T05:56:11.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:11 smithi115 ceph-mon[30349]: pgmap v568: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-05-09T05:56:11.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:11 smithi161 ceph-mon[33805]: pgmap v568: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-05-09T05:56:13.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:13 smithi195 ceph-mon[33844]: pgmap v569: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-05-09T05:56:13.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:13 smithi115 ceph-mon[30349]: pgmap v569: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-05-09T05:56:13.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:13 smithi161 ceph-mon[33805]: pgmap v569: 129 pgs: 129 active+clean; 15 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-05-09T05:56:15.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:15 smithi195 ceph-mon[33844]: pgmap v570: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-05-09T05:56:15.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:15 smithi195 ceph-mon[33844]: osdmap e115: 12 total, 12 up, 12 in 2024-05-09T05:56:15.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:15 smithi115 ceph-mon[30349]: pgmap v570: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-05-09T05:56:15.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:15 smithi115 ceph-mon[30349]: osdmap e115: 12 total, 12 up, 12 in 2024-05-09T05:56:15.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:15 smithi161 ceph-mon[33805]: pgmap v570: 129 pgs: 129 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2024-05-09T05:56:15.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:15 smithi161 ceph-mon[33805]: osdmap e115: 12 total, 12 up, 12 in 2024-05-09T05:56:17.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:17 smithi195 ceph-mon[33844]: pgmap v572: 129 pgs: 6 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 103 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-05-09T05:56:17.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:17 smithi115 ceph-mon[30349]: pgmap v572: 129 pgs: 6 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 103 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-05-09T05:56:17.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:17 smithi161 ceph-mon[33805]: pgmap v572: 129 pgs: 6 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 103 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-05-09T05:56:19.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:19 smithi195 ceph-mon[33844]: pgmap v573: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 13 MiB/s wr, 3.10k op/s 2024-05-09T05:56:19.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:19 smithi115 ceph-mon[30349]: pgmap v573: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 13 MiB/s wr, 3.10k op/s 2024-05-09T05:56:19.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:19 smithi161 ceph-mon[33805]: pgmap v573: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 13 MiB/s wr, 3.10k op/s 2024-05-09T05:56:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.261+0000 7f4af31c7640 1 -- 172.21.15.115:0/2103057276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aec11d250 msgr2=0x7f4aec11f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.261+0000 7f4af31c7640 1 --2- 172.21.15.115:0/2103057276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aec11d250 0x7f4aec11f670 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009900 tx=0x7f4ae002a4d0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.261+0000 7f4af31c7640 1 -- 172.21.15.115:0/2103057276 shutdown_connections 2024-05-09T05:56:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.261+0000 7f4af31c7640 1 --2- 172.21.15.115:0/2103057276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aec11d250 0x7f4aec11f670 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.261+0000 7f4af31c7640 1 --2- 172.21.15.115:0/2103057276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4aec11a8f0 0x7f4aec11cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.261+0000 7f4af31c7640 1 --2- 172.21.15.115:0/2103057276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4aec10a200 0x7f4aec10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.261+0000 7f4af31c7640 1 -- 172.21.15.115:0/2103057276 >> 172.21.15.115:0/2103057276 conn(0x7f4aec06e900 msgr2=0x7f4aec06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.278+0000 7f4af31c7640 1 -- 172.21.15.115:0/2103057276 shutdown_connections 2024-05-09T05:56:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.278+0000 7f4af31c7640 1 -- 172.21.15.115:0/2103057276 wait complete. 2024-05-09T05:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.279+0000 7f4af31c7640 1 Processor -- start 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.284+0000 7f4af31c7640 1 -- start start 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.284+0000 7f4af31c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aec10a200 0x7f4aec1a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.284+0000 7f4af31c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4aec11a8f0 0x7f4aec1a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.285+0000 7f4af31c7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4aec11d250 0x7f4aec1adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.285+0000 7f4af31c7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec122ba0 con 0x7f4aec10a200 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.285+0000 7f4af31c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec122a20 con 0x7f4aec11a8f0 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.285+0000 7f4af31c7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec122d20 con 0x7f4aec11d250 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.285+0000 7f4af1764640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4aec11d250 0x7f4aec1adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.285+0000 7f4af1764640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4aec11d250 0x7f4aec1adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:38818/0 (socket says 172.21.15.115:38818) 2024-05-09T05:56:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.285+0000 7f4af1764640 1 -- 172.21.15.115:0/713852335 learned_addr learned my addr 172.21.15.115:0/713852335 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.285+0000 7f4aebfff640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4aec11a8f0 0x7f4aec1a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.286+0000 7f4af0f63640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aec10a200 0x7f4aec1a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.286+0000 7f4af1764640 1 -- 172.21.15.115:0/713852335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4aec11a8f0 msgr2=0x7f4aec1a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.286+0000 7f4af1764640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4aec11a8f0 0x7f4aec1a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.286+0000 7f4af1764640 1 -- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aec10a200 msgr2=0x7f4aec1a9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.286+0000 7f4af1764640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aec10a200 0x7f4aec1a9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.286+0000 7f4af1764640 1 -- 172.21.15.115:0/713852335 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec1ae580 con 0x7f4aec11d250 2024-05-09T05:56:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.289+0000 7f4af1764640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4aec11d250 0x7f4aec1adf40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009850 tx=0x7f4ae0003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.289+0000 7f4ae9ffb640 1 -- 172.21.15.115:0/713852335 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae003f070 con 0x7f4aec11d250 2024-05-09T05:56:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.290+0000 7f4af31c7640 1 -- 172.21.15.115:0/713852335 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec1ae810 con 0x7f4aec11d250 2024-05-09T05:56:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.290+0000 7f4af31c7640 1 -- 172.21.15.115:0/713852335 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec1aed00 con 0x7f4aec11d250 2024-05-09T05:56:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.290+0000 7f4ae9ffb640 1 -- 172.21.15.115:0/713852335 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ae003bd00 con 0x7f4aec11d250 2024-05-09T05:56:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.290+0000 7f4ae9ffb640 1 -- 172.21.15.115:0/713852335 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0035030 con 0x7f4aec11d250 2024-05-09T05:56:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.297+0000 7f4afa077740 1 Processor -- start 2024-05-09T05:56:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.297+0000 7f4afa077740 1 -- start start 2024-05-09T05:56:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4afa077740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x562736613680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4afa077740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x562736613bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4afa077740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736614100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4afa077740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5627365787c0 con 0x5627362701b0 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4afa077740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5627365140e0 con 0x5627362472f0 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4afa077740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562736574780 con 0x56273624f4d0 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4ae9ffb640 1 -- 172.21.15.115:0/713852335 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4ae0042070 con 0x7f4aec11d250 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736614100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736614100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:38832/0 (socket says 172.21.15.115:38832) 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4aebfff640 1 -- 172.21.15.115:0/3388512835 learned_addr learned my addr 172.21.15.115:0/3388512835 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4af0f63640 1 --2- 172.21.15.115:0/3388512835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x562736613bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4aebfff640 1 -- 172.21.15.115:0/3388512835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 msgr2=0x562736613bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4aebfff640 1 --2- 172.21.15.115:0/3388512835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x562736613bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4aebfff640 1 -- 172.21.15.115:0/3388512835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 msgr2=0x562736613680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4aebfff640 1 --2- 172.21.15.115:0/3388512835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x562736613680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.298+0000 7f4aebfff640 1 -- 172.21.15.115:0/3388512835 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273653d7c0 con 0x56273624f4d0 2024-05-09T05:56:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.299+0000 7f4aebfff640 1 --2- 172.21.15.115:0/3388512835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736614100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009a70 tx=0x7f4adc009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7c58959b487a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.300+0000 7f4ae9ffb640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ad40b2b60 0x7f4ad40b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.300+0000 7f4ae9ffb640 1 -- 172.21.15.115:0/713852335 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f4ae0103670 con 0x7f4aec11d250 2024-05-09T05:56:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.301+0000 7f4af0f63640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ad40b2b60 0x7f4ad40b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.301+0000 7f4af0f63640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ad40b2b60 0x7f4ad40b5070 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x562736614a30 tx=0x7f4ae400e040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.302+0000 7f4ad2ffd640 1 -- 172.21.15.115:0/3388512835 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc018020 con 0x56273624f4d0 2024-05-09T05:56:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.302+0000 7f4ad2ffd640 1 -- 172.21.15.115:0/3388512835 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4adc010bc0 con 0x56273624f4d0 2024-05-09T05:56:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.303+0000 7f4afa077740 1 -- 172.21.15.115:0/3388512835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 msgr2=0x562736614100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.303+0000 7f4afa077740 1 --2- 172.21.15.115:0/3388512835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736614100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009a70 tx=0x7f4adc009f30 comp rx=0 tx=0).stop 2024-05-09T05:56:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.304+0000 7f4afa077740 1 -- 172.21.15.115:0/3388512835 shutdown_connections 2024-05-09T05:56:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.304+0000 7f4afa077740 1 --2- 172.21.15.115:0/3388512835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736614100 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.304+0000 7f4afa077740 1 --2- 172.21.15.115:0/3388512835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x562736613bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.304+0000 7f4afa077740 1 --2- 172.21.15.115:0/3388512835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x562736613680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.304+0000 7f4afa077740 1 -- 172.21.15.115:0/3388512835 >> 172.21.15.115:0/3388512835 conn(0x562736264300 msgr2=0x56273660fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.310+0000 7f4afa077740 1 -- 172.21.15.115:0/3388512835 shutdown_connections 2024-05-09T05:56:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.310+0000 7f4afa077740 1 -- 172.21.15.115:0/3388512835 wait complete. 2024-05-09T05:56:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.311+0000 7f4afa077740 1 -- start start 2024-05-09T05:56:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.311+0000 7f4afa077740 1 Processor -- start 2024-05-09T05:56:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4afa077740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273661dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4afa077740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56273624f4d0 0x56273661e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4af0f63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273661dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4af0f63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273661dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:45174/0 (socket says 172.21.15.115:45174) 2024-05-09T05:56:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56273624f4d0 0x56273661e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4aebfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56273624f4d0 0x56273661e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54130/0 (socket says 172.21.15.115:54130) 2024-05-09T05:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4af0f63640 1 -- 172.21.15.115:0/4023752900 learned_addr learned my addr 172.21.15.115:0/4023752900 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4afa077740 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627362701b0 0x562736623d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562736620800 con 0x56273624f4d0 2024-05-09T05:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.312+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5627364ff9c0 con 0x5627362472f0 2024-05-09T05:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.313+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562736620980 con 0x5627362701b0 2024-05-09T05:56:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.313+0000 7f4af1764640 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627362701b0 0x562736623d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.313+0000 7f4af1764640 1 -- 172.21.15.115:0/4023752900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 msgr2=0x56273661dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.313+0000 7f4af1764640 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273661dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.313+0000 7f4af1764640 1 -- 172.21.15.115:0/4023752900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56273624f4d0 msgr2=0x56273661e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.313+0000 7f4af1764640 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56273624f4d0 0x56273661e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.313+0000 7f4af1764640 1 -- 172.21.15.115:0/4023752900 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627365261f0 con 0x5627362701b0 2024-05-09T05:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.313+0000 7f4af1764640 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627362701b0 0x562736623d60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0100970 tx=0x7f4ae00c9d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b2830bd4aa62f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.313+0000 7f4afa077740 10 client.0 _ll_register_callbacks cb 0x56273652c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:56:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.314+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4adc010bc0 con 0x5627362701b0 2024-05-09T05:56:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.314+0000 7f4acbfff640 10 client.28303 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:56:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.316+0000 7f4af0f63640 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273661dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:56:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.316+0000 7f4aebfff640 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56273624f4d0 0x56273661e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:56:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.332+0000 7f4acbfff640 1 -- 172.21.15.115:0/4023752900 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0049050 con 0x5627362701b0 2024-05-09T05:56:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.332+0000 7f4acbfff640 1 -- 172.21.15.115:0/4023752900 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f4ae0100bf0 con 0x5627362701b0 2024-05-09T05:56:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.347+0000 7f4acbfff640 1 -- 172.21.15.115:0/4023752900 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f4ae0100df0 con 0x5627362701b0 2024-05-09T05:56:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.347+0000 7f4afa077740 10 client.28303 fetch_fsmap learned FSMap version 67 2024-05-09T05:56:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.347+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({fsmap=67}) -- 0x5627362db790 con 0x5627362701b0 2024-05-09T05:56:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.350+0000 7f4acbfff640 1 -- 172.21.15.115:0/4023752900 <== mon.1 v2:172.21.15.195:3300/0 4 ==== fsmap(e 67) ==== 8366+0+0 (secure 0 0 0) 0x7f4ae00ca760 con 0x5627362701b0 2024-05-09T05:56:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.350+0000 7f4afa077740 10 client.28303 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T05:56:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.350+0000 7f4afa077740 10 client.28303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:56:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.350+0000 7f4afa077740 20 client.28303 populate_metadata read hostname 'smithi115' 2024-05-09T05:56:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.350+0000 7f4afa077740 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56273622ed80 0x56273662ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.350+0000 7f4afa077740 4 client.28303 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:56:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.350+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627365863d0 con 0x56273622ed80 2024-05-09T05:56:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.353+0000 7f4af0f63640 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56273622ed80 0x56273662ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.353+0000 7f4af0f63640 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56273622ed80 0x56273662ff50 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af467dbfbcd4f8c server_cookie=569ed3b55040726a in_seq=0 out_seq=0 2024-05-09T05:56:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.355+0000 7f4acbfff640 10 client.28303 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.359+0000 7f4acbfff640 1 -- 172.21.15.115:0/4023752900 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5627365863d0 con 0x56273622ed80 2024-05-09T05:56:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.359+0000 7f4acbfff640 10 client.28303 handle_command_reply: tid=0 2024-05-09T05:56:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.359+0000 7f4afa077740 1 client.28303 shutdown 2024-05-09T05:56:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.360+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627362701b0 msgr2=0x562736623d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.360+0000 7f4afa077740 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627362701b0 0x562736623d60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0100970 tx=0x7f4ae00c9d30 comp rx=0 tx=0).stop 2024-05-09T05:56:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.363+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 shutdown_connections 2024-05-09T05:56:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.363+0000 7f4afa077740 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56273622ed80 0x56273662ff50 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.363+0000 7f4afa077740 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5627362701b0 0x562736623d60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.363+0000 7f4afa077740 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56273624f4d0 0x56273661e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.363+0000 7f4afa077740 1 --2- 172.21.15.115:0/4023752900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273661dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.363+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 >> 172.21.15.115:0/4023752900 conn(0x562736264300 msgr2=0x56273660e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.363+0000 7f4acbfff640 0 client.28303 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.365+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 shutdown_connections 2024-05-09T05:56:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.365+0000 7f4afa077740 1 -- 172.21.15.115:0/4023752900 wait complete. 2024-05-09T05:56:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.365+0000 7f4afa077740 20 client.28303 trim_cache size 0 max 16384 2024-05-09T05:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.417+0000 7f4afa077740 1 Processor -- start 2024-05-09T05:56:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.417+0000 7f4afa077740 1 -- start start 2024-05-09T05:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.418+0000 7f4afa077740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x5627366179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.418+0000 7f4afa077740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736623cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.418+0000 7f4af0f63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x5627366179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.418+0000 7f4af0f63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x5627366179e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:45186/0 (socket says 172.21.15.115:45186) 2024-05-09T05:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.418+0000 7f4af0f63640 1 -- 172.21.15.115:0/2466303500 learned_addr learned my addr 172.21.15.115:0/2466303500 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.418+0000 7f4afa077740 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x562736624210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.418+0000 7f4aebfff640 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736623cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.419+0000 7f4af1764640 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x562736624210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.419+0000 7f4afa077740 1 -- 172.21.15.115:0/2466303500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8010060 con 0x5627362701b0 2024-05-09T05:56:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.419+0000 7f4afa077740 1 -- 172.21.15.115:0/2466303500 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8011ba0 con 0x5627362472f0 2024-05-09T05:56:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.419+0000 7f4afa077740 1 -- 172.21.15.115:0/2466303500 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8010730 con 0x56273624f4d0 2024-05-09T05:56:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.420+0000 7f4aebfff640 1 -- 172.21.15.115:0/2466303500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 msgr2=0x5627366179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.420+0000 7f4aebfff640 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x5627366179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.420+0000 7f4aebfff640 1 -- 172.21.15.115:0/2466303500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 msgr2=0x562736624210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.420+0000 7f4aebfff640 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x562736624210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.420+0000 7f4aebfff640 1 -- 172.21.15.115:0/2466303500 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627363525f0 con 0x56273624f4d0 2024-05-09T05:56:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.420+0000 7f4aebfff640 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736623cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0106e0 tx=0x7f4adc009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab9863f89537cc89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.421+0000 7f4ad27fc640 1 -- 172.21.15.115:0/2466303500 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc026070 con 0x56273624f4d0 2024-05-09T05:56:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.421+0000 7f4ad27fc640 1 -- 172.21.15.115:0/2466303500 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4adc014470 con 0x56273624f4d0 2024-05-09T05:56:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.421+0000 7f4ad27fc640 1 -- 172.21.15.115:0/2466303500 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc018030 con 0x56273624f4d0 2024-05-09T05:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.423+0000 7f4afa077740 1 -- 172.21.15.115:0/2466303500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 msgr2=0x562736623cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.423+0000 7f4afa077740 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736623cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0106e0 tx=0x7f4adc009120 comp rx=0 tx=0).stop 2024-05-09T05:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.426+0000 7f4afa077740 1 -- 172.21.15.115:0/2466303500 shutdown_connections 2024-05-09T05:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.426+0000 7f4afa077740 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x562736624210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.426+0000 7f4afa077740 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x562736623cd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.426+0000 7f4afa077740 1 --2- 172.21.15.115:0/2466303500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x5627366179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.426+0000 7f4afa077740 1 -- 172.21.15.115:0/2466303500 >> 172.21.15.115:0/2466303500 conn(0x562736264300 msgr2=0x5627366172d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.426+0000 7f4afa077740 1 -- 172.21.15.115:0/2466303500 shutdown_connections 2024-05-09T05:56:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.426+0000 7f4afa077740 1 -- 172.21.15.115:0/2466303500 wait complete. 2024-05-09T05:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.427+0000 7f4afa077740 1 -- start start 2024-05-09T05:56:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.427+0000 7f4afa077740 1 Processor -- start 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.427+0000 7f4afa077740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273662a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.427+0000 7f4afa077740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x56273662a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.427+0000 7f4afa077740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x5627366258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.427+0000 7f4afa077740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad80114d0 con 0x5627362701b0 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.428+0000 7f4afa077740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8010e00 con 0x5627362472f0 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.428+0000 7f4afa077740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad800e060 con 0x56273624f4d0 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.428+0000 7f4afa077740 10 client.0 _ll_register_callbacks cb 0x7f4ae0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.428+0000 7f4af1764640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x5627366258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.428+0000 7f4af1764640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x5627366258b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54152/0 (socket says 172.21.15.115:54152) 2024-05-09T05:56:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.428+0000 7f4af1764640 1 -- 172.21.15.115:0/3085031177 learned_addr learned my addr 172.21.15.115:0/3085031177 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.428+0000 7f4af0f63640 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273662a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.428+0000 7f4aebfff640 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x56273662a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.429+0000 7f4af1764640 1 -- 172.21.15.115:0/3085031177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 msgr2=0x56273662a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.429+0000 7f4af1764640 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x56273662a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.429+0000 7f4af1764640 1 -- 172.21.15.115:0/3085031177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 msgr2=0x56273662a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.429+0000 7f4af1764640 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273662a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.429+0000 7f4af1764640 1 -- 172.21.15.115:0/3085031177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56273653ced0 con 0x5627362701b0 2024-05-09T05:56:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.433+0000 7f4af1764640 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x5627366258b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0100a20 tx=0x7f4ae0045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5befe4bad56533f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.434+0000 7f4acbfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:56:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.434+0000 7f4acbfff640 1 -- 172.21.15.115:0/3085031177 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00ca500 con 0x5627362701b0 2024-05-09T05:56:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.434+0000 7f4acbfff640 1 -- 172.21.15.115:0/3085031177 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f4ae00c6af0 con 0x5627362701b0 2024-05-09T05:56:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.434+0000 7f4afa077740 1 -- 172.21.15.115:0/3085031177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4adc014470 con 0x5627362701b0 2024-05-09T05:56:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.435+0000 7f4acbfff640 1 -- 172.21.15.115:0/3085031177 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f4ae00ce8b0 con 0x5627362701b0 2024-05-09T05:56:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.443+0000 7f4afa077740 10 client.18984 fetch_fsmap learned FSMap version 67 2024-05-09T05:56:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.443+0000 7f4afa077740 1 -- 172.21.15.115:0/3085031177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x562736697bd0 con 0x5627362701b0 2024-05-09T05:56:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.450+0000 7f4acbfff640 1 -- 172.21.15.115:0/3085031177 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4adc010bc0 con 0x5627362701b0 2024-05-09T05:56:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.451+0000 7f4acbfff640 1 -- 172.21.15.115:0/3085031177 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 67) ==== 8366+0+0 (secure 0 0 0) 0x7f4ae00d2ed0 con 0x5627362701b0 2024-05-09T05:56:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.451+0000 7f4afa077740 10 client.18984 fetch_fsmap finished waiting for FSMap version 67 2024-05-09T05:56:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.451+0000 7f4afa077740 10 client.18984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:56:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.451+0000 7f4afa077740 20 client.18984 populate_metadata read hostname 'smithi115' 2024-05-09T05:56:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.451+0000 7f4afa077740 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5627362041c0 0x56273669a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.451+0000 7f4afa077740 4 client.18984 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:56:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.452+0000 7f4afa077740 1 -- 172.21.15.115:0/3085031177 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4adc014470 con 0x5627362041c0 2024-05-09T05:56:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.453+0000 7f4af0f63640 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5627362041c0 0x56273669a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.463+0000 7f4af0f63640 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5627362041c0 0x56273669a150 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c9f8a55aa00166 server_cookie=c80ab5ff4e3f83a in_seq=0 out_seq=0 2024-05-09T05:56:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:20.465+0000 7f4acbfff640 10 client.18984 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:21.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:21 smithi195 ceph-mon[33844]: pgmap v574: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-05-09T05:56:21.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:21 smithi115 ceph-mon[30349]: pgmap v574: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-05-09T05:56:21.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:21 smithi161 ceph-mon[33805]: pgmap v574: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-05-09T05:56:23.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:23 smithi195 ceph-mon[33844]: pgmap v575: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 3.37k op/s 2024-05-09T05:56:23.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:23 smithi115 ceph-mon[30349]: pgmap v575: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 3.37k op/s 2024-05-09T05:56:23.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:23 smithi161 ceph-mon[33805]: pgmap v575: 129 pgs: 19 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 60 active+clean; 15 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 3.37k op/s 2024-05-09T05:56:24.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:24 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:56:24.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:24 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:56:24.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:24 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:56:25.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:25 smithi195 ceph-mon[33844]: pgmap v576: 129 pgs: 18 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 3.06k op/s 2024-05-09T05:56:25.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:25 smithi115 ceph-mon[30349]: pgmap v576: 129 pgs: 18 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 3.06k op/s 2024-05-09T05:56:25.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:25 smithi161 ceph-mon[33805]: pgmap v576: 129 pgs: 18 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 3.06k op/s 2024-05-09T05:56:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.250+0000 7f4acbfff640 1 -- 172.21.15.115:0/3085031177 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f4adc014470 con 0x5627362041c0 2024-05-09T05:56:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.250+0000 7f4acbfff640 10 client.18984 handle_command_reply: tid=0 2024-05-09T05:56:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.250+0000 7f4afa077740 1 client.18984 shutdown 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.253+0000 7f4afa077740 1 -- 172.21.15.115:0/3085031177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 msgr2=0x5627366258b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.253+0000 7f4afa077740 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x5627366258b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0100a20 tx=0x7f4ae0045000 comp rx=0 tx=0).stop 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4afa077740 1 -- 172.21.15.115:0/3085031177 shutdown_connections 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4afa077740 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5627362701b0 0x5627366258b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4afa077740 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5627362041c0 0x56273669a150 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4afa077740 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56273624f4d0 0x56273662a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4afa077740 1 --2- 172.21.15.115:0/3085031177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5627362472f0 0x56273662a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4afa077740 1 -- 172.21.15.115:0/3085031177 >> 172.21.15.115:0/3085031177 conn(0x562736264300 msgr2=0x56273660f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4acbfff640 0 client.18984 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4afa077740 1 -- 172.21.15.115:0/3085031177 shutdown_connections 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4afa077740 1 -- 172.21.15.115:0/3085031177 wait complete. 2024-05-09T05:56:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.254+0000 7f4afa077740 20 client.18984 trim_cache size 0 max 16384 2024-05-09T05:56:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:56:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:56:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:56:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.256+0000 7f4af31c7640 1 -- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ad40b2b60 msgr2=0x7f4ad40b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.256+0000 7f4af31c7640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ad40b2b60 0x7f4ad40b5070 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x562736614a30 tx=0x7f4ae400e040 comp rx=0 tx=0).stop 2024-05-09T05:56:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.256+0000 7f4af31c7640 1 -- 172.21.15.115:0/713852335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4aec11d250 msgr2=0x7f4aec1adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.256+0000 7f4af31c7640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4aec11d250 0x7f4aec1adf40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009850 tx=0x7f4ae0003e10 comp rx=0 tx=0).stop 2024-05-09T05:56:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.256+0000 7f4af31c7640 1 -- 172.21.15.115:0/713852335 shutdown_connections 2024-05-09T05:56:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.256+0000 7f4af31c7640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4ad40b2b60 0x7f4ad40b5070 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.257+0000 7f4af31c7640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4aec11d250 0x7f4aec1adf40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.257+0000 7f4af31c7640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4aec11a8f0 0x7f4aec1a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.257+0000 7f4af31c7640 1 --2- 172.21.15.115:0/713852335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aec10a200 0x7f4aec1a9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.257+0000 7f4af31c7640 1 -- 172.21.15.115:0/713852335 >> 172.21.15.115:0/713852335 conn(0x7f4aec06e900 msgr2=0x7f4aec11ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.258+0000 7f4af31c7640 1 -- 172.21.15.115:0/713852335 shutdown_connections 2024-05-09T05:56:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:26.258+0000 7f4af31c7640 1 -- 172.21.15.115:0/713852335 wait complete. 2024-05-09T05:56:27.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:27 smithi195 ceph-mon[33844]: pgmap v577: 129 pgs: 18 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 2.89k op/s 2024-05-09T05:56:27.895 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:27 smithi115 ceph-mon[30349]: pgmap v577: 129 pgs: 18 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 2.89k op/s 2024-05-09T05:56:27.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:27 smithi161 ceph-mon[33805]: pgmap v577: 129 pgs: 18 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 2.89k op/s 2024-05-09T05:56:29.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:29 smithi195 ceph-mon[33844]: pgmap v578: 129 pgs: 16 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 71 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-05-09T05:56:29.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:29 smithi115 ceph-mon[30349]: pgmap v578: 129 pgs: 16 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 71 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-05-09T05:56:29.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:29 smithi161 ceph-mon[33805]: pgmap v578: 129 pgs: 16 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 71 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-05-09T05:56:31.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:31 smithi195 ceph-mon[33844]: pgmap v579: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-05-09T05:56:31.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:31 smithi115 ceph-mon[30349]: pgmap v579: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-05-09T05:56:31.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:31 smithi161 ceph-mon[33805]: pgmap v579: 129 pgs: 14 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 81 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-05-09T05:56:33.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:33 smithi195 ceph-mon[33844]: pgmap v580: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 2.77k op/s 2024-05-09T05:56:33.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:33 smithi115 ceph-mon[30349]: pgmap v580: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 2.77k op/s 2024-05-09T05:56:33.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:33 smithi161 ceph-mon[33805]: pgmap v580: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 2.77k op/s 2024-05-09T05:56:34.365 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.3 2024-05-09T05:56:34.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:56:35.148 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:56:35.601 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:35 smithi115 ceph-mon[30349]: pgmap v581: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 3.49k op/s 2024-05-09T05:56:35.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:35 smithi195 ceph-mon[33844]: pgmap v581: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 3.49k op/s 2024-05-09T05:56:35.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:35 smithi161 ceph-mon[33805]: pgmap v581: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 3.49k op/s 2024-05-09T05:56:37.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:37 smithi115 ceph-mon[30349]: pgmap v582: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-05-09T05:56:37.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:37 smithi195 ceph-mon[33844]: pgmap v582: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-05-09T05:56:37.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:37 smithi161 ceph-mon[33805]: pgmap v582: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-05-09T05:56:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.465+0000 7f8e7ed14640 1 -- 172.21.15.115:0/1285478030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e700a5840 msgr2=0x7f8e700a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.465+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/1285478030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e700a5840 0x7f8e700a5c50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8e780687c0 tx=0x7f8e7808f250 comp rx=0 tx=0).stop 2024-05-09T05:56:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.467+0000 7f8e7ed14640 1 -- 172.21.15.115:0/1285478030 shutdown_connections 2024-05-09T05:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.467+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/1285478030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e700b8710 0x7f8e700bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.467+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/1285478030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e700b5db0 0x7f8e700b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.467+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/1285478030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e700a5840 0x7f8e700a5c50 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.467+0000 7f8e7ed14640 1 -- 172.21.15.115:0/1285478030 >> 172.21.15.115:0/1285478030 conn(0x7f8e7001af20 msgr2=0x7f8e7001b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.470+0000 7f8e7ed14640 1 -- 172.21.15.115:0/1285478030 shutdown_connections 2024-05-09T05:56:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.471+0000 7f8e7ed14640 1 -- 172.21.15.115:0/1285478030 wait complete. 2024-05-09T05:56:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.472+0000 7f8e7ed14640 1 Processor -- start 2024-05-09T05:56:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.472+0000 7f8e7ed14640 1 -- start start 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e7ed14640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e700a5840 0x7f8e700aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e7ed14640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e700b5db0 0x7f8e700aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e7ed14640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e700b8710 0x7f8e700af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e7ed14640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e700be130 con 0x7f8e700b8710 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e7ed14640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8e700bdfb0 con 0x7f8e700a5840 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e7ed14640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8e700be2b0 con 0x7f8e700b5db0 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e700b5db0 0x7f8e700aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e700b5db0 0x7f8e700aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46424/0 (socket says 172.21.15.115:46424) 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e77fff640 1 -- 172.21.15.115:0/3696094341 learned_addr learned my addr 172.21.15.115:0/3696094341 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.473+0000 7f8e7d2b1640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e700b8710 0x7f8e700af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.474+0000 7f8e7d2b1640 1 -- 172.21.15.115:0/3696094341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e700b5db0 msgr2=0x7f8e700aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.474+0000 7f8e7d2b1640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e700b5db0 0x7f8e700aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.474+0000 7f8e7d2b1640 1 -- 172.21.15.115:0/3696094341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e700a5840 msgr2=0x7f8e700aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.474+0000 7f8e7d2b1640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e700a5840 0x7f8e700aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.474+0000 7f8e7d2b1640 1 -- 172.21.15.115:0/3696094341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e700b3440 con 0x7f8e700b8710 2024-05-09T05:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.475+0000 7f8e7d2b1640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e700b8710 0x7f8e700af510 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00b3e0 tx=0x7f8e6c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.475+0000 7f8e75ffb640 1 -- 172.21.15.115:0/3696094341 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c015020 con 0x7f8e700b8710 2024-05-09T05:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.475+0000 7f8e75ffb640 1 -- 172.21.15.115:0/3696094341 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e6c011cc0 con 0x7f8e700b8710 2024-05-09T05:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.475+0000 7f8e75ffb640 1 -- 172.21.15.115:0/3696094341 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c010d40 con 0x7f8e700b8710 2024-05-09T05:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.475+0000 7f8e7ed14640 1 -- 172.21.15.115:0/3696094341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e700b3730 con 0x7f8e700b8710 2024-05-09T05:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.475+0000 7f8e7ed14640 1 -- 172.21.15.115:0/3696094341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e700b3c00 con 0x7f8e700b8710 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.481+0000 7f8e85bc8740 1 Processor -- start 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.484+0000 7f8e75ffb640 1 -- 172.21.15.115:0/3696094341 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f8e6c018020 con 0x7f8e700b8710 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.485+0000 7f8e75ffb640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e680b2a90 0x7f8e680b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.485+0000 7f8e75ffb640 1 -- 172.21.15.115:0/3696094341 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f8e6c0e2120 con 0x7f8e700b8710 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.488+0000 7f8e85bc8740 1 -- start start 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.488+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.488+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.488+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 0x55bcca3c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.488+0000 7f8e85bc8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bcca32d7c0 con 0x55bcca0044d0 2024-05-09T05:56:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.488+0000 7f8e85bc8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bcca2c90e0 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.488+0000 7f8e85bc8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcca329780 con 0x55bcca0251b0 2024-05-09T05:56:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.492+0000 7f8e7cab0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.492+0000 7f8e7cab0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46426/0 (socket says 172.21.15.115:46426) 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.492+0000 7f8e7cab0640 1 -- 172.21.15.115:0/2878301464 learned_addr learned my addr 172.21.15.115:0/2878301464 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.492+0000 7f8e77fff640 1 --2- 172.21.15.115:0/2878301464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.492+0000 7f8e77fff640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e680b2a90 0x7f8e680b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.492+0000 7f8e7cab0640 1 -- 172.21.15.115:0/2878301464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 msgr2=0x55bcca3c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.492+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/2878301464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.492+0000 7f8e7cab0640 1 -- 172.21.15.115:0/2878301464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 msgr2=0x55bcca3c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.492+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/2878301464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 0x55bcca3c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.493+0000 7f8e7cab0640 1 -- 172.21.15.115:0/2878301464 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcca2f27c0 con 0x55bcca0251b0 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.493+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/2878301464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3c8840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e78052170 tx=0x7f8e78069110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ba303fb6841523 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.493+0000 7f8e62ffd640 1 -- 172.21.15.115:0/2878301464 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7806b050 con 0x55bcca0251b0 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.493+0000 7f8e62ffd640 1 -- 172.21.15.115:0/2878301464 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e78093040 con 0x55bcca0251b0 2024-05-09T05:56:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.494+0000 7f8e62ffd640 1 -- 172.21.15.115:0/2878301464 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78069e30 con 0x55bcca0251b0 2024-05-09T05:56:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.496+0000 7f8e85bc8740 1 -- 172.21.15.115:0/2878301464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 msgr2=0x55bcca3c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.496+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/2878301464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3c8840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e78052170 tx=0x7f8e78069110 comp rx=0 tx=0).stop 2024-05-09T05:56:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.498+0000 7f8e77fff640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e680b2a90 0x7f8e680b4fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55bcca3c9bf0 tx=0x7f8e64009190 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.503+0000 7f8e85bc8740 1 -- 172.21.15.115:0/2878301464 shutdown_connections 2024-05-09T05:56:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.503+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/2878301464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 0x55bcca3c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.503+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/2878301464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.503+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/2878301464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3c8840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.503+0000 7f8e85bc8740 1 -- 172.21.15.115:0/2878301464 >> 172.21.15.115:0/2878301464 conn(0x55bcca019300 msgr2=0x55bcca3c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.507+0000 7f8e85bc8740 1 -- 172.21.15.115:0/2878301464 shutdown_connections 2024-05-09T05:56:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.507+0000 7f8e85bc8740 1 -- 172.21.15.115:0/2878301464 wait complete. 2024-05-09T05:56:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.508+0000 7f8e85bc8740 1 -- start start 2024-05-09T05:56:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.508+0000 7f8e85bc8740 1 Processor -- start 2024-05-09T05:56:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.508+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.508+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0044d0 0x55bcca3d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.508+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0251b0 0x55bcca3d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.509+0000 7f8e85bc8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bcca3f7920 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.509+0000 7f8e85bc8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bcca2b49c0 con 0x55bcca0251b0 2024-05-09T05:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.509+0000 7f8e85bc8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcca3f7aa0 con 0x55bcca0044d0 2024-05-09T05:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.509+0000 7f8e85bc8740 10 client.0 _ll_register_callbacks cb 0x55bcca2e1020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.510+0000 7f8e7d2b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0251b0 0x55bcca3d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.510+0000 7f8e7d2b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0251b0 0x55bcca3d2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:45766/0 (socket says 172.21.15.115:45766) 2024-05-09T05:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.510+0000 7f8e7d2b1640 1 -- 172.21.15.115:0/1746634073 learned_addr learned my addr 172.21.15.115:0/1746634073 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.511+0000 7f8e77fff640 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0044d0 0x55bcca3d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.514+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.515+0000 7f8e7cab0640 1 -- 172.21.15.115:0/1746634073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0044d0 msgr2=0x55bcca3d4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.515+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0044d0 0x55bcca3d4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.515+0000 7f8e7cab0640 1 -- 172.21.15.115:0/1746634073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0251b0 msgr2=0x55bcca3d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.515+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0251b0 0x55bcca3d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.515+0000 7f8e7cab0640 1 -- 172.21.15.115:0/1746634073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcca2db1f0 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.515+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d4a50 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8e7808ffa0 tx=0x7f8e78068d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bf7031abc225e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.516+0000 7f8e57fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.516+0000 7f8e57fff640 1 -- 172.21.15.115:0/1746634073 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e780af060 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.516+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1746634073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e78093040 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.516+0000 7f8e57fff640 1 -- 172.21.15.115:0/1746634073 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f8e7806b260 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.517+0000 7f8e57fff640 1 -- 172.21.15.115:0/1746634073 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f8e780b6050 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.517+0000 7f8e85bc8740 10 client.18993 fetch_fsmap learned FSMap version 68 2024-05-09T05:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.517+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1746634073 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55bcca090790 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.518+0000 7f8e57fff640 1 -- 172.21.15.115:0/1746634073 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e78093040 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.519+0000 7f8e57fff640 1 -- 172.21.15.115:0/1746634073 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 68) ==== 8370+0+0 (secure 0 0 0) 0x7f8e780a2420 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.520+0000 7f8e85bc8740 10 client.18993 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T05:56:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.520+0000 7f8e85bc8740 10 client.18993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:56:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.520+0000 7f8e85bc8740 20 client.18993 populate_metadata read hostname 'smithi115' 2024-05-09T05:56:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.520+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bcc9fe3d80 0x55bcca407210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.520+0000 7f8e85bc8740 4 client.18993 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:56:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.520+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1746634073 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcca33b3d0 con 0x55bcc9fe3d80 2024-05-09T05:56:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.523+0000 7f8e77fff640 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bcc9fe3d80 0x55bcca407210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.528+0000 7f8e77fff640 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bcc9fe3d80 0x55bcca407210 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2291281a457e1cd server_cookie=89fab8b62f00dfaf in_seq=0 out_seq=0 2024-05-09T05:56:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.528+0000 7f8e57fff640 10 client.18993 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.530+0000 7f8e57fff640 1 -- 172.21.15.115:0/1746634073 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bcca33b3d0 con 0x55bcc9fe3d80 2024-05-09T05:56:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.530+0000 7f8e57fff640 10 client.18993 handle_command_reply: tid=0 2024-05-09T05:56:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.530+0000 7f8e85bc8740 1 client.18993 shutdown 2024-05-09T05:56:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.539+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1746634073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 msgr2=0x55bcca3d4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.539+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d4a50 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8e7808ffa0 tx=0x7f8e78068d90 comp rx=0 tx=0).stop 2024-05-09T05:56:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.539+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1746634073 shutdown_connections 2024-05-09T05:56:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.539+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0251b0 0x55bcca3d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.539+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bcc9fe3d80 0x55bcca407210 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.539+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0044d0 0x55bcca3d4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.539+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1746634073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d4a50 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.539+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1746634073 >> 172.21.15.115:0/1746634073 conn(0x55bcca019300 msgr2=0x55bcca3c3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.539+0000 7f8e57fff640 0 client.18993 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.540+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1746634073 shutdown_connections 2024-05-09T05:56:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.540+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1746634073 wait complete. 2024-05-09T05:56:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.540+0000 7f8e85bc8740 20 client.18993 trim_cache size 0 max 16384 2024-05-09T05:56:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.577+0000 7f8e85bc8740 1 Processor -- start 2024-05-09T05:56:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.577+0000 7f8e85bc8740 1 -- start start 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.577+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.577+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 0x55bcca3c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca40c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e7cab0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e7cab0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:45770/0 (socket says 172.21.15.115:45770) 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 0x55bcca3c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 0x55bcca3c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33648/0 (socket says 172.21.15.115:33648) 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e7cab0640 1 -- 172.21.15.115:0/1948870783 learned_addr learned my addr 172.21.15.115:0/1948870783 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e7d2b1640 1 --2- 172.21.15.115:0/1948870783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca40c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1948870783 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7000f650 con 0x55bcca0044d0 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1948870783 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8e700111c0 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1948870783 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7000fd20 con 0x55bcca0251b0 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e77fff640 1 -- 172.21.15.115:0/1948870783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 msgr2=0x55bcca40c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e77fff640 1 --2- 172.21.15.115:0/1948870783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca40c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e77fff640 1 -- 172.21.15.115:0/1948870783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 msgr2=0x55bcca3d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e77fff640 1 --2- 172.21.15.115:0/1948870783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.578+0000 7f8e77fff640 1 -- 172.21.15.115:0/1948870783 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcca1075f0 con 0x55bcca0044d0 2024-05-09T05:56:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.579+0000 7f8e77fff640 1 --2- 172.21.15.115:0/1948870783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 0x55bcca3c3f70 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x55bcca3c5f80 tx=0x7f8e64006a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115e4339c1a6703c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.579+0000 7f8e627fc640 1 -- 172.21.15.115:0/1948870783 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64019e10 con 0x55bcca0044d0 2024-05-09T05:56:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.579+0000 7f8e627fc640 1 -- 172.21.15.115:0/1948870783 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e64029030 con 0x55bcca0044d0 2024-05-09T05:56:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.579+0000 7f8e627fc640 1 -- 172.21.15.115:0/1948870783 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e640291e0 con 0x55bcca0044d0 2024-05-09T05:56:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.580+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1948870783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 msgr2=0x55bcca3c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.580+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1948870783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 0x55bcca3c3f70 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x55bcca3c5f80 tx=0x7f8e64006a00 comp rx=0 tx=0).stop 2024-05-09T05:56:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.580+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1948870783 shutdown_connections 2024-05-09T05:56:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.580+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1948870783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca40c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.580+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1948870783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcca0044d0 0x55bcca3c3f70 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.580+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/1948870783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.580+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1948870783 >> 172.21.15.115:0/1948870783 conn(0x55bcca019300 msgr2=0x55bcca3c3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.580+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1948870783 shutdown_connections 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.580+0000 7f8e85bc8740 1 -- 172.21.15.115:0/1948870783 wait complete. 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e85bc8740 1 -- start start 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e85bc8740 1 Processor -- start 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e85bc8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0044d0 0x55bcca3f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e7cab0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e7cab0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3f9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33654/0 (socket says 172.21.15.115:33654) 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e7cab0640 1 -- 172.21.15.115:0/797939969 learned_addr learned my addr 172.21.15.115:0/797939969 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70010af0 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.581+0000 7f8e77fff640 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0044d0 0x55bcca3f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e7d2b1640 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70010420 con 0x55bcca0044d0 2024-05-09T05:56:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8e700a6b10 con 0x55bcca0251b0 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e7cab0640 1 -- 172.21.15.115:0/797939969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 msgr2=0x55bcca3fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3fa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e7cab0640 1 -- 172.21.15.115:0/797939969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0044d0 msgr2=0x55bcca3f9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0044d0 0x55bcca3f9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e7cab0640 1 -- 172.21.15.115:0/797939969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcca2f1ed0 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e77fff640 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0044d0 0x55bcca3f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e85bc8740 10 client.0 _ll_register_callbacks cb 0x7f8e7808fdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e7cab0640 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3f9a00 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8e78051620 tx=0x7f8e780518c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a30735edd0d602a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e57fff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e57fff640 1 -- 172.21.15.115:0/797939969 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e780af040 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e57fff640 1 -- 172.21.15.115:0/797939969 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f8e780a2b30 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.582+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e64029030 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e57fff640 1 -- 172.21.15.115:0/797939969 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f8e78069ad0 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e85bc8740 10 client.18999 fetch_fsmap learned FSMap version 68 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({fsmap=68}) -- 0x55bcca46e4b0 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e57fff640 1 -- 172.21.15.115:0/797939969 <== mon.0 v2:172.21.15.115:3300/0 4 ==== fsmap(e 68) ==== 8370+0+0 (secure 0 0 0) 0x7f8e780b9030 con 0x55bcc9ffc2f0 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e85bc8740 10 client.18999 fetch_fsmap finished waiting for FSMap version 68 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e85bc8740 10 client.18999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e85bc8740 20 client.18999 populate_metadata read hostname 'smithi115' 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bcc9fb91c0 0x55bcca470b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e85bc8740 4 client.18999 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8e64029030 con 0x55bcc9fb91c0 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.583+0000 7f8e77fff640 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bcc9fb91c0 0x55bcca470b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.584+0000 7f8e57fff640 10 client.18999 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:39.584+0000 7f8e77fff640 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bcc9fb91c0 0x55bcca470b50 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9410e37ef14e2322 server_cookie=2e7fc08ed0778955 in_seq=0 out_seq=0 2024-05-09T05:56:39.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:39 smithi195 ceph-mon[33844]: pgmap v583: 129 pgs: 14 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.89k op/s 2024-05-09T05:56:39.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:39 smithi115 ceph-mon[30349]: pgmap v583: 129 pgs: 14 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.89k op/s 2024-05-09T05:56:39.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:39 smithi161 ceph-mon[33805]: pgmap v583: 129 pgs: 14 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.89k op/s 2024-05-09T05:56:41.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:41 smithi115 ceph-mon[30349]: pgmap v584: 129 pgs: 14 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 3.26k op/s 2024-05-09T05:56:41.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:41 smithi161 ceph-mon[33805]: pgmap v584: 129 pgs: 14 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 3.26k op/s 2024-05-09T05:56:42.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:41 smithi195 ceph-mon[33844]: pgmap v584: 129 pgs: 14 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 3.26k op/s 2024-05-09T05:56:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e57fff640 1 -- 172.21.15.115:0/797939969 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f8e64029030 con 0x55bcc9fb91c0 2024-05-09T05:56:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e57fff640 10 client.18999 handle_command_reply: tid=0 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 client.18999 shutdown 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 msgr2=0x55bcca3f9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3f9a00 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8e78051620 tx=0x7f8e780518c0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 shutdown_connections 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bcc9fb91c0 0x55bcca470b50 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcca0251b0 0x55bcca3fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bcca0044d0 0x55bcca3f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 --2- 172.21.15.115:0/797939969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc9ffc2f0 0x55bcca3f9a00 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 >> 172.21.15.115:0/797939969 conn(0x55bcca019300 msgr2=0x55bcca3d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e57fff640 0 client.18999 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 shutdown_connections 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 1 -- 172.21.15.115:0/797939969 wait complete. 2024-05-09T05:56:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.882+0000 7f8e85bc8740 20 client.18999 trim_cache size 0 max 16384 2024-05-09T05:56:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:56:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:56:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:56:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.883+0000 7f8e7ed14640 1 -- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e680b2a90 msgr2=0x7f8e680b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.883+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e680b2a90 0x7f8e680b4fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55bcca3c9bf0 tx=0x7f8e64009190 comp rx=0 tx=0).stop 2024-05-09T05:56:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.883+0000 7f8e7ed14640 1 -- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e700b8710 msgr2=0x7f8e700af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.883+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e700b8710 0x7f8e700af510 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00b3e0 tx=0x7f8e6c00b8a0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.883+0000 7f8e7ed14640 1 -- 172.21.15.115:0/3696094341 shutdown_connections 2024-05-09T05:56:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.883+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f8e680b2a90 0x7f8e680b4fa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.884+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e700b8710 0x7f8e700af510 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.884+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e700b5db0 0x7f8e700aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.884+0000 7f8e7ed14640 1 --2- 172.21.15.115:0/3696094341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e700a5840 0x7f8e700aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.884+0000 7f8e7ed14640 1 -- 172.21.15.115:0/3696094341 >> 172.21.15.115:0/3696094341 conn(0x7f8e7001af20 msgr2=0x7f8e700b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.884+0000 7f8e7ed14640 1 -- 172.21.15.115:0/3696094341 shutdown_connections 2024-05-09T05:56:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:42.884+0000 7f8e7ed14640 1 -- 172.21.15.115:0/3696094341 wait complete. 2024-05-09T05:56:43.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:43 smithi115 ceph-mon[30349]: pgmap v585: 129 pgs: 14 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-05-09T05:56:43.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:43 smithi161 ceph-mon[33805]: pgmap v585: 129 pgs: 14 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-05-09T05:56:44.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:43 smithi195 ceph-mon[33844]: pgmap v585: 129 pgs: 14 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-05-09T05:56:45.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:45 smithi115 ceph-mon[30349]: pgmap v586: 129 pgs: 13 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 90 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 3.23k op/s 2024-05-09T05:56:45.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:45 smithi161 ceph-mon[33805]: pgmap v586: 129 pgs: 13 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 90 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 3.23k op/s 2024-05-09T05:56:46.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:45 smithi195 ceph-mon[33844]: pgmap v586: 129 pgs: 13 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 90 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 3.23k op/s 2024-05-09T05:56:47.010 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:46 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:56:47.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:46 smithi195 ceph-mon[33844]: pgmap v587: 129 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 94 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.6 MiB/s wr, 2.44k op/s 2024-05-09T05:56:47.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:46 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:56:47.141 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:46 smithi115 ceph-mon[30349]: pgmap v587: 129 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 94 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.6 MiB/s wr, 2.44k op/s 2024-05-09T05:56:47.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:46 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:56:47.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:46 smithi161 ceph-mon[33805]: pgmap v587: 129 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 94 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.6 MiB/s wr, 2.44k op/s 2024-05-09T05:56:49.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:49 smithi115 ceph-mon[30349]: pgmap v588: 129 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 94 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-09T05:56:49.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:49 smithi161 ceph-mon[33805]: pgmap v588: 129 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 94 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-09T05:56:49.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:49 smithi195 ceph-mon[33844]: pgmap v588: 129 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 94 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-09T05:56:51.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:51 smithi115 ceph-mon[30349]: pgmap v589: 129 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 94 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:56:51.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:51 smithi161 ceph-mon[33805]: pgmap v589: 129 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 94 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:56:51.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:51 smithi195 ceph-mon[33844]: pgmap v589: 129 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 94 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-09T05:56:52.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:52 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:56:52.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:52 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:56:52.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:52 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:56:53.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:53 smithi115 ceph-mon[30349]: pgmap v590: 129 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 96 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.9 MiB/s wr, 2.02k op/s 2024-05-09T05:56:53.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:53 smithi195 ceph-mon[33844]: pgmap v590: 129 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 96 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.9 MiB/s wr, 2.02k op/s 2024-05-09T05:56:53.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:53 smithi161 ceph-mon[33805]: pgmap v590: 129 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 96 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.9 MiB/s wr, 2.02k op/s 2024-05-09T05:56:55.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:55 smithi195 ceph-mon[33844]: pgmap v591: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-09T05:56:55.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:55 smithi115 ceph-mon[30349]: pgmap v591: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-09T05:56:55.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:55 smithi161 ceph-mon[33805]: pgmap v591: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-09T05:56:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.094+0000 7ff38cd3b640 1 -- 172.21.15.115:0/4051429205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff38810a210 msgr2=0x7ff38810a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.094+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/4051429205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff38810a210 0x7ff38810a6a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff380009ca0 tx=0x7ff38002a8e0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.096+0000 7ff38cd3b640 1 -- 172.21.15.115:0/4051429205 shutdown_connections 2024-05-09T05:56:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.096+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/4051429205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff38810a210 0x7ff38810a6a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.096+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/4051429205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3881103f0 0x7ff388109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.096+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/4051429205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff38810fa10 0x7ff38810fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.096+0000 7ff38cd3b640 1 -- 172.21.15.115:0/4051429205 >> 172.21.15.115:0/4051429205 conn(0x7ff38806f810 msgr2=0x7ff38806fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.104+0000 7ff38cd3b640 1 -- 172.21.15.115:0/4051429205 shutdown_connections 2024-05-09T05:56:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.106+0000 7ff38cd3b640 1 -- 172.21.15.115:0/4051429205 wait complete. 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.107+0000 7ff38cd3b640 1 Processor -- start 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.107+0000 7ff38cd3b640 1 -- start start 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.108+0000 7ff38cd3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38810a210 0x7ff3881203a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.108+0000 7ff38cd3b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff38810fa10 0x7ff3881194c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.108+0000 7ff38cd3b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3881103f0 0x7ff388119b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.108+0000 7ff38cd3b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff388122960 con 0x7ff38810a210 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.108+0000 7ff38cd3b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881227e0 con 0x7ff38810fa10 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.108+0000 7ff38cd3b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff388122ae0 con 0x7ff3881103f0 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.109+0000 7ff3877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38810a210 0x7ff3881203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.109+0000 7ff3877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38810a210 0x7ff3881203a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47346/0 (socket says 172.21.15.115:47346) 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.109+0000 7ff3877fe640 1 -- 172.21.15.115:0/3660830781 learned_addr learned my addr 172.21.15.115:0/3660830781 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.109+0000 7ff387fff640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3881103f0 0x7ff388119b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.110+0000 7ff386ffd640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff38810fa10 0x7ff3881194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.110+0000 7ff387fff640 1 -- 172.21.15.115:0/3660830781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff38810fa10 msgr2=0x7ff3881194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.110+0000 7ff387fff640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff38810fa10 0x7ff3881194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.110+0000 7ff387fff640 1 -- 172.21.15.115:0/3660830781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38810a210 msgr2=0x7ff3881203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.110+0000 7ff387fff640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38810a210 0x7ff3881203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.110+0000 7ff387fff640 1 -- 172.21.15.115:0/3660830781 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38811dec0 con 0x7ff3881103f0 2024-05-09T05:56:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.112+0000 7ff387fff640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3881103f0 0x7ff388119b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff380005e30 tx=0x7ff380003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.112+0000 7ff384ff9640 1 -- 172.21.15.115:0/3660830781 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380003d20 con 0x7ff3881103f0 2024-05-09T05:56:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.112+0000 7ff38cd3b640 1 -- 172.21.15.115:0/3660830781 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38811e150 con 0x7ff3881103f0 2024-05-09T05:56:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.113+0000 7ff38cd3b640 1 -- 172.21.15.115:0/3660830781 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38811e6a0 con 0x7ff3881103f0 2024-05-09T05:56:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.113+0000 7ff384ff9640 1 -- 172.21.15.115:0/3660830781 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3800044d0 con 0x7ff3881103f0 2024-05-09T05:56:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.113+0000 7ff384ff9640 1 -- 172.21.15.115:0/3660830781 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380030bb0 con 0x7ff3881103f0 2024-05-09T05:56:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.120+0000 7ff394e51740 1 Processor -- start 2024-05-09T05:56:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.120+0000 7ff384ff9640 1 -- 172.21.15.115:0/3660830781 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff380030d50 con 0x7ff3881103f0 2024-05-09T05:56:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.121+0000 7ff384ff9640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3700b2b60 0x7ff3700b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.121+0000 7ff394e51740 1 -- start start 2024-05-09T05:56:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff384ff9640 1 -- 172.21.15.115:0/3660830781 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7ff3801039a0 con 0x7ff3881103f0 2024-05-09T05:56:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff394e51740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bcc41b0 0x559c1c067840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff394e51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc9b2f0 0x559c1c067d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff394e51740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 0x559c1c0682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff394e51740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c1bfcc7c0 con 0x559c1bcc41b0 2024-05-09T05:56:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff394e51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c1bf680e0 con 0x559c1bc9b2f0 2024-05-09T05:56:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff394e51740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559c1bfc8780 con 0x559c1bca34d0 2024-05-09T05:56:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff387fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 0x559c1c0682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff387fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 0x559c1c0682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:59350/0 (socket says 172.21.15.115:59350) 2024-05-09T05:56:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff387fff640 1 -- 172.21.15.115:0/1471104276 learned_addr learned my addr 172.21.15.115:0/1471104276 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.122+0000 7ff3877fe640 1 --2- 172.21.15.115:0/1471104276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bcc41b0 0x559c1c067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.123+0000 7ff387fff640 1 -- 172.21.15.115:0/1471104276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc9b2f0 msgr2=0x559c1c067d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:56:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.123+0000 7ff387fff640 1 --2- 172.21.15.115:0/1471104276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc9b2f0 0x559c1c067d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.123+0000 7ff387fff640 1 -- 172.21.15.115:0/1471104276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bcc41b0 msgr2=0x559c1c067840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.123+0000 7ff387fff640 1 --2- 172.21.15.115:0/1471104276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bcc41b0 0x559c1c067840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.123+0000 7ff387fff640 1 -- 172.21.15.115:0/1471104276 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c1bf917c0 con 0x559c1bca34d0 2024-05-09T05:56:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.124+0000 7ff387fff640 1 --2- 172.21.15.115:0/1471104276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 0x559c1c0682c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff3800ca640 tx=0x7ff3800ca350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1feecd5b6e1d915b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.125+0000 7ff36dffb640 1 -- 172.21.15.115:0/1471104276 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3800cb040 con 0x559c1bca34d0 2024-05-09T05:56:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.125+0000 7ff386ffd640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3700b2b60 0x7ff3700b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.125+0000 7ff36dffb640 1 -- 172.21.15.115:0/1471104276 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff380036040 con 0x559c1bca34d0 2024-05-09T05:56:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.125+0000 7ff36dffb640 1 -- 172.21.15.115:0/1471104276 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380100f20 con 0x559c1bca34d0 2024-05-09T05:56:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.126+0000 7ff394e51740 1 -- 172.21.15.115:0/1471104276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 msgr2=0x559c1c0682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.126+0000 7ff394e51740 1 --2- 172.21.15.115:0/1471104276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 0x559c1c0682c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff3800ca640 tx=0x7ff3800ca350 comp rx=0 tx=0).stop 2024-05-09T05:56:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.126+0000 7ff394e51740 1 -- 172.21.15.115:0/1471104276 shutdown_connections 2024-05-09T05:56:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.126+0000 7ff394e51740 1 --2- 172.21.15.115:0/1471104276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 0x559c1c0682c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.126+0000 7ff394e51740 1 --2- 172.21.15.115:0/1471104276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc9b2f0 0x559c1c067d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.127+0000 7ff394e51740 1 --2- 172.21.15.115:0/1471104276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bcc41b0 0x559c1c067840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.127+0000 7ff394e51740 1 -- 172.21.15.115:0/1471104276 >> 172.21.15.115:0/1471104276 conn(0x559c1bcb8300 msgr2=0x559c1c063ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.127+0000 7ff394e51740 1 -- 172.21.15.115:0/1471104276 shutdown_connections 2024-05-09T05:56:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.127+0000 7ff394e51740 1 -- 172.21.15.115:0/1471104276 wait complete. 2024-05-09T05:56:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.128+0000 7ff394e51740 1 -- start start 2024-05-09T05:56:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.128+0000 7ff394e51740 1 Processor -- start 2024-05-09T05:56:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.129+0000 7ff394e51740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bca34d0 0x559c1c0739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.129+0000 7ff394e51740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bcc41b0 0x559c1c073f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.129+0000 7ff394e51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.129+0000 7ff394e51740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559c1c074730 con 0x559c1bca34d0 2024-05-09T05:56:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.129+0000 7ff394e51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c1bf539c0 con 0x559c1bc82d80 2024-05-09T05:56:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.129+0000 7ff394e51740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559c1c0748b0 con 0x559c1bcc41b0 2024-05-09T05:56:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.130+0000 7ff387fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.130+0000 7ff387fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:34578/0 (socket says 172.21.15.115:34578) 2024-05-09T05:56:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.130+0000 7ff387fff640 1 -- 172.21.15.115:0/2014898971 learned_addr learned my addr 172.21.15.115:0/2014898971 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.130+0000 7ff394e51740 10 client.0 _ll_register_callbacks cb 0x559c1bf80020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:56:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.130+0000 7ff387fff640 1 -- 172.21.15.115:0/2014898971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bcc41b0 msgr2=0x559c1c073f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:56:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.130+0000 7ff387fff640 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bcc41b0 0x559c1c073f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.130+0000 7ff387fff640 1 -- 172.21.15.115:0/2014898971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bca34d0 msgr2=0x559c1c0739d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:56:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.130+0000 7ff387fff640 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bca34d0 0x559c1c0739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.130+0000 7ff387fff640 1 -- 172.21.15.115:0/2014898971 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c1bf7a1f0 con 0x559c1bc82d80 2024-05-09T05:56:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.131+0000 7ff387fff640 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c071de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff380031040 tx=0x7ff3801011d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452192f929eb347c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.132+0000 7ff386ffd640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3700b2b60 0x7ff3700b5070 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x559c1c068bf0 tx=0x7ff37c008040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.132+0000 7ff366ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:56:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.132+0000 7ff366ffd640 1 -- 172.21.15.115:0/2014898971 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3800cb040 con 0x559c1bc82d80 2024-05-09T05:56:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.132+0000 7ff394e51740 1 -- 172.21.15.115:0/2014898971 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff380036040 con 0x559c1bc82d80 2024-05-09T05:56:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.132+0000 7ff366ffd640 1 -- 172.21.15.115:0/2014898971 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7ff3800cadb0 con 0x559c1bc82d80 2024-05-09T05:56:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.138+0000 7ff366ffd640 1 -- 172.21.15.115:0/2014898971 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7ff3800ca940 con 0x559c1bc82d80 2024-05-09T05:56:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.138+0000 7ff394e51740 10 client.28190 fetch_fsmap learned FSMap version 70 2024-05-09T05:56:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.138+0000 7ff394e51740 1 -- 172.21.15.115:0/2014898971 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x559c1bd2f790 con 0x559c1bc82d80 2024-05-09T05:56:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.141+0000 7ff366ffd640 1 -- 172.21.15.115:0/2014898971 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff380036040 con 0x559c1bc82d80 2024-05-09T05:56:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.141+0000 7ff366ffd640 1 -- 172.21.15.115:0/2014898971 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 70) ==== 8382+0+0 (secure 0 0 0) 0x7ff380004b40 con 0x559c1bc82d80 2024-05-09T05:56:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.143+0000 7ff394e51740 10 client.28190 fetch_fsmap finished waiting for FSMap version 70 2024-05-09T05:56:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.143+0000 7ff394e51740 10 client.28190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:56:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.143+0000 7ff394e51740 20 client.28190 populate_metadata read hostname 'smithi115' 2024-05-09T05:56:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.143+0000 7ff394e51740 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c1bc581c0 0x559c1c083bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.143+0000 7ff394e51740 4 client.28190 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:56:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.143+0000 7ff394e51740 1 -- 172.21.15.115:0/2014898971 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c1bfda3d0 con 0x559c1bc581c0 2024-05-09T05:56:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.146+0000 7ff3877fe640 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c1bc581c0 0x559c1c083bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.157+0000 7ff3877fe640 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c1bc581c0 0x559c1c083bb0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994d82fe7a7281dc server_cookie=fd9a5b60cb64e34f in_seq=0 out_seq=0 2024-05-09T05:56:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.157+0000 7ff366ffd640 10 client.28190 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.161+0000 7ff366ffd640 1 -- 172.21.15.115:0/2014898971 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559c1bfda3d0 con 0x559c1bc581c0 2024-05-09T05:56:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.161+0000 7ff366ffd640 10 client.28190 handle_command_reply: tid=0 2024-05-09T05:56:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.161+0000 7ff394e51740 1 client.28190 shutdown 2024-05-09T05:56:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.162+0000 7ff394e51740 1 -- 172.21.15.115:0/2014898971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 msgr2=0x559c1c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.162+0000 7ff394e51740 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c071de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff380031040 tx=0x7ff3801011d0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.166+0000 7ff394e51740 1 -- 172.21.15.115:0/2014898971 shutdown_connections 2024-05-09T05:56:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.166+0000 7ff394e51740 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c071de0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.166+0000 7ff394e51740 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c1bc581c0 0x559c1c083bb0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.166+0000 7ff394e51740 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bcc41b0 0x559c1c073f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.166+0000 7ff394e51740 1 --2- 172.21.15.115:0/2014898971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bca34d0 0x559c1c0739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.166+0000 7ff394e51740 1 -- 172.21.15.115:0/2014898971 >> 172.21.15.115:0/2014898971 conn(0x559c1bcb8300 msgr2=0x559c1c062860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.167+0000 7ff366ffd640 0 client.28190 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.173+0000 7ff394e51740 1 -- 172.21.15.115:0/2014898971 shutdown_connections 2024-05-09T05:56:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.177+0000 7ff394e51740 1 -- 172.21.15.115:0/2014898971 wait complete. 2024-05-09T05:56:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.177+0000 7ff394e51740 20 client.28190 trim_cache size 0 max 16384 2024-05-09T05:56:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.207+0000 7ff394e51740 1 Processor -- start 2024-05-09T05:56:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.207+0000 7ff394e51740 1 -- start start 2024-05-09T05:56:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.208+0000 7ff394e51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c06bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.208+0000 7ff394e51740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 0x559c1c079b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.208+0000 7ff394e51740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bcc41b0 0x559c1c07a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.208+0000 7ff394e51740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff3740100d0 con 0x559c1bcc41b0 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.208+0000 7ff394e51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff374011c40 con 0x559c1bc82d80 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.208+0000 7ff394e51740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff3740107a0 con 0x559c1bca34d0 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.208+0000 7ff3877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c06bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.208+0000 7ff3877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c06bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:34582/0 (socket says 172.21.15.115:34582) 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.208+0000 7ff3877fe640 1 -- 172.21.15.115:0/3948827469 learned_addr learned my addr 172.21.15.115:0/3948827469 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.209+0000 7ff3877fe640 1 -- 172.21.15.115:0/3948827469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 msgr2=0x559c1c079b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.209+0000 7ff3877fe640 1 --2- 172.21.15.115:0/3948827469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 0x559c1c079b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.209+0000 7ff3877fe640 1 -- 172.21.15.115:0/3948827469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bcc41b0 msgr2=0x559c1c07a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.209+0000 7ff3877fe640 1 --2- 172.21.15.115:0/3948827469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bcc41b0 0x559c1c07a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.209+0000 7ff3877fe640 1 -- 172.21.15.115:0/3948827469 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c1c073d40 con 0x559c1bc82d80 2024-05-09T05:56:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.210+0000 7ff3877fe640 1 --2- 172.21.15.115:0/3948827469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c06bc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x559c1c0746a0 tx=0x7ff378010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1bb37db2db70065 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.214+0000 7ff36d7fa640 1 -- 172.21.15.115:0/3948827469 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378010e60 con 0x559c1bc82d80 2024-05-09T05:56:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.214+0000 7ff36d7fa640 1 -- 172.21.15.115:0/3948827469 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff378004250 con 0x559c1bc82d80 2024-05-09T05:56:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.214+0000 7ff36d7fa640 1 -- 172.21.15.115:0/3948827469 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37801c680 con 0x559c1bc82d80 2024-05-09T05:56:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.215+0000 7ff394e51740 1 -- 172.21.15.115:0/3948827469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 msgr2=0x559c1c06bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.215+0000 7ff394e51740 1 --2- 172.21.15.115:0/3948827469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c06bc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x559c1c0746a0 tx=0x7ff378010800 comp rx=0 tx=0).stop 2024-05-09T05:56:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.215+0000 7ff394e51740 1 -- 172.21.15.115:0/3948827469 shutdown_connections 2024-05-09T05:56:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.215+0000 7ff394e51740 1 --2- 172.21.15.115:0/3948827469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bcc41b0 0x559c1c07a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.215+0000 7ff394e51740 1 --2- 172.21.15.115:0/3948827469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bca34d0 0x559c1c079b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.215+0000 7ff394e51740 1 --2- 172.21.15.115:0/3948827469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bc82d80 0x559c1c06bc40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.215+0000 7ff394e51740 1 -- 172.21.15.115:0/3948827469 >> 172.21.15.115:0/3948827469 conn(0x559c1bcb8300 msgr2=0x559c1c06b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.216+0000 7ff394e51740 1 -- 172.21.15.115:0/3948827469 shutdown_connections 2024-05-09T05:56:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.216+0000 7ff394e51740 1 -- 172.21.15.115:0/3948827469 wait complete. 2024-05-09T05:56:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.217+0000 7ff394e51740 1 -- start start 2024-05-09T05:56:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.217+0000 7ff394e51740 1 Processor -- start 2024-05-09T05:56:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.217+0000 7ff394e51740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bc82d80 0x559c1c077400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff394e51740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bca34d0 0x559c1c077940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff394e51740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bcc41b0 0x559c1c077e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff394e51740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff374011570 con 0x559c1bca34d0 2024-05-09T05:56:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff394e51740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff374010ea0 con 0x559c1bcc41b0 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff394e51740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff37400e0d0 con 0x559c1bc82d80 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff394e51740 10 client.0 _ll_register_callbacks cb 0x7ff3800ca710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff387fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bcc41b0 0x559c1c077e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff387fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bcc41b0 0x559c1c077e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:34592/0 (socket says 172.21.15.115:34592) 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff387fff640 1 -- 172.21.15.115:0/4164139842 learned_addr learned my addr 172.21.15.115:0/4164139842 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff3877fe640 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bc82d80 0x559c1c077400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff387fff640 1 -- 172.21.15.115:0/4164139842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bc82d80 msgr2=0x559c1c077400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff387fff640 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bc82d80 0x559c1c077400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff387fff640 1 -- 172.21.15.115:0/4164139842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bca34d0 msgr2=0x559c1c077940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff387fff640 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bca34d0 0x559c1c077940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.218+0000 7ff387fff640 1 -- 172.21.15.115:0/4164139842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c1bda65f0 con 0x559c1bcc41b0 2024-05-09T05:56:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.219+0000 7ff387fff640 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bcc41b0 0x559c1c077e80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3800c7bb0 tx=0x7ff380100e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e699df8b65f2a5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.219+0000 7ff366ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:56:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.221+0000 7ff366ffd640 1 -- 172.21.15.115:0/4164139842 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380101130 con 0x559c1bcc41b0 2024-05-09T05:56:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.221+0000 7ff366ffd640 1 -- 172.21.15.115:0/4164139842 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7ff380004b40 con 0x559c1bcc41b0 2024-05-09T05:56:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.223+0000 7ff394e51740 1 -- 172.21.15.115:0/4164139842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff378004250 con 0x559c1bcc41b0 2024-05-09T05:56:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.224+0000 7ff366ffd640 1 -- 172.21.15.115:0/4164139842 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7ff3800cb040 con 0x559c1bcc41b0 2024-05-09T05:56:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.224+0000 7ff394e51740 10 client.28196 fetch_fsmap learned FSMap version 70 2024-05-09T05:56:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.224+0000 7ff394e51740 1 -- 172.21.15.115:0/4164139842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x559c1bf90ed0 con 0x559c1bcc41b0 2024-05-09T05:56:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.224+0000 7ff366ffd640 1 -- 172.21.15.115:0/4164139842 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff378004250 con 0x559c1bcc41b0 2024-05-09T05:56:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.224+0000 7ff366ffd640 1 -- 172.21.15.115:0/4164139842 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 70) ==== 8382+0+0 (secure 0 0 0) 0x7ff3800c62b0 con 0x559c1bcc41b0 2024-05-09T05:56:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.224+0000 7ff394e51740 10 client.28196 fetch_fsmap finished waiting for FSMap version 70 2024-05-09T05:56:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.224+0000 7ff394e51740 10 client.28196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:56:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.224+0000 7ff394e51740 20 client.28196 populate_metadata read hostname 'smithi115' 2024-05-09T05:56:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.224+0000 7ff394e51740 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c1c0ec010 0x559c1c0ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.225+0000 7ff394e51740 4 client.28196 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:56:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.225+0000 7ff394e51740 1 -- 172.21.15.115:0/4164139842 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c1bfd4240 con 0x559c1c0ec010 2024-05-09T05:56:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.226+0000 7ff3877fe640 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c1c0ec010 0x559c1c0ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.233+0000 7ff3877fe640 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c1c0ec010 0x559c1c0ee430 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da09ba8a0147fa08 server_cookie=f325858d7c33cef2 in_seq=0 out_seq=0 2024-05-09T05:56:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:56.233+0000 7ff366ffd640 10 client.28196 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:57.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:57 smithi195 ceph-mon[33844]: pgmap v592: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-09T05:56:57.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:57 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:56:57.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:57 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3321762598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:56:57.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:57 smithi115 ceph-mon[30349]: pgmap v592: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-09T05:56:57.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:57 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:56:57.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:57 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3321762598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:56:57.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:57 smithi161 ceph-mon[33805]: pgmap v592: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-09T05:56:57.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:57 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:56:57.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:57 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3321762598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:56:58.136 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:56:58.136 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:56:50.589592+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:56:58.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/4164560525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4108780 msgr2=0x7f2aa41100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/4164560525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4108780 0x7f2aa41100e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f2a940099c0 tx=0x7f2a94009cd0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/4164560525 shutdown_connections 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/4164560525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4108780 0x7f2aa41100e0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/4164560525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa4103550 0x7f2aa4103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/4164560525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aa4107e30 0x7f2aa4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/4164560525 >> 172.21.15.115:0/4164560525 conn(0x7f2aa4078e00 msgr2=0x7f2aa4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/4164560525 shutdown_connections 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/4164560525 wait complete. 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 Processor -- start 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.764+0000 7f2aa9ad2700 1 -- start start 2024-05-09T05:56:58.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa9ad2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4103550 0x7f2aa419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa9ad2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aa4107e30 0x7f2aa419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa9ad2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa4108780 0x7f2aa4199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa9ad2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4113260 con 0x7f2aa4103550 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa9ad2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41130e0 con 0x7f2aa4107e30 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa9ad2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41133e0 con 0x7f2aa4108780 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4103550 0x7f2aa419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4103550 0x7f2aa419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47404/0 (socket says 172.21.15.115:47404) 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa37fe700 1 -- 172.21.15.115:0/3321762598 learned_addr learned my addr 172.21.15.115:0/3321762598 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa2ffd700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aa4107e30 0x7f2aa419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa3fff700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa4108780 0x7f2aa4199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa37fe700 1 -- 172.21.15.115:0/3321762598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa4108780 msgr2=0x7f2aa4199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa37fe700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa4108780 0x7f2aa4199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa37fe700 1 -- 172.21.15.115:0/3321762598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aa4107e30 msgr2=0x7f2aa419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa37fe700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aa4107e30 0x7f2aa419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa37fe700 1 -- 172.21.15.115:0/3321762598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa419a0f0 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa37fe700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4103550 0x7f2aa419f820 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0060b0 tx=0x7f2a9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.765+0000 7f2aa0ff9700 1 -- 172.21.15.115:0/3321762598 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c01a070 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.766+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa419a3e0 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.766+0000 7f2aa0ff9700 1 -- 172.21.15.115:0/3321762598 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a9c00dd20 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.766+0000 7f2aa0ff9700 1 -- 172.21.15.115:0/3321762598 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c014600 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.766+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa41a66b0 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.767+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa40512c0 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.768+0000 7f2aa0ff9700 1 -- 172.21.15.115:0/3321762598 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f2a9c01d070 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.769+0000 7f2aa0ff9700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2a900b29f0 0x7f2a900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.770+0000 7f2aa0ff9700 1 -- 172.21.15.115:0/3321762598 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f2a9c0e23c0 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.771+0000 7f2aa2ffd700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2a900b29f0 0x7f2a900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.775+0000 7f2aa2ffd700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2a900b29f0 0x7f2a900b4f00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4108730 tx=0x7f2a8c0073d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:56.776+0000 7f2aa0ff9700 1 -- 172.21.15.115:0/3321762598 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a9c010070 con 0x7f2aa4103550 2024-05-09T05:56:58.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.179+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2aa419af70 con 0x7f2aa4103550 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.183+0000 7f2aa0ff9700 1 -- 172.21.15.115:0/3321762598 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+11959 (secure 0 0 0) 0x7f2a9c0a9360 con 0x7f2aa4103550 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.189+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2a900b29f0 msgr2=0x7f2a900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.189+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2a900b29f0 0x7f2a900b4f00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4108730 tx=0x7f2a8c0073d0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.189+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4103550 msgr2=0x7f2aa419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.189+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4103550 0x7f2aa419f820 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0060b0 tx=0x7f2a9c00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.193+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 shutdown_connections 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.193+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f2a900b29f0 0x7f2a900b4f00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.193+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa4108780 0x7f2aa4199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.193+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aa4103550 0x7f2aa419f820 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.193+0000 7f2aa9ad2700 1 --2- 172.21.15.115:0/3321762598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aa4107e30 0x7f2aa419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.193+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 >> 172.21.15.115:0/3321762598 conn(0x7f2aa4078e00 msgr2=0x7f2aa4100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:58.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.194+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 shutdown_connections 2024-05-09T05:56:58.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:56:57.194+0000 7f2aa9ad2700 1 -- 172.21.15.115:0/3321762598 wait complete. 2024-05-09T05:56:58.141 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 70 2024-05-09T05:56:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.148+0000 7ff366ffd640 1 -- 172.21.15.115:0/4164139842 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559c1bfd4240 con 0x559c1c0ec010 2024-05-09T05:56:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.148+0000 7ff366ffd640 10 client.28196 handle_command_reply: tid=0 2024-05-09T05:56:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.148+0000 7ff394e51740 1 client.28196 shutdown 2024-05-09T05:56:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.153+0000 7ff394e51740 1 -- 172.21.15.115:0/4164139842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bcc41b0 msgr2=0x559c1c077e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.153+0000 7ff394e51740 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bcc41b0 0x559c1c077e80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3800c7bb0 tx=0x7ff380100e40 comp rx=0 tx=0).stop 2024-05-09T05:56:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.153+0000 7ff394e51740 1 -- 172.21.15.115:0/4164139842 shutdown_connections 2024-05-09T05:56:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.153+0000 7ff394e51740 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c1bcc41b0 0x559c1c077e80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.153+0000 7ff394e51740 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559c1bca34d0 0x559c1c077940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.153+0000 7ff394e51740 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x559c1c0ec010 0x559c1c0ee430 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.153+0000 7ff394e51740 1 --2- 172.21.15.115:0/4164139842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559c1bc82d80 0x559c1c077400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.153+0000 7ff394e51740 1 -- 172.21.15.115:0/4164139842 >> 172.21.15.115:0/4164139842 conn(0x559c1bcb8300 msgr2=0x559c1c062860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.153+0000 7ff366ffd640 0 client.28196 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:56:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.156+0000 7ff394e51740 1 -- 172.21.15.115:0/4164139842 shutdown_connections 2024-05-09T05:56:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.156+0000 7ff394e51740 1 -- 172.21.15.115:0/4164139842 wait complete. 2024-05-09T05:56:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.156+0000 7ff394e51740 20 client.28196 trim_cache size 0 max 16384 2024-05-09T05:56:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:56:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:56:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:56:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.163+0000 7ff38cd3b640 1 -- 172.21.15.115:0/3660830781 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3700b2b60 msgr2=0x7ff3700b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.163+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3700b2b60 0x7ff3700b5070 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x559c1c068bf0 tx=0x7ff37c008040 comp rx=0 tx=0).stop 2024-05-09T05:56:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.163+0000 7ff38cd3b640 1 -- 172.21.15.115:0/3660830781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3881103f0 msgr2=0x7ff388119b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:56:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.163+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3881103f0 0x7ff388119b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff380005e30 tx=0x7ff380003970 comp rx=0 tx=0).stop 2024-05-09T05:56:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.169+0000 7ff38cd3b640 1 -- 172.21.15.115:0/3660830781 shutdown_connections 2024-05-09T05:56:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.169+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3700b2b60 0x7ff3700b5070 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.169+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3881103f0 0x7ff388119b90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.169+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff38810fa10 0x7ff3881194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.169+0000 7ff38cd3b640 1 --2- 172.21.15.115:0/3660830781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38810a210 0x7ff3881203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:56:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.169+0000 7ff38cd3b640 1 -- 172.21.15.115:0/3660830781 >> 172.21.15.115:0/3660830781 conn(0x7ff38806f810 msgr2=0x7ff38810d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:56:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.182+0000 7ff38cd3b640 1 -- 172.21.15.115:0/3660830781 shutdown_connections 2024-05-09T05:56:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:56:58.182+0000 7ff38cd3b640 1 -- 172.21.15.115:0/3660830781 wait complete. 2024-05-09T05:56:59.445 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:56:59 smithi115 ceph-mon[30349]: pgmap v593: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.2 MiB/s wr, 2.07k op/s 2024-05-09T05:56:59.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:56:59 smithi195 ceph-mon[33844]: pgmap v593: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.2 MiB/s wr, 2.07k op/s 2024-05-09T05:56:59.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:56:59 smithi161 ceph-mon[33805]: pgmap v593: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.2 MiB/s wr, 2.07k op/s 2024-05-09T05:57:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:01 smithi115 ceph-mon[30349]: pgmap v594: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-05-09T05:57:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:01 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:57:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:01 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:57:01.735 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:01 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:57:01.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:01 smithi195 ceph-mon[33844]: pgmap v594: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-05-09T05:57:01.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:01 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:57:01.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:01 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:57:01.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:01 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:57:01.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:01 smithi161 ceph-mon[33805]: pgmap v594: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-05-09T05:57:01.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:01 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:57:01.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:01 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:57:01.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:01 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:57:03.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:03 smithi195 ceph-mon[33844]: pgmap v595: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 2.25k op/s 2024-05-09T05:57:03.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:03 smithi115 ceph-mon[30349]: pgmap v595: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 2.25k op/s 2024-05-09T05:57:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:03 smithi161 ceph-mon[33805]: pgmap v595: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 2.25k op/s 2024-05-09T05:57:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:05 smithi195 ceph-mon[33844]: pgmap v596: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2024-05-09T05:57:05.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:05 smithi115 ceph-mon[30349]: pgmap v596: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2024-05-09T05:57:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:05 smithi161 ceph-mon[33805]: pgmap v596: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 2.73k op/s 2024-05-09T05:57:07.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:07 smithi195 ceph-mon[33844]: pgmap v597: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 MiB/s wr, 2.22k op/s 2024-05-09T05:57:07.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:07 smithi115 ceph-mon[30349]: pgmap v597: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 MiB/s wr, 2.22k op/s 2024-05-09T05:57:07.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:07 smithi161 ceph-mon[33805]: pgmap v597: 129 pgs: 9 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 110 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 MiB/s wr, 2.22k op/s 2024-05-09T05:57:08.286 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 141.3 --await-for 45.0' 2024-05-09T05:57:08.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 141.3 --await-for 45.0 2024-05-09T05:57:08.756 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:57:09.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:09 smithi195 ceph-mon[33844]: pgmap v598: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-05-09T05:57:09.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:09 smithi115 ceph-mon[30349]: pgmap v598: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-05-09T05:57:09.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:09 smithi161 ceph-mon[33805]: pgmap v598: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-05-09T05:57:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 -- 172.21.15.115:0/487544638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40501075e0 msgr2=0x7f405010ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 --2- 172.21.15.115:0/487544638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40501075e0 0x7f405010ed80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4048009ca0 tx=0x7f404802a8c0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 -- 172.21.15.115:0/487544638 shutdown_connections 2024-05-09T05:57:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 --2- 172.21.15.115:0/487544638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40501075e0 0x7f405010ed80 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 --2- 172.21.15.115:0/487544638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4050106b20 0x7f4050106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 --2- 172.21.15.115:0/487544638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4050102590 0x7f40501029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 -- 172.21.15.115:0/487544638 >> 172.21.15.115:0/487544638 conn(0x7f40500fd9d0 msgr2=0x7f40500ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 -- 172.21.15.115:0/487544638 shutdown_connections 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 -- 172.21.15.115:0/487544638 wait complete. 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 Processor -- start 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.350+0000 7f4057861640 1 -- start start 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4057861640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4050102590 0x7f4050198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4057861640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4050106b20 0x7f4050198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4057861640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40501075e0 0x7f405019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4057861640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4050111b70 con 0x7f4050106b20 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4057861640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f40501119f0 con 0x7f40501075e0 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4057861640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4050111cf0 con 0x7f4050102590 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4054dfc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4050106b20 0x7f4050198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f40555fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4050102590 0x7f4050198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4054dfc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4050106b20 0x7f4050198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41322/0 (socket says 172.21.15.115:41322) 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4054dfc640 1 -- 172.21.15.115:0/1651930751 learned_addr learned my addr 172.21.15.115:0/1651930751 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4055dfe640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40501075e0 0x7f405019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4054dfc640 1 -- 172.21.15.115:0/1651930751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4050102590 msgr2=0x7f4050198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4054dfc640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4050102590 0x7f4050198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4054dfc640 1 -- 172.21.15.115:0/1651930751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40501075e0 msgr2=0x7f405019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4054dfc640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40501075e0 0x7f405019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4054dfc640 1 -- 172.21.15.115:0/1651930751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405019d470 con 0x7f4050106b20 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4055dfe640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40501075e0 0x7f405019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f4054dfc640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4050106b20 0x7f4050198b70 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f404000e9c0 tx=0x7f404000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f40467fc640 1 -- 172.21.15.115:0/1651930751 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404000cd80 con 0x7f4050106b20 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.351+0000 7f40467fc640 1 -- 172.21.15.115:0/1651930751 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40400054b0 con 0x7f4050106b20 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.352+0000 7f4057861640 1 -- 172.21.15.115:0/1651930751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405019d760 con 0x7f4050106b20 2024-05-09T05:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.352+0000 7f40467fc640 1 -- 172.21.15.115:0/1651930751 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040010430 con 0x7f4050106b20 2024-05-09T05:57:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.352+0000 7f4057861640 1 -- 172.21.15.115:0/1651930751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40501a50e0 con 0x7f4050106b20 2024-05-09T05:57:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.354+0000 7f405e7ec740 1 Processor -- start 2024-05-09T05:57:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.354+0000 7f405e7ec740 1 -- start start 2024-05-09T05:57:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.354+0000 7f40467fc640 1 -- 172.21.15.115:0/1651930751 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4040004f20 con 0x7f4050106b20 2024-05-09T05:57:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.354+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1fb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde354d0 0x55b4ce1fc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f405e7ec740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce15e7c0 con 0x55b4cde354d0 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f405e7ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce0fa0e0 con 0x55b4cde561b0 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f405e7ec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce15a780 con 0x55b4cde2d2f0 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4055dfe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4054dfc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde354d0 0x55b4ce1fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4055dfe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1fb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:56856/0 (socket says 172.21.15.115:56856) 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4055dfe640 1 -- 172.21.15.115:0/2767896216 learned_addr learned my addr 172.21.15.115:0/2767896216 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f40555fd640 1 --2- 172.21.15.115:0/2767896216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4055dfe640 1 -- 172.21.15.115:0/2767896216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 msgr2=0x55b4ce1fbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4055dfe640 1 --2- 172.21.15.115:0/2767896216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4055dfe640 1 -- 172.21.15.115:0/2767896216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde354d0 msgr2=0x55b4ce1fc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4055dfe640 1 --2- 172.21.15.115:0/2767896216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde354d0 0x55b4ce1fc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4055dfe640 1 -- 172.21.15.115:0/2767896216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ce1237c0 con 0x55b4cde561b0 2024-05-09T05:57:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f4055dfe640 1 --2- 172.21.15.115:0/2767896216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1fb890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40480094b0 tx=0x7f4048009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757fdb66643c9fe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f40467fc640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f403c0b2950 0x7f403c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.355+0000 7f403b7fe640 1 -- 172.21.15.115:0/2767896216 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048031980 con 0x55b4cde561b0 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f40555fd640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f403c0b2950 0x7f403c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f403b7fe640 1 -- 172.21.15.115:0/2767896216 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4048036040 con 0x55b4cde561b0 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f403b7fe640 1 -- 172.21.15.115:0/2767896216 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048004370 con 0x55b4cde561b0 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f40467fc640 1 -- 172.21.15.115:0/1651930751 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f40400e2fd0 con 0x7f4050106b20 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f405e7ec740 1 -- 172.21.15.115:0/2767896216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 msgr2=0x55b4ce1fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f405e7ec740 1 --2- 172.21.15.115:0/2767896216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1fb890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40480094b0 tx=0x7f4048009a20 comp rx=0 tx=0).stop 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f40555fd640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f403c0b2950 0x7f403c0b4e60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55b4ce1fcc40 tx=0x7f404c00c4a0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f405e7ec740 1 -- 172.21.15.115:0/2767896216 shutdown_connections 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f405e7ec740 1 --2- 172.21.15.115:0/2767896216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde354d0 0x55b4ce1fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f405e7ec740 1 --2- 172.21.15.115:0/2767896216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.356+0000 7f405e7ec740 1 --2- 172.21.15.115:0/2767896216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1fb890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 -- 172.21.15.115:0/2767896216 >> 172.21.15.115:0/2767896216 conn(0x55b4cde4a300 msgr2=0x55b4ce1f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 -- 172.21.15.115:0/2767896216 shutdown_connections 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 -- 172.21.15.115:0/2767896216 wait complete. 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 -- start start 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 Processor -- start 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce207950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde354d0 0x55b4ce207e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde561b0 0x55b4ce205e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce208960 con 0x55b4cde561b0 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce0e59c0 con 0x55b4cde354d0 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce208ae0 con 0x55b4cde2d2f0 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f405e7ec740 10 client.0 _ll_register_callbacks cb 0x55b4ce112020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f4055dfe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde561b0 0x55b4ce205e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f40555fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce207950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f4055dfe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde561b0 0x55b4ce205e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41352/0 (socket says 172.21.15.115:41352) 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f40555fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce207950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:39394/0 (socket says 172.21.15.115:39394) 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.357+0000 7f4055dfe640 1 -- 172.21.15.115:0/3348917751 learned_addr learned my addr 172.21.15.115:0/3348917751 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4054dfc640 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde354d0 0x55b4ce207e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4055dfe640 1 -- 172.21.15.115:0/3348917751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 msgr2=0x55b4ce207950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4055dfe640 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce207950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4055dfe640 1 -- 172.21.15.115:0/3348917751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde354d0 msgr2=0x55b4ce207e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4055dfe640 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde354d0 0x55b4ce207e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4055dfe640 1 -- 172.21.15.115:0/3348917751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ce10c1f0 con 0x55b4cde561b0 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f40555fd640 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce207950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4055dfe640 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde561b0 0x55b4ce205e60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f404802a900 tx=0x7f40480315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6eb16d38ca692a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:57:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4038ff9640 1 -- 172.21.15.115:0/3348917751 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048052020 con 0x55b4cde561b0 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4038ff9640 1 -- 172.21.15.115:0/3348917751 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f4048003b70 con 0x55b4cde561b0 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f405e7ec740 1 -- 172.21.15.115:0/3348917751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4048036040 con 0x55b4cde561b0 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4038ff9640 1 -- 172.21.15.115:0/3348917751 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f404805d050 con 0x55b4cde561b0 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f405e7ec740 10 client.19023 fetch_fsmap learned FSMap version 70 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f405e7ec740 1 -- 172.21.15.115:0/3348917751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55b4cdec1790 con 0x55b4cde561b0 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.358+0000 7f4038ff9640 1 -- 172.21.15.115:0/3348917751 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4048036040 con 0x55b4cde561b0 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f4038ff9640 1 -- 172.21.15.115:0/3348917751 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 70) ==== 8382+0+0 (secure 0 0 0) 0x7f404804d030 con 0x55b4cde561b0 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f405e7ec740 10 client.19023 fetch_fsmap finished waiting for FSMap version 70 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f405e7ec740 10 client.19023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f405e7ec740 20 client.19023 populate_metadata read hostname 'smithi115' 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f405e7ec740 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b4cde14d80 0x55b4ce218040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f405e7ec740 4 client.19023 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f405e7ec740 1 -- 172.21.15.115:0/3348917751 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4ce16c3d0 con 0x55b4cde14d80 2024-05-09T05:57:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f40555fd640 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b4cde14d80 0x55b4ce218040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f4038ff9640 10 client.19023 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.359+0000 7f40555fd640 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b4cde14d80 0x55b4ce218040 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57162c36850a0b2e server_cookie=a45b09c118514a91 in_seq=0 out_seq=0 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f4038ff9640 1 -- 172.21.15.115:0/3348917751 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b4ce16c3d0 con 0x55b4cde14d80 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f4038ff9640 10 client.19023 handle_command_reply: tid=0 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 client.19023 shutdown 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 -- 172.21.15.115:0/3348917751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde561b0 msgr2=0x55b4ce205e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde561b0 0x55b4ce205e60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f404802a900 tx=0x7f40480315d0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 -- 172.21.15.115:0/3348917751 shutdown_connections 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde561b0 0x55b4ce205e60 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde354d0 0x55b4ce207e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b4cde14d80 0x55b4ce218040 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 --2- 172.21.15.115:0/3348917751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde2d2f0 0x55b4ce207950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f4038ff9640 0 client.19023 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 -- 172.21.15.115:0/3348917751 >> 172.21.15.115:0/3348917751 conn(0x55b4cde4a300 msgr2=0x55b4ce1fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.360+0000 7f405e7ec740 1 -- 172.21.15.115:0/3348917751 shutdown_connections 2024-05-09T05:57:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.361+0000 7f405e7ec740 1 -- 172.21.15.115:0/3348917751 wait complete. 2024-05-09T05:57:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.361+0000 7f405e7ec740 20 client.19023 trim_cache size 0 max 16384 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.375+0000 7f405e7ec740 1 Processor -- start 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f405e7ec740 1 -- start start 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 0x55b4ce21b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1ff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f405e7ec740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40501af130 con 0x55b4cde2d2f0 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f405e7ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f40501b0ca0 con 0x55b4cde561b0 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f405e7ec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40501af800 con 0x55b4cde354d0 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f40555fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f40555fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41368/0 (socket says 172.21.15.115:41368) 2024-05-09T05:57:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f40555fd640 1 -- 172.21.15.115:0/2109269383 learned_addr learned my addr 172.21.15.115:0/2109269383 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f4055dfe640 1 --2- 172.21.15.115:0/2109269383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f4054dfc640 1 --2- 172.21.15.115:0/2109269383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 0x55b4ce21b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f40555fd640 1 -- 172.21.15.115:0/2109269383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 msgr2=0x55b4ce21b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f40555fd640 1 --2- 172.21.15.115:0/2109269383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 0x55b4ce21b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f40555fd640 1 -- 172.21.15.115:0/2109269383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 msgr2=0x55b4ce1ff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f40555fd640 1 --2- 172.21.15.115:0/2109269383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1ff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f40555fd640 1 -- 172.21.15.115:0/2109269383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4cdf385f0 con 0x55b4cde2d2f0 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f40555fd640 1 --2- 172.21.15.115:0/2109269383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fdc80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f404c018580 tx=0x7f404c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3374636fee87c674 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f403affd640 1 -- 172.21.15.115:0/2109269383 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c005020 con 0x55b4cde2d2f0 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f403affd640 1 -- 172.21.15.115:0/2109269383 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f404c005650 con 0x55b4cde2d2f0 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.376+0000 7f403affd640 1 -- 172.21.15.115:0/2109269383 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c02a200 con 0x55b4cde2d2f0 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.377+0000 7f405e7ec740 1 -- 172.21.15.115:0/2109269383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 msgr2=0x55b4ce1fdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.377+0000 7f405e7ec740 1 --2- 172.21.15.115:0/2109269383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fdc80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f404c018580 tx=0x7f404c015000 comp rx=0 tx=0).stop 2024-05-09T05:57:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.377+0000 7f405e7ec740 1 -- 172.21.15.115:0/2109269383 shutdown_connections 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.377+0000 7f405e7ec740 1 --2- 172.21.15.115:0/2109269383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.377+0000 7f405e7ec740 1 --2- 172.21.15.115:0/2109269383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 0x55b4ce21b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.377+0000 7f405e7ec740 1 --2- 172.21.15.115:0/2109269383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce1fdc80 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.377+0000 7f405e7ec740 1 -- 172.21.15.115:0/2109269383 >> 172.21.15.115:0/2109269383 conn(0x55b4cde4a300 msgr2=0x55b4ce1fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.377+0000 7f405e7ec740 1 -- 172.21.15.115:0/2109269383 shutdown_connections 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.377+0000 7f405e7ec740 1 -- 172.21.15.115:0/2109269383 wait complete. 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f405e7ec740 1 -- start start 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f405e7ec740 1 Processor -- start 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce279470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 0x55b4ce2799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f405e7ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f405e7ec740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f40501b05d0 con 0x55b4cde2d2f0 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f405e7ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f40501aff00 con 0x55b4cde561b0 2024-05-09T05:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f405e7ec740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40501ad130 con 0x55b4cde354d0 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f405e7ec740 10 client.0 _ll_register_callbacks cb 0x7f40400e69f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f40555fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce279470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f40555fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce279470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41376/0 (socket says 172.21.15.115:41376) 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f40555fd640 1 -- 172.21.15.115:0/390371122 learned_addr learned my addr 172.21.15.115:0/390371122 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f4054dfc640 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 0x55b4ce2799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f4055dfe640 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f40555fd640 1 -- 172.21.15.115:0/390371122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 msgr2=0x55b4ce2799b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f40555fd640 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 0x55b4ce2799b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f40555fd640 1 -- 172.21.15.115:0/390371122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 msgr2=0x55b4ce1f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f40555fd640 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f40555fd640 1 -- 172.21.15.115:0/390371122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ce122ed0 con 0x55b4cde2d2f0 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f40555fd640 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce279470 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f404c004540 tx=0x7f404c0193b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fce828dde9d405 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.378+0000 7f4038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:57:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f4038ff9640 1 -- 172.21.15.115:0/390371122 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c018a40 con 0x55b4cde2d2f0 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f4038ff9640 1 -- 172.21.15.115:0/390371122 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f404c018be0 con 0x55b4cde2d2f0 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f405e7ec740 1 -- 172.21.15.115:0/390371122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f404c005650 con 0x55b4cde2d2f0 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f4038ff9640 1 -- 172.21.15.115:0/390371122 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f404c047050 con 0x55b4cde2d2f0 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f405e7ec740 10 client.19029 fetch_fsmap learned FSMap version 70 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f405e7ec740 1 -- 172.21.15.115:0/390371122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55b4ce2819f0 con 0x55b4cde2d2f0 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f4038ff9640 1 -- 172.21.15.115:0/390371122 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f404c005650 con 0x55b4cde2d2f0 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f4038ff9640 1 -- 172.21.15.115:0/390371122 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 70) ==== 8382+0+0 (secure 0 0 0) 0x7f404c00b040 con 0x55b4cde2d2f0 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f405e7ec740 10 client.19029 fetch_fsmap finished waiting for FSMap version 70 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f405e7ec740 10 client.19029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f405e7ec740 20 client.19029 populate_metadata read hostname 'smithi115' 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f405e7ec740 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b4cddea1c0 0x55b4ce284150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f405e7ec740 4 client.19029 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f405e7ec740 1 -- 172.21.15.115:0/390371122 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4ce166240 con 0x55b4cddea1c0 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.379+0000 7f4054dfc640 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b4cddea1c0 0x55b4ce284150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.380+0000 7f4038ff9640 10 client.19029 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:11.380+0000 7f4054dfc640 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b4cddea1c0 0x55b4ce284150 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935f5782e359b794 server_cookie=d3bec363d3241098 in_seq=0 out_seq=0 2024-05-09T05:57:11.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:11 smithi195 ceph-mon[33844]: pgmap v599: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-05-09T05:57:11.892 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:11 smithi115 ceph-mon[30349]: pgmap v599: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-05-09T05:57:11.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:11 smithi161 ceph-mon[33805]: pgmap v599: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-05-09T05:57:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.413+0000 7f4038ff9640 1 -- 172.21.15.115:0/390371122 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b4ce166240 con 0x55b4cddea1c0 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.413+0000 7f4038ff9640 10 client.19029 handle_command_reply: tid=0 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.413+0000 7f405e7ec740 1 client.19029 shutdown 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.414+0000 7f405e7ec740 1 -- 172.21.15.115:0/390371122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 msgr2=0x55b4ce279470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.414+0000 7f405e7ec740 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce279470 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f404c004540 tx=0x7f404c0193b0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.414+0000 7f405e7ec740 1 -- 172.21.15.115:0/390371122 shutdown_connections 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.414+0000 7f405e7ec740 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4cde561b0 0x55b4ce1f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.414+0000 7f405e7ec740 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b4cddea1c0 0x55b4ce284150 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.414+0000 7f405e7ec740 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b4cde354d0 0x55b4ce2799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.414+0000 7f405e7ec740 1 --2- 172.21.15.115:0/390371122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b4cde2d2f0 0x55b4ce279470 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.414+0000 7f405e7ec740 1 -- 172.21.15.115:0/390371122 >> 172.21.15.115:0/390371122 conn(0x55b4cde4a300 msgr2=0x55b4ce1fb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.414+0000 7f4038ff9640 0 client.19029 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.418+0000 7f405e7ec740 1 -- 172.21.15.115:0/390371122 shutdown_connections 2024-05-09T05:57:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.418+0000 7f405e7ec740 1 -- 172.21.15.115:0/390371122 wait complete. 2024-05-09T05:57:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.420+0000 7f405e7ec740 20 client.19029 trim_cache size 0 max 16384 2024-05-09T05:57:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:57:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:57:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.423+0000 7f4057861640 1 -- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f403c0b2950 msgr2=0x7f403c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.423+0000 7f4057861640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f403c0b2950 0x7f403c0b4e60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55b4ce1fcc40 tx=0x7f404c00c4a0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.423+0000 7f4057861640 1 -- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4050106b20 msgr2=0x7f4050198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.423+0000 7f4057861640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4050106b20 0x7f4050198b70 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f404000e9c0 tx=0x7f404000ee80 comp rx=0 tx=0).stop 2024-05-09T05:57:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.430+0000 7f4057861640 1 -- 172.21.15.115:0/1651930751 shutdown_connections 2024-05-09T05:57:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.430+0000 7f4057861640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f403c0b2950 0x7f403c0b4e60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.430+0000 7f4057861640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40501075e0 0x7f405019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.430+0000 7f4057861640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4050106b20 0x7f4050198b70 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.430+0000 7f4057861640 1 --2- 172.21.15.115:0/1651930751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4050102590 0x7f4050198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.430+0000 7f4057861640 1 -- 172.21.15.115:0/1651930751 >> 172.21.15.115:0/1651930751 conn(0x7f40500fd9d0 msgr2=0x7f4050104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.430+0000 7f4057861640 1 -- 172.21.15.115:0/1651930751 shutdown_connections 2024-05-09T05:57:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:12.430+0000 7f4057861640 1 -- 172.21.15.115:0/1651930751 wait complete. 2024-05-09T05:57:13.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:13 smithi195 ceph-mon[33844]: pgmap v600: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.6 MiB/s wr, 2.18k op/s 2024-05-09T05:57:13.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:13 smithi115 ceph-mon[30349]: pgmap v600: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.6 MiB/s wr, 2.18k op/s 2024-05-09T05:57:13.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:13 smithi161 ceph-mon[33805]: pgmap v600: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.6 MiB/s wr, 2.18k op/s 2024-05-09T05:57:15.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:15 smithi195 ceph-mon[33844]: pgmap v601: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-09T05:57:15.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:15 smithi115 ceph-mon[30349]: pgmap v601: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-09T05:57:15.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:15 smithi161 ceph-mon[33805]: pgmap v601: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-09T05:57:17.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:17 smithi195 ceph-mon[33844]: pgmap v602: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 2.01k op/s 2024-05-09T05:57:17.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:17 smithi115 ceph-mon[30349]: pgmap v602: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 2.01k op/s 2024-05-09T05:57:17.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:17 smithi161 ceph-mon[33805]: pgmap v602: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 2.01k op/s 2024-05-09T05:57:19.862 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:19 smithi195 ceph-mon[33844]: pgmap v603: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 2.12k op/s 2024-05-09T05:57:19.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:19 smithi115 ceph-mon[30349]: pgmap v603: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 2.12k op/s 2024-05-09T05:57:19.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:19 smithi161 ceph-mon[33805]: pgmap v603: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 2.12k op/s 2024-05-09T05:57:20.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:20 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:57:20.878 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:20 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:57:20.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:20 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:57:21.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:21 smithi195 ceph-mon[33844]: pgmap v604: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-09T05:57:21.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:21 smithi115 ceph-mon[30349]: pgmap v604: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-09T05:57:21.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:21 smithi161 ceph-mon[33805]: pgmap v604: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-09T05:57:23.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:23 smithi195 ceph-mon[33844]: pgmap v605: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.2 MiB/s wr, 2.07k op/s 2024-05-09T05:57:23.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:23 smithi115 ceph-mon[30349]: pgmap v605: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.2 MiB/s wr, 2.07k op/s 2024-05-09T05:57:23.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:23 smithi161 ceph-mon[33805]: pgmap v605: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.2 MiB/s wr, 2.07k op/s 2024-05-09T05:57:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.674+0000 7fcc587b2640 1 -- 172.21.15.115:0/2371202724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5011a8f0 msgr2=0x7fcc5011cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.674+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2371202724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5011a8f0 0x7fcc5011cd10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009be0 tx=0x7fcc4c02a750 comp rx=0 tx=0).stop 2024-05-09T05:57:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.677+0000 7fcc587b2640 1 -- 172.21.15.115:0/2371202724 shutdown_connections 2024-05-09T05:57:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.677+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2371202724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc5011d250 0x7fcc5011f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.677+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2371202724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5011a8f0 0x7fcc5011cd10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.677+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2371202724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc5010a200 0x7fcc5010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.677+0000 7fcc587b2640 1 -- 172.21.15.115:0/2371202724 >> 172.21.15.115:0/2371202724 conn(0x7fcc5006e900 msgr2=0x7fcc5006ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.683+0000 7fcc587b2640 1 -- 172.21.15.115:0/2371202724 shutdown_connections 2024-05-09T05:57:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.684+0000 7fcc587b2640 1 -- 172.21.15.115:0/2371202724 wait complete. 2024-05-09T05:57:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.685+0000 7fcc587b2640 1 Processor -- start 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.688+0000 7fcc587b2640 1 -- start start 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.688+0000 7fcc587b2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5010a200 0x7fcc501a9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.688+0000 7fcc587b2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc5011a8f0 0x7fcc501a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.688+0000 7fcc587b2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc5011d250 0x7fcc501add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.688+0000 7fcc587b2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50122b30 con 0x7fcc5011d250 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.688+0000 7fcc587b2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501229b0 con 0x7fcc5010a200 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.688+0000 7fcc587b2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50122cb0 con 0x7fcc5011a8f0 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.690+0000 7fcc5654e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5010a200 0x7fcc501a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.690+0000 7fcc5654e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5010a200 0x7fcc501a9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:35168/0 (socket says 172.21.15.115:35168) 2024-05-09T05:57:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.690+0000 7fcc5654e640 1 -- 172.21.15.115:0/2031738380 learned_addr learned my addr 172.21.15.115:0/2031738380 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.692+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc5011d250 0x7fcc501add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.693+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc5011a8f0 0x7fcc501a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.695+0000 7fcc5654e640 1 -- 172.21.15.115:0/2031738380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc5011a8f0 msgr2=0x7fcc501a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.695+0000 7fcc5654e640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc5011a8f0 0x7fcc501a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.695+0000 7fcc5654e640 1 -- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc5011d250 msgr2=0x7fcc501add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.695+0000 7fcc5654e640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc5011d250 0x7fcc501add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.695+0000 7fcc5654e640 1 -- 172.21.15.115:0/2031738380 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc501ae390 con 0x7fcc5010a200 2024-05-09T05:57:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.700+0000 7fcc5654e640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5010a200 0x7fcc501a9530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800e9c0 tx=0x7fcc4800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.701+0000 7fcc477fe640 1 -- 172.21.15.115:0/2031738380 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4800ccc0 con 0x7fcc5010a200 2024-05-09T05:57:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.701+0000 7fcc477fe640 1 -- 172.21.15.115:0/2031738380 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc480054b0 con 0x7fcc5010a200 2024-05-09T05:57:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.702+0000 7fcc477fe640 1 -- 172.21.15.115:0/2031738380 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48010430 con 0x7fcc5010a200 2024-05-09T05:57:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.707+0000 7fcc587b2640 1 -- 172.21.15.115:0/2031738380 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc501ae680 con 0x7fcc5010a200 2024-05-09T05:57:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.707+0000 7fcc587b2640 1 -- 172.21.15.115:0/2031738380 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc501aeb50 con 0x7fcc5010a200 2024-05-09T05:57:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.711+0000 7fcc5f663740 1 Processor -- start 2024-05-09T05:57:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.712+0000 7fcc5f663740 1 -- start start 2024-05-09T05:57:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.712+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee801b0 0x5651ff223680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.712+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee572f0 0x5651ff223bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.712+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee5f4d0 0x5651ff224100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.713+0000 7fcc5f663740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5651ff1887c0 con 0x5651fee572f0 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.713+0000 7fcc5f663740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651ff1240e0 con 0x5651fee5f4d0 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.713+0000 7fcc5f663740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5651ff184780 con 0x5651fee801b0 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.715+0000 7fcc56d4f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee801b0 0x5651ff223680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.715+0000 7fcc56d4f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee801b0 0x5651ff223680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:43046/0 (socket says 172.21.15.115:43046) 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.715+0000 7fcc56d4f640 1 -- 172.21.15.115:0/1830261122 learned_addr learned my addr 172.21.15.115:0/1830261122 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.715+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/1830261122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee5f4d0 0x5651ff224100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.715+0000 7fcc56d4f640 1 -- 172.21.15.115:0/1830261122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee5f4d0 msgr2=0x5651ff224100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.715+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/1830261122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee5f4d0 0x5651ff224100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.715+0000 7fcc56d4f640 1 -- 172.21.15.115:0/1830261122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee572f0 msgr2=0x5651ff223bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.715+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/1830261122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee572f0 0x5651ff223bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.715+0000 7fcc56d4f640 1 -- 172.21.15.115:0/1830261122 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ff14d7c0 con 0x5651fee801b0 2024-05-09T05:57:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.716+0000 7fcc477fe640 1 -- 172.21.15.115:0/2031738380 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fcc48004e90 con 0x7fcc5010a200 2024-05-09T05:57:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.716+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/1830261122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee801b0 0x5651ff223680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000dbb0 tx=0x7fcc4000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f756adcb5a012f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.717+0000 7fcc477fe640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc3c0b2a40 0x7fcc3c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.717+0000 7fcc477fe640 1 -- 172.21.15.115:0/2031738380 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7fcc480e4e70 con 0x7fcc5010a200 2024-05-09T05:57:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.717+0000 7fcc44ff9640 1 -- 172.21.15.115:0/1830261122 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4000dea0 con 0x5651fee801b0 2024-05-09T05:57:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.717+0000 7fcc44ff9640 1 -- 172.21.15.115:0/1830261122 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc400035f0 con 0x5651fee801b0 2024-05-09T05:57:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.717+0000 7fcc44ff9640 1 -- 172.21.15.115:0/1830261122 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc400044c0 con 0x5651fee801b0 2024-05-09T05:57:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.721+0000 7fcc5f663740 1 -- 172.21.15.115:0/1830261122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee801b0 msgr2=0x5651ff223680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.721+0000 7fcc5f663740 1 --2- 172.21.15.115:0/1830261122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee801b0 0x5651ff223680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000dbb0 tx=0x7fcc4000f730 comp rx=0 tx=0).stop 2024-05-09T05:57:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.722+0000 7fcc5654e640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc3c0b2a40 0x7fcc3c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.724+0000 7fcc5f663740 1 -- 172.21.15.115:0/1830261122 shutdown_connections 2024-05-09T05:57:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.724+0000 7fcc5f663740 1 --2- 172.21.15.115:0/1830261122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee5f4d0 0x5651ff224100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.724+0000 7fcc5f663740 1 --2- 172.21.15.115:0/1830261122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee572f0 0x5651ff223bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.724+0000 7fcc5f663740 1 --2- 172.21.15.115:0/1830261122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee801b0 0x5651ff223680 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.724+0000 7fcc5f663740 1 -- 172.21.15.115:0/1830261122 >> 172.21.15.115:0/1830261122 conn(0x5651fee74300 msgr2=0x5651ff21fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.731+0000 7fcc5f663740 1 -- 172.21.15.115:0/1830261122 shutdown_connections 2024-05-09T05:57:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.731+0000 7fcc5f663740 1 -- 172.21.15.115:0/1830261122 wait complete. 2024-05-09T05:57:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.731+0000 7fcc5f663740 1 -- start start 2024-05-09T05:57:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.731+0000 7fcc5f663740 1 Processor -- start 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee572f0 0x5651ff22f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff22fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee801b0 0x5651ff22dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc5f663740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5651ff2307b0 con 0x5651fee801b0 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc5f663740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651ff10f9c0 con 0x5651fee572f0 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc5f663740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5651ff230930 con 0x5651fee5f4d0 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc5f663740 10 client.0 _ll_register_callbacks cb 0x5651ff13c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc56d4f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee572f0 0x5651ff22f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc56d4f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee572f0 0x5651ff22f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:35186/0 (socket says 172.21.15.115:35186) 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc56d4f640 1 -- 172.21.15.115:0/3174398474 learned_addr learned my addr 172.21.15.115:0/3174398474 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.733+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff22fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.734+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee801b0 0x5651ff22dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.734+0000 7fcc5654e640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc3c0b2a40 0x7fcc3c0b4f50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcc480058c0 tx=0x7fcc480abab0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.734+0000 7fcc56d4f640 1 -- 172.21.15.115:0/3174398474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 msgr2=0x5651ff22fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.734+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff22fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.734+0000 7fcc56d4f640 1 -- 172.21.15.115:0/3174398474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee801b0 msgr2=0x5651ff22dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.734+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee801b0 0x5651ff22dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.734+0000 7fcc56d4f640 1 -- 172.21.15.115:0/3174398474 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ff1361f0 con 0x5651fee572f0 2024-05-09T05:57:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.734+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee572f0 0x5651ff22f810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000ded0 tx=0x7fcc40003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e27c245dfe8257a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.736+0000 7fcc39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:57:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.736+0000 7fcc39ffb640 1 -- 172.21.15.115:0/3174398474 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4001f020 con 0x5651fee572f0 2024-05-09T05:57:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.736+0000 7fcc39ffb640 1 -- 172.21.15.115:0/3174398474 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7fcc400039a0 con 0x5651fee572f0 2024-05-09T05:57:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.736+0000 7fcc5f663740 1 -- 172.21.15.115:0/3174398474 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc400035f0 con 0x5651fee572f0 2024-05-09T05:57:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.737+0000 7fcc39ffb640 1 -- 172.21.15.115:0/3174398474 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fcc40025040 con 0x5651fee572f0 2024-05-09T05:57:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.739+0000 7fcc5f663740 10 client.28229 fetch_fsmap learned FSMap version 71 2024-05-09T05:57:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.739+0000 7fcc5f663740 1 -- 172.21.15.115:0/3174398474 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5651feeeb790 con 0x5651fee572f0 2024-05-09T05:57:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.740+0000 7fcc39ffb640 1 -- 172.21.15.115:0/3174398474 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc400035f0 con 0x5651fee572f0 2024-05-09T05:57:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.740+0000 7fcc39ffb640 1 -- 172.21.15.115:0/3174398474 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 71) ==== 8386+0+0 (secure 0 0 0) 0x7fcc400117b0 con 0x5651fee572f0 2024-05-09T05:57:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.740+0000 7fcc5f663740 10 client.28229 fetch_fsmap finished waiting for FSMap version 71 2024-05-09T05:57:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.740+0000 7fcc5f663740 10 client.28229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:57:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.740+0000 7fcc5f663740 20 client.28229 populate_metadata read hostname 'smithi115' 2024-05-09T05:57:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.740+0000 7fcc5f663740 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5651fee3ed80 0x5651ff2403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.740+0000 7fcc5f663740 4 client.28229 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:57:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.740+0000 7fcc5f663740 1 -- 172.21.15.115:0/3174398474 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651ff1963d0 con 0x5651fee3ed80 2024-05-09T05:57:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.743+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5651fee3ed80 0x5651ff2403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.748+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5651fee3ed80 0x5651ff2403c0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bca4a7c0c996730 server_cookie=580c1b38a56f6a64 in_seq=0 out_seq=0 2024-05-09T05:57:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.748+0000 7fcc39ffb640 10 client.28229 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.751+0000 7fcc39ffb640 1 -- 172.21.15.115:0/3174398474 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651ff1963d0 con 0x5651fee3ed80 2024-05-09T05:57:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.751+0000 7fcc39ffb640 10 client.28229 handle_command_reply: tid=0 2024-05-09T05:57:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.751+0000 7fcc5f663740 1 client.28229 shutdown 2024-05-09T05:57:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.751+0000 7fcc5f663740 1 -- 172.21.15.115:0/3174398474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee572f0 msgr2=0x5651ff22f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.751+0000 7fcc5f663740 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee572f0 0x5651ff22f810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000ded0 tx=0x7fcc40003f30 comp rx=0 tx=0).stop 2024-05-09T05:57:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.752+0000 7fcc5f663740 1 -- 172.21.15.115:0/3174398474 shutdown_connections 2024-05-09T05:57:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.752+0000 7fcc5f663740 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee801b0 0x5651ff22dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.752+0000 7fcc5f663740 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5651fee3ed80 0x5651ff2403c0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.752+0000 7fcc5f663740 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff22fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.752+0000 7fcc5f663740 1 --2- 172.21.15.115:0/3174398474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee572f0 0x5651ff22f810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.752+0000 7fcc5f663740 1 -- 172.21.15.115:0/3174398474 >> 172.21.15.115:0/3174398474 conn(0x5651fee74300 msgr2=0x5651ff21e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.753+0000 7fcc39ffb640 0 client.28229 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.759+0000 7fcc5f663740 1 -- 172.21.15.115:0/3174398474 shutdown_connections 2024-05-09T05:57:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.766+0000 7fcc5f663740 1 -- 172.21.15.115:0/3174398474 wait complete. 2024-05-09T05:57:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.766+0000 7fcc5f663740 20 client.28229 trim_cache size 0 max 16384 2024-05-09T05:57:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.809+0000 7fcc5f663740 1 Processor -- start 2024-05-09T05:57:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.811+0000 7fcc5f663740 1 -- start start 2024-05-09T05:57:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.812+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee572f0 0x5651ff21ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.812+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff245b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.812+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 0x5651ff2460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.813+0000 7fcc5f663740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501c8640 con 0x5651fee572f0 2024-05-09T05:57:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.813+0000 7fcc55d4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff245b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.813+0000 7fcc55d4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff245b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:43056/0 (socket says 172.21.15.115:43056) 2024-05-09T05:57:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.813+0000 7fcc55d4d640 1 -- 172.21.15.115:0/1358297697 learned_addr learned my addr 172.21.15.115:0/1358297697 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.813+0000 7fcc5f663740 1 -- 172.21.15.115:0/1358297697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501ca1e0 con 0x5651fee801b0 2024-05-09T05:57:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.813+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/1358297697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 0x5651ff2460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.813+0000 7fcc5f663740 1 -- 172.21.15.115:0/1358297697 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501c8d10 con 0x5651fee5f4d0 2024-05-09T05:57:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.813+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/1358297697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee572f0 0x5651ff21ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.814+0000 7fcc55d4d640 1 -- 172.21.15.115:0/1358297697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 msgr2=0x5651ff2460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.814+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/1358297697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 0x5651ff2460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.814+0000 7fcc55d4d640 1 -- 172.21.15.115:0/1358297697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee572f0 msgr2=0x5651ff21ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.814+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/1358297697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee572f0 0x5651ff21ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.814+0000 7fcc55d4d640 1 -- 172.21.15.115:0/1358297697 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fef625f0 con 0x5651fee5f4d0 2024-05-09T05:57:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.814+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/1358297697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff245b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009420 tx=0x7fcc4c02ac50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc945ea50f552d08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.815+0000 7fcc3bfff640 1 -- 172.21.15.115:0/1358297697 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c04a020 con 0x5651fee5f4d0 2024-05-09T05:57:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.815+0000 7fcc3bfff640 1 -- 172.21.15.115:0/1358297697 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc4c037040 con 0x5651fee5f4d0 2024-05-09T05:57:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.815+0000 7fcc3bfff640 1 -- 172.21.15.115:0/1358297697 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c036040 con 0x5651fee5f4d0 2024-05-09T05:57:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.816+0000 7fcc5f663740 1 -- 172.21.15.115:0/1358297697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 msgr2=0x5651ff245b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.816+0000 7fcc5f663740 1 --2- 172.21.15.115:0/1358297697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff245b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009420 tx=0x7fcc4c02ac50 comp rx=0 tx=0).stop 2024-05-09T05:57:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.817+0000 7fcc5f663740 1 -- 172.21.15.115:0/1358297697 shutdown_connections 2024-05-09T05:57:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.817+0000 7fcc5f663740 1 --2- 172.21.15.115:0/1358297697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 0x5651ff2460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.817+0000 7fcc5f663740 1 --2- 172.21.15.115:0/1358297697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee5f4d0 0x5651ff245b80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.817+0000 7fcc5f663740 1 --2- 172.21.15.115:0/1358297697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee572f0 0x5651ff21ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.817+0000 7fcc5f663740 1 -- 172.21.15.115:0/1358297697 >> 172.21.15.115:0/1358297697 conn(0x5651fee74300 msgr2=0x5651ff21f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.819+0000 7fcc5f663740 1 -- 172.21.15.115:0/1358297697 shutdown_connections 2024-05-09T05:57:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.819+0000 7fcc5f663740 1 -- 172.21.15.115:0/1358297697 wait complete. 2024-05-09T05:57:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.820+0000 7fcc5f663740 1 -- start start 2024-05-09T05:57:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.820+0000 7fcc5f663740 1 Processor -- start 2024-05-09T05:57:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee572f0 0x5651ff232740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc5f663740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee5f4d0 0x5651ff232c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc56d4f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee572f0 0x5651ff232740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc56d4f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee572f0 0x5651ff232740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:43062/0 (socket says 172.21.15.115:43062) 2024-05-09T05:57:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc56d4f640 1 -- 172.21.15.115:0/955253736 learned_addr learned my addr 172.21.15.115:0/955253736 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc5f663740 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 0x5651ff2331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501c9b10 con 0x5651fee5f4d0 2024-05-09T05:57:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501c9410 con 0x5651fee801b0 2024-05-09T05:57:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5010b520 con 0x5651fee572f0 2024-05-09T05:57:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.821+0000 7fcc5f663740 10 client.0 _ll_register_callbacks cb 0x7fcc40015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:57:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.823+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 0x5651ff2331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.823+0000 7fcc56d4f640 1 -- 172.21.15.115:0/955253736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 msgr2=0x5651ff2331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.823+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 0x5651ff2331c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.823+0000 7fcc56d4f640 1 -- 172.21.15.115:0/955253736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee5f4d0 msgr2=0x5651ff232c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.823+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee5f4d0 0x5651ff232c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.823+0000 7fcc56d4f640 1 -- 172.21.15.115:0/955253736 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ff14ced0 con 0x5651fee572f0 2024-05-09T05:57:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.823+0000 7fcc56d4f640 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee572f0 0x5651ff232740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc40011ea0 tx=0x7fcc40003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e277fe44ddd047bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.823+0000 7fcc39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:57:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.824+0000 7fcc39ffb640 1 -- 172.21.15.115:0/955253736 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4001f040 con 0x5651fee572f0 2024-05-09T05:57:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.824+0000 7fcc39ffb640 1 -- 172.21.15.115:0/955253736 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7fcc400050e0 con 0x5651fee572f0 2024-05-09T05:57:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.826+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc4c037040 con 0x5651fee572f0 2024-05-09T05:57:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.827+0000 7fcc39ffb640 1 -- 172.21.15.115:0/955253736 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fcc40011440 con 0x5651fee572f0 2024-05-09T05:57:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.828+0000 7fcc5f663740 10 client.28378 fetch_fsmap learned FSMap version 71 2024-05-09T05:57:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.828+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5651ff2a7640 con 0x5651fee572f0 2024-05-09T05:57:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.828+0000 7fcc39ffb640 1 -- 172.21.15.115:0/955253736 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc4c037040 con 0x5651fee572f0 2024-05-09T05:57:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.829+0000 7fcc39ffb640 1 -- 172.21.15.115:0/955253736 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 71) ==== 8386+0+0 (secure 0 0 0) 0x7fcc40003680 con 0x5651fee572f0 2024-05-09T05:57:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.831+0000 7fcc5f663740 10 client.28378 fetch_fsmap finished waiting for FSMap version 71 2024-05-09T05:57:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.831+0000 7fcc5f663740 10 client.28378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:57:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.831+0000 7fcc5f663740 20 client.28378 populate_metadata read hostname 'smithi115' 2024-05-09T05:57:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.831+0000 7fcc5f663740 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5651fee141c0 0x5651ff2a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.831+0000 7fcc5f663740 4 client.28378 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:57:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.831+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651ff190240 con 0x5651fee141c0 2024-05-09T05:57:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.835+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5651fee141c0 0x5651ff2a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.841+0000 7fcc55d4d640 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5651fee141c0 0x5651ff2a9e60 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f794d792bb6fa27 server_cookie=9e43259f1babec7d in_seq=0 out_seq=0 2024-05-09T05:57:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:25.841+0000 7fcc39ffb640 10 client.28378 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:25.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:25 smithi195 ceph-mon[33844]: pgmap v606: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-09T05:57:25.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:25 smithi115 ceph-mon[30349]: pgmap v606: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-09T05:57:25.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:25 smithi161 ceph-mon[33805]: pgmap v606: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-09T05:57:27.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:27 smithi195 ceph-mon[33844]: pgmap v607: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-05-09T05:57:27.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:27 smithi115 ceph-mon[30349]: pgmap v607: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-05-09T05:57:27.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:27 smithi161 ceph-mon[33805]: pgmap v607: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-05-09T05:57:29.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:29 smithi195 ceph-mon[33844]: pgmap v608: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-09T05:57:29.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:29 smithi115 ceph-mon[30349]: pgmap v608: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-09T05:57:29.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:29 smithi161 ceph-mon[33805]: pgmap v608: 129 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 111 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-09T05:57:31.572 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:31 smithi195 ceph-mon[33844]: pgmap v609: 129 pgs: 7 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 115 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 2.90k op/s 2024-05-09T05:57:31.769 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:31 smithi115 ceph-mon[30349]: pgmap v609: 129 pgs: 7 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 115 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 2.90k op/s 2024-05-09T05:57:31.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:31 smithi161 ceph-mon[33805]: pgmap v609: 129 pgs: 7 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 115 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 2.90k op/s 2024-05-09T05:57:32.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.431+0000 7f03334da700 1 -- 172.21.15.115:0/1147699983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10fca0 msgr2=0x7f032c1100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:32.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.431+0000 7f03334da700 1 --2- 172.21.15.115:0/1147699983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10fca0 0x7f032c1100b0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f0324013930 tx=0x7f0324013c40 comp rx=0 tx=0).stop 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.432+0000 7f03334da700 1 -- 172.21.15.115:0/1147699983 shutdown_connections 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.432+0000 7f03334da700 1 --2- 172.21.15.115:0/1147699983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10fca0 0x7f032c1100b0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.432+0000 7f03334da700 1 --2- 172.21.15.115:0/1147699983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f032c06b570 0x7f032c10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.432+0000 7f03334da700 1 --2- 172.21.15.115:0/1147699983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f032c10b400 0x7f032c10f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.432+0000 7f03334da700 1 -- 172.21.15.115:0/1147699983 >> 172.21.15.115:0/1147699983 conn(0x7f032c06f6e0 msgr2=0x7f032c06fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.434+0000 7f03334da700 1 -- 172.21.15.115:0/1147699983 shutdown_connections 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.434+0000 7f03334da700 1 -- 172.21.15.115:0/1147699983 wait complete. 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03334da700 1 Processor -- start 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03334da700 1 -- start start 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03334da700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10b400 0x7f032c1b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03334da700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f032c10fca0 0x7f032c1b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03334da700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f032c1b6bc0 0x7f032c1b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03334da700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f032c118890 con 0x7f032c10b400 2024-05-09T05:57:32.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03334da700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f032c118710 con 0x7f032c10fca0 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03334da700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f032c118a10 con 0x7f032c1b6bc0 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03324d8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10b400 0x7f032c1b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03324d8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10b400 0x7f032c1b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37834/0 (socket says 172.21.15.115:37834) 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.435+0000 7f03324d8700 1 -- 172.21.15.115:0/319356384 learned_addr learned my addr 172.21.15.115:0/319356384 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.438+0000 7f0331cd7700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f032c10fca0 0x7f032c1b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.439+0000 7f0332cd9700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f032c1b6bc0 0x7f032c1b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.444+0000 7f0331cd7700 1 -- 172.21.15.115:0/319356384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f032c1b6bc0 msgr2=0x7f032c1b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.444+0000 7f0331cd7700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f032c1b6bc0 0x7f032c1b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.444+0000 7f0331cd7700 1 -- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10b400 msgr2=0x7f032c1b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.444+0000 7f0331cd7700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10b400 0x7f032c1b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.444+0000 7f0331cd7700 1 -- 172.21.15.115:0/319356384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c1b7670 con 0x7f032c10fca0 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.444+0000 7f03324d8700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10b400 0x7f032c1b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.444+0000 7f0331cd7700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f032c10fca0 0x7f032c1b2930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f032c06d0e0 tx=0x7f031c00b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.446+0000 7f03237fe700 1 -- 172.21.15.115:0/319356384 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c015020 con 0x7f032c10fca0 2024-05-09T05:57:32.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.446+0000 7f03334da700 1 -- 172.21.15.115:0/319356384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c1b7960 con 0x7f032c10fca0 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.446+0000 7f03334da700 1 -- 172.21.15.115:0/319356384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c1bf150 con 0x7f032c10fca0 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.447+0000 7f03237fe700 1 -- 172.21.15.115:0/319356384 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f031c00be50 con 0x7f032c10fca0 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.448+0000 7f03237fe700 1 -- 172.21.15.115:0/319356384 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c010b00 con 0x7f032c10fca0 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.451+0000 7f03237fe700 1 -- 172.21.15.115:0/319356384 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f031c018020 con 0x7f032c10fca0 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.451+0000 7f0343152b80 1 Processor -- start 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f0343152b80 1 -- start start 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f03237fe700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f03180b2b80 0x7f03180b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f0332cd9700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f03180b2b80 0x7f03180b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f03237fe700 1 -- 172.21.15.115:0/319356384 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f031c0e1ca0 con 0x7f032c10fca0 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f0343152b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 0x556e758a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f0343152b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e75721630 0x556e758a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f0343152b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7589ef90 0x556e7589f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f0343152b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556e757c1180 con 0x556e7589ef90 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f0343152b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556e7575f980 con 0x556e7580b2f0 2024-05-09T05:57:32.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f0343152b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e757bd140 con 0x556e75721630 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f03324d8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 0x556e758a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f03324d8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 0x556e758a4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:35216/0 (socket says 172.21.15.115:35216) 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f03324d8700 1 -- 172.21.15.115:0/1297861622 learned_addr learned my addr 172.21.15.115:0/1297861622 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.452+0000 7f0332cd9700 1 --2- 172.21.15.115:0/1297861622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7589ef90 0x556e7589f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0331cd7700 1 --2- 172.21.15.115:0/1297861622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e75721630 0x556e758a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0332cd9700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f03180b2b80 0x7f03180b5090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0324009e60 tx=0x7f03240096e0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0332cd9700 1 -- 172.21.15.115:0/1297861622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e75721630 msgr2=0x556e758a5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0332cd9700 1 --2- 172.21.15.115:0/1297861622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e75721630 0x556e758a5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0332cd9700 1 -- 172.21.15.115:0/1297861622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 msgr2=0x556e758a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0332cd9700 1 --2- 172.21.15.115:0/1297861622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 0x556e758a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0332cd9700 1 -- 172.21.15.115:0/1297861622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e75757500 con 0x556e7589ef90 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0332cd9700 1 --2- 172.21.15.115:0/1297861622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7589ef90 0x556e7589f3a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f0324014010 tx=0x7f0324011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27ae7675bf34ca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0320ff9700 1 -- 172.21.15.115:0/1297861622 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032401b040 con 0x556e7589ef90 2024-05-09T05:57:32.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0320ff9700 1 -- 172.21.15.115:0/1297861622 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0324024030 con 0x556e7589ef90 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.453+0000 7f0320ff9700 1 -- 172.21.15.115:0/1297861622 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240206b0 con 0x556e7589ef90 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.454+0000 7f0343152b80 1 -- 172.21.15.115:0/1297861622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7589ef90 msgr2=0x556e7589f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.454+0000 7f0343152b80 1 --2- 172.21.15.115:0/1297861622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7589ef90 0x556e7589f3a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f0324014010 tx=0x7f0324011000 comp rx=0 tx=0).stop 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.454+0000 7f0343152b80 1 -- 172.21.15.115:0/1297861622 shutdown_connections 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.454+0000 7f0343152b80 1 --2- 172.21.15.115:0/1297861622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 0x556e758a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.454+0000 7f0343152b80 1 --2- 172.21.15.115:0/1297861622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7589ef90 0x556e7589f3a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.454+0000 7f0343152b80 1 --2- 172.21.15.115:0/1297861622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e75721630 0x556e758a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.454+0000 7f0343152b80 1 -- 172.21.15.115:0/1297861622 >> 172.21.15.115:0/1297861622 conn(0x556e75781830 msgr2=0x556e75899ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.455+0000 7f0343152b80 1 -- 172.21.15.115:0/1297861622 shutdown_connections 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.455+0000 7f0343152b80 1 -- 172.21.15.115:0/1297861622 wait complete. 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.455+0000 7f0343152b80 1 -- start start 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.455+0000 7f0343152b80 1 Processor -- start 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.455+0000 7f0343152b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e75721630 0x556e758a8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.455+0000 7f0343152b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7580b2f0 0x556e758a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.455+0000 7f0343152b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 0x556e758a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.455+0000 7f0331cd7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7580b2f0 0x556e758a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.455+0000 7f0331cd7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7580b2f0 0x556e758a8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37838/0 (socket says 172.21.15.115:37838) 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0331cd7700 1 -- 172.21.15.115:0/1169665165 learned_addr learned my addr 172.21.15.115:0/1169665165 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556e7563cd30 con 0x556e7580b2f0 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f03324d8700 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e75721630 0x556e758a8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556e7577b2e0 con 0x556e75721630 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556e7589fdf0 con 0x556e7589ef90 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0343152b80 10 client.0 _ll_register_callbacks cb 0x556e75632850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0332cd9700 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 0x556e758a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0331cd7700 1 -- 172.21.15.115:0/1169665165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 msgr2=0x556e758a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0331cd7700 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 0x556e758a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0331cd7700 1 -- 172.21.15.115:0/1169665165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e75721630 msgr2=0x556e758a8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0331cd7700 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e75721630 0x556e758a8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0331cd7700 1 -- 172.21.15.115:0/1169665165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e7563c620 con 0x556e7580b2f0 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f03324d8700 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e75721630 0x556e758a8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0331cd7700 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7580b2f0 0x556e758a8f80 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f031c011f70 tx=0x7f031c0df530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7177924afc6e6750 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:32.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0315ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0315ffb700 1 -- 172.21.15.115:0/1169665165 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c0a93a0 con 0x556e7580b2f0 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0315ffb700 1 -- 172.21.15.115:0/1169665165 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f031c0a9110 con 0x556e7580b2f0 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0324024030 con 0x556e7580b2f0 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0315ffb700 1 -- 172.21.15.115:0/1169665165 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f031c0a8ca0 con 0x556e7580b2f0 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0343152b80 10 client.19053 fetch_fsmap learned FSMap version 71 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.456+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x556e758b6380 con 0x556e7580b2f0 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.457+0000 7f0315ffb700 1 -- 172.21.15.115:0/1169665165 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0324024030 con 0x556e7580b2f0 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.457+0000 7f0315ffb700 1 -- 172.21.15.115:0/1169665165 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 71) ==== 8386+0+0 (secure 0 0 0) 0x7f031c0c3020 con 0x556e7580b2f0 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.457+0000 7f0343152b80 10 client.19053 fetch_fsmap finished waiting for FSMap version 71 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.457+0000 7f0343152b80 10 client.19053 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.457+0000 7f0343152b80 20 client.19053 populate_metadata read hostname 'smithi115' 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.457+0000 7f0343152b80 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e758b89c0 0x556e758bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.457+0000 7f0343152b80 4 client.19053 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.457+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e757ceb60 con 0x556e758b89c0 2024-05-09T05:57:32.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.458+0000 7f03324d8700 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e758b89c0 0x556e758bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.458+0000 7f03324d8700 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e758b89c0 0x556e758bae10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddfdee841c783400 server_cookie=c34ee11651feab2f in_seq=0 out_seq=0 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.458+0000 7f0315ffb700 10 client.19053 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0315ffb700 1 -- 172.21.15.115:0/1169665165 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e757ceb60 con 0x556e758b89c0 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0315ffb700 10 client.19053 handle_command_reply: tid=0 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0343152b80 1 client.19053 shutdown 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7580b2f0 msgr2=0x556e758a8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0343152b80 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7580b2f0 0x556e758a8f80 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f031c011f70 tx=0x7f031c0df530 comp rx=0 tx=0).stop 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 shutdown_connections 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0343152b80 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e75721630 0x556e758a8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0343152b80 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e758b89c0 0x556e758bae10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0343152b80 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 0x556e758a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0315ffb700 0 client.19053 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:32.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.459+0000 7f0343152b80 1 --2- 172.21.15.115:0/1169665165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e7580b2f0 0x556e758a8f80 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.460+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 >> 172.21.15.115:0/1169665165 conn(0x556e75781830 msgr2=0x556e758991d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.460+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 shutdown_connections 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.460+0000 7f0343152b80 1 -- 172.21.15.115:0/1169665165 wait complete. 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.460+0000 7f0343152b80 20 client.19053 trim_cache size 0 max 16384 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.479+0000 7f0343152b80 1 Processor -- start 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0343152b80 1 -- start start 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0343152b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e7589e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0343152b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 0x556e758a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0343152b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 0x556e758aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f03324d8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e7589e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0343152b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f031000ed90 con 0x556e75721630 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0343152b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0310010930 con 0x556e7580b2f0 2024-05-09T05:57:32.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0343152b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f031000f460 con 0x556e7589ef90 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f03324d8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e7589e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37846/0 (socket says 172.21.15.115:37846) 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f03324d8700 1 -- 172.21.15.115:0/2567440746 learned_addr learned my addr 172.21.15.115:0/2567440746 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0331cd7700 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 0x556e758a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0332cd9700 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 0x556e758aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f03324d8700 1 -- 172.21.15.115:0/2567440746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 msgr2=0x556e758aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f03324d8700 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 0x556e758aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f03324d8700 1 -- 172.21.15.115:0/2567440746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 msgr2=0x556e758a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f03324d8700 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 0x556e758a9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f03324d8700 1 -- 172.21.15.115:0/2567440746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e75899900 con 0x556e75721630 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.480+0000 7f0332cd9700 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 0x556e758aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.481+0000 7f03324d8700 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e7589e9b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x556e758bb560 tx=0x7f0328008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd57e1eb34059ffc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.481+0000 7f0317fff700 1 -- 172.21.15.115:0/2567440746 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328022070 con 0x556e75721630 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.481+0000 7f0317fff700 1 -- 172.21.15.115:0/2567440746 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f032801e2d0 con 0x556e75721630 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.481+0000 7f0317fff700 1 -- 172.21.15.115:0/2567440746 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328016720 con 0x556e75721630 2024-05-09T05:57:32.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 -- 172.21.15.115:0/2567440746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 msgr2=0x556e7589e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e7589e9b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x556e758bb560 tx=0x7f0328008040 comp rx=0 tx=0).stop 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 -- 172.21.15.115:0/2567440746 shutdown_connections 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7589ef90 0x556e758aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e7589e9b0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 --2- 172.21.15.115:0/2567440746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7580b2f0 0x556e758a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 -- 172.21.15.115:0/2567440746 >> 172.21.15.115:0/2567440746 conn(0x556e75781830 msgr2=0x556e758aa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 -- 172.21.15.115:0/2567440746 shutdown_connections 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 -- 172.21.15.115:0/2567440746 wait complete. 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 -- start start 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.482+0000 7f0343152b80 1 Processor -- start 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0343152b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e758b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0343152b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7580b2f0 0x556e758b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0343152b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7589ef90 0x556e758abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0343152b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0310010260 con 0x556e75721630 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0343152b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f031000fb60 con 0x556e7589ef90 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0343152b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f031000cd50 con 0x556e7580b2f0 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f03324d8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e758b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0343152b80 10 client.0 _ll_register_callbacks cb 0x556e758a4770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f03324d8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e758b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37862/0 (socket says 172.21.15.115:37862) 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f03324d8700 1 -- 172.21.15.115:0/3359425780 learned_addr learned my addr 172.21.15.115:0/3359425780 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0332cd9700 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7589ef90 0x556e758abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f03324d8700 1 -- 172.21.15.115:0/3359425780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7580b2f0 msgr2=0x556e758b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f03324d8700 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7580b2f0 0x556e758b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f03324d8700 1 -- 172.21.15.115:0/3359425780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7589ef90 msgr2=0x556e758abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f03324d8700 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7589ef90 0x556e758abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f03324d8700 1 -- 172.21.15.115:0/3359425780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e758ac560 con 0x556e75721630 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0332cd9700 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7589ef90 0x556e758abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:57:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f03324d8700 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e758b0970 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f0328008010 tx=0x7f032801e480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44eaf0eea534fdbb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0315ffb700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0315ffb700 1 -- 172.21.15.115:0/3359425780 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032802d070 con 0x556e75721630 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0315ffb700 1 -- 172.21.15.115:0/3359425780 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f0328027450 con 0x556e75721630 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.483+0000 7f0343152b80 1 -- 172.21.15.115:0/3359425780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f032801e2d0 con 0x556e75721630 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0315ffb700 1 -- 172.21.15.115:0/3359425780 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f0328039050 con 0x556e75721630 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0343152b80 10 client.19059 fetch_fsmap learned FSMap version 71 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0343152b80 1 -- 172.21.15.115:0/3359425780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x556e75924a30 con 0x556e75721630 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0315ffb700 1 -- 172.21.15.115:0/3359425780 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f032801e2d0 con 0x556e75721630 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0315ffb700 1 -- 172.21.15.115:0/3359425780 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 71) ==== 8386+0+0 (secure 0 0 0) 0x7f03280167a0 con 0x556e75721630 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0343152b80 10 client.19059 fetch_fsmap finished waiting for FSMap version 71 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0343152b80 10 client.19059 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0343152b80 20 client.19059 populate_metadata read hostname 'smithi115' 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0343152b80 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e759250c0 0x556e759274e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:32.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0343152b80 4 client.19059 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.3, "await_for": 45.0}] 2024-05-09T05:57:32.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0343152b80 1 -- 172.21.15.115:0/3359425780 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.3, "await_for": 45.0}) -- 0x556e757c8c00 con 0x556e759250c0 2024-05-09T05:57:32.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.484+0000 7f0331cd7700 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e759250c0 0x556e759274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:32.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.485+0000 7f0315ffb700 10 client.19059 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:32.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:31.485+0000 7f0331cd7700 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e759250c0 0x556e759274e0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a987112122d0c3 server_cookie=eafb5f340e64e433 in_seq=0 out_seq=0 2024-05-09T05:57:33.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:33 smithi195 ceph-mon[33844]: pgmap v610: 129 pgs: 7 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 115 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.1 MiB/s wr, 2.29k op/s 2024-05-09T05:57:33.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:33 smithi115 ceph-mon[30349]: pgmap v610: 129 pgs: 7 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 115 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.1 MiB/s wr, 2.29k op/s 2024-05-09T05:57:33.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:33 smithi161 ceph-mon[33805]: pgmap v610: 129 pgs: 7 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 115 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.1 MiB/s wr, 2.29k op/s 2024-05-09T05:57:35.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:35 smithi195 ceph-mon[33844]: pgmap v611: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-05-09T05:57:35.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:35 smithi195 ceph-mon[33844]: osdmap e116: 12 total, 12 up, 12 in 2024-05-09T05:57:35.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:35 smithi115 ceph-mon[30349]: pgmap v611: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-05-09T05:57:35.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:35 smithi115 ceph-mon[30349]: osdmap e116: 12 total, 12 up, 12 in 2024-05-09T05:57:35.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:35 smithi161 ceph-mon[33805]: pgmap v611: 129 pgs: 4 active+clean+snaptrim, 125 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 12 MiB/s wr, 3.08k op/s 2024-05-09T05:57:35.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:35 smithi161 ceph-mon[33805]: osdmap e116: 12 total, 12 up, 12 in 2024-05-09T05:57:37.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:37 smithi195 ceph-mon[33844]: pgmap v613: 129 pgs: 8 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 116 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.9 MiB/s wr, 2.48k op/s 2024-05-09T05:57:37.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:37 smithi115 ceph-mon[30349]: pgmap v613: 129 pgs: 8 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 116 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.9 MiB/s wr, 2.48k op/s 2024-05-09T05:57:37.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:37 smithi161 ceph-mon[33805]: pgmap v613: 129 pgs: 8 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 116 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 9.9 MiB/s wr, 2.48k op/s 2024-05-09T05:57:39.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:39 smithi195 ceph-mon[33844]: pgmap v614: 129 pgs: 13 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 110 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 10 MiB/s wr, 2.51k op/s 2024-05-09T05:57:39.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:39 smithi195 ceph-mon[33844]: osdmap e117: 12 total, 12 up, 12 in 2024-05-09T05:57:39.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:39 smithi115 ceph-mon[30349]: pgmap v614: 129 pgs: 13 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 110 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 10 MiB/s wr, 2.51k op/s 2024-05-09T05:57:39.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:39 smithi115 ceph-mon[30349]: osdmap e117: 12 total, 12 up, 12 in 2024-05-09T05:57:39.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:39 smithi161 ceph-mon[33805]: pgmap v614: 129 pgs: 13 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 110 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 10 MiB/s wr, 2.51k op/s 2024-05-09T05:57:39.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:39 smithi161 ceph-mon[33805]: osdmap e117: 12 total, 12 up, 12 in 2024-05-09T05:57:40.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0315ffb700 1 -- 172.21.15.115:0/3359425780 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556e757c8c00 con 0x556e759250c0 2024-05-09T05:57:40.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0315ffb700 10 client.19059 handle_command_reply: tid=0 2024-05-09T05:57:40.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0343152b80 1 client.19059 shutdown 2024-05-09T05:57:40.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0343152b80 1 -- 172.21.15.115:0/3359425780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 msgr2=0x556e758b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0343152b80 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e758b0970 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f0328008010 tx=0x7f032801e480 comp rx=0 tx=0).stop 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0343152b80 1 -- 172.21.15.115:0/3359425780 shutdown_connections 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0343152b80 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e7589ef90 0x556e758abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0343152b80 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556e759250c0 0x556e759274e0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0343152b80 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556e75721630 0x556e758b0970 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0343152b80 1 --2- 172.21.15.115:0/3359425780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556e7580b2f0 0x556e758b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0315ffb700 0 client.19059 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.595+0000 7f0343152b80 1 -- 172.21.15.115:0/3359425780 >> 172.21.15.115:0/3359425780 conn(0x556e75781830 msgr2=0x556e7589e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.596+0000 7f0343152b80 1 -- 172.21.15.115:0/3359425780 shutdown_connections 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.596+0000 7f0343152b80 1 -- 172.21.15.115:0/3359425780 wait complete. 2024-05-09T05:57:40.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.596+0000 7f0343152b80 20 client.19059 trim_cache size 0 max 16384 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 71, 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 50, 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "ac200e3d": { 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 50, 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 141.3, 2024-05-09T05:57:40.585 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:40.586 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.596+0000 7f03334da700 1 -- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f03180b2b80 msgr2=0x7f03180b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.596+0000 7f03334da700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f03180b2b80 0x7f03180b5090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0324009e60 tx=0x7f03240096e0 comp rx=0 tx=0).stop 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.596+0000 7f03334da700 1 -- 172.21.15.115:0/319356384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f032c10fca0 msgr2=0x7f032c1b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.596+0000 7f03334da700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f032c10fca0 0x7f032c1b2930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f032c06d0e0 tx=0x7f031c00b750 comp rx=0 tx=0).stop 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.597+0000 7f03334da700 1 -- 172.21.15.115:0/319356384 shutdown_connections 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.597+0000 7f03334da700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f03180b2b80 0x7f03180b5090 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.597+0000 7f03334da700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f032c1b6bc0 0x7f032c1b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.597+0000 7f03334da700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c10b400 0x7f032c1b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.597+0000 7f03334da700 1 --2- 172.21.15.115:0/319356384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f032c10fca0 0x7f032c1b2930 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.597+0000 7f03334da700 1 -- 172.21.15.115:0/319356384 >> 172.21.15.115:0/319356384 conn(0x7f032c06f6e0 msgr2=0x7f032c10e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.597+0000 7f03334da700 1 -- 172.21.15.115:0/319356384 shutdown_connections 2024-05-09T05:57:40.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:40.597+0000 7f03334da700 1 -- 172.21.15.115:0/319356384 wait complete. 2024-05-09T05:57:41.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:41 smithi195 ceph-mon[33844]: pgmap v616: 129 pgs: 20 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 102 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 7.6 MiB/s wr, 1.91k op/s 2024-05-09T05:57:41.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:41 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:57:41.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:41 smithi115 ceph-mon[30349]: pgmap v616: 129 pgs: 20 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 102 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 7.6 MiB/s wr, 1.91k op/s 2024-05-09T05:57:41.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:41 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:57:41.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:41 smithi161 ceph-mon[33805]: pgmap v616: 129 pgs: 20 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 102 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 7.6 MiB/s wr, 1.91k op/s 2024-05-09T05:57:41.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:41 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:57:43.631 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ac200e3d', quiesce took 69.3 seconds. Will release after: -48.0 2024-05-09T05:57:43.632 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:57:43.887 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:57:43.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:43 smithi115 ceph-mon[30349]: pgmap v617: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.4 MiB/s wr, 612 op/s 2024-05-09T05:57:43.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:43 smithi161 ceph-mon[33805]: pgmap v617: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.4 MiB/s wr, 612 op/s 2024-05-09T05:57:44.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:43 smithi195 ceph-mon[33844]: pgmap v617: 129 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 112 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.4 MiB/s wr, 612 op/s 2024-05-09T05:57:45.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.801+0000 7f2584710700 1 -- 172.21.15.115:0/1021870265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 msgr2=0x7f257c102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:45.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.801+0000 7f2584710700 1 --2- 172.21.15.115:0/1021870265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 0x7f257c102ba0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2578009990 tx=0x7f2578009ca0 comp rx=0 tx=0).stop 2024-05-09T05:57:45.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.801+0000 7f2584710700 1 -- 172.21.15.115:0/1021870265 shutdown_connections 2024-05-09T05:57:45.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.801+0000 7f2584710700 1 --2- 172.21.15.115:0/1021870265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257c1030e0 0x7f257c111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:45.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.801+0000 7f2584710700 1 --2- 172.21.15.115:0/1021870265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c109300 0x7f257c109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:45.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.801+0000 7f2584710700 1 --2- 172.21.15.115:0/1021870265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 0x7f257c102ba0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:45.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.801+0000 7f2584710700 1 -- 172.21.15.115:0/1021870265 >> 172.21.15.115:0/1021870265 conn(0x7f257c0fe520 msgr2=0x7f257c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:45.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.801+0000 7f2584710700 1 -- 172.21.15.115:0/1021870265 shutdown_connections 2024-05-09T05:57:45.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.801+0000 7f2584710700 1 -- 172.21.15.115:0/1021870265 wait complete. 2024-05-09T05:57:45.790 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:45 smithi115 ceph-mon[30349]: pgmap v618: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 2.3 MiB/s wr, 592 op/s 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.802+0000 7f2584710700 1 Processor -- start 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.802+0000 7f2584710700 1 -- start start 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.802+0000 7f2584710700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 0x7f257c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.802+0000 7f2584710700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257c1030e0 0x7f257c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.802+0000 7f2584710700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c109300 0x7f257c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f2584710700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f257c114450 con 0x7f257c102790 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f2584710700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1142d0 con 0x7f257c1030e0 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f2584710700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1145d0 con 0x7f257c109300 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25824ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 0x7f257c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:45.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25824ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 0x7f257c19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42436/0 (socket says 172.21.15.115:42436) 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25824ac700 1 -- 172.21.15.115:0/2777162483 learned_addr learned my addr 172.21.15.115:0/2777162483 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f2581cab700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257c1030e0 0x7f257c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f2582cad700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c109300 0x7f257c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25824ac700 1 -- 172.21.15.115:0/2777162483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c109300 msgr2=0x7f257c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25824ac700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c109300 0x7f257c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25824ac700 1 -- 172.21.15.115:0/2777162483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257c1030e0 msgr2=0x7f257c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25824ac700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257c1030e0 0x7f257c19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25824ac700 1 -- 172.21.15.115:0/2777162483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c19fcf0 con 0x7f257c102790 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f2582cad700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c109300 0x7f257c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25824ac700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 0x7f257c19ae20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d950 tx=0x7f256c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.803+0000 7f25737fe700 1 -- 172.21.15.115:0/2777162483 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c017070 con 0x7f257c102790 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.804+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257c19ffe0 con 0x7f257c102790 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.804+0000 7f25737fe700 1 -- 172.21.15.115:0/2777162483 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f256c009730 con 0x7f257c102790 2024-05-09T05:57:45.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.804+0000 7f25737fe700 1 -- 172.21.15.115:0/2777162483 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c003a00 con 0x7f257c102790 2024-05-09T05:57:45.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.804+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257c1a7900 con 0x7f257c102790 2024-05-09T05:57:45.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.805+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257c0512c0 con 0x7f257c102790 2024-05-09T05:57:45.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.806+0000 7f25737fe700 1 -- 172.21.15.115:0/2777162483 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f256c003ba0 con 0x7f257c102790 2024-05-09T05:57:45.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.807+0000 7f25737fe700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f25680b29f0 0x7f25680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:45.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.807+0000 7f2581cab700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f25680b29f0 0x7f25680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:45.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.807+0000 7f25737fe700 1 -- 172.21.15.115:0/2777162483 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9514+0+0 (secure 0 0 0) 0x7f256c0e2510 con 0x7f257c102790 2024-05-09T05:57:45.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.807+0000 7f2581cab700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f25680b29f0 0x7f25680b4f00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2578006010 tx=0x7f2578005960 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:45.812+0000 7f25737fe700 1 -- 172.21.15.115:0/2777162483 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f256c020030 con 0x7f257c102790 2024-05-09T05:57:45.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:45 smithi161 ceph-mon[33805]: pgmap v618: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 2.3 MiB/s wr, 592 op/s 2024-05-09T05:57:46.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.064+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f257c02e5f0 con 0x7f257c102790 2024-05-09T05:57:46.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.064+0000 7f25737fe700 1 -- 172.21.15.115:0/2777162483 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+11958 (secure 0 0 0) 0x7f256c0a9b10 con 0x7f257c102790 2024-05-09T05:57:46.052 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:57:46.052 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":72,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:57:40.589987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27646":{"gid":27646,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/197242519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":197242519},{"type":"v1","addr":"172.21.15.195:6833","nonce":197242519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:57:46.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f25680b29f0 msgr2=0x7f25680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:46.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f25680b29f0 0x7f25680b4f00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2578006010 tx=0x7f2578005960 comp rx=0 tx=0).stop 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 msgr2=0x7f257c19ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 0x7f257c19ae20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d950 tx=0x7f256c00dc60 comp rx=0 tx=0).stop 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 shutdown_connections 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f25680b29f0 0x7f25680b4f00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c109300 0x7f257c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f257c102790 0x7f257c19ae20 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 --2- 172.21.15.115:0/2777162483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f257c1030e0 0x7f257c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 >> 172.21.15.115:0/2777162483 conn(0x7f257c0fe520 msgr2=0x7f257c107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 shutdown_connections 2024-05-09T05:57:46.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:46.067+0000 7f2584710700 1 -- 172.21.15.115:0/2777162483 wait complete. 2024-05-09T05:57:46.056 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 72 2024-05-09T05:57:46.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:45 smithi195 ceph-mon[33844]: pgmap v618: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 2.3 MiB/s wr, 592 op/s 2024-05-09T05:57:46.682 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:46 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2777162483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:57:46.710 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id ac200e3d --release --await' 2024-05-09T05:57:46.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id ac200e3d --release --await 2024-05-09T05:57:46.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:46 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2777162483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:57:46.956 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:57:47.011 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:46 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2777162483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:57:47.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:47 smithi115 ceph-mon[30349]: pgmap v619: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 531 op/s 2024-05-09T05:57:47.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:47 smithi161 ceph-mon[33805]: pgmap v619: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 531 op/s 2024-05-09T05:57:48.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:47 smithi195 ceph-mon[33844]: pgmap v619: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 531 op/s 2024-05-09T05:57:48.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.771+0000 7f171688a700 1 -- 172.21.15.115:0/3194751231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17101027a0 msgr2=0x7f1710102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.771+0000 7f171688a700 1 --2- 172.21.15.115:0/3194751231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17101027a0 0x7f1710102bb0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f1704009990 tx=0x7f1704009ca0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.771+0000 7f171688a700 1 -- 172.21.15.115:0/3194751231 shutdown_connections 2024-05-09T05:57:48.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.771+0000 7f171688a700 1 --2- 172.21.15.115:0/3194751231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1710109310 0x7f1710109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.771+0000 7f171688a700 1 --2- 172.21.15.115:0/3194751231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17101030f0 0x7f1710111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.771+0000 7f171688a700 1 --2- 172.21.15.115:0/3194751231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17101027a0 0x7f1710102bb0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.771+0000 7f171688a700 1 -- 172.21.15.115:0/3194751231 >> 172.21.15.115:0/3194751231 conn(0x7f17100fe4f0 msgr2=0x7f1710100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:48.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.772+0000 7f171688a700 1 -- 172.21.15.115:0/3194751231 shutdown_connections 2024-05-09T05:57:48.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.772+0000 7f171688a700 1 -- 172.21.15.115:0/3194751231 wait complete. 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.772+0000 7f171688a700 1 Processor -- start 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.772+0000 7f171688a700 1 -- start start 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f171688a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17101027a0 0x7f171019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f171688a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17101030f0 0x7f171019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f171688a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1710109310 0x7f171019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f171688a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f17101144e0 con 0x7f1710109310 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f171688a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1710114360 con 0x7f17101027a0 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f171688a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1710114660 con 0x7f17101030f0 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f1714e27700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1710109310 0x7f171019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f1714e27700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1710109310 0x7f171019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42458/0 (socket says 172.21.15.115:42458) 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f1714e27700 1 -- 172.21.15.115:0/1929845192 learned_addr learned my addr 172.21.15.115:0/1929845192 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f170f7fe700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17101030f0 0x7f171019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f170ffff700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17101027a0 0x7f171019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f1714e27700 1 -- 172.21.15.115:0/1929845192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17101030f0 msgr2=0x7f171019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f1714e27700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17101030f0 0x7f171019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f1714e27700 1 -- 172.21.15.115:0/1929845192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17101027a0 msgr2=0x7f171019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f1714e27700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17101027a0 0x7f171019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f1714e27700 1 -- 172.21.15.115:0/1929845192 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171019fde0 con 0x7f1710109310 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f170ffff700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17101027a0 0x7f171019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f1714e27700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1710109310 0x7f171019f740 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f170800ada0 tx=0x7f170800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f170d7fa700 1 -- 172.21.15.115:0/1929845192 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708017020 con 0x7f1710109310 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f170d7fa700 1 -- 172.21.15.115:0/1929845192 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f170800ce10 con 0x7f1710109310 2024-05-09T05:57:48.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.773+0000 7f171688a700 1 -- 172.21.15.115:0/1929845192 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17101a00d0 con 0x7f1710109310 2024-05-09T05:57:48.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.774+0000 7f170d7fa700 1 -- 172.21.15.115:0/1929845192 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708005440 con 0x7f1710109310 2024-05-09T05:57:48.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.774+0000 7f171688a700 1 -- 172.21.15.115:0/1929845192 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17101a79f0 con 0x7f1710109310 2024-05-09T05:57:48.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.776+0000 7f170d7fa700 1 -- 172.21.15.115:0/1929845192 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f1708004ad0 con 0x7f1710109310 2024-05-09T05:57:48.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.777+0000 7f170d7fa700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f16f80b29f0 0x7f16f80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.777+0000 7f170ffff700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f16f80b29f0 0x7f16f80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.777+0000 7f170d7fa700 1 -- 172.21.15.115:0/1929845192 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9514+0+0 (secure 0 0 0) 0x7f17080e2220 con 0x7f1710109310 2024-05-09T05:57:48.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.777+0000 7f170ffff700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f16f80b29f0 0x7f16f80b4f00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f171006ca70 tx=0x7f1700008040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:48.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f17252a0b80 1 Processor -- start 2024-05-09T05:57:48.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f17252a0b80 1 -- start start 2024-05-09T05:57:48.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40c9ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 0x556f40c9efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 0x556f40c98f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f17252a0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556f40bbb0f0 con 0x556f40c05260 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f17252a0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556f40b598f0 con 0x556f40c98b50 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f17252a0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f40bb70b0 con 0x556f40b1b5a0 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f170ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40c9ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f170ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40c9ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42464/0 (socket says 172.21.15.115:42464) 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f170ffff700 1 -- 172.21.15.115:0/4276086754 learned_addr learned my addr 172.21.15.115:0/4276086754 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f1714e27700 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 0x556f40c98f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f170f7fe700 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 0x556f40c9efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f170ffff700 1 -- 172.21.15.115:0/4276086754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 msgr2=0x556f40c9efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f170ffff700 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 0x556f40c9efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f170ffff700 1 -- 172.21.15.115:0/4276086754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 msgr2=0x556f40c98f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:48.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.778+0000 7f170ffff700 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 0x556f40c98f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.779+0000 7f170ffff700 1 -- 172.21.15.115:0/4276086754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f40b51470 con 0x556f40c05260 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.779+0000 7f1714e27700 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 0x556f40c98f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.779+0000 7f170ffff700 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40c9ea90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f170000c910 tx=0x7f170000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284c4cc8b1597817 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.779+0000 7f16f67fc700 1 -- 172.21.15.115:0/4276086754 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700023070 con 0x556f40c05260 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.779+0000 7f16f67fc700 1 -- 172.21.15.115:0/4276086754 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1700013dc0 con 0x556f40c05260 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.779+0000 7f16f67fc700 1 -- 172.21.15.115:0/4276086754 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170001ed90 con 0x556f40c05260 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 -- 172.21.15.115:0/4276086754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 msgr2=0x556f40c9ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40c9ea90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f170000c910 tx=0x7f170000f000 comp rx=0 tx=0).stop 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 -- 172.21.15.115:0/4276086754 shutdown_connections 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 0x556f40c98f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40c9ea90 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 --2- 172.21.15.115:0/4276086754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 0x556f40c9efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 -- 172.21.15.115:0/4276086754 >> 172.21.15.115:0/4276086754 conn(0x556f40b7b7a0 msgr2=0x556f40c93a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:48.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 -- 172.21.15.115:0/4276086754 shutdown_connections 2024-05-09T05:57:48.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 -- 172.21.15.115:0/4276086754 wait complete. 2024-05-09T05:57:48.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 -- start start 2024-05-09T05:57:48.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.780+0000 7f17252a0b80 1 Processor -- start 2024-05-09T05:57:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40b1b5a0 0x556f40ca27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c05260 0x556f40ca2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c98b50 0x556f40ca3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f17252a0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556f40a36ca0 con 0x556f40b1b5a0 2024-05-09T05:57:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f17252a0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556f40b75250 con 0x556f40c05260 2024-05-09T05:57:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f17252a0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f40cae930 con 0x556f40c98b50 2024-05-09T05:57:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f17252a0b80 10 client.0 _ll_register_callbacks cb 0x556f40a2c7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:57:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40b1b5a0 0x556f40ca27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40b1b5a0 0x556f40ca27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42472/0 (socket says 172.21.15.115:42472) 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170ffff700 1 -- 172.21.15.115:0/3151781222 learned_addr learned my addr 172.21.15.115:0/3151781222 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f1714e27700 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c98b50 0x556f40ca3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170f7fe700 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c05260 0x556f40ca2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170ffff700 1 -- 172.21.15.115:0/3151781222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c98b50 msgr2=0x556f40ca3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170ffff700 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c98b50 0x556f40ca3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170ffff700 1 -- 172.21.15.115:0/3151781222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c05260 msgr2=0x556f40ca2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170ffff700 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c05260 0x556f40ca2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170ffff700 1 -- 172.21.15.115:0/3151781222 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f40a36590 con 0x556f40b1b5a0 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170f7fe700 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c05260 0x556f40ca2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f170ffff700 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40b1b5a0 0x556f40ca27e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1700021040 tx=0x7f170001eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548b6da9c96d0e76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f16ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f16ef7fe700 1 -- 172.21.15.115:0/3151781222 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170002c070 con 0x556f40b1b5a0 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f16ef7fe700 1 -- 172.21.15.115:0/3151781222 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9514+0+0 (secure 0 0 0) 0x7f1700013730 con 0x556f40b1b5a0 2024-05-09T05:57:48.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.781+0000 7f17252a0b80 1 -- 172.21.15.115:0/3151781222 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1700013dc0 con 0x556f40b1b5a0 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f16ef7fe700 1 -- 172.21.15.115:0/3151781222 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f1700038050 con 0x556f40b1b5a0 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f17252a0b80 10 client.19077 fetch_fsmap learned FSMap version 72 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f17252a0b80 1 -- 172.21.15.115:0/3151781222 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x556f40cb02d0 con 0x556f40b1b5a0 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f16ef7fe700 1 -- 172.21.15.115:0/3151781222 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1700013dc0 con 0x556f40b1b5a0 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f16ef7fe700 1 -- 172.21.15.115:0/3151781222 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 72) ==== 8382+0+0 (secure 0 0 0) 0x7f1700028860 con 0x556f40b1b5a0 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f17252a0b80 10 client.19077 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f17252a0b80 10 client.19077 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f17252a0b80 20 client.19077 populate_metadata read hostname 'smithi115' 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f40cb27a0 0x556f40cb4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f17252a0b80 4 client.19077 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f17252a0b80 1 -- 172.21.15.115:0/3151781222 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f40bc8ad0 con 0x556f40cb27a0 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.782+0000 7f170f7fe700 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f40cb27a0 0x556f40cb4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.783+0000 7f170f7fe700 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f40cb27a0 0x556f40cb4bc0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=811a28bc346a4a50 server_cookie=81eac562b0e0c6c7 in_seq=0 out_seq=0 2024-05-09T05:57:48.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.783+0000 7f16ef7fe700 10 client.19077 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f16ef7fe700 1 -- 172.21.15.115:0/3151781222 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f40bc8ad0 con 0x556f40cb27a0 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f16ef7fe700 10 client.19077 handle_command_reply: tid=0 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 client.19077 shutdown 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 -- 172.21.15.115:0/3151781222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40b1b5a0 msgr2=0x556f40ca27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40b1b5a0 0x556f40ca27e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1700021040 tx=0x7f170001eb20 comp rx=0 tx=0).stop 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 -- 172.21.15.115:0/3151781222 shutdown_connections 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f40cb27a0 0x556f40cb4bc0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c98b50 0x556f40ca3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40b1b5a0 0x556f40ca27e0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3151781222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c05260 0x556f40ca2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f16ef7fe700 0 client.19077 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:48.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 -- 172.21.15.115:0/3151781222 >> 172.21.15.115:0/3151781222 conn(0x556f40b7b7a0 msgr2=0x556f40c93750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:48.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 -- 172.21.15.115:0/3151781222 shutdown_connections 2024-05-09T05:57:48.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 1 -- 172.21.15.115:0/3151781222 wait complete. 2024-05-09T05:57:48.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.784+0000 7f17252a0b80 20 client.19077 trim_cache size 0 max 16384 2024-05-09T05:57:48.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.803+0000 7f17252a0b80 1 Processor -- start 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.803+0000 7f17252a0b80 1 -- start start 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40b1b5a0 0x556f40ca1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c05260 0x556f40c9cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c98b50 0x556f40cba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f17252a0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f17101b0710 con 0x556f40c98b50 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f17252a0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f17101b22b0 con 0x556f40b1b5a0 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f17252a0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17101b0de0 con 0x556f40c05260 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f1714e27700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c98b50 0x556f40cba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f1714e27700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c98b50 0x556f40cba180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42486/0 (socket says 172.21.15.115:42486) 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f1714e27700 1 -- 172.21.15.115:0/3958070418 learned_addr learned my addr 172.21.15.115:0/3958070418 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f170ffff700 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40b1b5a0 0x556f40ca1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f170f7fe700 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c05260 0x556f40c9cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f1714e27700 1 -- 172.21.15.115:0/3958070418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c05260 msgr2=0x556f40c9cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f1714e27700 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c05260 0x556f40c9cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f1714e27700 1 -- 172.21.15.115:0/3958070418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40b1b5a0 msgr2=0x556f40ca1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f1714e27700 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40b1b5a0 0x556f40ca1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f1714e27700 1 -- 172.21.15.115:0/3958070418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f40c992a0 con 0x556f40c98b50 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f170ffff700 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40b1b5a0 0x556f40ca1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f170f7fe700 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c05260 0x556f40c9cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.804+0000 7f1714e27700 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c98b50 0x556f40cba180 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f17080a9140 tx=0x7f17080a8d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63b5b410cdd6be7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f16f5ffb700 1 -- 172.21.15.115:0/3958070418 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17080ad560 con 0x556f40c98b50 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f16f5ffb700 1 -- 172.21.15.115:0/3958070418 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17080b1f80 con 0x556f40c98b50 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f16f5ffb700 1 -- 172.21.15.115:0/3958070418 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17080b1af0 con 0x556f40c98b50 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f17252a0b80 1 -- 172.21.15.115:0/3958070418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c98b50 msgr2=0x556f40cba180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c98b50 0x556f40cba180 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f17080a9140 tx=0x7f17080a8d90 comp rx=0 tx=0).stop 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f17252a0b80 1 -- 172.21.15.115:0/3958070418 shutdown_connections 2024-05-09T05:57:48.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40b1b5a0 0x556f40ca1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c98b50 0x556f40cba180 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f17252a0b80 1 --2- 172.21.15.115:0/3958070418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40c05260 0x556f40c9cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f17252a0b80 1 -- 172.21.15.115:0/3958070418 >> 172.21.15.115:0/3958070418 conn(0x556f40b7b7a0 msgr2=0x556f40c9c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f17252a0b80 1 -- 172.21.15.115:0/3958070418 shutdown_connections 2024-05-09T05:57:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.805+0000 7f17252a0b80 1 -- 172.21.15.115:0/3958070418 wait complete. 2024-05-09T05:57:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f17252a0b80 1 -- start start 2024-05-09T05:57:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f17252a0b80 1 Processor -- start 2024-05-09T05:57:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 0x556f40caab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40ca7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f17252a0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 0x556f40ca7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f17252a0b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f17101b1be0 con 0x556f40c05260 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f17252a0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f17101b14e0 con 0x556f40c98b50 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f17252a0b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f17101ae6d0 con 0x556f40b1b5a0 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f17252a0b80 10 client.0 _ll_register_callbacks cb 0x556f40a2c7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f170f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40ca7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f170f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40ca7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42498/0 (socket says 172.21.15.115:42498) 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f170f7fe700 1 -- 172.21.15.115:0/2473603309 learned_addr learned my addr 172.21.15.115:0/2473603309 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f1714e27700 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 0x556f40ca7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.806+0000 7f170ffff700 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 0x556f40caab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f170f7fe700 1 -- 172.21.15.115:0/2473603309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 msgr2=0x556f40caab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f170f7fe700 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 0x556f40caab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f170f7fe700 1 -- 172.21.15.115:0/2473603309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 msgr2=0x556f40ca7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f170f7fe700 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 0x556f40ca7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f170f7fe700 1 -- 172.21.15.115:0/2473603309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f40ca82e0 con 0x556f40c05260 2024-05-09T05:57:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f170f7fe700 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40ca7830 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f171006cc90 tx=0x7f1704005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e48b636767b190 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f16ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f16ef7fe700 1 -- 172.21.15.115:0/2473603309 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704029070 con 0x556f40c05260 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f16ef7fe700 1 -- 172.21.15.115:0/2473603309 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9514+0+0 (secure 0 0 0) 0x7f17040039c0 con 0x556f40c05260 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f17252a0b80 1 -- 172.21.15.115:0/2473603309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17080b1f80 con 0x556f40c05260 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f16ef7fe700 1 -- 172.21.15.115:0/2473603309 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f1704035050 con 0x556f40c05260 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f17252a0b80 10 client.19083 fetch_fsmap learned FSMap version 72 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f17252a0b80 1 -- 172.21.15.115:0/2473603309 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x556f40d1da30 con 0x556f40c05260 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f16ef7fe700 1 -- 172.21.15.115:0/2473603309 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17080b1f80 con 0x556f40c05260 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f16ef7fe700 1 -- 172.21.15.115:0/2473603309 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 72) ==== 8382+0+0 (secure 0 0 0) 0x7f170400b770 con 0x556f40c05260 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f17252a0b80 10 client.19083 fetch_fsmap finished waiting for FSMap version 72 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f17252a0b80 10 client.19083 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f17252a0b80 20 client.19083 populate_metadata read hostname 'smithi115' 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.807+0000 7f17252a0b80 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f40d1e1a0 0x556f40d205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.808+0000 7f17252a0b80 4 client.19083 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ac200e3d", "release": true, "await": true}] 2024-05-09T05:57:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.808+0000 7f17252a0b80 1 -- 172.21.15.115:0/2473603309 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ac200e3d", "release": true, "await": true}) -- 0x556f40bc2b70 con 0x556f40d1e1a0 2024-05-09T05:57:48.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.808+0000 7f170ffff700 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f40d1e1a0 0x556f40d205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:57:48.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.808+0000 7f16ef7fe700 10 client.19083 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:48.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.808+0000 7f170ffff700 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f40d1e1a0 0x556f40d205c0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb56a872bf7f80c server_cookie=dab65c928a192aae in_seq=0 out_seq=0 2024-05-09T05:57:48.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f16ef7fe700 1 -- 172.21.15.115:0/2473603309 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556f40bc2b70 con 0x556f40d1e1a0 2024-05-09T05:57:48.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f16ef7fe700 10 client.19083 handle_command_reply: tid=0 2024-05-09T05:57:48.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 client.19083 shutdown 2024-05-09T05:57:48.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 -- 172.21.15.115:0/2473603309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 msgr2=0x556f40ca7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40ca7830 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f171006cc90 tx=0x7f1704005390 comp rx=0 tx=0).stop 2024-05-09T05:57:48.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 -- 172.21.15.115:0/2473603309 shutdown_connections 2024-05-09T05:57:48.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f40c98b50 0x556f40ca7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f40d1e1a0 0x556f40d205c0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f40b1b5a0 0x556f40caab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 --2- 172.21.15.115:0/2473603309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f40c05260 0x556f40ca7830 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 -- 172.21.15.115:0/2473603309 >> 172.21.15.115:0/2473603309 conn(0x556f40b7b7a0 msgr2=0x556f40ca1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f16ef7fe700 0 client.19083 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.832+0000 7f17252a0b80 1 -- 172.21.15.115:0/2473603309 shutdown_connections 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.833+0000 7f17252a0b80 1 -- 172.21.15.115:0/2473603309 wait complete. 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.833+0000 7f17252a0b80 20 client.19083 trim_cache size 0 max 16384 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 72, 2024-05-09T05:57:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:57:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 52, 2024-05-09T05:57:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:57:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "ac200e3d": { 2024-05-09T05:57:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 52, 2024-05-09T05:57:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:57:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 141.3, 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:57:48.824 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.833+0000 7f171688a700 1 -- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f16f80b29f0 msgr2=0x7f16f80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.833+0000 7f171688a700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f16f80b29f0 0x7f16f80b4f00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f171006ca70 tx=0x7f1700008040 comp rx=0 tx=0).stop 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.833+0000 7f171688a700 1 -- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1710109310 msgr2=0x7f171019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.833+0000 7f171688a700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1710109310 0x7f171019f740 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f170800ada0 tx=0x7f170800c330 comp rx=0 tx=0).stop 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.834+0000 7f171688a700 1 -- 172.21.15.115:0/1929845192 shutdown_connections 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.834+0000 7f171688a700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f16f80b29f0 0x7f16f80b4f00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.834+0000 7f171688a700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17101027a0 0x7f171019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.834+0000 7f171688a700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1710109310 0x7f171019f740 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.834+0000 7f171688a700 1 --2- 172.21.15.115:0/1929845192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17101030f0 0x7f171019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.834+0000 7f171688a700 1 -- 172.21.15.115:0/1929845192 >> 172.21.15.115:0/1929845192 conn(0x7f17100fe4f0 msgr2=0x7f1710107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.834+0000 7f171688a700 1 -- 172.21.15.115:0/1929845192 shutdown_connections 2024-05-09T05:57:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:57:48.834+0000 7f171688a700 1 -- 172.21.15.115:0/1929845192 wait complete. 2024-05-09T05:57:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.834+0000 7fcc39ffb640 1 -- 172.21.15.115:0/955253736 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5651ff190240 con 0x5651fee141c0 2024-05-09T05:57:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.834+0000 7fcc39ffb640 10 client.28378 handle_command_reply: tid=0 2024-05-09T05:57:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.834+0000 7fcc5f663740 1 client.28378 shutdown 2024-05-09T05:57:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee572f0 msgr2=0x5651ff232740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee572f0 0x5651ff232740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc40011ea0 tx=0x7fcc40003db0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 shutdown_connections 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651fee801b0 0x5651ff2331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5651fee5f4d0 0x5651ff232c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5651fee141c0 0x5651ff2a9e60 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 --2- 172.21.15.115:0/955253736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5651fee572f0 0x5651ff232740 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc39ffb640 0 client.28378 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 >> 172.21.15.115:0/955253736 conn(0x5651fee74300 msgr2=0x5651ff2272d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 shutdown_connections 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 1 -- 172.21.15.115:0/955253736 wait complete. 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.835+0000 7fcc5f663740 20 client.28378 trim_cache size 0 max 16384 2024-05-09T05:57:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:57:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:57:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:57:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 -- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc3c0b2a40 msgr2=0x7fcc3c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc3c0b2a40 0x7fcc3c0b4f50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcc480058c0 tx=0x7fcc480abab0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 -- 172.21.15.115:0/2031738380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5010a200 msgr2=0x7fcc501a9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5010a200 0x7fcc501a9530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800e9c0 tx=0x7fcc4800ee80 comp rx=0 tx=0).stop 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 -- 172.21.15.115:0/2031738380 shutdown_connections 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fcc3c0b2a40 0x7fcc3c0b4f50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc5011d250 0x7fcc501add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc5011a8f0 0x7fcc501a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 --2- 172.21.15.115:0/2031738380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc5010a200 0x7fcc501a9530 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 -- 172.21.15.115:0/2031738380 >> 172.21.15.115:0/2031738380 conn(0x7fcc5006e900 msgr2=0x7fcc5011ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 -- 172.21.15.115:0/2031738380 shutdown_connections 2024-05-09T05:57:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:57:48.836+0000 7fcc587b2640 1 -- 172.21.15.115:0/2031738380 wait complete. 2024-05-09T05:57:49.662 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ac200e3d', total seconds elapsed: 75.3 2024-05-09T05:57:49.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:49 smithi115 ceph-mon[30349]: pgmap v620: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 569 KiB/s wr, 147 op/s 2024-05-09T05:57:49.894 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:49 smithi115 ceph-mon[30349]: osdmap e118: 12 total, 12 up, 12 in 2024-05-09T05:57:49.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:49 smithi161 ceph-mon[33805]: pgmap v620: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 569 KiB/s wr, 147 op/s 2024-05-09T05:57:49.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:49 smithi161 ceph-mon[33805]: osdmap e118: 12 total, 12 up, 12 in 2024-05-09T05:57:50.109 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:49 smithi195 ceph-mon[33844]: pgmap v620: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 569 KiB/s wr, 147 op/s 2024-05-09T05:57:50.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:49 smithi195 ceph-mon[33844]: osdmap e118: 12 total, 12 up, 12 in 2024-05-09T05:57:51.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:51 smithi161 ceph-mon[33805]: pgmap v622: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 348 KiB/s wr, 97 op/s 2024-05-09T05:57:51.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:51 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:57:52.109 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:51 smithi195 ceph-mon[33844]: pgmap v622: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 348 KiB/s wr, 97 op/s 2024-05-09T05:57:52.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:51 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:57:52.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:51 smithi115 ceph-mon[30349]: pgmap v622: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 348 KiB/s wr, 97 op/s 2024-05-09T05:57:52.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:51 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:57:53.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:53 smithi161 ceph-mon[33805]: pgmap v623: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 509 KiB/s wr, 136 op/s 2024-05-09T05:57:54.109 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:53 smithi195 ceph-mon[33844]: pgmap v623: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 509 KiB/s wr, 136 op/s 2024-05-09T05:57:54.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:53 smithi115 ceph-mon[30349]: pgmap v623: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 509 KiB/s wr, 136 op/s 2024-05-09T05:57:56.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:55 smithi195 ceph-mon[33844]: pgmap v624: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.0 MiB/s wr, 739 op/s 2024-05-09T05:57:56.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:55 smithi115 ceph-mon[30349]: pgmap v624: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.0 MiB/s wr, 739 op/s 2024-05-09T05:57:56.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:55 smithi161 ceph-mon[33805]: pgmap v624: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.0 MiB/s wr, 739 op/s 2024-05-09T05:57:58.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:57 smithi195 ceph-mon[33844]: pgmap v625: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 855 op/s 2024-05-09T05:57:58.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:57 smithi115 ceph-mon[30349]: pgmap v625: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 855 op/s 2024-05-09T05:57:58.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:57 smithi161 ceph-mon[33805]: pgmap v625: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 855 op/s 2024-05-09T05:58:00.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:57:59 smithi195 ceph-mon[33844]: pgmap v626: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.6 MiB/s wr, 903 op/s 2024-05-09T05:58:00.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:57:59 smithi115 ceph-mon[30349]: pgmap v626: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.6 MiB/s wr, 903 op/s 2024-05-09T05:58:00.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:57:59 smithi161 ceph-mon[33805]: pgmap v626: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.6 MiB/s wr, 903 op/s 2024-05-09T05:58:01.943 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:01 smithi115 ceph-mon[30349]: pgmap v627: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 901 op/s 2024-05-09T05:58:01.943 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:01 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:58:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.001+0000 7f26af32b640 1 -- 172.21.15.115:0/587396013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26a00afc10 msgr2=0x7f26a00a97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.001+0000 7f26af32b640 1 --2- 172.21.15.115:0/587396013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26a00afc10 0x7f26a00a97b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009900 tx=0x7f26a402a410 comp rx=0 tx=0).stop 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.002+0000 7f26af32b640 1 -- 172.21.15.115:0/587396013 shutdown_connections 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.002+0000 7f26af32b640 1 --2- 172.21.15.115:0/587396013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a00aa360 0x7f26a00b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.002+0000 7f26af32b640 1 --2- 172.21.15.115:0/587396013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00b00b0 0x7f26a00a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.002+0000 7f26af32b640 1 --2- 172.21.15.115:0/587396013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26a00afc10 0x7f26a00a97b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.002+0000 7f26af32b640 1 -- 172.21.15.115:0/587396013 >> 172.21.15.115:0/587396013 conn(0x7f26a00a5290 msgr2=0x7f26a00a76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.002+0000 7f26af32b640 1 -- 172.21.15.115:0/587396013 shutdown_connections 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.002+0000 7f26af32b640 1 -- 172.21.15.115:0/587396013 wait complete. 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.002+0000 7f26af32b640 1 Processor -- start 2024-05-09T05:58:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26af32b640 1 -- start start 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26af32b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00aa360 0x7f26a01407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26af32b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26a00afc10 0x7f26a0140d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26af32b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a00b00b0 0x7f26a0144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26af32b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f26a00b9cb0 con 0x7f26a00afc10 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26af32b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f26a00b9b30 con 0x7f26a00aa360 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26af32b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f26a00b9e30 con 0x7f26a00b00b0 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26adb28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26a00afc10 0x7f26a0140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26ae329640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00aa360 0x7f26a01407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26adb28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26a00afc10 0x7f26a0140d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45396/0 (socket says 172.21.15.115:45396) 2024-05-09T05:58:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26adb28640 1 -- 172.21.15.115:0/220953970 learned_addr learned my addr 172.21.15.115:0/220953970 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26aeb2a640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a00b00b0 0x7f26a0144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26ae329640 1 -- 172.21.15.115:0/220953970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a00b00b0 msgr2=0x7f26a0144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26ae329640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a00b00b0 0x7f26a0144fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26ae329640 1 -- 172.21.15.115:0/220953970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26a00afc10 msgr2=0x7f26a0140d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26ae329640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26a00afc10 0x7f26a0140d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.003+0000 7f26ae329640 1 -- 172.21.15.115:0/220953970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a0145610 con 0x7f26a00aa360 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.004+0000 7f26ae329640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00aa360 0x7f26a01407b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009230 tx=0x7f26a4005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.004+0000 7f269f7fe640 1 -- 172.21.15.115:0/220953970 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a403f070 con 0x7f26a00aa360 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.004+0000 7f26af32b640 1 -- 172.21.15.115:0/220953970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a01458a0 con 0x7f26a00aa360 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.004+0000 7f269f7fe640 1 -- 172.21.15.115:0/220953970 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26a4003a80 con 0x7f26a00aa360 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.004+0000 7f269f7fe640 1 -- 172.21.15.115:0/220953970 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4035070 con 0x7f26a00aa360 2024-05-09T05:58:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.004+0000 7f26af32b640 1 -- 172.21.15.115:0/220953970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a0145db0 con 0x7f26a00aa360 2024-05-09T05:58:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.007+0000 7f26b7446740 1 Processor -- start 2024-05-09T05:58:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26b7446740 1 -- start start 2024-05-09T05:58:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f269f7fe640 1 -- 172.21.15.115:0/220953970 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f26a4043400 con 0x7f26a00aa360 2024-05-09T05:58:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26b7446740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 0x55b950af0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26b7446740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950af0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26b7446740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95072c4d0 0x55b950af12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26b7446740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b950a557c0 con 0x55b9507242f0 2024-05-09T05:58:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26b7446740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b9509f10e0 con 0x55b95074d1b0 2024-05-09T05:58:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26b7446740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b950a51780 con 0x55b95072c4d0 2024-05-09T05:58:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26ae329640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950af0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26ae329640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950af0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45398/0 (socket says 172.21.15.115:45398) 2024-05-09T05:58:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.008+0000 7f26ae329640 1 -- 172.21.15.115:0/92312566 learned_addr learned my addr 172.21.15.115:0/92312566 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f269f7fe640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f26940b2960 0x7f26940b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f26aeb2a640 1 --2- 172.21.15.115:0/92312566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 0x55b950af0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f269f7fe640 1 -- 172.21.15.115:0/220953970 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9470+0+0 (secure 0 0 0) 0x7f26a4102a90 con 0x7f26a00aa360 2024-05-09T05:58:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f26ae329640 1 -- 172.21.15.115:0/92312566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95072c4d0 msgr2=0x55b950af12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f26ae329640 1 --2- 172.21.15.115:0/92312566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95072c4d0 0x55b950af12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f26ae329640 1 -- 172.21.15.115:0/92312566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 msgr2=0x55b950af0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f26ae329640 1 --2- 172.21.15.115:0/92312566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 0x55b950af0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f26ae329640 1 -- 172.21.15.115:0/92312566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b950a1a7c0 con 0x55b9507242f0 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f26aeb2a640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f26940b2960 0x7f26940b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.009+0000 7f26ae329640 1 --2- 172.21.15.115:0/92312566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950af0d80 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f26a40c6ce0 tx=0x7f26a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da22ddf2e5c1829 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.010+0000 7f269cbfe640 1 -- 172.21.15.115:0/92312566 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4048070 con 0x55b9507242f0 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.010+0000 7f269cbfe640 1 -- 172.21.15.115:0/92312566 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26a410a040 con 0x55b9507242f0 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.010+0000 7f269cbfe640 1 -- 172.21.15.115:0/92312566 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a40d2170 con 0x55b9507242f0 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.011+0000 7f26b7446740 1 -- 172.21.15.115:0/92312566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 msgr2=0x55b950af0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.011+0000 7f26aeb2a640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f26940b2960 0x7f26940b4e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26a8051dc0 tx=0x7f26a8064250 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.011+0000 7f26b7446740 1 --2- 172.21.15.115:0/92312566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950af0d80 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f26a40c6ce0 tx=0x7f26a4009260 comp rx=0 tx=0).stop 2024-05-09T05:58:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.011+0000 7f26b7446740 1 -- 172.21.15.115:0/92312566 shutdown_connections 2024-05-09T05:58:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.011+0000 7f26b7446740 1 --2- 172.21.15.115:0/92312566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95072c4d0 0x55b950af12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.011+0000 7f26b7446740 1 --2- 172.21.15.115:0/92312566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950af0d80 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.011+0000 7f26b7446740 1 --2- 172.21.15.115:0/92312566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 0x55b950af0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.011+0000 7f26b7446740 1 -- 172.21.15.115:0/92312566 >> 172.21.15.115:0/92312566 conn(0x55b950741300 msgr2=0x55b950aecff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.012+0000 7f26b7446740 1 -- 172.21.15.115:0/92312566 shutdown_connections 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.012+0000 7f26b7446740 1 -- 172.21.15.115:0/92312566 wait complete. 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.012+0000 7f26b7446740 1 -- start start 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.012+0000 7f26b7446740 1 Processor -- start 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26b7446740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9507242f0 0x55b950afc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26b7446740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95072c4d0 0x55b950afce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26b7446740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 0x55b950afad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26b7446740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b950afd730 con 0x55b95072c4d0 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26b7446740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b9509dc9c0 con 0x55b95074d1b0 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26b7446740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b950afd8b0 con 0x55b9507242f0 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26adb28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95072c4d0 0x55b950afce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26b7446740 10 client.0 _ll_register_callbacks cb 0x55b950a09020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26adb28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95072c4d0 0x55b950afce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45406/0 (socket says 172.21.15.115:45406) 2024-05-09T05:58:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26adb28640 1 -- 172.21.15.115:0/804516890 learned_addr learned my addr 172.21.15.115:0/804516890 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26aeb2a640 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 0x55b950afad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26ae329640 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9507242f0 0x55b950afc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26adb28640 1 -- 172.21.15.115:0/804516890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9507242f0 msgr2=0x55b950afc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26adb28640 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9507242f0 0x55b950afc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26adb28640 1 -- 172.21.15.115:0/804516890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 msgr2=0x55b950afad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26adb28640 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 0x55b950afad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.013+0000 7f26adb28640 1 -- 172.21.15.115:0/804516890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b950a031f0 con 0x55b95072c4d0 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.014+0000 7f26adb28640 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95072c4d0 0x55b950afce20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f2698004700 tx=0x7f2698011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dd60c68884d603 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.014+0000 7f2691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.014+0000 7f2691ffb640 1 -- 172.21.15.115:0/804516890 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269800fd30 con 0x55b95072c4d0 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.014+0000 7f2691ffb640 1 -- 172.21.15.115:0/804516890 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9470+0+0 (secure 0 0 0) 0x7f2698004030 con 0x55b95072c4d0 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.014+0000 7f26b7446740 1 -- 172.21.15.115:0/804516890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a410a040 con 0x55b95072c4d0 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.014+0000 7f2691ffb640 1 -- 172.21.15.115:0/804516890 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f2698024050 con 0x55b95072c4d0 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.014+0000 7f26b7446740 10 client.19095 fetch_fsmap learned FSMap version 73 2024-05-09T05:58:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.014+0000 7f26b7446740 1 -- 172.21.15.115:0/804516890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55b9507b8790 con 0x55b95072c4d0 2024-05-09T05:58:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.014+0000 7f2691ffb640 1 -- 172.21.15.115:0/804516890 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26a410a040 con 0x55b95072c4d0 2024-05-09T05:58:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.015+0000 7f2691ffb640 1 -- 172.21.15.115:0/804516890 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 73) ==== 8386+0+0 (secure 0 0 0) 0x7f2698019420 con 0x55b95072c4d0 2024-05-09T05:58:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.015+0000 7f26b7446740 10 client.19095 fetch_fsmap finished waiting for FSMap version 73 2024-05-09T05:58:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.015+0000 7f26b7446740 10 client.19095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.015+0000 7f26b7446740 20 client.19095 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.015+0000 7f26b7446740 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b95070bd80 0x55b950b0cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.015+0000 7f26b7446740 4 client.19095 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:58:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.015+0000 7f26b7446740 1 -- 172.21.15.115:0/804516890 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b950a633d0 con 0x55b95070bd80 2024-05-09T05:58:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.017+0000 7f26ae329640 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b95070bd80 0x55b950b0cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.017+0000 7f26ae329640 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b95070bd80 0x55b950b0cf70 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ab16bf5e1bf863 server_cookie=d05963e7f7f9ff44 in_seq=0 out_seq=0 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.017+0000 7f2691ffb640 10 client.19095 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.018+0000 7f2691ffb640 1 -- 172.21.15.115:0/804516890 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b950a633d0 con 0x55b95070bd80 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.018+0000 7f2691ffb640 10 client.19095 handle_command_reply: tid=0 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 client.19095 shutdown 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 -- 172.21.15.115:0/804516890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95072c4d0 msgr2=0x55b950afce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95072c4d0 0x55b950afce20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f2698004700 tx=0x7f2698011b50 comp rx=0 tx=0).stop 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 -- 172.21.15.115:0/804516890 shutdown_connections 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95074d1b0 0x55b950afad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95072c4d0 0x55b950afce20 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b95070bd80 0x55b950b0cf70 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 --2- 172.21.15.115:0/804516890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9507242f0 0x55b950afc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 -- 172.21.15.115:0/804516890 >> 172.21.15.115:0/804516890 conn(0x55b950741300 msgr2=0x55b950aeb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f2691ffb640 0 client.19095 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 -- 172.21.15.115:0/804516890 shutdown_connections 2024-05-09T05:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 1 -- 172.21.15.115:0/804516890 wait complete. 2024-05-09T05:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.019+0000 7f26b7446740 20 client.19095 trim_cache size 0 max 16384 2024-05-09T05:58:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.048+0000 7f26b7446740 1 Processor -- start 2024-05-09T05:58:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.048+0000 7f26b7446740 1 -- start start 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26b7446740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9507242f0 0x55b950b02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26b7446740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95072c4d0 0x55b950b03040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26b7446740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95074d1b0 0x55b950afad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26b7446740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0157200 con 0x55b95074d1b0 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26b7446740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0158da0 con 0x55b9507242f0 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26b7446740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01578d0 con 0x55b95072c4d0 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26aeb2a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95074d1b0 0x55b950afad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26aeb2a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95074d1b0 0x55b950afad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45412/0 (socket says 172.21.15.115:45412) 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26aeb2a640 1 -- 172.21.15.115:0/791531263 learned_addr learned my addr 172.21.15.115:0/791531263 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.049+0000 7f26ae329640 1 --2- 172.21.15.115:0/791531263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9507242f0 0x55b950b02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.050+0000 7f26aeb2a640 1 -- 172.21.15.115:0/791531263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95072c4d0 msgr2=0x55b950b03040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:58:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.050+0000 7f26aeb2a640 1 --2- 172.21.15.115:0/791531263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95072c4d0 0x55b950b03040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.050+0000 7f26aeb2a640 1 -- 172.21.15.115:0/791531263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9507242f0 msgr2=0x55b950b02b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.050+0000 7f26aeb2a640 1 --2- 172.21.15.115:0/791531263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9507242f0 0x55b950b02b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.050+0000 7f26aeb2a640 1 -- 172.21.15.115:0/791531263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b95082f5f0 con 0x55b95074d1b0 2024-05-09T05:58:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.050+0000 7f26aeb2a640 1 --2- 172.21.15.115:0/791531263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95074d1b0 0x55b950afad40 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f26a8068b90 tx=0x7f26a8069110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b2913a07f16d41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.050+0000 7f2693fff640 1 -- 172.21.15.115:0/791531263 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8078e10 con 0x55b95074d1b0 2024-05-09T05:58:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.050+0000 7f2693fff640 1 -- 172.21.15.115:0/791531263 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26a8084d70 con 0x55b95074d1b0 2024-05-09T05:58:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.050+0000 7f2693fff640 1 -- 172.21.15.115:0/791531263 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8076b00 con 0x55b95074d1b0 2024-05-09T05:58:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.052+0000 7f26b7446740 1 -- 172.21.15.115:0/791531263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95074d1b0 msgr2=0x55b950afad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.052+0000 7f26b7446740 1 --2- 172.21.15.115:0/791531263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95074d1b0 0x55b950afad40 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f26a8068b90 tx=0x7f26a8069110 comp rx=0 tx=0).stop 2024-05-09T05:58:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.052+0000 7f26b7446740 1 -- 172.21.15.115:0/791531263 shutdown_connections 2024-05-09T05:58:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.052+0000 7f26b7446740 1 --2- 172.21.15.115:0/791531263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b95074d1b0 0x55b950afad40 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.052+0000 7f26b7446740 1 --2- 172.21.15.115:0/791531263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95072c4d0 0x55b950b03040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.052+0000 7f26b7446740 1 --2- 172.21.15.115:0/791531263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9507242f0 0x55b950b02b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.052+0000 7f26b7446740 1 -- 172.21.15.115:0/791531263 >> 172.21.15.115:0/791531263 conn(0x55b950741300 msgr2=0x55b950af9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.053+0000 7f26b7446740 1 -- 172.21.15.115:0/791531263 shutdown_connections 2024-05-09T05:58:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.056+0000 7f26b7446740 1 -- 172.21.15.115:0/791531263 wait complete. 2024-05-09T05:58:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.056+0000 7f26b7446740 1 -- start start 2024-05-09T05:58:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.056+0000 7f26b7446740 1 Processor -- start 2024-05-09T05:58:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26b7446740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950affde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26b7446740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95072c4d0 0x55b950b00320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26b7446740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95074d1b0 0x55b950b6d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26b7446740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01586d0 con 0x55b9507242f0 2024-05-09T05:58:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26b7446740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0157fd0 con 0x55b95072c4d0 2024-05-09T05:58:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26b7446740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0155200 con 0x55b95074d1b0 2024-05-09T05:58:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26b7446740 10 client.0 _ll_register_callbacks cb 0x7f2698009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26adb28640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95072c4d0 0x55b950b00320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26ae329640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950affde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26adb28640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95072c4d0 0x55b950b00320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:59564/0 (socket says 172.21.15.115:59564) 2024-05-09T05:58:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26adb28640 1 -- 172.21.15.115:0/394465681 learned_addr learned my addr 172.21.15.115:0/394465681 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.057+0000 7f26aeb2a640 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95074d1b0 0x55b950b6d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.058+0000 7f26adb28640 1 -- 172.21.15.115:0/394465681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95074d1b0 msgr2=0x55b950b6d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.058+0000 7f26adb28640 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95074d1b0 0x55b950b6d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.058+0000 7f26adb28640 1 -- 172.21.15.115:0/394465681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 msgr2=0x55b950affde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.058+0000 7f26adb28640 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950affde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.058+0000 7f26adb28640 1 -- 172.21.15.115:0/394465681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b950a19ed0 con 0x55b95072c4d0 2024-05-09T05:58:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.058+0000 7f26adb28640 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95072c4d0 0x55b950b00320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f26980199d0 tx=0x7f269800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b245400977cc234c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.058+0000 7f2691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:58:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.059+0000 7f2691ffb640 1 -- 172.21.15.115:0/394465681 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269800fb30 con 0x55b95072c4d0 2024-05-09T05:58:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.059+0000 7f2691ffb640 1 -- 172.21.15.115:0/394465681 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9470+0+0 (secure 0 0 0) 0x7f269800fcd0 con 0x55b95072c4d0 2024-05-09T05:58:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.059+0000 7f26b7446740 1 -- 172.21.15.115:0/394465681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a8084d70 con 0x55b95072c4d0 2024-05-09T05:58:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.059+0000 7f2691ffb640 1 -- 172.21.15.115:0/394465681 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f2698019a50 con 0x55b95072c4d0 2024-05-09T05:58:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.059+0000 7f26b7446740 10 client.28280 fetch_fsmap learned FSMap version 73 2024-05-09T05:58:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.059+0000 7f26b7446740 1 -- 172.21.15.115:0/394465681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55b950b749f0 con 0x55b95072c4d0 2024-05-09T05:58:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.060+0000 7f2691ffb640 1 -- 172.21.15.115:0/394465681 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26a8084d70 con 0x55b95072c4d0 2024-05-09T05:58:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.060+0000 7f2691ffb640 1 -- 172.21.15.115:0/394465681 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 73) ==== 8386+0+0 (secure 0 0 0) 0x7f2698032030 con 0x55b95072c4d0 2024-05-09T05:58:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.060+0000 7f26b7446740 10 client.28280 fetch_fsmap finished waiting for FSMap version 73 2024-05-09T05:58:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.060+0000 7f26b7446740 10 client.28280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.060+0000 7f26b7446740 20 client.28280 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.060+0000 7f26b7446740 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b9506e11c0 0x55b950b771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.060+0000 7f26b7446740 4 client.28280 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:58:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.060+0000 7f26b7446740 1 -- 172.21.15.115:0/394465681 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b950a5d240 con 0x55b9506e11c0 2024-05-09T05:58:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.060+0000 7f26ae329640 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b9506e11c0 0x55b950b771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.061+0000 7f2691ffb640 10 client.28280 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:02.061+0000 7f26ae329640 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b9506e11c0 0x55b950b771e0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b58b82c6de4b220 server_cookie=56c1f3fb53eb98ca in_seq=0 out_seq=0 2024-05-09T05:58:02.109 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:01 smithi195 ceph-mon[33844]: pgmap v627: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 901 op/s 2024-05-09T05:58:02.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:01 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:58:02.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:01 smithi161 ceph-mon[33805]: pgmap v627: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 901 op/s 2024-05-09T05:58:02.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:01 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:58:02.941 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:02 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:58:02.941 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:02 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:58:02.941 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:02 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:58:03.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:02 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:58:03.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:02 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:58:03.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:02 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:58:03.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:02 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:58:03.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:02 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:58:03.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:02 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:58:04.109 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:03 smithi195 ceph-mon[33844]: pgmap v628: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 867 op/s 2024-05-09T05:58:04.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:03 smithi115 ceph-mon[30349]: pgmap v628: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 867 op/s 2024-05-09T05:58:04.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:03 smithi161 ceph-mon[33805]: pgmap v628: 129 pgs: 129 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 867 op/s 2024-05-09T05:58:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.315+0000 7f2691ffb640 1 -- 172.21.15.115:0/394465681 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b950a5d240 con 0x55b9506e11c0 2024-05-09T05:58:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f2691ffb640 10 client.28280 handle_command_reply: tid=0 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 client.28280 shutdown 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 -- 172.21.15.115:0/394465681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95072c4d0 msgr2=0x55b950b00320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95072c4d0 0x55b950b00320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f26980199d0 tx=0x7f269800f790 comp rx=0 tx=0).stop 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 -- 172.21.15.115:0/394465681 shutdown_connections 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b9506e11c0 0x55b950b771e0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b95074d1b0 0x55b950b6d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b95072c4d0 0x55b950b00320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 --2- 172.21.15.115:0/394465681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b9507242f0 0x55b950affde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f2691ffb640 0 client.28280 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 -- 172.21.15.115:0/394465681 >> 172.21.15.115:0/394465681 conn(0x55b950741300 msgr2=0x55b950aeb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 -- 172.21.15.115:0/394465681 shutdown_connections 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 1 -- 172.21.15.115:0/394465681 wait complete. 2024-05-09T05:58:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.316+0000 7f26b7446740 20 client.28280 trim_cache size 0 max 16384 2024-05-09T05:58:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:58:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:58:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 -- 172.21.15.115:0/220953970 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f26940b2960 msgr2=0x7f26940b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f26940b2960 0x7f26940b4e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26a8051dc0 tx=0x7f26a8064250 comp rx=0 tx=0).stop 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 -- 172.21.15.115:0/220953970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00aa360 msgr2=0x7f26a01407b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00aa360 0x7f26a01407b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009230 tx=0x7f26a4005e10 comp rx=0 tx=0).stop 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 -- 172.21.15.115:0/220953970 shutdown_connections 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f26940b2960 0x7f26940b4e70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26a00b00b0 0x7f26a0144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26a00afc10 0x7f26a0140d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 --2- 172.21.15.115:0/220953970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00aa360 0x7f26a01407b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 -- 172.21.15.115:0/220953970 >> 172.21.15.115:0/220953970 conn(0x7f26a00a5290 msgr2=0x7f26a00a60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 -- 172.21.15.115:0/220953970 shutdown_connections 2024-05-09T05:58:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:04.317+0000 7f26af32b640 1 -- 172.21.15.115:0/220953970 wait complete. 2024-05-09T05:58:05.111 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: *** Caught signal (Segmentation fault) ** 2024-05-09T05:58:05.111 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: in thread 7ff992502700 thread_name:md_log_replay 2024-05-09T05:58:05.111 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:58:05.111 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff9a38c4d20] 2024-05-09T05:58:05.111 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x55d36a446da1] 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x55d36a447238] 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55d36a4418af] 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55d36a102cff] 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 6: (MDLog::_replay_thread()+0x1f38) [0x55d36a4a792e] 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55d36a4b0e6d] 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 8: (Thread::entry_wrapper()+0x43) [0x7ff9a4d179f5] 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 9: (Thread::_entry_func(void*)+0xd) [0x7ff9a4d17a11] 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7ff9a38ba1ca] 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 11: clone() 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2024-05-09T05:58:04.643+0000 7ff992502700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: in thread 7ff992502700 thread_name:md_log_replay 2024-05-09T05:58:05.112 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff9a38c4d20] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x55d36a446da1] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x55d36a447238] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55d36a4418af] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55d36a102cff] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 6: (MDLog::_replay_thread()+0x1f38) [0x55d36a4a792e] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55d36a4b0e6d] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 8: (Thread::entry_wrapper()+0x43) [0x7ff9a4d179f5] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 9: (Thread::_entry_func(void*)+0xd) [0x7ff9a4d17a11] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7ff9a38ba1ca] 2024-05-09T05:58:05.113 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 11: clone() 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 0> 2024-05-09T05:58:04.643+0000 7ff992502700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: in thread 7ff992502700 thread_name:md_log_replay 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff9a38c4d20] 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x55d36a446da1] 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x55d36a447238] 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55d36a4418af] 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55d36a102cff] 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 6: (MDLog::_replay_thread()+0x1f38) [0x55d36a4a792e] 2024-05-09T05:58:05.114 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55d36a4b0e6d] 2024-05-09T05:58:05.115 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 8: (Thread::entry_wrapper()+0x43) [0x7ff9a4d179f5] 2024-05-09T05:58:05.115 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 9: (Thread::_entry_func(void*)+0xd) [0x7ff9a4d17a11] 2024-05-09T05:58:05.115 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7ff9a38ba1ca] 2024-05-09T05:58:05.115 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 11: clone() 2024-05-09T05:58:05.115 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:58:05.115 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2024-05-09T05:58:05.115 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 0> 2024-05-09T05:58:04.643+0000 7ff992502700 -1 *** Caught signal (Segmentation fault) ** 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: in thread 7ff992502700 thread_name:md_log_replay 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff9a38c4d20] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_find_before_node(unsigned long, unsigned long const&, unsigned long) const+0x33) [0x55d36a446da1] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 3: (std::_Hashtable, std::__detail::_Identity, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, unsigned long const&)+0x2a) [0x55d36a447238] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 4: (MDSTableClient::got_journaled_ack(unsigned long)+0x3df) [0x55d36a4418af] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 5: (ETableClient::replay(MDSRank*)+0x5f) [0x55d36a102cff] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 6: (MDLog::_replay_thread()+0x1f38) [0x55d36a4a792e] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 7: (MDLog::ReplayThread::entry()+0x11) [0x55d36a4b0e6d] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 8: (Thread::entry_wrapper()+0x43) [0x7ff9a4d179f5] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 9: (Thread::_entry_func(void*)+0xd) [0x7ff9a4d17a11] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 10: /lib64/libpthread.so.0(+0x81ca) [0x7ff9a38ba1ca] 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 11: clone() 2024-05-09T05:58:05.116 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T05:58:05.117 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[70216]: 2024-05-09T05:58:05.610 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:05 smithi195 podman[71598]: 2024-05-09 05:58:05.189454923 +0000 UTC m=+0.165089073 container died 0bf632ef37eef0d17b32414d3d309e119e8d7fc25bd231c2098f9e9c798ffdb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T05:58:06.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:05 smithi195 ceph-mon[33844]: pgmap v629: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.8 MiB/s wr, 971 op/s 2024-05-09T05:58:06.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:05 smithi195 ceph-mon[33844]: osdmap e119: 12 total, 12 up, 12 in 2024-05-09T05:58:06.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:05 smithi115 ceph-mon[30349]: pgmap v629: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.8 MiB/s wr, 971 op/s 2024-05-09T05:58:06.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:05 smithi115 ceph-mon[30349]: osdmap e119: 12 total, 12 up, 12 in 2024-05-09T05:58:06.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:05 smithi161 ceph-mon[33805]: pgmap v629: 129 pgs: 129 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.8 MiB/s wr, 971 op/s 2024-05-09T05:58:06.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:05 smithi161 ceph-mon[33805]: osdmap e119: 12 total, 12 up, 12 in 2024-05-09T05:58:08.109 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:07 smithi195 ceph-mon[33844]: pgmap v631: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.5 MiB/s wr, 391 op/s 2024-05-09T05:58:08.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:07 smithi115 ceph-mon[30349]: pgmap v631: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.5 MiB/s wr, 391 op/s 2024-05-09T05:58:08.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:07 smithi161 ceph-mon[33805]: pgmap v631: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.5 MiB/s wr, 391 op/s 2024-05-09T05:58:09.110 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:08 smithi195 podman[71598]: 2024-05-09 05:58:08.917106564 +0000 UTC m=+3.892740703 container remove 0bf632ef37eef0d17b32414d3d309e119e8d7fc25bd231c2098f9e9c798ffdb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T05:58:09.110 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:08 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-05-09T05:58:09.110 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:08 smithi195 ceph-mon[33844]: pgmap v632: 129 pgs: 6 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 101 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.3 MiB/s wr, 359 op/s 2024-05-09T05:58:09.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:08 smithi115 ceph-mon[30349]: pgmap v632: 129 pgs: 6 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 101 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.3 MiB/s wr, 359 op/s 2024-05-09T05:58:09.154 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:08 smithi161 ceph-mon[33805]: pgmap v632: 129 pgs: 6 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 101 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.3 MiB/s wr, 359 op/s 2024-05-09T05:58:09.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:09 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-05-09T05:58:09.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:09 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Consumed 1.765s CPU time. 2024-05-09T05:58:11.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:11 smithi161 ceph-mon[33805]: pgmap v633: 129 pgs: 12 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 88 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-05-09T05:58:11.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:11 smithi195 ceph-mon[33844]: pgmap v633: 129 pgs: 12 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 88 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-05-09T05:58:11.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:11 smithi115 ceph-mon[30349]: pgmap v633: 129 pgs: 12 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 88 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-05-09T05:58:13.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:13 smithi115 ceph-mon[30349]: pgmap v634: 129 pgs: 10 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 98 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 956 KiB/s wr, 256 op/s 2024-05-09T05:58:13.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:13 smithi161 ceph-mon[33805]: pgmap v634: 129 pgs: 10 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 98 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 956 KiB/s wr, 256 op/s 2024-05-09T05:58:13.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:13 smithi195 ceph-mon[33844]: pgmap v634: 129 pgs: 10 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 98 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 956 KiB/s wr, 256 op/s 2024-05-09T05:58:15.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:15 smithi161 ceph-mon[33805]: pgmap v635: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 781 KiB/s wr, 204 op/s 2024-05-09T05:58:15.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:15 smithi195 ceph-mon[33844]: pgmap v635: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 781 KiB/s wr, 204 op/s 2024-05-09T05:58:15.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:15 smithi115 ceph-mon[30349]: pgmap v635: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 781 KiB/s wr, 204 op/s 2024-05-09T05:58:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.480+0000 7f36d278d640 1 -- 172.21.15.115:0/2223184883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36cc111df0 msgr2=0x7f36cc112280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.480+0000 7f36d278d640 1 --2- 172.21.15.115:0/2223184883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36cc111df0 0x7f36cc112280 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009860 tx=0x7f36c002a3d0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.480+0000 7f36d278d640 1 -- 172.21.15.115:0/2223184883 shutdown_connections 2024-05-09T05:58:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.480+0000 7f36d278d640 1 --2- 172.21.15.115:0/2223184883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36cc112910 0x7f36cc118ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.480+0000 7f36d278d640 1 --2- 172.21.15.115:0/2223184883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36cc111df0 0x7f36cc112280 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.480+0000 7f36d278d640 1 --2- 172.21.15.115:0/2223184883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36cc10a4a0 0x7f36cc10a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.480+0000 7f36d278d640 1 -- 172.21.15.115:0/2223184883 >> 172.21.15.115:0/2223184883 conn(0x7f36cc06e900 msgr2=0x7f36cc06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.480+0000 7f36d278d640 1 -- 172.21.15.115:0/2223184883 shutdown_connections 2024-05-09T05:58:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.481+0000 7f36d278d640 1 -- 172.21.15.115:0/2223184883 wait complete. 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.481+0000 7f36d278d640 1 Processor -- start 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.481+0000 7f36d278d640 1 -- start start 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.482+0000 7f36d278d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36cc10a4a0 0x7f36cc1a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.482+0000 7f36d278d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36cc111df0 0x7f36cc1a3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.482+0000 7f36d278d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36cc112910 0x7f36cc1a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.482+0000 7f36d278d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc11c520 con 0x7f36cc10a4a0 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.482+0000 7f36d278d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc11c3a0 con 0x7f36cc111df0 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.482+0000 7f36d0d2a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36cc112910 0x7f36cc1a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.482+0000 7f36d0d2a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36cc112910 0x7f36cc1a77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:39944/0 (socket says 172.21.15.115:39944) 2024-05-09T05:58:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.482+0000 7f36d0d2a640 1 -- 172.21.15.115:0/1039945620 learned_addr learned my addr 172.21.15.115:0/1039945620 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.482+0000 7f36d278d640 1 -- 172.21.15.115:0/1039945620 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc11c6a0 con 0x7f36cc112910 2024-05-09T05:58:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.484+0000 7f36cbfff640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36cc10a4a0 0x7f36cc1a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.485+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36cc111df0 0x7f36cc1a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.485+0000 7f36d0d2a640 1 -- 172.21.15.115:0/1039945620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36cc111df0 msgr2=0x7f36cc1a3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.485+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36cc111df0 0x7f36cc1a3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.485+0000 7f36d0d2a640 1 -- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36cc10a4a0 msgr2=0x7f36cc1a2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.485+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36cc10a4a0 0x7f36cc1a2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.485+0000 7f36d0d2a640 1 -- 172.21.15.115:0/1039945620 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc1a7e00 con 0x7f36cc112910 2024-05-09T05:58:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.485+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36cc111df0 0x7f36cc1a3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:58:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.485+0000 7f36cbfff640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36cc10a4a0 0x7f36cc1a2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:58:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.486+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36cc112910 0x7f36cc1a77c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f36c400dac0 tx=0x7f36c400dec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.486+0000 7f36c97fa640 1 -- 172.21.15.115:0/1039945620 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4017070 con 0x7f36cc112910 2024-05-09T05:58:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.486+0000 7f36c97fa640 1 -- 172.21.15.115:0/1039945620 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36c4003e00 con 0x7f36cc112910 2024-05-09T05:58:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.486+0000 7f36c97fa640 1 -- 172.21.15.115:0/1039945620 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4010460 con 0x7f36cc112910 2024-05-09T05:58:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.486+0000 7f36d278d640 1 -- 172.21.15.115:0/1039945620 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36cc1a80f0 con 0x7f36cc112910 2024-05-09T05:58:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.487+0000 7f36d278d640 1 -- 172.21.15.115:0/1039945620 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36cc1a85c0 con 0x7f36cc112910 2024-05-09T05:58:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.490+0000 7f36d9647740 1 Processor -- start 2024-05-09T05:58:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.490+0000 7f36c97fa640 1 -- 172.21.15.115:0/1039945620 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f36c4004020 con 0x7f36cc112910 2024-05-09T05:58:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.490+0000 7f36d9647740 1 -- start start 2024-05-09T05:58:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36d9647740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f59441b0 0x5647f5cf66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36d9647740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5cf6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36d9647740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59234d0 0x5647f5cf5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36d9647740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5647f5c4c7c0 con 0x5647f59441b0 2024-05-09T05:58:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36d9647740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5647f5be80e0 con 0x5647f591b2f0 2024-05-09T05:58:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36d9647740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5647f5c48780 con 0x5647f59234d0 2024-05-09T05:58:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5cf6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5cf6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49322/0 (socket says 172.21.15.115:49322) 2024-05-09T05:58:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36cbfff640 1 -- 172.21.15.115:0/2075838150 learned_addr learned my addr 172.21.15.115:0/2075838150 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/2075838150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f59441b0 0x5647f5cf66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.491+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/2075838150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59234d0 0x5647f5cf5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36cbfff640 1 -- 172.21.15.115:0/2075838150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59234d0 msgr2=0x5647f5cf5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36cbfff640 1 --2- 172.21.15.115:0/2075838150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59234d0 0x5647f5cf5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36cbfff640 1 -- 172.21.15.115:0/2075838150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f59441b0 msgr2=0x5647f5cf66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36cbfff640 1 --2- 172.21.15.115:0/2075838150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f59441b0 0x5647f5cf66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36cbfff640 1 -- 172.21.15.115:0/2075838150 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f5c117c0 con 0x5647f591b2f0 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36cbfff640 1 --2- 172.21.15.115:0/2075838150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5cf6c30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f36bc011510 tx=0x7f36bc0119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2cf5783a9e0c20 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36c97fa640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36b40b29d0 0x7f36b40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36b27fc640 1 -- 172.21.15.115:0/2075838150 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc004020 con 0x5647f591b2f0 2024-05-09T05:58:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36b27fc640 1 -- 172.21.15.115:0/2075838150 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36bc0041c0 con 0x5647f591b2f0 2024-05-09T05:58:17.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.492+0000 7f36b27fc640 1 -- 172.21.15.115:0/2075838150 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc00f820 con 0x5647f591b2f0 2024-05-09T05:58:17.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.493+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36b40b29d0 0x7f36b40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.493+0000 7f36c97fa640 1 -- 172.21.15.115:0/1039945620 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9526+0+0 (secure 0 0 0) 0x7f36c40e22e0 con 0x7f36cc112910 2024-05-09T05:58:17.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.493+0000 7f36d9647740 1 -- 172.21.15.115:0/2075838150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 msgr2=0x5647f5cf6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:17.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.493+0000 7f36d9647740 1 --2- 172.21.15.115:0/2075838150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5cf6c30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f36bc011510 tx=0x7f36bc0119d0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.482 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.493+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36b40b29d0 0x7f36b40b4ee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f36c0005cd0 tx=0x7f36c0009500 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.494+0000 7f36d9647740 1 -- 172.21.15.115:0/2075838150 shutdown_connections 2024-05-09T05:58:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.494+0000 7f36d9647740 1 --2- 172.21.15.115:0/2075838150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59234d0 0x5647f5cf5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.494+0000 7f36d9647740 1 --2- 172.21.15.115:0/2075838150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5cf6c30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.494+0000 7f36d9647740 1 --2- 172.21.15.115:0/2075838150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f59441b0 0x5647f5cf66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.494+0000 7f36d9647740 1 -- 172.21.15.115:0/2075838150 >> 172.21.15.115:0/2075838150 conn(0x5647f5938300 msgr2=0x5647f5ce37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.494+0000 7f36d9647740 1 -- 172.21.15.115:0/2075838150 shutdown_connections 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.494+0000 7f36d9647740 1 -- 172.21.15.115:0/2075838150 wait complete. 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.494+0000 7f36d9647740 1 -- start start 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.494+0000 7f36d9647740 1 Processor -- start 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36d9647740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cefaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36d9647740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 0x5647f5ce7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36d9647740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5ce8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36d9647740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5647f5cf5dc0 con 0x5647f591b2f0 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36d9647740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5647f5bd39c0 con 0x5647f59234d0 2024-05-09T05:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36d9647740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5647f5cf5f40 con 0x5647f59441b0 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cefaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cefaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41216/0 (socket says 172.21.15.115:41216) 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36cbfff640 1 -- 172.21.15.115:0/1908416644 learned_addr learned my addr 172.21.15.115:0/1908416644 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 0x5647f5ce7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36d9647740 10 client.0 _ll_register_callbacks cb 0x5647f5c00020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.495+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5ce8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36cbfff640 1 -- 172.21.15.115:0/1908416644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 msgr2=0x5647f5ce8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36cbfff640 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5ce8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36cbfff640 1 -- 172.21.15.115:0/1908416644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 msgr2=0x5647f5ce7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36cbfff640 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 0x5647f5ce7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36cbfff640 1 -- 172.21.15.115:0/1908416644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f5bfa1f0 con 0x5647f591b2f0 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36cbfff640 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cefaa0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f36bc01b040 tx=0x7f36bc0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08a2abb3bd0cd54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:58:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36ab7fe640 1 -- 172.21.15.115:0/1908416644 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc027070 con 0x5647f591b2f0 2024-05-09T05:58:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36ab7fe640 1 -- 172.21.15.115:0/1908416644 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9526+0+0 (secure 0 0 0) 0x7f36bc005510 con 0x5647f591b2f0 2024-05-09T05:58:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.496+0000 7f36d9647740 1 -- 172.21.15.115:0/1908416644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36bc0041c0 con 0x5647f591b2f0 2024-05-09T05:58:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.497+0000 7f36ab7fe640 1 -- 172.21.15.115:0/1908416644 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f36bc033050 con 0x5647f591b2f0 2024-05-09T05:58:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.497+0000 7f36d9647740 10 client.19113 fetch_fsmap learned FSMap version 73 2024-05-09T05:58:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.497+0000 7f36d9647740 1 -- 172.21.15.115:0/1908416644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5647f59af790 con 0x5647f591b2f0 2024-05-09T05:58:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.497+0000 7f36ab7fe640 1 -- 172.21.15.115:0/1908416644 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36bc0041c0 con 0x5647f591b2f0 2024-05-09T05:58:17.489 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.497+0000 7f36ab7fe640 1 -- 172.21.15.115:0/1908416644 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 73) ==== 8386+0+0 (secure 0 0 0) 0x7f36bc004aa0 con 0x5647f591b2f0 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.497+0000 7f36d9647740 10 client.19113 fetch_fsmap finished waiting for FSMap version 73 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.498+0000 7f36d9647740 10 client.19113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.498+0000 7f36d9647740 20 client.19113 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.498+0000 7f36d9647740 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5647f5902d80 0x5647f5d003c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.498+0000 7f36d9647740 4 client.19113 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.498+0000 7f36d9647740 1 -- 172.21.15.115:0/1908416644 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647f5c5a3d0 con 0x5647f5902d80 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.498+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5647f5902d80 0x5647f5d003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.499+0000 7f36ab7fe640 10 client.19113 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.499+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5647f5902d80 0x5647f5d003c0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb06c0d664cf531 server_cookie=eba65973bddfb9bd in_seq=0 out_seq=0 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.499+0000 7f36ab7fe640 1 -- 172.21.15.115:0/1908416644 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647f5c5a3d0 con 0x5647f5902d80 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.499+0000 7f36ab7fe640 10 client.19113 handle_command_reply: tid=0 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.499+0000 7f36d9647740 1 client.19113 shutdown 2024-05-09T05:58:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 -- 172.21.15.115:0/1908416644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 msgr2=0x5647f5cefaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cefaa0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f36bc01b040 tx=0x7f36bc0045a0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 -- 172.21.15.115:0/1908416644 shutdown_connections 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5647f5902d80 0x5647f5d003c0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5ce8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 0x5647f5ce7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 --2- 172.21.15.115:0/1908416644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cefaa0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 -- 172.21.15.115:0/1908416644 >> 172.21.15.115:0/1908416644 conn(0x5647f5938300 msgr2=0x5647f5ce37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36ab7fe640 0 client.19113 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 -- 172.21.15.115:0/1908416644 shutdown_connections 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 1 -- 172.21.15.115:0/1908416644 wait complete. 2024-05-09T05:58:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.500+0000 7f36d9647740 20 client.19113 trim_cache size 0 max 16384 2024-05-09T05:58:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.523+0000 7f36d9647740 1 Processor -- start 2024-05-09T05:58:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.523+0000 7f36d9647740 1 -- start start 2024-05-09T05:58:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36d9647740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cf84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36d9647740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 0x5647f5cef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36d9647740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5ce8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cf84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cf84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41222/0 (socket says 172.21.15.115:41222) 2024-05-09T05:58:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36cbfff640 1 -- 172.21.15.115:0/3470184981 learned_addr learned my addr 172.21.15.115:0/3470184981 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36cb7fe640 1 --2- 172.21.15.115:0/3470184981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 0x5647f5cef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36d9647740 1 -- 172.21.15.115:0/3470184981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1b78e0 con 0x5647f591b2f0 2024-05-09T05:58:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36d9647740 1 -- 172.21.15.115:0/3470184981 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1b9480 con 0x5647f59234d0 2024-05-09T05:58:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/3470184981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5ce8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.524+0000 7f36d9647740 1 -- 172.21.15.115:0/3470184981 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1b7fb0 con 0x5647f59441b0 2024-05-09T05:58:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.525+0000 7f36d0d2a640 1 -- 172.21.15.115:0/3470184981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 msgr2=0x5647f5cef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.525+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/3470184981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 0x5647f5cef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.525+0000 7f36d0d2a640 1 -- 172.21.15.115:0/3470184981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 msgr2=0x5647f5cf84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.525+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/3470184981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cf84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.525+0000 7f36d0d2a640 1 -- 172.21.15.115:0/3470184981 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f5a265f0 con 0x5647f59441b0 2024-05-09T05:58:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.525+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/3470184981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5ce8310 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f36c40df640 tx=0x7f36c40a8e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9a0387ba53b9a09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.525+0000 7f36b1ffb640 1 -- 172.21.15.115:0/3470184981 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4020040 con 0x5647f59441b0 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.525+0000 7f36b1ffb640 1 -- 172.21.15.115:0/3470184981 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36c40ad580 con 0x5647f59441b0 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.525+0000 7f36b1ffb640 1 -- 172.21.15.115:0/3470184981 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c40ad120 con 0x5647f59441b0 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.526+0000 7f36d9647740 1 -- 172.21.15.115:0/3470184981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 msgr2=0x5647f5ce8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.526+0000 7f36d9647740 1 --2- 172.21.15.115:0/3470184981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5ce8310 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f36c40df640 tx=0x7f36c40a8e10 comp rx=0 tx=0).stop 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.527+0000 7f36d9647740 1 -- 172.21.15.115:0/3470184981 shutdown_connections 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.527+0000 7f36d9647740 1 --2- 172.21.15.115:0/3470184981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5ce8310 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.527+0000 7f36d9647740 1 --2- 172.21.15.115:0/3470184981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f59234d0 0x5647f5cef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.527+0000 7f36d9647740 1 --2- 172.21.15.115:0/3470184981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f591b2f0 0x5647f5cf84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.527+0000 7f36d9647740 1 -- 172.21.15.115:0/3470184981 >> 172.21.15.115:0/3470184981 conn(0x5647f5938300 msgr2=0x5647f5cf32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.527+0000 7f36d9647740 1 -- 172.21.15.115:0/3470184981 shutdown_connections 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.527+0000 7f36d9647740 1 -- 172.21.15.115:0/3470184981 wait complete. 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.527+0000 7f36d9647740 1 -- start start 2024-05-09T05:58:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.527+0000 7f36d9647740 1 Processor -- start 2024-05-09T05:58:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d9647740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5d05ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d9647740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f59234d0 0x5647f5d032e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d9647740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5d03820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d9647740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1b8db0 con 0x5647f59234d0 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d9647740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1b86b0 con 0x5647f591b2f0 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d9647740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1b58e0 con 0x5647f59441b0 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d9647740 10 client.0 _ll_register_callbacks cb 0x7f36bc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d0d2a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5d03820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5d05ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d0d2a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5d03820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:39972/0 (socket says 172.21.15.115:39972) 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5d05ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:49358/0 (socket says 172.21.15.115:49358) 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.528+0000 7f36d0d2a640 1 -- 172.21.15.115:0/763716070 learned_addr learned my addr 172.21.15.115:0/763716070 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36d0d2a640 1 -- 172.21.15.115:0/763716070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 msgr2=0x5647f5d05ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5d05ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36d0d2a640 1 -- 172.21.15.115:0/763716070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f59234d0 msgr2=0x5647f5d032e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f59234d0 0x5647f5d032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36d0d2a640 1 -- 172.21.15.115:0/763716070 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f5c10ed0 con 0x5647f59441b0 2024-05-09T05:58:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36d0d2a640 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5d03820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36c40a6440 tx=0x7f36c40e2270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28fbd6b537ab5ca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:58:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36ab7fe640 1 -- 172.21.15.115:0/763716070 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4020070 con 0x5647f59441b0 2024-05-09T05:58:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36ab7fe640 1 -- 172.21.15.115:0/763716070 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9526+0+0 (secure 0 0 0) 0x7f36c4003970 con 0x5647f59441b0 2024-05-09T05:58:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.529+0000 7f36d9647740 1 -- 172.21.15.115:0/763716070 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c40ad580 con 0x5647f59441b0 2024-05-09T05:58:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.530+0000 7f36ab7fe640 1 -- 172.21.15.115:0/763716070 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f36c40ad330 con 0x5647f59441b0 2024-05-09T05:58:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.530+0000 7f36d9647740 10 client.28432 fetch_fsmap learned FSMap version 73 2024-05-09T05:58:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.530+0000 7f36d9647740 1 -- 172.21.15.115:0/763716070 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5647f5d8d2b0 con 0x5647f59441b0 2024-05-09T05:58:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.530+0000 7f36ab7fe640 1 -- 172.21.15.115:0/763716070 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36c40ad580 con 0x5647f59441b0 2024-05-09T05:58:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.531+0000 7f36ab7fe640 1 -- 172.21.15.115:0/763716070 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 73) ==== 8386+0+0 (secure 0 0 0) 0x7f36c4023060 con 0x5647f59441b0 2024-05-09T05:58:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.531+0000 7f36d9647740 10 client.28432 fetch_fsmap finished waiting for FSMap version 73 2024-05-09T05:58:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.531+0000 7f36d9647740 10 client.28432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.531+0000 7f36d9647740 20 client.28432 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.531+0000 7f36d9647740 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5647f58d81c0 0x5647f5d8f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.531+0000 7f36d9647740 4 client.28432 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:58:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.531+0000 7f36d9647740 1 -- 172.21.15.115:0/763716070 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647f5c54240 con 0x5647f58d81c0 2024-05-09T05:58:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.532+0000 7f36cbfff640 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5647f58d81c0 0x5647f5d8f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.532+0000 7f36ab7fe640 10 client.28432 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.532+0000 7f36cbfff640 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5647f58d81c0 0x5647f5d8f960 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a88d7f7b213b927 server_cookie=bad0cb76d87e12c in_seq=0 out_seq=0 2024-05-09T05:58:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.546+0000 7f36ab7fe640 1 -- 172.21.15.115:0/763716070 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5647f5c54240 con 0x5647f58d81c0 2024-05-09T05:58:17.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.546+0000 7f36ab7fe640 10 client.28432 handle_command_reply: tid=0 2024-05-09T05:58:17.535 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.547+0000 7f36d9647740 1 client.28432 shutdown 2024-05-09T05:58:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.547+0000 7f36d9647740 1 -- 172.21.15.115:0/763716070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 msgr2=0x5647f5d03820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.547+0000 7f36d9647740 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5d03820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36c40a6440 tx=0x7f36c40e2270 comp rx=0 tx=0).stop 2024-05-09T05:58:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.547+0000 7f36d9647740 1 -- 172.21.15.115:0/763716070 shutdown_connections 2024-05-09T05:58:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.547+0000 7f36d9647740 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5647f58d81c0 0x5647f5d8f960 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.547+0000 7f36ab7fe640 0 client.28432 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.548+0000 7f36d9647740 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5647f59441b0 0x5647f5d03820 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.548+0000 7f36d9647740 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5647f59234d0 0x5647f5d032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.548+0000 7f36d9647740 1 --2- 172.21.15.115:0/763716070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647f591b2f0 0x5647f5d05ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.548+0000 7f36d9647740 1 -- 172.21.15.115:0/763716070 >> 172.21.15.115:0/763716070 conn(0x5647f5938300 msgr2=0x5647f5cf9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.548+0000 7f36d9647740 1 -- 172.21.15.115:0/763716070 shutdown_connections 2024-05-09T05:58:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.548+0000 7f36d9647740 1 -- 172.21.15.115:0/763716070 wait complete. 2024-05-09T05:58:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.548+0000 7f36d9647740 20 client.28432 trim_cache size 0 max 16384 2024-05-09T05:58:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:58:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:58:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:58:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 -- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36b40b29d0 msgr2=0x7f36b40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36b40b29d0 0x7f36b40b4ee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f36c0005cd0 tx=0x7f36c0009500 comp rx=0 tx=0).stop 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 -- 172.21.15.115:0/1039945620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36cc112910 msgr2=0x7f36cc1a77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36cc112910 0x7f36cc1a77c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f36c400dac0 tx=0x7f36c400dec0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 -- 172.21.15.115:0/1039945620 shutdown_connections 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f36b40b29d0 0x7f36b40b4ee0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36cc112910 0x7f36cc1a77c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36cc111df0 0x7f36cc1a3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 --2- 172.21.15.115:0/1039945620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36cc10a4a0 0x7f36cc1a2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 -- 172.21.15.115:0/1039945620 >> 172.21.15.115:0/1039945620 conn(0x7f36cc06e900 msgr2=0x7f36cc073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 -- 172.21.15.115:0/1039945620 shutdown_connections 2024-05-09T05:58:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:17.549+0000 7f36d278d640 1 -- 172.21.15.115:0/1039945620 wait complete. 2024-05-09T05:58:17.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:17 smithi161 ceph-mon[33805]: pgmap v636: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 724 KiB/s wr, 190 op/s 2024-05-09T05:58:17.661 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:17 smithi195 ceph-mon[33844]: pgmap v636: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 724 KiB/s wr, 190 op/s 2024-05-09T05:58:17.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:17 smithi115 ceph-mon[30349]: pgmap v636: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 724 KiB/s wr, 190 op/s 2024-05-09T05:58:19.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:19 smithi161 ceph-mon[33805]: pgmap v637: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 804 KiB/s wr, 211 op/s 2024-05-09T05:58:19.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:19 smithi161 ceph-mon[33805]: osdmap e120: 12 total, 12 up, 12 in 2024-05-09T05:58:19.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:19 smithi195 ceph-mon[33844]: pgmap v637: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 804 KiB/s wr, 211 op/s 2024-05-09T05:58:19.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:19 smithi195 ceph-mon[33844]: osdmap e120: 12 total, 12 up, 12 in 2024-05-09T05:58:19.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:19 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 4. 2024-05-09T05:58:19.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:19 smithi195 systemd[1]: Stopped Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:58:19.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:19 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Consumed 1.765s CPU time. 2024-05-09T05:58:19.860 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:19 smithi195 systemd[1]: Starting Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T05:58:19.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:19 smithi115 ceph-mon[30349]: pgmap v637: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 804 KiB/s wr, 211 op/s 2024-05-09T05:58:19.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:19 smithi115 ceph-mon[30349]: osdmap e120: 12 total, 12 up, 12 in 2024-05-09T05:58:20.192 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:19 smithi195 podman[71683]: 2024-05-09 05:58:19.817795377 +0000 UTC m=+0.027622774 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:58:20.192 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:19 smithi195 podman[71683]: 2024-05-09 05:58:19.960087043 +0000 UTC m=+0.169914422 container create 7e9e6957cbf73ee7d9633237d138dc3606d4a1e108df23e1bd111bdd5561c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:58:20.610 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:20 smithi195 ceph-mon[33844]: daemon mds.l restarted 2024-05-09T05:58:20.610 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:20 smithi195 podman[71683]: 2024-05-09 05:58:20.2051144 +0000 UTC m=+0.414941797 container init 7e9e6957cbf73ee7d9633237d138dc3606d4a1e108df23e1bd111bdd5561c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True) 2024-05-09T05:58:20.610 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:20 smithi195 podman[71683]: 2024-05-09 05:58:20.209552243 +0000 UTC m=+0.419379629 container start 7e9e6957cbf73ee7d9633237d138dc3606d4a1e108df23e1bd111bdd5561c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T05:58:20.610 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:20 smithi195 ceph-mds[71700]: -- 172.21.15.195:0/2843108120 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618f10b1a40 con 0x5618f10f9180 2024-05-09T05:58:20.610 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:20 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[71696]: starting mds.l at 2024-05-09T05:58:20.611 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:20 smithi195 bash[71683]: 7e9e6957cbf73ee7d9633237d138dc3606d4a1e108df23e1bd111bdd5561c8c1 2024-05-09T05:58:20.611 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 05:58:20 smithi195 systemd[1]: Started Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T05:58:20.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:20 smithi115 ceph-mon[30349]: daemon mds.l restarted 2024-05-09T05:58:20.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:20 smithi161 ceph-mon[33805]: daemon mds.l restarted 2024-05-09T05:58:21.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:21 smithi195 ceph-mon[33844]: pgmap v639: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 MiB/s wr, 316 op/s 2024-05-09T05:58:21.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:21 smithi195 ceph-mon[33844]: osdmap e121: 12 total, 12 up, 12 in 2024-05-09T05:58:21.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:21 smithi195 ceph-mon[33844]: mds.? [v2:172.21.15.195:6832/3680433414,v1:172.21.15.195:6833/3680433414] up:boot 2024-05-09T05:58:21.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:21 smithi195 ceph-mon[33844]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:58:21.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:21 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:58:21.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:21 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:58:21.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:21 smithi115 ceph-mon[30349]: pgmap v639: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 MiB/s wr, 316 op/s 2024-05-09T05:58:21.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:21 smithi115 ceph-mon[30349]: osdmap e121: 12 total, 12 up, 12 in 2024-05-09T05:58:21.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:21 smithi115 ceph-mon[30349]: mds.? [v2:172.21.15.195:6832/3680433414,v1:172.21.15.195:6833/3680433414] up:boot 2024-05-09T05:58:21.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:21 smithi115 ceph-mon[30349]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:58:21.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:21 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:58:21.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:21 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:58:21.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:21 smithi161 ceph-mon[33805]: pgmap v639: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 MiB/s wr, 316 op/s 2024-05-09T05:58:21.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:21 smithi161 ceph-mon[33805]: osdmap e121: 12 total, 12 up, 12 in 2024-05-09T05:58:21.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:21 smithi161 ceph-mon[33805]: mds.? [v2:172.21.15.195:6832/3680433414,v1:172.21.15.195:6833/3680433414] up:boot 2024-05-09T05:58:21.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:21 smithi161 ceph-mon[33805]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-09T05:58:21.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:21 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T05:58:21.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:21 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:58:23.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:23 smithi195 ceph-mon[33844]: pgmap v641: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.0 MiB/s wr, 272 op/s 2024-05-09T05:58:23.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:23 smithi115 ceph-mon[30349]: pgmap v641: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.0 MiB/s wr, 272 op/s 2024-05-09T05:58:23.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:23 smithi161 ceph-mon[33805]: pgmap v641: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.0 MiB/s wr, 272 op/s 2024-05-09T05:58:25.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:25 smithi195 ceph-mon[33844]: pgmap v642: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.6 MiB/s wr, 417 op/s 2024-05-09T05:58:25.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:25 smithi115 ceph-mon[30349]: pgmap v642: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.6 MiB/s wr, 417 op/s 2024-05-09T05:58:25.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:25 smithi161 ceph-mon[33805]: pgmap v642: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.6 MiB/s wr, 417 op/s 2024-05-09T05:58:27.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:27 smithi195 ceph-mon[33844]: pgmap v643: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.5 MiB/s wr, 404 op/s 2024-05-09T05:58:27.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:27 smithi115 ceph-mon[30349]: pgmap v643: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.5 MiB/s wr, 404 op/s 2024-05-09T05:58:27.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:27 smithi161 ceph-mon[33805]: pgmap v643: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.5 MiB/s wr, 404 op/s 2024-05-09T05:58:29.221 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-09T05:58:29.221 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.95 2024-05-09T05:58:29.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:58:29.508 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:58:29.633 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:29 smithi115 ceph-mon[30349]: pgmap v644: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.3 MiB/s wr, 333 op/s 2024-05-09T05:58:29.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:29 smithi195 ceph-mon[33844]: pgmap v644: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.3 MiB/s wr, 333 op/s 2024-05-09T05:58:29.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:29 smithi161 ceph-mon[33805]: pgmap v644: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.3 MiB/s wr, 333 op/s 2024-05-09T05:58:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 -- 172.21.15.115:0/4254696834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c104da0 msgr2=0x7f473c1051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 --2- 172.21.15.115:0/4254696834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c104da0 0x7f473c1051b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f4738009900 tx=0x7f473802a410 comp rx=0 tx=0).stop 2024-05-09T05:58:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 -- 172.21.15.115:0/4254696834 shutdown_connections 2024-05-09T05:58:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 --2- 172.21.15.115:0/4254696834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c10e1f0 0x7f473c1106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 --2- 172.21.15.115:0/4254696834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c10b800 0x7f473c10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 --2- 172.21.15.115:0/4254696834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c104da0 0x7f473c1051b0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 -- 172.21.15.115:0/4254696834 >> 172.21.15.115:0/4254696834 conn(0x7f473c0782e0 msgr2=0x7f473c078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 -- 172.21.15.115:0/4254696834 shutdown_connections 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 -- 172.21.15.115:0/4254696834 wait complete. 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.684+0000 7f4743d66640 1 Processor -- start 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4743d66640 1 -- start start 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4743d66640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c104da0 0x7f473c102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4743d66640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c10b800 0x7f473c102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4743d66640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c10e1f0 0x7f473c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4743d66640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f473c113d80 con 0x7f473c10b800 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4743d66640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f473c113c00 con 0x7f473c10e1f0 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4743d66640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f473c113f00 con 0x7f473c104da0 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4741301640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c10b800 0x7f473c102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4741301640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c10b800 0x7f473c102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59712/0 (socket says 172.21.15.115:59712) 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4741301640 1 -- 172.21.15.115:0/63337759 learned_addr learned my addr 172.21.15.115:0/63337759 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4741b02640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c104da0 0x7f473c102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4742303640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c10e1f0 0x7f473c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4741301640 1 -- 172.21.15.115:0/63337759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c104da0 msgr2=0x7f473c102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4741301640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c104da0 0x7f473c102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4741301640 1 -- 172.21.15.115:0/63337759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c10e1f0 msgr2=0x7f473c1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4741301640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c10e1f0 0x7f473c1030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.685+0000 7f4741301640 1 -- 172.21.15.115:0/63337759 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c1a8ba0 con 0x7f473c10b800 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.686+0000 7f4741301640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c10b800 0x7f473c102ba0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b450 tx=0x7f472c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.686+0000 7f4732ffd640 1 -- 172.21.15.115:0/63337759 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c015020 con 0x7f473c10b800 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.686+0000 7f4732ffd640 1 -- 172.21.15.115:0/63337759 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f472c011d20 con 0x7f473c10b800 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.686+0000 7f4743d66640 1 -- 172.21.15.115:0/63337759 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c1a8e90 con 0x7f473c10b800 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.686+0000 7f4732ffd640 1 -- 172.21.15.115:0/63337759 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c010e30 con 0x7f473c10b800 2024-05-09T05:58:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.686+0000 7f4743d66640 1 -- 172.21.15.115:0/63337759 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c1a92a0 con 0x7f473c10b800 2024-05-09T05:58:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.688+0000 7f4732ffd640 1 -- 172.21.15.115:0/63337759 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f472c018020 con 0x7f473c10b800 2024-05-09T05:58:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.689+0000 7f4732ffd640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f47280b2950 0x7f47280b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.689+0000 7f474ac16740 1 Processor -- start 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.689+0000 7f474ac16740 1 -- start start 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.689+0000 7f4741b02640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f47280b2950 0x7f47280b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.689+0000 7f4732ffd640 1 -- 172.21.15.115:0/63337759 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f472c0e2310 con 0x7f473c10b800 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f474ac16740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b631b0 0x555f34f06680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f474ac16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f06bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f474ac16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b424d0 0x555f34f07100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f474ac16740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f34e6b7c0 con 0x555f34b631b0 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f474ac16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555f34e070e0 con 0x555f34b3a2f0 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f474ac16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f34e67780 con 0x555f34b424d0 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b631b0 0x555f34f06680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b631b0 0x555f34f06680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59714/0 (socket says 172.21.15.115:59714) 2024-05-09T05:58:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 -- 172.21.15.115:0/1284677662 learned_addr learned my addr 172.21.15.115:0/1284677662 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f47280b2950 0x7f47280b4e60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f47380057c0 tx=0x7f47380058d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 -- 172.21.15.115:0/1284677662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b424d0 msgr2=0x555f34f07100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 --2- 172.21.15.115:0/1284677662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b424d0 0x555f34f07100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 -- 172.21.15.115:0/1284677662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 msgr2=0x555f34f06bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 --2- 172.21.15.115:0/1284677662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f06bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 -- 172.21.15.115:0/1284677662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f34e307c0 con 0x555f34b631b0 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4741b02640 1 --2- 172.21.15.115:0/1284677662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b631b0 0x555f34f06680 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f473802a930 tx=0x7f473803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e2b7541f5bb5bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4727fff640 1 -- 172.21.15.115:0/1284677662 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738035050 con 0x555f34b631b0 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.690+0000 7f4727fff640 1 -- 172.21.15.115:0/1284677662 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47380044d0 con 0x555f34b631b0 2024-05-09T05:58:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f4727fff640 1 -- 172.21.15.115:0/1284677662 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473803c6a0 con 0x555f34b631b0 2024-05-09T05:58:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f474ac16740 1 -- 172.21.15.115:0/1284677662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b631b0 msgr2=0x555f34f06680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f474ac16740 1 --2- 172.21.15.115:0/1284677662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b631b0 0x555f34f06680 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f473802a930 tx=0x7f473803e000 comp rx=0 tx=0).stop 2024-05-09T05:58:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f474ac16740 1 -- 172.21.15.115:0/1284677662 shutdown_connections 2024-05-09T05:58:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f474ac16740 1 --2- 172.21.15.115:0/1284677662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b424d0 0x555f34f07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f474ac16740 1 --2- 172.21.15.115:0/1284677662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f06bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f474ac16740 1 --2- 172.21.15.115:0/1284677662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b631b0 0x555f34f06680 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f474ac16740 1 -- 172.21.15.115:0/1284677662 >> 172.21.15.115:0/1284677662 conn(0x555f34b57300 msgr2=0x555f34f02e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f474ac16740 1 -- 172.21.15.115:0/1284677662 shutdown_connections 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.691+0000 7f474ac16740 1 -- 172.21.15.115:0/1284677662 wait complete. 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f474ac16740 1 -- start start 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f474ac16740 1 Processor -- start 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f474ac16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b3a2f0 0x555f34f12850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f474ac16740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f12d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f474ac16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b631b0 0x555f34f10c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f474ac16740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555f34f13750 con 0x555f34b424d0 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f474ac16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555f34df29c0 con 0x555f34b631b0 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f474ac16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f34f138d0 con 0x555f34b3a2f0 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f474ac16740 10 client.0 _ll_register_callbacks cb 0x555f34e1f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f4741301640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f12d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f4741301640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f12d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59724/0 (socket says 172.21.15.115:59724) 2024-05-09T05:58:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f4741301640 1 -- 172.21.15.115:0/1909977988 learned_addr learned my addr 172.21.15.115:0/1909977988 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f4741b02640 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b3a2f0 0x555f34f12850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.692+0000 7f4742303640 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b631b0 0x555f34f10c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f4741301640 1 -- 172.21.15.115:0/1909977988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b3a2f0 msgr2=0x555f34f12850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f4741301640 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b3a2f0 0x555f34f12850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f4741301640 1 -- 172.21.15.115:0/1909977988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b631b0 msgr2=0x555f34f10c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f4741301640 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b631b0 0x555f34f10c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f4741301640 1 -- 172.21.15.115:0/1909977988 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f34e191f0 con 0x555f34b424d0 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f4741301640 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f12d90 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f472c009fd0 tx=0x7f472c0a8e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b12468e8d65aae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f47257fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f47257fa640 1 -- 172.21.15.115:0/1909977988 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c0a9360 con 0x555f34b424d0 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f47257fa640 1 -- 172.21.15.115:0/1909977988 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f472c011890 con 0x555f34b424d0 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f474ac16740 1 -- 172.21.15.115:0/1909977988 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47380044d0 con 0x555f34b424d0 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f47257fa640 1 -- 172.21.15.115:0/1909977988 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f472c0ad710 con 0x555f34b424d0 2024-05-09T05:58:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f474ac16740 10 client.19134 fetch_fsmap learned FSMap version 75 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f474ac16740 1 -- 172.21.15.115:0/1909977988 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x555f34bce790 con 0x555f34b424d0 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f47257fa640 1 -- 172.21.15.115:0/1909977988 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47380044d0 con 0x555f34b424d0 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f47257fa640 1 -- 172.21.15.115:0/1909977988 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 75) ==== 8386+0+0 (secure 0 0 0) 0x7f472c0a4cd0 con 0x555f34b424d0 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f474ac16740 10 client.19134 fetch_fsmap finished waiting for FSMap version 75 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f474ac16740 10 client.19134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.693+0000 7f474ac16740 20 client.19134 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.694+0000 7f474ac16740 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555f34b21d80 0x555f34f23040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.694+0000 7f474ac16740 4 client.19134 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.694+0000 7f474ac16740 1 -- 172.21.15.115:0/1909977988 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f34e793d0 con 0x555f34b21d80 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.694+0000 7f4741b02640 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555f34b21d80 0x555f34f23040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.694+0000 7f47257fa640 10 client.19134 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.694+0000 7f4741b02640 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555f34b21d80 0x555f34f23040 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da82ddd4a15b8f33 server_cookie=4a95f2ac4a877cd2 in_seq=0 out_seq=0 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f47257fa640 1 -- 172.21.15.115:0/1909977988 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f34e793d0 con 0x555f34b21d80 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f47257fa640 10 client.19134 handle_command_reply: tid=0 2024-05-09T05:58:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 client.19134 shutdown 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 -- 172.21.15.115:0/1909977988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 msgr2=0x555f34f12d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f12d90 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f472c009fd0 tx=0x7f472c0a8e70 comp rx=0 tx=0).stop 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 -- 172.21.15.115:0/1909977988 shutdown_connections 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b631b0 0x555f34f10c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f12d90 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555f34b21d80 0x555f34f23040 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f47257fa640 0 client.19134 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 --2- 172.21.15.115:0/1909977988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b3a2f0 0x555f34f12850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 -- 172.21.15.115:0/1909977988 >> 172.21.15.115:0/1909977988 conn(0x555f34b57300 msgr2=0x555f34f01700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 -- 172.21.15.115:0/1909977988 shutdown_connections 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 1 -- 172.21.15.115:0/1909977988 wait complete. 2024-05-09T05:58:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.695+0000 7f474ac16740 20 client.19134 trim_cache size 0 max 16384 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.713+0000 7f474ac16740 1 Processor -- start 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.713+0000 7f474ac16740 1 -- start start 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f474ac16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b21d80 0x555f34f0aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f474ac16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f07420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f474ac16740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f18b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f474ac16740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1b3170 con 0x555f34b424d0 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f474ac16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1b4d10 con 0x555f34b3a2f0 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f474ac16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1b3840 con 0x555f34b21d80 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4742303640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f18b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4742303640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f18b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59734/0 (socket says 172.21.15.115:59734) 2024-05-09T05:58:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4742303640 1 -- 172.21.15.115:0/3691361303 learned_addr learned my addr 172.21.15.115:0/3691361303 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4741b02640 1 --2- 172.21.15.115:0/3691361303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b21d80 0x555f34f0aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4741301640 1 --2- 172.21.15.115:0/3691361303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f07420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4742303640 1 -- 172.21.15.115:0/3691361303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b21d80 msgr2=0x555f34f0aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4742303640 1 --2- 172.21.15.115:0/3691361303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b21d80 0x555f34f0aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4742303640 1 -- 172.21.15.115:0/3691361303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 msgr2=0x555f34f07420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4742303640 1 --2- 172.21.15.115:0/3691361303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f07420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4742303640 1 -- 172.21.15.115:0/3691361303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f34c455f0 con 0x555f34b424d0 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f4742303640 1 --2- 172.21.15.115:0/3691361303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f18b60 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f4734011600 tx=0x7f4734011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28aadcab304ff771 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.714+0000 7f47277fe640 1 -- 172.21.15.115:0/3691361303 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734003a30 con 0x555f34b424d0 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.715+0000 7f47277fe640 1 -- 172.21.15.115:0/3691361303 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4734003bd0 con 0x555f34b424d0 2024-05-09T05:58:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.715+0000 7f47277fe640 1 -- 172.21.15.115:0/3691361303 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473400f8a0 con 0x555f34b424d0 2024-05-09T05:58:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 -- 172.21.15.115:0/3691361303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 msgr2=0x555f34f18b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 --2- 172.21.15.115:0/3691361303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f18b60 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f4734011600 tx=0x7f4734011ac0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 -- 172.21.15.115:0/3691361303 shutdown_connections 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 --2- 172.21.15.115:0/3691361303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b424d0 0x555f34f18b60 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 --2- 172.21.15.115:0/3691361303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f07420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 --2- 172.21.15.115:0/3691361303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b21d80 0x555f34f0aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 -- 172.21.15.115:0/3691361303 >> 172.21.15.115:0/3691361303 conn(0x555f34b57300 msgr2=0x555f34f0a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 -- 172.21.15.115:0/3691361303 shutdown_connections 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 -- 172.21.15.115:0/3691361303 wait complete. 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 -- start start 2024-05-09T05:58:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.716+0000 7f474ac16740 1 Processor -- start 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f474ac16740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b21d80 0x555f34f16640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f474ac16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f16b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f474ac16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b424d0 0x555f34f170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f474ac16740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1b4640 con 0x555f34b21d80 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f474ac16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1b3f40 con 0x555f34b3a2f0 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f474ac16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1b1150 con 0x555f34b424d0 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f474ac16740 10 client.0 _ll_register_callbacks cb 0x7f472c0107f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741b02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b21d80 0x555f34f16640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741b02640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b21d80 0x555f34f16640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59746/0 (socket says 172.21.15.115:59746) 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741b02640 1 -- 172.21.15.115:0/636274665 learned_addr learned my addr 172.21.15.115:0/636274665 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741301640 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f16b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4742303640 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b424d0 0x555f34f170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741b02640 1 -- 172.21.15.115:0/636274665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b424d0 msgr2=0x555f34f170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741b02640 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b424d0 0x555f34f170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741b02640 1 -- 172.21.15.115:0/636274665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 msgr2=0x555f34f16b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741b02640 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f16b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741b02640 1 -- 172.21.15.115:0/636274665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f34e2fed0 con 0x555f34b21d80 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.717+0000 7f4741301640 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f16b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f4741b02640 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b21d80 0x555f34f16640 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4738003fd0 tx=0x7f47380042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621d4a73d9924cc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f47257fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f47257fa640 1 -- 172.21.15.115:0/636274665 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473805f070 con 0x555f34b21d80 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f47257fa640 1 -- 172.21.15.115:0/636274665 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f47380037f0 con 0x555f34b21d80 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f474ac16740 1 -- 172.21.15.115:0/636274665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4734003bd0 con 0x555f34b21d80 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f47257fa640 1 -- 172.21.15.115:0/636274665 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f4738068050 con 0x555f34b21d80 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f474ac16740 10 client.19140 fetch_fsmap learned FSMap version 75 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f474ac16740 1 -- 172.21.15.115:0/636274665 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x555f34f8b2a0 con 0x555f34b21d80 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f47257fa640 1 -- 172.21.15.115:0/636274665 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4734003bd0 con 0x555f34b21d80 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f47257fa640 1 -- 172.21.15.115:0/636274665 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 75) ==== 8386+0+0 (secure 0 0 0) 0x7f4738062020 con 0x555f34b21d80 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f474ac16740 10 client.19140 fetch_fsmap finished waiting for FSMap version 75 2024-05-09T05:58:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f474ac16740 10 client.19140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.718+0000 7f474ac16740 20 client.19140 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.719+0000 7f474ac16740 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555f34b631b0 0x555f34f8d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.719+0000 7f474ac16740 4 client.19140 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:58:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.719+0000 7f474ac16740 1 -- 172.21.15.115:0/636274665 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f34e73240 con 0x555f34b631b0 2024-05-09T05:58:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.719+0000 7f4741301640 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555f34b631b0 0x555f34f8d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.719+0000 7f4741301640 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555f34b631b0 0x555f34f8d5e0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55b2e9d35361787 server_cookie=e3aa3f96edc420aa in_seq=0 out_seq=0 2024-05-09T05:58:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.719+0000 7f47257fa640 10 client.19140 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f47257fa640 1 -- 172.21.15.115:0/636274665 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555f34e73240 con 0x555f34b631b0 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f47257fa640 10 client.19140 handle_command_reply: tid=0 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f474ac16740 1 client.19140 shutdown 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f474ac16740 1 -- 172.21.15.115:0/636274665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b21d80 msgr2=0x555f34f16640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f474ac16740 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b21d80 0x555f34f16640 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4738003fd0 tx=0x7f47380042c0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f474ac16740 1 -- 172.21.15.115:0/636274665 shutdown_connections 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f474ac16740 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x555f34b631b0 0x555f34f8d5e0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f474ac16740 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f34b424d0 0x555f34f170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f47257fa640 0 client.19140 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f474ac16740 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f34b3a2f0 0x555f34f16b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f474ac16740 1 --2- 172.21.15.115:0/636274665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555f34b21d80 0x555f34f16640 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.725+0000 7f474ac16740 1 -- 172.21.15.115:0/636274665 >> 172.21.15.115:0/636274665 conn(0x555f34b57300 msgr2=0x555f34f0a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f474ac16740 1 -- 172.21.15.115:0/636274665 shutdown_connections 2024-05-09T05:58:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f474ac16740 1 -- 172.21.15.115:0/636274665 wait complete. 2024-05-09T05:58:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f474ac16740 20 client.19140 trim_cache size 0 max 16384 2024-05-09T05:58:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:58:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:58:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:58:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 -- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f47280b2950 msgr2=0x7f47280b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f47280b2950 0x7f47280b4e60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f47380057c0 tx=0x7f47380058d0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 -- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c10b800 msgr2=0x7f473c102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c10b800 0x7f473c102ba0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b450 tx=0x7f472c00b910 comp rx=0 tx=0).stop 2024-05-09T05:58:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 -- 172.21.15.115:0/63337759 shutdown_connections 2024-05-09T05:58:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f47280b2950 0x7f47280b4e60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c10e1f0 0x7f473c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f473c10b800 0x7f473c102ba0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 --2- 172.21.15.115:0/63337759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c104da0 0x7f473c102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 -- 172.21.15.115:0/63337759 >> 172.21.15.115:0/63337759 conn(0x7f473c0782e0 msgr2=0x7f473c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 -- 172.21.15.115:0/63337759 shutdown_connections 2024-05-09T05:58:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:30.726+0000 7f4743d66640 1 -- 172.21.15.115:0/63337759 wait complete. 2024-05-09T05:58:31.728 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:31 smithi115 ceph-mon[30349]: pgmap v645: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 351 op/s 2024-05-09T05:58:31.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:31 smithi195 ceph-mon[33844]: pgmap v645: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 351 op/s 2024-05-09T05:58:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 -- 172.21.15.115:0/851800249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed8109300 msgr2=0x7f3ed8109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/851800249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed8109300 0x7f3ed8109710 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009990 tx=0x7f3ed4009ca0 comp rx=0 tx=0).stop 2024-05-09T05:58:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 -- 172.21.15.115:0/851800249 shutdown_connections 2024-05-09T05:58:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/851800249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed81031d0 0x7f3ed8111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/851800249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed8109300 0x7f3ed8109710 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/851800249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed8102880 0x7f3ed8102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 -- 172.21.15.115:0/851800249 >> 172.21.15.115:0/851800249 conn(0x7f3ed80fe6f0 msgr2=0x7f3ed8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:31.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 -- 172.21.15.115:0/851800249 shutdown_connections 2024-05-09T05:58:31.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 -- 172.21.15.115:0/851800249 wait complete. 2024-05-09T05:58:31.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.885+0000 7f3edf6a2700 1 Processor -- start 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edf6a2700 1 -- start start 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edf6a2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed8102880 0x7f3ed8103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edf6a2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed81031d0 0x7f3ed8103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edf6a2700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed8109300 0x7f3ed81044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edf6a2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8114440 con 0x7f3ed81031d0 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edf6a2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed81142c0 con 0x7f3ed8102880 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edf6a2700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed81145c0 con 0x7f3ed8109300 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edcc3d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed81031d0 0x7f3ed8103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edd43e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed8102880 0x7f3ed8103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edcc3d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed81031d0 0x7f3ed8103fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59772/0 (socket says 172.21.15.115:59772) 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edcc3d700 1 -- 172.21.15.115:0/172322438 learned_addr learned my addr 172.21.15.115:0/172322438 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3eddc3f700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed8109300 0x7f3ed81044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:31.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edcc3d700 1 -- 172.21.15.115:0/172322438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed8109300 msgr2=0x7f3ed81044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edcc3d700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed8109300 0x7f3ed81044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edcc3d700 1 -- 172.21.15.115:0/172322438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed8102880 msgr2=0x7f3ed8103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edcc3d700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed8102880 0x7f3ed8103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edcc3d700 1 -- 172.21.15.115:0/172322438 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed81a91e0 con 0x7f3ed81031d0 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3eddc3f700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed8109300 0x7f3ed81044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.886+0000 7f3edd43e700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed8102880 0x7f3ed8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.887+0000 7f3edcc3d700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed81031d0 0x7f3ed8103fa0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800d980 tx=0x7f3ec800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.887+0000 7f3ece7fc700 1 -- 172.21.15.115:0/172322438 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8017070 con 0x7f3ed81031d0 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.887+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed81a94d0 con 0x7f3ed81031d0 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.887+0000 7f3ece7fc700 1 -- 172.21.15.115:0/172322438 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ec8009dc0 con 0x7f3ed81031d0 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.887+0000 7f3ece7fc700 1 -- 172.21.15.115:0/172322438 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800f3c0 con 0x7f3ed81031d0 2024-05-09T05:58:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.887+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed81a9a60 con 0x7f3ed81031d0 2024-05-09T05:58:31.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.888+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed8104f40 con 0x7f3ed81031d0 2024-05-09T05:58:31.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.890+0000 7f3ece7fc700 1 -- 172.21.15.115:0/172322438 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f3ec8009930 con 0x7f3ed81031d0 2024-05-09T05:58:31.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.891+0000 7f3ece7fc700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ec40b29f0 0x7f3ec40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:31.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.891+0000 7f3edd43e700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ec40b29f0 0x7f3ec40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:31.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.891+0000 7f3ece7fc700 1 -- 172.21.15.115:0/172322438 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f3ec80e2490 con 0x7f3ed81031d0 2024-05-09T05:58:31.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.891+0000 7f3edd43e700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ec40b29f0 0x7f3ec40b4f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400b650 tx=0x7f3ed4005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:31.895+0000 7f3ece7fc700 1 -- 172.21.15.115:0/172322438 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ec80a9b30 con 0x7f3ed81031d0 2024-05-09T05:58:31.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:31 smithi161 ceph-mon[33805]: pgmap v645: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 351 op/s 2024-05-09T05:58:32.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.210+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed810c8a0 con 0x7f3ed81031d0 2024-05-09T05:58:32.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.211+0000 7f3ece7fc700 1 -- 172.21.15.115:0/172322438 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+11963 (secure 0 0 0) 0x7f3ec80a9430 con 0x7f3ed81031d0 2024-05-09T05:58:32.200 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:58:32.200 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":75,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:58:20.434266+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28438":{"gid":28438,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/3680433414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3680433414},{"type":"v1","addr":"172.21.15.195:6833","nonce":3680433414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24440,24425,24451,24469,24457]},"id":1}]} 2024-05-09T05:58:32.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ec40b29f0 msgr2=0x7f3ec40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:32.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ec40b29f0 0x7f3ec40b4f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400b650 tx=0x7f3ed4005fb0 comp rx=0 tx=0).stop 2024-05-09T05:58:32.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed81031d0 msgr2=0x7f3ed8103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:32.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed81031d0 0x7f3ed8103fa0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800d980 tx=0x7f3ec800dc90 comp rx=0 tx=0).stop 2024-05-09T05:58:32.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 shutdown_connections 2024-05-09T05:58:32.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f3ec40b29f0 0x7f3ec40b4f00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:32.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed8102880 0x7f3ed8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:32.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ed8109300 0x7f3ed81044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:32.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 --2- 172.21.15.115:0/172322438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ed81031d0 0x7f3ed8103fa0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:32.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 >> 172.21.15.115:0/172322438 conn(0x7f3ed80fe6f0 msgr2=0x7f3ed810a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:32.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 shutdown_connections 2024-05-09T05:58:32.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:32.214+0000 7f3edf6a2700 1 -- 172.21.15.115:0/172322438 wait complete. 2024-05-09T05:58:32.204 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 75 2024-05-09T05:58:32.597 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:32 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/172322438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:58:32.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:32 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/172322438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:58:32.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:32 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/172322438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:58:32.972 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 140.95 --await-for 45.0' 2024-05-09T05:58:32.973 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 140.95 --await-for 45.0 2024-05-09T05:58:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:58:33.732 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:33 smithi115 ceph-mon[30349]: pgmap v646: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 1.1 MiB/s wr, 302 op/s 2024-05-09T05:58:33.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:33 smithi195 ceph-mon[33844]: pgmap v646: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 1.1 MiB/s wr, 302 op/s 2024-05-09T05:58:33.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:33 smithi161 ceph-mon[33805]: pgmap v646: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 1.1 MiB/s wr, 302 op/s 2024-05-09T05:58:34.807 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:34 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:58:34.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:34 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:58:34.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:34 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:58:35.746 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:35 smithi115 ceph-mon[30349]: pgmap v647: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 346 op/s 2024-05-09T05:58:35.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.803+0000 7f7066e48700 1 -- 172.21.15.115:0/1141627056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70601038f0 msgr2=0x7f7060103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:35.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.803+0000 7f7066e48700 1 --2- 172.21.15.115:0/1141627056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70601038f0 0x7f7060103d00 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f704c009990 tx=0x7f704c009ca0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.803+0000 7f7066e48700 1 -- 172.21.15.115:0/1141627056 shutdown_connections 2024-05-09T05:58:35.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.803+0000 7f7066e48700 1 --2- 172.21.15.115:0/1141627056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7060104240 0x7f7060110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.803+0000 7f7066e48700 1 --2- 172.21.15.115:0/1141627056 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7060108200 0x7f7060108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.803+0000 7f7066e48700 1 --2- 172.21.15.115:0/1141627056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70601038f0 0x7f7060103d00 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.803+0000 7f7066e48700 1 -- 172.21.15.115:0/1141627056 >> 172.21.15.115:0/1141627056 conn(0x7f70600fe6f0 msgr2=0x7f7060100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:35.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.803+0000 7f7066e48700 1 -- 172.21.15.115:0/1141627056 shutdown_connections 2024-05-09T05:58:35.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.803+0000 7f7066e48700 1 -- 172.21.15.115:0/1141627056 wait complete. 2024-05-09T05:58:35.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7066e48700 1 Processor -- start 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7066e48700 1 -- start start 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7066e48700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70601038f0 0x7f706019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7066e48700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7060104240 0x7f706019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7066e48700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7060108200 0x7f7060199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7066e48700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7060113230 con 0x7f7060104240 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7066e48700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f70601130b0 con 0x7f7060108200 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7066e48700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70601133b0 con 0x7f70601038f0 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7064be4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70601038f0 0x7f706019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7064be4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70601038f0 0x7f706019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:44634/0 (socket says 172.21.15.115:44634) 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f7064be4700 1 -- 172.21.15.115:0/4073717925 learned_addr learned my addr 172.21.15.115:0/4073717925 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f705ffff700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7060104240 0x7f706019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.804+0000 7f70653e5700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7060108200 0x7f7060199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f7064be4700 1 -- 172.21.15.115:0/4073717925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7060108200 msgr2=0x7f7060199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f7064be4700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7060108200 0x7f7060199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f7064be4700 1 -- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7060104240 msgr2=0x7f706019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f7064be4700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7060104240 0x7f706019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f7064be4700 1 -- 172.21.15.115:0/4073717925 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706019a0e0 con 0x7f70601038f0 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f7064be4700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70601038f0 0x7f706019f810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f705400e940 tx=0x7f705400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f705dffb700 1 -- 172.21.15.115:0/4073717925 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054004220 con 0x7f70601038f0 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f7066e48700 1 -- 172.21.15.115:0/4073717925 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706019a3d0 con 0x7f70601038f0 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f705dffb700 1 -- 172.21.15.115:0/4073717925 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70540043c0 con 0x7f70601038f0 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f705dffb700 1 -- 172.21.15.115:0/4073717925 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054003bf0 con 0x7f70601038f0 2024-05-09T05:58:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.805+0000 7f7066e48700 1 -- 172.21.15.115:0/4073717925 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70601027e0 con 0x7f70601038f0 2024-05-09T05:58:35.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.809+0000 7f705dffb700 1 -- 172.21.15.115:0/4073717925 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f7054021440 con 0x7f70601038f0 2024-05-09T05:58:35.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.810+0000 7f705dffb700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f70500b29f0 0x7f70500b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.810+0000 7f705ffff700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f70500b29f0 0x7f70500b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.810+0000 7f705dffb700 1 -- 172.21.15.115:0/4073717925 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f70540e2ae0 con 0x7f70601038f0 2024-05-09T05:58:35.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.810+0000 7f707585eb80 1 Processor -- start 2024-05-09T05:58:35.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.810+0000 7f707585eb80 1 -- start start 2024-05-09T05:58:35.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f705ffff700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f70500b29f0 0x7f70500b4f00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7060106920 tx=0x7f704c005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:35.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f707585eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b067444af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f707585eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 0x55b067445050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f707585eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b0674530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f707585eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b067367240 con 0x55b0672c7640 2024-05-09T05:58:35.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f707585eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b06730cb40 con 0x55b067452cc0 2024-05-09T05:58:35.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f707585eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b067363200 con 0x55b067341ea0 2024-05-09T05:58:35.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f7064be4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b067444af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f705ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 0x55b067445050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f7064be4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b067444af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:44648/0 (socket says 172.21.15.115:44648) 2024-05-09T05:58:35.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f7064be4700 1 -- 172.21.15.115:0/725156900 learned_addr learned my addr 172.21.15.115:0/725156900 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:35.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f70653e5700 1 --2- 172.21.15.115:0/725156900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b0674530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f7064be4700 1 -- 172.21.15.115:0/725156900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 msgr2=0x55b0674530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f7064be4700 1 --2- 172.21.15.115:0/725156900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b0674530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f7064be4700 1 -- 172.21.15.115:0/725156900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 msgr2=0x55b067445050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f7064be4700 1 --2- 172.21.15.115:0/725156900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 0x55b067445050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.811+0000 7f7064be4700 1 -- 172.21.15.115:0/725156900 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0672fd6a0 con 0x55b067341ea0 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.812+0000 7f7064be4700 1 --2- 172.21.15.115:0/725156900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b067444af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f70540e04f0 tx=0x7f70540a9270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9efd3117858ca11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.812+0000 7f704affd700 1 -- 172.21.15.115:0/725156900 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70540b2230 con 0x55b067341ea0 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.812+0000 7f704affd700 1 -- 172.21.15.115:0/725156900 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70540a9590 con 0x55b067341ea0 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.812+0000 7f704affd700 1 -- 172.21.15.115:0/725156900 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70540a9ce0 con 0x55b067341ea0 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.813+0000 7f707585eb80 1 -- 172.21.15.115:0/725156900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 msgr2=0x55b067444af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.813+0000 7f707585eb80 1 --2- 172.21.15.115:0/725156900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b067444af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f70540e04f0 tx=0x7f70540a9270 comp rx=0 tx=0).stop 2024-05-09T05:58:35.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.813+0000 7f707585eb80 1 -- 172.21.15.115:0/725156900 shutdown_connections 2024-05-09T05:58:35.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.813+0000 7f707585eb80 1 --2- 172.21.15.115:0/725156900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b0674530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.813+0000 7f707585eb80 1 --2- 172.21.15.115:0/725156900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b067444af0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.813+0000 7f707585eb80 1 --2- 172.21.15.115:0/725156900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 0x55b067445050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.813+0000 7f707585eb80 1 -- 172.21.15.115:0/725156900 >> 172.21.15.115:0/725156900 conn(0x55b0672cb110 msgr2=0x55b06743fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:35.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.813+0000 7f707585eb80 1 -- 172.21.15.115:0/725156900 shutdown_connections 2024-05-09T05:58:35.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.813+0000 7f707585eb80 1 -- 172.21.15.115:0/725156900 wait complete. 2024-05-09T05:58:35.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f707585eb80 1 -- start start 2024-05-09T05:58:35.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f707585eb80 1 Processor -- start 2024-05-09T05:58:35.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f707585eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b067456e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f707585eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067341ea0 0x55b067457360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f707585eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b0674578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f707585eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55b067453a90 con 0x55b067341ea0 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f707585eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b067301cc0 con 0x55b067452cc0 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f707585eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b067459e20 con 0x55b0672c7640 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f705ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067341ea0 0x55b067457360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f7064be4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b067456e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f705ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067341ea0 0x55b067457360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49158/0 (socket says 172.21.15.115:49158) 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f70653e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b0674578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.814+0000 7f705ffff700 1 -- 172.21.15.115:0/1047922585 learned_addr learned my addr 172.21.15.115:0/1047922585 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f707585eb80 10 client.0 _ll_register_callbacks cb 0x55b0671d8860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f7064be4700 1 -- 172.21.15.115:0/1047922585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 msgr2=0x55b0674578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f7064be4700 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b0674578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f7064be4700 1 -- 172.21.15.115:0/1047922585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067341ea0 msgr2=0x55b067457360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f7064be4700 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067341ea0 0x55b067457360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f7064be4700 1 -- 172.21.15.115:0/1047922585 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0671e2630 con 0x55b0672c7640 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f707585eb80 1 -- 172.21.15.115:0/1047922585 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70540a9590 con 0x55b0672c7640 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f7064be4700 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b067456e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f70540e04f0 tx=0x7f70540a9f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa62589113c288e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f703ffff700 10 client.28471 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f703ffff700 1 -- 172.21.15.115:0/1047922585 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70540b2230 con 0x55b0672c7640 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f703ffff700 1 -- 172.21.15.115:0/1047922585 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f70540b2660 con 0x55b0672c7640 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f703ffff700 1 -- 172.21.15.115:0/1047922585 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f70540a9a80 con 0x55b0672c7640 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f707585eb80 10 client.28471 fetch_fsmap learned FSMap version 76 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.815+0000 7f707585eb80 1 -- 172.21.15.115:0/1047922585 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({fsmap=76}) -- 0x55b067464740 con 0x55b0672c7640 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.816+0000 7f703ffff700 1 -- 172.21.15.115:0/1047922585 <== mon.1 v2:172.21.15.195:3300/0 4 ==== fsmap(e 76) ==== 8382+0+0 (secure 0 0 0) 0x7f70540701b0 con 0x55b0672c7640 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.816+0000 7f707585eb80 10 client.28471 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.816+0000 7f707585eb80 10 client.28471 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:58:35.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.816+0000 7f707585eb80 20 client.28471 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.816+0000 7f707585eb80 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b067466c10 0x55b067469030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.816+0000 7f707585eb80 4 client.28471 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.816+0000 7f707585eb80 1 -- 172.21.15.115:0/1047922585 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b067374c20 con 0x55b067466c10 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.816+0000 7f705ffff700 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b067466c10 0x55b067469030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.817+0000 7f703ffff700 10 client.28471 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.817+0000 7f705ffff700 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b067466c10 0x55b067469030 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c248fa8c8151a9e4 server_cookie=c2971bc192e56286 in_seq=0 out_seq=0 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.817+0000 7f703ffff700 1 -- 172.21.15.115:0/1047922585 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b067374c20 con 0x55b067466c10 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.817+0000 7f703ffff700 10 client.28471 handle_command_reply: tid=0 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 client.28471 shutdown 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 -- 172.21.15.115:0/1047922585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 msgr2=0x55b067456e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b067456e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f70540e04f0 tx=0x7f70540a9f90 comp rx=0 tx=0).stop 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 -- 172.21.15.115:0/1047922585 shutdown_connections 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b0674578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b067466c10 0x55b067469030 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b067456e20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 --2- 172.21.15.115:0/1047922585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067341ea0 0x55b067457360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f703ffff700 0 client.28471 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:35.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 -- 172.21.15.115:0/1047922585 >> 172.21.15.115:0/1047922585 conn(0x55b0672cb110 msgr2=0x55b06743ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:35.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 -- 172.21.15.115:0/1047922585 shutdown_connections 2024-05-09T05:58:35.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 1 -- 172.21.15.115:0/1047922585 wait complete. 2024-05-09T05:58:35.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.818+0000 7f707585eb80 20 client.28471 trim_cache size 0 max 16384 2024-05-09T05:58:35.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.839+0000 7f707585eb80 1 Processor -- start 2024-05-09T05:58:35.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.839+0000 7f707585eb80 1 -- start start 2024-05-09T05:58:35.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.839+0000 7f707585eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b0674626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.839+0000 7f707585eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067341ea0 0x55b0674592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.839+0000 7f707585eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067452cc0 0x55b067451500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f707585eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70601ae190 con 0x55b067452cc0 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f707585eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f70601afd00 con 0x55b067341ea0 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f707585eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70601ae860 con 0x55b0672c7640 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f70653e5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067452cc0 0x55b067451500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f705ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067341ea0 0x55b0674592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f7064be4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b0674626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f705ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067341ea0 0x55b0674592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51816/0 (socket says 172.21.15.115:51816) 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f705ffff700 1 -- 172.21.15.115:0/3691034276 learned_addr learned my addr 172.21.15.115:0/3691034276 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f705ffff700 1 -- 172.21.15.115:0/3691034276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 msgr2=0x55b0674626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f705ffff700 1 --2- 172.21.15.115:0/3691034276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b0674626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f705ffff700 1 -- 172.21.15.115:0/3691034276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067452cc0 msgr2=0x55b067451500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:35.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f705ffff700 1 --2- 172.21.15.115:0/3691034276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067452cc0 0x55b067451500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f705ffff700 1 -- 172.21.15.115:0/3691034276 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b067454be0 con 0x55b067341ea0 2024-05-09T05:58:35.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f70653e5700 1 --2- 172.21.15.115:0/3691034276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067452cc0 0x55b067451500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:58:35.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f7064be4700 1 --2- 172.21.15.115:0/3691034276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b0674626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:58:35.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f705ffff700 1 --2- 172.21.15.115:0/3691034276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067341ea0 0x55b0674592c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f704c0050a0 tx=0x7f704c021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625706f8f5bada72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:35.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.840+0000 7f704a7fc700 1 -- 172.21.15.115:0/3691034276 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c034630 con 0x55b067341ea0 2024-05-09T05:58:35.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.841+0000 7f704a7fc700 1 -- 172.21.15.115:0/3691034276 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f704c003680 con 0x55b067341ea0 2024-05-09T05:58:35.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.841+0000 7f704a7fc700 1 -- 172.21.15.115:0/3691034276 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c00f6a0 con 0x55b067341ea0 2024-05-09T05:58:35.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 -- 172.21.15.115:0/3691034276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067341ea0 msgr2=0x55b0674592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:35.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 --2- 172.21.15.115:0/3691034276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067341ea0 0x55b0674592c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f704c0050a0 tx=0x7f704c021000 comp rx=0 tx=0).stop 2024-05-09T05:58:35.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 -- 172.21.15.115:0/3691034276 shutdown_connections 2024-05-09T05:58:35.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 --2- 172.21.15.115:0/3691034276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b067452cc0 0x55b067451500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 --2- 172.21.15.115:0/3691034276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0672c7640 0x55b0674626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 --2- 172.21.15.115:0/3691034276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067341ea0 0x55b0674592c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 -- 172.21.15.115:0/3691034276 >> 172.21.15.115:0/3691034276 conn(0x55b0672cb110 msgr2=0x55b067455600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:35.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 -- 172.21.15.115:0/3691034276 shutdown_connections 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 -- 172.21.15.115:0/3691034276 wait complete. 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 -- start start 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.842+0000 7f707585eb80 1 Processor -- start 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f707585eb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 0x55b06746bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f707585eb80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b06746a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f707585eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b06746a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f707585eb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f70601af630 con 0x55b0672c7640 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f707585eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f70601aef30 con 0x55b067452cc0 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f707585eb80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70601ac130 con 0x55b067341ea0 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f70653e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b06746a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f70653e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b06746a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51820/0 (socket says 172.21.15.115:51820) 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f70653e5700 1 -- 172.21.15.115:0/2850719938 learned_addr learned my addr 172.21.15.115:0/2850719938 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f705ffff700 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b06746a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f707585eb80 10 client.0 _ll_register_callbacks cb 0x55b0671d8860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:35.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f7064be4700 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 0x55b06746bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f70653e5700 1 -- 172.21.15.115:0/2850719938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 msgr2=0x55b06746a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f70653e5700 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b06746a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f70653e5700 1 -- 172.21.15.115:0/2850719938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 msgr2=0x55b06746bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f70653e5700 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 0x55b06746bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f70653e5700 1 -- 172.21.15.115:0/2850719938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b06746acb0 con 0x55b067452cc0 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f7064be4700 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 0x55b06746bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f70653e5700 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b06746a770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7058011d10 tx=0x7f7058007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ed889bcd93c7dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.843+0000 7f703ffff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.844+0000 7f703ffff700 1 -- 172.21.15.115:0/2850719938 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058007d40 con 0x55b067452cc0 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.844+0000 7f703ffff700 1 -- 172.21.15.115:0/2850719938 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f7058003a00 con 0x55b067452cc0 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.844+0000 7f707585eb80 1 -- 172.21.15.115:0/2850719938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704c003680 con 0x55b067452cc0 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.844+0000 7f703ffff700 1 -- 172.21.15.115:0/2850719938 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f705802a050 con 0x55b067452cc0 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.844+0000 7f707585eb80 10 client.28340 fetch_fsmap learned FSMap version 76 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.844+0000 7f707585eb80 1 -- 172.21.15.115:0/2850719938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55b0674d1d10 con 0x55b067452cc0 2024-05-09T05:58:35.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.844+0000 7f703ffff700 1 -- 172.21.15.115:0/2850719938 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f704c003680 con 0x55b067452cc0 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f703ffff700 1 -- 172.21.15.115:0/2850719938 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 76) ==== 8382+0+0 (secure 0 0 0) 0x7f7058021460 con 0x55b067452cc0 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f707585eb80 10 client.28340 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f707585eb80 10 client.28340 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f707585eb80 20 client.28340 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f707585eb80 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b0674d2490 0x55b0674d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f707585eb80 4 client.28340 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.95, "await_for": 45.0}] 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f707585eb80 1 -- 172.21.15.115:0/2850719938 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.95, "await_for": 45.0}) -- 0x55b06736ecc0 con 0x55b0674d2490 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f7064be4700 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b0674d2490 0x55b0674d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f703ffff700 10 client.28340 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:35.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.845+0000 7f7064be4700 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b0674d2490 0x55b0674d48b0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a5159873c64fde server_cookie=96adf8e294a83f13 in_seq=0 out_seq=0 2024-05-09T05:58:35.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:35 smithi195 ceph-mon[33844]: pgmap v647: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 346 op/s 2024-05-09T05:58:35.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:35 smithi161 ceph-mon[33805]: pgmap v647: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 346 op/s 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f703ffff700 1 -- 172.21.15.115:0/2850719938 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b06736ecc0 con 0x55b0674d2490 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f703ffff700 10 client.28340 handle_command_reply: tid=0 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 client.28340 shutdown 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 -- 172.21.15.115:0/2850719938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 msgr2=0x55b06746a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b06746a770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7058011d10 tx=0x7f7058007500 comp rx=0 tx=0).stop 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 -- 172.21.15.115:0/2850719938 shutdown_connections 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b067452cc0 0x55b06746a770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55b0674d2490 0x55b0674d48b0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55b0672c7640 0x55b06746bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 --2- 172.21.15.115:0/2850719938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b067341ea0 0x55b06746a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f703ffff700 0 client.28340 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 -- 172.21.15.115:0/2850719938 >> 172.21.15.115:0/2850719938 conn(0x55b0672cb110 msgr2=0x55b067451260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 -- 172.21.15.115:0/2850719938 shutdown_connections 2024-05-09T05:58:35.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 1 -- 172.21.15.115:0/2850719938 wait complete. 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.939+0000 7f707585eb80 20 client.28340 trim_cache size 0 max 16384 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 76, 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 54, 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "f2e9c488": { 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 54, 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:58:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 140.9, 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:35.930 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 -- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f70500b29f0 msgr2=0x7f70500b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f70500b29f0 0x7f70500b4f00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7060106920 tx=0x7f704c005fb0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 -- 172.21.15.115:0/4073717925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70601038f0 msgr2=0x7f706019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70601038f0 0x7f706019f810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f705400e940 tx=0x7f705400ec50 comp rx=0 tx=0).stop 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 -- 172.21.15.115:0/4073717925 shutdown_connections 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f70500b29f0 0x7f70500b4f00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7060108200 0x7f7060199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70601038f0 0x7f706019f810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 --2- 172.21.15.115:0/4073717925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7060104240 0x7f706019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 -- 172.21.15.115:0/4073717925 >> 172.21.15.115:0/4073717925 conn(0x7f70600fe6f0 msgr2=0x7f7060109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 -- 172.21.15.115:0/4073717925 shutdown_connections 2024-05-09T05:58:35.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:35.940+0000 7f7066e48700 1 -- 172.21.15.115:0/4073717925 wait complete. 2024-05-09T05:58:36.635 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f2e9c488', quiesce took 7.4 seconds. Will release after: 13.549999999999999 2024-05-09T05:58:37.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:37 smithi195 ceph-mon[33844]: pgmap v648: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 898 KiB/s wr, 233 op/s 2024-05-09T05:58:37.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:37 smithi115 ceph-mon[30349]: pgmap v648: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 898 KiB/s wr, 233 op/s 2024-05-09T05:58:37.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:37 smithi161 ceph-mon[33805]: pgmap v648: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 898 KiB/s wr, 233 op/s 2024-05-09T05:58:39.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:39 smithi195 ceph-mon[33844]: pgmap v649: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 806 KiB/s wr, 212 op/s 2024-05-09T05:58:39.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:39 smithi115 ceph-mon[30349]: pgmap v649: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 806 KiB/s wr, 212 op/s 2024-05-09T05:58:39.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:39 smithi161 ceph-mon[33805]: pgmap v649: 129 pgs: 129 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 806 KiB/s wr, 212 op/s 2024-05-09T05:58:41.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:41 smithi195 ceph-mon[33844]: pgmap v650: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 749 KiB/s wr, 198 op/s 2024-05-09T05:58:41.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:41 smithi115 ceph-mon[30349]: pgmap v650: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 749 KiB/s wr, 198 op/s 2024-05-09T05:58:41.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:41 smithi161 ceph-mon[33805]: pgmap v650: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 749 KiB/s wr, 198 op/s 2024-05-09T05:58:43.758 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:43 smithi115 ceph-mon[30349]: pgmap v651: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 419 KiB/s wr, 113 op/s 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.844+0000 7f4cd7e29640 1 -- 172.21.15.115:0/870098519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0108bd0 msgr2=0x7f4cd0102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.844+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/870098519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0108bd0 0x7f4cd0102770 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009900 tx=0x7f4ccc02a410 comp rx=0 tx=0).stop 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.844+0000 7f4cd7e29640 1 -- 172.21.15.115:0/870098519 shutdown_connections 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.844+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/870098519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0103320 0x7f4cd010f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.844+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/870098519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd0109070 0x7f4cd0102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.844+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/870098519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0108bd0 0x7f4cd0102770 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.844+0000 7f4cd7e29640 1 -- 172.21.15.115:0/870098519 >> 172.21.15.115:0/870098519 conn(0x7f4cd00782d0 msgr2=0x7f4cd0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.844+0000 7f4cd7e29640 1 -- 172.21.15.115:0/870098519 shutdown_connections 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.844+0000 7f4cd7e29640 1 -- 172.21.15.115:0/870098519 wait complete. 2024-05-09T05:58:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.845+0000 7f4cd7e29640 1 Processor -- start 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.845+0000 7f4cd7e29640 1 -- start start 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.845+0000 7f4cd7e29640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0103320 0x7f4cd019b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.845+0000 7f4cd7e29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd0108bd0 0x7f4cd019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.845+0000 7f4cd7e29640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0109070 0x7f4cd019c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.845+0000 7f4cd7e29640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0112c70 con 0x7f4cd0109070 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.845+0000 7f4cd7e29640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0112af0 con 0x7f4cd0103320 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.845+0000 7f4cd7e29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0112df0 con 0x7f4cd0108bd0 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.845+0000 7f4cd63c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0109070 0x7f4cd019c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd63c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0109070 0x7f4cd019c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39586/0 (socket says 172.21.15.115:39586) 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd63c6640 1 -- 172.21.15.115:0/455141408 learned_addr learned my addr 172.21.15.115:0/455141408 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd0108bd0 0x7f4cd019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0103320 0x7f4cd019b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd63c6640 1 -- 172.21.15.115:0/455141408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd0108bd0 msgr2=0x7f4cd019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd63c6640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd0108bd0 0x7f4cd019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd63c6640 1 -- 172.21.15.115:0/455141408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0103320 msgr2=0x7f4cd019b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd63c6640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0103320 0x7f4cd019b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd63c6640 1 -- 172.21.15.115:0/455141408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd01a6970 con 0x7f4cd0109070 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0103320 0x7f4cd019b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd63c6640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0109070 0x7f4cd019c0d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800eb10 tx=0x7f4cc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cc6ffd640 1 -- 172.21.15.115:0/455141408 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8004b20 con 0x7f4cd0109070 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd7e29640 1 -- 172.21.15.115:0/455141408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd01a6c60 con 0x7f4cd0109070 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cc6ffd640 1 -- 172.21.15.115:0/455141408 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cc8004cc0 con 0x7f4cd0109070 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cc6ffd640 1 -- 172.21.15.115:0/455141408 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8010690 con 0x7f4cd0109070 2024-05-09T05:58:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.846+0000 7f4cd7e29640 1 -- 172.21.15.115:0/455141408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd01a7160 con 0x7f4cd0109070 2024-05-09T05:58:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.849+0000 7f4cdedb1740 1 Processor -- start 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.849+0000 7f4cc6ffd640 1 -- 172.21.15.115:0/455141408 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4cc8005020 con 0x7f4cd0109070 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.849+0000 7f4cdedb1740 1 -- start start 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618450081b0 0x5618453ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561844fdf2f0 0x5618453abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fe74d0 0x5618453ac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5618453107c0 con 0x5618450081b0 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5618452ac0e0 con 0x561844fe74d0 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56184530c780 con 0x561844fdf2f0 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd53c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618450081b0 0x5618453ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd53c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618450081b0 0x5618453ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39594/0 (socket says 172.21.15.115:39594) 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd53c4640 1 -- 172.21.15.115:0/3240083894 learned_addr learned my addr 172.21.15.115:0/3240083894 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd53c4640 1 -- 172.21.15.115:0/3240083894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561844fdf2f0 msgr2=0x5618453abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/3240083894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561844fdf2f0 0x5618453abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd53c4640 1 -- 172.21.15.115:0/3240083894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fe74d0 msgr2=0x5618453ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/3240083894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fe74d0 0x5618453ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd53c4640 1 -- 172.21.15.115:0/3240083894 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618452d57c0 con 0x5618450081b0 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cc6ffd640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4cbc0b2950 0x7f4cbc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/3240083894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618450081b0 0x5618453ab680 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000bcc0 tx=0x7f4cc000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ec99d4151fbf9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4cbc0b2950 0x7f4cbc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cc6ffd640 1 -- 172.21.15.115:0/455141408 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f4cc80e3040 con 0x7f4cd0109070 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.850+0000 7f4cbbfff640 1 -- 172.21.15.115:0/3240083894 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0015070 con 0x5618450081b0 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.851+0000 7f4cbbfff640 1 -- 172.21.15.115:0/3240083894 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cc001a8f0 con 0x5618450081b0 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.851+0000 7f4cbbfff640 1 -- 172.21.15.115:0/3240083894 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00117c0 con 0x5618450081b0 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.851+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3240083894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618450081b0 msgr2=0x5618453ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.851+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4cbc0b2950 0x7f4cbc0b4e60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5618453aca30 tx=0x7f4ccc000bc0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.851+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/3240083894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618450081b0 0x5618453ab680 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000bcc0 tx=0x7f4cc000f730 comp rx=0 tx=0).stop 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3240083894 shutdown_connections 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/3240083894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fe74d0 0x5618453ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/3240083894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561844fdf2f0 0x5618453abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/3240083894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5618450081b0 0x5618453ab680 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3240083894 >> 172.21.15.115:0/3240083894 conn(0x561844ffc300 msgr2=0x5618453a7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3240083894 shutdown_connections 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3240083894 wait complete. 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 -- start start 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 Processor -- start 2024-05-09T05:58:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 0x5618453b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 0x5618453b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5618453b87b0 con 0x561844fe74d0 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5618452979c0 con 0x561844fdf2f0 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.852+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5618453b8930 con 0x5618450081b0 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cdedb1740 10 client.0 _ll_register_callbacks cb 0x5618452c4020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd53c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd53c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39606/0 (socket says 172.21.15.115:39606) 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd53c4640 1 -- 172.21.15.115:0/758382439 learned_addr learned my addr 172.21.15.115:0/758382439 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 0x5618453b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd63c6640 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 0x5618453b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd53c4640 1 -- 172.21.15.115:0/758382439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 msgr2=0x5618453b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 0x5618453b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd53c4640 1 -- 172.21.15.115:0/758382439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 msgr2=0x5618453b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 0x5618453b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd53c4640 1 -- 172.21.15.115:0/758382439 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618452be1f0 con 0x561844fe74d0 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd63c6640 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 0x5618453b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453b7c40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000b560 tx=0x7f4cc001a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec503afadd634217 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cb97fa640 1 -- 172.21.15.115:0/758382439 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0015070 con 0x561844fe74d0 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cb97fa640 1 -- 172.21.15.115:0/758382439 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f4cc0004030 con 0x561844fe74d0 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cdedb1740 1 -- 172.21.15.115:0/758382439 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc001a8f0 con 0x561844fe74d0 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cb97fa640 1 -- 172.21.15.115:0/758382439 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4cc002d050 con 0x561844fe74d0 2024-05-09T05:58:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cdedb1740 10 client.19173 fetch_fsmap learned FSMap version 76 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.853+0000 7f4cdedb1740 1 -- 172.21.15.115:0/758382439 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x561845073790 con 0x561844fe74d0 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.854+0000 7f4cb97fa640 1 -- 172.21.15.115:0/758382439 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cc001a8f0 con 0x561844fe74d0 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.854+0000 7f4cb97fa640 1 -- 172.21.15.115:0/758382439 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 76) ==== 8382+0+0 (secure 0 0 0) 0x7f4cc0004330 con 0x561844fe74d0 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.854+0000 7f4cdedb1740 10 client.19173 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.854+0000 7f4cdedb1740 10 client.19173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.854+0000 7f4cdedb1740 20 client.19173 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.854+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561844fc6d80 0x5618453c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.854+0000 7f4cdedb1740 4 client.19173 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.854+0000 7f4cdedb1740 1 -- 172.21.15.115:0/758382439 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56184531e3d0 con 0x561844fc6d80 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.854+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561844fc6d80 0x5618453c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.855+0000 7f4cb97fa640 10 client.19173 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.855+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561844fc6d80 0x5618453c7ff0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d695ae249f74154 server_cookie=3af735d50c024e5a in_seq=0 out_seq=0 2024-05-09T05:58:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.855+0000 7f4cb97fa640 1 -- 172.21.15.115:0/758382439 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56184531e3d0 con 0x561844fc6d80 2024-05-09T05:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.855+0000 7f4cb97fa640 10 client.19173 handle_command_reply: tid=0 2024-05-09T05:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.855+0000 7f4cdedb1740 1 client.19173 shutdown 2024-05-09T05:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 -- 172.21.15.115:0/758382439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 msgr2=0x5618453b7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453b7c40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000b560 tx=0x7f4cc001a710 comp rx=0 tx=0).stop 2024-05-09T05:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 -- 172.21.15.115:0/758382439 shutdown_connections 2024-05-09T05:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561844fc6d80 0x5618453c7ff0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 0x5618453b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453b7c40 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cb97fa640 0 client.19173 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/758382439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 0x5618453b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 -- 172.21.15.115:0/758382439 >> 172.21.15.115:0/758382439 conn(0x561844ffc300 msgr2=0x5618453a6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 -- 172.21.15.115:0/758382439 shutdown_connections 2024-05-09T05:58:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 1 -- 172.21.15.115:0/758382439 wait complete. 2024-05-09T05:58:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.856+0000 7f4cdedb1740 20 client.19173 trim_cache size 0 max 16384 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cdedb1740 1 Processor -- start 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cdedb1740 1 -- start start 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 0x5618453a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 0x5618453cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd01b1070 con 0x561844fe74d0 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd01b2be0 con 0x561844fdf2f0 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd01b1740 con 0x5618450081b0 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd53c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd53c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39622/0 (socket says 172.21.15.115:39622) 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd53c4640 1 -- 172.21.15.115:0/3557176716 learned_addr learned my addr 172.21.15.115:0/3557176716 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/3557176716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 0x5618453a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd53c4640 1 -- 172.21.15.115:0/3557176716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 msgr2=0x5618453cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/3557176716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 0x5618453cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd53c4640 1 -- 172.21.15.115:0/3557176716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 msgr2=0x5618453a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/3557176716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 0x5618453a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd53c4640 1 -- 172.21.15.115:0/3557176716 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618450ea5f0 con 0x561844fe74d0 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.871+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/3557176716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453af940 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00058c0 tx=0x7f4cc0004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14adb9f860624232 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cbb7fe640 1 -- 172.21.15.115:0/3557176716 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0015070 con 0x561844fe74d0 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cbb7fe640 1 -- 172.21.15.115:0/3557176716 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cc0036070 con 0x561844fe74d0 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cbb7fe640 1 -- 172.21.15.115:0/3557176716 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc001f070 con 0x561844fe74d0 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3557176716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 msgr2=0x5618453af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/3557176716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453af940 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00058c0 tx=0x7f4cc0004680 comp rx=0 tx=0).stop 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3557176716 shutdown_connections 2024-05-09T05:58:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/3557176716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618450081b0 0x5618453cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/3557176716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453af940 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/3557176716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561844fdf2f0 0x5618453a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3557176716 >> 172.21.15.115:0/3557176716 conn(0x561844ffc300 msgr2=0x5618453a6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3557176716 shutdown_connections 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.872+0000 7f4cdedb1740 1 -- 172.21.15.115:0/3557176716 wait complete. 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cdedb1740 1 -- start start 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cdedb1740 1 Processor -- start 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561844fdf2f0 0x5618453c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cdedb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618450081b0 0x5618453ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd01b2510 con 0x561844fe74d0 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd01b1e10 con 0x5618450081b0 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cdedb1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd01af050 con 0x561844fdf2f0 2024-05-09T05:58:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cdedb1740 10 client.0 _ll_register_callbacks cb 0x7f4cc000bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd53c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd53c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39634/0 (socket says 172.21.15.115:39634) 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd53c4640 1 -- 172.21.15.115:0/710890630 learned_addr learned my addr 172.21.15.115:0/710890630 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561844fdf2f0 0x5618453c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd63c6640 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618450081b0 0x5618453ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd53c4640 1 -- 172.21.15.115:0/710890630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561844fdf2f0 msgr2=0x5618453c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561844fdf2f0 0x5618453c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd53c4640 1 -- 172.21.15.115:0/710890630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618450081b0 msgr2=0x5618453ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618450081b0 0x5618453ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.873+0000 7f4cd53c4640 1 -- 172.21.15.115:0/710890630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618452d4ed0 con 0x561844fe74d0 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cd63c6640 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618450081b0 0x5618453ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cd53c4640 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453ca460 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000be50 tx=0x7f4cc0004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2faeff6bd54acd6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cb97fa640 1 -- 172.21.15.115:0/710890630 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0015030 con 0x561844fe74d0 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cb97fa640 1 -- 172.21.15.115:0/710890630 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f4cc0004e30 con 0x561844fe74d0 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cdedb1740 1 -- 172.21.15.115:0/710890630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc0036070 con 0x561844fe74d0 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cb97fa640 1 -- 172.21.15.115:0/710890630 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4cc0048050 con 0x561844fe74d0 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cdedb1740 10 client.19179 fetch_fsmap learned FSMap version 76 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cdedb1740 1 -- 172.21.15.115:0/710890630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56184542f430 con 0x561844fe74d0 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cb97fa640 1 -- 172.21.15.115:0/710890630 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cc0036070 con 0x561844fe74d0 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cb97fa640 1 -- 172.21.15.115:0/710890630 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 76) ==== 8382+0+0 (secure 0 0 0) 0x7f4cc0011680 con 0x561844fe74d0 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cdedb1740 10 client.19179 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cdedb1740 10 client.19179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cdedb1740 20 client.19179 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561844f9c1c0 0x561845431af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cdedb1740 4 client.19179 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:58:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.874+0000 7f4cdedb1740 1 -- 172.21.15.115:0/710890630 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561845318240 con 0x561844f9c1c0 2024-05-09T05:58:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.875+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561844f9c1c0 0x561845431af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.875+0000 7f4cb97fa640 10 client.19179 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.875+0000 7f4cd5bc5640 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561844f9c1c0 0x561845431af0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3b7fcc659b0370 server_cookie=fc406fce63dad213 in_seq=0 out_seq=0 2024-05-09T05:58:43.868 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:43 smithi195 ceph-mon[33844]: pgmap v651: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 419 KiB/s wr, 113 op/s 2024-05-09T05:58:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.885+0000 7f4cb97fa640 1 -- 172.21.15.115:0/710890630 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561845318240 con 0x561844f9c1c0 2024-05-09T05:58:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.885+0000 7f4cb97fa640 10 client.19179 handle_command_reply: tid=0 2024-05-09T05:58:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.885+0000 7f4cdedb1740 1 client.19179 shutdown 2024-05-09T05:58:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 -- 172.21.15.115:0/710890630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 msgr2=0x5618453ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453ca460 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000be50 tx=0x7f4cc0004cf0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 -- 172.21.15.115:0/710890630 shutdown_connections 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618450081b0 0x5618453ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x561844fe74d0 0x5618453ca460 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x561844f9c1c0 0x561845431af0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 --2- 172.21.15.115:0/710890630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561844fdf2f0 0x5618453c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cb97fa640 0 client.19179 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 -- 172.21.15.115:0/710890630 >> 172.21.15.115:0/710890630 conn(0x561844ffc300 msgr2=0x5618453b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 -- 172.21.15.115:0/710890630 shutdown_connections 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 1 -- 172.21.15.115:0/710890630 wait complete. 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cdedb1740 20 client.19179 trim_cache size 0 max 16384 2024-05-09T05:58:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:58:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:58:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:58:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cd7e29640 1 -- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4cbc0b2950 msgr2=0x7f4cbc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4cbc0b2950 0x7f4cbc0b4e60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5618453aca30 tx=0x7f4ccc000bc0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cd7e29640 1 -- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0109070 msgr2=0x7f4cd019c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0109070 0x7f4cd019c0d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800eb10 tx=0x7f4cc800c2d0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.886+0000 7f4cd7e29640 1 -- 172.21.15.115:0/455141408 shutdown_connections 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.887+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f4cbc0b2950 0x7f4cbc0b4e60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.887+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd0109070 0x7f4cd019c0d0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.887+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd0108bd0 0x7f4cd019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.887+0000 7f4cd7e29640 1 --2- 172.21.15.115:0/455141408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0103320 0x7f4cd019b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.887+0000 7f4cd7e29640 1 -- 172.21.15.115:0/455141408 >> 172.21.15.115:0/455141408 conn(0x7f4cd00782d0 msgr2=0x7f4cd0104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.887+0000 7f4cd7e29640 1 -- 172.21.15.115:0/455141408 shutdown_connections 2024-05-09T05:58:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:43.887+0000 7f4cd7e29640 1 -- 172.21.15.115:0/455141408 wait complete. 2024-05-09T05:58:43.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:43 smithi161 ceph-mon[33805]: pgmap v651: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 419 KiB/s wr, 113 op/s 2024-05-09T05:58:45.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:45 smithi195 ceph-mon[33844]: pgmap v652: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 395 KiB/s wr, 111 op/s 2024-05-09T05:58:45.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:45 smithi115 ceph-mon[30349]: pgmap v652: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 395 KiB/s wr, 111 op/s 2024-05-09T05:58:45.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:45 smithi161 ceph-mon[33805]: pgmap v652: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 395 KiB/s wr, 111 op/s 2024-05-09T05:58:47.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:47 smithi195 ceph-mon[33844]: pgmap v653: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 165 KiB/s wr, 51 op/s 2024-05-09T05:58:47.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:47 smithi115 ceph-mon[30349]: pgmap v653: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 165 KiB/s wr, 51 op/s 2024-05-09T05:58:47.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:47 smithi161 ceph-mon[33805]: pgmap v653: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 165 KiB/s wr, 51 op/s 2024-05-09T05:58:49.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:49 smithi195 ceph-mon[33844]: pgmap v654: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 129 KiB/s wr, 43 op/s 2024-05-09T05:58:49.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:49 smithi115 ceph-mon[30349]: pgmap v654: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 129 KiB/s wr, 43 op/s 2024-05-09T05:58:49.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:49 smithi161 ceph-mon[33805]: pgmap v654: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 129 KiB/s wr, 43 op/s 2024-05-09T05:58:50.186 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:58:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:58:51.806 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:51 smithi115 ceph-mon[30349]: pgmap v655: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 37 op/s 2024-05-09T05:58:51.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:51 smithi195 ceph-mon[33844]: pgmap v655: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 37 op/s 2024-05-09T05:58:51.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:51 smithi161 ceph-mon[33805]: pgmap v655: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 37 op/s 2024-05-09T05:58:52.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.342+0000 7ff3e5725700 1 -- 172.21.15.115:0/2068753129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 msgr2=0x7ff3e0109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.342+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2068753129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 0x7ff3e0109310 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40099c0 tx=0x7ff3d4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.342+0000 7ff3e5725700 1 -- 172.21.15.115:0/2068753129 shutdown_connections 2024-05-09T05:58:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.342+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2068753129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3e0109850 0x7ff3e01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.342+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2068753129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3e0102ae0 0x7ff3e0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.342+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2068753129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 0x7ff3e0109310 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.342+0000 7ff3e5725700 1 -- 172.21.15.115:0/2068753129 >> 172.21.15.115:0/2068753129 conn(0x7ff3e00fe520 msgr2=0x7ff3e0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.342+0000 7ff3e5725700 1 -- 172.21.15.115:0/2068753129 shutdown_connections 2024-05-09T05:58:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.342+0000 7ff3e5725700 1 -- 172.21.15.115:0/2068753129 wait complete. 2024-05-09T05:58:52.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3e5725700 1 Processor -- start 2024-05-09T05:58:52.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3e5725700 1 -- start start 2024-05-09T05:58:52.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3e5725700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3e0102ae0 0x7ff3e019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:52.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3e5725700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 0x7ff3e019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:52.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3e5725700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3e0109850 0x7ff3e019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:52.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3e5725700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0114310 con 0x7ff3e0108f00 2024-05-09T05:58:52.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3e5725700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0114190 con 0x7ff3e0109850 2024-05-09T05:58:52.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3e5725700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0114490 con 0x7ff3e0102ae0 2024-05-09T05:58:52.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3de7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 0x7ff3e019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:52.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.343+0000 7ff3de7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 0x7ff3e019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39658/0 (socket says 172.21.15.115:39658) 2024-05-09T05:58:52.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3de7fc700 1 -- 172.21.15.115:0/2247118747 learned_addr learned my addr 172.21.15.115:0/2247118747 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:52.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3deffd700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3e0102ae0 0x7ff3e019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:52.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3df7fe700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3e0109850 0x7ff3e019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:52.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3de7fc700 1 -- 172.21.15.115:0/2247118747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3e0102ae0 msgr2=0x7ff3e019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:52.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3de7fc700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3e0102ae0 0x7ff3e019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3de7fc700 1 -- 172.21.15.115:0/2247118747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3e0109850 msgr2=0x7ff3e019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3de7fc700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3e0109850 0x7ff3e019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3de7fc700 1 -- 172.21.15.115:0/2247118747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e019fc10 con 0x7ff3e0108f00 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3de7fc700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 0x7ff3e019b2e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400b580 tx=0x7ff3d40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3c7fff700 1 -- 172.21.15.115:0/2247118747 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4027070 con 0x7ff3e0108f00 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3c7fff700 1 -- 172.21.15.115:0/2247118747 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3d400fc80 con 0x7ff3e0108f00 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e019fea0 con 0x7ff3e0108f00 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3c7fff700 1 -- 172.21.15.115:0/2247118747 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4022da0 con 0x7ff3e0108f00 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.344+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e01a0380 con 0x7ff3e0108f00 2024-05-09T05:58:52.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.345+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e00512c0 con 0x7ff3e0108f00 2024-05-09T05:58:52.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.348+0000 7ff3c7fff700 1 -- 172.21.15.115:0/2247118747 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7ff3d402a070 con 0x7ff3e0108f00 2024-05-09T05:58:52.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.348+0000 7ff3c7fff700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3c80b29f0 0x7ff3c80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:52.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.348+0000 7ff3deffd700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3c80b29f0 0x7ff3c80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:52.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.348+0000 7ff3c7fff700 1 -- 172.21.15.115:0/2247118747 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7ff3d40e38c0 con 0x7ff3e0108f00 2024-05-09T05:58:52.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.349+0000 7ff3deffd700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3c80b29f0 0x7ff3c80b4f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0007be0 tx=0x7ff3d00073d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:52.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.352+0000 7ff3c7fff700 1 -- 172.21.15.115:0/2247118747 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3d405fdf0 con 0x7ff3e0108f00 2024-05-09T05:58:52.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.604+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3e002e5f0 con 0x7ff3e0108f00 2024-05-09T05:58:52.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.605+0000 7ff3c7fff700 1 -- 172.21.15.115:0/2247118747 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11962 (secure 0 0 0) 0x7ff3d4013070 con 0x7ff3e0108f00 2024-05-09T05:58:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:58:52.592 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:58:33.383061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28438":{"gid":28438,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/3680433414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3680433414},{"type":"v1","addr":"172.21.15.195:6833","nonce":3680433414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:58:52.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3c80b29f0 msgr2=0x7ff3c80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:52.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3c80b29f0 0x7ff3c80b4f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0007be0 tx=0x7ff3d00073d0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 msgr2=0x7ff3e019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:52.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 0x7ff3e019b2e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400b580 tx=0x7ff3d40057c0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 shutdown_connections 2024-05-09T05:58:52.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7ff3c80b29f0 0x7ff3c80b4f00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3e0109850 0x7ff3e019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3e0102ae0 0x7ff3e019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 --2- 172.21.15.115:0/2247118747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e0108f00 0x7ff3e019b2e0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:52.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 >> 172.21.15.115:0/2247118747 conn(0x7ff3e00fe520 msgr2=0x7ff3e0100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:52.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 shutdown_connections 2024-05-09T05:58:52.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:52.607+0000 7ff3e5725700 1 -- 172.21.15.115:0/2247118747 wait complete. 2024-05-09T05:58:52.595 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 76 2024-05-09T05:58:53.333 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id f2e9c488 --release --await' 2024-05-09T05:58:53.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id f2e9c488 --release --await 2024-05-09T05:58:53.580 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:58:53.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:53 smithi115 ceph-mon[30349]: pgmap v656: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-09T05:58:53.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:53 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2247118747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:58:53.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:53 smithi195 ceph-mon[33844]: pgmap v656: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-09T05:58:53.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:53 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2247118747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:58:53.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:53 smithi161 ceph-mon[33805]: pgmap v656: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-09T05:58:53.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:53 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2247118747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:58:55.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.500+0000 7fd17a20d700 1 -- 172.21.15.115:0/1319129198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd174108750 msgr2=0x7fd1741100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.500+0000 7fd17a20d700 1 --2- 172.21.15.115:0/1319129198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd174108750 0x7fd1741100b0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b0d0 tx=0x7fd16c00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.500+0000 7fd17a20d700 1 -- 172.21.15.115:0/1319129198 shutdown_connections 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.500+0000 7fd17a20d700 1 --2- 172.21.15.115:0/1319129198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1741034d0 0x7fd1741038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.500+0000 7fd17a20d700 1 --2- 172.21.15.115:0/1319129198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd174108750 0x7fd1741100b0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.500+0000 7fd17a20d700 1 --2- 172.21.15.115:0/1319129198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd174107e00 0x7fd174108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.500+0000 7fd17a20d700 1 -- 172.21.15.115:0/1319129198 >> 172.21.15.115:0/1319129198 conn(0x7fd1740fef20 msgr2=0x7fd174101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.500+0000 7fd17a20d700 1 -- 172.21.15.115:0/1319129198 shutdown_connections 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.500+0000 7fd17a20d700 1 -- 172.21.15.115:0/1319129198 wait complete. 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd17a20d700 1 Processor -- start 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd17a20d700 1 -- start start 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd17a20d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1741034d0 0x7fd17419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd17a20d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd174107e00 0x7fd17419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd17a20d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd174108750 0x7fd174199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd17a20d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741132b0 con 0x7fd1741034d0 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd17a20d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd174113130 con 0x7fd174107e00 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd17a20d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd174113430 con 0x7fd174108750 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1741034d0 0x7fd17419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1741034d0 0x7fd17419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42076/0 (socket says 172.21.15.115:42076) 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd1737fe700 1 -- 172.21.15.115:0/296425432 learned_addr learned my addr 172.21.15.115:0/296425432 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd16affd700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd174107e00 0x7fd17419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.501+0000 7fd173fff700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd174108750 0x7fd174199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd1737fe700 1 -- 172.21.15.115:0/296425432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd174108750 msgr2=0x7fd174199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd1737fe700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd174108750 0x7fd174199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd1737fe700 1 -- 172.21.15.115:0/296425432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd174107e00 msgr2=0x7fd17419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd1737fe700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd174107e00 0x7fd17419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd1737fe700 1 -- 172.21.15.115:0/296425432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17419a120 con 0x7fd1741034d0 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd1737fe700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1741034d0 0x7fd17419f850 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd16400d9e0 tx=0x7fd16400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd1717fa700 1 -- 172.21.15.115:0/296425432 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16401b070 con 0x7fd1741034d0 2024-05-09T05:58:55.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd1717fa700 1 -- 172.21.15.115:0/296425432 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd16400ec50 con 0x7fd1741034d0 2024-05-09T05:58:55.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd17a20d700 1 -- 172.21.15.115:0/296425432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17419a410 con 0x7fd1741034d0 2024-05-09T05:58:55.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd1717fa700 1 -- 172.21.15.115:0/296425432 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1640145a0 con 0x7fd1741034d0 2024-05-09T05:58:55.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.502+0000 7fd17a20d700 1 -- 172.21.15.115:0/296425432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17406b610 con 0x7fd1741034d0 2024-05-09T05:58:55.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.505+0000 7fd1717fa700 1 -- 172.21.15.115:0/296425432 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fd16401e070 con 0x7fd1741034d0 2024-05-09T05:58:55.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd1717fa700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd1600b29f0 0x7fd1600b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd188c23b80 1 Processor -- start 2024-05-09T05:58:55.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd1717fa700 1 -- 172.21.15.115:0/296425432 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7fd1640e3f20 con 0x7fd1741034d0 2024-05-09T05:58:55.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd16affd700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd1600b29f0 0x7fd1600b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd188c23b80 1 -- start start 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0daf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd188c23b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c1aaffd0f0 con 0x55c1ab0dab50 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd188c23b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c1aaf9b8f0 con 0x55c1ab047260 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd188c23b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c1aaff90b0 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd16affd700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd1600b29f0 0x7fd1600b4f00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd1741024c0 tx=0x7fd15c00a300 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd173fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0daf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd173fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0daf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42088/0 (socket says 172.21.15.115:42088) 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd173fff700 1 -- 172.21.15.115:0/1930193460 learned_addr learned my addr 172.21.15.115:0/1930193460 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.506+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd16affd700 1 --2- 172.21.15.115:0/1930193460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd173fff700 1 -- 172.21.15.115:0/1930193460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1aaf5d5a0 msgr2=0x55c1ab0e0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd173fff700 1 --2- 172.21.15.115:0/1930193460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd173fff700 1 -- 172.21.15.115:0/1930193460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 msgr2=0x55c1ab0e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:55.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd173fff700 1 --2- 172.21.15.115:0/1930193460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd173fff700 1 -- 172.21.15.115:0/1930193460 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1aaf93470 con 0x55c1ab0dab50 2024-05-09T05:58:55.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd1737fe700 1 --2- 172.21.15.115:0/1930193460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:58:55.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd173fff700 1 --2- 172.21.15.115:0/1930193460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0daf60 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009e30 tx=0x7fd16c0090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76781f25b6d1edb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:55.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd169ffb700 1 -- 172.21.15.115:0/1930193460 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c020030 con 0x55c1ab0dab50 2024-05-09T05:58:55.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd169ffb700 1 -- 172.21.15.115:0/1930193460 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd16c0092c0 con 0x55c1ab0dab50 2024-05-09T05:58:55.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.507+0000 7fd169ffb700 1 -- 172.21.15.115:0/1930193460 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c005050 con 0x55c1ab0dab50 2024-05-09T05:58:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 -- 172.21.15.115:0/1930193460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 msgr2=0x55c1ab0daf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1930193460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0daf60 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009e30 tx=0x7fd16c0090d0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 -- 172.21.15.115:0/1930193460 shutdown_connections 2024-05-09T05:58:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1930193460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1930193460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0daf60 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1930193460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 -- 172.21.15.115:0/1930193460 >> 172.21.15.115:0/1930193460 conn(0x55c1aafbd7a0 msgr2=0x55c1ab0d5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 -- 172.21.15.115:0/1930193460 shutdown_connections 2024-05-09T05:58:55.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 -- 172.21.15.115:0/1930193460 wait complete. 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.508+0000 7fd188c23b80 1 -- start start 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd188c23b80 1 Processor -- start 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab047260 0x55c1ab0e4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0e5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd188c23b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55c1aae78ca0 con 0x55c1ab0dab50 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd188c23b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c1aafb7250 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd188c23b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c1ab0db970 con 0x55c1ab047260 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd188c23b80 10 client.0 _ll_register_callbacks cb 0x55c1aae6e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd173fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd173fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0e5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42102/0 (socket says 172.21.15.115:42102) 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd173fff700 1 -- 172.21.15.115:0/2488145284 learned_addr learned my addr 172.21.15.115:0/2488145284 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd16affd700 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab047260 0x55c1ab0e4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd1737fe700 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd173fff700 1 -- 172.21.15.115:0/2488145284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab047260 msgr2=0x55c1ab0e4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd173fff700 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab047260 0x55c1ab0e4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd173fff700 1 -- 172.21.15.115:0/2488145284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1aaf5d5a0 msgr2=0x55c1ab0e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd173fff700 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.509+0000 7fd173fff700 1 -- 172.21.15.115:0/2488145284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1aae78590 con 0x55c1ab0dab50 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd173fff700 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0e5160 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b0d0 tx=0x7fd16c007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f49f97e661f6a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd152ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd152ffd700 1 -- 172.21.15.115:0/2488145284 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c02b070 con 0x55c1ab0dab50 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd152ffd700 1 -- 172.21.15.115:0/2488145284 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7fd16c004c50 con 0x55c1ab0dab50 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd188c23b80 1 -- 172.21.15.115:0/2488145284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16c0092c0 con 0x55c1ab0dab50 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd152ffd700 1 -- 172.21.15.115:0/2488145284 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd16c020040 con 0x55c1ab0dab50 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd188c23b80 10 client.19197 fetch_fsmap learned FSMap version 76 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd188c23b80 1 -- 172.21.15.115:0/2488145284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55c1ab0f2010 con 0x55c1ab0dab50 2024-05-09T05:58:55.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd152ffd700 1 -- 172.21.15.115:0/2488145284 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd16c0092c0 con 0x55c1ab0dab50 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd152ffd700 1 -- 172.21.15.115:0/2488145284 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 76) ==== 8382+0+0 (secure 0 0 0) 0x7fd16c011070 con 0x55c1ab0dab50 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd188c23b80 10 client.19197 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd188c23b80 10 client.19197 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd188c23b80 20 client.19197 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd188c23b80 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c1ab0f44e0 0x55c1ab0f6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd188c23b80 4 client.19197 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.510+0000 7fd188c23b80 1 -- 172.21.15.115:0/2488145284 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1ab00aad0 con 0x55c1ab0f44e0 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.511+0000 7fd1737fe700 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c1ab0f44e0 0x55c1ab0f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.511+0000 7fd1737fe700 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c1ab0f44e0 0x55c1ab0f6900 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df18f4ccc27f1f97 server_cookie=cce78750352e6fcb in_seq=0 out_seq=0 2024-05-09T05:58:55.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.511+0000 7fd152ffd700 10 client.19197 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd152ffd700 1 -- 172.21.15.115:0/2488145284 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c1ab00aad0 con 0x55c1ab0f44e0 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd152ffd700 10 client.19197 handle_command_reply: tid=0 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 client.19197 shutdown 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 -- 172.21.15.115:0/2488145284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 msgr2=0x55c1ab0e5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0e5160 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b0d0 tx=0x7fd16c007e90 comp rx=0 tx=0).stop 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 -- 172.21.15.115:0/2488145284 shutdown_connections 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c1ab0f44e0 0x55c1ab0f6900 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1ab0dab50 0x55c1ab0e5160 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 --2- 172.21.15.115:0/2488145284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab047260 0x55c1ab0e4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 -- 172.21.15.115:0/2488145284 >> 172.21.15.115:0/2488145284 conn(0x55c1aafbd7a0 msgr2=0x55c1ab0d5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd152ffd700 0 client.19197 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 -- 172.21.15.115:0/2488145284 shutdown_connections 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 1 -- 172.21.15.115:0/2488145284 wait complete. 2024-05-09T05:58:55.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.512+0000 7fd188c23b80 20 client.19197 trim_cache size 0 max 16384 2024-05-09T05:58:55.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.532+0000 7fd188c23b80 1 Processor -- start 2024-05-09T05:58:55.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.532+0000 7fd188c23b80 1 -- start start 2024-05-09T05:58:55.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.532+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.532+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0df3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.532+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab0dab50 0x55c1ab0df8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.532+0000 7fd188c23b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741ae080 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.532+0000 7fd188c23b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741afbf0 con 0x55c1ab047260 2024-05-09T05:58:55.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.532+0000 7fd188c23b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741ae750 con 0x55c1ab0dab50 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.532+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42108/0 (socket says 172.21.15.115:42108) 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1737fe700 1 -- 172.21.15.115:0/3545050163 learned_addr learned my addr 172.21.15.115:0/3545050163 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd173fff700 1 --2- 172.21.15.115:0/3545050163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab0dab50 0x55c1ab0df8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd16affd700 1 --2- 172.21.15.115:0/3545050163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0df3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1737fe700 1 -- 172.21.15.115:0/3545050163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab0dab50 msgr2=0x55c1ab0df8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1737fe700 1 --2- 172.21.15.115:0/3545050163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab0dab50 0x55c1ab0df8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1737fe700 1 -- 172.21.15.115:0/3545050163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 msgr2=0x55c1ab0df3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1737fe700 1 --2- 172.21.15.115:0/3545050163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0df3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1737fe700 1 -- 172.21.15.115:0/3545050163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1ab0d7290 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1737fe700 1 --2- 172.21.15.115:0/3545050163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e38d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd1640e14f0 tx=0x7fd1640a6fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eac42e8040976c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1697fa700 1 -- 172.21.15.115:0/3545050163 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1640b3e30 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1697fa700 1 -- 172.21.15.115:0/3545050163 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1640c4070 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.533+0000 7fd1697fa700 1 -- 172.21.15.115:0/3545050163 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1640b39a0 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 -- 172.21.15.115:0/3545050163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 msgr2=0x55c1ab0e38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:55.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 --2- 172.21.15.115:0/3545050163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e38d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd1640e14f0 tx=0x7fd1640a6fc0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 -- 172.21.15.115:0/3545050163 shutdown_connections 2024-05-09T05:58:55.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 --2- 172.21.15.115:0/3545050163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab0dab50 0x55c1ab0df8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 --2- 172.21.15.115:0/3545050163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0e38d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 --2- 172.21.15.115:0/3545050163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0df3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 -- 172.21.15.115:0/3545050163 >> 172.21.15.115:0/3545050163 conn(0x55c1aafbd7a0 msgr2=0x55c1ab0e4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 -- 172.21.15.115:0/3545050163 shutdown_connections 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 -- 172.21.15.115:0/3545050163 wait complete. 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 -- start start 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.534+0000 7fd188c23b80 1 Processor -- start 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd188c23b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab0dab50 0x55c1ab0f8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd188c23b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741af520 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd188c23b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741aee20 con 0x55c1ab047260 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd188c23b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741ac020 con 0x55c1ab0dab50 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd188c23b80 10 client.0 _ll_register_callbacks cb 0x55c1aae6e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:55.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42110/0 (socket says 172.21.15.115:42110) 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd1737fe700 1 -- 172.21.15.115:0/1217477845 learned_addr learned my addr 172.21.15.115:0/1217477845 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd16affd700 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd1737fe700 1 -- 172.21.15.115:0/1217477845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab0dab50 msgr2=0x55c1ab0f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd1737fe700 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab0dab50 0x55c1ab0f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd1737fe700 1 -- 172.21.15.115:0/1217477845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 msgr2=0x55c1ab0f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd1737fe700 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd1737fe700 1 -- 172.21.15.115:0/1217477845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1ab0db1b0 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd1737fe700 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0f9b80 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd1640a7050 tx=0x7fd164016b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c64b52e3dfb494 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd152ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.535+0000 7fd152ffd700 1 -- 172.21.15.115:0/1217477845 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1640ab440 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd152ffd700 1 -- 172.21.15.115:0/1217477845 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7fd1640b39a0 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd188c23b80 1 -- 172.21.15.115:0/1217477845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1640c4070 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd152ffd700 1 -- 172.21.15.115:0/1217477845 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd1640b3ba0 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd188c23b80 10 client.19203 fetch_fsmap learned FSMap version 76 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd188c23b80 1 -- 172.21.15.115:0/1217477845 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55c1ab163d20 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd152ffd700 1 -- 172.21.15.115:0/1217477845 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1640c4070 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd152ffd700 1 -- 172.21.15.115:0/1217477845 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 76) ==== 8382+0+0 (secure 0 0 0) 0x7fd16400e040 con 0x55c1aaf5d5a0 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd188c23b80 10 client.19203 fetch_fsmap finished waiting for FSMap version 76 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd188c23b80 10 client.19203 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd188c23b80 20 client.19203 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c1ab164490 0x55c1ab1668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd188c23b80 4 client.19203 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f2e9c488", "release": true, "await": true}] 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd188c23b80 1 -- 172.21.15.115:0/1217477845 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f2e9c488", "release": true, "await": true}) -- 0x55c1ab004b70 con 0x55c1ab164490 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.536+0000 7fd16affd700 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c1ab164490 0x55c1ab1668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.537+0000 7fd16affd700 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c1ab164490 0x55c1ab1668b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfeaccd6a0d8775 server_cookie=f7c5803c2fb9d747 in_seq=0 out_seq=0 2024-05-09T05:58:55.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.537+0000 7fd152ffd700 10 client.19203 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:55.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd152ffd700 1 -- 172.21.15.115:0/1217477845 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c1ab004b70 con 0x55c1ab164490 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd152ffd700 10 client.19203 handle_command_reply: tid=0 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 client.19203 shutdown 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 -- 172.21.15.115:0/1217477845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 msgr2=0x55c1ab0f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0f9b80 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd1640a7050 tx=0x7fd164016b90 comp rx=0 tx=0).stop 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 -- 172.21.15.115:0/1217477845 shutdown_connections 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55c1ab164490 0x55c1ab1668b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1ab0dab50 0x55c1ab0f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c1aaf5d5a0 0x55c1ab0f9b80 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 --2- 172.21.15.115:0/1217477845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1ab047260 0x55c1ab0f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 -- 172.21.15.115:0/1217477845 >> 172.21.15.115:0/1217477845 conn(0x55c1aafbd7a0 msgr2=0x55c1ab0e5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd152ffd700 0 client.19203 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 -- 172.21.15.115:0/1217477845 shutdown_connections 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 1 -- 172.21.15.115:0/1217477845 wait complete. 2024-05-09T05:58:55.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.552+0000 7fd188c23b80 20 client.19203 trim_cache size 0 max 16384 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 76, 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 56, 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "f2e9c488": { 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 56, 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:58:55.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 140.9, 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:58:55.542 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:58:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 -- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd1600b29f0 msgr2=0x7fd1600b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd1600b29f0 0x7fd1600b4f00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd1741024c0 tx=0x7fd15c00a300 comp rx=0 tx=0).stop 2024-05-09T05:58:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 -- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1741034d0 msgr2=0x7fd17419f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1741034d0 0x7fd17419f850 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd16400d9e0 tx=0x7fd16400dcf0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 -- 172.21.15.115:0/296425432 shutdown_connections 2024-05-09T05:58:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fd1600b29f0 0x7fd1600b4f00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd174108750 0x7fd174199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1741034d0 0x7fd17419f850 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 --2- 172.21.15.115:0/296425432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd174107e00 0x7fd17419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 -- 172.21.15.115:0/296425432 >> 172.21.15.115:0/296425432 conn(0x7fd1740fef20 msgr2=0x7fd174107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 -- 172.21.15.115:0/296425432 shutdown_connections 2024-05-09T05:58:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:55.553+0000 7fd17a20d700 1 -- 172.21.15.115:0/296425432 wait complete. 2024-05-09T05:58:55.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:55 smithi195 ceph-mon[33844]: pgmap v657: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T05:58:55.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:55 smithi115 ceph-mon[30349]: pgmap v657: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T05:58:55.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:55 smithi161 ceph-mon[33805]: pgmap v657: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-05-09T05:58:56.366 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f2e9c488', total seconds elapsed: 27.1 2024-05-09T05:58:56.367 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.95 2024-05-09T05:58:56.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:58:56.662 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:58:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.013+0000 7f33d6870640 1 -- 172.21.15.115:0/2421706167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d01021d0 msgr2=0x7f33d01025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.013+0000 7f33d6870640 1 --2- 172.21.15.115:0/2421706167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d01021d0 0x7f33d01025e0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009900 tx=0x7f33c402a3f0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.013+0000 7f33d6870640 1 -- 172.21.15.115:0/2421706167 shutdown_connections 2024-05-09T05:58:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.013+0000 7f33d6870640 1 --2- 172.21.15.115:0/2421706167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33d010e6d0 0x7f33d0110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.013+0000 7f33d6870640 1 --2- 172.21.15.115:0/2421706167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d01021d0 0x7f33d01025e0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.013+0000 7f33d6870640 1 --2- 172.21.15.115:0/2421706167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d0109d20 0x7f33d0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.013+0000 7f33d6870640 1 -- 172.21.15.115:0/2421706167 >> 172.21.15.115:0/2421706167 conn(0x7f33d00779e0 msgr2=0x7f33d0077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.013+0000 7f33d6870640 1 -- 172.21.15.115:0/2421706167 shutdown_connections 2024-05-09T05:58:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.013+0000 7f33d6870640 1 -- 172.21.15.115:0/2421706167 wait complete. 2024-05-09T05:58:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.014+0000 7f33d6870640 1 Processor -- start 2024-05-09T05:58:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.014+0000 7f33d6870640 1 -- start start 2024-05-09T05:58:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d6870640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d01021d0 0x7f33d019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d6870640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33d0109d20 0x7f33d019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d6870640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d010e6d0 0x7f33d019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d6870640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0113d40 con 0x7f33d010e6d0 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d6870640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0113bc0 con 0x7f33d0109d20 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d6870640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0113ec0 con 0x7f33d01021d0 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d4e0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d010e6d0 0x7f33d019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33cf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33d0109d20 0x7f33d019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33cffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d01021d0 0x7f33d019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d4e0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d010e6d0 0x7f33d019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42122/0 (socket says 172.21.15.115:42122) 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d4e0d640 1 -- 172.21.15.115:0/2290341353 learned_addr learned my addr 172.21.15.115:0/2290341353 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33cf7fe640 1 -- 172.21.15.115:0/2290341353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d01021d0 msgr2=0x7f33d019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d01021d0 0x7f33d019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33cf7fe640 1 -- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d010e6d0 msgr2=0x7f33d019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d010e6d0 0x7f33d019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33cf7fe640 1 -- 172.21.15.115:0/2290341353 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d019f640 con 0x7f33d0109d20 2024-05-09T05:58:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33cffff640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d01021d0 0x7f33d019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:58:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33d4e0d640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d010e6d0 0x7f33d019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:58:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.015+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33d0109d20 0x7f33d019ad40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009230 tx=0x7f33c4005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.016+0000 7f33cd7fa640 1 -- 172.21.15.115:0/2290341353 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c403f070 con 0x7f33d0109d20 2024-05-09T05:58:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.016+0000 7f33cd7fa640 1 -- 172.21.15.115:0/2290341353 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33c4003a80 con 0x7f33d0109d20 2024-05-09T05:58:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.016+0000 7f33cd7fa640 1 -- 172.21.15.115:0/2290341353 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4035070 con 0x7f33d0109d20 2024-05-09T05:58:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.016+0000 7f33d6870640 1 -- 172.21.15.115:0/2290341353 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d019f8d0 con 0x7f33d0109d20 2024-05-09T05:58:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.016+0000 7f33d6870640 1 -- 172.21.15.115:0/2290341353 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d019fd40 con 0x7f33d0109d20 2024-05-09T05:58:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33dd800740 1 Processor -- start 2024-05-09T05:58:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33dd800740 1 -- start start 2024-05-09T05:58:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33dd800740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e72c1b0 0x556f2ead3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33dd800740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2ead3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33dd800740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2ead4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33dd800740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556f2ea347c0 con 0x556f2e7032f0 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33dd800740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556f2e9d00e0 con 0x556f2e72c1b0 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33dd800740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f2ea30780 con 0x556f2e70b4d0 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33cffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2ead3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33d4e0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e72c1b0 0x556f2ead3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33d4e0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e72c1b0 0x556f2ead3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51624/0 (socket says 172.21.15.115:51624) 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33cffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2ead3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42124/0 (socket says 172.21.15.115:42124) 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33d4e0d640 1 -- 172.21.15.115:0/3144207106 learned_addr learned my addr 172.21.15.115:0/3144207106 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.019+0000 7f33cd7fa640 1 -- 172.21.15.115:0/2290341353 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f33c4043400 con 0x7f33d0109d20 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33d4e0d640 1 -- 172.21.15.115:0/3144207106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 msgr2=0x556f2ead4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2ead4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33d4e0d640 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2ead4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33d4e0d640 1 -- 172.21.15.115:0/3144207106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 msgr2=0x556f2ead3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33d4e0d640 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2ead3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33d4e0d640 1 -- 172.21.15.115:0/3144207106 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2e9f97c0 con 0x556f2e72c1b0 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33cffff640 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2ead3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2ead4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33d4e0d640 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e72c1b0 0x556f2ead3a90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f33c8011880 tx=0x7f33c8011d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96489fe81ad456c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.020+0000 7f33cd7fa640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33b80b2950 0x7f33b80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.021+0000 7f33cffff640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33b80b2950 0x7f33b80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.021+0000 7f33b67fc640 1 -- 172.21.15.115:0/3144207106 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8011310 con 0x556f2e72c1b0 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.021+0000 7f33cd7fa640 1 -- 172.21.15.115:0/2290341353 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f33c4102ab0 con 0x7f33d0109d20 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.021+0000 7f33b67fc640 1 -- 172.21.15.115:0/3144207106 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33c800fe70 con 0x556f2e72c1b0 2024-05-09T05:58:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.021+0000 7f33b67fc640 1 -- 172.21.15.115:0/3144207106 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c80193e0 con 0x556f2e72c1b0 2024-05-09T05:58:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33cffff640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33b80b2950 0x7f33b80b4e60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009fd0 tx=0x7f33c0005f90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33dd800740 1 -- 172.21.15.115:0/3144207106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e72c1b0 msgr2=0x556f2ead3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33dd800740 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e72c1b0 0x556f2ead3a90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f33c8011880 tx=0x7f33c8011d40 comp rx=0 tx=0).stop 2024-05-09T05:58:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33dd800740 1 -- 172.21.15.115:0/3144207106 shutdown_connections 2024-05-09T05:58:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33dd800740 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2ead4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33dd800740 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2ead3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33dd800740 1 --2- 172.21.15.115:0/3144207106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e72c1b0 0x556f2ead3a90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33dd800740 1 -- 172.21.15.115:0/3144207106 >> 172.21.15.115:0/3144207106 conn(0x556f2e720300 msgr2=0x556f2ead0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33dd800740 1 -- 172.21.15.115:0/3144207106 shutdown_connections 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.022+0000 7f33dd800740 1 -- 172.21.15.115:0/3144207106 wait complete. 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.023+0000 7f33dd800740 1 -- start start 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.023+0000 7f33dd800740 1 Processor -- start 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33dd800740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 0x556f2eadfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33dd800740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e70b4d0 0x556f2eae0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33dd800740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e72c1b0 0x556f2eaddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33dd800740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556f2eae0bc0 con 0x556f2e70b4d0 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33dd800740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556f2e9bb9c0 con 0x556f2e7032f0 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33dd800740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556f2eae0d40 con 0x556f2e72c1b0 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33dd800740 10 client.0 _ll_register_callbacks cb 0x556f2e9e8020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33d4e0d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e72c1b0 0x556f2eaddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33cffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 0x556f2eadfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33d4e0d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e72c1b0 0x556f2eaddff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:45114/0 (socket says 172.21.15.115:45114) 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33cffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 0x556f2eadfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51630/0 (socket says 172.21.15.115:51630) 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33d4e0d640 1 -- 172.21.15.115:0/251319601 learned_addr learned my addr 172.21.15.115:0/251319601 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e70b4d0 0x556f2eae0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33d4e0d640 1 -- 172.21.15.115:0/251319601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 msgr2=0x556f2eadfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33d4e0d640 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 0x556f2eadfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33d4e0d640 1 -- 172.21.15.115:0/251319601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e70b4d0 msgr2=0x556f2eae0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33d4e0d640 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e70b4d0 0x556f2eae0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33d4e0d640 1 -- 172.21.15.115:0/251319601 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2e9e21f0 con 0x556f2e72c1b0 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33d4e0d640 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e72c1b0 0x556f2eaddff0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f33c8010b90 tx=0x7f33c800fb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5d28440a892ed2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.024+0000 7f33af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33af7fe640 1 -- 172.21.15.115:0/251319601 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8029060 con 0x556f2e72c1b0 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33af7fe640 1 -- 172.21.15.115:0/251319601 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f33c8004c50 con 0x556f2e72c1b0 2024-05-09T05:58:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33dd800740 1 -- 172.21.15.115:0/251319601 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33c800fe70 con 0x556f2e72c1b0 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33af7fe640 1 -- 172.21.15.115:0/251319601 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f33c8030050 con 0x556f2e72c1b0 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33dd800740 10 client.28516 fetch_fsmap learned FSMap version 77 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33dd800740 1 -- 172.21.15.115:0/251319601 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x556f2e797790 con 0x556f2e72c1b0 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33af7fe640 1 -- 172.21.15.115:0/251319601 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33c800fe70 con 0x556f2e72c1b0 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33af7fe640 1 -- 172.21.15.115:0/251319601 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f33c8018070 con 0x556f2e72c1b0 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33dd800740 10 client.28516 fetch_fsmap finished waiting for FSMap version 77 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33dd800740 10 client.28516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.025+0000 7f33dd800740 20 client.28516 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.026+0000 7f33dd800740 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f2e6ead80 0x556f2eaf0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.026+0000 7f33dd800740 4 client.28516 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.026+0000 7f33dd800740 1 -- 172.21.15.115:0/251319601 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f2ea423d0 con 0x556f2e6ead80 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.026+0000 7f33cffff640 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f2e6ead80 0x556f2eaf0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.026+0000 7f33af7fe640 10 client.28516 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.026+0000 7f33cffff640 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f2e6ead80 0x556f2eaf0240 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e017c580f0fabed server_cookie=f4bad7e3da7fe113 in_seq=0 out_seq=0 2024-05-09T05:58:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33af7fe640 1 -- 172.21.15.115:0/251319601 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f2ea423d0 con 0x556f2e6ead80 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33af7fe640 10 client.28516 handle_command_reply: tid=0 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 client.28516 shutdown 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 -- 172.21.15.115:0/251319601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e72c1b0 msgr2=0x556f2eaddff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e72c1b0 0x556f2eaddff0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f33c8010b90 tx=0x7f33c800fb40 comp rx=0 tx=0).stop 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 -- 172.21.15.115:0/251319601 shutdown_connections 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f2e6ead80 0x556f2eaf0240 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33af7fe640 0 client.28516 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e72c1b0 0x556f2eaddff0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e70b4d0 0x556f2eae0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 --2- 172.21.15.115:0/251319601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 0x556f2eadfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 -- 172.21.15.115:0/251319601 >> 172.21.15.115:0/251319601 conn(0x556f2e720300 msgr2=0x556f2ead57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.027+0000 7f33dd800740 1 -- 172.21.15.115:0/251319601 shutdown_connections 2024-05-09T05:58:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.028+0000 7f33dd800740 1 -- 172.21.15.115:0/251319601 wait complete. 2024-05-09T05:58:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.028+0000 7f33dd800740 20 client.28516 trim_cache size 0 max 16384 2024-05-09T05:58:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.047+0000 7f33dd800740 1 Processor -- start 2024-05-09T05:58:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.047+0000 7f33dd800740 1 -- start start 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.047+0000 7f33dd800740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e6ead80 0x556f2eae01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.047+0000 7f33dd800740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 0x556f2eaf5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.047+0000 7f33dd800740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2eaf5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.047+0000 7f33dd800740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01aef10 con 0x556f2e6ead80 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.047+0000 7f33dd800740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01b0a80 con 0x556f2e7032f0 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.047+0000 7f33dd800740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01af5e0 con 0x556f2e70b4d0 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e6ead80 0x556f2eae01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e6ead80 0x556f2eae01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42134/0 (socket says 172.21.15.115:42134) 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cffff640 1 -- 172.21.15.115:0/1161337860 learned_addr learned my addr 172.21.15.115:0/1161337860 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33d4e0d640 1 --2- 172.21.15.115:0/1161337860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2eaf5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/1161337860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 0x556f2eaf5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cffff640 1 -- 172.21.15.115:0/1161337860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 msgr2=0x556f2eaf5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cffff640 1 --2- 172.21.15.115:0/1161337860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2eaf5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cffff640 1 -- 172.21.15.115:0/1161337860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 msgr2=0x556f2eaf5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cffff640 1 --2- 172.21.15.115:0/1161337860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 0x556f2eaf5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cffff640 1 -- 172.21.15.115:0/1161337860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2e80e5f0 con 0x556f2e6ead80 2024-05-09T05:58:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33cffff640 1 --2- 172.21.15.115:0/1161337860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e6ead80 0x556f2eae01f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f33d0075640 tx=0x7f33c001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b0d3126d2346f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33b5ffb640 1 -- 172.21.15.115:0/1161337860 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0005020 con 0x556f2e6ead80 2024-05-09T05:58:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33b5ffb640 1 -- 172.21.15.115:0/1161337860 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33c002b420 con 0x556f2e6ead80 2024-05-09T05:58:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.048+0000 7f33b5ffb640 1 -- 172.21.15.115:0/1161337860 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c00047a0 con 0x556f2e6ead80 2024-05-09T05:58:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 -- 172.21.15.115:0/1161337860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e6ead80 msgr2=0x556f2eae01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 --2- 172.21.15.115:0/1161337860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e6ead80 0x556f2eae01f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f33d0075640 tx=0x7f33c001a000 comp rx=0 tx=0).stop 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 -- 172.21.15.115:0/1161337860 shutdown_connections 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 --2- 172.21.15.115:0/1161337860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2eaf5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 --2- 172.21.15.115:0/1161337860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e7032f0 0x556f2eaf5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 --2- 172.21.15.115:0/1161337860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e6ead80 0x556f2eae01f0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 -- 172.21.15.115:0/1161337860 >> 172.21.15.115:0/1161337860 conn(0x556f2e720300 msgr2=0x556f2eadfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 -- 172.21.15.115:0/1161337860 shutdown_connections 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 -- 172.21.15.115:0/1161337860 wait complete. 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.049+0000 7f33dd800740 1 -- start start 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33dd800740 1 Processor -- start 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33dd800740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e6ead80 0x556f2eb505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33dd800740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2eb50b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33dd800740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2eae5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33dd800740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01b03b0 con 0x556f2e7032f0 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33dd800740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01afcb0 con 0x556f2e6ead80 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33dd800740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01acef0 con 0x556f2e70b4d0 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33dd800740 10 client.0 _ll_register_callbacks cb 0x7f33c800f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33cf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2eb50b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33cf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2eb50b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42144/0 (socket says 172.21.15.115:42144) 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33cf7fe640 1 -- 172.21.15.115:0/3421699805 learned_addr learned my addr 172.21.15.115:0/3421699805 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33cffff640 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e6ead80 0x556f2eb505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33d4e0d640 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2eae5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33cf7fe640 1 -- 172.21.15.115:0/3421699805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 msgr2=0x556f2eae5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2eae5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33cf7fe640 1 -- 172.21.15.115:0/3421699805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e6ead80 msgr2=0x556f2eb505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e6ead80 0x556f2eb505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.050+0000 7f33cf7fe640 1 -- 172.21.15.115:0/3421699805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2e9f8ed0 con 0x556f2e7032f0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33cf7fe640 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2eb50b20 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f33c40c9800 tx=0x7f33c4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe7a6d53b57f617 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33af7fe640 1 -- 172.21.15.115:0/3421699805 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4048040 con 0x556f2e7032f0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33af7fe640 1 -- 172.21.15.115:0/3421699805 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f33c40d2420 con 0x556f2e7032f0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33dd800740 1 -- 172.21.15.115:0/3421699805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33c002b420 con 0x556f2e7032f0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33af7fe640 1 -- 172.21.15.115:0/3421699805 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f33c40c5290 con 0x556f2e7032f0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33dd800740 10 client.19221 fetch_fsmap learned FSMap version 77 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33dd800740 1 -- 172.21.15.115:0/3421699805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x556f2eb57c30 con 0x556f2e7032f0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33af7fe640 1 -- 172.21.15.115:0/3421699805 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33c002b420 con 0x556f2e7032f0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33af7fe640 1 -- 172.21.15.115:0/3421699805 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f33c404b060 con 0x556f2e7032f0 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33dd800740 10 client.19221 fetch_fsmap finished waiting for FSMap version 77 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33dd800740 10 client.19221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:58:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33dd800740 20 client.19221 populate_metadata read hostname 'smithi115' 2024-05-09T05:58:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33dd800740 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f2e72c1b0 0x556f2eb5a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33dd800740 4 client.19221 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:58:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.051+0000 7f33dd800740 1 -- 172.21.15.115:0/3421699805 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f2ea3c240 con 0x556f2e72c1b0 2024-05-09T05:58:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.052+0000 7f33cffff640 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f2e72c1b0 0x556f2eb5a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.052+0000 7f33af7fe640 10 client.19221 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.052+0000 7f33cffff640 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f2e72c1b0 0x556f2eb5a3b0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd315b5e3243f1b server_cookie=dc47b671df1a8565 in_seq=0 out_seq=0 2024-05-09T05:58:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.062+0000 7f33af7fe640 1 -- 172.21.15.115:0/3421699805 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556f2ea3c240 con 0x556f2e72c1b0 2024-05-09T05:58:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.062+0000 7f33af7fe640 10 client.19221 handle_command_reply: tid=0 2024-05-09T05:58:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.062+0000 7f33dd800740 1 client.19221 shutdown 2024-05-09T05:58:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.062+0000 7f33dd800740 1 -- 172.21.15.115:0/3421699805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 msgr2=0x556f2eb50b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2eb50b20 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f33c40c9800 tx=0x7f33c4009260 comp rx=0 tx=0).stop 2024-05-09T05:58:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 1 -- 172.21.15.115:0/3421699805 shutdown_connections 2024-05-09T05:58:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556f2e72c1b0 0x556f2eb5a3b0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556f2e70b4d0 0x556f2eae5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556f2e7032f0 0x556f2eb50b20 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 1 --2- 172.21.15.115:0/3421699805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556f2e6ead80 0x556f2eb505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 1 -- 172.21.15.115:0/3421699805 >> 172.21.15.115:0/3421699805 conn(0x556f2e720300 msgr2=0x556f2ead5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33af7fe640 0 client.19221 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:58:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 1 -- 172.21.15.115:0/3421699805 shutdown_connections 2024-05-09T05:58:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 1 -- 172.21.15.115:0/3421699805 wait complete. 2024-05-09T05:58:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.063+0000 7f33dd800740 20 client.19221 trim_cache size 0 max 16384 2024-05-09T05:58:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:58:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:58:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:58:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 -- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33b80b2950 msgr2=0x7f33b80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33b80b2950 0x7f33b80b4e60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009fd0 tx=0x7f33c0005f90 comp rx=0 tx=0).stop 2024-05-09T05:58:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 -- 172.21.15.115:0/2290341353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33d0109d20 msgr2=0x7f33d019ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33d0109d20 0x7f33d019ad40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009230 tx=0x7f33c4005e10 comp rx=0 tx=0).stop 2024-05-09T05:58:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 -- 172.21.15.115:0/2290341353 shutdown_connections 2024-05-09T05:58:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f33b80b2950 0x7f33b80b4e60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33d010e6d0 0x7f33d019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33d0109d20 0x7f33d019ad40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 --2- 172.21.15.115:0/2290341353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d01021d0 0x7f33d019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 -- 172.21.15.115:0/2290341353 >> 172.21.15.115:0/2290341353 conn(0x7f33d00779e0 msgr2=0x7f33d010a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 -- 172.21.15.115:0/2290341353 shutdown_connections 2024-05-09T05:58:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:58:57.064+0000 7f33d6870640 1 -- 172.21.15.115:0/2290341353 wait complete. 2024-05-09T05:58:57.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:57 smithi195 ceph-mon[33844]: pgmap v658: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-05-09T05:58:57.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:57 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:58:57.868 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:57 smithi115 ceph-mon[30349]: pgmap v658: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-05-09T05:58:57.868 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:57 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:58:57.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:57 smithi161 ceph-mon[33805]: pgmap v658: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-05-09T05:58:57.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:57 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:58:58.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.488+0000 7fb274543700 1 -- 172.21.15.115:0/4032689261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb26c108f20 msgr2=0x7fb26c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:58.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.488+0000 7fb274543700 1 --2- 172.21.15.115:0/4032689261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb26c108f20 0x7fb26c109330 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0099c0 tx=0x7fb25c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.488+0000 7fb274543700 1 -- 172.21.15.115:0/4032689261 shutdown_connections 2024-05-09T05:58:58.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.488+0000 7fb274543700 1 --2- 172.21.15.115:0/4032689261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26c109870 0x7fb26c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.488+0000 7fb274543700 1 --2- 172.21.15.115:0/4032689261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb26c1045f0 0x7fb26c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.488+0000 7fb274543700 1 --2- 172.21.15.115:0/4032689261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb26c108f20 0x7fb26c109330 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.488+0000 7fb274543700 1 -- 172.21.15.115:0/4032689261 >> 172.21.15.115:0/4032689261 conn(0x7fb26c100040 msgr2=0x7fb26c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:58.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.488+0000 7fb274543700 1 -- 172.21.15.115:0/4032689261 shutdown_connections 2024-05-09T05:58:58.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.488+0000 7fb274543700 1 -- 172.21.15.115:0/4032689261 wait complete. 2024-05-09T05:58:58.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb274543700 1 Processor -- start 2024-05-09T05:58:58.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb274543700 1 -- start start 2024-05-09T05:58:58.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb274543700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb26c1045f0 0x7fb26c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:58.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb274543700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26c108f20 0x7fb26c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:58.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb274543700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb26c109870 0x7fb26c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb274543700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c1143a0 con 0x7fb26c109870 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb274543700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c114220 con 0x7fb26c108f20 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb274543700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c114520 con 0x7fb26c1045f0 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb2722df700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb26c1045f0 0x7fb26c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb271ade700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26c108f20 0x7fb26c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb271ade700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26c108f20 0x7fb26c076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:51680/0 (socket says 172.21.15.115:51680) 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb272ae0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb26c109870 0x7fb26c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb271ade700 1 -- 172.21.15.115:0/77901293 learned_addr learned my addr 172.21.15.115:0/77901293 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.489+0000 7fb2722df700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb26c1045f0 0x7fb26c075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:45160/0 (socket says 172.21.15.115:45160) 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb2722df700 1 -- 172.21.15.115:0/77901293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26c108f20 msgr2=0x7fb26c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb2722df700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26c108f20 0x7fb26c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb2722df700 1 -- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb26c109870 msgr2=0x7fb26c076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb2722df700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb26c109870 0x7fb26c076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb2722df700 1 -- 172.21.15.115:0/77901293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c1a91d0 con 0x7fb26c1045f0 2024-05-09T05:58:58.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb272ae0700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb26c109870 0x7fb26c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:58:58.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb2722df700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb26c1045f0 0x7fb26c075ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9f0 tx=0x7fb26800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:58.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb2637fe700 1 -- 172.21.15.115:0/77901293 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26800c9c0 con 0x7fb26c1045f0 2024-05-09T05:58:58.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb2637fe700 1 -- 172.21.15.115:0/77901293 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb268010430 con 0x7fb26c1045f0 2024-05-09T05:58:58.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c1a94c0 con 0x7fb26c1045f0 2024-05-09T05:58:58.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb2637fe700 1 -- 172.21.15.115:0/77901293 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268018680 con 0x7fb26c1045f0 2024-05-09T05:58:58.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.490+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c1a9a00 con 0x7fb26c1045f0 2024-05-09T05:58:58.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.491+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c0512c0 con 0x7fb26c1045f0 2024-05-09T05:58:58.480 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.493+0000 7fb2637fe700 1 -- 172.21.15.115:0/77901293 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fb26800cb60 con 0x7fb26c1045f0 2024-05-09T05:58:58.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.494+0000 7fb2637fe700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb2580b29f0 0x7fb2580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:58:58.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.494+0000 7fb271ade700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb2580b29f0 0x7fb2580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:58:58.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.494+0000 7fb2637fe700 1 -- 172.21.15.115:0/77901293 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7fb2680e2ca0 con 0x7fb26c1045f0 2024-05-09T05:58:58.481 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.495+0000 7fb271ade700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb2580b29f0 0x7fb2580b4f00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009750 tx=0x7fb25c01a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:58:58.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.498+0000 7fb2637fe700 1 -- 172.21.15.115:0/77901293 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2680aa340 con 0x7fb26c1045f0 2024-05-09T05:58:58.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.806+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb26c02e5f0 con 0x7fb26c1045f0 2024-05-09T05:58:58.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.807+0000 7fb2637fe700 1 -- 172.21.15.115:0/77901293 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+11960 (secure 0 0 0) 0x7fb2680a9c40 con 0x7fb26c1045f0 2024-05-09T05:58:58.794 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:58:58.794 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":77,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:58:55.590837+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28438":{"gid":28438,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/3680433414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3680433414},{"type":"v1","addr":"172.21.15.195:6833","nonce":3680433414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24425,24440,24451,24457,24469]},"id":1}]} 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.810+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb2580b29f0 msgr2=0x7fb2580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.810+0000 7fb274543700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb2580b29f0 0x7fb2580b4f00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009750 tx=0x7fb25c01a040 comp rx=0 tx=0).stop 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.810+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb26c1045f0 msgr2=0x7fb26c075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.810+0000 7fb274543700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb26c1045f0 0x7fb26c075ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9f0 tx=0x7fb26800ed00 comp rx=0 tx=0).stop 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.811+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 shutdown_connections 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.811+0000 7fb274543700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fb2580b29f0 0x7fb2580b4f00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.811+0000 7fb274543700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb26c109870 0x7fb26c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.811+0000 7fb274543700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb26c1045f0 0x7fb26c075ac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.811+0000 7fb274543700 1 --2- 172.21.15.115:0/77901293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26c108f20 0x7fb26c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:58:58.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.811+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 >> 172.21.15.115:0/77901293 conn(0x7fb26c100040 msgr2=0x7fb26c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:58:58.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.811+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 shutdown_connections 2024-05-09T05:58:58.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:58:58.811+0000 7fb274543700 1 -- 172.21.15.115:0/77901293 wait complete. 2024-05-09T05:58:58.799 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 77 2024-05-09T05:58:59.496 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 90 --expiration 140.95 --await-for 45.0' 2024-05-09T05:58:59.496 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db / --timeout 90 --expiration 140.95 --await-for 45.0 2024-05-09T05:58:59.777 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:58:59.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:59 smithi195 ceph-mon[33844]: pgmap v659: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-05-09T05:58:59.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:58:59 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/77901293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:58:59.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:59 smithi115 ceph-mon[30349]: pgmap v659: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-05-09T05:58:59.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:58:59 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/77901293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:58:59.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:59 smithi161 ceph-mon[33805]: pgmap v659: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-05-09T05:58:59.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:58:59 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/77901293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:59:01.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:01 smithi115 ceph-mon[30349]: pgmap v660: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 171 KiB/s wr, 54 op/s 2024-05-09T05:59:01.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:01 smithi195 ceph-mon[33844]: pgmap v660: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 171 KiB/s wr, 54 op/s 2024-05-09T05:59:01.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:01 smithi161 ceph-mon[33805]: pgmap v660: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 171 KiB/s wr, 54 op/s 2024-05-09T05:59:02.508 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:02 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:59:02.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.581+0000 7f0fb9817700 1 -- 172.21.15.115:0/3282285735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb41027e0 msgr2=0x7f0fb4102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:02.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.581+0000 7f0fb9817700 1 --2- 172.21.15.115:0/3282285735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb41027e0 0x7f0fb4102bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80098f0 tx=0x7f0fa8009c00 comp rx=0 tx=0).stop 2024-05-09T05:59:02.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.581+0000 7f0fb9817700 1 -- 172.21.15.115:0/3282285735 shutdown_connections 2024-05-09T05:59:02.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.581+0000 7f0fb9817700 1 --2- 172.21.15.115:0/3282285735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb4103130 0x7f0fb4111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.581+0000 7f0fb9817700 1 --2- 172.21.15.115:0/3282285735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb410a3e0 0x7f0fb410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.581+0000 7f0fb9817700 1 --2- 172.21.15.115:0/3282285735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb41027e0 0x7f0fb4102bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.581+0000 7f0fb9817700 1 -- 172.21.15.115:0/3282285735 >> 172.21.15.115:0/3282285735 conn(0x7f0fb40fe6f0 msgr2=0x7f0fb4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:02.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.581+0000 7f0fb9817700 1 -- 172.21.15.115:0/3282285735 shutdown_connections 2024-05-09T05:59:02.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.581+0000 7f0fb9817700 1 -- 172.21.15.115:0/3282285735 wait complete. 2024-05-09T05:59:02.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.582+0000 7f0fb9817700 1 Processor -- start 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.582+0000 7f0fb9817700 1 -- start start 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb9817700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb41027e0 0x7f0fb419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb9817700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb4103130 0x7f0fb419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb9817700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb410a3e0 0x7f0fb4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb9817700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41121b0 con 0x7f0fb4103130 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb9817700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4112030 con 0x7f0fb41027e0 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb9817700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4112330 con 0x7f0fb410a3e0 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb41027e0 0x7f0fb419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb2ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb41027e0 0x7f0fb419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:42134/0 (socket says 172.21.15.115:42134) 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/2284966594 learned_addr learned my addr 172.21.15.115:0/2284966594 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb410a3e0 0x7f0fb4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb4103130 0x7f0fb419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb37fe700 1 -- 172.21.15.115:0/2284966594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb41027e0 msgr2=0x7f0fb419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb41027e0 0x7f0fb419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb37fe700 1 -- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb4103130 msgr2=0x7f0fb419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb4103130 0x7f0fb419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.583+0000 7f0fb37fe700 1 -- 172.21.15.115:0/2284966594 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb4199040 con 0x7f0fb410a3e0 2024-05-09T05:59:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.584+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb410a3e0 0x7f0fb4198990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00efc0 tx=0x7f0fac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:02.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.584+0000 7f0fb8815700 1 -- 172.21.15.115:0/2284966594 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0092e0 con 0x7f0fb410a3e0 2024-05-09T05:59:02.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.584+0000 7f0fb9817700 1 -- 172.21.15.115:0/2284966594 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb41992d0 con 0x7f0fb410a3e0 2024-05-09T05:59:02.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.584+0000 7f0fb9817700 1 -- 172.21.15.115:0/2284966594 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb41a5600 con 0x7f0fb410a3e0 2024-05-09T05:59:02.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.584+0000 7f0fb8815700 1 -- 172.21.15.115:0/2284966594 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fac004bb0 con 0x7f0fb410a3e0 2024-05-09T05:59:02.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.584+0000 7f0fb8815700 1 -- 172.21.15.115:0/2284966594 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac004020 con 0x7f0fb410a3e0 2024-05-09T05:59:02.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.588+0000 7f0fb8815700 1 -- 172.21.15.115:0/2284966594 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0fac01b030 con 0x7f0fb410a3e0 2024-05-09T05:59:02.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.589+0000 7f0fb8815700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f9c0b2a70 0x7f0f9c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.589+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f9c0b2a70 0x7f0f9c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.589+0000 7f0fb8815700 1 -- 172.21.15.115:0/2284966594 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f0fac0e23f0 con 0x7f0fb410a3e0 2024-05-09T05:59:02.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fc822db80 1 Processor -- start 2024-05-09T05:59:02.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fc822db80 1 -- start start 2024-05-09T05:59:02.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f9c0b2a70 0x7f0f9c0b4f80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4199ad0 tx=0x7f0fa4011040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:02.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x558943495000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fc822db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5589433b7190 con 0x558943317640 2024-05-09T05:59:02.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fb2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fb2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:35930/0 (socket says 172.21.15.115:35930) 2024-05-09T05:59:02.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/207512861 learned_addr learned my addr 172.21.15.115:0/207512861 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:02.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.590+0000 7f0fc822db80 1 -- 172.21.15.115:0/207512861 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558943355990 con 0x558943494bf0 2024-05-09T05:59:02.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fc822db80 1 -- 172.21.15.115:0/207512861 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5589433b3150 con 0x558943401300 2024-05-09T05:59:02.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x558943495000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fb37fe700 1 -- 172.21.15.115:0/207512861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 msgr2=0x55894349ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:02.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fb37fe700 1 -- 172.21.15.115:0/207512861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 msgr2=0x55894349b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:02.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fb37fe700 1 -- 172.21.15.115:0/207512861 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55894334d510 con 0x558943494bf0 2024-05-09T05:59:02.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:59:02.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x558943495000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0df800 tx=0x7f0fac0a8fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bb3d497a7c5aa6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:02.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.591+0000 7f0f99ffb700 1 -- 172.21.15.115:0/207512861 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0a9420 con 0x558943494bf0 2024-05-09T05:59:02.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.592+0000 7f0f99ffb700 1 -- 172.21.15.115:0/207512861 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fac0ad7d0 con 0x558943494bf0 2024-05-09T05:59:02.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.592+0000 7f0f99ffb700 1 -- 172.21.15.115:0/207512861 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0b1d60 con 0x558943494bf0 2024-05-09T05:59:02.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.592+0000 7f0fc822db80 1 -- 172.21.15.115:0/207512861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 msgr2=0x558943495000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:02.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.592+0000 7f0fc822db80 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x558943495000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0df800 tx=0x7f0fac0a8fb0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.593+0000 7f0fc822db80 1 -- 172.21.15.115:0/207512861 shutdown_connections 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.593+0000 7f0fc822db80 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x558943495000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.593+0000 7f0fc822db80 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.593+0000 7f0fc822db80 1 --2- 172.21.15.115:0/207512861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.593+0000 7f0fc822db80 1 -- 172.21.15.115:0/207512861 >> 172.21.15.115:0/207512861 conn(0x558943377840 msgr2=0x55894348faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.593+0000 7f0fc822db80 1 -- 172.21.15.115:0/207512861 shutdown_connections 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.593+0000 7f0fc822db80 1 -- 172.21.15.115:0/207512861 wait complete. 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.593+0000 7f0fc822db80 1 -- start start 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fc822db80 1 Processor -- start 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x55894349f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fc822db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x558943232d40 con 0x558943317640 2024-05-09T05:59:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fc822db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5589433712f0 con 0x558943494bf0 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fc822db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5589434a17f0 con 0x558943401300 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fc822db80 10 client.0 _ll_register_callbacks cb 0x558943228860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fb2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fb2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34292/0 (socket says 172.21.15.115:34292) 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.594+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/1373240467 learned_addr learned my addr 172.21.15.115:0/1373240467 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x55894349f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/1373240467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 msgr2=0x55894349ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/1373240467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 msgr2=0x55894349f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x55894349f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/1373240467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558943232630 con 0x558943317640 2024-05-09T05:59:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349e820 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009780 tx=0x7f0fa4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a19f294d2a4c7af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0f92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0f92ffd700 1 -- 172.21.15.115:0/1373240467 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa402c070 con 0x558943317640 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fc822db80 1 -- 172.21.15.115:0/1373240467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fac0ad7d0 con 0x558943317640 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0f92ffd700 1 -- 172.21.15.115:0/1373240467 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f0fa401c870 con 0x558943317640 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0f92ffd700 1 -- 172.21.15.115:0/1373240467 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f0fa4036050 con 0x558943317640 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fc822db80 10 client.19239 fetch_fsmap learned FSMap version 77 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.595+0000 7f0fc822db80 1 -- 172.21.15.115:0/1373240467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x5589434ac2b0 con 0x558943317640 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.596+0000 7f0f92ffd700 1 -- 172.21.15.115:0/1373240467 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fa4028440 con 0x558943317640 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.596+0000 7f0f92ffd700 1 -- 172.21.15.115:0/1373240467 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f0fa4039020 con 0x558943317640 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.596+0000 7f0fc822db80 10 client.19239 fetch_fsmap finished waiting for FSMap version 77 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.596+0000 7f0fc822db80 10 client.19239 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.596+0000 7f0fc822db80 20 client.19239 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.596+0000 7f0fc822db80 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5589434ae940 0x5589434b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.596+0000 7f0fc822db80 4 client.19239 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.596+0000 7f0fc822db80 1 -- 172.21.15.115:0/1373240467 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0fac0ad7d0 con 0x5589434ae940 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.596+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5589434ae940 0x5589434b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.597+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5589434ae940 0x5589434b0d60 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4157aada3fc63aa3 server_cookie=9dfc619b50e9370c in_seq=0 out_seq=0 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.597+0000 7f0f92ffd700 10 client.19239 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0f92ffd700 1 -- 172.21.15.115:0/1373240467 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f0fac0ad7d0 con 0x5589434ae940 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0f92ffd700 10 client.19239 handle_command_reply: tid=0 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 client.19239 shutdown 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 -- 172.21.15.115:0/1373240467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 msgr2=0x55894349e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349e820 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009780 tx=0x7f0fa4004750 comp rx=0 tx=0).stop 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 -- 172.21.15.115:0/1373240467 shutdown_connections 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x55894349f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5589434ae940 0x5589434b0d60 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943317640 0x55894349e820 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 --2- 172.21.15.115:0/1373240467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943401300 0x55894349ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 -- 172.21.15.115:0/1373240467 >> 172.21.15.115:0/1373240467 conn(0x558943377840 msgr2=0x55894348f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:02.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0f92ffd700 0 client.19239 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:02.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 -- 172.21.15.115:0/1373240467 shutdown_connections 2024-05-09T05:59:02.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.598+0000 7f0fc822db80 1 -- 172.21.15.115:0/1373240467 wait complete. 2024-05-09T05:59:02.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.599+0000 7f0fc822db80 20 client.19239 trim_cache size 0 max 16384 2024-05-09T05:59:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fc822db80 1 Processor -- start 2024-05-09T05:59:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fc822db80 1 -- start start 2024-05-09T05:59:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434b6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x55894349e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fc822db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41ae170 con 0x558943401300 2024-05-09T05:59:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fc822db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41afce0 con 0x558943494bf0 2024-05-09T05:59:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fc822db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41ae840 con 0x558943317640 2024-05-09T05:59:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fb27fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fb27fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34306/0 (socket says 172.21.15.115:34306) 2024-05-09T05:59:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fb27fc700 1 -- 172.21.15.115:0/2163766406 learned_addr learned my addr 172.21.15.115:0/2163766406 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.621+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/2163766406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/2163766406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 msgr2=0x55894349e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/2163766406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x55894349e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/2163766406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 msgr2=0x5589434b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/2163766406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/2163766406 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558943491670 con 0x558943317640 2024-05-09T05:59:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/2163766406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/2163766406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434b6540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4005950 tx=0x7f0fa4009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2c51ce12ae18cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0f997fa700 1 -- 172.21.15.115:0/2163766406 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa402c040 con 0x558943317640 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0f997fa700 1 -- 172.21.15.115:0/2163766406 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fa403e070 con 0x558943317640 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.622+0000 7f0f997fa700 1 -- 172.21.15.115:0/2163766406 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4022040 con 0x558943317640 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.623+0000 7f0fc822db80 1 -- 172.21.15.115:0/2163766406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 msgr2=0x5589434b6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.623+0000 7f0fc822db80 1 --2- 172.21.15.115:0/2163766406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434b6540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4005950 tx=0x7f0fa4009ea0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.624+0000 7f0fc822db80 1 -- 172.21.15.115:0/2163766406 shutdown_connections 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.624+0000 7f0fc822db80 1 --2- 172.21.15.115:0/2163766406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x55894349e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.624+0000 7f0fc822db80 1 --2- 172.21.15.115:0/2163766406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434b6540 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.624+0000 7f0fc822db80 1 --2- 172.21.15.115:0/2163766406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.624+0000 7f0fc822db80 1 -- 172.21.15.115:0/2163766406 >> 172.21.15.115:0/2163766406 conn(0x558943377840 msgr2=0x55894349d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.624+0000 7f0fc822db80 1 -- 172.21.15.115:0/2163766406 shutdown_connections 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.624+0000 7f0fc822db80 1 -- 172.21.15.115:0/2163766406 wait complete. 2024-05-09T05:59:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.624+0000 7f0fc822db80 1 -- start start 2024-05-09T05:59:02.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.624+0000 7f0fc822db80 1 Processor -- start 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fc822db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x5589434a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fc822db80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41af610 con 0x558943401300 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fc822db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41aef10 con 0x558943494bf0 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fc822db80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41ac110 con 0x558943317640 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fc822db80 10 client.0 _ll_register_callbacks cb 0x558943228860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434a38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:35942/0 (socket says 172.21.15.115:35942) 2024-05-09T05:59:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/3971144876 learned_addr learned my addr 172.21.15.115:0/3971144876 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb37fe700 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x5589434a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/3971144876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 msgr2=0x5589434a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x5589434a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/3971144876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 msgr2=0x5589434a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.625+0000 7f0fb2ffd700 1 -- 172.21.15.115:0/3971144876 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558943519100 con 0x558943317640 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.626+0000 7f0fb2ffd700 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434a38a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400f450 tx=0x7f0fa401aee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd1a7e4d384cfa11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.626+0000 7f0f92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.626+0000 7f0f92ffd700 1 -- 172.21.15.115:0/3971144876 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4046070 con 0x558943317640 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.626+0000 7f0f92ffd700 1 -- 172.21.15.115:0/3971144876 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9350+0+0 (secure 0 0 0) 0x7f0fa4005230 con 0x558943317640 2024-05-09T05:59:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.626+0000 7f0fc822db80 1 -- 172.21.15.115:0/3971144876 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fa403e070 con 0x558943317640 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.626+0000 7f0f92ffd700 1 -- 172.21.15.115:0/3971144876 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f0fa4052050 con 0x558943317640 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.626+0000 7f0fc822db80 10 client.28543 fetch_fsmap learned FSMap version 77 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.626+0000 7f0fc822db80 1 -- 172.21.15.115:0/3971144876 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55894351a1d0 con 0x558943317640 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.627+0000 7f0f92ffd700 1 -- 172.21.15.115:0/3971144876 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fa401cb00 con 0x558943317640 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.627+0000 7f0f92ffd700 1 -- 172.21.15.115:0/3971144876 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f0fa4039030 con 0x558943317640 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.627+0000 7f0fc822db80 10 client.28543 fetch_fsmap finished waiting for FSMap version 77 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.627+0000 7f0fc822db80 10 client.28543 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.627+0000 7f0fc822db80 20 client.28543 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.627+0000 7f0fc822db80 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55894351a8c0 0x55894351cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.627+0000 7f0fc822db80 4 client.28543 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.95, "await_for": 45.0}] 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.627+0000 7f0fc822db80 1 -- 172.21.15.115:0/3971144876 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.95, "await_for": 45.0}) -- 0x7f0fa403e070 con 0x55894351a8c0 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.627+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55894351a8c0 0x55894351cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.628+0000 7f0fb27fc700 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55894351a8c0 0x55894351cce0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152515051679f9c9 server_cookie=1c8e9128b57e4fcc in_seq=0 out_seq=0 2024-05-09T05:59:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.628+0000 7f0f92ffd700 10 client.28543 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:02.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0f92ffd700 1 -- 172.21.15.115:0/3971144876 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f0fa403e070 con 0x55894351a8c0 2024-05-09T05:59:02.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0f92ffd700 10 client.28543 handle_command_reply: tid=0 2024-05-09T05:59:02.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 client.28543 shutdown 2024-05-09T05:59:02.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 -- 172.21.15.115:0/3971144876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 msgr2=0x5589434a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434a38a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400f450 tx=0x7f0fa401aee0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 -- 172.21.15.115:0/3971144876 shutdown_connections 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558943494bf0 0x5589434a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55894351a8c0 0x55894351cce0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558943317640 0x5589434a38a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 --2- 172.21.15.115:0/3971144876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558943401300 0x5589434a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 -- 172.21.15.115:0/3971144876 >> 172.21.15.115:0/3971144876 conn(0x558943377840 msgr2=0x558943498230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0f92ffd700 0 client.28543 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 -- 172.21.15.115:0/3971144876 shutdown_connections 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 1 -- 172.21.15.115:0/3971144876 wait complete. 2024-05-09T05:59:02.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.755+0000 7f0fc822db80 20 client.28543 trim_cache size 0 max 16384 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 77, 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 58, 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "8ecd7ce4": { 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 58, 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:59:02.745 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 140.9, 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "QUIESCED", 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:02.746 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:59:02.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 -- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f9c0b2a70 msgr2=0x7f0f9c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:02.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f9c0b2a70 0x7f0f9c0b4f80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4199ad0 tx=0x7f0fa4011040 comp rx=0 tx=0).stop 2024-05-09T05:59:02.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 -- 172.21.15.115:0/2284966594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb410a3e0 msgr2=0x7f0fb4198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:02.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb410a3e0 0x7f0fb4198990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00efc0 tx=0x7f0fac00c3a0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 -- 172.21.15.115:0/2284966594 shutdown_connections 2024-05-09T05:59:02.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f0f9c0b2a70 0x7f0f9c0b4f80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb41027e0 0x7f0fb419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb410a3e0 0x7f0fb4198990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 --2- 172.21.15.115:0/2284966594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fb4103130 0x7f0fb419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:02.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 -- 172.21.15.115:0/2284966594 >> 172.21.15.115:0/2284966594 conn(0x7f0fb40fe6f0 msgr2=0x7f0fb4101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:02.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 -- 172.21.15.115:0/2284966594 shutdown_connections 2024-05-09T05:59:02.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:02.756+0000 7f0fb9817700 1 -- 172.21.15.115:0/2284966594 wait complete. 2024-05-09T05:59:02.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:02 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:59:02.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:02 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:59:03.560 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8ecd7ce4', quiesce took 7.2 seconds. Will release after: 13.75 2024-05-09T05:59:03.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:03 smithi195 ceph-mon[33844]: pgmap v661: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 177 KiB/s wr, 53 op/s 2024-05-09T05:59:03.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:03 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:03 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:03 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:03 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:03 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:03 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:03 smithi115 ceph-mon[30349]: pgmap v661: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 177 KiB/s wr, 53 op/s 2024-05-09T05:59:03.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:03 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:03 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:03 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:03 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:03 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:03 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:03 smithi161 ceph-mon[33805]: pgmap v661: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 177 KiB/s wr, 53 op/s 2024-05-09T05:59:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:03 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:03 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:03 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:03 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:03 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:03 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: pgmap v662: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 386 KiB/s wr, 114 op/s 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:05 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: pgmap v662: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 386 KiB/s wr, 114 op/s 2024-05-09T05:59:05.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:05 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: pgmap v662: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 386 KiB/s wr, 114 op/s 2024-05-09T05:59:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:05.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:05 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: Unable to set osd_memory_target on smithi195 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi115 to 176.1M 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: Unable to set osd_memory_target on smithi115 to 184750899: error parsing value: Value '184750899' is below minimum 939524096 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:59:06.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:59:06.861 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:06.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-09T05:59:06.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: Unable to set osd_memory_target on smithi195 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T05:59:06.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi115 to 176.1M 2024-05-09T05:59:06.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: Unable to set osd_memory_target on smithi115 to 184750899: error parsing value: Value '184750899' is below minimum 939524096 2024-05-09T05:59:06.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:59:06.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:59:06.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:06.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi195 to 176.1M 2024-05-09T05:59:06.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: Unable to set osd_memory_target on smithi195 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-09T05:59:06.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi115 to 176.1M 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: Unable to set osd_memory_target on smithi115 to 184750899: error parsing value: Value '184750899' is below minimum 939524096 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:59:06.905 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T05:59:07.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:07 smithi195 ceph-mon[33844]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-05-09T05:59:07.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:07 smithi195 ceph-mon[33844]: Unable to set osd_memory_target on smithi161 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T05:59:07.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:07 smithi195 ceph-mon[33844]: pgmap v663: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 409 KiB/s wr, 118 op/s 2024-05-09T05:59:07.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:07 smithi115 ceph-mon[30349]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-05-09T05:59:07.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:07 smithi115 ceph-mon[30349]: Unable to set osd_memory_target on smithi161 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T05:59:07.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:07 smithi115 ceph-mon[30349]: pgmap v663: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 409 KiB/s wr, 118 op/s 2024-05-09T05:59:07.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:07 smithi161 ceph-mon[33805]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-05-09T05:59:07.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:07 smithi161 ceph-mon[33805]: Unable to set osd_memory_target on smithi161 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-05-09T05:59:07.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:07 smithi161 ceph-mon[33805]: pgmap v663: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 409 KiB/s wr, 118 op/s 2024-05-09T05:59:09.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:09 smithi195 ceph-mon[33844]: pgmap v664: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 409 KiB/s wr, 118 op/s 2024-05-09T05:59:09.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:09 smithi115 ceph-mon[30349]: pgmap v664: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 409 KiB/s wr, 118 op/s 2024-05-09T05:59:09.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:09 smithi161 ceph-mon[33805]: pgmap v664: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 409 KiB/s wr, 118 op/s 2024-05-09T05:59:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.247+0000 7f883f292640 1 -- 172.21.15.115:0/3011310889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 msgr2=0x7f883811cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.247+0000 7f883f292640 1 --2- 172.21.15.115:0/3011310889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 0x7f883811cd10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8830009900 tx=0x7f883002a4b0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.247+0000 7f883f292640 1 -- 172.21.15.115:0/3011310889 shutdown_connections 2024-05-09T05:59:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.247+0000 7f883f292640 1 --2- 172.21.15.115:0/3011310889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883811d250 0x7f883811f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.249+0000 7f883f292640 1 --2- 172.21.15.115:0/3011310889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 0x7f883811cd10 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.249+0000 7f883f292640 1 --2- 172.21.15.115:0/3011310889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883810a200 0x7f883810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.249+0000 7f883f292640 1 -- 172.21.15.115:0/3011310889 >> 172.21.15.115:0/3011310889 conn(0x7f883806e900 msgr2=0x7f883806ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.250+0000 7f883f292640 1 -- 172.21.15.115:0/3011310889 shutdown_connections 2024-05-09T05:59:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.250+0000 7f883f292640 1 -- 172.21.15.115:0/3011310889 wait complete. 2024-05-09T05:59:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.251+0000 7f883f292640 1 Processor -- start 2024-05-09T05:59:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.254+0000 7f883f292640 1 -- start start 2024-05-09T05:59:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.255+0000 7f883f292640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883810a200 0x7f883811a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.255+0000 7f883f292640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 0x7f8838114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.255+0000 7f883f292640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883811d250 0x7f8838114ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.255+0000 7f883f292640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8838112840 con 0x7f883811a8f0 2024-05-09T05:59:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.256+0000 7f883f292640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88381126c0 con 0x7f883810a200 2024-05-09T05:59:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.256+0000 7f883f292640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88381129c0 con 0x7f883811d250 2024-05-09T05:59:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.256+0000 7f883c82d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 0x7f8838114440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.256+0000 7f883c82d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 0x7f8838114440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34322/0 (socket says 172.21.15.115:34322) 2024-05-09T05:59:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.256+0000 7f883c82d640 1 -- 172.21.15.115:0/3054949649 learned_addr learned my addr 172.21.15.115:0/3054949649 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.257+0000 7f883c82d640 1 -- 172.21.15.115:0/3054949649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883811d250 msgr2=0x7f8838114ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T05:59:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.257+0000 7f883c82d640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883811d250 0x7f8838114ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.257+0000 7f883c82d640 1 -- 172.21.15.115:0/3054949649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883810a200 msgr2=0x7f883811a330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T05:59:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.257+0000 7f883c82d640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883810a200 0x7f883811a330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.257+0000 7f883c82d640 1 -- 172.21.15.115:0/3054949649 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88381be160 con 0x7f883811a8f0 2024-05-09T05:59:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.258+0000 7f883c82d640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 0x7f8838114440 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f8830005ab0 tx=0x7f8830003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.258+0000 7f88367fc640 1 -- 172.21.15.115:0/3054949649 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883003f070 con 0x7f883811a8f0 2024-05-09T05:59:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.258+0000 7f88367fc640 1 -- 172.21.15.115:0/3054949649 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f883003bd00 con 0x7f883811a8f0 2024-05-09T05:59:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.258+0000 7f88367fc640 1 -- 172.21.15.115:0/3054949649 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830035030 con 0x7f883811a8f0 2024-05-09T05:59:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.259+0000 7f883f292640 1 -- 172.21.15.115:0/3054949649 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381be330 con 0x7f883811a8f0 2024-05-09T05:59:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.259+0000 7f883f292640 1 -- 172.21.15.115:0/3054949649 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381be740 con 0x7f883811a8f0 2024-05-09T05:59:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.266+0000 7f884621d740 1 Processor -- start 2024-05-09T05:59:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.268+0000 7f884621d740 1 -- start start 2024-05-09T05:59:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.268+0000 7f884621d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 0x55bef2d82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.268+0000 7f884621d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29b62f0 0x55bef2d82bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.268+0000 7f884621d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d83100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.268+0000 7f884621d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bef2ce77c0 con 0x55bef29be4d0 2024-05-09T05:59:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.268+0000 7f884621d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bef2c830e0 con 0x55bef29df1b0 2024-05-09T05:59:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.268+0000 7f884621d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bef2ce3780 con 0x55bef29b62f0 2024-05-09T05:59:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.269+0000 7f88367fc640 1 -- 172.21.15.115:0/3054949649 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f8830042070 con 0x7f883811a8f0 2024-05-09T05:59:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.269+0000 7f883d82f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 0x55bef2d82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.269+0000 7f883d02e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29b62f0 0x55bef2d82bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.269+0000 7f883d02e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29b62f0 0x55bef2d82bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:35960/0 (socket says 172.21.15.115:35960) 2024-05-09T05:59:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.269+0000 7f883d82f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 0x55bef2d82680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:42202/0 (socket says 172.21.15.115:42202) 2024-05-09T05:59:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.269+0000 7f883d82f640 1 -- 172.21.15.115:0/659386414 learned_addr learned my addr 172.21.15.115:0/659386414 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.270+0000 7f88367fc640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f88200b2b60 0x7f88200b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.271+0000 7f88367fc640 1 -- 172.21.15.115:0/3054949649 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9526+0+0 (secure 0 0 0) 0x7f8830102e70 con 0x7f883811a8f0 2024-05-09T05:59:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.271+0000 7f883c82d640 1 --2- 172.21.15.115:0/659386414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d83100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.272+0000 7f883c82d640 1 -- 172.21.15.115:0/659386414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29b62f0 msgr2=0x55bef2d82bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.272+0000 7f883c82d640 1 --2- 172.21.15.115:0/659386414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29b62f0 0x55bef2d82bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.272+0000 7f883c82d640 1 -- 172.21.15.115:0/659386414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 msgr2=0x55bef2d82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.272+0000 7f883c82d640 1 --2- 172.21.15.115:0/659386414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 0x55bef2d82680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.272+0000 7f883c82d640 1 -- 172.21.15.115:0/659386414 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef2cac7c0 con 0x55bef29be4d0 2024-05-09T05:59:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.272+0000 7f883c82d640 1 --2- 172.21.15.115:0/659386414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d83100 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f8830100170 tx=0x7f88300c9770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a8c0eb3f809b5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.273+0000 7f881f7fe640 1 -- 172.21.15.115:0/659386414 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88300e4070 con 0x55bef29be4d0 2024-05-09T05:59:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.273+0000 7f881f7fe640 1 -- 172.21.15.115:0/659386414 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88300c9d00 con 0x55bef29be4d0 2024-05-09T05:59:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.273+0000 7f881f7fe640 1 -- 172.21.15.115:0/659386414 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830049070 con 0x55bef29be4d0 2024-05-09T05:59:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.273+0000 7f883d82f640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f88200b2b60 0x7f88200b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.274+0000 7f884621d740 1 -- 172.21.15.115:0/659386414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 msgr2=0x55bef2d83100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.274+0000 7f884621d740 1 --2- 172.21.15.115:0/659386414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d83100 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f8830100170 tx=0x7f88300c9770 comp rx=0 tx=0).stop 2024-05-09T05:59:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.274+0000 7f883d82f640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f88200b2b60 0x7f88200b5070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55bef2d7f7f0 tx=0x7f8828004ab0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.276+0000 7f884621d740 1 -- 172.21.15.115:0/659386414 shutdown_connections 2024-05-09T05:59:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.276+0000 7f884621d740 1 --2- 172.21.15.115:0/659386414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d83100 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.276+0000 7f884621d740 1 --2- 172.21.15.115:0/659386414 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29b62f0 0x55bef2d82bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.276+0000 7f884621d740 1 --2- 172.21.15.115:0/659386414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 0x55bef2d82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.276+0000 7f884621d740 1 -- 172.21.15.115:0/659386414 >> 172.21.15.115:0/659386414 conn(0x55bef29d3300 msgr2=0x55bef2d7ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.277+0000 7f884621d740 1 -- 172.21.15.115:0/659386414 shutdown_connections 2024-05-09T05:59:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.281+0000 7f884621d740 1 -- 172.21.15.115:0/659386414 wait complete. 2024-05-09T05:59:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.281+0000 7f884621d740 1 -- start start 2024-05-09T05:59:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.282+0000 7f884621d740 1 Processor -- start 2024-05-09T05:59:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.282+0000 7f884621d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 0x55bef2d8e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.282+0000 7f884621d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 0x55bef2d8ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.282+0000 7f884621d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29df1b0 0x55bef2d8cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.282+0000 7f884621d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55bef2d8f570 con 0x55bef29df1b0 2024-05-09T05:59:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.282+0000 7f884621d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bef2c6e9c0 con 0x55bef29b62f0 2024-05-09T05:59:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.282+0000 7f884621d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bef2d8f6f0 con 0x55bef29be4d0 2024-05-09T05:59:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.282+0000 7f884621d740 10 client.0 _ll_register_callbacks cb 0x55bef2c9b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.283+0000 7f883d82f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29df1b0 0x55bef2d8cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.283+0000 7f883d82f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29df1b0 0x55bef2d8cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34350/0 (socket says 172.21.15.115:34350) 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.283+0000 7f883d82f640 1 -- 172.21.15.115:0/2380190199 learned_addr learned my addr 172.21.15.115:0/2380190199 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.284+0000 7f883d82f640 1 -- 172.21.15.115:0/2380190199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 msgr2=0x55bef2d8ec20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.284+0000 7f883d82f640 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 0x55bef2d8ec20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.284+0000 7f883d82f640 1 -- 172.21.15.115:0/2380190199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 msgr2=0x55bef2d8e6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.284+0000 7f883d02e640 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 0x55bef2d8e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.284+0000 7f883d82f640 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 0x55bef2d8e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.284+0000 7f883d82f640 1 -- 172.21.15.115:0/2380190199 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef2c951f0 con 0x55bef29df1b0 2024-05-09T05:59:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.284+0000 7f883d02e640 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 0x55bef2d8e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:59:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.285+0000 7f883d82f640 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29df1b0 0x55bef2d8cbd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8828011e40 tx=0x7f8828007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8555b18fe4a4164a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.286+0000 7f881cff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:59:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.289+0000 7f881cff9640 1 -- 172.21.15.115:0/2380190199 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828022020 con 0x55bef29df1b0 2024-05-09T05:59:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.289+0000 7f884621d740 1 -- 172.21.15.115:0/2380190199 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88300c9d00 con 0x55bef29df1b0 2024-05-09T05:59:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.289+0000 7f881cff9640 1 -- 172.21.15.115:0/2380190199 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9526+0+0 (secure 0 0 0) 0x7f882801e450 con 0x55bef29df1b0 2024-05-09T05:59:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.290+0000 7f881cff9640 1 -- 172.21.15.115:0/2380190199 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f882802d050 con 0x55bef29df1b0 2024-05-09T05:59:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.292+0000 7f884621d740 10 client.19257 fetch_fsmap learned FSMap version 77 2024-05-09T05:59:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.292+0000 7f884621d740 1 -- 172.21.15.115:0/2380190199 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55bef2a4a790 con 0x55bef29df1b0 2024-05-09T05:59:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.295+0000 7f881cff9640 1 -- 172.21.15.115:0/2380190199 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88300c9d00 con 0x55bef29df1b0 2024-05-09T05:59:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.295+0000 7f881cff9640 1 -- 172.21.15.115:0/2380190199 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f882800f040 con 0x55bef29df1b0 2024-05-09T05:59:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.298+0000 7f884621d740 10 client.19257 fetch_fsmap finished waiting for FSMap version 77 2024-05-09T05:59:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.298+0000 7f884621d740 10 client.19257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:59:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.298+0000 7f884621d740 20 client.19257 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.298+0000 7f884621d740 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bef299dd80 0x55bef2d9eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.298+0000 7f884621d740 4 client.19257 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:59:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.298+0000 7f884621d740 1 -- 172.21.15.115:0/2380190199 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bef2cf53d0 con 0x55bef299dd80 2024-05-09T05:59:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.303+0000 7f883d02e640 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bef299dd80 0x55bef2d9eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.311+0000 7f883d02e640 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bef299dd80 0x55bef2d9eee0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c19ee6e64b8337 server_cookie=a89f556b46c72c15 in_seq=0 out_seq=0 2024-05-09T05:59:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.311+0000 7f881cff9640 10 client.19257 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.313+0000 7f881cff9640 1 -- 172.21.15.115:0/2380190199 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bef2cf53d0 con 0x55bef299dd80 2024-05-09T05:59:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.313+0000 7f881cff9640 10 client.19257 handle_command_reply: tid=0 2024-05-09T05:59:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.314+0000 7f884621d740 1 client.19257 shutdown 2024-05-09T05:59:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.319+0000 7f884621d740 1 -- 172.21.15.115:0/2380190199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29df1b0 msgr2=0x55bef2d8cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.319+0000 7f884621d740 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29df1b0 0x55bef2d8cbd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8828011e40 tx=0x7f8828007410 comp rx=0 tx=0).stop 2024-05-09T05:59:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.323+0000 7f884621d740 1 -- 172.21.15.115:0/2380190199 shutdown_connections 2024-05-09T05:59:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.323+0000 7f884621d740 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29df1b0 0x55bef2d8cbd0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.323+0000 7f884621d740 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bef299dd80 0x55bef2d9eee0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.323+0000 7f884621d740 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 0x55bef2d8ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.323+0000 7f884621d740 1 --2- 172.21.15.115:0/2380190199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 0x55bef2d8e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.323+0000 7f884621d740 1 -- 172.21.15.115:0/2380190199 >> 172.21.15.115:0/2380190199 conn(0x55bef29d3300 msgr2=0x55bef2d843a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.323+0000 7f881cff9640 0 client.19257 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.327+0000 7f884621d740 1 -- 172.21.15.115:0/2380190199 shutdown_connections 2024-05-09T05:59:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.329+0000 7f884621d740 1 -- 172.21.15.115:0/2380190199 wait complete. 2024-05-09T05:59:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.330+0000 7f884621d740 20 client.19257 trim_cache size 0 max 16384 2024-05-09T05:59:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.360+0000 7f884621d740 1 Processor -- start 2024-05-09T05:59:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.360+0000 7f884621d740 1 -- start start 2024-05-09T05:59:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.360+0000 7f884621d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29b62f0 0x55bef2d8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.361+0000 7f884621d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 0x55bef2d82b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.361+0000 7f884621d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 0x55bef2d83060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.361+0000 7f883c82d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 0x55bef2d82b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.361+0000 7f883c82d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 0x55bef2d82b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:35968/0 (socket says 172.21.15.115:35968) 2024-05-09T05:59:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.361+0000 7f883c82d640 1 -- 172.21.15.115:0/1909535408 learned_addr learned my addr 172.21.15.115:0/1909535408 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.361+0000 7f884621d740 1 -- 172.21.15.115:0/1909535408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f88240100d0 con 0x55bef29b62f0 2024-05-09T05:59:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.361+0000 7f883d82f640 1 --2- 172.21.15.115:0/1909535408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 0x55bef2d83060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.361+0000 7f884621d740 1 -- 172.21.15.115:0/1909535408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8824011c40 con 0x55bef29df1b0 2024-05-09T05:59:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.361+0000 7f884621d740 1 -- 172.21.15.115:0/1909535408 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88240107a0 con 0x55bef29be4d0 2024-05-09T05:59:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.362+0000 7f883d02e640 1 --2- 172.21.15.115:0/1909535408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29b62f0 0x55bef2d8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.362+0000 7f883d02e640 1 -- 172.21.15.115:0/1909535408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 msgr2=0x55bef2d82b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.362+0000 7f883d02e640 1 --2- 172.21.15.115:0/1909535408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 0x55bef2d82b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.362+0000 7f883d02e640 1 -- 172.21.15.115:0/1909535408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 msgr2=0x55bef2d83060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.362+0000 7f883d02e640 1 --2- 172.21.15.115:0/1909535408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 0x55bef2d83060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.362+0000 7f883d02e640 1 -- 172.21.15.115:0/1909535408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef2ac15f0 con 0x55bef29b62f0 2024-05-09T05:59:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.363+0000 7f883d02e640 1 --2- 172.21.15.115:0/1909535408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29b62f0 0x55bef2d8d120 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x55bef2d9f650 tx=0x7f882c009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64e490ac204865f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.363+0000 7f881effd640 1 -- 172.21.15.115:0/1909535408 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c00f040 con 0x55bef29b62f0 2024-05-09T05:59:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.363+0000 7f881effd640 1 -- 172.21.15.115:0/1909535408 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f882c0180e0 con 0x55bef29b62f0 2024-05-09T05:59:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.364+0000 7f881effd640 1 -- 172.21.15.115:0/1909535408 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c021690 con 0x55bef29b62f0 2024-05-09T05:59:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.365+0000 7f884621d740 1 -- 172.21.15.115:0/1909535408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29b62f0 msgr2=0x55bef2d8d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.365+0000 7f884621d740 1 --2- 172.21.15.115:0/1909535408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29b62f0 0x55bef2d8d120 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x55bef2d9f650 tx=0x7f882c009c80 comp rx=0 tx=0).stop 2024-05-09T05:59:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.365+0000 7f884621d740 1 -- 172.21.15.115:0/1909535408 shutdown_connections 2024-05-09T05:59:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.365+0000 7f884621d740 1 --2- 172.21.15.115:0/1909535408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29df1b0 0x55bef2d83060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.365+0000 7f884621d740 1 --2- 172.21.15.115:0/1909535408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29be4d0 0x55bef2d82b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.365+0000 7f884621d740 1 --2- 172.21.15.115:0/1909535408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29b62f0 0x55bef2d8d120 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.365+0000 7f884621d740 1 -- 172.21.15.115:0/1909535408 >> 172.21.15.115:0/1909535408 conn(0x55bef29d3300 msgr2=0x55bef2d82410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.367+0000 7f884621d740 1 -- 172.21.15.115:0/1909535408 shutdown_connections 2024-05-09T05:59:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.367+0000 7f884621d740 1 -- 172.21.15.115:0/1909535408 wait complete. 2024-05-09T05:59:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.367+0000 7f884621d740 1 -- start start 2024-05-09T05:59:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.368+0000 7f884621d740 1 Processor -- start 2024-05-09T05:59:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.368+0000 7f884621d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 0x55bef2d94770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.368+0000 7f884621d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d94cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.369+0000 7f884621d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29df1b0 0x55bef2d8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.369+0000 7f884621d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8824011570 con 0x55bef29be4d0 2024-05-09T05:59:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.369+0000 7f884621d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8824010e70 con 0x55bef29b62f0 2024-05-09T05:59:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.369+0000 7f884621d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f882400e0d0 con 0x55bef29df1b0 2024-05-09T05:59:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.369+0000 7f884621d740 10 client.0 _ll_register_callbacks cb 0x7f8828007a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.370+0000 7f883c82d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d94cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.370+0000 7f883c82d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d94cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34376/0 (socket says 172.21.15.115:34376) 2024-05-09T05:59:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.370+0000 7f883c82d640 1 -- 172.21.15.115:0/4017049408 learned_addr learned my addr 172.21.15.115:0/4017049408 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.371+0000 7f883d02e640 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 0x55bef2d94770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.371+0000 7f883c82d640 1 -- 172.21.15.115:0/4017049408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29df1b0 msgr2=0x55bef2d8fd10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:59:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.371+0000 7f883c82d640 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29df1b0 0x55bef2d8fd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.371+0000 7f883c82d640 1 -- 172.21.15.115:0/4017049408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 msgr2=0x55bef2d94770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.371+0000 7f883c82d640 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 0x55bef2d94770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.371+0000 7f883c82d640 1 -- 172.21.15.115:0/4017049408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef2cabed0 con 0x55bef29be4d0 2024-05-09T05:59:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.371+0000 7f883c82d640 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d94cd0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f88300c5d80 tx=0x7f88300c61c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9246a69fefb26bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.372+0000 7f881cff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:59:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.375+0000 7f881cff9640 1 -- 172.21.15.115:0/4017049408 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830049040 con 0x55bef29be4d0 2024-05-09T05:59:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.375+0000 7f881cff9640 1 -- 172.21.15.115:0/4017049408 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9526+0+0 (secure 0 0 0) 0x7f88300e41e0 con 0x55bef29be4d0 2024-05-09T05:59:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.377+0000 7f884621d740 1 -- 172.21.15.115:0/4017049408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882c0180e0 con 0x55bef29be4d0 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.379+0000 7f881cff9640 1 -- 172.21.15.115:0/4017049408 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f88300d8050 con 0x55bef29be4d0 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.379+0000 7f884621d740 10 client.19263 fetch_fsmap learned FSMap version 77 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.379+0000 7f884621d740 1 -- 172.21.15.115:0/4017049408 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55bef2e06a30 con 0x55bef29be4d0 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.380+0000 7f881cff9640 1 -- 172.21.15.115:0/4017049408 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f882c0180e0 con 0x55bef29be4d0 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.380+0000 7f881cff9640 1 -- 172.21.15.115:0/4017049408 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 77) ==== 8378+0+0 (secure 0 0 0) 0x7f88300d26d0 con 0x55bef29be4d0 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.380+0000 7f884621d740 10 client.19263 fetch_fsmap finished waiting for FSMap version 77 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.380+0000 7f884621d740 10 client.19263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.380+0000 7f884621d740 20 client.19263 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.380+0000 7f884621d740 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bef29731c0 0x55bef2e08ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.380+0000 7f884621d740 4 client.19263 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:59:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.380+0000 7f884621d740 1 -- 172.21.15.115:0/4017049408 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bef2cef240 con 0x55bef29731c0 2024-05-09T05:59:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.382+0000 7f883d02e640 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bef29731c0 0x55bef2e08ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.383+0000 7f883d02e640 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bef29731c0 0x55bef2e08ed0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e728c914b0d7438 server_cookie=2e3876aa410bfdd9 in_seq=0 out_seq=0 2024-05-09T05:59:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.383+0000 7f881cff9640 10 client.19263 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.486+0000 7f881cff9640 1 -- 172.21.15.115:0/4017049408 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bef2cef240 con 0x55bef29731c0 2024-05-09T05:59:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.486+0000 7f881cff9640 10 client.19263 handle_command_reply: tid=0 2024-05-09T05:59:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 client.19263 shutdown 2024-05-09T05:59:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 -- 172.21.15.115:0/4017049408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 msgr2=0x55bef2d94cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d94cd0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f88300c5d80 tx=0x7f88300c61c0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 -- 172.21.15.115:0/4017049408 shutdown_connections 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55bef29731c0 0x55bef2e08ed0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bef29df1b0 0x55bef2d8fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bef29be4d0 0x55bef2d94cd0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 --2- 172.21.15.115:0/4017049408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bef29b62f0 0x55bef2d94770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 -- 172.21.15.115:0/4017049408 >> 172.21.15.115:0/4017049408 conn(0x55bef29d3300 msgr2=0x55bef2d82e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f881cff9640 0 client.19263 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 -- 172.21.15.115:0/4017049408 shutdown_connections 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 1 -- 172.21.15.115:0/4017049408 wait complete. 2024-05-09T05:59:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.487+0000 7f884621d740 20 client.19263 trim_cache size 0 max 16384 2024-05-09T05:59:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:59:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:59:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:59:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.489+0000 7f883f292640 1 -- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f88200b2b60 msgr2=0x7f88200b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.489+0000 7f883f292640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f88200b2b60 0x7f88200b5070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55bef2d7f7f0 tx=0x7f8828004ab0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.489+0000 7f883f292640 1 -- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 msgr2=0x7f8838114440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.489+0000 7f883f292640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 0x7f8838114440 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f8830005ab0 tx=0x7f8830003d80 comp rx=0 tx=0).stop 2024-05-09T05:59:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.490+0000 7f883f292640 1 -- 172.21.15.115:0/3054949649 shutdown_connections 2024-05-09T05:59:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.490+0000 7f883f292640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f88200b2b60 0x7f88200b5070 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.490+0000 7f883f292640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f883811d250 0x7f8838114ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.490+0000 7f883f292640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f883811a8f0 0x7f8838114440 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.490+0000 7f883f292640 1 --2- 172.21.15.115:0/3054949649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883810a200 0x7f883811a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.490+0000 7f883f292640 1 -- 172.21.15.115:0/3054949649 >> 172.21.15.115:0/3054949649 conn(0x7f883806e900 msgr2=0x7f883811ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.491+0000 7f883f292640 1 -- 172.21.15.115:0/3054949649 shutdown_connections 2024-05-09T05:59:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:10.491+0000 7f883f292640 1 -- 172.21.15.115:0/3054949649 wait complete. 2024-05-09T05:59:10.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:10 smithi195 ceph-mon[33844]: osdmap e122: 12 total, 12 up, 12 in 2024-05-09T05:59:10.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:10 smithi115 ceph-mon[30349]: osdmap e122: 12 total, 12 up, 12 in 2024-05-09T05:59:10.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:10 smithi161 ceph-mon[33805]: osdmap e122: 12 total, 12 up, 12 in 2024-05-09T05:59:11.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:11 smithi195 ceph-mon[33844]: pgmap v666: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 290 KiB/s wr, 90 op/s 2024-05-09T05:59:11.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:11 smithi115 ceph-mon[30349]: pgmap v666: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 290 KiB/s wr, 90 op/s 2024-05-09T05:59:11.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:11 smithi161 ceph-mon[33805]: pgmap v666: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 290 KiB/s wr, 90 op/s 2024-05-09T05:59:13.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:13 smithi195 ceph-mon[33844]: pgmap v667: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 278 KiB/s wr, 87 op/s 2024-05-09T05:59:13.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:13 smithi115 ceph-mon[30349]: pgmap v667: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 278 KiB/s wr, 87 op/s 2024-05-09T05:59:13.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:13 smithi161 ceph-mon[33805]: pgmap v667: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 278 KiB/s wr, 87 op/s 2024-05-09T05:59:15.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:15 smithi195 ceph-mon[33844]: pgmap v668: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 KiB/s wr, 19 op/s 2024-05-09T05:59:15.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:15 smithi115 ceph-mon[30349]: pgmap v668: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 KiB/s wr, 19 op/s 2024-05-09T05:59:15.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:15 smithi161 ceph-mon[33805]: pgmap v668: 129 pgs: 16 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 KiB/s wr, 19 op/s 2024-05-09T05:59:17.311 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:59:17.555 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:59:17.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:17 smithi115 ceph-mon[30349]: pgmap v669: 129 pgs: 13 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-05-09T05:59:17.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:17 smithi195 ceph-mon[33844]: pgmap v669: 129 pgs: 13 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-05-09T05:59:17.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:17 smithi161 ceph-mon[33805]: pgmap v669: 129 pgs: 13 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 83 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-05-09T05:59:19.715 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:19 smithi115 ceph-mon[30349]: pgmap v670: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-05-09T05:59:19.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 -- 172.21.15.115:0/2787838698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42601034c0 msgr2=0x7f42601038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 --2- 172.21.15.115:0/2787838698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42601034c0 0x7f42601038d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f425c0099c0 tx=0x7f425c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 -- 172.21.15.115:0/2787838698 shutdown_connections 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 --2- 172.21.15.115:0/2787838698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4260108740 0x7f42601100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 --2- 172.21.15.115:0/2787838698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42601034c0 0x7f42601038d0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 --2- 172.21.15.115:0/2787838698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4260107df0 0x7f4260108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 -- 172.21.15.115:0/2787838698 >> 172.21.15.115:0/2787838698 conn(0x7f42600fedf0 msgr2=0x7f4260101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 -- 172.21.15.115:0/2787838698 shutdown_connections 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 -- 172.21.15.115:0/2787838698 wait complete. 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 Processor -- start 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.850+0000 7f4267608700 1 -- start start 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4267608700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42601034c0 0x7f426019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:19.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4267608700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4260107df0 0x7f426019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4267608700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4260108740 0x7f4260199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4267608700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f42601132a0 con 0x7f4260107df0 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4267608700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4260113120 con 0x7f4260108740 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4267608700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4260113420 con 0x7f42601034c0 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4264ba3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4260107df0 0x7f426019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4264ba3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4260107df0 0x7f426019fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42452/0 (socket says 172.21.15.115:42452) 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4264ba3700 1 -- 172.21.15.115:0/2258288272 learned_addr learned my addr 172.21.15.115:0/2258288272 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f42653a4700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42601034c0 0x7f426019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4265ba5700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4260108740 0x7f4260199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4264ba3700 1 -- 172.21.15.115:0/2258288272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42601034c0 msgr2=0x7f426019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4264ba3700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42601034c0 0x7f426019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4264ba3700 1 -- 172.21.15.115:0/2258288272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4260108740 msgr2=0x7f4260199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4264ba3700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4260108740 0x7f4260199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4264ba3700 1 -- 172.21.15.115:0/2258288272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426019a170 con 0x7f4260107df0 2024-05-09T05:59:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f42653a4700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42601034c0 0x7f426019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:59:19.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f4264ba3700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4260107df0 0x7f426019fde0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f425000d980 tx=0x7f425000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:19.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.851+0000 7f42567fc700 1 -- 172.21.15.115:0/2258288272 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250017070 con 0x7f4260107df0 2024-05-09T05:59:19.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.852+0000 7f42567fc700 1 -- 172.21.15.115:0/2258288272 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4250009dc0 con 0x7f4260107df0 2024-05-09T05:59:19.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.852+0000 7f42567fc700 1 -- 172.21.15.115:0/2258288272 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425000f3c0 con 0x7f4260107df0 2024-05-09T05:59:19.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.852+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426019a460 con 0x7f4260107df0 2024-05-09T05:59:19.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.852+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42600759f0 con 0x7f4260107df0 2024-05-09T05:59:19.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.853+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4260104970 con 0x7f4260107df0 2024-05-09T05:59:19.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.854+0000 7f42567fc700 1 -- 172.21.15.115:0/2258288272 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4250009930 con 0x7f4260107df0 2024-05-09T05:59:19.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.855+0000 7f42567fc700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f424c0b29f0 0x7f424c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:19.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.855+0000 7f42653a4700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f424c0b29f0 0x7f424c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:19.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.855+0000 7f42567fc700 1 -- 172.21.15.115:0/2258288272 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9526+0+0 (secure 0 0 0) 0x7f42500e24f0 con 0x7f4260107df0 2024-05-09T05:59:19.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.855+0000 7f42653a4700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f424c0b29f0 0x7f424c0b4f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f425c000c00 tx=0x7f425c005fb0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:19.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:19.858+0000 7f42567fc700 1 -- 172.21.15.115:0/2258288272 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42500a9ae0 con 0x7f4260107df0 2024-05-09T05:59:19.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:19 smithi195 ceph-mon[33844]: pgmap v670: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-05-09T05:59:19.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:19 smithi161 ceph-mon[33805]: pgmap v670: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-05-09T05:59:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.107+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42600512c0 con 0x7f4260107df0 2024-05-09T05:59:20.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.107+0000 7f42567fc700 1 -- 172.21.15.115:0/2258288272 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+11959 (secure 0 0 0) 0x7f42500a93e0 con 0x7f4260107df0 2024-05-09T05:59:20.095 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:59:20.096 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":78,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:59:19.298648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28438":{"gid":28438,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/3680433414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3680433414},{"type":"v1","addr":"172.21.15.195:6833","nonce":3680433414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24469,24457,24451,24425,24440]},"id":1}]} 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f424c0b29f0 msgr2=0x7f424c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f424c0b29f0 0x7f424c0b4f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f425c000c00 tx=0x7f425c005fb0 comp rx=0 tx=0).stop 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4260107df0 msgr2=0x7f426019fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4260107df0 0x7f426019fde0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f425000d980 tx=0x7f425000dc90 comp rx=0 tx=0).stop 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 shutdown_connections 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f424c0b29f0 0x7f424c0b4f00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4260108740 0x7f4260199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42601034c0 0x7f426019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 --2- 172.21.15.115:0/2258288272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4260107df0 0x7f426019fde0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:20.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 >> 172.21.15.115:0/2258288272 conn(0x7f42600fedf0 msgr2=0x7f4260101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:20.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 shutdown_connections 2024-05-09T05:59:20.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:20.110+0000 7f4267608700 1 -- 172.21.15.115:0/2258288272 wait complete. 2024-05-09T05:59:20.099 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 78 2024-05-09T05:59:20.712 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:20 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:20.712 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:20 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2258288272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:59:20.740 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 8ecd7ce4 --release --await' 2024-05-09T05:59:20.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --set-id 8ecd7ce4 --release --await 2024-05-09T05:59:20.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:20 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:20.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:20 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2258288272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:59:20.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:20 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:21.270 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:20 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2258288272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:59:21.271 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:59:21.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:21 smithi195 ceph-mon[33844]: pgmap v671: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-09T05:59:21.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:21 smithi115 ceph-mon[30349]: pgmap v671: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-09T05:59:21.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:21 smithi161 ceph-mon[33805]: pgmap v671: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-05-09T05:59:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 -- 172.21.15.115:0/2746747177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 msgr2=0x7f44341051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 --2- 172.21.15.115:0/2746747177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 0x7f44341051b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4430009900 tx=0x7f443002a410 comp rx=0 tx=0).stop 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 -- 172.21.15.115:0/2746747177 shutdown_connections 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 --2- 172.21.15.115:0/2746747177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f443410e1f0 0x7f44341106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 --2- 172.21.15.115:0/2746747177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443410b800 0x7f443410dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 --2- 172.21.15.115:0/2746747177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 0x7f44341051b0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 -- 172.21.15.115:0/2746747177 >> 172.21.15.115:0/2746747177 conn(0x7f44340782e0 msgr2=0x7f4434078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 -- 172.21.15.115:0/2746747177 shutdown_connections 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 -- 172.21.15.115:0/2746747177 wait complete. 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.620+0000 7f443c2c8640 1 Processor -- start 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443c2c8640 1 -- start start 2024-05-09T05:59:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443c2c8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 0x7f443419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443c2c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443410b800 0x7f443419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443c2c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f443410e1f0 0x7f443419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443c2c8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4434113d00 con 0x7f4434104da0 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443c2c8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4434113b80 con 0x7f443410b800 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443c2c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4434113e80 con 0x7f443410e1f0 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443a064640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 0x7f443419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443a064640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 0x7f443419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35880/0 (socket says 172.21.15.115:35880) 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443a064640 1 -- 172.21.15.115:0/186634488 learned_addr learned my addr 172.21.15.115:0/186634488 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f4439863640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443410b800 0x7f443419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443a865640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f443410e1f0 0x7f443419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443a064640 1 -- 172.21.15.115:0/186634488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f443410e1f0 msgr2=0x7f443419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443a064640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f443410e1f0 0x7f443419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443a064640 1 -- 172.21.15.115:0/186634488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443410b800 msgr2=0x7f443419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443a064640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443410b800 0x7f443419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.621+0000 7f443a064640 1 -- 172.21.15.115:0/186634488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443419f5c0 con 0x7f4434104da0 2024-05-09T05:59:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.622+0000 7f443a064640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 0x7f443419a760 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4430009230 tx=0x7f44300319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.622+0000 7f442b7fe640 1 -- 172.21.15.115:0/186634488 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443003f030 con 0x7f4434104da0 2024-05-09T05:59:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.622+0000 7f443c2c8640 1 -- 172.21.15.115:0/186634488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443419f850 con 0x7f4434104da0 2024-05-09T05:59:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.622+0000 7f442b7fe640 1 -- 172.21.15.115:0/186634488 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44300043d0 con 0x7f4434104da0 2024-05-09T05:59:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.622+0000 7f442b7fe640 1 -- 172.21.15.115:0/186634488 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430035070 con 0x7f4434104da0 2024-05-09T05:59:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.622+0000 7f443c2c8640 1 -- 172.21.15.115:0/186634488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443419fd90 con 0x7f4434104da0 2024-05-09T05:59:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.624+0000 7f444317c740 1 Processor -- start 2024-05-09T05:59:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.624+0000 7f442b7fe640 1 -- 172.21.15.115:0/186634488 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f4430042030 con 0x7f4434104da0 2024-05-09T05:59:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.624+0000 7f444317c740 1 -- start start 2024-05-09T05:59:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f444317c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989378530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f444317c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f989378a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f444317c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f989378fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f444317c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f9892dd7c0 con 0x55f988fd51b0 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f444317c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f9892790e0 con 0x55f988fb44d0 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f444317c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f9892d9780 con 0x55f988fac2f0 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a865640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989378530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a865640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989378530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35890/0 (socket says 172.21.15.115:35890) 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a865640 1 -- 172.21.15.115:0/2502802556 learned_addr learned my addr 172.21.15.115:0/2502802556 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a064640 1 --2- 172.21.15.115:0/2502802556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f989378a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f4439863640 1 --2- 172.21.15.115:0/2502802556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f989378fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a865640 1 -- 172.21.15.115:0/2502802556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 msgr2=0x55f989378a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a865640 1 --2- 172.21.15.115:0/2502802556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f989378a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a865640 1 -- 172.21.15.115:0/2502802556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 msgr2=0x55f989378fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a865640 1 --2- 172.21.15.115:0/2502802556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f989378fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f442b7fe640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f44200b2950 0x7f44200b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a865640 1 -- 172.21.15.115:0/2502802556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9892a27c0 con 0x55f988fd51b0 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a064640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f44200b2950 0x7f44200b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.625+0000 7f443a865640 1 --2- 172.21.15.115:0/2502802556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989378530 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f442c009f40 tx=0x7f442c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9344abd6b5ff227 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.626+0000 7f442b7fe640 1 -- 172.21.15.115:0/186634488 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9526+0+0 (secure 0 0 0) 0x7f4430102e50 con 0x7f4434104da0 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.626+0000 7f4428bfe640 1 -- 172.21.15.115:0/2502802556 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c003b70 con 0x55f988fd51b0 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.626+0000 7f4428bfe640 1 -- 172.21.15.115:0/2502802556 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f442c003d10 con 0x55f988fd51b0 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.626+0000 7f4428bfe640 1 -- 172.21.15.115:0/2502802556 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c011900 con 0x55f988fd51b0 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 -- 172.21.15.115:0/2502802556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 msgr2=0x55f989378530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f443a064640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f44200b2950 0x7f44200b4e60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4430100260 tx=0x7f44301069a0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 --2- 172.21.15.115:0/2502802556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989378530 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f442c009f40 tx=0x7f442c010860 comp rx=0 tx=0).stop 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 -- 172.21.15.115:0/2502802556 shutdown_connections 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 --2- 172.21.15.115:0/2502802556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f989378fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 --2- 172.21.15.115:0/2502802556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f989378a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 --2- 172.21.15.115:0/2502802556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989378530 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 -- 172.21.15.115:0/2502802556 >> 172.21.15.115:0/2502802556 conn(0x55f988fc9300 msgr2=0x55f989374e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 -- 172.21.15.115:0/2502802556 shutdown_connections 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 -- 172.21.15.115:0/2502802556 wait complete. 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 -- start start 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 Processor -- start 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f989388b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f989389070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989386fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f989389bc0 con 0x55f988fd51b0 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f9892649c0 con 0x55f988fb44d0 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f989389d40 con 0x55f988fac2f0 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.627+0000 7f444317c740 10 client.0 _ll_register_callbacks cb 0x55f989291020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989386fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989386fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35896/0 (socket says 172.21.15.115:35896) 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 -- 172.21.15.115:0/2969653916 learned_addr learned my addr 172.21.15.115:0/2969653916 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f989388b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f4439863640 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f989389070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 -- 172.21.15.115:0/2969653916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 msgr2=0x55f989388b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f989388b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 -- 172.21.15.115:0/2969653916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 msgr2=0x55f989389070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f989389070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 -- 172.21.15.115:0/2969653916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f98928b1f0 con 0x55f988fd51b0 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f443a865640 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989386fd0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f442c010d60 tx=0x7f442c004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d068ef47f3ad78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f441dffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f441dffb640 1 -- 172.21.15.115:0/2969653916 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c011e60 con 0x55f988fd51b0 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f441dffb640 1 -- 172.21.15.115:0/2969653916 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9526+0+0 (secure 0 0 0) 0x7f442c004c40 con 0x55f988fd51b0 2024-05-09T05:59:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f444317c740 1 -- 172.21.15.115:0/2969653916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442c003d10 con 0x55f988fd51b0 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f441dffb640 1 -- 172.21.15.115:0/2969653916 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f442c031050 con 0x55f988fd51b0 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f444317c740 10 client.19281 fetch_fsmap learned FSMap version 78 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.628+0000 7f444317c740 1 -- 172.21.15.115:0/2969653916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55f989040790 con 0x55f988fd51b0 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.629+0000 7f441dffb640 1 -- 172.21.15.115:0/2969653916 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f442c003d10 con 0x55f988fd51b0 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.629+0000 7f441dffb640 1 -- 172.21.15.115:0/2969653916 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 78) ==== 8374+0+0 (secure 0 0 0) 0x7f442c01b5b0 con 0x55f988fd51b0 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.629+0000 7f444317c740 10 client.19281 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.629+0000 7f444317c740 10 client.19281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.629+0000 7f444317c740 20 client.19281 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.629+0000 7f444317c740 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f988f93d80 0x55f989399280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.629+0000 7f444317c740 4 client.19281 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.629+0000 7f444317c740 1 -- 172.21.15.115:0/2969653916 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9892eb3d0 con 0x55f988f93d80 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.629+0000 7f4439863640 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f988f93d80 0x55f989399280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.630+0000 7f441dffb640 10 client.19281 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.630+0000 7f4439863640 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f988f93d80 0x55f989399280 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6996982c959515c server_cookie=6c4733bb3c0f5c5d in_seq=0 out_seq=0 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.630+0000 7f441dffb640 1 -- 172.21.15.115:0/2969653916 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9892eb3d0 con 0x55f988f93d80 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.630+0000 7f441dffb640 10 client.19281 handle_command_reply: tid=0 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.630+0000 7f444317c740 1 client.19281 shutdown 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 -- 172.21.15.115:0/2969653916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 msgr2=0x55f989386fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989386fd0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f442c010d60 tx=0x7f442c004900 comp rx=0 tx=0).stop 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 -- 172.21.15.115:0/2969653916 shutdown_connections 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989386fd0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f989389070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f988f93d80 0x55f989399280 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 --2- 172.21.15.115:0/2969653916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f989388b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f441dffb640 0 client.19281 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 -- 172.21.15.115:0/2969653916 >> 172.21.15.115:0/2969653916 conn(0x55f988fc9300 msgr2=0x55f98937a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 -- 172.21.15.115:0/2969653916 shutdown_connections 2024-05-09T05:59:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 1 -- 172.21.15.115:0/2969653916 wait complete. 2024-05-09T05:59:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.631+0000 7f444317c740 20 client.19281 trim_cache size 0 max 16384 2024-05-09T05:59:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f444317c740 1 Processor -- start 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f444317c740 1 -- start start 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f444317c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f98937a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f444317c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f98937a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f444317c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989374710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f444317c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44341af0a0 con 0x55f988fd51b0 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f444317c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b0be0 con 0x55f988fb44d0 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f444317c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f44341af770 con 0x55f988fac2f0 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f443a865640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989374710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f443a865640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989374710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35902/0 (socket says 172.21.15.115:35902) 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f443a865640 1 -- 172.21.15.115:0/4117235086 learned_addr learned my addr 172.21.15.115:0/4117235086 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f443a865640 1 --2- 172.21.15.115:0/4117235086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f98937a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f4439863640 1 --2- 172.21.15.115:0/4117235086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f98937a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f443a865640 1 -- 172.21.15.115:0/4117235086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 msgr2=0x55f98937a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.646+0000 7f443a865640 1 --2- 172.21.15.115:0/4117235086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f98937a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.647+0000 7f443a865640 1 -- 172.21.15.115:0/4117235086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 msgr2=0x55f98937a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.647+0000 7f443a865640 1 --2- 172.21.15.115:0/4117235086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f98937a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.647+0000 7f443a865640 1 -- 172.21.15.115:0/4117235086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9890b75f0 con 0x55f988fd51b0 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.647+0000 7f443a865640 1 --2- 172.21.15.115:0/4117235086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989374710 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f442c006010 tx=0x7f442c0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a0aa038371975b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.647+0000 7f441ffff640 1 -- 172.21.15.115:0/4117235086 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c024040 con 0x55f988fd51b0 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.647+0000 7f441ffff640 1 -- 172.21.15.115:0/4117235086 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f442c012e50 con 0x55f988fd51b0 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.647+0000 7f441ffff640 1 -- 172.21.15.115:0/4117235086 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c003f70 con 0x55f988fd51b0 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 -- 172.21.15.115:0/4117235086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 msgr2=0x55f989374710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 --2- 172.21.15.115:0/4117235086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989374710 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f442c006010 tx=0x7f442c0021f0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 -- 172.21.15.115:0/4117235086 shutdown_connections 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 --2- 172.21.15.115:0/4117235086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fd51b0 0x55f989374710 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 --2- 172.21.15.115:0/4117235086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fb44d0 0x55f98937a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 --2- 172.21.15.115:0/4117235086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fac2f0 0x55f98937a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 -- 172.21.15.115:0/4117235086 >> 172.21.15.115:0/4117235086 conn(0x55f988fc9300 msgr2=0x55f9893858a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 -- 172.21.15.115:0/4117235086 shutdown_connections 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 -- 172.21.15.115:0/4117235086 wait complete. 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 -- start start 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 Processor -- start 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fac2f0 0x55f98939c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fb44d0 0x55f9893999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fd51b0 0x55f989399f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.648+0000 7f444317c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b0510 con 0x55f988fac2f0 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f444317c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44341afe40 con 0x55f988fd51b0 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f444317c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f44341ad0a0 con 0x55f988fb44d0 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f444317c740 10 client.0 _ll_register_callbacks cb 0x7f4430106870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f443a865640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fac2f0 0x55f98939c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f443a865640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fac2f0 0x55f98939c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35916/0 (socket says 172.21.15.115:35916) 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f443a865640 1 -- 172.21.15.115:0/3479009811 learned_addr learned my addr 172.21.15.115:0/3479009811 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f443a865640 1 -- 172.21.15.115:0/3479009811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fb44d0 msgr2=0x55f9893999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f443a865640 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fb44d0 0x55f9893999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f443a865640 1 -- 172.21.15.115:0/3479009811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fd51b0 msgr2=0x55f989399f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f443a865640 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fd51b0 0x55f989399f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f443a865640 1 -- 172.21.15.115:0/3479009811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9892a1ed0 con 0x55f988fac2f0 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f443a865640 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fac2f0 0x55f98939c680 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f442c0108a0 tx=0x7f442c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2834b62118703514 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f441dffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f441dffb640 1 -- 172.21.15.115:0/3479009811 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c011500 con 0x55f988fac2f0 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f441dffb640 1 -- 172.21.15.115:0/3479009811 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9526+0+0 (secure 0 0 0) 0x7f442c01b620 con 0x55f988fac2f0 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f444317c740 1 -- 172.21.15.115:0/3479009811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442c012e50 con 0x55f988fac2f0 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f441dffb640 1 -- 172.21.15.115:0/3479009811 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f442c048050 con 0x55f988fac2f0 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f444317c740 10 client.19287 fetch_fsmap learned FSMap version 78 2024-05-09T05:59:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.649+0000 7f444317c740 1 -- 172.21.15.115:0/3479009811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55f989400f20 con 0x55f988fac2f0 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.650+0000 7f441dffb640 1 -- 172.21.15.115:0/3479009811 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f442c012e50 con 0x55f988fac2f0 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.650+0000 7f441dffb640 1 -- 172.21.15.115:0/3479009811 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 78) ==== 8374+0+0 (secure 0 0 0) 0x7f442c03f430 con 0x55f988fac2f0 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.650+0000 7f444317c740 10 client.19287 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.650+0000 7f444317c740 10 client.19287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.650+0000 7f444317c740 20 client.19287 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.650+0000 7f444317c740 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f988f691c0 0x55f989403590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.650+0000 7f444317c740 4 client.19287 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.650+0000 7f444317c740 1 -- 172.21.15.115:0/3479009811 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9892e5240 con 0x55f988f691c0 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.650+0000 7f4439863640 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f988f691c0 0x55f989403590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.651+0000 7f441dffb640 10 client.19287 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.651+0000 7f4439863640 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f988f691c0 0x55f989403590 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f46b3c7686f3ea9a server_cookie=591a1eb29f43dabd in_seq=0 out_seq=0 2024-05-09T05:59:23.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:23 smithi115 ceph-mon[30349]: pgmap v672: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-05-09T05:59:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f441dffb640 1 -- 172.21.15.115:0/3479009811 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f9892e5240 con 0x55f988f691c0 2024-05-09T05:59:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f441dffb640 10 client.19287 handle_command_reply: tid=0 2024-05-09T05:59:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 client.19287 shutdown 2024-05-09T05:59:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 -- 172.21.15.115:0/3479009811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fac2f0 msgr2=0x55f98939c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fac2f0 0x55f98939c680 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f442c0108a0 tx=0x7f442c009480 comp rx=0 tx=0).stop 2024-05-09T05:59:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 -- 172.21.15.115:0/3479009811 shutdown_connections 2024-05-09T05:59:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f988fd51b0 0x55f989399f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55f988f691c0 0x55f989403590 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f988fb44d0 0x55f9893999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f441dffb640 0 client.19287 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 --2- 172.21.15.115:0/3479009811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f988fac2f0 0x55f98939c680 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 -- 172.21.15.115:0/3479009811 >> 172.21.15.115:0/3479009811 conn(0x55f988fc9300 msgr2=0x55f98937a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 -- 172.21.15.115:0/3479009811 shutdown_connections 2024-05-09T05:59:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.661+0000 7f444317c740 1 -- 172.21.15.115:0/3479009811 wait complete. 2024-05-09T05:59:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f444317c740 20 client.19287 trim_cache size 0 max 16384 2024-05-09T05:59:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:59:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:59:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:59:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 -- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f44200b2950 msgr2=0x7f44200b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f44200b2950 0x7f44200b4e60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4430100260 tx=0x7f44301069a0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 -- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 msgr2=0x7f443419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 0x7f443419a760 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4430009230 tx=0x7f44300319a0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 -- 172.21.15.115:0/186634488 shutdown_connections 2024-05-09T05:59:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f44200b2950 0x7f44200b4e60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f443410e1f0 0x7f443419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443410b800 0x7f443419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 --2- 172.21.15.115:0/186634488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4434104da0 0x7f443419a760 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 -- 172.21.15.115:0/186634488 >> 172.21.15.115:0/186634488 conn(0x7f44340782e0 msgr2=0x7f44341012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 -- 172.21.15.115:0/186634488 shutdown_connections 2024-05-09T05:59:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:23.662+0000 7f443c2c8640 1 -- 172.21.15.115:0/186634488 wait complete. 2024-05-09T05:59:23.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:23 smithi195 ceph-mon[33844]: pgmap v672: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-05-09T05:59:23.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:23 smithi161 ceph-mon[33805]: pgmap v672: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-05-09T05:59:25.359 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:24 smithi195 ceph-mon[33844]: osdmap e123: 12 total, 12 up, 12 in 2024-05-09T05:59:25.359 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:24 smithi195 ceph-mon[33844]: pgmap v674: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-05-09T05:59:25.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:24 smithi115 ceph-mon[30349]: osdmap e123: 12 total, 12 up, 12 in 2024-05-09T05:59:25.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:24 smithi115 ceph-mon[30349]: pgmap v674: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-05-09T05:59:25.404 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:24 smithi161 ceph-mon[33805]: osdmap e123: 12 total, 12 up, 12 in 2024-05-09T05:59:25.404 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:24 smithi161 ceph-mon[33805]: pgmap v674: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 -- 172.21.15.115:0/757616384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c109850 msgr2=0x7f6d8c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 --2- 172.21.15.115:0/757616384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c109850 0x7f6d8c1111b0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b0d0 tx=0x7f6d8400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 -- 172.21.15.115:0/757616384 shutdown_connections 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 --2- 172.21.15.115:0/757616384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c109850 0x7f6d8c1111b0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 --2- 172.21.15.115:0/757616384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d8c1045d0 0x7f6d8c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 --2- 172.21.15.115:0/757616384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d8c108f00 0x7f6d8c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 -- 172.21.15.115:0/757616384 >> 172.21.15.115:0/757616384 conn(0x7f6d8c0ffec0 msgr2=0x7f6d8c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 -- 172.21.15.115:0/757616384 shutdown_connections 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 -- 172.21.15.115:0/757616384 wait complete. 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 Processor -- start 2024-05-09T05:59:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.400+0000 7f6d928df700 1 -- start start 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d928df700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c1045d0 0x7f6d8c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d928df700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d8c108f00 0x7f6d8c1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d928df700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d8c109850 0x7f6d8c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d928df700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c114330 con 0x7f6d8c1045d0 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d928df700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1141b0 con 0x7f6d8c108f00 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d928df700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1144b0 con 0x7f6d8c109850 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c1045d0 0x7f6d8c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c1045d0 0x7f6d8c1a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35926/0 (socket says 172.21.15.115:35926) 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8bfff700 1 -- 172.21.15.115:0/235550674 learned_addr learned my addr 172.21.15.115:0/235550674 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d90e7c700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d8c109850 0x7f6d8c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d8c108f00 0x7f6d8c1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8bfff700 1 -- 172.21.15.115:0/235550674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d8c109850 msgr2=0x7f6d8c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d8c109850 0x7f6d8c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8bfff700 1 -- 172.21.15.115:0/235550674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d8c108f00 msgr2=0x7f6d8c1a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d8c108f00 0x7f6d8c1a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8bfff700 1 -- 172.21.15.115:0/235550674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c19b1d0 con 0x7f6d8c1045d0 2024-05-09T05:59:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d8c108f00 0x7f6d8c1a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:59:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.401+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c1045d0 0x7f6d8c1a08b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.402+0000 7f6d897fa700 1 -- 172.21.15.115:0/235550674 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c016070 con 0x7f6d8c1045d0 2024-05-09T05:59:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.402+0000 7f6d897fa700 1 -- 172.21.15.115:0/235550674 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d7c009dd0 con 0x7f6d8c1045d0 2024-05-09T05:59:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.402+0000 7f6d928df700 1 -- 172.21.15.115:0/235550674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c19b4c0 con 0x7f6d8c1045d0 2024-05-09T05:59:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.402+0000 7f6d897fa700 1 -- 172.21.15.115:0/235550674 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00f3c0 con 0x7f6d8c1045d0 2024-05-09T05:59:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.402+0000 7f6d928df700 1 -- 172.21.15.115:0/235550674 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c076b00 con 0x7f6d8c1045d0 2024-05-09T05:59:26.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.405+0000 7f6d897fa700 1 -- 172.21.15.115:0/235550674 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f6d7c019070 con 0x7f6d8c1045d0 2024-05-09T05:59:26.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6da12f5b80 1 Processor -- start 2024-05-09T05:59:26.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6da12f5b80 1 -- start start 2024-05-09T05:59:26.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6d897fa700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6d740b29f0 0x7f6d740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e364a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6d90e7c700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6d740b29f0 0x7f6d740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e364fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55759e2810f0 con 0x55759e1e15a0 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55759e21f8f0 con 0x55759e2cb260 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6d897fa700 1 -- 172.21.15.115:0/235550674 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f6d7c0e2610 con 0x7f6d8c1045d0 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.406+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55759e27d0b0 con 0x55759e35eb50 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e364fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e364fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35932/0 (socket says 172.21.15.115:35932) 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8b7fe700 1 -- 172.21.15.115:0/390994265 learned_addr learned my addr 172.21.15.115:0/390994265 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e364a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d90e7c700 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d90e7c700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6d740b29f0 0x7f6d740b4f00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009fd0 tx=0x7f6d84009d10 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8b7fe700 1 -- 172.21.15.115:0/390994265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 msgr2=0x55759e35ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8b7fe700 1 -- 172.21.15.115:0/390994265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 msgr2=0x55759e364a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e364a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8b7fe700 1 -- 172.21.15.115:0/390994265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55759e217470 con 0x55759e1e15a0 2024-05-09T05:59:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d90e7c700 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:59:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e364fd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6d80008bb0 tx=0x7f6d80008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a647df90d853ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d727fc700 1 -- 172.21.15.115:0/390994265 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80014070 con 0x55759e1e15a0 2024-05-09T05:59:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d727fc700 1 -- 172.21.15.115:0/390994265 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d8000ed10 con 0x55759e1e15a0 2024-05-09T05:59:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.407+0000 7f6d727fc700 1 -- 172.21.15.115:0/390994265 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d800056c0 con 0x55759e1e15a0 2024-05-09T05:59:26.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.408+0000 7f6da12f5b80 1 -- 172.21.15.115:0/390994265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 msgr2=0x55759e364fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:26.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.408+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e364fd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6d80008bb0 tx=0x7f6d80008ec0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.408+0000 7f6da12f5b80 1 -- 172.21.15.115:0/390994265 shutdown_connections 2024-05-09T05:59:26.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.408+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e364a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.408+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.408+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/390994265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e364fd0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.408+0000 7f6da12f5b80 1 -- 172.21.15.115:0/390994265 >> 172.21.15.115:0/390994265 conn(0x55759e2417a0 msgr2=0x55759e359a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:26.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.408+0000 7f6da12f5b80 1 -- 172.21.15.115:0/390994265 shutdown_connections 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.408+0000 7f6da12f5b80 1 -- 172.21.15.115:0/390994265 wait complete. 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6da12f5b80 1 -- start start 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6da12f5b80 1 Processor -- start 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e1e15a0 0x55759e368720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e2cb260 0x55759e368c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e35eb50 0x55759e3691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55759e0fcca0 con 0x55759e2cb260 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55759e23b250 con 0x55759e35eb50 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55759e374900 con 0x55759e1e15a0 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6da12f5b80 10 client.0 _ll_register_callbacks cb 0x55759e0f27c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6d8b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e2cb260 0x55759e368c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6d8b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e2cb260 0x55759e368c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35940/0 (socket says 172.21.15.115:35940) 2024-05-09T05:59:26.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6d8b7fe700 1 -- 172.21.15.115:0/3182737248 learned_addr learned my addr 172.21.15.115:0/3182737248 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6d90e7c700 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e35eb50 0x55759e3691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.409+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e1e15a0 0x55759e368720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d8b7fe700 1 -- 172.21.15.115:0/3182737248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e1e15a0 msgr2=0x55759e368720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e1e15a0 0x55759e368720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d8b7fe700 1 -- 172.21.15.115:0/3182737248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e35eb50 msgr2=0x55759e3691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e35eb50 0x55759e3691a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d8b7fe700 1 -- 172.21.15.115:0/3182737248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55759e0fc590 con 0x55759e2cb260 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e2cb260 0x55759e368c60 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6d80004980 tx=0x7f6d80004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d1297d40b4b407 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/3182737248 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80027060 con 0x55759e2cb260 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/3182737248 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f6d8000eb10 con 0x55759e2cb260 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6da12f5b80 1 -- 172.21.15.115:0/3182737248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d8000ed10 con 0x55759e2cb260 2024-05-09T05:59:26.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/3182737248 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f6d8002e050 con 0x55759e2cb260 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6da12f5b80 10 client.19299 fetch_fsmap learned FSMap version 78 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.410+0000 7f6da12f5b80 1 -- 172.21.15.115:0/3182737248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55759e376200 con 0x55759e2cb260 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.411+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/3182737248 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d80004fc0 con 0x55759e2cb260 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.411+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/3182737248 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 78) ==== 8374+0+0 (secure 0 0 0) 0x7f6d80022410 con 0x55759e2cb260 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.411+0000 7f6da12f5b80 10 client.19299 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.411+0000 7f6da12f5b80 10 client.19299 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.411+0000 7f6da12f5b80 20 client.19299 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.411+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55759e3786d0 0x55759e37aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.411+0000 7f6da12f5b80 4 client.19299 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.411+0000 7f6da12f5b80 1 -- 172.21.15.115:0/3182737248 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6d8000ed10 con 0x55759e3786d0 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.411+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55759e3786d0 0x55759e37aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.412+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55759e3786d0 0x55759e37aaf0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae94b8449171ca8 server_cookie=6a7c161c47306d90 in_seq=0 out_seq=0 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.412+0000 7f6d6b7fe700 10 client.19299 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.412+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/3182737248 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f6d8000ed10 con 0x55759e3786d0 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.412+0000 7f6d6b7fe700 10 client.19299 handle_command_reply: tid=0 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 client.19299 shutdown 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 -- 172.21.15.115:0/3182737248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e2cb260 msgr2=0x55759e368c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e2cb260 0x55759e368c60 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6d80004980 tx=0x7f6d80004a60 comp rx=0 tx=0).stop 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 -- 172.21.15.115:0/3182737248 shutdown_connections 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e35eb50 0x55759e3691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55759e3786d0 0x55759e37aaf0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e1e15a0 0x55759e368720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/3182737248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e2cb260 0x55759e368c60 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6d6b7fe700 0 client.19299 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 -- 172.21.15.115:0/3182737248 >> 172.21.15.115:0/3182737248 conn(0x55759e2417a0 msgr2=0x55759e359750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 -- 172.21.15.115:0/3182737248 shutdown_connections 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 1 -- 172.21.15.115:0/3182737248 wait complete. 2024-05-09T05:59:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.413+0000 7f6da12f5b80 20 client.19299 trim_cache size 0 max 16384 2024-05-09T05:59:26.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.433+0000 7f6da12f5b80 1 Processor -- start 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.433+0000 7f6da12f5b80 1 -- start start 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e368fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e367720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b2750 con 0x55759e1e15a0 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b4290 con 0x55759e2cb260 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b2e20 con 0x55759e35eb50 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e368fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e368fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35946/0 (socket says 172.21.15.115:35946) 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8bfff700 1 -- 172.21.15.115:0/1499153471 learned_addr learned my addr 172.21.15.115:0/1499153471 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e367720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d90e7c700 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8bfff700 1 -- 172.21.15.115:0/1499153471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 msgr2=0x55759e35db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:26.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8bfff700 1 -- 172.21.15.115:0/1499153471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 msgr2=0x55759e367720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:26.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e367720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8bfff700 1 -- 172.21.15.115:0/1499153471 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55759e366c90 con 0x55759e1e15a0 2024-05-09T05:59:26.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d90e7c700 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:59:26.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e368fe0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005950 tx=0x7f6d7c0a5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8daaad554a4739d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:26.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d71ffb700 1 -- 172.21.15.115:0/1499153471 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c020040 con 0x55759e1e15a0 2024-05-09T05:59:26.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d71ffb700 1 -- 172.21.15.115:0/1499153471 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d7c0b23c0 con 0x55759e1e15a0 2024-05-09T05:59:26.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.434+0000 7f6d71ffb700 1 -- 172.21.15.115:0/1499153471 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0e2260 con 0x55759e1e15a0 2024-05-09T05:59:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.435+0000 7f6da12f5b80 1 -- 172.21.15.115:0/1499153471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 msgr2=0x55759e368fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.435+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e368fe0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005950 tx=0x7f6d7c0a5550 comp rx=0 tx=0).stop 2024-05-09T05:59:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.435+0000 7f6da12f5b80 1 -- 172.21.15.115:0/1499153471 shutdown_connections 2024-05-09T05:59:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.435+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e35db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.435+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e368fe0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.435+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/1499153471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e367720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.435+0000 7f6da12f5b80 1 -- 172.21.15.115:0/1499153471 >> 172.21.15.115:0/1499153471 conn(0x55759e2417a0 msgr2=0x55759e367010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.435+0000 7f6da12f5b80 1 -- 172.21.15.115:0/1499153471 shutdown_connections 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 1 -- 172.21.15.115:0/1499153471 wait complete. 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 1 -- start start 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 1 Processor -- start 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e37d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e37ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e37beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b3bc0 con 0x55759e1e15a0 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b34f0 con 0x55759e2cb260 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b0730 con 0x55759e35eb50 2024-05-09T05:59:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e37d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6da12f5b80 10 client.0 _ll_register_callbacks cb 0x55759e35f680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.436+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e37d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35950/0 (socket says 172.21.15.115:35950) 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d8bfff700 1 -- 172.21.15.115:0/4168535212 learned_addr learned my addr 172.21.15.115:0/4168535212 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d90e7c700 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e37beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e37ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d8bfff700 1 -- 172.21.15.115:0/4168535212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 msgr2=0x55759e37beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e37beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d8bfff700 1 -- 172.21.15.115:0/4168535212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 msgr2=0x55759e37ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e37ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d8bfff700 1 -- 172.21.15.115:0/4168535212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55759e37c4c0 con 0x55759e1e15a0 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e37ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d8bfff700 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e37d990 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0a55e0 tx=0x7f6d7c0ada90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ce6db837d18d5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/4168535212 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c020040 con 0x55759e1e15a0 2024-05-09T05:59:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/4168535212 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f6d7c0e2260 con 0x55759e1e15a0 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6da12f5b80 1 -- 172.21.15.115:0/4168535212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d7c0b23c0 con 0x55759e1e15a0 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/4168535212 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f6d7c066170 con 0x55759e1e15a0 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6da12f5b80 10 client.19305 fetch_fsmap learned FSMap version 78 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6da12f5b80 1 -- 172.21.15.115:0/4168535212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55759e3e3560 con 0x55759e1e15a0 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.437+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/4168535212 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d7c0b23c0 con 0x55759e1e15a0 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/4168535212 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 78) ==== 8374+0+0 (secure 0 0 0) 0x7f6d7c0a87c0 con 0x55759e1e15a0 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6da12f5b80 10 client.19305 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6da12f5b80 10 client.19305 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6da12f5b80 20 client.19305 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55759e3e3c30 0x55759e3e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6da12f5b80 4 client.19305 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8ecd7ce4", "release": true, "await": true}] 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6da12f5b80 1 -- 172.21.15.115:0/4168535212 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8ecd7ce4", "release": true, "await": true}) -- 0x55759e28ead0 con 0x55759e3e3c30 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55759e3e3c30 0x55759e3e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6d6b7fe700 10 client.19305 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:26.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.438+0000 7f6d8b7fe700 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55759e3e3c30 0x55759e3e6050 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=625aac58d468c836 server_cookie=724e9ba1445e5eae in_seq=0 out_seq=0 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6d6b7fe700 1 -- 172.21.15.115:0/4168535212 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55759e28ead0 con 0x55759e3e3c30 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6d6b7fe700 10 client.19305 handle_command_reply: tid=0 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 client.19305 shutdown 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 -- 172.21.15.115:0/4168535212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 msgr2=0x55759e37d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e37d990 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0a55e0 tx=0x7f6d7c0ada90 comp rx=0 tx=0).stop 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 -- 172.21.15.115:0/4168535212 shutdown_connections 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55759e3e3c30 0x55759e3e6050 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55759e35eb50 0x55759e37beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55759e1e15a0 0x55759e37d990 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 --2- 172.21.15.115:0/4168535212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55759e2cb260 0x55759e37ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 -- 172.21.15.115:0/4168535212 >> 172.21.15.115:0/4168535212 conn(0x55759e2417a0 msgr2=0x55759e3623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6d6b7fe700 0 client.19305 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 -- 172.21.15.115:0/4168535212 shutdown_connections 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.454+0000 7f6da12f5b80 1 -- 172.21.15.115:0/4168535212 wait complete. 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6da12f5b80 20 client.19305 trim_cache size 0 max 16384 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 78, 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 60, 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": { 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stdout: "8ecd7ce4": { 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stdout: "version": 60, 2024-05-09T05:59:26.444 INFO:teuthology.orchestra.run.smithi115.stdout: "age_ref": 0.0, 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "timeout": 90.0, 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "expiration": 140.9, 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "members": { 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "file:/": { 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "excluded": false, 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "state": { 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RELEASED", 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:26.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:26.446 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T05:59:26.446 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T05:59:26.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 -- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6d740b29f0 msgr2=0x7f6d740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6d740b29f0 0x7f6d740b4f00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009fd0 tx=0x7f6d84009d10 comp rx=0 tx=0).stop 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 -- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c1045d0 msgr2=0x7f6d8c1a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c1045d0 0x7f6d8c1a08b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00dc60 comp rx=0 tx=0).stop 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 -- 172.21.15.115:0/235550674 shutdown_connections 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f6d740b29f0 0x7f6d740b4f00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d8c109850 0x7f6d8c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d8c1045d0 0x7f6d8c1a08b0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 --2- 172.21.15.115:0/235550674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d8c108f00 0x7f6d8c1a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 -- 172.21.15.115:0/235550674 >> 172.21.15.115:0/235550674 conn(0x7f6d8c0ffec0 msgr2=0x7f6d8c1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.455+0000 7f6d928df700 1 -- 172.21.15.115:0/235550674 shutdown_connections 2024-05-09T05:59:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:26.456+0000 7f6d928df700 1 -- 172.21.15.115:0/235550674 wait complete. 2024-05-09T05:59:27.125 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8ecd7ce4', total seconds elapsed: 30.8 2024-05-09T05:59:27.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:27 smithi195 ceph-mon[33844]: pgmap v675: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-05-09T05:59:27.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:27 smithi115 ceph-mon[30349]: pgmap v675: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-05-09T05:59:27.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:27 smithi161 ceph-mon[33805]: pgmap v675: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-05-09T05:59:29.703 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:29 smithi195 ceph-mon[33844]: pgmap v676: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-05-09T05:59:29.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:29 smithi115 ceph-mon[30349]: pgmap v676: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-05-09T05:59:29.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:29 smithi161 ceph-mon[33805]: pgmap v676: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-05-09T05:59:31.692 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:31 smithi115 ceph-mon[30349]: pgmap v677: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 143 KiB/s wr, 46 op/s 2024-05-09T05:59:31.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:31 smithi195 ceph-mon[33844]: pgmap v677: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 143 KiB/s wr, 46 op/s 2024-05-09T05:59:31.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:31 smithi161 ceph-mon[33805]: pgmap v677: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 143 KiB/s wr, 46 op/s 2024-05-09T05:59:33.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:33 smithi115 ceph-mon[30349]: pgmap v678: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 160 KiB/s wr, 50 op/s 2024-05-09T05:59:33.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:33 smithi195 ceph-mon[33844]: pgmap v678: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 160 KiB/s wr, 50 op/s 2024-05-09T05:59:33.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:33 smithi161 ceph-mon[33805]: pgmap v678: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 160 KiB/s wr, 50 op/s 2024-05-09T05:59:35.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:35 smithi195 ceph-mon[33844]: pgmap v679: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 411 KiB/s wr, 113 op/s 2024-05-09T05:59:35.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:35 smithi115 ceph-mon[30349]: pgmap v679: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 411 KiB/s wr, 113 op/s 2024-05-09T05:59:35.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:35 smithi161 ceph-mon[33805]: pgmap v679: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 411 KiB/s wr, 113 op/s 2024-05-09T05:59:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.795+0000 7f3179184640 1 -- 172.21.15.115:0/1149157110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3174104420 msgr2=0x7f31741107f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.795+0000 7f3179184640 1 --2- 172.21.15.115:0/1149157110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3174104420 0x7f31741107f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f315c009860 tx=0x7f315c02a340 comp rx=0 tx=0).stop 2024-05-09T05:59:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.795+0000 7f3179184640 1 -- 172.21.15.115:0/1149157110 shutdown_connections 2024-05-09T05:59:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.795+0000 7f3179184640 1 --2- 172.21.15.115:0/1149157110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3174104420 0x7f31741107f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.795+0000 7f3179184640 1 --2- 172.21.15.115:0/1149157110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f317410a170 0x7f3174103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.795+0000 7f3179184640 1 --2- 172.21.15.115:0/1149157110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3174109cd0 0x7f3174103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.795+0000 7f3179184640 1 -- 172.21.15.115:0/1149157110 >> 172.21.15.115:0/1149157110 conn(0x7f31740793d0 msgr2=0x7f3174079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.795+0000 7f3179184640 1 -- 172.21.15.115:0/1149157110 shutdown_connections 2024-05-09T05:59:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.795+0000 7f3179184640 1 -- 172.21.15.115:0/1149157110 wait complete. 2024-05-09T05:59:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.796+0000 7f3179184640 1 Processor -- start 2024-05-09T05:59:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.796+0000 7f3179184640 1 -- start start 2024-05-09T05:59:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3179184640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3174104420 0x7f3174074fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3179184640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3174109cd0 0x7f3174075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3179184640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317410a170 0x7f3174075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3179184640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3174113d90 con 0x7f3174109cd0 2024-05-09T05:59:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3179184640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3174113c10 con 0x7f317410a170 2024-05-09T05:59:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3179184640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3174113f10 con 0x7f3174104420 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3172d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3174104420 0x7f3174074fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f317259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3174109cd0 0x7f3174075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3172d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3174104420 0x7f3174074fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:51592/0 (socket says 172.21.15.115:51592) 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f317359e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317410a170 0x7f3174075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3172d9d640 1 -- 172.21.15.115:0/253148816 learned_addr learned my addr 172.21.15.115:0/253148816 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f317359e640 1 -- 172.21.15.115:0/253148816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3174104420 msgr2=0x7f3174074fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f317359e640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3174104420 0x7f3174074fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f317359e640 1 -- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3174109cd0 msgr2=0x7f3174075510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f317359e640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3174109cd0 0x7f3174075510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f317359e640 1 -- 172.21.15.115:0/253148816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31741a8b10 con 0x7f317410a170 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f3172d9d640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3174104420 0x7f3174074fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f317259c640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3174109cd0 0x7f3174075510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:59:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.797+0000 7f317359e640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317410a170 0x7f3174075a50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f315c009780 tx=0x7f315c02ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.798+0000 7f316bfff640 1 -- 172.21.15.115:0/253148816 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c037040 con 0x7f317410a170 2024-05-09T05:59:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.798+0000 7f3179184640 1 -- 172.21.15.115:0/253148816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31741a8da0 con 0x7f317410a170 2024-05-09T05:59:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.798+0000 7f316bfff640 1 -- 172.21.15.115:0/253148816 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f315c030bb0 con 0x7f317410a170 2024-05-09T05:59:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.798+0000 7f3179184640 1 -- 172.21.15.115:0/253148816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31741a91f0 con 0x7f317410a170 2024-05-09T05:59:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.799+0000 7f316bfff640 1 -- 172.21.15.115:0/253148816 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c0035f0 con 0x7f317410a170 2024-05-09T05:59:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.801+0000 7f316bfff640 1 -- 172.21.15.115:0/253148816 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f315c043020 con 0x7f317410a170 2024-05-09T05:59:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.801+0000 7f3180039740 1 Processor -- start 2024-05-09T05:59:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f3180039740 1 -- start start 2024-05-09T05:59:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f3180039740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d961b0 0x562a26139570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f3180039740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f3180039740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a26139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f3180039740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562a2609e7c0 con 0x562a25d6d2f0 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f3180039740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562a2603a0e0 con 0x562a25d754d0 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f3180039740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a2609a780 con 0x562a25d961b0 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f316bfff640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31600b29d0 0x7f31600b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f317259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d961b0 0x562a26139570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f317259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d961b0 0x562a26139570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:51600/0 (socket says 172.21.15.115:51600) 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f317259c640 1 -- 172.21.15.115:0/1748302401 learned_addr learned my addr 172.21.15.115:0/1748302401 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f317259c640 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a26139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f3172d9d640 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.802+0000 7f316bfff640 1 -- 172.21.15.115:0/253148816 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f315c101ab0 con 0x7f317410a170 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f317259c640 1 -- 172.21.15.115:0/1748302401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 msgr2=0x562a26139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:36.792 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f317259c640 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a26139ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f317259c640 1 -- 172.21.15.115:0/1748302401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 msgr2=0x562a26139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f317259c640 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f317259c640 1 -- 172.21.15.115:0/1748302401 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a260637c0 con 0x562a25d961b0 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f3172d9d640 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f317359e640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31600b29d0 0x7f31600b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f317259c640 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d961b0 0x562a26139570 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3164013b60 tx=0x7f3164004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37442bdbdc4f821 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f31697fa640 1 -- 172.21.15.115:0/1748302401 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164013c50 con 0x562a25d961b0 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f31697fa640 1 -- 172.21.15.115:0/1748302401 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31640044c0 con 0x562a25d961b0 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.803+0000 7f31697fa640 1 -- 172.21.15.115:0/1748302401 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164005280 con 0x562a25d961b0 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f3180039740 1 -- 172.21.15.115:0/1748302401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d961b0 msgr2=0x562a26139570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f317359e640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31600b29d0 0x7f31600b4ee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f315c0c8aa0 tx=0x7f315c095440 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f3180039740 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d961b0 0x562a26139570 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3164013b60 tx=0x7f3164004730 comp rx=0 tx=0).stop 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f3180039740 1 -- 172.21.15.115:0/1748302401 shutdown_connections 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f3180039740 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a26139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f3180039740 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f3180039740 1 --2- 172.21.15.115:0/1748302401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d961b0 0x562a26139570 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f3180039740 1 -- 172.21.15.115:0/1748302401 >> 172.21.15.115:0/1748302401 conn(0x562a25d8a300 msgr2=0x562a26135e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f3180039740 1 -- 172.21.15.115:0/1748302401 shutdown_connections 2024-05-09T05:59:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.804+0000 7f3180039740 1 -- 172.21.15.115:0/1748302401 wait complete. 2024-05-09T05:59:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3180039740 1 -- start start 2024-05-09T05:59:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3180039740 1 Processor -- start 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3180039740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d6d2f0 0x562a2614c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3180039740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a2614cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3180039740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d961b0 0x562a26152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3180039740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x562a2614efc0 con 0x562a25d961b0 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3180039740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562a260259c0 con 0x562a25d754d0 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3180039740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562a2614f140 con 0x562a25d6d2f0 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3180039740 10 client.0 _ll_register_callbacks cb 0x562a26052020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3172d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a2614cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3172d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a2614cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:52748/0 (socket says 172.21.15.115:52748) 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.805+0000 7f3172d9d640 1 -- 172.21.15.115:0/3056901464 learned_addr learned my addr 172.21.15.115:0/3056901464 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f317259c640 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d6d2f0 0x562a2614c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f317259c640 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d961b0 0x562a26152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f3172d9d640 1 -- 172.21.15.115:0/3056901464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d6d2f0 msgr2=0x562a2614c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f3172d9d640 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d6d2f0 0x562a2614c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f3172d9d640 1 -- 172.21.15.115:0/3056901464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d961b0 msgr2=0x562a26152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f3172d9d640 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d961b0 0x562a26152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f3172d9d640 1 -- 172.21.15.115:0/3056901464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a2604c1f0 con 0x562a25d754d0 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f317259c640 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d961b0 0x562a26152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f3172d9d640 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a2614cab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f316c011dd0 tx=0x7f316c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc14919ee47cda3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f315a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f315a7fc640 1 -- 172.21.15.115:0/3056901464 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c01f060 con 0x562a25d754d0 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f315a7fc640 1 -- 172.21.15.115:0/3056901464 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f316c011260 con 0x562a25d754d0 2024-05-09T05:59:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f3180039740 1 -- 172.21.15.115:0/3056901464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a260ac3d0 con 0x562a25d754d0 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f315a7fc640 1 -- 172.21.15.115:0/3056901464 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f316c026050 con 0x562a25d754d0 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f3180039740 10 client.28457 fetch_fsmap learned FSMap version 78 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.806+0000 7f3180039740 1 -- 172.21.15.115:0/3056901464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x562a25e01790 con 0x562a25d754d0 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f315a7fc640 1 -- 172.21.15.115:0/3056901464 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562a260ac3d0 con 0x562a25d754d0 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f315a7fc640 1 -- 172.21.15.115:0/3056901464 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 78) ==== 8374+0+0 (secure 0 0 0) 0x7f316c019400 con 0x562a25d754d0 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f3180039740 10 client.28457 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f3180039740 10 client.28457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f3180039740 20 client.28457 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f3180039740 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a25d54d80 0x562a2615e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f3180039740 4 client.28457 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f3180039740 1 -- 172.21.15.115:0/3056901464 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a260a6240 con 0x562a25d54d80 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f317259c640 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a25d54d80 0x562a2615e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f315a7fc640 10 client.28457 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.807+0000 7f317259c640 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a25d54d80 0x562a2615e970 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92001d96432ef24f server_cookie=4dcf197af7e8a4f7 in_seq=0 out_seq=0 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.808+0000 7f315a7fc640 1 -- 172.21.15.115:0/3056901464 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a260a6240 con 0x562a25d54d80 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.808+0000 7f315a7fc640 10 client.28457 handle_command_reply: tid=0 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.808+0000 7f3180039740 1 client.28457 shutdown 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 -- 172.21.15.115:0/3056901464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 msgr2=0x562a2614cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a2614cab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f316c011dd0 tx=0x7f316c00f790 comp rx=0 tx=0).stop 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 -- 172.21.15.115:0/3056901464 shutdown_connections 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d961b0 0x562a26152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d754d0 0x562a2614cab0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a25d54d80 0x562a2615e970 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 --2- 172.21.15.115:0/3056901464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d6d2f0 0x562a2614c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f315a7fc640 0 client.28457 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 -- 172.21.15.115:0/3056901464 >> 172.21.15.115:0/3056901464 conn(0x562a25d8a300 msgr2=0x562a2613b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 -- 172.21.15.115:0/3056901464 shutdown_connections 2024-05-09T05:59:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 1 -- 172.21.15.115:0/3056901464 wait complete. 2024-05-09T05:59:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.809+0000 7f3180039740 20 client.28457 trim_cache size 0 max 16384 2024-05-09T05:59:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.824+0000 7f3180039740 1 Processor -- start 2024-05-09T05:59:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.824+0000 7f3180039740 1 -- start start 2024-05-09T05:59:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3180039740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26135f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3180039740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a26152440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3180039740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 0x562a26145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3180039740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f315400fe90 con 0x562a25d6d2f0 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3180039740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f31540119d0 con 0x562a25d961b0 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3180039740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3154010560 con 0x562a25d754d0 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f317259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26135f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f317259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26135f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34752/0 (socket says 172.21.15.115:34752) 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f317259c640 1 -- 172.21.15.115:0/2390102329 learned_addr learned my addr 172.21.15.115:0/2390102329 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3172d9d640 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a26152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f317259c640 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 0x562a26145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3172d9d640 1 -- 172.21.15.115:0/2390102329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 msgr2=0x562a26145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3172d9d640 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 0x562a26145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3172d9d640 1 -- 172.21.15.115:0/2390102329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 msgr2=0x562a26135f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3172d9d640 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26135f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3172d9d640 1 -- 172.21.15.115:0/2390102329 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a25e785f0 con 0x562a25d754d0 2024-05-09T05:59:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f317259c640 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26135f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:59:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.825+0000 7f3172d9d640 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a26152440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f316c011da0 tx=0x7f316c00fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e47dd8ec763809 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.826+0000 7f3168ff9640 1 -- 172.21.15.115:0/2390102329 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c02f070 con 0x562a25d754d0 2024-05-09T05:59:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.826+0000 7f3168ff9640 1 -- 172.21.15.115:0/2390102329 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f316c003e00 con 0x562a25d754d0 2024-05-09T05:59:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.826+0000 7f3168ff9640 1 -- 172.21.15.115:0/2390102329 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c005370 con 0x562a25d754d0 2024-05-09T05:59:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 -- 172.21.15.115:0/2390102329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 msgr2=0x562a26152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a26152440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f316c011da0 tx=0x7f316c00fc90 comp rx=0 tx=0).stop 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 -- 172.21.15.115:0/2390102329 shutdown_connections 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 0x562a26145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a26152440 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 --2- 172.21.15.115:0/2390102329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a26135f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 -- 172.21.15.115:0/2390102329 >> 172.21.15.115:0/2390102329 conn(0x562a25d8a300 msgr2=0x562a26135810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 -- 172.21.15.115:0/2390102329 shutdown_connections 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 -- 172.21.15.115:0/2390102329 wait complete. 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 -- start start 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.827+0000 7f3180039740 1 Processor -- start 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f3180039740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a261634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f3180039740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a261639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f3180039740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 0x562a261c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f3180039740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3154011300 con 0x562a25d6d2f0 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f3180039740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3154010c30 con 0x562a25d961b0 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f3180039740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f315400de90 con 0x562a25d754d0 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f3180039740 10 client.0 _ll_register_callbacks cb 0x7f316c0119e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a261634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a261634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34766/0 (socket says 172.21.15.115:34766) 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 -- 172.21.15.115:0/1728015246 learned_addr learned my addr 172.21.15.115:0/1728015246 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f3172d9d640 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a261639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 0x562a261c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 -- 172.21.15.115:0/1728015246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 msgr2=0x562a261639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a261639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 -- 172.21.15.115:0/1728015246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 msgr2=0x562a261c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 0x562a261c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 -- 172.21.15.115:0/1728015246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a26062ed0 con 0x562a25d6d2f0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f3172d9d640 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a261639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f317259c640 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a261634b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f31640050d0 tx=0x7f3164015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0677de3f37fcca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f315a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.828+0000 7f315a7fc640 1 -- 172.21.15.115:0/1728015246 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164005560 con 0x562a25d6d2f0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f315a7fc640 1 -- 172.21.15.115:0/1728015246 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f316402b040 con 0x562a25d6d2f0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3180039740 1 -- 172.21.15.115:0/1728015246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f316c003e00 con 0x562a25d6d2f0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f315a7fc640 1 -- 172.21.15.115:0/1728015246 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f3164042050 con 0x562a25d6d2f0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3180039740 10 client.19323 fetch_fsmap learned FSMap version 78 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3180039740 1 -- 172.21.15.115:0/1728015246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x562a261c6000 con 0x562a25d6d2f0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f315a7fc640 1 -- 172.21.15.115:0/1728015246 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f316c003e00 con 0x562a25d6d2f0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f315a7fc640 1 -- 172.21.15.115:0/1728015246 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 78) ==== 8374+0+0 (secure 0 0 0) 0x7f31640305b0 con 0x562a25d6d2f0 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3180039740 10 client.19323 fetch_fsmap finished waiting for FSMap version 78 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3180039740 10 client.19323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3180039740 20 client.19323 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3180039740 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a25d2a1c0 0x562a261c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3180039740 4 client.19323 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3180039740 1 -- 172.21.15.115:0/1728015246 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a26021350 con 0x562a25d2a1c0 2024-05-09T05:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.829+0000 7f3172d9d640 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a25d2a1c0 0x562a261c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.830+0000 7f315a7fc640 10 client.19323 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.830+0000 7f3172d9d640 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a25d2a1c0 0x562a261c8660 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4fd9ec8b9b6fbe server_cookie=6cd29138b52f9bb3 in_seq=0 out_seq=0 2024-05-09T05:59:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.834+0000 7f315a7fc640 1 -- 172.21.15.115:0/1728015246 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a26021350 con 0x562a25d2a1c0 2024-05-09T05:59:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.834+0000 7f315a7fc640 10 client.19323 handle_command_reply: tid=0 2024-05-09T05:59:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.834+0000 7f3180039740 1 client.19323 shutdown 2024-05-09T05:59:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 -- 172.21.15.115:0/1728015246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 msgr2=0x562a261634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a261634b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f31640050d0 tx=0x7f3164015040 comp rx=0 tx=0).stop 2024-05-09T05:59:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 -- 172.21.15.115:0/1728015246 shutdown_connections 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562a25d961b0 0x562a261c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x562a25d2a1c0 0x562a261c8660 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562a25d754d0 0x562a261639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 --2- 172.21.15.115:0/1728015246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562a25d6d2f0 0x562a261634b0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f315a7fc640 0 client.19323 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 -- 172.21.15.115:0/1728015246 >> 172.21.15.115:0/1728015246 conn(0x562a25d8a300 msgr2=0x562a26134700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 -- 172.21.15.115:0/1728015246 shutdown_connections 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 1 -- 172.21.15.115:0/1728015246 wait complete. 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3180039740 20 client.19323 trim_cache size 0 max 16384 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:59:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:59:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3179184640 1 -- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31600b29d0 msgr2=0x7f31600b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.835+0000 7f3179184640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31600b29d0 0x7f31600b4ee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f315c0c8aa0 tx=0x7f315c095440 comp rx=0 tx=0).stop 2024-05-09T05:59:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 -- 172.21.15.115:0/253148816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317410a170 msgr2=0x7f3174075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317410a170 0x7f3174075a50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f315c009780 tx=0x7f315c02ae20 comp rx=0 tx=0).stop 2024-05-09T05:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 -- 172.21.15.115:0/253148816 shutdown_connections 2024-05-09T05:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f31600b29d0 0x7f31600b4ee0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317410a170 0x7f3174075a50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3174109cd0 0x7f3174075510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 --2- 172.21.15.115:0/253148816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3174104420 0x7f3174074fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 -- 172.21.15.115:0/253148816 >> 172.21.15.115:0/253148816 conn(0x7f31740793d0 msgr2=0x7f3174102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 -- 172.21.15.115:0/253148816 shutdown_connections 2024-05-09T05:59:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:36.836+0000 7f3179184640 1 -- 172.21.15.115:0/253148816 wait complete. 2024-05-09T05:59:37.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:37 smithi195 ceph-mon[33844]: pgmap v680: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 388 KiB/s wr, 107 op/s 2024-05-09T05:59:37.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:37 smithi115 ceph-mon[30349]: pgmap v680: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 388 KiB/s wr, 107 op/s 2024-05-09T05:59:37.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:37 smithi161 ceph-mon[33805]: pgmap v680: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 388 KiB/s wr, 107 op/s 2024-05-09T05:59:39.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:39 smithi195 ceph-mon[33844]: pgmap v681: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 441 KiB/s wr, 121 op/s 2024-05-09T05:59:39.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:39 smithi115 ceph-mon[30349]: pgmap v681: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 441 KiB/s wr, 121 op/s 2024-05-09T05:59:39.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:39 smithi161 ceph-mon[33805]: pgmap v681: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 441 KiB/s wr, 121 op/s 2024-05-09T05:59:41.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:41 smithi195 ceph-mon[33844]: pgmap v682: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 626 KiB/s wr, 167 op/s 2024-05-09T05:59:41.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:41 smithi115 ceph-mon[30349]: pgmap v682: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 626 KiB/s wr, 167 op/s 2024-05-09T05:59:41.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:41 smithi161 ceph-mon[33805]: pgmap v682: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 626 KiB/s wr, 167 op/s 2024-05-09T05:59:42.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:42 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:42.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:42 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:42.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:42 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:43.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:43 smithi195 ceph-mon[33844]: pgmap v683: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 543 KiB/s wr, 143 op/s 2024-05-09T05:59:43.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:43 smithi115 ceph-mon[30349]: pgmap v683: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 543 KiB/s wr, 143 op/s 2024-05-09T05:59:43.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:43 smithi161 ceph-mon[33805]: pgmap v683: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 543 KiB/s wr, 143 op/s 2024-05-09T05:59:44.011 INFO:tasks.workunit.client.0.smithi115.stdout:Syncing()...1 sec 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout:FFSB benchmark finished at: Thu May 9 05:59:44 2024 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout:Results: 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout:Benchmark took 440.51 sec 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout:Total Results 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout:=============== 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout: ======= ============ ========= ======= =========== ========== 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout: write : 988160 2243.20 100.000% 100.000% 8.76MB/sec 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout:- 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout:2243.20 Transactions per Second 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.012 INFO:tasks.workunit.client.0.smithi115.stdout:Throughput Results 2024-05-09T05:59:44.013 INFO:tasks.workunit.client.0.smithi115.stdout:=================== 2024-05-09T05:59:44.013 INFO:tasks.workunit.client.0.smithi115.stdout:Write Throughput: 8.76MB/sec 2024-05-09T05:59:44.016 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.016 INFO:tasks.workunit.client.0.smithi115.stdout:System Call Latency statistics in millisecs 2024-05-09T05:59:44.016 INFO:tasks.workunit.client.0.smithi115.stdout:===== 2024-05-09T05:59:44.016 INFO:tasks.workunit.client.0.smithi115.stdout: Min Avg Max Total Calls 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: ======== ======== ======== ============ 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout:[ open] 0.551000 270.281123 27961.412109 772 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[0] 5000.000000 - 10000.000000 : 11 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[1] 2000.000000 - 5000.000000 : 7 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[2] 1000.000000 - 2000.000000 : 4 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[3] 500.000000 - 1000.000000 : 2 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[4] 200.000000 - 500.000000 : 4 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[5] 100.000000 - 200.000000 : 26 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[6] 50.000000 - 100.000000 : 146 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[7] 20.000000 - 50.000000 : 296 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[8] 10.000000 - 20.000000 : 149 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[9] 5.000000 - 10.000000 : 62 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[10] 2.000000 - 5.000000 : 34 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[11] 1.000000 - 2.000000 : 21 2024-05-09T05:59:44.017 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[12] 0.500000 - 1.000000 : 6 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout:[ write] 0.090000 11.086273 63242.667969 988160 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[0] 5000.000000 - 10000.000000 : 28 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[1] 2000.000000 - 5000.000000 : 68 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[2] 1000.000000 - 2000.000000 : 58 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[3] 500.000000 - 1000.000000 : 133 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[4] 200.000000 - 500.000000 : 894 2024-05-09T05:59:44.018 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[5] 100.000000 - 200.000000 : 4138 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[6] 50.000000 - 100.000000 : 17484 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[7] 20.000000 - 50.000000 : 72899 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[8] 10.000000 - 20.000000 : 92357 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[9] 5.000000 - 10.000000 : 102632 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[10] 2.000000 - 5.000000 : 92924 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[11] 1.000000 - 2.000000 : 40231 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[12] 0.500000 - 1.000000 : 48154 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[13] 0.200000 - 0.500000 : 244284 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[14] 0.100000 - 0.200000 : 271680 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[15] 0.050000 - 0.100000 : 54 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.019 INFO:tasks.workunit.client.0.smithi115.stdout:[ lseek] 0.000000 0.000882 4.012000 988160 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[10] 2.000000 - 5.000000 : 2 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[11] 1.000000 - 2.000000 : 2 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[12] 0.500000 - 1.000000 : 1 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[13] 0.200000 - 0.500000 : 2 2024-05-09T05:59:44.020 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[14] 0.100000 - 0.200000 : 7 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[15] 0.050000 - 0.100000 : 11 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[16] 0.020000 - 0.050000 : 148 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[17] 0.010000 - 0.020000 : 703 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[18] 0.000000 - 0.010000 : 987284 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout:[ close] 0.024000 3.127505 107.350998 772 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[6] 50.000000 - 100.000000 : 6 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[7] 20.000000 - 50.000000 : 35 2024-05-09T05:59:44.021 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[8] 10.000000 - 20.000000 : 23 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[9] 5.000000 - 10.000000 : 28 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[10] 2.000000 - 5.000000 : 42 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[11] 1.000000 - 2.000000 : 20 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[12] 0.500000 - 1.000000 : 26 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[13] 0.200000 - 0.500000 : 38 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[14] 0.100000 - 0.200000 : 145 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[15] 0.050000 - 0.100000 : 68 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[16] 0.020000 - 0.050000 : 340 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.022 INFO:tasks.workunit.client.0.smithi115.stdout: 2024-05-09T05:59:44.023 INFO:tasks.workunit.client.0.smithi115.stdout:0.3% User Time 2024-05-09T05:59:44.023 INFO:tasks.workunit.client.0.smithi115.stdout:21.0% System Time 2024-05-09T05:59:44.023 INFO:tasks.workunit.client.0.smithi115.stdout:21.4% CPU Utilization 2024-05-09T05:59:44.084 INFO:tasks.workunit.client.0.smithi115.stderr:+ cd .. 2024-05-09T05:59:44.093 INFO:tasks.workunit.client.0.smithi115.stderr:+ rm -r tmp ffsb 2024-05-09T05:59:45.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:45 smithi195 ceph-mon[33844]: pgmap v684: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 696 KiB/s wr, 185 op/s 2024-05-09T05:59:45.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:45 smithi115 ceph-mon[30349]: pgmap v684: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 696 KiB/s wr, 185 op/s 2024-05-09T05:59:45.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:45 smithi161 ceph-mon[33805]: pgmap v684: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 696 KiB/s wr, 185 op/s 2024-05-09T05:59:47.804 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T05:59:47.804 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T05:59:47.859 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:47 smithi115 ceph-mon[30349]: pgmap v685: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 495 KiB/s wr, 132 op/s 2024-05-09T05:59:47.859 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:47 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:47.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:47 smithi195 ceph-mon[33844]: pgmap v685: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 495 KiB/s wr, 132 op/s 2024-05-09T05:59:47.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:47 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:47.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:47 smithi161 ceph-mon[33805]: pgmap v685: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 495 KiB/s wr, 132 op/s 2024-05-09T05:59:47.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:47 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:47.917 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-05-09T05:59:47.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T05:59:48.468 DEBUG:teuthology.parallel:result is None 2024-05-09T05:59:48.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T05:59:48.787 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T05:59:48.787 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-09T05:59:48.798 INFO:tasks.quiescer:joining Quiescers 2024-05-09T05:59:48.799 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-09T05:59:48.799 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:59:49.293 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:59:49.753 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:49 smithi115 ceph-mon[30349]: pgmap v686: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 521 KiB/s wr, 142 op/s 2024-05-09T05:59:49.753 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:49 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:49.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:49 smithi195 ceph-mon[33844]: pgmap v686: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 521 KiB/s wr, 142 op/s 2024-05-09T05:59:49.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:49 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:49.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:49 smithi161 ceph-mon[33805]: pgmap v686: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 521 KiB/s wr, 142 op/s 2024-05-09T05:59:49.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:49 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:49.997+0000 7f018ae13640 1 -- 172.21.15.115:0/3172050711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0184103310 msgr2=0x7f018410f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:49.997+0000 7f018ae13640 1 --2- 172.21.15.115:0/3172050711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0184103310 0x7f018410f6e0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f017c009ca0 tx=0x7f017c02a8f0 comp rx=0 tx=0).stop 2024-05-09T05:59:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:49.998+0000 7f018ae13640 1 -- 172.21.15.115:0/3172050711 shutdown_connections 2024-05-09T05:59:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:49.998+0000 7f018ae13640 1 --2- 172.21.15.115:0/3172050711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0184103310 0x7f018410f6e0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:49.998+0000 7f018ae13640 1 --2- 172.21.15.115:0/3172050711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0184109060 0x7f0184102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:49.998+0000 7f018ae13640 1 --2- 172.21.15.115:0/3172050711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0184108bc0 0x7f0184102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:49.998+0000 7f018ae13640 1 -- 172.21.15.115:0/3172050711 >> 172.21.15.115:0/3172050711 conn(0x7f01840781b0 msgr2=0x7f0184078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.003+0000 7f018ae13640 1 -- 172.21.15.115:0/3172050711 shutdown_connections 2024-05-09T05:59:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.004+0000 7f018ae13640 1 -- 172.21.15.115:0/3172050711 wait complete. 2024-05-09T05:59:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.005+0000 7f018ae13640 1 Processor -- start 2024-05-09T05:59:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.005+0000 7f018ae13640 1 -- start start 2024-05-09T05:59:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.005+0000 7f018ae13640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0184103310 0x7f01841a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f018ae13640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0184108bc0 0x7f01841a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f018ae13640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0184109060 0x7f01841a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f018ae13640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0184111c50 con 0x7f0184109060 2024-05-09T05:59:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f018ae13640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0184111ad0 con 0x7f0184103310 2024-05-09T05:59:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f018ae13640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0184111dd0 con 0x7f0184108bc0 2024-05-09T05:59:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f0188baf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0184103310 0x7f01841a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f0188baf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0184103310 0x7f01841a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:43574/0 (socket says 172.21.15.115:43574) 2024-05-09T05:59:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f0188baf640 1 -- 172.21.15.115:0/986332063 learned_addr learned my addr 172.21.15.115:0/986332063 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f0188baf640 1 -- 172.21.15.115:0/986332063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0184108bc0 msgr2=0x7f01841a14a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T05:59:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f0188baf640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0184108bc0 0x7f01841a14a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f0188baf640 1 -- 172.21.15.115:0/986332063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0184109060 msgr2=0x7f01841a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f0188baf640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0184109060 0x7f01841a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.006+0000 7f0188baf640 1 -- 172.21.15.115:0/986332063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01841a5da0 con 0x7f0184103310 2024-05-09T05:59:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.007+0000 7f0188baf640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0184103310 0x7f01841a0f40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f017000b470 tx=0x7f017000b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.008+0000 7f0181ffb640 1 -- 172.21.15.115:0/986332063 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170015020 con 0x7f0184103310 2024-05-09T05:59:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.008+0000 7f0181ffb640 1 -- 172.21.15.115:0/986332063 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0170011d20 con 0x7f0184103310 2024-05-09T05:59:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.008+0000 7f0181ffb640 1 -- 172.21.15.115:0/986332063 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170010da0 con 0x7f0184103310 2024-05-09T05:59:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.009+0000 7f018ae13640 1 -- 172.21.15.115:0/986332063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01841a6090 con 0x7f0184103310 2024-05-09T05:59:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.009+0000 7f018ae13640 1 -- 172.21.15.115:0/986332063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01841a6590 con 0x7f0184103310 2024-05-09T05:59:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.014+0000 7f0191da0740 1 Processor -- start 2024-05-09T05:59:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.017+0000 7f0191da0740 1 -- start start 2024-05-09T05:59:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.017+0000 7f0191da0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006c5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.017+0000 7f0191da0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 0x556c006c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.017+0000 7f0191da0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 0x556c006c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.017+0000 7f0191da0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556c0062a7c0 con 0x556c002f92f0 2024-05-09T05:59:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.017+0000 7f0191da0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556c005c60e0 con 0x556c003014d0 2024-05-09T05:59:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.017+0000 7f0191da0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556c00626780 con 0x556c003221b0 2024-05-09T05:59:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.020+0000 7f01893b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.020+0000 7f01893b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:58740/0 (socket says 172.21.15.115:58740) 2024-05-09T05:59:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.020+0000 7f01893b0640 1 -- 172.21.15.115:0/1727194636 learned_addr learned my addr 172.21.15.115:0/1727194636 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.020+0000 7f0188baf640 1 --2- 172.21.15.115:0/1727194636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 0x556c006c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.020+0000 7f0181ffb640 1 -- 172.21.15.115:0/986332063 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f0170018020 con 0x7f0184103310 2024-05-09T05:59:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.021+0000 7f0181ffb640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01740b2b60 0x7f01740b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.021+0000 7f0181ffb640 1 -- 172.21.15.115:0/986332063 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f01700e1f50 con 0x7f0184103310 2024-05-09T05:59:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.022+0000 7f0183fff640 1 --2- 172.21.15.115:0/1727194636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 0x556c006c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.022+0000 7f01893b0640 1 -- 172.21.15.115:0/1727194636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 msgr2=0x556c006c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.022+0000 7f01893b0640 1 --2- 172.21.15.115:0/1727194636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 0x556c006c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.022+0000 7f01893b0640 1 -- 172.21.15.115:0/1727194636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 msgr2=0x556c006c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.022+0000 7f01893b0640 1 --2- 172.21.15.115:0/1727194636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 0x556c006c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.022+0000 7f01893b0640 1 -- 172.21.15.115:0/1727194636 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c005ef7c0 con 0x556c003221b0 2024-05-09T05:59:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.022+0000 7f01893b0640 1 --2- 172.21.15.115:0/1727194636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006c5680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f017c0099e0 tx=0x7f017c009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caf1683b3221b2b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.024+0000 7f016effd640 1 -- 172.21.15.115:0/1727194636 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c031730 con 0x556c003221b0 2024-05-09T05:59:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.024+0000 7f016effd640 1 -- 172.21.15.115:0/1727194636 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f017c03f070 con 0x556c003221b0 2024-05-09T05:59:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.026+0000 7f01893b0640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01740b2b60 0x7f01740b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.026+0000 7f0191da0740 1 -- 172.21.15.115:0/1727194636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 msgr2=0x556c006c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.026+0000 7f01893b0640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01740b2b60 0x7f01740b5070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f017c043000 tx=0x7f017c004680 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.026+0000 7f0191da0740 1 --2- 172.21.15.115:0/1727194636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006c5680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f017c0099e0 tx=0x7f017c009a10 comp rx=0 tx=0).stop 2024-05-09T05:59:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.028+0000 7f0191da0740 1 -- 172.21.15.115:0/1727194636 shutdown_connections 2024-05-09T05:59:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.028+0000 7f0191da0740 1 --2- 172.21.15.115:0/1727194636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 0x556c006c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.028+0000 7f0191da0740 1 --2- 172.21.15.115:0/1727194636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 0x556c006c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.028+0000 7f0191da0740 1 --2- 172.21.15.115:0/1727194636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006c5680 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.028+0000 7f0191da0740 1 -- 172.21.15.115:0/1727194636 >> 172.21.15.115:0/1727194636 conn(0x556c00316300 msgr2=0x556c006c1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.028+0000 7f0191da0740 1 -- 172.21.15.115:0/1727194636 shutdown_connections 2024-05-09T05:59:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.029+0000 7f0191da0740 1 -- 172.21.15.115:0/1727194636 wait complete. 2024-05-09T05:59:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.029+0000 7f0191da0740 1 -- start start 2024-05-09T05:59:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.029+0000 7f0191da0740 1 Processor -- start 2024-05-09T05:59:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0191da0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c002f92f0 0x556c006cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0191da0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c003014d0 0x556c006d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0191da0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0188baf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c002f92f0 0x556c006cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0188baf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c002f92f0 0x556c006cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:43592/0 (socket says 172.21.15.115:43592) 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0188baf640 1 -- 172.21.15.115:0/298663188 learned_addr learned my addr 172.21.15.115:0/298663188 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556c006d27a0 con 0x556c003014d0 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556c005b19c0 con 0x556c002f92f0 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556c006d2920 con 0x556c003221b0 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f01893b0640 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.030+0000 7f0191da0740 10 client.0 _ll_register_callbacks cb 0x556c005de020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.031+0000 7f0183fff640 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c003014d0 0x556c006d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.031+0000 7f01893b0640 1 -- 172.21.15.115:0/298663188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c002f92f0 msgr2=0x556c006cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.031+0000 7f01893b0640 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c002f92f0 0x556c006cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.031+0000 7f01893b0640 1 -- 172.21.15.115:0/298663188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c003014d0 msgr2=0x556c006d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.031+0000 7f01893b0640 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c003014d0 0x556c006d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.031+0000 7f01893b0640 1 -- 172.21.15.115:0/298663188 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c005d81f0 con 0x556c003221b0 2024-05-09T05:59:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.032+0000 7f01893b0640 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5c70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f017c009ca0 tx=0x7f017c03a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6edd6455568ed8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.032+0000 7f0163fff640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:59:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.032+0000 7f0163fff640 1 -- 172.21.15.115:0/298663188 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c06f060 con 0x556c003221b0 2024-05-09T05:59:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.032+0000 7f0163fff640 1 -- 172.21.15.115:0/298663188 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f017c031530 con 0x556c003221b0 2024-05-09T05:59:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.033+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f017c03f070 con 0x556c003221b0 2024-05-09T05:59:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.033+0000 7f0163fff640 1 -- 172.21.15.115:0/298663188 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f017c004810 con 0x556c003221b0 2024-05-09T05:59:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.033+0000 7f0191da0740 10 client.28606 fetch_fsmap learned FSMap version 82 2024-05-09T05:59:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.033+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x556c0038d790 con 0x556c003221b0 2024-05-09T05:59:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.035+0000 7f0163fff640 1 -- 172.21.15.115:0/298663188 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f017c036030 con 0x556c003221b0 2024-05-09T05:59:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.035+0000 7f0163fff640 1 -- 172.21.15.115:0/298663188 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 82) ==== 8382+0+0 (secure 0 0 0) 0x7f017c0615b0 con 0x556c003221b0 2024-05-09T05:59:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.036+0000 7f0191da0740 10 client.28606 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T05:59:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.036+0000 7f0191da0740 10 client.28606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:59:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.036+0000 7f0191da0740 20 client.28606 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.036+0000 7f0191da0740 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c002e0d80 0x556c006e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.036+0000 7f0191da0740 4 client.28606 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:59:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.036+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f017c03f070 con 0x556c002e0d80 2024-05-09T05:59:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.037+0000 7f0188baf640 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c002e0d80 0x556c006e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.038+0000 7f0188baf640 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c002e0d80 0x556c006e1ad0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b85967d70a16f50 server_cookie=3b77d3ed82168b73 in_seq=0 out_seq=0 2024-05-09T05:59:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.039+0000 7f0163fff640 10 client.28606 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.042+0000 7f0163fff640 1 -- 172.21.15.115:0/298663188 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f017c03f070 con 0x556c002e0d80 2024-05-09T05:59:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.042+0000 7f0163fff640 10 client.28606 handle_command_reply: tid=0 2024-05-09T05:59:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.042+0000 7f0191da0740 1 client.28606 shutdown 2024-05-09T05:59:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.046+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 msgr2=0x556c006d5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.046+0000 7f0191da0740 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5c70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f017c009ca0 tx=0x7f017c03a000 comp rx=0 tx=0).stop 2024-05-09T05:59:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.047+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 shutdown_connections 2024-05-09T05:59:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.047+0000 7f0191da0740 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c002e0d80 0x556c006e1ad0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.047+0000 7f0191da0740 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5c70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.047+0000 7f0191da0740 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c003014d0 0x556c006d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.047+0000 7f0191da0740 1 --2- 172.21.15.115:0/298663188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c002f92f0 0x556c006cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.047+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 >> 172.21.15.115:0/298663188 conn(0x556c00316300 msgr2=0x556c006c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.048+0000 7f0163fff640 0 client.28606 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.050+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 shutdown_connections 2024-05-09T05:59:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.050+0000 7f0191da0740 1 -- 172.21.15.115:0/298663188 wait complete. 2024-05-09T05:59:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.052+0000 7f0191da0740 20 client.28606 trim_cache size 0 max 16384 2024-05-09T05:59:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.087+0000 7f0191da0740 1 Processor -- start 2024-05-09T05:59:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.089+0000 7f0191da0740 1 -- start start 2024-05-09T05:59:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.089+0000 7f0191da0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 0x556c006d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f0191da0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 0x556c006cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f0191da0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f0191da0740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f018406df00 con 0x556c002f92f0 2024-05-09T05:59:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f0191da0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f018406faa0 con 0x556c003014d0 2024-05-09T05:59:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f0191da0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f018406e5d0 con 0x556c003221b0 2024-05-09T05:59:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f0183fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 0x556c006cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f01893b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f01893b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:58754/0 (socket says 172.21.15.115:58754) 2024-05-09T05:59:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f01893b0640 1 -- 172.21.15.115:0/1280088989 learned_addr learned my addr 172.21.15.115:0/1280088989 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.090+0000 7f0188baf640 1 --2- 172.21.15.115:0/1280088989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 0x556c006d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.091+0000 7f0188baf640 1 -- 172.21.15.115:0/1280088989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 msgr2=0x556c006d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.091+0000 7f0188baf640 1 --2- 172.21.15.115:0/1280088989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.091+0000 7f0188baf640 1 -- 172.21.15.115:0/1280088989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 msgr2=0x556c006cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.091+0000 7f0188baf640 1 --2- 172.21.15.115:0/1280088989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 0x556c006cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.091+0000 7f0188baf640 1 -- 172.21.15.115:0/1280088989 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c004045f0 con 0x556c002f92f0 2024-05-09T05:59:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.091+0000 7f0188baf640 1 --2- 172.21.15.115:0/1280088989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 0x556c006d1c90 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f0170006fd0 tx=0x7f01700103e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1118ef18b3b98b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.094+0000 7f016e7fc640 1 -- 172.21.15.115:0/1280088989 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700a8f50 con 0x556c002f92f0 2024-05-09T05:59:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.094+0000 7f016e7fc640 1 -- 172.21.15.115:0/1280088989 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01700ad300 con 0x556c002f92f0 2024-05-09T05:59:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.094+0000 7f01893b0640 1 --2- 172.21.15.115:0/1280088989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:59:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.094+0000 7f016e7fc640 1 -- 172.21.15.115:0/1280088989 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700e5040 con 0x556c002f92f0 2024-05-09T05:59:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.096+0000 7f0191da0740 1 -- 172.21.15.115:0/1280088989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 msgr2=0x556c006d1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.096+0000 7f0191da0740 1 --2- 172.21.15.115:0/1280088989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 0x556c006d1c90 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f0170006fd0 tx=0x7f01700103e0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.099+0000 7f0191da0740 1 -- 172.21.15.115:0/1280088989 shutdown_connections 2024-05-09T05:59:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.099+0000 7f0191da0740 1 --2- 172.21.15.115:0/1280088989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c003221b0 0x556c006d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.099+0000 7f0191da0740 1 --2- 172.21.15.115:0/1280088989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003014d0 0x556c006cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.099+0000 7f0191da0740 1 --2- 172.21.15.115:0/1280088989 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c002f92f0 0x556c006d1c90 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.099+0000 7f0191da0740 1 -- 172.21.15.115:0/1280088989 >> 172.21.15.115:0/1280088989 conn(0x556c00316300 msgr2=0x556c006c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.101+0000 7f0191da0740 1 -- 172.21.15.115:0/1280088989 shutdown_connections 2024-05-09T05:59:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.101+0000 7f0191da0740 1 -- 172.21.15.115:0/1280088989 wait complete. 2024-05-09T05:59:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.102+0000 7f0191da0740 1 -- start start 2024-05-09T05:59:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.102+0000 7f0191da0740 1 Processor -- start 2024-05-09T05:59:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.103+0000 7f0191da0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c002f92f0 0x556c006dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.103+0000 7f0191da0740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c003014d0 0x556c006dc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.104+0000 7f0188baf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c002f92f0 0x556c006dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.104+0000 7f0188baf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c002f92f0 0x556c006dc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:58766/0 (socket says 172.21.15.115:58766) 2024-05-09T05:59:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.104+0000 7f0188baf640 1 -- 172.21.15.115:0/3564824580 learned_addr learned my addr 172.21.15.115:0/3564824580 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.104+0000 7f0191da0740 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003221b0 0x556c006d7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.104+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f018406f3d0 con 0x556c003014d0 2024-05-09T05:59:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.104+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f018406ecd0 con 0x556c003221b0 2024-05-09T05:59:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.104+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f018419f140 con 0x556c002f92f0 2024-05-09T05:59:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.104+0000 7f01893b0640 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003221b0 0x556c006d7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.105+0000 7f0191da0740 10 client.0 _ll_register_callbacks cb 0x7f017c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:59:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.105+0000 7f0188baf640 1 -- 172.21.15.115:0/3564824580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003221b0 msgr2=0x556c006d7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.105+0000 7f0188baf640 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003221b0 0x556c006d7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.105+0000 7f0188baf640 1 -- 172.21.15.115:0/3564824580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c003014d0 msgr2=0x556c006dc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:59:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.105+0000 7f0188baf640 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c003014d0 0x556c006dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.105+0000 7f0188baf640 1 -- 172.21.15.115:0/3564824580 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c005eeed0 con 0x556c002f92f0 2024-05-09T05:59:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.105+0000 7f0188baf640 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c002f92f0 0x556c006dc0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01700a8c50 tx=0x7f01700a5480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207a36316d0f61c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.105+0000 7f0163fff640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-05-09T05:59:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.109+0000 7f0163fff640 1 -- 172.21.15.115:0/3564824580 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700c8060 con 0x556c002f92f0 2024-05-09T05:59:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.109+0000 7f0163fff640 1 -- 172.21.15.115:0/3564824580 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f01700df6b0 con 0x556c002f92f0 2024-05-09T05:59:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.109+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01700ad300 con 0x556c002f92f0 2024-05-09T05:59:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.117+0000 7f0163fff640 1 -- 172.21.15.115:0/3564824580 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f01700a8f50 con 0x556c002f92f0 2024-05-09T05:59:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.117+0000 7f0191da0740 10 client.28612 fetch_fsmap learned FSMap version 82 2024-05-09T05:59:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.118+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x556c00749370 con 0x556c002f92f0 2024-05-09T05:59:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.125+0000 7f0163fff640 1 -- 172.21.15.115:0/3564824580 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01700ad300 con 0x556c002f92f0 2024-05-09T05:59:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.125+0000 7f0163fff640 1 -- 172.21.15.115:0/3564824580 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 82) ==== 8382+0+0 (secure 0 0 0) 0x7f01700a48c0 con 0x556c002f92f0 2024-05-09T05:59:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.125+0000 7f0191da0740 10 client.28612 fetch_fsmap finished waiting for FSMap version 82 2024-05-09T05:59:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.125+0000 7f0191da0740 10 client.28612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T05:59:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.125+0000 7f0191da0740 20 client.28612 populate_metadata read hostname 'smithi115' 2024-05-09T05:59:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.125+0000 7f0191da0740 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c002b61c0 0x556c0074ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.125+0000 7f0191da0740 4 client.28612 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T05:59:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.125+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c006383d0 con 0x556c002b61c0 2024-05-09T05:59:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.126+0000 7f0183fff640 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c002b61c0 0x556c0074ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.131+0000 7f0183fff640 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c002b61c0 0x556c0074ba00 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8719a3322bee1893 server_cookie=a5ae5e4f9eaca89e in_seq=0 out_seq=0 2024-05-09T05:59:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.132+0000 7f0163fff640 10 client.28612 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:50.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:50 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:50.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:50 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:50.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:50 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0163fff640 1 -- 172.21.15.115:0/3564824580 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556c006383d0 con 0x556c002b61c0 2024-05-09T05:59:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0163fff640 10 client.28612 handle_command_reply: tid=0 2024-05-09T05:59:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0191da0740 1 client.28612 shutdown 2024-05-09T05:59:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c002f92f0 msgr2=0x556c006dc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0191da0740 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c002f92f0 0x556c006dc0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01700a8c50 tx=0x7f01700a5480 comp rx=0 tx=0).stop 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 shutdown_connections 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0191da0740 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556c003221b0 0x556c006d7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0191da0740 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556c003014d0 0x556c006dc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0191da0740 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x556c002b61c0 0x556c0074ba00 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0191da0740 1 --2- 172.21.15.115:0/3564824580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556c002f92f0 0x556c006dc0e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 >> 172.21.15.115:0/3564824580 conn(0x556c00316300 msgr2=0x556c006ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.944+0000 7f0163fff640 0 client.28612 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.945+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 shutdown_connections 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.945+0000 7f0191da0740 1 -- 172.21.15.115:0/3564824580 wait complete. 2024-05-09T05:59:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.945+0000 7f0191da0740 20 client.28612 trim_cache size 0 max 16384 2024-05-09T05:59:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T05:59:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T05:59:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T05:59:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 -- 172.21.15.115:0/986332063 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01740b2b60 msgr2=0x7f01740b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01740b2b60 0x7f01740b5070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f017c043000 tx=0x7f017c004680 comp rx=0 tx=0).stop 2024-05-09T05:59:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 -- 172.21.15.115:0/986332063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0184103310 msgr2=0x7f01841a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0184103310 0x7f01841a0f40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f017000b470 tx=0x7f017000b930 comp rx=0 tx=0).stop 2024-05-09T05:59:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 -- 172.21.15.115:0/986332063 shutdown_connections 2024-05-09T05:59:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f01740b2b60 0x7f01740b5070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0184109060 0x7f01841a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0184108bc0 0x7f01841a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 --2- 172.21.15.115:0/986332063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0184103310 0x7f01841a0f40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 -- 172.21.15.115:0/986332063 >> 172.21.15.115:0/986332063 conn(0x7f01840781b0 msgr2=0x7f01841000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 -- 172.21.15.115:0/986332063 shutdown_connections 2024-05-09T05:59:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T05:59:50.947+0000 7f018ae13640 1 -- 172.21.15.115:0/986332063 wait complete. 2024-05-09T05:59:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.501+0000 7fa64ff21700 1 -- 172.21.15.115:0/3581104356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6481045d0 msgr2=0x7fa6481049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.502+0000 7fa64ff21700 1 --2- 172.21.15.115:0/3581104356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6481045d0 0x7fa6481049e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa6380099c0 tx=0x7fa638009cd0 comp rx=0 tx=0).stop 2024-05-09T05:59:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.502+0000 7fa64ff21700 1 -- 172.21.15.115:0/3581104356 shutdown_connections 2024-05-09T05:59:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.502+0000 7fa64ff21700 1 --2- 172.21.15.115:0/3581104356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa648109850 0x7fa6481111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.502+0000 7fa64ff21700 1 --2- 172.21.15.115:0/3581104356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6481045d0 0x7fa6481049e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.502+0000 7fa64ff21700 1 --2- 172.21.15.115:0/3581104356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa648108f00 0x7fa648109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.502+0000 7fa64ff21700 1 -- 172.21.15.115:0/3581104356 >> 172.21.15.115:0/3581104356 conn(0x7fa6480ffea0 msgr2=0x7fa6481022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.503+0000 7fa64ff21700 1 -- 172.21.15.115:0/3581104356 shutdown_connections 2024-05-09T05:59:51.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.503+0000 7fa64ff21700 1 -- 172.21.15.115:0/3581104356 wait complete. 2024-05-09T05:59:51.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.504+0000 7fa64ff21700 1 Processor -- start 2024-05-09T05:59:51.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.504+0000 7fa64ff21700 1 -- start start 2024-05-09T05:59:51.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.504+0000 7fa64ff21700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6481045d0 0x7fa648199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:51.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.504+0000 7fa64ff21700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa648108f00 0x7fa648199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:51.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.504+0000 7fa64dcbd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6481045d0 0x7fa648199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:51.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.504+0000 7fa64dcbd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6481045d0 0x7fa648199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54102/0 (socket says 172.21.15.115:54102) 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64d4bc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa648108f00 0x7fa648199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64d4bc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa648108f00 0x7fa648199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:43640/0 (socket says 172.21.15.115:43640) 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64ff21700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa648109850 0x7fa6481a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64d4bc700 1 -- 172.21.15.115:0/897129401 learned_addr learned my addr 172.21.15.115:0/897129401 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa648114380 con 0x7fa6481045d0 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa648114200 con 0x7fa648108f00 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa648114500 con 0x7fa648109850 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64e4be700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa648109850 0x7fa6481a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64dcbd700 1 -- 172.21.15.115:0/897129401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa648109850 msgr2=0x7fa6481a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:51.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64dcbd700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa648109850 0x7fa6481a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64dcbd700 1 -- 172.21.15.115:0/897129401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa648108f00 msgr2=0x7fa648199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64dcbd700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa648108f00 0x7fa648199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.505+0000 7fa64dcbd700 1 -- 172.21.15.115:0/897129401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6481a0df0 con 0x7fa6481045d0 2024-05-09T05:59:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.506+0000 7fa64dcbd700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6481045d0 0x7fa648199850 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098e0 tx=0x7fa6380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.506+0000 7fa63effd700 1 -- 172.21.15.115:0/897129401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638027070 con 0x7fa6481045d0 2024-05-09T05:59:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.506+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6481a1080 con 0x7fa6481045d0 2024-05-09T05:59:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.506+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6481a15b0 con 0x7fa6481045d0 2024-05-09T05:59:51.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.506+0000 7fa63effd700 1 -- 172.21.15.115:0/897129401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa63800fc00 con 0x7fa6481045d0 2024-05-09T05:59:51.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.506+0000 7fa63effd700 1 -- 172.21.15.115:0/897129401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638022c80 con 0x7fa6481045d0 2024-05-09T05:59:51.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.509+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c005330 con 0x7fa6481045d0 2024-05-09T05:59:51.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.511+0000 7fa63effd700 1 -- 172.21.15.115:0/897129401 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa63802a070 con 0x7fa6481045d0 2024-05-09T05:59:51.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.513+0000 7fa63effd700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6340b2af0 0x7fa6340b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:59:51.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.513+0000 7fa64d4bc700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6340b2af0 0x7fa6340b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:59:51.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.514+0000 7fa63effd700 1 -- 172.21.15.115:0/897129401 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fa6380e3600 con 0x7fa6481045d0 2024-05-09T05:59:51.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.514+0000 7fa64d4bc700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6340b2af0 0x7fa6340b5000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa648078820 tx=0x7fa64400b480 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:59:51.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:51.523+0000 7fa63effd700 1 -- 172.21.15.115:0/897129401 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa638013070 con 0x7fa6481045d0 2024-05-09T05:59:51.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:51 smithi115 ceph-mon[30349]: pgmap v687: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 652 KiB/s wr, 177 op/s 2024-05-09T05:59:51.641 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:51 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:51.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:51 smithi195 ceph-mon[33844]: pgmap v687: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 652 KiB/s wr, 177 op/s 2024-05-09T05:59:51.910 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:51 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:51.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:51 smithi161 ceph-mon[33805]: pgmap v687: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 652 KiB/s wr, 177 op/s 2024-05-09T05:59:51.912 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:51 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:52.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.019+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa62c0058f0 con 0x7fa6481045d0 2024-05-09T05:59:52.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.020+0000 7fa63effd700 1 -- 172.21.15.115:0/897129401 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+11969 (secure 0 0 0) 0x7fa6380aac80 con 0x7fa6481045d0 2024-05-09T05:59:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T05:59:52.011 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1419658869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1419658869},{"type":"v1","addr":"172.21.15.115:6835","nonce":1419658869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2746290424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2746290424},{"type":"v1","addr":"172.21.15.115:6837","nonce":2746290424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":84,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T05:33:54.732251+0000","modified":"2024-05-09T05:59:51.358371+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24457,"mds_2":24451,"mds_3":24440,"mds_4":24425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6839/3801478957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":3801478957},{"type":"v1","addr":"172.21.15.115:6839","nonce":3801478957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6841/1081505697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6840","nonce":1081505697},{"type":"v1","addr":"172.21.15.115:6841","nonce":1081505697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24386":{"gid":24386,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/634985009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":634985009},{"type":"v1","addr":"172.21.15.161:6833","nonce":634985009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/1779202002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1779202002},{"type":"v1","addr":"172.21.15.161:6835","nonce":1779202002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":25,"addr":"172.21.15.161:6837/3591780613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3591780613},{"type":"v1","addr":"172.21.15.161:6837","nonce":3591780613}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"b","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.161:6839/234220396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":234220396},{"type":"v1","addr":"172.21.15.161:6839","nonce":234220396}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":205,"addr":"172.21.15.195:6835/969435997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":969435997},{"type":"v1","addr":"172.21.15.195:6835","nonce":969435997}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.195:6837/4026198088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":4026198088},{"type":"v1","addr":"172.21.15.195:6837","nonce":4026198088}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.195:6839/1691623113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":1691623113},{"type":"v1","addr":"172.21.15.195:6839","nonce":1691623113}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28438":{"gid":28438,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6833/3680433414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3680433414},{"type":"v1","addr":"172.21.15.195:6833","nonce":3680433414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24457,24469,24451,24440,24425]},"id":1}]} 2024-05-09T05:59:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6340b2af0 msgr2=0x7fa6340b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6340b2af0 0x7fa6340b5000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa648078820 tx=0x7fa64400b480 comp rx=0 tx=0).stop 2024-05-09T05:59:52.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6481045d0 msgr2=0x7fa648199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:59:52.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6481045d0 0x7fa648199850 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098e0 tx=0x7fa6380047b0 comp rx=0 tx=0).stop 2024-05-09T05:59:52.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 shutdown_connections 2024-05-09T05:59:52.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa6340b2af0 0x7fa6340b5000 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:52.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa648109850 0x7fa6481a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:52.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6481045d0 0x7fa648199850 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:52.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 --2- 172.21.15.115:0/897129401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa648108f00 0x7fa648199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:59:52.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.031+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 >> 172.21.15.115:0/897129401 conn(0x7fa6480ffea0 msgr2=0x7fa648077510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:59:52.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.032+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 shutdown_connections 2024-05-09T05:59:52.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T05:59:52.033+0000 7fa64ff21700 1 -- 172.21.15.115:0/897129401 wait complete. 2024-05-09T05:59:52.029 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 84 2024-05-09T05:59:52.784 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --cancel --all' 2024-05-09T05:59:52.784 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- timeout 105 ceph tell mds.24469 quiesce db --cancel --all 2024-05-09T05:59:52.829 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:52 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:52.829 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:52 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/897129401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:59:52.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:52 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:52.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:52 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/897129401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:59:52.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:52 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:52.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:52 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/897129401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:59:53.312 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:59:53.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:53 smithi195 ceph-mon[33844]: pgmap v688: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 449 KiB/s wr, 129 op/s 2024-05-09T05:59:53.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:53 smithi115 ceph-mon[30349]: pgmap v688: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 449 KiB/s wr, 129 op/s 2024-05-09T05:59:53.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:53 smithi161 ceph-mon[33805]: pgmap v688: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 449 KiB/s wr, 129 op/s 2024-05-09T05:59:55.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:55 smithi115 ceph-mon[30349]: pgmap v689: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 454 KiB/s wr, 192 op/s 2024-05-09T05:59:55.359 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:55 smithi195 ceph-mon[33844]: pgmap v689: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 454 KiB/s wr, 192 op/s 2024-05-09T05:59:55.404 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:55 smithi161 ceph-mon[33805]: pgmap v689: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 454 KiB/s wr, 192 op/s 2024-05-09T05:59:56.359 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:56 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:56.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:56 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:56.404 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:56 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T05:59:57.359 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:57 smithi195 ceph-mon[33844]: pgmap v690: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 304 KiB/s wr, 170 op/s 2024-05-09T05:59:57.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:57 smithi115 ceph-mon[30349]: pgmap v690: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 304 KiB/s wr, 170 op/s 2024-05-09T05:59:57.404 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:57 smithi161 ceph-mon[33805]: pgmap v690: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 304 KiB/s wr, 170 op/s 2024-05-09T05:59:59.724 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 05:59:59 smithi115 ceph-mon[30349]: pgmap v691: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 283 KiB/s wr, 181 op/s 2024-05-09T05:59:59.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 05:59:59 smithi195 ceph-mon[33844]: pgmap v691: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 283 KiB/s wr, 181 op/s 2024-05-09T05:59:59.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 05:59:59 smithi161 ceph-mon[33805]: pgmap v691: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 283 KiB/s wr, 181 op/s 2024-05-09T06:00:00.390 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a[30345]: 2024-05-09T05:59:59.999+0000 7f5c15a39700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-09T06:00:00.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:00 smithi195 ceph-mon[33844]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-09T06:00:00.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:00 smithi115 ceph-mon[30349]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-09T06:00:00.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:00 smithi161 ceph-mon[33805]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-05-09T06:00:01.790 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:01 smithi115 ceph-mon[30349]: pgmap v692: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 219 KiB/s wr, 195 op/s 2024-05-09T06:00:01.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:01 smithi195 ceph-mon[33844]: pgmap v692: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 219 KiB/s wr, 195 op/s 2024-05-09T06:00:01.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:01 smithi161 ceph-mon[33805]: pgmap v692: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 219 KiB/s wr, 195 op/s 2024-05-09T06:00:02.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:02 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T06:00:02.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:02 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T06:00:02.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:02 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T06:00:03.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:03 smithi195 ceph-mon[33844]: pgmap v693: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 161 op/s 2024-05-09T06:00:03.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:03 smithi115 ceph-mon[30349]: pgmap v693: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 161 op/s 2024-05-09T06:00:03.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:03 smithi161 ceph-mon[33805]: pgmap v693: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 161 op/s 2024-05-09T06:00:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.135+0000 7f37792f1640 1 -- 172.21.15.115:0/3596257182 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770030820 con 0x7f377410e640 2024-05-09T06:00:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.136+0000 7f377c557640 1 -- 172.21.15.115:0/3596257182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377410e640 msgr2=0x7f3774110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.137+0000 7f377c557640 1 --2- 172.21.15.115:0/3596257182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377410e640 0x7f3774110a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3770009860 tx=0x7f377002a360 comp rx=0 tx=0).stop 2024-05-09T06:00:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.138+0000 7f377c557640 1 -- 172.21.15.115:0/3596257182 shutdown_connections 2024-05-09T06:00:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.138+0000 7f377c557640 1 --2- 172.21.15.115:0/3596257182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377410e640 0x7f3774110a60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.138+0000 7f377c557640 1 --2- 172.21.15.115:0/3596257182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3774101d70 0x7f3774102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.138+0000 7f377c557640 1 --2- 172.21.15.115:0/3596257182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3774108c10 0x7f3774101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.138+0000 7f377c557640 1 -- 172.21.15.115:0/3596257182 >> 172.21.15.115:0/3596257182 conn(0x7f37740fd6b0 msgr2=0x7f37740ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.144+0000 7f377c557640 1 -- 172.21.15.115:0/3596257182 shutdown_connections 2024-05-09T06:00:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.146+0000 7f377c557640 1 -- 172.21.15.115:0/3596257182 wait complete. 2024-05-09T06:00:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.148+0000 7f377c557640 1 Processor -- start 2024-05-09T06:00:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f377c557640 1 -- start start 2024-05-09T06:00:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f377c557640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3774101d70 0x7f3774074230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f377c557640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3774108c10 0x7f3774074770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f377c557640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377410e640 0x7f377406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f377c557640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f377406ca80 con 0x7f3774108c10 2024-05-09T06:00:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f377c557640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f377406c900 con 0x7f377410e640 2024-05-09T06:00:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f377c557640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f377406cc00 con 0x7f3774101d70 2024-05-09T06:00:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f3779af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3774108c10 0x7f3774074770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f3779af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3774108c10 0x7f3774074770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41252/0 (socket says 172.21.15.115:41252) 2024-05-09T06:00:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.150+0000 7f3779af2640 1 -- 172.21.15.115:0/836774968 learned_addr learned my addr 172.21.15.115:0/836774968 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.151+0000 7f377a2f3640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3774101d70 0x7f3774074230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.152+0000 7f377a2f3640 1 -- 172.21.15.115:0/836774968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377410e640 msgr2=0x7f377406d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T06:00:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.152+0000 7f377a2f3640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377410e640 0x7f377406d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.153+0000 7f377a2f3640 1 -- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3774108c10 msgr2=0x7f3774074770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.153+0000 7f377a2f3640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3774108c10 0x7f3774074770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.153+0000 7f377a2f3640 1 -- 172.21.15.115:0/836774968 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377406d9a0 con 0x7f3774101d70 2024-05-09T06:00:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.153+0000 7f3779af2640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3774108c10 0x7f3774074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:00:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.154+0000 7f377a2f3640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3774101d70 0x7f3774074230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3774075230 tx=0x7f376c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.155+0000 7f376b7fe640 1 -- 172.21.15.115:0/836774968 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c003970 con 0x7f3774101d70 2024-05-09T06:00:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.155+0000 7f376b7fe640 1 -- 172.21.15.115:0/836774968 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f376c004020 con 0x7f3774101d70 2024-05-09T06:00:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.156+0000 7f376b7fe640 1 -- 172.21.15.115:0/836774968 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c00b7b0 con 0x7f3774101d70 2024-05-09T06:00:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.157+0000 7f377c557640 1 -- 172.21.15.115:0/836774968 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377406dc90 con 0x7f3774101d70 2024-05-09T06:00:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.157+0000 7f377c557640 1 -- 172.21.15.115:0/836774968 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37741ad6f0 con 0x7f3774101d70 2024-05-09T06:00:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.160+0000 7f3783409740 1 Processor -- start 2024-05-09T06:00:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.161+0000 7f3783409740 1 -- start start 2024-05-09T06:00:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.161+0000 7f3783409740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f0ed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.161+0000 7f3783409740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed212f0 0x56295f0edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.161+0000 7f3783409740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.161+0000 7f3783409740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56295f0527c0 con 0x56295ed294d0 2024-05-09T06:00:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.161+0000 7f3783409740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56295efee0e0 con 0x56295ed4a1b0 2024-05-09T06:00:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.161+0000 7f3783409740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56295f04e780 con 0x56295ed212f0 2024-05-09T06:00:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.164+0000 7f3779af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.164+0000 7f3779af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0ee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41268/0 (socket says 172.21.15.115:41268) 2024-05-09T06:00:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.164+0000 7f3779af2640 1 -- 172.21.15.115:0/2850165403 learned_addr learned my addr 172.21.15.115:0/2850165403 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.164+0000 7f377aaf4640 1 --2- 172.21.15.115:0/2850165403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f0ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.165+0000 7f3779af2640 1 -- 172.21.15.115:0/2850165403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed212f0 msgr2=0x56295f0edbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T06:00:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.165+0000 7f3779af2640 1 --2- 172.21.15.115:0/2850165403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed212f0 0x56295f0edbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.165+0000 7f3779af2640 1 -- 172.21.15.115:0/2850165403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 msgr2=0x56295f0ed680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.165+0000 7f3779af2640 1 --2- 172.21.15.115:0/2850165403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f0ed680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.165+0000 7f3779af2640 1 -- 172.21.15.115:0/2850165403 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295f0177c0 con 0x56295ed294d0 2024-05-09T06:00:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.165+0000 7f3779af2640 1 --2- 172.21.15.115:0/2850165403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0ee100 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f376400f440 tx=0x7f376400f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9413da5d767d5af6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.170+0000 7f376b7fe640 1 -- 172.21.15.115:0/836774968 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f376c01d020 con 0x7f3774101d70 2024-05-09T06:00:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.172+0000 7f376b7fe640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37600b29d0 0x7f37600b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.172+0000 7f376b7fe640 1 -- 172.21.15.115:0/836774968 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f376c0e2890 con 0x7f3774101d70 2024-05-09T06:00:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.173+0000 7f377aaf4640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37600b29d0 0x7f37600b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.174+0000 7f377aaf4640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37600b29d0 0x7f37600b4ee0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56295f0ea7f0 tx=0x7f377002a860 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.179+0000 7f3768bfe640 1 -- 172.21.15.115:0/2850165403 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376401d070 con 0x56295ed294d0 2024-05-09T06:00:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.179+0000 7f3768bfe640 1 -- 172.21.15.115:0/2850165403 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3764010df0 con 0x56295ed294d0 2024-05-09T06:00:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.179+0000 7f3768bfe640 1 -- 172.21.15.115:0/2850165403 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376400d3a0 con 0x56295ed294d0 2024-05-09T06:00:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.180+0000 7f3783409740 1 -- 172.21.15.115:0/2850165403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 msgr2=0x56295f0ee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.180+0000 7f3783409740 1 --2- 172.21.15.115:0/2850165403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0ee100 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f376400f440 tx=0x7f376400f900 comp rx=0 tx=0).stop 2024-05-09T06:00:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.182+0000 7f3783409740 1 -- 172.21.15.115:0/2850165403 shutdown_connections 2024-05-09T06:00:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.182+0000 7f3783409740 1 --2- 172.21.15.115:0/2850165403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0ee100 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.182+0000 7f3783409740 1 --2- 172.21.15.115:0/2850165403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed212f0 0x56295f0edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.182+0000 7f3783409740 1 --2- 172.21.15.115:0/2850165403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f0ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.182+0000 7f3783409740 1 -- 172.21.15.115:0/2850165403 >> 172.21.15.115:0/2850165403 conn(0x56295ed3e300 msgr2=0x56295f0e9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.187+0000 7f3783409740 1 -- 172.21.15.115:0/2850165403 shutdown_connections 2024-05-09T06:00:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.187+0000 7f3783409740 1 -- 172.21.15.115:0/2850165403 wait complete. 2024-05-09T06:00:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.187+0000 7f3783409740 1 -- start start 2024-05-09T06:00:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.187+0000 7f3783409740 1 Processor -- start 2024-05-09T06:00:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.188+0000 7f3783409740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed212f0 0x56295f0f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.188+0000 7f3783409740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed294d0 0x56295f0f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.188+0000 7f3783409740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f0f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.188+0000 7f3783409740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x56295f0fa7b0 con 0x56295ed212f0 2024-05-09T06:00:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.188+0000 7f3783409740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56295efd99c0 con 0x56295ed4a1b0 2024-05-09T06:00:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.188+0000 7f3783409740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56295f0fa930 con 0x56295ed294d0 2024-05-09T06:00:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.188+0000 7f3783409740 10 client.0 _ll_register_callbacks cb 0x56295f006020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:00:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f377a2f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed212f0 0x56295f0f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f377a2f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed212f0 0x56295f0f9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41270/0 (socket says 172.21.15.115:41270) 2024-05-09T06:00:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f377a2f3640 1 -- 172.21.15.115:0/2334096771 learned_addr learned my addr 172.21.15.115:0/2334096771 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f377a2f3640 1 -- 172.21.15.115:0/2334096771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed294d0 msgr2=0x56295f0f9cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T06:00:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f377a2f3640 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed294d0 0x56295f0f9cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f377a2f3640 1 -- 172.21.15.115:0/2334096771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 msgr2=0x56295f0f7bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T06:00:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f377a2f3640 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f0f7bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f377a2f3640 1 -- 172.21.15.115:0/2334096771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295f0001f0 con 0x56295ed212f0 2024-05-09T06:00:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f377a2f3640 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed212f0 0x56295f0f9770 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f376c0a69f0 tx=0x7f376c0a9f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387ab8e3a016db24 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.189+0000 7f375dffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T06:00:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.190+0000 7f375dffb640 1 -- 172.21.15.115:0/2334096771 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c0e0080 con 0x56295ed212f0 2024-05-09T06:00:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.190+0000 7f375dffb640 1 -- 172.21.15.115:0/2334096771 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f376c0e2480 con 0x56295ed212f0 2024-05-09T06:00:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.192+0000 7f3783409740 1 -- 172.21.15.115:0/2334096771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3764010df0 con 0x56295ed212f0 2024-05-09T06:00:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.194+0000 7f375dffb640 1 -- 172.21.15.115:0/2334096771 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f376c0e2680 con 0x56295ed212f0 2024-05-09T06:00:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.198+0000 7f3783409740 10 client.19353 fetch_fsmap learned FSMap version 86 2024-05-09T06:00:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.198+0000 7f3783409740 1 -- 172.21.15.115:0/2334096771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x56295edb5790 con 0x56295ed212f0 2024-05-09T06:00:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.202+0000 7f375dffb640 1 -- 172.21.15.115:0/2334096771 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3764010df0 con 0x56295ed212f0 2024-05-09T06:00:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.202+0000 7f375dffb640 1 -- 172.21.15.115:0/2334096771 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f376c00b040 con 0x56295ed212f0 2024-05-09T06:00:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.204+0000 7f3783409740 10 client.19353 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T06:00:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.204+0000 7f3783409740 10 client.19353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T06:00:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.204+0000 7f3783409740 20 client.19353 populate_metadata read hostname 'smithi115' 2024-05-09T06:00:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.204+0000 7f3783409740 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56295ed08d80 0x56295f109ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.204+0000 7f3783409740 4 client.19353 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:00:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.204+0000 7f3783409740 1 -- 172.21.15.115:0/2334096771 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56295f0603d0 con 0x56295ed08d80 2024-05-09T06:00:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.210+0000 7f3779af2640 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56295ed08d80 0x56295f109ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.215+0000 7f3779af2640 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56295ed08d80 0x56295f109ff0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c7a85820dd00a3 server_cookie=930f148839e02b59 in_seq=0 out_seq=0 2024-05-09T06:00:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.215+0000 7f375dffb640 10 client.19353 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T06:00:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.218+0000 7f375dffb640 1 -- 172.21.15.115:0/2334096771 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56295f0603d0 con 0x56295ed08d80 2024-05-09T06:00:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.218+0000 7f375dffb640 10 client.19353 handle_command_reply: tid=0 2024-05-09T06:00:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.218+0000 7f3783409740 1 client.19353 shutdown 2024-05-09T06:00:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.221+0000 7f3783409740 1 -- 172.21.15.115:0/2334096771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed212f0 msgr2=0x56295f0f9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.221+0000 7f3783409740 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed212f0 0x56295f0f9770 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f376c0a69f0 tx=0x7f376c0a9f40 comp rx=0 tx=0).stop 2024-05-09T06:00:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.226+0000 7f3783409740 1 -- 172.21.15.115:0/2334096771 shutdown_connections 2024-05-09T06:00:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.226+0000 7f3783409740 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f0f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.226+0000 7f3783409740 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56295ed08d80 0x56295f109ff0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.226+0000 7f3783409740 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed294d0 0x56295f0f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.226+0000 7f3783409740 1 --2- 172.21.15.115:0/2334096771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed212f0 0x56295f0f9770 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.226+0000 7f3783409740 1 -- 172.21.15.115:0/2334096771 >> 172.21.15.115:0/2334096771 conn(0x56295ed3e300 msgr2=0x56295f0ef3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.227+0000 7f375dffb640 0 client.19353 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T06:00:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.227+0000 7f3783409740 1 -- 172.21.15.115:0/2334096771 shutdown_connections 2024-05-09T06:00:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.228+0000 7f3783409740 1 -- 172.21.15.115:0/2334096771 wait complete. 2024-05-09T06:00:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.230+0000 7f3783409740 20 client.19353 trim_cache size 0 max 16384 2024-05-09T06:00:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3783409740 1 Processor -- start 2024-05-09T06:00:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3783409740 1 -- start start 2024-05-09T06:00:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3783409740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed212f0 0x56295f0ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3783409740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f10f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3783409740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f10fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3783409740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37741c86a0 con 0x56295ed294d0 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3783409740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f37741ca2d0 con 0x56295ed4a1b0 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3783409740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f37741c8e00 con 0x56295ed212f0 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3779af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f10f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3779af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f10f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41280/0 (socket says 172.21.15.115:41280) 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f3779af2640 1 -- 172.21.15.115:0/755402833 learned_addr learned my addr 172.21.15.115:0/755402833 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f377a2f3640 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed212f0 0x56295f0ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.259+0000 7f377aaf4640 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f10fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f3779af2640 1 -- 172.21.15.115:0/755402833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed212f0 msgr2=0x56295f0ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f3779af2640 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed212f0 0x56295f0ef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f3779af2640 1 -- 172.21.15.115:0/755402833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 msgr2=0x56295f10fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f3779af2640 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f10fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f3779af2640 1 -- 172.21.15.115:0/755402833 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295ee2c5f0 con 0x56295ed294d0 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f377aaf4640 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f10fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:00:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f3779af2640 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f10f7f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f37640058c0 tx=0x7f3764000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf3a173160e4761 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f375ffff640 1 -- 172.21.15.115:0/755402833 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376401d040 con 0x56295ed294d0 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f375ffff640 1 -- 172.21.15.115:0/755402833 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3764018b00 con 0x56295ed294d0 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.260+0000 7f375ffff640 1 -- 172.21.15.115:0/755402833 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37640054b0 con 0x56295ed294d0 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 -- 172.21.15.115:0/755402833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 msgr2=0x56295f10f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f10f7f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f37640058c0 tx=0x7f3764000bc0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 -- 172.21.15.115:0/755402833 shutdown_connections 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed4a1b0 0x56295f10fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f10f7f0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 --2- 172.21.15.115:0/755402833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed212f0 0x56295f0ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 -- 172.21.15.115:0/755402833 >> 172.21.15.115:0/755402833 conn(0x56295ed3e300 msgr2=0x56295f0ef230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 -- 172.21.15.115:0/755402833 shutdown_connections 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 -- 172.21.15.115:0/755402833 wait complete. 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 -- start start 2024-05-09T06:00:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.261+0000 7f3783409740 1 Processor -- start 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3783409740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed212f0 0x56295f0ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3783409740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3783409740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed4a1b0 0x56295f0fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3783409740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f37741c9c00 con 0x56295ed294d0 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3783409740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f37741c9500 con 0x56295ed212f0 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3783409740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f37741c6700 con 0x56295ed4a1b0 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3783409740 10 client.0 _ll_register_callbacks cb 0x7f376c0a5640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3779af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3779af2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0fb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41282/0 (socket says 172.21.15.115:41282) 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3779af2640 1 -- 172.21.15.115:0/870164123 learned_addr learned my addr 172.21.15.115:0/870164123 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f377aaf4640 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed4a1b0 0x56295f0fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f377a2f3640 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed212f0 0x56295f0ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3779af2640 1 -- 172.21.15.115:0/870164123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed4a1b0 msgr2=0x56295f0fb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3779af2640 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed4a1b0 0x56295f0fb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3779af2640 1 -- 172.21.15.115:0/870164123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed212f0 msgr2=0x56295f0ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3779af2640 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed212f0 0x56295f0ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3779af2640 1 -- 172.21.15.115:0/870164123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295f016ed0 con 0x56295ed294d0 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f377aaf4640 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed4a1b0 0x56295f0fb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f377a2f3640 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed212f0 0x56295f0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f3779af2640 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0fb180 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f3764006010 tx=0x7f376400ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01ba3730ca3da13 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f375dffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.262+0000 7f375dffb640 1 -- 172.21.15.115:0/870164123 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376401d040 con 0x56295ed294d0 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f375dffb640 1 -- 172.21.15.115:0/870164123 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f3764019cc0 con 0x56295ed294d0 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f3783409740 1 -- 172.21.15.115:0/870164123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3764018b00 con 0x56295ed294d0 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f375dffb640 1 -- 172.21.15.115:0/870164123 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f3764018440 con 0x56295ed294d0 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f3783409740 10 client.19359 fetch_fsmap learned FSMap version 86 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f3783409740 1 -- 172.21.15.115:0/870164123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x56295f171d90 con 0x56295ed294d0 2024-05-09T06:00:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f375dffb640 1 -- 172.21.15.115:0/870164123 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3764018b00 con 0x56295ed294d0 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f375dffb640 1 -- 172.21.15.115:0/870164123 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7f3764010960 con 0x56295ed294d0 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f3783409740 10 client.19359 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f3783409740 10 client.19359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f3783409740 20 client.19359 populate_metadata read hostname 'smithi115' 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f3783409740 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56295ecde1c0 0x56295f174220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f3783409740 4 client.19359 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f3783409740 1 -- 172.21.15.115:0/870164123 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56295f05a240 con 0x56295ecde1c0 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.263+0000 7f377a2f3640 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56295ecde1c0 0x56295f174220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.264+0000 7f377a2f3640 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56295ecde1c0 0x56295f174220 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824ba7e4e0cac6be server_cookie=a8c8d0358c9a4a49 in_seq=0 out_seq=0 2024-05-09T06:00:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.264+0000 7f375dffb640 10 client.19359 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T06:00:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.471+0000 7f375dffb640 1 -- 172.21.15.115:0/870164123 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56295f05a240 con 0x56295ecde1c0 2024-05-09T06:00:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.471+0000 7f375dffb640 10 client.19359 handle_command_reply: tid=0 2024-05-09T06:00:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.471+0000 7f3783409740 1 client.19359 shutdown 2024-05-09T06:00:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.471+0000 7f3783409740 1 -- 172.21.15.115:0/870164123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 msgr2=0x56295f0fb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.471+0000 7f3783409740 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0fb180 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f3764006010 tx=0x7f376400ffc0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.471+0000 7f3783409740 1 -- 172.21.15.115:0/870164123 shutdown_connections 2024-05-09T06:00:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.472+0000 7f3783409740 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x56295ecde1c0 0x56295f174220 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.459 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.472+0000 7f375dffb640 0 client.19359 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T06:00:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.472+0000 7f3783409740 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56295ed4a1b0 0x56295f0fb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.472+0000 7f3783409740 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56295ed294d0 0x56295f0fb180 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.472+0000 7f3783409740 1 --2- 172.21.15.115:0/870164123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56295ed212f0 0x56295f0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.472+0000 7f3783409740 1 -- 172.21.15.115:0/870164123 >> 172.21.15.115:0/870164123 conn(0x56295ed3e300 msgr2=0x56295f0e8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.472+0000 7f3783409740 1 -- 172.21.15.115:0/870164123 shutdown_connections 2024-05-09T06:00:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.472+0000 7f3783409740 1 -- 172.21.15.115:0/870164123 wait complete. 2024-05-09T06:00:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.472+0000 7f3783409740 20 client.19359 trim_cache size 0 max 16384 2024-05-09T06:00:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:{ 2024-05-09T06:00:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout: "return_code": 0 2024-05-09T06:00:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stdout:} 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.473+0000 7f377c557640 1 -- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37600b29d0 msgr2=0x7f37600b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.473+0000 7f377c557640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37600b29d0 0x7f37600b4ee0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56295f0ea7f0 tx=0x7f377002a860 comp rx=0 tx=0).stop 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.473+0000 7f377c557640 1 -- 172.21.15.115:0/836774968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3774101d70 msgr2=0x7f3774074230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.473+0000 7f377c557640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3774101d70 0x7f3774074230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3774075230 tx=0x7f376c00c800 comp rx=0 tx=0).stop 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.474+0000 7f377c557640 1 -- 172.21.15.115:0/836774968 shutdown_connections 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.474+0000 7f377c557640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f37600b29d0 0x7f37600b4ee0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.474+0000 7f377c557640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f377410e640 0x7f377406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.474+0000 7f377c557640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3774108c10 0x7f3774074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.474+0000 7f377c557640 1 --2- 172.21.15.115:0/836774968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3774101d70 0x7f3774074230 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.474+0000 7f377c557640 1 -- 172.21.15.115:0/836774968 >> 172.21.15.115:0/836774968 conn(0x7f37740fd6b0 msgr2=0x7f377410fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.474+0000 7f377c557640 1 -- 172.21.15.115:0/836774968 shutdown_connections 2024-05-09T06:00:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi115.front.sepia.ceph.com.smithi115.stderr:2024-05-09T06:00:04.474+0000 7f377c557640 1 -- 172.21.15.115:0/836774968 wait complete. 2024-05-09T06:00:05.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:05 smithi195 ceph-mon[33844]: pgmap v694: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 KiB/s wr, 204 op/s 2024-05-09T06:00:05.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:05 smithi115 ceph-mon[30349]: pgmap v694: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 KiB/s wr, 204 op/s 2024-05-09T06:00:05.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:05 smithi161 ceph-mon[33805]: pgmap v694: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 KiB/s wr, 204 op/s 2024-05-09T06:00:06.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.015+0000 7fe96ac21700 1 -- 172.21.15.115:0/2780077860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9641034c0 msgr2=0x7fe9641038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:06.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.015+0000 7fe96ac21700 1 --2- 172.21.15.115:0/2780077860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9641034c0 0x7fe9641038d0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fe9580099c0 tx=0x7fe958009cd0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.015+0000 7fe96ac21700 1 -- 172.21.15.115:0/2780077860 shutdown_connections 2024-05-09T06:00:06.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.015+0000 7fe96ac21700 1 --2- 172.21.15.115:0/2780077860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe964108740 0x7fe9641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.015+0000 7fe96ac21700 1 --2- 172.21.15.115:0/2780077860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9641034c0 0x7fe9641038d0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.015+0000 7fe96ac21700 1 --2- 172.21.15.115:0/2780077860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe964107df0 0x7fe964108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.015+0000 7fe96ac21700 1 -- 172.21.15.115:0/2780077860 >> 172.21.15.115:0/2780077860 conn(0x7fe9640fedf0 msgr2=0x7fe964101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:06.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.015+0000 7fe96ac21700 1 -- 172.21.15.115:0/2780077860 shutdown_connections 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.015+0000 7fe96ac21700 1 -- 172.21.15.115:0/2780077860 wait complete. 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe96ac21700 1 Processor -- start 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe96ac21700 1 -- start start 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe96ac21700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9641034c0 0x7fe96419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe96ac21700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe964107df0 0x7fe96419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe96ac21700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe964108740 0x7fe96419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe96ac21700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe964113220 con 0x7fe964107df0 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe96ac21700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641130a0 con 0x7fe964108740 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe96ac21700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641133a0 con 0x7fe9641034c0 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe963fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe964107df0 0x7fe96419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe963fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe964107df0 0x7fe96419b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41298/0 (socket says 172.21.15.115:41298) 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe963fff700 1 -- 172.21.15.115:0/1811061483 learned_addr learned my addr 172.21.15.115:0/1811061483 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe9689bd700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9641034c0 0x7fe96419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.016+0000 7fe9691be700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe964108740 0x7fe96419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe963fff700 1 -- 172.21.15.115:0/1811061483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9641034c0 msgr2=0x7fe96419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:06.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe963fff700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9641034c0 0x7fe96419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe963fff700 1 -- 172.21.15.115:0/1811061483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe964108740 msgr2=0x7fe96419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:06.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe963fff700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe964108740 0x7fe96419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe963fff700 1 -- 172.21.15.115:0/1811061483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96419fbf0 con 0x7fe964107df0 2024-05-09T06:00:06.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe963fff700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe964107df0 0x7fe96419b2c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe95000e970 tx=0x7fe95000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe961ffb700 1 -- 172.21.15.115:0/1811061483 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950004a30 con 0x7fe964107df0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe961ffb700 1 -- 172.21.15.115:0/1811061483 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe95000cc70 con 0x7fe964107df0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe96ac21700 1 -- 172.21.15.115:0/1811061483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96419fee0 con 0x7fe964107df0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe961ffb700 1 -- 172.21.15.115:0/1811061483 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950005520 con 0x7fe964107df0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.017+0000 7fe96ac21700 1 -- 172.21.15.115:0/1811061483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641a7800 con 0x7fe964107df0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.020+0000 7fe961ffb700 1 -- 172.21.15.115:0/1811061483 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fe95000c7e0 con 0x7fe964107df0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.020+0000 7fe961ffb700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9540b2a70 0x7fe9540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.020+0000 7fe961ffb700 1 -- 172.21.15.115:0/1811061483 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fe9500e2a80 con 0x7fe964107df0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.020+0000 7fe9689bd700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9540b2a70 0x7fe9540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe9689bd700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9540b2a70 0x7fe9540b4f80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe958000c00 tx=0x7fe95801a040 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe979637b80 1 Processor -- start 2024-05-09T06:00:06.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe979637b80 1 -- start start 2024-05-09T06:00:06.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe979637b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536fc6d70 0x5565370c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe979637b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536f4c510 0x5565370c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe979637b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565370d3870 0x5565370d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe979637b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556536fec110 con 0x5565370d3870 2024-05-09T06:00:06.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe979637b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556536f91a10 con 0x556536f4c510 2024-05-09T06:00:06.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe979637b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556536fe80d0 con 0x556536fc6d70 2024-05-09T06:00:06.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe9691be700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565370d3870 0x5565370d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe9689bd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536fc6d70 0x5565370c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe963fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536f4c510 0x5565370c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe9691be700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565370d3870 0x5565370d5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41310/0 (socket says 172.21.15.115:41310) 2024-05-09T06:00:06.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe963fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536f4c510 0x5565370c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:44698/0 (socket says 172.21.15.115:44698) 2024-05-09T06:00:06.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.021+0000 7fe9689bd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536fc6d70 0x5565370c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:57482/0 (socket says 172.21.15.115:57482) 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe9691be700 1 -- 172.21.15.115:0/1288596291 learned_addr learned my addr 172.21.15.115:0/1288596291 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe963fff700 1 -- 172.21.15.115:0/1288596291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536fc6d70 msgr2=0x5565370c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe963fff700 1 --2- 172.21.15.115:0/1288596291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536fc6d70 0x5565370c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe963fff700 1 -- 172.21.15.115:0/1288596291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565370d3870 msgr2=0x5565370d5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe963fff700 1 --2- 172.21.15.115:0/1288596291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565370d3870 0x5565370d5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe963fff700 1 -- 172.21.15.115:0/1288596291 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556536f82570 con 0x556536f4c510 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe9691be700 1 --2- 172.21.15.115:0/1288596291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565370d3870 0x5565370d5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe9689bd700 1 --2- 172.21.15.115:0/1288596291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536fc6d70 0x5565370c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe963fff700 1 --2- 172.21.15.115:0/1288596291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536f4c510 0x5565370c9100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe9500dfe40 tx=0x7fe9500a91c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea74a39575a4596 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:06.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe94effd700 1 -- 172.21.15.115:0/1288596291 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500a94e0 con 0x556536f4c510 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe94effd700 1 -- 172.21.15.115:0/1288596291 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe95000ce20 con 0x556536f4c510 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.022+0000 7fe94effd700 1 -- 172.21.15.115:0/1288596291 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500a5c60 con 0x556536f4c510 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.023+0000 7fe979637b80 1 -- 172.21.15.115:0/1288596291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536f4c510 msgr2=0x5565370c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.023+0000 7fe979637b80 1 --2- 172.21.15.115:0/1288596291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536f4c510 0x5565370c9100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe9500dfe40 tx=0x7fe9500a91c0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.023+0000 7fe979637b80 1 -- 172.21.15.115:0/1288596291 shutdown_connections 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.023+0000 7fe979637b80 1 --2- 172.21.15.115:0/1288596291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5565370d3870 0x5565370d5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.023+0000 7fe979637b80 1 --2- 172.21.15.115:0/1288596291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536fc6d70 0x5565370c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.023+0000 7fe979637b80 1 --2- 172.21.15.115:0/1288596291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536f4c510 0x5565370c9100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.023+0000 7fe979637b80 1 -- 172.21.15.115:0/1288596291 >> 172.21.15.115:0/1288596291 conn(0x556536f4ffe0 msgr2=0x5565370c4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.023+0000 7fe979637b80 1 -- 172.21.15.115:0/1288596291 shutdown_connections 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.023+0000 7fe979637b80 1 -- 172.21.15.115:0/1288596291 wait complete. 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe979637b80 1 -- start start 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe979637b80 1 Processor -- start 2024-05-09T06:00:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe979637b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536f4c510 0x5565370d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe979637b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536fc6d70 0x5565370d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe979637b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565370d3870 0x5565370d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe979637b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5565370d6290 con 0x556536fc6d70 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe979637b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556536f86b90 con 0x5565370d3870 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe979637b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5565370d6410 con 0x556536f4c510 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe979637b80 10 client.0 _ll_register_callbacks cb 0x556536e5d730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe963fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536fc6d70 0x5565370d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe963fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536fc6d70 0x5565370d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41312/0 (socket says 172.21.15.115:41312) 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe963fff700 1 -- 172.21.15.115:0/657774169 learned_addr learned my addr 172.21.15.115:0/657774169 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe9691be700 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565370d3870 0x5565370d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.024+0000 7fe9689bd700 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536f4c510 0x5565370d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe963fff700 1 -- 172.21.15.115:0/657774169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536f4c510 msgr2=0x5565370d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe963fff700 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536f4c510 0x5565370d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe963fff700 1 -- 172.21.15.115:0/657774169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565370d3870 msgr2=0x5565370d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe963fff700 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565370d3870 0x5565370d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe963fff700 1 -- 172.21.15.115:0/657774169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556536e67500 con 0x556536fc6d70 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe9689bd700 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536f4c510 0x5565370d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe963fff700 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536fc6d70 0x5565370d0ea0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fe95000cfd0 tx=0x7fe9500cc380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a1e3201605b43a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe943fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe943fff700 1 -- 172.21.15.115:0/657774169 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500a94e0 con 0x556536fc6d70 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe943fff700 1 -- 172.21.15.115:0/657774169 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fe9500a9e40 con 0x556536fc6d70 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe979637b80 1 -- 172.21.15.115:0/657774169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95000ce20 con 0x556536fc6d70 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe943fff700 1 -- 172.21.15.115:0/657774169 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe9500aa0d0 con 0x556536fc6d70 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe979637b80 10 client.19371 fetch_fsmap learned FSMap version 86 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe979637b80 1 -- 172.21.15.115:0/657774169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x5565370e0fe0 con 0x556536fc6d70 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.025+0000 7fe943fff700 1 -- 172.21.15.115:0/657774169 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe95000ce20 con 0x556536fc6d70 2024-05-09T06:00:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe943fff700 1 -- 172.21.15.115:0/657774169 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7fe95008a080 con 0x556536fc6d70 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe979637b80 10 client.19371 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe979637b80 10 client.19371 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe979637b80 20 client.19371 populate_metadata read hostname 'smithi115' 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe979637b80 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565370e34b0 0x5565370e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe979637b80 4 client.19371 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe979637b80 1 -- 172.21.15.115:0/657774169 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556536ff9af0 con 0x5565370e34b0 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe9689bd700 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565370e34b0 0x5565370e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe9689bd700 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565370e34b0 0x5565370e58d0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1657827b3d1142c5 server_cookie=a6c271ea348263e2 in_seq=0 out_seq=0 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.026+0000 7fe943fff700 10 client.19371 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe943fff700 1 -- 172.21.15.115:0/657774169 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556536ff9af0 con 0x5565370e34b0 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe943fff700 10 client.19371 handle_command_reply: tid=0 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe979637b80 1 client.19371 shutdown 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe979637b80 1 -- 172.21.15.115:0/657774169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536fc6d70 msgr2=0x5565370d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:06.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe979637b80 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536fc6d70 0x5565370d0ea0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fe95000cfd0 tx=0x7fe9500cc380 comp rx=0 tx=0).stop 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe979637b80 1 -- 172.21.15.115:0/657774169 shutdown_connections 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe979637b80 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565370d3870 0x5565370d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe979637b80 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x5565370e34b0 0x5565370e58d0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe979637b80 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556536f4c510 0x5565370d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe979637b80 1 --2- 172.21.15.115:0/657774169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536fc6d70 0x5565370d0ea0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.027+0000 7fe943fff700 0 client.19371 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.028+0000 7fe979637b80 1 -- 172.21.15.115:0/657774169 >> 172.21.15.115:0/657774169 conn(0x556536f4ffe0 msgr2=0x5565370c3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.028+0000 7fe979637b80 1 -- 172.21.15.115:0/657774169 shutdown_connections 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.028+0000 7fe979637b80 1 -- 172.21.15.115:0/657774169 wait complete. 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.028+0000 7fe979637b80 20 client.19371 trim_cache size 0 max 16384 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.047+0000 7fe979637b80 1 Processor -- start 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.047+0000 7fe979637b80 1 -- start start 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.047+0000 7fe979637b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.047+0000 7fe979637b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 0x5565370dedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.047+0000 7fe979637b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 0x5565370e8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.047+0000 7fe979637b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b0520 con 0x556536f4c510 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.047+0000 7fe979637b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b2150 con 0x556536fc6d70 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.047+0000 7fe979637b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b0c80 con 0x5565370d3870 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9689bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9689bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370d2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41322/0 (socket says 172.21.15.115:41322) 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9689bd700 1 -- 172.21.15.115:0/2358725897 learned_addr learned my addr 172.21.15.115:0/2358725897 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9691be700 1 --2- 172.21.15.115:0/2358725897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 0x5565370e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe963fff700 1 --2- 172.21.15.115:0/2358725897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 0x5565370dedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9689bd700 1 -- 172.21.15.115:0/2358725897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 msgr2=0x5565370e8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9689bd700 1 --2- 172.21.15.115:0/2358725897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 0x5565370e8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9689bd700 1 -- 172.21.15.115:0/2358725897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 msgr2=0x5565370dedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9689bd700 1 --2- 172.21.15.115:0/2358725897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 0x5565370dedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9689bd700 1 -- 172.21.15.115:0/2358725897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565370ceed0 con 0x556536f4c510 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe9689bd700 1 --2- 172.21.15.115:0/2358725897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370d2a40 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fe95800a010 tx=0x7fe958009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d47dfb03e406765 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe94e7fc700 1 -- 172.21.15.115:0/2358725897 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958032670 con 0x556536f4c510 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe94e7fc700 1 -- 172.21.15.115:0/2358725897 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe958037070 con 0x556536f4c510 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.048+0000 7fe94e7fc700 1 -- 172.21.15.115:0/2358725897 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95800f460 con 0x556536f4c510 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.049+0000 7fe979637b80 1 -- 172.21.15.115:0/2358725897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 msgr2=0x5565370d2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.049+0000 7fe979637b80 1 --2- 172.21.15.115:0/2358725897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370d2a40 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fe95800a010 tx=0x7fe958009cd0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.049+0000 7fe979637b80 1 -- 172.21.15.115:0/2358725897 shutdown_connections 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.049+0000 7fe979637b80 1 --2- 172.21.15.115:0/2358725897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 0x5565370e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.049+0000 7fe979637b80 1 --2- 172.21.15.115:0/2358725897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370d2a40 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.049+0000 7fe979637b80 1 --2- 172.21.15.115:0/2358725897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 0x5565370dedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.049+0000 7fe979637b80 1 -- 172.21.15.115:0/2358725897 >> 172.21.15.115:0/2358725897 conn(0x556536f4ffe0 msgr2=0x5565370de6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.049+0000 7fe979637b80 1 -- 172.21.15.115:0/2358725897 shutdown_connections 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.049+0000 7fe979637b80 1 -- 172.21.15.115:0/2358725897 wait complete. 2024-05-09T06:00:06.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe979637b80 1 -- start start 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe979637b80 1 Processor -- start 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe979637b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe979637b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 0x5565370e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe979637b80 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 0x5565370e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe979637b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b1a80 con 0x556536f4c510 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe979637b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b1380 con 0x556536fc6d70 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe979637b80 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641ae4e0 con 0x5565370d3870 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe9689bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe979637b80 10 client.0 _ll_register_callbacks cb 0x556536e5d730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe9689bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370e74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41336/0 (socket says 172.21.15.115:41336) 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe9689bd700 1 -- 172.21.15.115:0/1849505504 learned_addr learned my addr 172.21.15.115:0/1849505504 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe963fff700 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 0x5565370e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.050+0000 7fe9691be700 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 0x5565370e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe9689bd700 1 -- 172.21.15.115:0/1849505504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 msgr2=0x5565370e7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe9689bd700 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 0x5565370e7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe9689bd700 1 -- 172.21.15.115:0/1849505504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 msgr2=0x5565370e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe9689bd700 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 0x5565370e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe9689bd700 1 -- 172.21.15.115:0/1849505504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653714da10 con 0x556536f4c510 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe9689bd700 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370e74b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fe958005250 tx=0x7fe958009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f701bc614d5ee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe943fff700 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe943fff700 1 -- 172.21.15.115:0/1849505504 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958032670 con 0x556536f4c510 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe943fff700 1 -- 172.21.15.115:0/1849505504 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fe95800b790 con 0x556536f4c510 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe979637b80 1 -- 172.21.15.115:0/1849505504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe958037070 con 0x556536f4c510 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe943fff700 1 -- 172.21.15.115:0/1849505504 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe95800b990 con 0x556536f4c510 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe979637b80 10 client.19377 fetch_fsmap learned FSMap version 86 2024-05-09T06:00:06.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe979637b80 1 -- 172.21.15.115:0/1849505504 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x55653714ebb0 con 0x556536f4c510 2024-05-09T06:00:06.674 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T06:00:06.674 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 86, 2024-05-09T06:00:06.674 INFO:teuthology.orchestra.run.smithi115.stdout: "leader": 24469, 2024-05-09T06:00:06.674 INFO:teuthology.orchestra.run.smithi115.stdout: "set_version": 60, 2024-05-09T06:00:06.674 INFO:teuthology.orchestra.run.smithi115.stdout: "sets": {} 2024-05-09T06:00:06.674 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T06:00:06.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe943fff700 1 -- 172.21.15.115:0/1849505504 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe958037070 con 0x556536f4c510 2024-05-09T06:00:06.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe943fff700 1 -- 172.21.15.115:0/1849505504 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 86) ==== 8402+0+0 (secure 0 0 0) 0x7fe958021070 con 0x556536f4c510 2024-05-09T06:00:06.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.051+0000 7fe979637b80 10 client.19377 fetch_fsmap finished waiting for FSMap version 86 2024-05-09T06:00:06.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe979637b80 10 client.19377 resolve_mds: validated gid 24469 aka daemon mds.i 2024-05-09T06:00:06.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe979637b80 20 client.19377 populate_metadata read hostname 'smithi115' 2024-05-09T06:00:06.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe979637b80 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55653714f210 0x556537151630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:06.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe979637b80 4 client.19377 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-09T06:00:06.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe979637b80 1 -- 172.21.15.115:0/1849505504 --> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x556536ff3b90 con 0x55653714f210 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe963fff700 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55653714f210 0x556537151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe943fff700 10 client.19377 ms_handle_connect on v2:172.21.15.195:6838/1691623113 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe963fff700 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55653714f210 0x556537151630 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4fe778e68beef7a server_cookie=a1fffa88d134f851 in_seq=0 out_seq=0 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe943fff700 1 -- 172.21.15.115:0/1849505504 <== mds.0 v2:172.21.15.195:6838/1691623113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x556536ff3b90 con 0x55653714f210 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.052+0000 7fe943fff700 10 client.19377 handle_command_reply: tid=0 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 client.19377 shutdown 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 -- 172.21.15.115:0/1849505504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 msgr2=0x5565370e74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370e74b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fe958005250 tx=0x7fe958009300 comp rx=0 tx=0).stop 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 -- 172.21.15.115:0/1849505504 shutdown_connections 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.195:6838/1691623113,v1:172.21.15.195:6839/1691623113] conn(0x55653714f210 0x556537151630 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5565370d3870 0x5565370e7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe943fff700 0 client.19377 ms_handle_reset on v2:172.21.15.195:6838/1691623113 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556536f4c510 0x5565370e74b0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 --2- 172.21.15.115:0/1849505504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556536fc6d70 0x5565370e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 -- 172.21.15.115:0/1849505504 >> 172.21.15.115:0/1849505504 conn(0x556536f4ffe0 msgr2=0x5565370d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 -- 172.21.15.115:0/1849505504 shutdown_connections 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 1 -- 172.21.15.115:0/1849505504 wait complete. 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe979637b80 20 client.19377 trim_cache size 0 max 16384 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe96ac21700 1 -- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9540b2a70 msgr2=0x7fe9540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe96ac21700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9540b2a70 0x7fe9540b4f80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe958000c00 tx=0x7fe95801a040 comp rx=0 tx=0).stop 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe96ac21700 1 -- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe964107df0 msgr2=0x7fe96419b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.053+0000 7fe96ac21700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe964107df0 0x7fe96419b2c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe95000e970 tx=0x7fe95000ec80 comp rx=0 tx=0).stop 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.054+0000 7fe96ac21700 1 -- 172.21.15.115:0/1811061483 shutdown_connections 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.054+0000 7fe96ac21700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fe9540b2a70 0x7fe9540b4f80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.054+0000 7fe96ac21700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe964108740 0x7fe96419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.054+0000 7fe96ac21700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9641034c0 0x7fe96419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.054+0000 7fe96ac21700 1 --2- 172.21.15.115:0/1811061483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe964107df0 0x7fe96419b2c0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.054+0000 7fe96ac21700 1 -- 172.21.15.115:0/1811061483 >> 172.21.15.115:0/1811061483 conn(0x7fe9640fedf0 msgr2=0x7fe964101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.054+0000 7fe96ac21700 1 -- 172.21.15.115:0/1811061483 shutdown_connections 2024-05-09T06:00:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:05.054+0000 7fe96ac21700 1 -- 172.21.15.115:0/1811061483 wait complete. 2024-05-09T06:00:06.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:06 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:00:06.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:06 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:00:06.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:06 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:00:07.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:07 smithi195 ceph-mon[33844]: pgmap v695: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 KiB/s wr, 171 op/s 2024-05-09T06:00:07.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:07 smithi115 ceph-mon[30349]: pgmap v695: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 KiB/s wr, 171 op/s 2024-05-09T06:00:07.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:07 smithi161 ceph-mon[33805]: pgmap v695: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 55 KiB/s wr, 171 op/s 2024-05-09T06:00:09.815 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:09 smithi115 ceph-mon[30349]: pgmap v696: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 161 op/s 2024-05-09T06:00:09.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:09 smithi195 ceph-mon[33844]: pgmap v696: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 161 op/s 2024-05-09T06:00:09.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:09 smithi161 ceph-mon[33805]: pgmap v696: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 161 op/s 2024-05-09T06:00:10.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:10 smithi195 ceph-mon[33844]: osdmap e124: 12 total, 12 up, 12 in 2024-05-09T06:00:10.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:10 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:00:10.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:10 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:00:10.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:10 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T06:00:10.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:10 smithi115 ceph-mon[30349]: osdmap e124: 12 total, 12 up, 12 in 2024-05-09T06:00:10.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:10 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:00:10.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:10 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:00:10.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:10 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T06:00:10.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:10 smithi161 ceph-mon[33805]: osdmap e124: 12 total, 12 up, 12 in 2024-05-09T06:00:10.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:10 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:00:10.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:10 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:00:10.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:10 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T06:00:11.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:11 smithi115 ceph-mon[30349]: pgmap v698: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 61 KiB/s wr, 178 op/s 2024-05-09T06:00:11.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:11 smithi195 ceph-mon[33844]: pgmap v698: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 61 KiB/s wr, 178 op/s 2024-05-09T06:00:11.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:11 smithi161 ceph-mon[33805]: pgmap v698: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 61 KiB/s wr, 178 op/s 2024-05-09T06:00:13.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:13 smithi195 ceph-mon[33844]: pgmap v699: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 60 KiB/s wr, 168 op/s 2024-05-09T06:00:13.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:13 smithi115 ceph-mon[30349]: pgmap v699: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 60 KiB/s wr, 168 op/s 2024-05-09T06:00:13.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:13 smithi161 ceph-mon[33805]: pgmap v699: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 15 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 60 KiB/s wr, 168 op/s 2024-05-09T06:00:15.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:15 smithi195 ceph-mon[33844]: pgmap v700: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 118 op/s 2024-05-09T06:00:15.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:15 smithi115 ceph-mon[30349]: pgmap v700: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 118 op/s 2024-05-09T06:00:15.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:15 smithi161 ceph-mon[33805]: pgmap v700: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 118 op/s 2024-05-09T06:00:17.378 INFO:tasks.quiescer:done joining Quiescers 2024-05-09T06:00:17.378 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-09T06:00:17.391 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T06:00:17.403 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-09T06:00:17.499 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-09T06:00:17.499 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T06:00:17.499 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T06:00:17.500 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-09T06:00:17.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:17 smithi115 ceph-mon[30349]: pgmap v701: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 17 KiB/s wr, 82 op/s 2024-05-09T06:00:17.764 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-09T06:00:17.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:17 smithi195 ceph-mon[33844]: pgmap v701: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 17 KiB/s wr, 82 op/s 2024-05-09T06:00:17.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:17 smithi161 ceph-mon[33805]: pgmap v701: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 17 KiB/s wr, 82 op/s 2024-05-09T06:00:17.959 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-09T06:00:18.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.173+0000 7f21a2f8d640 1 -- 172.21.15.115:0/4025863931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f219c11a8f0 msgr2=0x7f219c11cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:18.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.173+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/4025863931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f219c11a8f0 0x7f219c11cd10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f218401a910 tx=0x7f218403b500 comp rx=0 tx=0).stop 2024-05-09T06:00:18.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.174+0000 7f21a2f8d640 1 -- 172.21.15.115:0/4025863931 shutdown_connections 2024-05-09T06:00:18.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.174+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/4025863931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f219c11d250 0x7f219c11f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.174+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/4025863931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f219c11a8f0 0x7f219c11cd10 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.174+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/4025863931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f219c10a200 0x7f219c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.174+0000 7f21a2f8d640 1 -- 172.21.15.115:0/4025863931 >> 172.21.15.115:0/4025863931 conn(0x7f219c06e900 msgr2=0x7f219c06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:18.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.174+0000 7f21a2f8d640 1 -- 172.21.15.115:0/4025863931 shutdown_connections 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.174+0000 7f21a2f8d640 1 -- 172.21.15.115:0/4025863931 wait complete. 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.175+0000 7f21a2f8d640 1 Processor -- start 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.175+0000 7f21a2f8d640 1 -- start start 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.176+0000 7f21a2f8d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f219c10a200 0x7f219c1166a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.176+0000 7f21a2f8d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f219c11d250 0x7f219c116be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.176+0000 7f21a2f8d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f219c117120 0x7f219c1ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.176+0000 7f21a2f8d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f219c114900 con 0x7f219c10a200 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.176+0000 7f21a2f8d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f219c114780 con 0x7f219c117120 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.176+0000 7f21a152a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f219c117120 0x7f219c1ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.176+0000 7f21a152a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f219c117120 0x7f219c1ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:60728/0 (socket says 172.21.15.115:60728) 2024-05-09T06:00:18.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.176+0000 7f21a152a640 1 -- 172.21.15.115:0/3499011401 learned_addr learned my addr 172.21.15.115:0/3499011401 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:18.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.176+0000 7f21a2f8d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f219c114a80 con 0x7f219c11d250 2024-05-09T06:00:18.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.177+0000 7f219bfff640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f219c11d250 0x7f219c116be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:18.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.178+0000 7f21a152a640 1 -- 172.21.15.115:0/3499011401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f219c11d250 msgr2=0x7f219c116be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:18.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.178+0000 7f21a152a640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f219c11d250 0x7f219c116be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.178+0000 7f21a152a640 1 -- 172.21.15.115:0/3499011401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f219c10a200 msgr2=0x7f219c1166a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:18.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.178+0000 7f21a152a640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f219c10a200 0x7f219c1166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.178+0000 7f21a152a640 1 -- 172.21.15.115:0/3499011401 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c1ad690 con 0x7f219c117120 2024-05-09T06:00:18.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.179+0000 7f21a152a640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f219c117120 0x7f219c1ad150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f218c00b3f0 tx=0x7f218c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:18.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.180+0000 7f2199ffb640 1 -- 172.21.15.115:0/3499011401 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c015020 con 0x7f219c117120 2024-05-09T06:00:18.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.180+0000 7f2199ffb640 1 -- 172.21.15.115:0/3499011401 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f218c011cc0 con 0x7f219c117120 2024-05-09T06:00:18.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.181+0000 7f2199ffb640 1 -- 172.21.15.115:0/3499011401 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c010d70 con 0x7f219c117120 2024-05-09T06:00:18.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.181+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c1ad8c0 con 0x7f219c117120 2024-05-09T06:00:18.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.181+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c1adde0 con 0x7f219c117120 2024-05-09T06:00:18.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.183+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219c10b520 con 0x7f219c117120 2024-05-09T06:00:18.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.185+0000 7f2199ffb640 1 -- 172.21.15.115:0/3499011401 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f218c047d10 con 0x7f219c117120 2024-05-09T06:00:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.186+0000 7f2199ffb640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f21880b2b20 0x7f21880b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.186+0000 7f2199ffb640 1 -- 172.21.15.115:0/3499011401 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f218c0e2680 con 0x7f219c117120 2024-05-09T06:00:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.189+0000 7f21a0d29640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f21880b2b20 0x7f21880b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:18.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.190+0000 7f2199ffb640 1 -- 172.21.15.115:0/3499011401 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f218c01f020 con 0x7f219c117120 2024-05-09T06:00:18.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.192+0000 7f21a0d29640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f21880b2b20 0x7f21880b5030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f219400abb0 tx=0x7f2194009250 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:18.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.544+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f219c11b470 con 0x7f21880b2b20 2024-05-09T06:00:18.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.581+0000 7f2199ffb640 1 -- 172.21.15.115:0/3499011401 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f219c11b470 con 0x7f21880b2b20 2024-05-09T06:00:18.569 INFO:teuthology.orchestra.run.smithi115.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-09T00:00:00", "created": "2024-05-09T05:38:25", "first": "2024-05-09T05:39:00", "last": "2024-05-09T06:00:00", "last_pruned": "2024-05-09T06:00:03", "created_count": 22, "pruned_count": 16, "active": true} 2024-05-09T06:00:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.584+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f21880b2b20 msgr2=0x7f21880b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.584+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f21880b2b20 0x7f21880b5030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f219400abb0 tx=0x7f2194009250 comp rx=0 tx=0).stop 2024-05-09T06:00:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.584+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f219c117120 msgr2=0x7f219c1ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.584+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f219c117120 0x7f219c1ad150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f218c00b3f0 tx=0x7f218c00b8b0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.585+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 shutdown_connections 2024-05-09T06:00:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.585+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f21880b2b20 0x7f21880b5030 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.585+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f219c117120 0x7f219c1ad150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.585+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f219c11d250 0x7f219c116be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.585+0000 7f21a2f8d640 1 --2- 172.21.15.115:0/3499011401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f219c10a200 0x7f219c1166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.585+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 >> 172.21.15.115:0/3499011401 conn(0x7f219c06e900 msgr2=0x7f219c11ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.585+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 shutdown_connections 2024-05-09T06:00:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.585+0000 7f21a2f8d640 1 -- 172.21.15.115:0/3499011401 wait complete. 2024-05-09T06:00:18.591 DEBUG:teuthology.orchestra.run.smithi115:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-09T06:00:18.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.795+0000 7fa61929a640 1 -- 172.21.15.115:0/154281417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa614106b20 msgr2=0x7fa614106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:18.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.795+0000 7fa61929a640 1 --2- 172.21.15.115:0/154281417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa614106b20 0x7fa614106f50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa608009900 tx=0x7fa60802a420 comp rx=0 tx=0).stop 2024-05-09T06:00:18.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.795+0000 7fa61929a640 1 -- 172.21.15.115:0/154281417 shutdown_connections 2024-05-09T06:00:18.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.795+0000 7fa61929a640 1 --2- 172.21.15.115:0/154281417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6141075e0 0x7fa61410ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.795+0000 7fa61929a640 1 --2- 172.21.15.115:0/154281417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa614106b20 0x7fa614106f50 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa608009900 tx=0x7fa60802a420 comp rx=0 tx=0).stop 2024-05-09T06:00:18.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.795+0000 7fa61929a640 1 --2- 172.21.15.115:0/154281417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa614102590 0x7fa6141029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.795+0000 7fa61929a640 1 -- 172.21.15.115:0/154281417 >> 172.21.15.115:0/154281417 conn(0x7fa6140fd9d0 msgr2=0x7fa6140ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:18.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.797+0000 7fa61929a640 1 -- 172.21.15.115:0/154281417 shutdown_connections 2024-05-09T06:00:18.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.798+0000 7fa61929a640 1 -- 172.21.15.115:0/154281417 wait complete. 2024-05-09T06:00:18.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.799+0000 7fa61929a640 1 Processor -- start 2024-05-09T06:00:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.800+0000 7fa61929a640 1 -- start start 2024-05-09T06:00:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.800+0000 7fa61929a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa614102590 0x7fa61410e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.800+0000 7fa61929a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6141075e0 0x7fa61410ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.800+0000 7fa61929a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa61410d4b0 0x7fa6141ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.800+0000 7fa61929a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa6141124d0 con 0x7fa6141075e0 2024-05-09T06:00:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.800+0000 7fa61929a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa614112350 con 0x7fa614102590 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.800+0000 7fa61929a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa614112650 con 0x7fa61410d4b0 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.801+0000 7fa612ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa614102590 0x7fa61410e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.801+0000 7fa612ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa614102590 0x7fa61410e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:60754/0 (socket says 172.21.15.115:60754) 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.801+0000 7fa612ffd640 1 -- 172.21.15.115:0/3092831914 learned_addr learned my addr 172.21.15.115:0/3092831914 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.802+0000 7fa6137fe640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa61410d4b0 0x7fa6141ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.802+0000 7fa612ffd640 1 -- 172.21.15.115:0/3092831914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa61410d4b0 msgr2=0x7fa6141ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.802+0000 7fa612ffd640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa61410d4b0 0x7fa6141ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.802+0000 7fa612ffd640 1 -- 172.21.15.115:0/3092831914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6141075e0 msgr2=0x7fa61410ce40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.802+0000 7fa612ffd640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6141075e0 0x7fa61410ce40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.802+0000 7fa612ffd640 1 -- 172.21.15.115:0/3092831914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61410dc60 con 0x7fa614102590 2024-05-09T06:00:18.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.803+0000 7fa612ffd640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa614102590 0x7fa61410e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00ba50 tx=0x7fa60c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:18.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.803+0000 7fa5fbfff640 1 -- 172.21.15.115:0/3092831914 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c019070 con 0x7fa614102590 2024-05-09T06:00:18.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.804+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61419ebf0 con 0x7fa614102590 2024-05-09T06:00:18.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.804+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61419f110 con 0x7fa614102590 2024-05-09T06:00:18.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.805+0000 7fa5fbfff640 1 -- 172.21.15.115:0/3092831914 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa60c00cbe0 con 0x7fa614102590 2024-05-09T06:00:18.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.805+0000 7fa5fbfff640 1 -- 172.21.15.115:0/3092831914 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0075d0 con 0x7fa614102590 2024-05-09T06:00:18.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.805+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa614103610 con 0x7fa614102590 2024-05-09T06:00:18.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.809+0000 7fa5fbfff640 1 -- 172.21.15.115:0/3092831914 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fa60c01c070 con 0x7fa614102590 2024-05-09T06:00:18.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.810+0000 7fa5fbfff640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5fc0b2b70 0x7fa5fc0b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:18.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.810+0000 7fa6127fc640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5fc0b2b70 0x7fa5fc0b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:18.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.810+0000 7fa5fbfff640 1 -- 172.21.15.115:0/3092831914 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7fa60c0e2650 con 0x7fa614102590 2024-05-09T06:00:18.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.813+0000 7fa6127fc640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5fc0b2b70 0x7fa5fc0b5080 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa61406c060 tx=0x7fa608005fd0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:18.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:18.814+0000 7fa5fbfff640 1 -- 172.21.15.115:0/3092831914 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa60c0a9720 con 0x7fa614102590 2024-05-09T06:00:19.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.189+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 --> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa6140ffdf0 con 0x7fa5fc0b2b70 2024-05-09T06:00:19.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.209+0000 7fa5fbfff640 1 -- 172.21.15.115:0/3092831914 <== mgr.15327 v2:172.21.15.115:6800/1477729447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa6140ffdf0 con 0x7fa5fc0b2b70 2024-05-09T06:00:19.197 INFO:teuthology.orchestra.run.smithi115.stdout:/ 1m 6m3h 2024-05-09T06:00:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5fc0b2b70 msgr2=0x7fa5fc0b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5fc0b2b70 0x7fa5fc0b5080 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa61406c060 tx=0x7fa608005fd0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa614102590 msgr2=0x7fa61410e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa614102590 0x7fa61410e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00ba50 tx=0x7fa60c00bf10 comp rx=0 tx=0).stop 2024-05-09T06:00:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 shutdown_connections 2024-05-09T06:00:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fa5fc0b2b70 0x7fa5fc0b5080 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa61410d4b0 0x7fa6141ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6141075e0 0x7fa61410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 --2- 172.21.15.115:0/3092831914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa614102590 0x7fa61410e9f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 >> 172.21.15.115:0/3092831914 conn(0x7fa6140fd9d0 msgr2=0x7fa61406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 shutdown_connections 2024-05-09T06:00:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.213+0000 7fa61929a640 1 -- 172.21.15.115:0/3092831914 wait complete. 2024-05-09T06:00:19.216 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-09T06:00:19.227 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-09T06:00:19.227 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-09T06:00:19.228 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T06:00:19.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T06:00:19.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.405+0000 7f947ef27640 1 -- 172.21.15.115:0/2893329028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108b20 msgr2=0x7f94781026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.405+0000 7f947ef27640 1 --2- 172.21.15.115:0/2893329028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108b20 0x7f94781026c0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9470009900 tx=0x7f947002a410 comp rx=0 tx=0).stop 2024-05-09T06:00:19.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.405+0000 7f947ef27640 1 -- 172.21.15.115:0/2893329028 shutdown_connections 2024-05-09T06:00:19.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.405+0000 7f947ef27640 1 --2- 172.21.15.115:0/2893329028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9478103270 0x7f947810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.405+0000 7f947ef27640 1 --2- 172.21.15.115:0/2893329028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478108fc0 0x7f9478102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.405+0000 7f947ef27640 1 --2- 172.21.15.115:0/2893329028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108b20 0x7f94781026c0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.405+0000 7f947ef27640 1 -- 172.21.15.115:0/2893329028 >> 172.21.15.115:0/2893329028 conn(0x7f9478078240 msgr2=0x7f9478078680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.405+0000 7f947ef27640 1 -- 172.21.15.115:0/2893329028 shutdown_connections 2024-05-09T06:00:19.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.405+0000 7f947ef27640 1 -- 172.21.15.115:0/2893329028 wait complete. 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.406+0000 7f947ef27640 1 Processor -- start 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.406+0000 7f947ef27640 1 -- start start 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.406+0000 7f947ef27640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9478103270 0x7f94781a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.406+0000 7f947ef27640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478108b20 0x7f94781a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.406+0000 7f947ef27640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108fc0 0x7f947819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.406+0000 7f947ef27640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9478112bc0 con 0x7f9478108fc0 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.406+0000 7f947ef27640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9478112a40 con 0x7f9478108b20 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.406+0000 7f947ef27640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9478112d40 con 0x7f9478103270 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947d4c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108fc0 0x7f947819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947d4c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108fc0 0x7f947819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46478/0 (socket says 172.21.15.115:46478) 2024-05-09T06:00:19.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947d4c4640 1 -- 172.21.15.115:0/1899557479 learned_addr learned my addr 172.21.15.115:0/1899557479 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f9477fff640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478108b20 0x7f94781a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947ccc3640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9478103270 0x7f94781a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947d4c4640 1 -- 172.21.15.115:0/1899557479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9478103270 msgr2=0x7f94781a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947d4c4640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9478103270 0x7f94781a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947d4c4640 1 -- 172.21.15.115:0/1899557479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478108b20 msgr2=0x7f94781a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947d4c4640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478108b20 0x7f94781a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947d4c4640 1 -- 172.21.15.115:0/1899557479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947819cf00 con 0x7f9478108fc0 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f947d4c4640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108fc0 0x7f947819c860 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f946c00e970 tx=0x7f946c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.407+0000 7f9475ffb640 1 -- 172.21.15.115:0/1899557479 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00ccc0 con 0x7f9478108fc0 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.408+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947819d1f0 con 0x7f9478108fc0 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.408+0000 7f9475ffb640 1 -- 172.21.15.115:0/1899557479 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f946c00ce60 con 0x7f9478108fc0 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.408+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781a9450 con 0x7f9478108fc0 2024-05-09T06:00:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.408+0000 7f9475ffb640 1 -- 172.21.15.115:0/1899557479 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c010430 con 0x7f9478108fc0 2024-05-09T06:00:19.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.409+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947806b0d0 con 0x7f9478108fc0 2024-05-09T06:00:19.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.412+0000 7f9475ffb640 1 -- 172.21.15.115:0/1899557479 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f946c004ef0 con 0x7f9478108fc0 2024-05-09T06:00:19.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.413+0000 7f9475ffb640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94600b2a60 0x7f94600b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:19.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.413+0000 7f947ccc3640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94600b2a60 0x7f94600b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:19.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.413+0000 7f9475ffb640 1 -- 172.21.15.115:0/1899557479 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f946c0e3050 con 0x7f9478108fc0 2024-05-09T06:00:19.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.414+0000 7f947ccc3640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94600b2a60 0x7f94600b4f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f94700057c0 tx=0x7f9470032ba0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:19.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.418+0000 7f9475ffb640 1 -- 172.21.15.115:0/1899557479 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f946c054df0 con 0x7f9478108fc0 2024-05-09T06:00:19.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.682+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94781a96e0 con 0x7f9478108fc0 2024-05-09T06:00:19.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.686+0000 7f9475ffb640 1 -- 172.21.15.115:0/1899557479 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+25432 (secure 0 0 0) 0x7f946c0aa5e0 con 0x7f9478108fc0 2024-05-09T06:00:19.674 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T06:00:19.674 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":124,"fsid":"69de8a38-0dc4-11ef-bc97-c7b262605968","created":"2024-05-09T05:25:33.775456+0000","modified":"2024-05-09T06:00:09.493911+0000","last_up_change":"2024-05-09T05:32:50.268826+0000","last_in_change":"2024-05-09T05:32:30.467105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T05:28:58.644670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T05:33:47.793774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T05:33:51.611373+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T05:34:02.810308+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72834922-0c68-46a5-a24b-7bad43c395c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6803","nonce":2561526848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6805","nonce":2561526848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6809","nonce":2561526848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2561526848},{"type":"v1","addr":"172.21.15.115:6807","nonce":2561526848}]},"public_addr":"172.21.15.115:6803/2561526848","cluster_addr":"172.21.15.115:6805/2561526848","heartbeat_back_addr":"172.21.15.115:6809/2561526848","heartbeat_front_addr":"172.21.15.115:6807/2561526848","state":["exists","up"]},{"osd":1,"uuid":"b6b5e817-6c25-4b93-8eed-13aff270aac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6801","nonce":3600915673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6803","nonce":3600915673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6807","nonce":3600915673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3600915673},{"type":"v1","addr":"172.21.15.161:6805","nonce":3600915673}]},"public_addr":"172.21.15.161:6801/3600915673","cluster_addr":"172.21.15.161:6803/3600915673","heartbeat_back_addr":"172.21.15.161:6807/3600915673","heartbeat_front_addr":"172.21.15.161:6805/3600915673","state":["exists","up"]},{"osd":2,"uuid":"77c0b77f-2058-4131-a82c-b8f42780f350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6801","nonce":960731357}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6803","nonce":960731357}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6807","nonce":960731357}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":960731357},{"type":"v1","addr":"172.21.15.195:6805","nonce":960731357}]},"public_addr":"172.21.15.195:6801/960731357","cluster_addr":"172.21.15.195:6803/960731357","heartbeat_back_addr":"172.21.15.195:6807/960731357","heartbeat_front_addr":"172.21.15.195:6805/960731357","state":["exists","up"]},{"osd":3,"uuid":"588aa4a2-6263-494a-93e3-a9ad906119eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6811","nonce":1326312717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6813","nonce":1326312717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6817","nonce":1326312717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1326312717},{"type":"v1","addr":"172.21.15.115:6815","nonce":1326312717}]},"public_addr":"172.21.15.115:6811/1326312717","cluster_addr":"172.21.15.115:6813/1326312717","heartbeat_back_addr":"172.21.15.115:6817/1326312717","heartbeat_front_addr":"172.21.15.115:6815/1326312717","state":["exists","up"]},{"osd":4,"uuid":"70e4d345-0e86-4383-8051-be261d261fd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6809","nonce":3719283900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6811","nonce":3719283900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6815","nonce":3719283900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3719283900},{"type":"v1","addr":"172.21.15.161:6813","nonce":3719283900}]},"public_addr":"172.21.15.161:6809/3719283900","cluster_addr":"172.21.15.161:6811/3719283900","heartbeat_back_addr":"172.21.15.161:6815/3719283900","heartbeat_front_addr":"172.21.15.161:6813/3719283900","state":["exists","up"]},{"osd":5,"uuid":"8883101f-256e-4fa9-aa1f-0b5167462282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6809","nonce":3142086553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6811","nonce":3142086553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6815","nonce":3142086553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3142086553},{"type":"v1","addr":"172.21.15.195:6813","nonce":3142086553}]},"public_addr":"172.21.15.195:6809/3142086553","cluster_addr":"172.21.15.195:6811/3142086553","heartbeat_back_addr":"172.21.15.195:6815/3142086553","heartbeat_front_addr":"172.21.15.195:6813/3142086553","state":["exists","up"]},{"osd":6,"uuid":"7c3cc250-18ea-4a70-972e-3c58917ccced","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6819","nonce":3657552371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6821","nonce":3657552371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6825","nonce":3657552371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":3657552371},{"type":"v1","addr":"172.21.15.115:6823","nonce":3657552371}]},"public_addr":"172.21.15.115:6819/3657552371","cluster_addr":"172.21.15.115:6821/3657552371","heartbeat_back_addr":"172.21.15.115:6825/3657552371","heartbeat_front_addr":"172.21.15.115:6823/3657552371","state":["exists","up"]},{"osd":7,"uuid":"6c95dabd-b86f-4ca6-be1c-a783b8716848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6817","nonce":2577292327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6819","nonce":2577292327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6823","nonce":2577292327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2577292327},{"type":"v1","addr":"172.21.15.161:6821","nonce":2577292327}]},"public_addr":"172.21.15.161:6817/2577292327","cluster_addr":"172.21.15.161:6819/2577292327","heartbeat_back_addr":"172.21.15.161:6823/2577292327","heartbeat_front_addr":"172.21.15.161:6821/2577292327","state":["exists","up"]},{"osd":8,"uuid":"92757b9e-da23-40c2-920c-2ed2a247b772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6817","nonce":818250056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6819","nonce":818250056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6823","nonce":818250056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":818250056},{"type":"v1","addr":"172.21.15.195:6821","nonce":818250056}]},"public_addr":"172.21.15.195:6817/818250056","cluster_addr":"172.21.15.195:6819/818250056","heartbeat_back_addr":"172.21.15.195:6823/818250056","heartbeat_front_addr":"172.21.15.195:6821/818250056","state":["exists","up"]},{"osd":9,"uuid":"7d893599-0268-4ed6-9ea7-b74b6a623b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6827","nonce":4143175430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6829","nonce":4143175430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6833","nonce":4143175430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4143175430},{"type":"v1","addr":"172.21.15.115:6831","nonce":4143175430}]},"public_addr":"172.21.15.115:6827/4143175430","cluster_addr":"172.21.15.115:6829/4143175430","heartbeat_back_addr":"172.21.15.115:6833/4143175430","heartbeat_front_addr":"172.21.15.115:6831/4143175430","state":["exists","up"]},{"osd":10,"uuid":"777e8971-300a-4291-b81c-13794e9d8f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6825","nonce":1493802614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6827","nonce":1493802614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6831","nonce":1493802614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1493802614},{"type":"v1","addr":"172.21.15.161:6829","nonce":1493802614}]},"public_addr":"172.21.15.161:6825/1493802614","cluster_addr":"172.21.15.161:6827/1493802614","heartbeat_back_addr":"172.21.15.161:6831/1493802614","heartbeat_front_addr":"172.21.15.161:6829/1493802614","state":["exists","up"]},{"osd":11,"uuid":"debd622f-268c-4946-af2a-e687f95c4d89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6825","nonce":2489992228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6827","nonce":2489992228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6831","nonce":2489992228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2489992228},{"type":"v1","addr":"172.21.15.195:6829","nonce":2489992228}]},"public_addr":"172.21.15.195:6825/2489992228","cluster_addr":"172.21.15.195:6827/2489992228","heartbeat_back_addr":"172.21.15.195:6831/2489992228","heartbeat_front_addr":"172.21.15.195:6829/2489992228","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:07.540272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:31.973837+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:28:56.190220+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:21.169617+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:29:46.444531+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:11.547664+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:30:36.879693+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:02.659172+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:29.128462+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:31:55.522033+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:22.206103+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T05:32:49.023761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1352570099":"2024-05-10T05:38:11.918655+0000","172.21.15.115:6800/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:6800/2041699318":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/617068092":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/1072371912":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3822912073":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/1329653367":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/2041699318":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/4237548909":"2024-05-10T05:38:11.918655+0000","172.21.15.115:0/1853670702":"2024-05-10T05:38:11.918655+0000","172.21.15.115:6800/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/1212404088":"2024-05-10T05:26:06.415053+0000","172.21.15.115:6801/792641138":"2024-05-10T05:26:32.548125+0000","172.21.15.115:0/3376666830":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/1620364593":"2024-05-10T05:26:06.415053+0000","172.21.15.115:0/2102093122":"2024-05-10T05:26:32.548125+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]},{"pool":4,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]},{"pool":4,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T06:00:19.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94600b2a60 msgr2=0x7f94600b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94600b2a60 0x7f94600b4f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f94700057c0 tx=0x7f9470032ba0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108fc0 msgr2=0x7f947819c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108fc0 0x7f947819c860 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f946c00e970 tx=0x7f946c00ed70 comp rx=0 tx=0).stop 2024-05-09T06:00:19.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 shutdown_connections 2024-05-09T06:00:19.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f94600b2a60 0x7f94600b4f70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9478108fc0 0x7f947819c860 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478108b20 0x7f94781a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 --2- 172.21.15.115:0/1899557479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9478103270 0x7f94781a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 >> 172.21.15.115:0/1899557479 conn(0x7f9478078240 msgr2=0x7f94781012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:19.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.690+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 shutdown_connections 2024-05-09T06:00:19.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.691+0000 7f947ef27640 1 -- 172.21.15.115:0/1899557479 wait complete. 2024-05-09T06:00:19.696 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T05:28:58.644670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T05:33:47.793774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T05:33:51.611373+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T05:34:02.810308+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T06:00:19.697 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T06:00:19.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:19 smithi195 ceph-mon[33844]: pgmap v702: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 16 KiB/s wr, 67 op/s 2024-05-09T06:00:19.860 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:19 smithi195 ceph-mon[33844]: from='client.28514 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:00:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.882+0000 7fbbc5829640 1 -- 172.21.15.115:0/1557285212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc010e1b0 msgr2=0x7fbbc01106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.882+0000 7fbbc5829640 1 --2- 172.21.15.115:0/1557285212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc010e1b0 0x7fbbc01106a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009900 tx=0x7fbbb402a3f0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.882+0000 7fbbc5829640 1 -- 172.21.15.115:0/1557285212 shutdown_connections 2024-05-09T06:00:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.882+0000 7fbbc5829640 1 --2- 172.21.15.115:0/1557285212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc010e1b0 0x7fbbc01106a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.882+0000 7fbbc5829640 1 --2- 172.21.15.115:0/1557285212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc010b7c0 0x7fbbc010dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.882+0000 7fbbc5829640 1 --2- 172.21.15.115:0/1557285212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc0104d60 0x7fbbc0105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.882+0000 7fbbc5829640 1 -- 172.21.15.115:0/1557285212 >> 172.21.15.115:0/1557285212 conn(0x7fbbc0078240 msgr2=0x7fbbc0078680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.883+0000 7fbbc5829640 1 -- 172.21.15.115:0/1557285212 shutdown_connections 2024-05-09T06:00:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.883+0000 7fbbc5829640 1 -- 172.21.15.115:0/1557285212 wait complete. 2024-05-09T06:00:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.884+0000 7fbbc5829640 1 Processor -- start 2024-05-09T06:00:19.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.884+0000 7fbbc5829640 1 -- start start 2024-05-09T06:00:19.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.884+0000 7fbbc5829640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc0104d60 0x7fbbc0074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.884+0000 7fbbc5829640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc010b7c0 0x7fbbc010a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.884+0000 7fbbc5829640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc010e1b0 0x7fbbc010ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.884+0000 7fbbbeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc0104d60 0x7fbbc0074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.884+0000 7fbbbeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc0104d60 0x7fbbc0074560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:60792/0 (socket says 172.21.15.115:60792) 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.884+0000 7fbbbe7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc010b7c0 0x7fbbc010a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.885+0000 7fbbbe7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc010b7c0 0x7fbbc010a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.115:46342/0 (socket says 172.21.15.115:46342) 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.885+0000 7fbbbf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc010e1b0 0x7fbbc010ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.885+0000 7fbbbf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc010e1b0 0x7fbbc010ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46504/0 (socket says 172.21.15.115:46504) 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.885+0000 7fbbbeffd640 1 -- 172.21.15.115:0/3005671086 learned_addr learned my addr 172.21.15.115:0/3005671086 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.885+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc006ce10 con 0x7fbbc010e1b0 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.885+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc006cc90 con 0x7fbbc0104d60 2024-05-09T06:00:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.885+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc006cf90 con 0x7fbbc010b7c0 2024-05-09T06:00:19.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbbeffd640 1 -- 172.21.15.115:0/3005671086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc010b7c0 msgr2=0x7fbbc010a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbbeffd640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc010b7c0 0x7fbbc010a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbbeffd640 1 -- 172.21.15.115:0/3005671086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc010e1b0 msgr2=0x7fbbc010ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:19.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbbeffd640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc010e1b0 0x7fbbc010ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:19.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbbeffd640 1 -- 172.21.15.115:0/3005671086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc010b1d0 con 0x7fbbc0104d60 2024-05-09T06:00:19.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbbeffd640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc0104d60 0x7fbbc0074560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c920 tx=0x7fbbb000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:19.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbc4827640 1 -- 172.21.15.115:0/3005671086 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0007c70 con 0x7fbbc0104d60 2024-05-09T06:00:19.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc010b460 con 0x7fbbc0104d60 2024-05-09T06:00:19.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc01ad510 con 0x7fbbc0104d60 2024-05-09T06:00:19.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.886+0000 7fbbc4827640 1 -- 172.21.15.115:0/3005671086 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbb0007e10 con 0x7fbbc0104d60 2024-05-09T06:00:19.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.887+0000 7fbbc4827640 1 -- 172.21.15.115:0/3005671086 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb00186f0 con 0x7fbbc0104d60 2024-05-09T06:00:19.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.888+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbac005290 con 0x7fbbc0104d60 2024-05-09T06:00:19.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.891+0000 7fbbc4827640 1 -- 172.21.15.115:0/3005671086 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7fbbb0018890 con 0x7fbbc0104d60 2024-05-09T06:00:19.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.892+0000 7fbbc4827640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbba80b2a60 0x7fbba80b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:19.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.892+0000 7fbbc4827640 1 -- 172.21.15.115:0/3005671086 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7fbbb00e2cb0 con 0x7fbbc0104d60 2024-05-09T06:00:19.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.892+0000 7fbbbe7fc640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbba80b2a60 0x7fbba80b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:19.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.893+0000 7fbbbe7fc640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbba80b2a60 0x7fbba80b4f70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbbc006dd10 tx=0x7fbbb8009680 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:19.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:19.894+0000 7fbbc4827640 1 -- 172.21.15.115:0/3005671086 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbbb0014030 con 0x7fbbc0104d60 2024-05-09T06:00:19.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:19 smithi115 ceph-mon[30349]: pgmap v702: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 16 KiB/s wr, 67 op/s 2024-05-09T06:00:19.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:19 smithi115 ceph-mon[30349]: from='client.28514 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:00:19.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:19 smithi161 ceph-mon[33805]: pgmap v702: 129 pgs: 18 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 58 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 16 KiB/s wr, 67 op/s 2024-05-09T06:00:19.906 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:19 smithi161 ceph-mon[33805]: from='client.28514 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:00:20.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.222+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbbac005850 con 0x7fbbc0104d60 2024-05-09T06:00:20.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.223+0000 7fbbc4827640 1 -- 172.21.15.115:0/3005671086 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) ==== 93+0+10 (secure 0 0 0) 0x7fbbb00aa350 con 0x7fbbc0104d60 2024-05-09T06:00:20.210 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-05-09T06:00:20.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbba80b2a60 msgr2=0x7fbba80b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbba80b2a60 0x7fbba80b4f70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbbc006dd10 tx=0x7fbbb8009680 comp rx=0 tx=0).stop 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc0104d60 msgr2=0x7fbbc0074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc0104d60 0x7fbbc0074560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c920 tx=0x7fbbb000cde0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 shutdown_connections 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7fbba80b2a60 0x7fbba80b4f70 secure :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbbc006dd10 tx=0x7fbbb8009680 comp rx=0 tx=0).stop 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbc010e1b0 0x7fbbc010ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbc010b7c0 0x7fbbc010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 --2- 172.21.15.115:0/3005671086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc0104d60 0x7fbbc0074560 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 >> 172.21.15.115:0/3005671086 conn(0x7fbbc0078240 msgr2=0x7fbbc0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 shutdown_connections 2024-05-09T06:00:20.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.226+0000 7fbbc5829640 1 -- 172.21.15.115:0/3005671086 wait complete. 2024-05-09T06:00:20.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T06:00:20.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 -- 172.21.15.115:0/1850319496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5468104d80 msgr2=0x7f5468105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 --2- 172.21.15.115:0/1850319496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5468104d80 0x7f5468105190 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f5458009900 tx=0x7f545802a510 comp rx=0 tx=0).stop 2024-05-09T06:00:20.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 -- 172.21.15.115:0/1850319496 shutdown_connections 2024-05-09T06:00:20.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 --2- 172.21.15.115:0/1850319496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f546810e190 0x7f5468110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 --2- 172.21.15.115:0/1850319496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f546810b7e0 0x7f546810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 --2- 172.21.15.115:0/1850319496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5468104d80 0x7f5468105190 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 -- 172.21.15.115:0/1850319496 >> 172.21.15.115:0/1850319496 conn(0x7f5468078230 msgr2=0x7f5468078670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 -- 172.21.15.115:0/1850319496 shutdown_connections 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 -- 172.21.15.115:0/1850319496 wait complete. 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 Processor -- start 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.388+0000 7f546e34d640 1 -- start start 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f546e34d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5468104d80 0x7f546819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f546e34d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810b7e0 0x7f546819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f546e34d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f546810e190 0x7f546819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f546e34d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5468113ca0 con 0x7f546810b7e0 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f546e34d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5468113b20 con 0x7f546810e190 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f546e34d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5468113e20 con 0x7f5468104d80 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f545ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810b7e0 0x7f546819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f545ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810b7e0 0x7f546819ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46518/0 (socket says 172.21.15.115:46518) 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f545ffff640 1 -- 172.21.15.115:0/870159446 learned_addr learned my addr 172.21.15.115:0/870159446 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f546c8ea640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f546810e190 0x7f546819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:20.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f5467fff640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5468104d80 0x7f546819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f545ffff640 1 -- 172.21.15.115:0/870159446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5468104d80 msgr2=0x7f546819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f545ffff640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5468104d80 0x7f546819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f545ffff640 1 -- 172.21.15.115:0/870159446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f546810e190 msgr2=0x7f546819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f545ffff640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f546810e190 0x7f546819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f545ffff640 1 -- 172.21.15.115:0/870159446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546819f5e0 con 0x7f546810b7e0 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f545ffff640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810b7e0 0x7f546819ace0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f545000d770 tx=0x7f545000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f5465ffb640 1 -- 172.21.15.115:0/870159446 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450017070 con 0x7f546810b7e0 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f5465ffb640 1 -- 172.21.15.115:0/870159446 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54500054b0 con 0x7f546810b7e0 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f5465ffb640 1 -- 172.21.15.115:0/870159446 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450004c60 con 0x7f546810b7e0 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546819f8d0 con 0x7f546810b7e0 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.389+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546819fdd0 con 0x7f546810b7e0 2024-05-09T06:00:20.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.390+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546806b0d0 con 0x7f546810b7e0 2024-05-09T06:00:20.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.392+0000 7f5465ffb640 1 -- 172.21.15.115:0/870159446 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f5450004e00 con 0x7f546810b7e0 2024-05-09T06:00:20.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.393+0000 7f5465ffb640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54540b2960 0x7f54540b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:20.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.393+0000 7f5467fff640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54540b2960 0x7f54540b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:20.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.393+0000 7f5465ffb640 1 -- 172.21.15.115:0/870159446 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f54500e27b0 con 0x7f546810b7e0 2024-05-09T06:00:20.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.393+0000 7f5467fff640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54540b2960 0x7f54540b4e70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54580057c0 tx=0x7f54580058d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:20.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.396+0000 7f5465ffb640 1 -- 172.21.15.115:0/870159446 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f546819fdd0 con 0x7f546810b7e0 2024-05-09T06:00:20.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.614+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f54681a7340 con 0x7f546810b7e0 2024-05-09T06:00:20.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.614+0000 7f5465ffb640 1 -- 172.21.15.115:0/870159446 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) ==== 104+0+11 (secure 0 0 0) 0x7f54500a9750 con 0x7f546810b7e0 2024-05-09T06:00:20.602 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-05-09T06:00:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54540b2960 msgr2=0x7f54540b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54540b2960 0x7f54540b4e70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54580057c0 tx=0x7f54580058d0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810b7e0 msgr2=0x7f546819ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810b7e0 0x7f546819ace0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f545000d770 tx=0x7f545000dc30 comp rx=0 tx=0).stop 2024-05-09T06:00:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 shutdown_connections 2024-05-09T06:00:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f54540b2960 0x7f54540b4e70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f546810e190 0x7f546819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f546810b7e0 0x7f546819ace0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 --2- 172.21.15.115:0/870159446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5468104d80 0x7f546819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 >> 172.21.15.115:0/870159446 conn(0x7f5468078230 msgr2=0x7f54681012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 shutdown_connections 2024-05-09T06:00:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.616+0000 7f546e34d640 1 -- 172.21.15.115:0/870159446 wait complete. 2024-05-09T06:00:20.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T06:00:20.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.752+0000 7f68ae572640 1 -- 172.21.15.115:0/3431991240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810e1b0 msgr2=0x7f68a81106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.752+0000 7f68ae572640 1 --2- 172.21.15.115:0/3431991240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810e1b0 0x7f68a81106a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009ca0 tx=0x7f68a002a8c0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.752+0000 7f68ae572640 1 -- 172.21.15.115:0/3431991240 shutdown_connections 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.752+0000 7f68ae572640 1 --2- 172.21.15.115:0/3431991240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810e1b0 0x7f68a81106a0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.752+0000 7f68ae572640 1 --2- 172.21.15.115:0/3431991240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68a810b7c0 0x7f68a810dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.752+0000 7f68ae572640 1 --2- 172.21.15.115:0/3431991240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68a8104d60 0x7f68a8105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.752+0000 7f68ae572640 1 -- 172.21.15.115:0/3431991240 >> 172.21.15.115:0/3431991240 conn(0x7f68a8078240 msgr2=0x7f68a8078680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.752+0000 7f68ae572640 1 -- 172.21.15.115:0/3431991240 shutdown_connections 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.752+0000 7f68ae572640 1 -- 172.21.15.115:0/3431991240 wait complete. 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.753+0000 7f68ae572640 1 Processor -- start 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.753+0000 7f68ae572640 1 -- start start 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.753+0000 7f68ae572640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68a8104d60 0x7f68a819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:20.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.753+0000 7f68ae572640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810b7c0 0x7f68a819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.753+0000 7f68ae572640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68a810e1b0 0x7f68a819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.753+0000 7f68ae572640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8113d10 con 0x7f68a810b7c0 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.753+0000 7f68ae572640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8113b90 con 0x7f68a8104d60 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.753+0000 7f68ae572640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8113e90 con 0x7f68a810e1b0 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810b7c0 0x7f68a819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810b7c0 0x7f68a819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46534/0 (socket says 172.21.15.115:46534) 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a77fe640 1 -- 172.21.15.115:0/2114336136 learned_addr learned my addr 172.21.15.115:0/2114336136 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a7fff640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68a8104d60 0x7f68a819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68acb0f640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68a810e1b0 0x7f68a819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a77fe640 1 -- 172.21.15.115:0/2114336136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68a810e1b0 msgr2=0x7f68a819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a77fe640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68a810e1b0 0x7f68a819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a77fe640 1 -- 172.21.15.115:0/2114336136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68a8104d60 msgr2=0x7f68a819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a77fe640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68a8104d60 0x7f68a819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a77fe640 1 -- 172.21.15.115:0/2114336136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a819f550 con 0x7f68a810b7c0 2024-05-09T06:00:20.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68acb0f640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68a810e1b0 0x7f68a819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:00:20.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a7fff640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68a8104d60 0x7f68a819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:00:20.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a77fe640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810b7c0 0x7f68a819ac50 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e9f0 tx=0x7f689c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:20.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a57fa640 1 -- 172.21.15.115:0/2114336136 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c00cd80 con 0x7f68a810b7c0 2024-05-09T06:00:20.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a57fa640 1 -- 172.21.15.115:0/2114336136 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f689c0108c0 con 0x7f68a810b7c0 2024-05-09T06:00:20.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a819f840 con 0x7f68a810b7c0 2024-05-09T06:00:20.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68a57fa640 1 -- 172.21.15.115:0/2114336136 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c0187c0 con 0x7f68a810b7c0 2024-05-09T06:00:20.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.754+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a81a71c0 con 0x7f68a810b7c0 2024-05-09T06:00:20.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.755+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a8069410 con 0x7f68a810b7c0 2024-05-09T06:00:20.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.757+0000 7f68a57fa640 1 -- 172.21.15.115:0/2114336136 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f689c010430 con 0x7f68a810b7c0 2024-05-09T06:00:20.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.758+0000 7f68a57fa640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68900b29e0 0x7f68900b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:20.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.758+0000 7f68a7fff640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68900b29e0 0x7f68900b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:20.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.758+0000 7f68a57fa640 1 -- 172.21.15.115:0/2114336136 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f689c0e2d10 con 0x7f68a810b7c0 2024-05-09T06:00:20.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.758+0000 7f68a7fff640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68900b29e0 0x7f68900b4ef0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6898009fc0 tx=0x7f6898005f90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:20.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.763+0000 7f68a57fa640 1 -- 172.21.15.115:0/2114336136 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f689c0aa300 con 0x7f68a810b7c0 2024-05-09T06:00:20.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:20 smithi195 ceph-mon[33844]: from='client.28517 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:00:20.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:20 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1899557479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:00:20.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:20 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3005671086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:00:20.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:20 smithi115 ceph-mon[30349]: from='client.28517 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:00:20.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:20 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1899557479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:00:20.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:20 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3005671086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:00:20.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:20 smithi161 ceph-mon[33805]: from='client.28517 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T06:00:20.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:20 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1899557479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T06:00:20.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:20 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3005671086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T06:00:20.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.954+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f68a81a7390 con 0x7f68a810b7c0 2024-05-09T06:00:20.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.954+0000 7f68a57fa640 1 -- 172.21.15.115:0/2114336136 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) ==== 100+0+11 (secure 0 0 0) 0x7f689c0a9c00 con 0x7f68a810b7c0 2024-05-09T06:00:20.942 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2024-05-09T06:00:20.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.956+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68900b29e0 msgr2=0x7f68900b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.956+0000 7f68ae572640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68900b29e0 0x7f68900b4ef0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6898009fc0 tx=0x7f6898005f90 comp rx=0 tx=0).stop 2024-05-09T06:00:20.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.956+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810b7c0 msgr2=0x7f68a819ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:20.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.956+0000 7f68ae572640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810b7c0 0x7f68a819ac50 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e9f0 tx=0x7f689c00eeb0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.956+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 shutdown_connections 2024-05-09T06:00:20.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.956+0000 7f68ae572640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f68900b29e0 0x7f68900b4ef0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.956+0000 7f68ae572640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68a810e1b0 0x7f68a819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.957+0000 7f68ae572640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68a810b7c0 0x7f68a819ac50 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.957+0000 7f68ae572640 1 --2- 172.21.15.115:0/2114336136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68a8104d60 0x7f68a819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:20.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.957+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 >> 172.21.15.115:0/2114336136 conn(0x7f68a8078240 msgr2=0x7f68a8101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:20.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.957+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 shutdown_connections 2024-05-09T06:00:20.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:20.957+0000 7f68ae572640 1 -- 172.21.15.115:0/2114336136 wait complete. 2024-05-09T06:00:20.958 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T06:00:21.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 -- 172.21.15.115:0/771777702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f618410e6a0 msgr2=0x7f6184110ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:21.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 --2- 172.21.15.115:0/771777702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f618410e6a0 0x7f6184110ac0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f617c009ca0 tx=0x7f617c02a8c0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 -- 172.21.15.115:0/771777702 shutdown_connections 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 --2- 172.21.15.115:0/771777702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f618410e6a0 0x7f6184110ac0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 --2- 172.21.15.115:0/771777702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61841021a0 0x7f61841025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 --2- 172.21.15.115:0/771777702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6184109cf0 0x7f6184101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 -- 172.21.15.115:0/771777702 >> 172.21.15.115:0/771777702 conn(0x7f6184077a50 msgr2=0x7f6184077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 -- 172.21.15.115:0/771777702 shutdown_connections 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 -- 172.21.15.115:0/771777702 wait complete. 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 Processor -- start 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.099+0000 7f618a3e0640 1 -- start start 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f618a3e0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61841021a0 0x7f618419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f618a3e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6184109cf0 0x7f618419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f618a3e0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f618410e6a0 0x7f618419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f618a3e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6184113d60 con 0x7f6184109cf0 2024-05-09T06:00:21.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f618a3e0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6184113be0 con 0x7f61841021a0 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f618a3e0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6184113ee0 con 0x7f618410e6a0 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f617bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6184109cf0 0x7f618419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f617bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6184109cf0 0x7f618419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46542/0 (socket says 172.21.15.115:46542) 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f617bfff640 1 -- 172.21.15.115:0/3541244360 learned_addr learned my addr 172.21.15.115:0/3541244360 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f6183fff640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61841021a0 0x7f618419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f618897d640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f618410e6a0 0x7f618419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f617bfff640 1 -- 172.21.15.115:0/3541244360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f618410e6a0 msgr2=0x7f618419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f617bfff640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f618410e6a0 0x7f618419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f617bfff640 1 -- 172.21.15.115:0/3541244360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61841021a0 msgr2=0x7f618419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f617bfff640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61841021a0 0x7f618419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f617bfff640 1 -- 172.21.15.115:0/3541244360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618419f580 con 0x7f6184109cf0 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.100+0000 7f617bfff640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6184109cf0 0x7f618419ac80 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f616c00e9f0 tx=0x7f616c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.101+0000 7f6181ffb640 1 -- 172.21.15.115:0/3541244360 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c00ccc0 con 0x7f6184109cf0 2024-05-09T06:00:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.101+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618419f870 con 0x7f6184109cf0 2024-05-09T06:00:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.101+0000 7f6181ffb640 1 -- 172.21.15.115:0/3541244360 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f616c0054b0 con 0x7f6184109cf0 2024-05-09T06:00:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.101+0000 7f6181ffb640 1 -- 172.21.15.115:0/3541244360 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c010430 con 0x7f6184109cf0 2024-05-09T06:00:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.101+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841a71f0 con 0x7f6184109cf0 2024-05-09T06:00:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.102+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618406a290 con 0x7f6184109cf0 2024-05-09T06:00:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.103+0000 7f6181ffb640 1 -- 172.21.15.115:0/3541244360 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f616c0046c0 con 0x7f6184109cf0 2024-05-09T06:00:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.104+0000 7f6181ffb640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f61700b29e0 0x7f61700b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:21.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.104+0000 7f6183fff640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f61700b29e0 0x7f61700b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:21.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.104+0000 7f6181ffb640 1 -- 172.21.15.115:0/3541244360 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f616c0e2ed0 con 0x7f6184109cf0 2024-05-09T06:00:21.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.105+0000 7f6183fff640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f61700b29e0 0x7f61700b4ef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6184075650 tx=0x7f6174005f90 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:21.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.111+0000 7f6181ffb640 1 -- 172.21.15.115:0/3541244360 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f616c054df0 con 0x7f6184109cf0 2024-05-09T06:00:21.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.299+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6184108170 con 0x7f6184109cf0 2024-05-09T06:00:21.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.299+0000 7f6181ffb640 1 -- 172.21.15.115:0/3541244360 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v124) ==== 103+0+11 (secure 0 0 0) 0x7f616c0aa4c0 con 0x7f6184109cf0 2024-05-09T06:00:21.288 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2024-05-09T06:00:21.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.301+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f61700b29e0 msgr2=0x7f61700b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:21.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.301+0000 7f618a3e0640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f61700b29e0 0x7f61700b4ef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6184075650 tx=0x7f6174005f90 comp rx=0 tx=0).stop 2024-05-09T06:00:21.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.301+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6184109cf0 msgr2=0x7f618419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:21.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.301+0000 7f618a3e0640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6184109cf0 0x7f618419ac80 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f616c00e9f0 tx=0x7f616c00eeb0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.302+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 shutdown_connections 2024-05-09T06:00:21.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.302+0000 7f618a3e0640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f61700b29e0 0x7f61700b4ef0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.302+0000 7f618a3e0640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f618410e6a0 0x7f618419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.302+0000 7f618a3e0640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6184109cf0 0x7f618419ac80 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.302+0000 7f618a3e0640 1 --2- 172.21.15.115:0/3541244360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61841021a0 0x7f618419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.302+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 >> 172.21.15.115:0/3541244360 conn(0x7f6184077a50 msgr2=0x7f6184105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:21.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.302+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 shutdown_connections 2024-05-09T06:00:21.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.302+0000 7f618a3e0640 1 -- 172.21.15.115:0/3541244360 wait complete. 2024-05-09T06:00:21.301 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 -- 172.21.15.115:0/1519574877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 msgr2=0x7f85ec1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 --2- 172.21.15.115:0/1519574877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 0x7f85ec1038f0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009900 tx=0x7f85e802a410 comp rx=0 tx=0).stop 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 -- 172.21.15.115:0/1519574877 shutdown_connections 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 --2- 172.21.15.115:0/1519574877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85ec107e20 0x7f85ec10f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 --2- 172.21.15.115:0/1519574877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec1073a0 0x7f85ec1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 --2- 172.21.15.115:0/1519574877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 0x7f85ec1038f0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 -- 172.21.15.115:0/1519574877 >> 172.21.15.115:0/1519574877 conn(0x7f85ec0fd980 msgr2=0x7f85ec0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 -- 172.21.15.115:0/1519574877 shutdown_connections 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 -- 172.21.15.115:0/1519574877 wait complete. 2024-05-09T06:00:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 Processor -- start 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.429+0000 7f85f429f640 1 -- start start 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f429f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 0x7f85ec19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f429f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec1073a0 0x7f85ec19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f429f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85ec107e20 0x7f85ec19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f429f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec112bd0 con 0x7f85ec1034e0 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f429f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec112a50 con 0x7f85ec1073a0 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f429f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec112d50 con 0x7f85ec107e20 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f203b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 0x7f85ec19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f203b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 0x7f85ec19b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46568/0 (socket says 172.21.15.115:46568) 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f203b640 1 -- 172.21.15.115:0/2198706175 learned_addr learned my addr 172.21.15.115:0/2198706175 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f283c640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85ec107e20 0x7f85ec19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f183a640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec1073a0 0x7f85ec19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f203b640 1 -- 172.21.15.115:0/2198706175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85ec107e20 msgr2=0x7f85ec19c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f203b640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85ec107e20 0x7f85ec19c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f203b640 1 -- 172.21.15.115:0/2198706175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec1073a0 msgr2=0x7f85ec19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f203b640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec1073a0 0x7f85ec19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f203b640 1 -- 172.21.15.115:0/2198706175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec1a68a0 con 0x7f85ec1034e0 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f183a640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec1073a0 0x7f85ec19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85f203b640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 0x7f85ec19b590 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f85e8005940 tx=0x7f85e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.430+0000 7f85e37fe640 1 -- 172.21.15.115:0/2198706175 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e803f030 con 0x7f85ec1034e0 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.431+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec1a6b30 con 0x7f85ec1034e0 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.431+0000 7f85e37fe640 1 -- 172.21.15.115:0/2198706175 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85e8003c90 con 0x7f85ec1034e0 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.431+0000 7f85e37fe640 1 -- 172.21.15.115:0/2198706175 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8035070 con 0x7f85ec1034e0 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.431+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec1a7020 con 0x7f85ec1034e0 2024-05-09T06:00:21.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.431+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ec06a2c0 con 0x7f85ec1034e0 2024-05-09T06:00:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.433+0000 7f85e37fe640 1 -- 172.21.15.115:0/2198706175 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f85e8042030 con 0x7f85ec1034e0 2024-05-09T06:00:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.434+0000 7f85e37fe640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85d80b2960 0x7f85d80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:00:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.434+0000 7f85f183a640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85d80b2960 0x7f85d80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:00:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.434+0000 7f85e37fe640 1 -- 172.21.15.115:0/2198706175 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f85e8103300 con 0x7f85ec1034e0 2024-05-09T06:00:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.434+0000 7f85f183a640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85d80b2960 0x7f85d80b4e70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85dc005d00 tx=0x7f85dc005c70 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:00:21.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.437+0000 7f85e37fe640 1 -- 172.21.15.115:0/2198706175 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85e80ca8f0 con 0x7f85ec1034e0 2024-05-09T06:00:21.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:21 smithi115 ceph-mon[30349]: pgmap v703: 129 pgs: 9 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 188 B/s wr, 27 op/s 2024-05-09T06:00:21.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:21 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/870159446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T06:00:21.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:21 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2114336136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T06:00:21.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:21 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/3541244360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T06:00:21.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.671+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f85ec10a6f0 con 0x7f85ec1034e0 2024-05-09T06:00:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.680+0000 7f85e37fe640 1 -- 172.21.15.115:0/2198706175 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+816184 (secure 0 0 0) 0x7f85e80ca1f0 con 0x7f85ec1034e0 2024-05-09T06:00:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-09T06:00:21.678 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T06:00:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 25, 2024-05-09T06:00:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "active_gid": 15327, 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "active_name": "x", 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "active_addrs": { 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6800", 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1477729447 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6801", 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1477729447 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "active_addr": "172.21.15.115:6801/1477729447", 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "active_change": "2024-05-09T05:38:11.918915+0000", 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "available": true, 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "standbys": [ 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "gid": 24929, 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "y", 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_features": 4540701547738038271, 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "available_modules": [ 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "alerts", 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_destination": { 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_destination", 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Email address to send alerts to", 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_from_name": { 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_from_name", 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "Ceph", 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Email From: name", 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_host": { 2024-05-09T06:00:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_host", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SMTP server", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_password": { 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_password", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Password to authenticate with", 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_port": { 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_port", 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "465", 2024-05-09T06:00:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SMTP port", 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_sender": { 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_sender", 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SMTP envelope sender", 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_ssl": { 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_ssl", 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_user": { 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_user", 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "User to authenticate as", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "balancer", 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "active": { 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "active", 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "begin_time": { 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "begin_time", 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0000", 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "begin_weekday": { 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "begin_weekday", 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "0", 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "6", 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_compat_max_iterations": { 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crush_compat_max_iterations", 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "25", 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "250", 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_compat_metrics": { 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crush_compat_metrics", 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_compat_step": { 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crush_compat_step", 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0.5", 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "0.001", 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "0.999", 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "aggressiveness of optimization", 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "end_time": { 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "end_time", 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "2359", 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "end_weekday": { 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "end_weekday", 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "0", 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "6", 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "min_score": { 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "min_score", 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": { 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mode", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "upmap", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "crush-compat", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "none", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "read", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap", 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap-read" 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Balancer mode", 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_ids": { 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pool_ids", 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap_max_deviation": { 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "upmap_max_deviation", 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap_max_optimizations": { 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "upmap_max_optimizations", 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephadm", 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "agent_down_multiplier": { 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "agent_down_multiplier", 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "3.0", 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "agent_refresh_rate": { 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "agent_refresh_rate", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "20", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "agent_starting_port": { 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "agent_starting_port", 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "4721", 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_ptrace": { 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "allow_ptrace", 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "autotune_interval": { 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "autotune_interval", 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "autotune_memory_target_ratio": { 2024-05-09T06:00:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0.7", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm_log_destination": { 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephadm_log_destination", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "file", 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "file,syslog", 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "syslog" 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "cgroups_split": { 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cgroups_split", 2024-05-09T06:00:21.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "config_checks_enabled": { 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "config_checks_enabled", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.721 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "config_dashboard": { 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "config_dashboard", 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.722 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_alertmanager": { 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_alertmanager", 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T06:00:21.723 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_base": { 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_base", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.724 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Container image name, without the tag", 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_elasticsearch": { 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_elasticsearch", 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.725 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "elasticsearch container image", 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_grafana": { 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_grafana", 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:21.726 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_haproxy": { 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_haproxy", 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "HAproxy container image", 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_jaeger_agent": { 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_jaeger_agent", 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Jaeger agent container image", 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.728 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_jaeger_collector": { 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_jaeger_collector", 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Jaeger collector container image", 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_jaeger_query": { 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_jaeger_query", 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Jaeger query container image", 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_keepalived": { 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_keepalived", 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Keepalived container image", 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_loki": { 2024-05-09T06:00:21.731 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_loki", 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Loki container image", 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_node_exporter": { 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_node_exporter", 2024-05-09T06:00:21.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_nvmeof": { 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_nvmeof", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.733 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Nvme-of container image", 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_prometheus": { 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_prometheus", 2024-05-09T06:00:21.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_promtail": { 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_promtail", 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.735 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Promtail container image", 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_samba": { 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_samba", 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.736 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Samba/SMB container image", 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_snmp_gateway": { 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_snmp_gateway", 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SNMP Gateway container image", 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "container_init": { 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_init", 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.738 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "daemon_cache_timeout": { 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "daemon_cache_timeout", 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:21.739 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "default_cephadm_command_timeout": { 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "900", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.740 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "default_registry": { 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "default_registry", 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io", 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T06:00:21.741 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "device_cache_timeout": { 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "device_cache_timeout", 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "1800", 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "seconds to cache device inventory", 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.742 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "device_enhanced_scan": { 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "device_enhanced_scan", 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.743 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "facts_cache_timeout": { 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "facts_cache_timeout", 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "seconds to cache host facts data", 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.744 INFO:teuthology.orchestra.run.smithi115.stdout: "grafana_dashboards_path": { 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "grafana_dashboards_path", 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.745 INFO:teuthology.orchestra.run.smithi115.stdout: "host_check_interval": { 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "host_check_interval", 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to perform a host check", 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "hw_monitoring": { 2024-05-09T06:00:21.746 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "hw_monitoring", 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "inventory_list_all": { 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "inventory_list_all", 2024-05-09T06:00:21.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.749 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "log_refresh_metadata": { 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_refresh_metadata", 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T06:00:21.750 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.751 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.752 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.753 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.754 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "*", 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "max_count_per_host": { 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_count_per_host", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "max number of daemons per service per host", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.756 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "max_osd_draining_count": { 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_osd_draining_count", 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "migration_current": { 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "migration_current", 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "internal - do not modify", 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": { 2024-05-09T06:00:21.758 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mode", 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "root", 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm-package", 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "root" 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.759 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "oob_default_addr": { 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "oob_default_addr", 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "169.254.1.1", 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "prometheus_alerts_path": { 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "prometheus_alerts_path", 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_insecure": { 2024-05-09T06:00:21.761 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_insecure", 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_password": { 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_password", 2024-05-09T06:00:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_url": { 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_url", 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.763 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_username": { 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_username", 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "secure_monitoring_stack": { 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "secure_monitoring_stack", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.765 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "service_discovery_port": { 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "service_discovery_port", 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8765", 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "cephadm service discovery port", 2024-05-09T06:00:21.766 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.767 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "ssh_config_file": { 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssh_config_file", 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "use_agent": { 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "use_agent", 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "use_repo_digest": { 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "use_repo_digest", 2024-05-09T06:00:21.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_on_failed_host_check": { 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_on_failed_host_check", 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.770 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_on_stray_daemons": { 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_on_stray_daemons", 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.771 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_on_stray_hosts": { 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_on_stray_hosts", 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.772 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.773 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crash", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.774 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.775 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.776 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.777 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "retain_interval": { 2024-05-09T06:00:21.778 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "retain_interval", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "31536000", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.779 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_recent_interval": { 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_recent_interval", 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "1209600", 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "dashboard", 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.781 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.783 INFO:teuthology.orchestra.run.smithi115.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: "AUDIT_API_ENABLED": { 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T06:00:21.785 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T06:00:21.786 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T06:00:21.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.789 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.790 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.791 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.792 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.793 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "admin", 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.795 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.796 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_URL": { 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_URL", 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.797 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "admin", 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T06:00:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T06:00:21.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.800 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.801 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "[]", 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.802 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "{}", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.803 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.805 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T06:00:21.807 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T06:00:21.808 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T06:00:21.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T06:00:21.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.812 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.813 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.814 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8", 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.815 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "45", 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.817 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "admin", 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.818 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T06:00:21.819 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T06:00:21.820 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T06:00:21.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T06:00:21.822 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.823 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "cross_origin_url": { 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cross_origin_url", 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.824 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "crt_file": { 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crt_file", 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.825 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "debug": { 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "debug", 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.826 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Enable/disable debug options", 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "jwt_token_ttl": { 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "jwt_token_ttl", 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "28800", 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.827 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "key_file": { 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "key_file", 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.828 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.829 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.832 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.833 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "motd": { 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "motd", 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "The message of the day", 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "redirect_resolve_ip_addr": { 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "server_addr": { 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_addr", 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "::", 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.836 INFO:teuthology.orchestra.run.smithi115.stdout: "server_port": { 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_port", 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8080", 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.837 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "ssl": { 2024-05-09T06:00:21.838 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssl", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "ssl_server_port": { 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssl_server_port", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.839 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8443", 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_behaviour": { 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_behaviour", 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.840 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "redirect", 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "redirect" 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_error_status_code": { 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_error_status_code", 2024-05-09T06:00:21.841 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "500", 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "400", 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "599", 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "url_prefix": { 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "url_prefix", 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.842 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "devicehealth", 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.843 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "enable_monitoring": { 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enable_monitoring", 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "monitor device health metrics", 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.845 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.846 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.847 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.848 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "mark_out_threshold": { 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mark_out_threshold", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.849 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "2419200", 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": { 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pool_name", 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.850 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "device_health_metrics", 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "retention_period": { 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "retention_period", 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "15552000", 2024-05-09T06:00:21.851 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how long to retain device health metrics", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "scrape_frequency": { 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "scrape_frequency", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "86400", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.852 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "self_heal": { 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "self_heal", 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.853 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.854 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.855 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_threshold": { 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_threshold", 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "7257600", 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.856 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "diskprediction_local", 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.857 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.858 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.860 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.861 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "predict_interval": { 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "predict_interval", 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "86400", 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.862 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "predictor_model": { 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "predictor_model", 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "prophetstor", 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.864 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "influx", 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": false, 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "influxdb python module not found", 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "batch_size": { 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "batch_size", 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5000", 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "database": { 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "database", 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "ceph", 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T06:00:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "hostname": { 2024-05-09T06:00:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "hostname", 2024-05-09T06:00:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "InfluxDB server hostname", 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "30", 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "5", 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "password": { 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "password", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "password of InfluxDB server user", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "port": { 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "port", 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8086", 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "InfluxDB server port", 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "ssl": { 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssl", 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "false", 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "threads": { 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "threads", 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "32", 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "username": { 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "username", 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "username of InfluxDB server user", 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "verify_ssl": { 2024-05-09T06:00:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "verify_ssl", 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "true", 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "insights", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "iostat", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "k8sevents", 2024-05-09T06:00:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_event_retention_days": { 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ceph_event_retention_days", 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "7", 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "config_check_secs": { 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "config_check_secs", 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "10", 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "localpool", 2024-05-09T06:00:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "failure_domain": { 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "failure_domain", 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "host", 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "failure domain for any created local pool", 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": { 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "min_size", 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "default min_size for any created local pool", 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "num_rep": { 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "num_rep", 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "3", 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "default replica count for any created local pool", 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": { 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pg_num", 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "128", 2024-05-09T06:00:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "prefix": { 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "prefix", 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "name prefix for any created local pool", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "subtree": { 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "subtree", 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "rack", 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mds_autoscaler", 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mirroring", 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "nfs", 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "orchestrator", 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "fail_fs": { 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "fail_fs", 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "orchestrator": { 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "orchestrator", 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm", 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "rook", 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "test_orchestrator" 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Orchestrator backend", 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "osd_perf_query", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "osd_support", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.951 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pg_autoscaler", 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "threshold": { 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "threshold", 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "3.0", 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1.0", 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "scaling threshold", 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "progress", 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_pg_recovery_event": { 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "allow_pg_recovery_event", 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "enabled": { 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enabled", 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "max_completed_events": { 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_completed_events", 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "50", 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "number of past completed events to remember", 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how long the module is going to sleep", 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "prometheus", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "cache": { 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cache", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "exclude_perf_counters": { 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "exclude_perf_counters", 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_stats_pools": { 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_stats_pools", 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T06:00:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "300", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "scrape_interval": { 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "scrape_interval", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "15.0", 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "server_addr": { 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_addr", 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "::", 2024-05-09T06:00:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "server_port": { 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_port", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "9283", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "stale_cache_strategy": { 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "stale_cache_strategy", 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "log", 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_behaviour": { 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_behaviour", 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "default", 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "default", 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "error" 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_error_status_code": { 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_error_status_code", 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "500", 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "400", 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "599", 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_support", 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "max_concurrent_snap_create": { 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_concurrent_snap_create", 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "mirror_snapshot_schedule": { 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T06:00:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "trash_purge_schedule": { 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "trash_purge_schedule", 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "restful", 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "enable_auth": { 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enable_auth", 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "key_file": { 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "key_file", 2024-05-09T06:00:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "server_addr": { 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_addr", 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "server_port": { 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_port", 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rgw", 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rook", 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "storage_class": { 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "storage_class", 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "local", 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "selftest", 2024-05-09T06:00:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "roption1": { 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "roption1", 2024-05-09T06:00:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "roption2": { 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "roption2", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "xyz", 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption1": { 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption1", 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption2": { 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption2", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption3": { 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption3", 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption4": { 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption4", 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption5": { 2024-05-09T06:00:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption5", 2024-05-09T06:00:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption6": { 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption6", 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption7": { 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption7", 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "42", 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "testkey": { 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "testkey", 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "testlkey": { 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "testlkey", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "testnewline": { 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "testnewline", 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smb", 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "update_orchestration": { 2024-05-09T06:00:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "update_orchestration", 2024-05-09T06:00:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T06:00:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snap_schedule", 2024-05-09T06:00:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_m_granularity": { 2024-05-09T06:00:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "allow_m_granularity", 2024-05-09T06:00:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T06:00:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "dump_on_update": { 2024-05-09T06:00:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "dump_on_update", 2024-05-09T06:00:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "dump database to debug log on update", 2024-05-09T06:00:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "stats", 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.057 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.059 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.060 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.061 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.062 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "status", 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.063 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.064 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.065 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.066 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.068 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "telegraf", 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "address": { 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "address", 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "15", 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.072 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.072 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.073 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.073 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.073 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.073 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.074 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.075 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.076 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.077 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.078 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "telemetry", 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_basic": { 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_basic", 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.079 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_crash": { 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_crash", 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.080 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_device": { 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_device", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_ident": { 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_ident", 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_perf": { 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_perf", 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.083 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "contact": { 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "contact", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.084 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "description": { 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "description", 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "device_url": { 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "device_url", 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "enabled": { 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enabled", 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.087 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "24", 2024-05-09T06:00:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "8", 2024-05-09T06:00:22.088 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "last_opt_revision": { 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "last_opt_revision", 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "1", 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.089 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "leaderboard": { 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "leaderboard", 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.090 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "leaderboard_description": { 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "leaderboard_description", 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.091 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.092 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.093 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.094 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.095 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.096 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "organization": { 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "organization", 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.097 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: "proxy": { 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "proxy", 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.098 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.100 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "url": { 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "url", 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.101 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "test_orchestrator", 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.102 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.103 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.105 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.106 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.108 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.109 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "volumes", 2024-05-09T06:00:22.109 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.109 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.109 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.109 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.110 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.111 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.112 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.113 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.114 INFO:teuthology.orchestra.run.smithi115.stdout: "max_concurrent_clones": { 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_concurrent_clones", 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "4", 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.115 INFO:teuthology.orchestra.run.smithi115.stdout: "periodic_async_work": { 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "periodic_async_work", 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Periodically check for async work", 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "snapshot_clone_delay": { 2024-05-09T06:00:22.116 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snapshot_clone_delay", 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "snapshot_clone_no_wait": { 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T06:00:22.117 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.118 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.119 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix", 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: "discovery_interval": { 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "discovery_interval", 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.120 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "100", 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "identifier": { 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "identifier", 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.121 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.122 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.123 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.125 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.126 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.127 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.128 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "zabbix_host": { 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix_host", 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.129 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "zabbix_port": { 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix_port", 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10051", 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.131 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "zabbix_sender": { 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix_sender", 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: "gid": 25018, 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "z", 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_features": 4540701547738038271, 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: "available_modules": [ 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "alerts", 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.133 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.134 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.135 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.137 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.139 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_destination": { 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_destination", 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Email address to send alerts to", 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_from_name": { 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_from_name", 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "Ceph", 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Email From: name", 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.141 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_host": { 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_host", 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SMTP server", 2024-05-09T06:00:22.142 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_password": { 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_password", 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.143 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Password to authenticate with", 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_port": { 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_port", 2024-05-09T06:00:22.144 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.145 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.145 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.145 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "465", 2024-05-09T06:00:22.145 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.145 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.145 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.145 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SMTP port", 2024-05-09T06:00:22.146 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.146 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.146 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.146 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_sender": { 2024-05-09T06:00:22.146 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_sender", 2024-05-09T06:00:22.146 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.146 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SMTP envelope sender", 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_ssl": { 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_ssl", 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.147 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_user": { 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_user", 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.148 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "User to authenticate as", 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.149 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.150 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.150 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.150 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.150 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.150 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.150 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.150 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.151 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.151 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.151 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "balancer", 2024-05-09T06:00:22.151 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "active": { 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "active", 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.152 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "begin_time": { 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "begin_time", 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0000", 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.153 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "begin_weekday": { 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "begin_weekday", 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "0", 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "6", 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_compat_max_iterations": { 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crush_compat_max_iterations", 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "25", 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "250", 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_compat_metrics": { 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crush_compat_metrics", 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.156 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_compat_step": { 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crush_compat_step", 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0.5", 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "0.001", 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "0.999", 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.157 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "aggressiveness of optimization", 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "end_time": { 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "end_time", 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "2359", 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "end_weekday": { 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "end_weekday", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "0", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "6", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:00:22.159 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.160 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.162 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.162 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.162 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.162 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.162 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.162 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.162 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.162 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.163 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.164 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "min_score": { 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "min_score", 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.165 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": { 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mode", 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "upmap", 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "crush-compat", 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "none", 2024-05-09T06:00:22.166 INFO:teuthology.orchestra.run.smithi115.stdout: "read", 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap", 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap-read" 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Balancer mode", 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_ids": { 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pool_ids", 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.168 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.168 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.168 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.168 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T06:00:22.168 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.169 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.169 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.169 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:22.169 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:22.169 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.171 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap_max_deviation": { 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "upmap_max_deviation", 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.172 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap_max_optimizations": { 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "upmap_max_optimizations", 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.173 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephadm", 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.174 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.175 INFO:teuthology.orchestra.run.smithi115.stdout: "agent_down_multiplier": { 2024-05-09T06:00:22.175 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "agent_down_multiplier", 2024-05-09T06:00:22.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.175 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.175 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.175 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "3.0", 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: "agent_refresh_rate": { 2024-05-09T06:00:22.176 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "agent_refresh_rate", 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "20", 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "agent_starting_port": { 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "agent_starting_port", 2024-05-09T06:00:22.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "4721", 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_ptrace": { 2024-05-09T06:00:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "allow_ptrace", 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "autotune_interval": { 2024-05-09T06:00:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "autotune_interval", 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "autotune_memory_target_ratio": { 2024-05-09T06:00:22.180 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0.7", 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm_log_destination": { 2024-05-09T06:00:22.181 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephadm_log_destination", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "file", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "file,syslog", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "syslog" 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.182 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.183 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.183 INFO:teuthology.orchestra.run.smithi115.stdout: "cgroups_split": { 2024-05-09T06:00:22.183 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cgroups_split", 2024-05-09T06:00:22.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.183 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.183 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.183 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "config_checks_enabled": { 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "config_checks_enabled", 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.184 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "config_dashboard": { 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "config_dashboard", 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.185 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_alertmanager": { 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_alertmanager", 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.186 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_base": { 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_base", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Container image name, without the tag", 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_elasticsearch": { 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_elasticsearch", 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.188 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "elasticsearch container image", 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_grafana": { 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_grafana", 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.189 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_haproxy": { 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_haproxy", 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.190 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "HAproxy container image", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_jaeger_agent": { 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_jaeger_agent", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Jaeger agent container image", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_jaeger_collector": { 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_jaeger_collector", 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Jaeger collector container image", 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.192 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_jaeger_query": { 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_jaeger_query", 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.193 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Jaeger query container image", 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_keepalived": { 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_keepalived", 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T06:00:22.194 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Keepalived container image", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_loki": { 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_loki", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Loki container image", 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_node_exporter": { 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_node_exporter", 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:22.196 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_nvmeof": { 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_nvmeof", 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Nvme-of container image", 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.197 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_prometheus": { 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_prometheus", 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.198 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_promtail": { 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_promtail", 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.199 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Promtail container image", 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_samba": { 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_samba", 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T06:00:22.200 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Samba/SMB container image", 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_snmp_gateway": { 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_snmp_gateway", 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.201 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SNMP Gateway container image", 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "container_init": { 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_init", 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.202 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "daemon_cache_timeout": { 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "daemon_cache_timeout", 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.203 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: "default_cephadm_command_timeout": { 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.204 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "900", 2024-05-09T06:00:22.205 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.205 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.205 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.205 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T06:00:22.205 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.205 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.205 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.206 INFO:teuthology.orchestra.run.smithi115.stdout: "default_registry": { 2024-05-09T06:00:22.206 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "default_registry", 2024-05-09T06:00:22.206 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.206 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.206 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.207 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io", 2024-05-09T06:00:22.207 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.207 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.207 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.207 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T06:00:22.207 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.207 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.208 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.208 INFO:teuthology.orchestra.run.smithi115.stdout: "device_cache_timeout": { 2024-05-09T06:00:22.208 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "device_cache_timeout", 2024-05-09T06:00:22.208 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.208 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.209 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "1800", 2024-05-09T06:00:22.209 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.209 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.209 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.209 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "seconds to cache device inventory", 2024-05-09T06:00:22.209 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.210 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.210 INFO:teuthology.orchestra.run.smithi115.stdout: "device_enhanced_scan": { 2024-05-09T06:00:22.210 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "device_enhanced_scan", 2024-05-09T06:00:22.210 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.211 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.211 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.211 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.211 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.211 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.211 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.211 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T06:00:22.212 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.212 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.212 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.212 INFO:teuthology.orchestra.run.smithi115.stdout: "facts_cache_timeout": { 2024-05-09T06:00:22.212 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "facts_cache_timeout", 2024-05-09T06:00:22.212 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.213 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.213 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.213 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:22.213 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.213 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.213 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.214 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "seconds to cache host facts data", 2024-05-09T06:00:22.214 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.214 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.214 INFO:teuthology.orchestra.run.smithi115.stdout: "grafana_dashboards_path": { 2024-05-09T06:00:22.214 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "grafana_dashboards_path", 2024-05-09T06:00:22.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.215 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.215 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.215 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T06:00:22.215 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.215 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.215 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.216 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T06:00:22.216 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.216 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.216 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.216 INFO:teuthology.orchestra.run.smithi115.stdout: "host_check_interval": { 2024-05-09T06:00:22.216 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "host_check_interval", 2024-05-09T06:00:22.217 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.217 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.217 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.217 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.217 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.217 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.218 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.218 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to perform a host check", 2024-05-09T06:00:22.218 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.218 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.219 INFO:teuthology.orchestra.run.smithi115.stdout: "hw_monitoring": { 2024-05-09T06:00:22.219 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "hw_monitoring", 2024-05-09T06:00:22.219 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.219 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.219 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.219 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.220 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.220 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.220 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.220 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T06:00:22.220 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.221 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.221 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.221 INFO:teuthology.orchestra.run.smithi115.stdout: "inventory_list_all": { 2024-05-09T06:00:22.221 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "inventory_list_all", 2024-05-09T06:00:22.221 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.222 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.222 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.222 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.222 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.222 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.222 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.223 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T06:00:22.223 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.223 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.223 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.223 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.224 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.224 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.224 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.224 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.224 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.224 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.225 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.225 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.225 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.225 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.225 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.226 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.226 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.226 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.226 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.226 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.227 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.227 INFO:teuthology.orchestra.run.smithi115.stdout: "log_refresh_metadata": { 2024-05-09T06:00:22.227 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_refresh_metadata", 2024-05-09T06:00:22.227 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.227 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.228 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.228 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.228 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.228 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.228 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.228 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T06:00:22.229 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.229 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.229 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.229 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.229 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.230 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.230 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.230 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.230 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.230 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.231 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.231 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T06:00:22.231 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.231 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.231 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.232 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.232 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.232 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.232 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.232 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.233 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.233 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.233 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.233 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.233 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.233 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.233 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.234 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.234 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.234 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.234 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.234 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.234 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.235 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.235 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.235 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.235 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.235 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.236 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.236 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.236 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.236 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.236 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.236 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.236 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.236 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.237 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T06:00:22.238 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.238 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.238 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.238 INFO:teuthology.orchestra.run.smithi115.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T06:00:22.238 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T06:00:22.238 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.239 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.239 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.239 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "*", 2024-05-09T06:00:22.239 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.239 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.239 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.239 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T06:00:22.239 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.239 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "max_count_per_host": { 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_count_per_host", 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "max number of daemons per service per host", 2024-05-09T06:00:22.240 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.241 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.241 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.241 INFO:teuthology.orchestra.run.smithi115.stdout: "max_osd_draining_count": { 2024-05-09T06:00:22.241 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_osd_draining_count", 2024-05-09T06:00:22.241 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.241 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.241 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.241 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.242 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.242 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.242 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.242 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T06:00:22.242 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.242 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.243 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.243 INFO:teuthology.orchestra.run.smithi115.stdout: "migration_current": { 2024-05-09T06:00:22.243 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "migration_current", 2024-05-09T06:00:22.243 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.243 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.243 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.243 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "internal - do not modify", 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": { 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mode", 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.244 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "root", 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm-package", 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "root" 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "oob_default_addr": { 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "oob_default_addr", 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "169.254.1.1", 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.246 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "prometheus_alerts_path": { 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "prometheus_alerts_path", 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.247 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_insecure": { 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_insecure", 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_password": { 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_password", 2024-05-09T06:00:22.248 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_url": { 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_url", 2024-05-09T06:00:22.249 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.251 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_username": { 2024-05-09T06:00:22.251 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_username", 2024-05-09T06:00:22.251 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.251 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.251 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.251 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.252 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.252 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.252 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.252 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T06:00:22.252 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.252 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.252 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.253 INFO:teuthology.orchestra.run.smithi115.stdout: "secure_monitoring_stack": { 2024-05-09T06:00:22.253 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "secure_monitoring_stack", 2024-05-09T06:00:22.253 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.253 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.253 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.254 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.254 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.254 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.254 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.254 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T06:00:22.254 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.254 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.255 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.255 INFO:teuthology.orchestra.run.smithi115.stdout: "service_discovery_port": { 2024-05-09T06:00:22.255 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "service_discovery_port", 2024-05-09T06:00:22.255 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.255 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.256 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.256 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8765", 2024-05-09T06:00:22.256 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.256 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.256 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.256 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "cephadm service discovery port", 2024-05-09T06:00:22.256 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.257 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.257 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.257 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.257 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.257 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.258 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.258 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.258 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.258 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.258 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.258 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.259 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.259 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.259 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.259 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.259 INFO:teuthology.orchestra.run.smithi115.stdout: "ssh_config_file": { 2024-05-09T06:00:22.259 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssh_config_file", 2024-05-09T06:00:22.260 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.260 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.260 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.260 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.260 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.260 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.260 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.261 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T06:00:22.261 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.261 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.261 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.261 INFO:teuthology.orchestra.run.smithi115.stdout: "use_agent": { 2024-05-09T06:00:22.262 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "use_agent", 2024-05-09T06:00:22.262 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.262 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.262 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.262 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.262 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.263 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.263 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.263 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T06:00:22.263 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.263 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.263 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.264 INFO:teuthology.orchestra.run.smithi115.stdout: "use_repo_digest": { 2024-05-09T06:00:22.264 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "use_repo_digest", 2024-05-09T06:00:22.264 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.264 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.264 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.265 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.265 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.265 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.265 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.265 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T06:00:22.265 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.265 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.266 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.266 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_on_failed_host_check": { 2024-05-09T06:00:22.266 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_on_failed_host_check", 2024-05-09T06:00:22.266 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.266 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.267 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.267 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.267 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.267 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.267 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.267 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T06:00:22.268 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.268 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.268 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.268 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_on_stray_daemons": { 2024-05-09T06:00:22.268 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_on_stray_daemons", 2024-05-09T06:00:22.269 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.269 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.269 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.269 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.269 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.269 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.270 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.270 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T06:00:22.270 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.270 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.270 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.271 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_on_stray_hosts": { 2024-05-09T06:00:22.271 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_on_stray_hosts", 2024-05-09T06:00:22.271 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.271 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.271 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.271 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.272 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.272 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.272 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.272 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T06:00:22.272 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.272 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.273 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.273 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crash", 2024-05-09T06:00:22.273 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.274 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.274 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.274 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.274 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.274 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.275 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.275 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.275 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.275 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.275 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.275 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.276 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.276 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.276 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.276 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.276 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.276 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.276 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.276 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.277 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.277 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.277 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.277 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.278 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.279 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.280 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "retain_interval": { 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "retain_interval", 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "31536000", 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.281 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.282 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_recent_interval": { 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_recent_interval", 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "1209600", 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T06:00:22.283 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "dashboard", 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.284 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.285 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.286 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "AUDIT_API_ENABLED": { 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.287 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.288 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.289 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.290 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.291 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.292 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T06:00:22.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.294 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.296 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "admin", 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.297 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.298 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_URL": { 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_URL", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.299 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "admin", 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.301 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T06:00:22.303 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.304 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "[]", 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.305 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "{}", 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.306 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.307 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.308 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.309 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.310 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.311 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.314 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.315 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T06:00:22.316 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T06:00:22.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8", 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T06:00:22.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "45", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.319 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.320 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "admin", 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.321 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.322 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.323 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.324 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.325 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.326 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "cross_origin_url": { 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cross_origin_url", 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.327 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "crt_file": { 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crt_file", 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.328 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "debug": { 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "debug", 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Enable/disable debug options", 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.329 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "jwt_token_ttl": { 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "jwt_token_ttl", 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "28800", 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "key_file": { 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "key_file", 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.331 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.332 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "motd": { 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "motd", 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "The message of the day", 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "redirect_resolve_ip_addr": { 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "server_addr": { 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_addr", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "::", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "server_port": { 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_port", 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8080", 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "ssl": { 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssl", 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "ssl_server_port": { 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssl_server_port", 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8443", 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_behaviour": { 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_behaviour", 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "redirect", 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "redirect" 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_error_status_code": { 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_error_status_code", 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "500", 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "400", 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "599", 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "url_prefix": { 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "url_prefix", 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "devicehealth", 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "enable_monitoring": { 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enable_monitoring", 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "monitor device health metrics", 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "mark_out_threshold": { 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mark_out_threshold", 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "2419200", 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": { 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pool_name", 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "device_health_metrics", 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "retention_period": { 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "retention_period", 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "15552000", 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how long to retain device health metrics", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "scrape_frequency": { 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "scrape_frequency", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "86400", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "self_heal": { 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "self_heal", 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_threshold": { 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_threshold", 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "7257600", 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T06:00:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "diskprediction_local", 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "predict_interval": { 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "predict_interval", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "86400", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "predictor_model": { 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "predictor_model", 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "prophetstor", 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.368 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "influx", 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": false, 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "influxdb python module not found", 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "batch_size": { 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "batch_size", 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5000", 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "database": { 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "database", 2024-05-09T06:00:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "ceph", 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "hostname": { 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "hostname", 2024-05-09T06:00:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "InfluxDB server hostname", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "30", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "5", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "password": { 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "password", 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "password of InfluxDB server user", 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "port": { 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "port", 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8086", 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "InfluxDB server port", 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "ssl": { 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssl", 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "false", 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "threads": { 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "threads", 2024-05-09T06:00:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "32", 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "username": { 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "username", 2024-05-09T06:00:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "username of InfluxDB server user", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "verify_ssl": { 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "verify_ssl", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "true", 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "insights", 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "iostat", 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "k8sevents", 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_event_retention_days": { 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ceph_event_retention_days", 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "7", 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "config_check_secs": { 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "config_check_secs", 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "10", 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "localpool", 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "failure_domain": { 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "failure_domain", 2024-05-09T06:00:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "host", 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "failure domain for any created local pool", 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": { 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "min_size", 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "default min_size for any created local pool", 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "num_rep": { 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "num_rep", 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "3", 2024-05-09T06:00:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "default replica count for any created local pool", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": { 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pg_num", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "128", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "prefix": { 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "prefix", 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "name prefix for any created local pool", 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "subtree": { 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "subtree", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "rack", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T06:00:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mds_autoscaler", 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mirroring", 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.426 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.427 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.428 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "nfs", 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.429 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.430 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.431 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.432 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.433 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.434 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.435 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "orchestrator", 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "fail_fs": { 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "fail_fs", 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.436 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.437 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.438 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.439 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.440 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.441 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "orchestrator": { 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "orchestrator", 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.442 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm", 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "rook", 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "test_orchestrator" 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Orchestrator backend", 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.443 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "osd_perf_query", 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.444 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.445 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.446 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.447 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.448 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.449 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.450 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.450 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.450 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.450 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.450 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.450 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.451 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.451 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.451 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.451 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.451 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.451 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.452 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "osd_support", 2024-05-09T06:00:22.452 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.452 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.452 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.452 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.452 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.452 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.452 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.453 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.453 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.453 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.453 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.453 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.453 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.453 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.453 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.454 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.454 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.454 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.454 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.454 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.454 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.455 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.455 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.455 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.455 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.455 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.456 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.456 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.456 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.456 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.456 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.456 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.456 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.456 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.457 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.458 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.458 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.458 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.458 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.458 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.458 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.459 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.459 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.459 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.459 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.459 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.459 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.460 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.460 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.460 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.460 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.461 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.461 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.461 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.461 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.461 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.461 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.462 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.462 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.462 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.462 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.462 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.463 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.463 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.463 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.463 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.463 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.463 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.464 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.464 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.464 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.464 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.464 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.465 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.465 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.466 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pg_autoscaler", 2024-05-09T06:00:22.466 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.466 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.466 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.466 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.466 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.467 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.467 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.467 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.467 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.467 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.468 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.468 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.468 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.468 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.468 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.468 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.469 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.469 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.469 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.469 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.469 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.469 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.470 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.470 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.470 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.470 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.470 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.471 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.471 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.471 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.471 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.471 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.471 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.472 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.472 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.472 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.472 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.474 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.474 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.474 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.474 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.474 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.475 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.475 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.475 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.475 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.475 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.475 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.476 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.476 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.476 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.476 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.476 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.477 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.477 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.477 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.477 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.477 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.477 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.478 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.478 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.478 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.478 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:22.478 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:22.479 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.479 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.479 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.479 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:22.479 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.479 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.480 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.480 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.480 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.480 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.480 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.481 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.481 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.481 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.481 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.481 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.481 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.482 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.482 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.482 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.482 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.482 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.483 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.483 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.483 INFO:teuthology.orchestra.run.smithi115.stdout: "threshold": { 2024-05-09T06:00:22.483 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "threshold", 2024-05-09T06:00:22.483 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.484 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.484 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.484 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "3.0", 2024-05-09T06:00:22.484 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1.0", 2024-05-09T06:00:22.484 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.484 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.485 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "scaling threshold", 2024-05-09T06:00:22.485 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T06:00:22.485 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.485 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.486 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "progress", 2024-05-09T06:00:22.486 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.486 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.486 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.487 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_pg_recovery_event": { 2024-05-09T06:00:22.487 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "allow_pg_recovery_event", 2024-05-09T06:00:22.487 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.487 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.487 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.488 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.488 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.488 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.488 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.488 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T06:00:22.488 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.488 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.489 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.489 INFO:teuthology.orchestra.run.smithi115.stdout: "enabled": { 2024-05-09T06:00:22.489 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enabled", 2024-05-09T06:00:22.489 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.489 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.490 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.490 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.490 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.490 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.490 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.490 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.491 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.491 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.491 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.492 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.492 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.492 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.492 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.492 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.492 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.492 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.493 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.493 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.493 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.493 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.493 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.493 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.494 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.494 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.494 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.494 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.494 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.494 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.495 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.495 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.495 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.495 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.495 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.496 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.496 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.496 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.496 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.496 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.496 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.497 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.497 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.497 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.497 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.497 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.497 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.498 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.498 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.498 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.498 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.498 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.499 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.499 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.499 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.499 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.499 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.499 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.499 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.500 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.500 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.500 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.500 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.500 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.500 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.500 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.501 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.501 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.501 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.501 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.501 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.501 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.501 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.502 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.502 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.502 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.502 INFO:teuthology.orchestra.run.smithi115.stdout: "max_completed_events": { 2024-05-09T06:00:22.502 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_completed_events", 2024-05-09T06:00:22.502 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.503 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.503 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.503 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "50", 2024-05-09T06:00:22.503 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.503 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.503 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.503 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "number of past completed events to remember", 2024-05-09T06:00:22.503 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.503 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how long the module is going to sleep", 2024-05-09T06:00:22.504 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.505 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.505 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.505 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.505 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.505 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.506 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.506 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.506 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.506 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.506 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.506 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.507 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.507 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.507 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.507 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.508 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "prometheus", 2024-05-09T06:00:22.508 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.508 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.508 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.508 INFO:teuthology.orchestra.run.smithi115.stdout: "cache": { 2024-05-09T06:00:22.509 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cache", 2024-05-09T06:00:22.509 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.509 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.509 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.509 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.509 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "exclude_perf_counters": { 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "exclude_perf_counters", 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.510 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.511 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.511 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.511 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.511 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.511 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T06:00:22.511 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T06:00:22.511 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.512 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.512 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.512 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.512 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.512 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.512 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.513 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.513 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.513 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.513 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.513 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.513 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.514 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.514 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.514 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.514 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.514 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.514 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.514 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.515 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.515 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.515 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.515 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.515 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.515 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.516 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.516 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.516 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.516 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.516 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.516 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.516 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.517 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.517 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.517 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.517 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.517 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.517 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.518 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.519 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_stats_pools": { 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_stats_pools", 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.520 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "300", 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.521 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "scrape_interval": { 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "scrape_interval", 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "15.0", 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.522 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "server_addr": { 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_addr", 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "::", 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.523 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "server_port": { 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_port", 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "9283", 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.524 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.525 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "stale_cache_strategy": { 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "stale_cache_strategy", 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "log", 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.526 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_behaviour": { 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_behaviour", 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "default", 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "default", 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: "error" 2024-05-09T06:00:22.527 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_error_status_code": { 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_error_status_code", 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "500", 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "400", 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "599", 2024-05-09T06:00:22.528 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_support", 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.529 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.530 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.531 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.532 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "max_concurrent_snap_create": { 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_concurrent_snap_create", 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.534 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "mirror_snapshot_schedule": { 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.535 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.536 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "trash_purge_schedule": { 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "trash_purge_schedule", 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.537 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "restful", 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.538 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "enable_auth": { 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enable_auth", 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.539 INFO:teuthology.orchestra.run.smithi115.stdout: "key_file": { 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "key_file", 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.541 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.542 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.543 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.544 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "server_addr": { 2024-05-09T06:00:22.545 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_addr", 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "server_port": { 2024-05-09T06:00:22.546 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_port", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.547 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rgw", 2024-05-09T06:00:22.548 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.549 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.550 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.551 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.552 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.553 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rook", 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.555 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.556 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.557 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.558 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.559 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.559 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.559 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.559 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.559 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "storage_class": { 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "storage_class", 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "local", 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "selftest", 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.624 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.625 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.626 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "roption1": { 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "roption1", 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.627 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "roption2": { 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "roption2", 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "xyz", 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.628 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption1": { 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption1", 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption2": { 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption2", 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption3": { 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption3", 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption4": { 2024-05-09T06:00:22.631 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption4", 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption5": { 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption5", 2024-05-09T06:00:22.632 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption6": { 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption6", 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.633 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption7": { 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption7", 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.634 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "42", 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.635 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "testkey": { 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "testkey", 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.636 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "testlkey": { 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "testlkey", 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.637 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "testnewline": { 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "testnewline", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.638 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smb", 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.639 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.640 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.641 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.642 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.643 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.644 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "update_orchestration": { 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "update_orchestration", 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snap_schedule", 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_m_granularity": { 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "allow_m_granularity", 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.646 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "dump_on_update": { 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "dump_on_update", 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.647 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "dump database to debug log on update", 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.648 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.649 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.650 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.651 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.652 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.653 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "stats", 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.654 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.655 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.656 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.657 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.658 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.659 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "status", 2024-05-09T06:00:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.661 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.662 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.663 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.664 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.665 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.666 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "telegraf", 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "address": { 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "address", 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.667 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.668 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "15", 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.669 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.672 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.673 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.674 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "telemetry", 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_basic": { 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_basic", 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.675 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_crash": { 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_crash", 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.676 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_device": { 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_device", 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.677 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_ident": { 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_ident", 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.678 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_perf": { 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_perf", 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.679 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "contact": { 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "contact", 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.680 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "description": { 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "description", 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.681 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "device_url": { 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "device_url", 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.682 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "enabled": { 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enabled", 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.683 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "24", 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "8", 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.684 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "last_opt_revision": { 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "last_opt_revision", 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "1", 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "leaderboard": { 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "leaderboard", 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.686 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "leaderboard_description": { 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "leaderboard_description", 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.687 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.688 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.689 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.690 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "organization": { 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "organization", 2024-05-09T06:00:22.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "proxy": { 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "proxy", 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.693 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.694 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "url": { 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "url", 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.695 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "test_orchestrator", 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.696 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.697 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.698 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.699 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.701 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "volumes", 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.702 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.703 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.704 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.705 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "max_concurrent_clones": { 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_concurrent_clones", 2024-05-09T06:00:22.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "4", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "periodic_async_work": { 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "periodic_async_work", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Periodically check for async work", 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "snapshot_clone_delay": { 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snapshot_clone_delay", 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.709 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "snapshot_clone_no_wait": { 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.710 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.711 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.711 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.711 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T06:00:22.711 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.711 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.711 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.711 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.712 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.712 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.712 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.712 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.712 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.712 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix", 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "discovery_interval": { 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "discovery_interval", 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "100", 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "identifier": { 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "identifier", 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.714 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.715 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.716 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.718 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.720 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "zabbix_host": { 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix_host", 2024-05-09T06:00:22.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "zabbix_port": { 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix_port", 2024-05-09T06:00:22.722 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10051", 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "zabbix_sender": { 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix_sender", 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.723 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.724 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "modules": [ 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm", 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "dashboard", 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "iostat", 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "nfs", 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "restful", 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_schedule" 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "available_modules": [ 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "alerts", 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.725 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.726 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.729 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.730 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_destination": { 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_destination", 2024-05-09T06:00:22.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Email address to send alerts to", 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_from_name": { 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_from_name", 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.732 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "Ceph", 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Email From: name", 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_host": { 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_host", 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.733 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SMTP server", 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_password": { 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_password", 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.734 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Password to authenticate with", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_port": { 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_port", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "465", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.735 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SMTP port", 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_sender": { 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_sender", 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.736 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SMTP envelope sender", 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_ssl": { 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_ssl", 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.737 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "smtp_user": { 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smtp_user", 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.738 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "User to authenticate as", 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.739 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "balancer", 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "active": { 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "active", 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "begin_time": { 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "begin_time", 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0000", 2024-05-09T06:00:22.741 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "begin_weekday": { 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "begin_weekday", 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.742 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "0", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "6", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_compat_max_iterations": { 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crush_compat_max_iterations", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "25", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "250", 2024-05-09T06:00:22.743 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_compat_metrics": { 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crush_compat_metrics", 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T06:00:22.744 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_compat_step": { 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crush_compat_step", 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0.5", 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "0.001", 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "0.999", 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "aggressiveness of optimization", 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.745 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "end_time": { 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "end_time", 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "2359", 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "end_weekday": { 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "end_weekday", 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "0", 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "6", 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.747 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.748 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.751 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "min_score": { 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "min_score", 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.752 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": { 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mode", 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "upmap", 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "crush-compat", 2024-05-09T06:00:22.753 INFO:teuthology.orchestra.run.smithi115.stdout: "none", 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "read", 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap", 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap-read" 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Balancer mode", 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_ids": { 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pool_ids", 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.755 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap_max_deviation": { 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "upmap_max_deviation", 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "upmap_max_optimizations": { 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "upmap_max_optimizations", 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephadm", 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "agent_down_multiplier": { 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "agent_down_multiplier", 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "3.0", 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "agent_refresh_rate": { 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "agent_refresh_rate", 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "20", 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "agent_starting_port": { 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "agent_starting_port", 2024-05-09T06:00:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "4721", 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_ptrace": { 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "allow_ptrace", 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.762 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "autotune_interval": { 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "autotune_interval", 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.763 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "autotune_memory_target_ratio": { 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0.7", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.764 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm_log_destination": { 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephadm_log_destination", 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.765 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "file", 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "file,syslog", 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "syslog" 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "cgroups_split": { 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cgroups_split", 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.766 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "config_checks_enabled": { 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "config_checks_enabled", 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.767 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "config_dashboard": { 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "config_dashboard", 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.768 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_alertmanager": { 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_alertmanager", 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T06:00:22.769 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_base": { 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_base", 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Container image name, without the tag", 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_elasticsearch": { 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_elasticsearch", 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T06:00:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "elasticsearch container image", 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_grafana": { 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_grafana", 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_haproxy": { 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_haproxy", 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.773 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "HAproxy container image", 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_jaeger_agent": { 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_jaeger_agent", 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.774 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Jaeger agent container image", 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_jaeger_collector": { 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_jaeger_collector", 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.775 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Jaeger collector container image", 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_jaeger_query": { 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_jaeger_query", 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Jaeger query container image", 2024-05-09T06:00:22.776 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_keepalived": { 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_keepalived", 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Keepalived container image", 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.777 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_loki": { 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_loki", 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Loki container image", 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.778 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_node_exporter": { 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_node_exporter", 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_nvmeof": { 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_nvmeof", 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Nvme-of container image", 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_prometheus": { 2024-05-09T06:00:22.780 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_prometheus", 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Prometheus container image", 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_promtail": { 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_promtail", 2024-05-09T06:00:22.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Promtail container image", 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_samba": { 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_samba", 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.782 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Samba/SMB container image", 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image_snmp_gateway": { 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_image_snmp_gateway", 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.783 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "SNMP Gateway container image", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "container_init": { 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "container_init", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.784 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "daemon_cache_timeout": { 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "daemon_cache_timeout", 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.785 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "default_cephadm_command_timeout": { 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "900", 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T06:00:22.786 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "default_registry": { 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "default_registry", 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "docker.io", 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "device_cache_timeout": { 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "device_cache_timeout", 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "1800", 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "seconds to cache device inventory", 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "device_enhanced_scan": { 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "device_enhanced_scan", 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "facts_cache_timeout": { 2024-05-09T06:00:22.789 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "facts_cache_timeout", 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "seconds to cache host facts data", 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.790 INFO:teuthology.orchestra.run.smithi115.stdout: "grafana_dashboards_path": { 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "grafana_dashboards_path", 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "host_check_interval": { 2024-05-09T06:00:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "host_check_interval", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to perform a host check", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "hw_monitoring": { 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "hw_monitoring", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.792 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "inventory_list_all": { 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "inventory_list_all", 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.793 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.794 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.795 INFO:teuthology.orchestra.run.smithi115.stdout: "log_refresh_metadata": { 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_refresh_metadata", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.796 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.797 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.798 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.800 INFO:teuthology.orchestra.run.smithi115.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "*", 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "max_count_per_host": { 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_count_per_host", 2024-05-09T06:00:22.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "max number of daemons per service per host", 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.802 INFO:teuthology.orchestra.run.smithi115.stdout: "max_osd_draining_count": { 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_osd_draining_count", 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "migration_current": { 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "migration_current", 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "internal - do not modify", 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.804 INFO:teuthology.orchestra.run.smithi115.stdout: "mode": { 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mode", 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "root", 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm-package", 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "root" 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.805 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "oob_default_addr": { 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "oob_default_addr", 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "169.254.1.1", 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "prometheus_alerts_path": { 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "prometheus_alerts_path", 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.807 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_insecure": { 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_insecure", 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_password": { 2024-05-09T06:00:22.808 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_password", 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_url": { 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_url", 2024-05-09T06:00:22.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.810 INFO:teuthology.orchestra.run.smithi115.stdout: "registry_username": { 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "registry_username", 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "secure_monitoring_stack": { 2024-05-09T06:00:22.811 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "secure_monitoring_stack", 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "service_discovery_port": { 2024-05-09T06:00:22.812 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "service_discovery_port", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8765", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "cephadm service discovery port", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.813 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "ssh_config_file": { 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssh_config_file", 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.814 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "use_agent": { 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "use_agent", 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.815 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "use_repo_digest": { 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "use_repo_digest", 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.816 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_on_failed_host_check": { 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_on_failed_host_check", 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.817 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_on_stray_daemons": { 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_on_stray_daemons", 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.818 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_on_stray_hosts": { 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_on_stray_hosts", 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.819 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crash", 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.822 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.823 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.824 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "retain_interval": { 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "retain_interval", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "31536000", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.825 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.826 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_recent_interval": { 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_recent_interval", 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "1209600", 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.827 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "dashboard", 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.828 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.829 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.830 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "AUDIT_API_ENABLED": { 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.831 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.832 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.833 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.834 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.835 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.837 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.838 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T06:00:22.840 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T06:00:22.841 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "admin", 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T06:00:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_URL": { 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_URL", 2024-05-09T06:00:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "admin", 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "[]", 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "{}", 2024-05-09T06:00:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8", 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "45", 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T06:00:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "admin", 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T06:00:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T06:00:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "cross_origin_url": { 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cross_origin_url", 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "crt_file": { 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "crt_file", 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "debug": { 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "debug", 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Enable/disable debug options", 2024-05-09T06:00:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "jwt_token_ttl": { 2024-05-09T06:00:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "jwt_token_ttl", 2024-05-09T06:00:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "28800", 2024-05-09T06:00:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "key_file": { 2024-05-09T06:00:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "key_file", 2024-05-09T06:00:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "motd": { 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "motd", 2024-05-09T06:00:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "The message of the day", 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "redirect_resolve_ip_addr": { 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "server_addr": { 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_addr", 2024-05-09T06:00:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "::", 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "server_port": { 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_port", 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8080", 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "ssl": { 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssl", 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "ssl_server_port": { 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssl_server_port", 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8443", 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_behaviour": { 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_behaviour", 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "redirect", 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "redirect" 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_error_status_code": { 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_error_status_code", 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "500", 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "400", 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "599", 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "url_prefix": { 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "url_prefix", 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "devicehealth", 2024-05-09T06:00:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "enable_monitoring": { 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enable_monitoring", 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "monitor device health metrics", 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "mark_out_threshold": { 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mark_out_threshold", 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "2419200", 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": { 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pool_name", 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "device_health_metrics", 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "retention_period": { 2024-05-09T06:00:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "retention_period", 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "15552000", 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how long to retain device health metrics", 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "scrape_frequency": { 2024-05-09T06:00:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "scrape_frequency", 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "86400", 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "self_heal": { 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "self_heal", 2024-05-09T06:00:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "warn_threshold": { 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "warn_threshold", 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "7257600", 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "diskprediction_local", 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "predict_interval": { 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "predict_interval", 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "86400", 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "predictor_model": { 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "predictor_model", 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "prophetstor", 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "600", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "influx", 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": false, 2024-05-09T06:00:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "influxdb python module not found", 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "batch_size": { 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "batch_size", 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5000", 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "database": { 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "database", 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "ceph", 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "hostname": { 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "hostname", 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "InfluxDB server hostname", 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "30", 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "5", 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "password": { 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "password", 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "password of InfluxDB server user", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "port": { 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "port", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "8086", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "InfluxDB server port", 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.932 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "ssl": { 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ssl", 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "false", 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.933 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "threads": { 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "threads", 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "32", 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.934 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "username": { 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "username", 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "username of InfluxDB server user", 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.935 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "verify_ssl": { 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "verify_ssl", 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "true", 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "insights", 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.938 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.940 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.941 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.942 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "iostat", 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.943 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.945 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.946 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.947 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "k8sevents", 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_event_retention_days": { 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "ceph_event_retention_days", 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "7", 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "config_check_secs": { 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "config_check_secs", 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "10", 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "localpool", 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "failure_domain": { 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "failure_domain", 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "host", 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "failure domain for any created local pool", 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": { 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "min_size", 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85d80b2960 msgr2=0x7f85d80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85d80b2960 0x7f85d80b4e70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85dc005d00 tx=0x7f85dc005c70 comp rx=0 tx=0).stop 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "default min_size for any created local pool", 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "num_rep": { 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "num_rep", 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "3", 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "default replica count for any created local pool", 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": { 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pg_num", 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "128", 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "prefix": { 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "prefix", 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "name prefix for any created local pool", 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "subtree": { 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "subtree", 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "rack", 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mds_autoscaler", 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mirroring", 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "nfs", 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "orchestrator", 2024-05-09T06:00:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "fail_fs": { 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "fail_fs", 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "orchestrator": { 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "orchestrator", 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm", 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "rook", 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "test_orchestrator" 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Orchestrator backend", 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "osd_perf_query", 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "osd_support", 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "pg_autoscaler", 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "threshold": { 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "threshold", 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "3.0", 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1.0", 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "scaling threshold", 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "progress", 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_pg_recovery_event": { 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "allow_pg_recovery_event", 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "enabled": { 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enabled", 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "max_completed_events": { 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_completed_events", 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "50", 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "number of past completed events to remember", 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "sleep_interval": { 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sleep_interval", 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "5", 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "how long the module is going to sleep", 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "prometheus", 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "cache": { 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cache", 2024-05-09T06:00:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "exclude_perf_counters": { 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "exclude_perf_counters", 2024-05-09T06:00:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T06:00:23.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 msgr2=0x7f85ec19b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:00:23.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 0x7f85ec19b590 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f85e8005940 tx=0x7f85e80042f0 comp rx=0 tx=0).stop 2024-05-09T06:00:23.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 shutdown_connections 2024-05-09T06:00:23.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f85d80b2960 0x7f85d80b4e70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:23.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85ec107e20 0x7f85ec19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:23.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec1073a0 0x7f85ec19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:23.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 --2- 172.21.15.115:0/2198706175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85ec1034e0 0x7f85ec19b590 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:00:23.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 >> 172.21.15.115:0/2198706175 conn(0x7f85ec0fd980 msgr2=0x7f85ec105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:00:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T06:00:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_stats_pools": { 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_stats_pools", 2024-05-09T06:00:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "300", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "scrape_interval": { 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "scrape_interval", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "15.0", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "server_addr": { 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_addr", 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "::", 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "server_port": { 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_port", 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "9283", 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "stale_cache_strategy": { 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "stale_cache_strategy", 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "log", 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_behaviour": { 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_behaviour", 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "default", 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "default", 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "error" 2024-05-09T06:00:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "standby_error_status_code": { 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "standby_error_status_code", 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "500", 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "400", 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "599", 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_support", 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.047 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.048 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "max_concurrent_snap_create": { 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_concurrent_snap_create", 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10", 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.049 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "mirror_snapshot_schedule": { 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.050 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.051 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "trash_purge_schedule": { 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "trash_purge_schedule", 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.052 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "restful", 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "enable_auth": { 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enable_auth", 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.053 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "key_file": { 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "key_file", 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.054 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.055 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.056 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.057 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.058 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.059 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "server_addr": { 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_addr", 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "server_port": { 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "server_port", 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.061 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.062 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rgw", 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.063 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.064 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.066 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.067 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.068 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rook", 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.069 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.070 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.071 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.073 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.074 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "storage_class": { 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "storage_class", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "local", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.075 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "selftest", 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.076 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.077 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.078 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.079 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: "roption1": { 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "roption1", 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: "roption2": { 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "roption2", 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "xyz", 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption1": { 2024-05-09T06:00:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption1", 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption2": { 2024-05-09T06:00:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption2", 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption3": { 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption3", 2024-05-09T06:00:23.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "float", 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption4": { 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption4", 2024-05-09T06:00:23.086 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption5": { 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption5", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.087 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption6": { 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption6", 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "rwoption7": { 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rwoption7", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "1", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "42", 2024-05-09T06:00:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.090 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "testkey": { 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "testkey", 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.091 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "testlkey": { 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "testlkey", 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.092 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "testnewline": { 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "testnewline", 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.093 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "smb", 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.094 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.686+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 shutdown_connections 2024-05-09T06:00:23.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:00:21.687+0000 7f85f429f640 1 -- 172.21.15.115:0/2198706175 wait complete. 2024-05-09T06:00:23.096 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.096 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.100 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.101 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "update_orchestration": { 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "update_orchestration", 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.102 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snap_schedule", 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.103 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_m_granularity": { 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "allow_m_granularity", 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T06:00:23.104 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.105 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.105 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.105 INFO:teuthology.orchestra.run.smithi115.stdout: "dump_on_update": { 2024-05-09T06:00:23.105 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "dump_on_update", 2024-05-09T06:00:23.105 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.105 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.105 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "dump database to debug log on update", 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.106 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.108 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.109 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.110 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.111 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "stats", 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.112 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.113 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.114 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.115 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.116 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.117 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "status", 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.118 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.119 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.120 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.121 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.122 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.123 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "telegraf", 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.124 INFO:teuthology.orchestra.run.smithi115.stdout: "address": { 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "address", 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:23.125 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "15", 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.126 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.127 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.128 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.129 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.130 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.131 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "telemetry", 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.132 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_basic": { 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_basic", 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.133 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_crash": { 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_crash", 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.134 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_device": { 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_device", 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.135 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_ident": { 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_ident", 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "channel_perf": { 2024-05-09T06:00:23.136 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "channel_perf", 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "contact": { 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "contact", 2024-05-09T06:00:23.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "description": { 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "description", 2024-05-09T06:00:23.138 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "device_url": { 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "device_url", 2024-05-09T06:00:23.139 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "enabled": { 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "enabled", 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.140 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "24", 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "8", 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "last_opt_revision": { 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "last_opt_revision", 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.142 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "1", 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "leaderboard": { 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "leaderboard", 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.143 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "leaderboard_description": { 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "leaderboard_description", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.144 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.145 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.147 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.149 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "organization": { 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "organization", 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "proxy": { 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "proxy", 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.152 INFO:teuthology.orchestra.run.smithi115.stdout: "url": { 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "url", 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "test_orchestrator", 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.154 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.155 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.156 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.157 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.158 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "volumes", 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.160 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.161 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.162 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.163 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.164 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "max_concurrent_clones": { 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "max_concurrent_clones", 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "4", 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T06:00:23.165 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "periodic_async_work": { 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "periodic_async_work", 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Periodically check for async work", 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "snapshot_clone_delay": { 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snapshot_clone_delay", 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.167 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "snapshot_clone_no_wait": { 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "True", 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix", 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "can_run": true, 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "error_string": "", 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "module_options": { 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "discovery_interval": { 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "discovery_interval", 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "uint", 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "100", 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.170 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "identifier": { 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "identifier", 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.171 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "interval": { 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "interval", 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "secs", 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "60", 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.172 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "log_level": { 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_level", 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.173 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster": { 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster", 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_cluster_level": { 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_cluster_level", 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "info", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [ 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "critical", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "debug", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "error", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "info", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "warning" 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": { 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "log_to_file", 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "bool", 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "False", 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "sqlite3_killpoint": { 2024-05-09T06:00:23.177 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "sqlite3_killpoint", 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "dev", 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "0", 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "zabbix_host": { 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix_host", 2024-05-09T06:00:23.178 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "", 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "zabbix_port": { 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix_port", 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "int", 2024-05-09T06:00:23.179 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "10051", 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "zabbix_sender": { 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "zabbix_sender", 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "str", 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "level": "advanced", 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 0, 2024-05-09T06:00:23.180 INFO:teuthology.orchestra.run.smithi115.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "min": "", 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "max": "", 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "enum_allowed": [], 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "desc": "", 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "long_desc": "", 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "tags": [], 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "see_also": [] 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "services": { 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "dashboard": "https://172.21.15.115:8443/" 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "always_on_modules": { 2024-05-09T06:00:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus": [ 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "balancer", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "devicehealth", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "orchestrator", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscaler", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "progress", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_support", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "status", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "telemetry", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "volumes" 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific": [ 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "balancer", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-05-09T06:00:23.182 INFO:teuthology.orchestra.run.smithi115.stdout: "devicehealth", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "orchestrator", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscaler", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "progress", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_support", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "status", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "telemetry", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "volumes" 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy": [ 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "balancer", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "devicehealth", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "orchestrator", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscaler", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "progress", 2024-05-09T06:00:23.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_support", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "status", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "telemetry", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "volumes" 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "reef": [ 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "balancer", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "devicehealth", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "orchestrator", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscaler", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "progress", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_support", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "status", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "telemetry", 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: "volumes" 2024-05-09T06:00:23.184 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "squid": [ 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "balancer", 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "crash", 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "devicehealth", 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "orchestrator", 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscaler", 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "progress", 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_support", 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "status", 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "telemetry", 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "volumes" 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "last_failure_osd_epoch": 88, 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: "active_clients": [ 2024-05-09T06:00:23.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "devicehealth", 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:0", 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 504017164 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "libcephsqlite", 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-05-09T06:00:23.186 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:0", 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1918947412 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "rbd_support", 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:0", 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1136799915 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snap_schedule", 2024-05-09T06:00:23.187 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:0", 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2612736857 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "snap_schedule", 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:0", 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1769859937 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "volumes", 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:0", 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3054356125 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-09T06:00:23.189 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T06:00:23.224 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T06:00:23.225 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T06:00:23.225 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:22 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/2198706175' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T06:00:23.225 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:21 smithi161 ceph-mon[33805]: pgmap v703: 129 pgs: 9 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 188 B/s wr, 27 op/s 2024-05-09T06:00:23.225 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:21 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/870159446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T06:00:23.226 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:21 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2114336136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T06:00:23.226 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:21 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/3541244360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T06:00:23.226 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:22 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/2198706175' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T06:00:23.226 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:21 smithi195 ceph-mon[33844]: pgmap v703: 129 pgs: 9 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 95 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 188 B/s wr, 27 op/s 2024-05-09T06:00:23.226 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:21 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/870159446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T06:00:23.226 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:21 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2114336136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T06:00:23.226 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:21 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/3541244360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-09T06:00:23.226 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:22 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/2198706175' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T06:00:23.665 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T06:00:23.788 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:23 smithi115 ceph-mon[30349]: pgmap v704: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 22 op/s 2024-05-09T06:00:23.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:23 smithi195 ceph-mon[33844]: pgmap v704: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 22 op/s 2024-05-09T06:00:23.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:23 smithi161 ceph-mon[33805]: pgmap v704: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 22 op/s 2024-05-09T06:00:25.820 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:25 smithi115 ceph-mon[30349]: pgmap v705: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 170 B/s wr, 18 op/s 2024-05-09T06:00:25.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:25 smithi195 ceph-mon[33844]: pgmap v705: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 170 B/s wr, 18 op/s 2024-05-09T06:00:25.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:25 smithi161 ceph-mon[33805]: pgmap v705: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 170 B/s wr, 18 op/s 2024-05-09T06:00:27.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:27 smithi115 ceph-mon[30349]: pgmap v706: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-09T06:00:27.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:27 smithi195 ceph-mon[33844]: pgmap v706: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-09T06:00:27.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:27 smithi161 ceph-mon[33805]: pgmap v706: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-09T06:00:29.744 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:29 smithi115 ceph-mon[30349]: pgmap v707: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-09T06:00:29.744 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:29 smithi115 ceph-mon[30349]: osdmap e125: 12 total, 12 up, 12 in 2024-05-09T06:00:29.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:29 smithi195 ceph-mon[33844]: pgmap v707: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-09T06:00:29.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:29 smithi195 ceph-mon[33844]: osdmap e125: 12 total, 12 up, 12 in 2024-05-09T06:00:29.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:29 smithi161 ceph-mon[33805]: pgmap v707: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-09T06:00:29.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:29 smithi161 ceph-mon[33805]: osdmap e125: 12 total, 12 up, 12 in 2024-05-09T06:00:30.050 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T06:00:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:00:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 3139, 2024-05-09T06:00:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 3139, 2024-05-09T06:00:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2362883, 2024-05-09T06:00:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 997379, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 6, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 2, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.311296906, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.090320979, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.079594407, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.118858464, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3138, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.159194682, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000050731 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1846, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1778, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2382926, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 688110, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 5, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.201611568, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.064745972, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.080122816, 2024-05-09T06:00:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.058550660, 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1777, 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.101698979, 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000057230 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2060752, 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 272416 2024-05-09T06:00:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 4, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 329, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 748, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 813166, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 8, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.022210481, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.015971147, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.003329185, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000023212, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 326, 2024-05-09T06:00:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.032062027, 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000098349 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 812848 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7416, 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.207030345, 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000027916 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-05-09T06:00:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-05-09T06:00:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 305, 2024-05-09T06:00:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2024-05-09T06:00:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 305, 2024-05-09T06:00:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-05-09T06:00:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 304, 2024-05-09T06:00:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-05-09T06:00:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-05-09T06:00:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-05-09T06:00:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 1, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-05-09T06:00:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-05-09T06:00:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-05-09T06:00:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-05-09T06:00:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 668, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 2, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 5625090, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 5620606, 2024-05-09T06:00:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 668 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 136, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 369, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 233, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 69, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 89, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 20, 2024-05-09T06:00:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 305, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 602, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 297, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 124272, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215440 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-05-09T06:00:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-05-09T06:00:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-05-09T06:00:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-05-09T06:00:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-05-09T06:00:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-05-09T06:00:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 3, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 1, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 3, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-05-09T06:00:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 3153, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:00:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-05-09T06:00:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-05-09T06:00:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 190945, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 54, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:00:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9560, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 184, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 670976, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 19202, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 4443, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 21, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 3, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 4443, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4443, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 37.096368353, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.008349396 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-05-09T06:00:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4443, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4444 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 4443, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 4443, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 2794, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1647, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-05-09T06:00:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 1, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-05-09T06:00:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-05-09T06:00:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 125, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 38, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 5, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 7, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 2, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-05-09T06:00:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T06:00:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 4985, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 4267574, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 4985, 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4985, 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 4267574, 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4443, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 1430791, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4443, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1430791, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4443, 2024-05-09T06:00:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 4443, 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4443, 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 4443, 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55f518b288f0": { 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:30.097 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T06:00:31.055 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-09T06:00:31.055 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-09T06:00:31.055 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-09T06:00:31.055 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-09T06:00:31.055 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T06:00:31.056 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-05-09T06:00:31.056 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T06:00:31.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T06:00:31.327 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T06:00:31.730 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:31 smithi115 ceph-mon[30349]: pgmap v709: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-09T06:00:31.846 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:31 smithi195 ceph-mon[33844]: pgmap v709: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-09T06:00:31.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:31 smithi161 ceph-mon[33805]: pgmap v709: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-09T06:00:33.796 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:33 smithi115 ceph-mon[30349]: pgmap v710: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:33.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:33 smithi195 ceph-mon[33844]: pgmap v710: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:33.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:33 smithi161 ceph-mon[33805]: pgmap v710: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:35.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:35 smithi195 ceph-mon[33844]: pgmap v711: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:35.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:35 smithi115 ceph-mon[30349]: pgmap v711: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:35.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:35 smithi161 ceph-mon[33805]: pgmap v711: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:37.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:37 smithi195 ceph-mon[33844]: pgmap v712: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:37.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:37 smithi115 ceph-mon[30349]: pgmap v712: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:37.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:37 smithi161 ceph-mon[33805]: pgmap v712: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:38.435 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T06:00:38.435 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:00:38.435 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 0, 2024-05-09T06:00:38.435 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 2, 2024-05-09T06:00:38.435 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 0, 2024-05-09T06:00:38.435 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 0, 2024-05-09T06:00:38.435 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 4, 2024-05-09T06:00:38.435 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 0, 2024-05-09T06:00:38.435 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.001193224, 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.000284374, 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:38.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 7, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 335, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 3392, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 295072, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 6, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.023202602, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.016874721, 2024-05-09T06:00:38.437 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.002760792, 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 333, 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.029821003, 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000089552 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 295072 2024-05-09T06:00:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 446, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 436, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1625664, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 269712, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 2, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 1, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.070621232, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.025466116, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.045606888, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.008417003, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 435, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.017155335, 2024-05-09T06:00:38.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000039437 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1625664, 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 269712 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 0, 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-05-09T06:00:38.440 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:00:38.441 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:00:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 148754, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 31, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:00:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 0, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 0, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 928, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 2, 2024-05-09T06:00:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 449, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 1562651, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 449, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-05-09T06:00:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 449, 2024-05-09T06:00:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1562651, 2024-05-09T06:00:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:38.446 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T06:00:39.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:39 smithi195 ceph-mon[33844]: pgmap v713: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:39.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:39 smithi115 ceph-mon[30349]: pgmap v713: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:39.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:39 smithi161 ceph-mon[33805]: pgmap v713: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:41.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:41 smithi195 ceph-mon[33844]: pgmap v714: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T06:00:41.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:41 smithi115 ceph-mon[30349]: pgmap v714: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T06:00:41.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:41 smithi161 ceph-mon[33805]: pgmap v714: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T06:00:43.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:43 smithi195 ceph-mon[33844]: pgmap v715: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T06:00:43.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:43 smithi115 ceph-mon[30349]: pgmap v715: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T06:00:43.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:43 smithi161 ceph-mon[33805]: pgmap v715: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T06:00:45.359 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:45 smithi195 ceph-mon[33844]: pgmap v716: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T06:00:45.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:45 smithi115 ceph-mon[30349]: pgmap v716: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T06:00:45.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:45 smithi161 ceph-mon[33805]: pgmap v716: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T06:00:47.545 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T06:00:47.545 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T06:00:47.545 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-09T06:00:47.545 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-09T06:00:47.545 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-09T06:00:47.545 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-09T06:00:47.545 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T06:00:47.546 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T06:00:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T06:00:47.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:47 smithi195 ceph-mon[33844]: pgmap v717: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:47.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:47 smithi115 ceph-mon[30349]: pgmap v717: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:47.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:47 smithi161 ceph-mon[33805]: pgmap v717: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:49.706 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:49 smithi161 ceph-mon[33805]: pgmap v718: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-09T06:00:49.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:49 smithi195 ceph-mon[33844]: pgmap v718: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-09T06:00:49.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:49 smithi115 ceph-mon[30349]: pgmap v718: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-09T06:00:51.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:51 smithi115 ceph-mon[30349]: pgmap v719: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T06:00:51.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:51 smithi195 ceph-mon[33844]: pgmap v719: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T06:00:51.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:51 smithi161 ceph-mon[33805]: pgmap v719: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-09T06:00:52.031 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T06:00:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:00:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2050, 2024-05-09T06:00:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1979, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2571118, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 753722, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 5, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 1, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.253879470, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.096411751, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.094787231, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.060643479, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1979, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.137936143, 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000069699 2024-05-09T06:00:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2066192, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 277040 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2949, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 3279, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1348431, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 1670707, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 7, 2024-05-09T06:00:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.219821140, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.068473338, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.056406200, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.078243190, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3276, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.156182550, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000047674 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 733504 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 4, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 6, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1629, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 1284, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 7, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.001978197, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.000056469, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.001584060, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000077438, 2024-05-09T06:00:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4, 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.002186798, 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000546699 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-05-09T06:00:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7493, 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.196992801, 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000026290 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-05-09T06:00:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 23, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 23, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 23, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 1, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2024-05-09T06:00:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 0, 2024-05-09T06:00:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:00:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-05-09T06:00:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-05-09T06:00:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 378, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-05-09T06:00:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 2, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 4793405, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 4790800, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 378 2024-05-09T06:00:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 4, 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 143, 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 139, 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 2, 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 14, 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 12, 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 23, 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 162, 2024-05-09T06:00:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 139, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 121132, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215440 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-05-09T06:00:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-05-09T06:00:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-05-09T06:00:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-05-09T06:00:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:00:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 3, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 1, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 3, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2024-05-09T06:00:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 3174, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:00:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-05-09T06:00:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 199025, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 58, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-05-09T06:00:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9320, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 178, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-05-09T06:00:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 29920, 2024-05-09T06:00:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 124, 2024-05-09T06:00:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-05-09T06:00:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-05-09T06:00:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-05-09T06:00:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-05-09T06:00:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 4450, 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 21, 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 3, 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 4450, 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4450, 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 50.066147568, 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.011250819 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4450, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4451 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 4450, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 4450, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 2812, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1636, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2024-05-09T06:00:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 1, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-05-09T06:00:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-05-09T06:00:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 125, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 38, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 6, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 8, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 2, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-05-09T06:00:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2024-05-09T06:00:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-05-09T06:00:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2024-05-09T06:00:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-05-09T06:00:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2024-05-09T06:00:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:00:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-05-09T06:00:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 4999, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 3440627, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 4999, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4999, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3440627, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4450, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 599106, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4450, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 599106, 2024-05-09T06:00:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4450, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 4450, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4450, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 4450, 2024-05-09T06:00:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-05-09T06:00:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x563928504410": { 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-05-09T06:00:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-05-09T06:00:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:52.079 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T06:00:52.897 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-09T06:00:52.897 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-09T06:00:52.897 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-05-09T06:00:52.897 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-09T06:00:52.897 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T06:00:52.897 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-05-09T06:00:52.897 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T06:00:52.898 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T06:00:53.146 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T06:00:53.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:53 smithi115 ceph-mon[30349]: pgmap v720: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T06:00:53.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:53 smithi195 ceph-mon[33844]: pgmap v720: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T06:00:53.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:53 smithi161 ceph-mon[33805]: pgmap v720: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T06:00:54.940 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-09T06:00:54.940 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 452, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 442, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1627296, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 273552, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 4, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 0, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.074282505, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.028087711, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.045634246, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.008894988, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 441, 2024-05-09T06:00:54.941 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.022658577, 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000051379 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1627296, 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 273552 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 3, 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 338, 2024-05-09T06:00:54.942 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 480, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 295504, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 6, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.025063979, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.018945781, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.002400115, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 335, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.021525046, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000064253 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:54.943 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 295504 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 4, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 4, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2912, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 448, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 2, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 1, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.000697585, 2024-05-09T06:00:54.944 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.000029357, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.000628896, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000769836, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000256612 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-09T06:00:54.945 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 0, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:00:54.946 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:00:54.947 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 148721, 2024-05-09T06:00:54.948 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 30, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-05-09T06:00:54.949 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 928, 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 2, 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-05-09T06:00:54.950 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 455, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 1563431, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 455, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 455, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1563431, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:54.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-09T06:00:54.952 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-09T06:00:55.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:55 smithi115 ceph-mon[30349]: pgmap v721: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T06:00:55.729 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T06:00:55.729 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T06:00:55.730 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-09T06:00:55.730 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-09T06:00:55.730 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-09T06:00:55.730 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-09T06:00:55.730 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T06:00:55.730 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T06:00:55.806 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:55 smithi161 ceph-mon[33805]: pgmap v721: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T06:00:55.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:55 smithi195 ceph-mon[33844]: pgmap v721: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-09T06:00:56.038 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T06:00:57.600 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:57 smithi161 ceph-mon[33805]: pgmap v722: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:57.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:57 smithi195 ceph-mon[33844]: pgmap v722: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:57.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:57 smithi115 ceph-mon[30349]: pgmap v722: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:00:58.365 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 3623, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 3791, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1679826, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 2089919, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 23, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 8, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.285903191, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.106215529, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.175830633, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.020084570, 2024-05-09T06:00:58.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3789, 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.192982498, 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000050932 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6253, 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001476521, 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000236 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 4816, 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 26032 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 3415, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 4155, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1809279, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 3842692, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 28, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.294447528, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.097236866, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.166947000, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.039556615, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4153, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.236422965, 2024-05-09T06:00:58.368 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000056928 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5080, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001101200, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000216 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 5728, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 159920 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1156, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1360, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2575156, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1396182, 2024-05-09T06:00:58.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 10, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 7, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.143186617, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.054813540, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.073969904, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.021746112, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1359, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.082103682, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000060414 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 234, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000047850, 2024-05-09T06:00:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000204 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2083232, 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 953552 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1110, 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.493329447, 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000444440 2024-05-09T06:00:58.371 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 652, 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.041673870, 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000063916 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-09T06:00:58.372 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 700, 2024-05-09T06:00:58.373 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 696, 2024-05-09T06:00:58.373 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-09T06:00:58.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 696, 2024-05-09T06:00:58.373 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 29.980834315, 2024-05-09T06:00:58.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.043075911 2024-05-09T06:00:58.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.373 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-09T06:00:58.373 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 4, 2024-05-09T06:00:58.373 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 221, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 1, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 1, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 219, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 219, 2024-05-09T06:00:58.374 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 296, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 2, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 2, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 4, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 1239, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 1095, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 1, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 11, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-09T06:00:58.375 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 6, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 276, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 8, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 185, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 725, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 269, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 7409369197, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 725, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:00:58.376 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 335, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 335, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 390, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 6, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 491, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 335, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 691, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 180, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 4, 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 334 2024-05-09T06:00:58.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 19, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 10, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 67, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 165, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 98, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-09T06:00:58.378 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 15, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 2185, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 6, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-09T06:00:58.379 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 668, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 668, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 2, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-05-09T06:00:58.380 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 5625090, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 307, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 23.655746630, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.077054549 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.381 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 153, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 378, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 225, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 87, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 99, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 12, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 221, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 616, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 395, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 2, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 175, 2024-05-09T06:00:58.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 173, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 131312, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223632 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 1204, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 105, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 700, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 36, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 521, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.012486948, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004162316 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.383 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 214, 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 17.983873590, 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.084036792 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.385 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-09T06:00:58.385 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 165, 2024-05-09T06:00:58.385 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 10.982392541, 2024-05-09T06:00:58.385 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.066559954 2024-05-09T06:00:58.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.385 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-09T06:00:58.385 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.385 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.386 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-09T06:00:58.387 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.388 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.388 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.388 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-09T06:00:58.388 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.388 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 76, 2024-05-09T06:00:58.389 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.058907837, 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000775103 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 73, 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.070152118, 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000960987 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.390 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 67, 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.685553616, 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.010232143 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.391 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-09T06:00:58.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-09T06:00:58.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 98, 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.187467665, 2024-05-09T06:00:58.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001912935 2024-05-09T06:00:58.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.395 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:00:58.395 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:00:58.395 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-09T06:00:58.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.395 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 532, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 266, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 1340, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.396 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:00:58.397 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:00:58.398 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:00:58.399 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:00:58.400 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 833607, 2024-05-09T06:00:58.400 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 244, 2024-05-09T06:00:58.400 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:00:58.400 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-09T06:00:58.400 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-09T06:00:58.400 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9320, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 178, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 693813, 2024-05-09T06:00:58.401 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 20951, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 1828, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 1481941, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 28, 2024-05-09T06:00:58.402 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 1828, 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1828, 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 140.723140993, 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.076982024 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1828, 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1934 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.403 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 1828, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 729, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 1099, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 4, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 311, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 47, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 640, 2024-05-09T06:00:58.404 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 725, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-09T06:00:58.405 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-09T06:00:58.406 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 125, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 38, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-09T06:00:58.407 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 138, 2024-05-09T06:00:58.408 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-05-09T06:00:58.408 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 39 2024-05-09T06:00:58.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.408 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-09T06:00:58.408 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-05-09T06:00:58.408 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:00:58.408 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-09T06:00:58.408 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 2294 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 34, 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 5, 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 628, 2024-05-09T06:00:58.409 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 98, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 8190, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 5293359, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 8190, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 8190, 2024-05-09T06:00:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 5293359, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 1828, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 1497327, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1828, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1497327, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:00:58.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 1828, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 1828, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1828, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1828, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 98, 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 15386, 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 98, 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:00:58.414 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4, 2024-05-09T06:00:58.415 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 15386, 2024-05-09T06:00:58.415 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:00:58.415 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.415 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.415 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.415 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:00:58.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:00:58.415 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x563f0d104680": { 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 668, 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1430786, 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 668, 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:00:58.416 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 307, 2024-05-09T06:00:58.417 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1430786, 2024-05-09T06:00:58.417 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:00:58.417 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:00:58.417 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:00:58.417 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:00:58.417 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:00:58.417 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:00:58.417 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-09T06:00:59.378 INFO:tasks.check_counter:Daemon mds.b mds.exported=6 2024-05-09T06:00:59.378 INFO:tasks.check_counter:Daemon mds.b mds.imported=8 2024-05-09T06:00:59.378 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-09T06:00:59.378 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-05-09T06:00:59.378 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T06:00:59.378 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-05-09T06:00:59.378 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T06:00:59.378 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T06:00:59.628 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T06:00:59.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:00:59 smithi161 ceph-mon[33805]: pgmap v723: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-09T06:00:59.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:00:59 smithi195 ceph-mon[33844]: pgmap v723: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-09T06:00:59.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:00:59 smithi115 ceph-mon[30349]: pgmap v723: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-09T06:01:01.724 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:01 smithi161 ceph-mon[33805]: pgmap v724: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T06:01:01.760 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:01 smithi115 ceph-mon[30349]: pgmap v724: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T06:01:01.807 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-09T06:01:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:01:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4746, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 4746, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 13917304, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1508411, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.397795497, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.117376891, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.117686813, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.134133114, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4745, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.169623569, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000035747 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 979, 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 911, 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3429949, 2024-05-09T06:01:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 416280, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.126591138, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.039252840, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.067473481, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.026468656, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 910, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.048140751, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000052901 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2064912, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 280432 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 339, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 748, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 824622, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-05-09T06:01:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.019542827, 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.013080269, 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.003013070, 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000059967, 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 336, 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.027367575, 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000081451 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T06:01:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 824304 2024-05-09T06:01:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-09T06:01:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-09T06:01:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-09T06:01:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-09T06:01:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-09T06:01:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-09T06:01:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8781, 2024-05-09T06:01:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.209357559, 2024-05-09T06:01:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000023842 2024-05-09T06:01:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-09T06:01:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-09T06:01:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-09T06:01:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-09T06:01:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-05-09T06:01:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-05-09T06:01:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-09T06:01:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-09T06:01:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-05-09T06:01:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-09T06:01:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-09T06:01:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-05-09T06:01:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-09T06:01:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-09T06:01:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 839, 2024-05-09T06:01:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 169, 2024-05-09T06:01:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 670, 2024-05-09T06:01:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-05-09T06:01:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 598, 2024-05-09T06:01:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-05-09T06:01:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-05-09T06:01:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-05-09T06:01:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-05-09T06:01:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-05-09T06:01:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-05-09T06:01:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-05-09T06:01:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-05-09T06:01:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-09T06:01:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-09T06:01:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-09T06:01:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-09T06:01:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-09T06:01:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-05-09T06:01:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-05-09T06:01:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-05-09T06:01:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-05-09T06:01:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-09T06:01:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-05-09T06:01:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-09T06:01:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-05-09T06:01:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-05-09T06:01:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-05-09T06:01:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T06:01:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-09T06:01:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:01:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:01:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-05-09T06:01:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T06:01:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T06:01:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-05-09T06:01:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-05-09T06:01:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T06:01:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-05-09T06:01:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T06:01:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:01:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:01:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-05-09T06:01:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-05-09T06:01:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-05-09T06:01:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-05-09T06:01:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-09T06:01:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-05-09T06:01:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-05-09T06:01:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-05-09T06:01:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-05-09T06:01:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-05-09T06:01:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-05-09T06:01:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-09T06:01:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:01:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-05-09T06:01:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-05-09T06:01:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-09T06:01:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-09T06:01:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-09T06:01:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:01:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:01:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-09T06:01:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-09T06:01:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-05-09T06:01:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-05-09T06:01:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T06:01:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-05-09T06:01:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-09T06:01:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-09T06:01:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-09T06:01:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 2101, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-05-09T06:01:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 6, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 18033851, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 18031763, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 2101 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 666, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 1031, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 365, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 26, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 117, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 91, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 839, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 1182, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 343, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-05-09T06:01:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 125976, 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215440 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-05-09T06:01:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-09T06:01:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-09T06:01:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 2604, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:01:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:01:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 190968, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 55, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-09T06:01:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9320, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 178, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 2041224, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 5681, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-09T06:01:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-09T06:01:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-09T06:01:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-09T06:01:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-09T06:01:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-09T06:01:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-09T06:01:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 5170, 2024-05-09T06:01:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-05-09T06:01:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 5170, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5170, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 10.586777042, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002047732 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5170, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 5171 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 5170, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 5170, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-09T06:01:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 3008, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 2160, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-05-09T06:01:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:01:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-09T06:01:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-09T06:01:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-09T06:01:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-09T06:01:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-09T06:01:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-09T06:01:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-09T06:01:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-09T06:01:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-09T06:01:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-09T06:01:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-09T06:01:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-09T06:01:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-09T06:01:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-09T06:01:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-09T06:01:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-09T06:01:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-09T06:01:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 125, 2024-05-09T06:01:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-09T06:01:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 38, 2024-05-09T06:01:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 5, 2024-05-09T06:01:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 6, 2024-05-09T06:01:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 1, 2024-05-09T06:01:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-09T06:01:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-05-09T06:01:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-05-09T06:01:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-05-09T06:01:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-09T06:01:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-05-09T06:01:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:01:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-09T06:01:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-09T06:01:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-09T06:01:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-05-09T06:01:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-09T06:01:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-09T06:01:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T06:01:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-09T06:01:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-05-09T06:01:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-05-09T06:01:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-05-09T06:01:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-09T06:01:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:01:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-09T06:01:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 5725, 2024-05-09T06:01:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 16801755, 2024-05-09T06:01:01.857 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 5725, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 5725, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 16801755, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:01.858 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 5170, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 13842275, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 5170, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 13842275, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:01.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 5170, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 5170, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 5170, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 5170, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.860 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-09T06:01:01.861 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5584a59428f0": { 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:01.862 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:01.863 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-09T06:01:01.864 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:01 smithi195 ceph-mon[33844]: pgmap v724: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-09T06:01:02.694 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T06:01:02.694 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T06:01:02.694 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-05-09T06:01:02.694 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-09T06:01:02.694 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T06:01:02.694 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-09T06:01:02.694 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T06:01:02.694 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T06:01:02.941 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T06:01:03.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:03 smithi161 ceph-mon[33805]: pgmap v725: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T06:01:03.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:03 smithi195 ceph-mon[33844]: pgmap v725: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T06:01:03.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:03 smithi115 ceph-mon[30349]: pgmap v725: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T06:01:04.770 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 535, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 812, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 393895, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1612858, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 13, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.074118393, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.027957220, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.021853018, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.016084089, 2024-05-09T06:01:04.771 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 810, 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.087036562, 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000107452 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 240, 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000069162, 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000288 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 581104 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:01:04.772 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1417, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1278, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2749166, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1184224, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 39, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.169095681, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.052992131, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.109905546, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.021252365, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1277, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.090250473, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000070673 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.773 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1037, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000237986, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000229 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2083792, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 300080 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 2442, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2857, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1348725, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1972592, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 14, 2024-05-09T06:01:04.774 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.249459521, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.089755323, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.149441269, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.022170051, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2855, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.156661891, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000054872 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4272, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001249537, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000292 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.775 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 156336 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 510, 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.112233981, 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000220066 2024-05-09T06:01:04.776 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 593, 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.026333559, 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000044407 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-09T06:01:04.777 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 189, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 189, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 189, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 9.157255977, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.048451089 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-09T06:01:04.778 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 11, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 1, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 10, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 10, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 140, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 2, 2024-05-09T06:01:04.779 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 338, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 294, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 5, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 2, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 129, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 2, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 38, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-05-09T06:01:04.780 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 269, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 7409369197, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 6, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 130, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 100, 2024-05-09T06:01:04.781 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 304, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 137, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 1, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 45 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-05-09T06:01:04.782 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 8, 2024-05-09T06:01:04.783 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 22, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 22, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:01:04.784 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 15, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 207, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 2, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.785 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 378, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 378, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 2, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-05-09T06:01:04.786 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4793405, 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 191, 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 6.018545987, 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.031510711 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-05-09T06:01:04.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 16, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 154, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 138, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 13, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 26, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 13, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 11, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 173, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 162, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 24, 2024-05-09T06:01:04.788 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 24, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 125800, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223632 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 238, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 9, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 189, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 36, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 120, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.009773861, 2024-05-09T06:01:04.789 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004886930 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.790 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 84, 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 9.040035434, 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.107619469 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.791 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.792 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.793 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.794 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.794 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.794 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-09T06:01:04.794 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.794 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 78, 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.064220866, 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000823344 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001817489, 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000605829 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.796 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.797 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.798 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.799 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 22, 2024-05-09T06:01:04.801 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.041408327, 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001882196 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-09T06:01:04.802 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 52, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 26, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 1346, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-09T06:01:04.803 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:01:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:01:04.805 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 304999, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 111, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-09T06:01:04.806 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9320, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 178, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 65952, 2024-05-09T06:01:04.807 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 259, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 917, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 624815, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 33, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 917, 2024-05-09T06:01:04.808 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-09T06:01:04.809 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 917, 2024-05-09T06:01:04.809 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 132.206335599, 2024-05-09T06:01:04.809 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.144172666 2024-05-09T06:01:04.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.809 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-09T06:01:04.809 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-09T06:01:04.809 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 917, 2024-05-09T06:01:04.809 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1020 2024-05-09T06:01:04.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 917, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 1, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 916, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 192, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 39, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-09T06:01:04.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 584, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:01:04.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-09T06:01:04.812 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 125, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-09T06:01:04.813 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 38, 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 18, 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 6, 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 140, 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 34 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-05-09T06:01:04.814 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 229 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 26, 2024-05-09T06:01:04.815 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 1, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 572, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 22, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 4390, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 4063465, 2024-05-09T06:01:04.816 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 4390, 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4390, 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 4063465, 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-09T06:01:04.817 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:04.818 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-09T06:01:04.818 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:04.818 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:04.818 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:04.818 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:04.818 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:04.818 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 917, 2024-05-09T06:01:04.818 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 628269, 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 917, 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 628269, 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:04.819 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 917, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 917, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 917, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 917, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:04.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 22, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 3454, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 22, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 3454, 2024-05-09T06:01:04.821 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55df0d5ea410": { 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:04.822 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 378, 2024-05-09T06:01:04.823 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 599101, 2024-05-09T06:01:04.823 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:04.823 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 378, 2024-05-09T06:01:04.823 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:01:04.823 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:01:04.823 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 191, 2024-05-09T06:01:04.823 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 599101, 2024-05-09T06:01:04.823 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:04.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:04.824 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:04.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:04.824 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:04.824 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:04.824 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-09T06:01:05.569 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:05 smithi161 ceph-mon[33805]: pgmap v726: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-09T06:01:05.598 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-05-09T06:01:05.598 INFO:tasks.check_counter:Daemon mds.h mds.imported=2 2024-05-09T06:01:05.598 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-09T06:01:05.598 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-09T06:01:05.598 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T06:01:05.598 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-09T06:01:05.598 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T06:01:05.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T06:01:05.844 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T06:01:05.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:05 smithi195 ceph-mon[33844]: pgmap v726: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-09T06:01:05.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:05 smithi115 ceph-mon[30349]: pgmap v726: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-09T06:01:07.568 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:07 smithi161 ceph-mon[33805]: pgmap v727: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-09T06:01:07.613 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-09T06:01:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:01:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 558, 2024-05-09T06:01:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 490, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2067106, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 283956, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.090556998, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.030082779, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.057925363, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.011506336, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 489, 2024-05-09T06:01:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.023686016, 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000048437 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2066272, 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 283632 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1362, 2024-05-09T06:01:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1700, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 334902, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1170629, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.148551638, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.050463867, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.032228829, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.050298811, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1697, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.076191697, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000044897 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 738144 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 3357, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 3357, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3270057, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1066760, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-05-09T06:01:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.359095411, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.117383584, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.093289673, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.119739151, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3356, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.155078831, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000046209 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 7945, 2024-05-09T06:01:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.241115070, 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000030348 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-05-09T06:01:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-09T06:01:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 40, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 40, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 40, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-09T06:01:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-09T06:01:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-09T06:01:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-09T06:01:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-09T06:01:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-05-09T06:01:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-05-09T06:01:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-05-09T06:01:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-05-09T06:01:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-09T06:01:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T06:01:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-05-09T06:01:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-09T06:01:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-05-09T06:01:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 1305, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 3, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 6472848, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 6470243, 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-09T06:01:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 1305 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 4, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 431, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 427, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 2, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 45, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 43, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 40, 2024-05-09T06:01:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 503, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 463, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 122352, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215440 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-05-09T06:01:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-09T06:01:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-09T06:01:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-09T06:01:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-09T06:01:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-05-09T06:01:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 3, 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 3232, 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-09T06:01:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:01:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-09T06:01:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 190961, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 55, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:01:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9320, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 178, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 41888, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 192, 2024-05-09T06:01:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 4714, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 4714, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4714, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 21.680542509, 2024-05-09T06:01:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004599181 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4714, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4715 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 4714, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 4714, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 2902, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-05-09T06:01:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1810, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-09T06:01:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-09T06:01:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 125, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-09T06:01:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 38, 2024-05-09T06:01:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 5, 2024-05-09T06:01:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 6, 2024-05-09T06:01:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 1, 2024-05-09T06:01:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-09T06:01:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-05-09T06:01:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-05-09T06:01:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-05-09T06:01:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-05-09T06:01:07.655 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-05-09T06:01:07.656 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-05-09T06:01:07.656 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-09T06:01:07.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.656 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:01:07.656 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:07.656 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-09T06:01:07.656 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:07.656 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 5273, 2024-05-09T06:01:07.656 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 5166282, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 5273, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 5273, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 5166282, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:07.657 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-09T06:01:07.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:07.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:07.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:07.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:07.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:07.658 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 4714, 2024-05-09T06:01:07.658 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2278549, 2024-05-09T06:01:07.658 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4714, 2024-05-09T06:01:07.658 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2278549, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:07.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:07.660 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 4714, 2024-05-09T06:01:07.660 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 4714, 2024-05-09T06:01:07.660 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4714, 2024-05-09T06:01:07.660 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 4714, 2024-05-09T06:01:07.660 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:07.660 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.660 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.660 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:01:07.661 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-09T06:01:07.662 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-09T06:01:07.662 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:07.662 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.662 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.662 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.662 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:07.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-09T06:01:07.662 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5648e48b88f0": { 2024-05-09T06:01:07.662 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-09T06:01:07.663 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-09T06:01:07.664 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-09T06:01:07.664 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-09T06:01:07.664 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-09T06:01:07.664 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:07.664 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-09T06:01:07.664 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-09T06:01:07.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:07 smithi195 ceph-mon[33844]: pgmap v727: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-09T06:01:07.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:07 smithi115 ceph-mon[30349]: pgmap v727: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-09T06:01:08.470 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-09T06:01:08.470 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-09T06:01:08.470 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-05-09T06:01:08.470 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-09T06:01:08.470 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T06:01:08.470 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-05-09T06:01:08.470 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T06:01:08.470 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T06:01:08.804 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T06:01:09.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:09 smithi195 ceph-mon[33844]: pgmap v728: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-09T06:01:09.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:09 smithi115 ceph-mon[30349]: pgmap v728: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-09T06:01:09.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:09 smithi161 ceph-mon[33805]: pgmap v728: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-09T06:01:10.857 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:10 smithi195 ceph-mon[33844]: osdmap e126: 12 total, 12 up, 12 in 2024-05-09T06:01:10.857 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:10 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:01:10.858 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-09T06:01:10.858 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:01:10.858 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 2484, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 2414, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1238887, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 2188374, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 11, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.217631594, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.081454079, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.098828095, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.040512709, 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:10.859 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2412, 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.134048757, 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000055575 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2254, 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000544431, 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000241 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:01:10.860 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 6393, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 6901, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 3172447, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 16834092, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 11, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 6, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.580064217, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.250049781, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.285368881, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.065134444, 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:10.861 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 6900, 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.356933099, 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000051729 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9057, 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.002296198, 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000253 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 741472 2024-05-09T06:01:10.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 3908, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 5668, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 3215018, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 2730994, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 37, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 8, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.443296115, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.154979705, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.305317680, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.021040498, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5666, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.226462952, 2024-05-09T06:01:10.863 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000039968 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 7459, 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001917962, 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000257 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 2100208, 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 297696 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.864 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3397, 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.972945900, 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000286413 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 869, 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.038544321, 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000044354 2024-05-09T06:01:10.865 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 1086, 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 851, 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 851, 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 42.070615585, 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.049436681 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 235, 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 4, 2024-05-09T06:01:10.866 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 20, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 1, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 804, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 161, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 628, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 15, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 557, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 358, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 2, 2024-05-09T06:01:10.867 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 2, 2024-05-09T06:01:10.868 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 6, 2024-05-09T06:01:10.868 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 1843, 2024-05-09T06:01:10.868 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 1128, 2024-05-09T06:01:10.868 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 193, 2024-05-09T06:01:10.868 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 20, 2024-05-09T06:01:10.868 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 4, 2024-05-09T06:01:10.868 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-05-09T06:01:10.868 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-05-09T06:01:10.868 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-05-09T06:01:10.869 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-05-09T06:01:10.869 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 15, 2024-05-09T06:01:10.869 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 328, 2024-05-09T06:01:10.869 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 38, 2024-05-09T06:01:10.869 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 786, 2024-05-09T06:01:10.869 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-05-09T06:01:10.869 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 546, 2024-05-09T06:01:10.870 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-05-09T06:01:10.870 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 269, 2024-05-09T06:01:10.870 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 7409369197, 2024-05-09T06:01:10.870 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 6, 2024-05-09T06:01:10.870 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 546, 2024-05-09T06:01:10.870 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-05-09T06:01:10.871 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:01:10.871 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:01:10.871 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 103, 2024-05-09T06:01:10.871 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T06:01:10.871 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 103, 2024-05-09T06:01:10.871 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 443, 2024-05-09T06:01:10.871 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 24, 2024-05-09T06:01:10.872 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 430, 2024-05-09T06:01:10.872 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 1249, 2024-05-09T06:01:10.872 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 1, 2024-05-09T06:01:10.872 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:01:10.872 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:01:10.872 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 1854, 2024-05-09T06:01:10.873 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 1298, 2024-05-09T06:01:10.873 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 6, 2024-05-09T06:01:10.873 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 270 2024-05-09T06:01:10.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.873 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-05-09T06:01:10.873 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-05-09T06:01:10.873 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-05-09T06:01:10.874 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-05-09T06:01:10.874 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 24, 2024-05-09T06:01:10.874 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 38, 2024-05-09T06:01:10.874 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 2, 2024-05-09T06:01:10.874 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-05-09T06:01:10.874 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:01:10.875 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 50, 2024-05-09T06:01:10.875 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 48, 2024-05-09T06:01:10.875 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-05-09T06:01:10.875 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-05-09T06:01:10.875 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-05-09T06:01:10.875 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:01:10.875 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:01:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-05-09T06:01:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-05-09T06:01:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 15, 2024-05-09T06:01:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 1010, 2024-05-09T06:01:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T06:01:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 15, 2024-05-09T06:01:10.877 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-05-09T06:01:10.877 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 1, 2024-05-09T06:01:10.877 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-05-09T06:01:10.877 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-05-09T06:01:10.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.877 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-05-09T06:01:10.878 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-05-09T06:01:10.878 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 2101, 2024-05-09T06:01:10.878 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-05-09T06:01:10.878 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-05-09T06:01:10.878 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 2101, 2024-05-09T06:01:10.878 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-05-09T06:01:10.879 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-05-09T06:01:10.879 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 6, 2024-05-09T06:01:10.879 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-05-09T06:01:10.879 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-05-09T06:01:10.879 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 6, 2024-05-09T06:01:10.879 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 2, 2024-05-09T06:01:10.880 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-05-09T06:01:10.880 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-05-09T06:01:10.880 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-05-09T06:01:10.880 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 18033851, 2024-05-09T06:01:10.880 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 0, 2024-05-09T06:01:10.880 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-05-09T06:01:10.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1369, 2024-05-09T06:01:10.881 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 14.079744633, 2024-05-09T06:01:10.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.010284692 2024-05-09T06:01:10.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.881 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 0 2024-05-09T06:01:10.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.882 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-05-09T06:01:10.882 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 683, 2024-05-09T06:01:10.882 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 1047, 2024-05-09T06:01:10.882 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 364, 2024-05-09T06:01:10.882 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 47, 2024-05-09T06:01:10.882 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 132, 2024-05-09T06:01:10.883 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 85, 2024-05-09T06:01:10.883 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 804, 2024-05-09T06:01:10.883 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 1210, 2024-05-09T06:01:10.883 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 406, 2024-05-09T06:01:10.883 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 2, 2024-05-09T06:01:10.884 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 319, 2024-05-09T06:01:10.884 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 317, 2024-05-09T06:01:10.884 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 131272, 2024-05-09T06:01:10.884 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 223632 2024-05-09T06:01:10.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.884 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-05-09T06:01:10.885 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 1275, 2024-05-09T06:01:10.885 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 252, 2024-05-09T06:01:10.885 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 1086, 2024-05-09T06:01:10.885 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 34, 2024-05-09T06:01:10.885 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 808, 2024-05-09T06:01:10.885 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-05-09T06:01:10.885 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 126, 2024-05-09T06:01:10.886 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.182109989, 2024-05-09T06:01:10.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001445317 2024-05-09T06:01:10.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.886 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-05-09T06:01:10.886 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 8, 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.003001005, 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000375125 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-05-09T06:01:10.887 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.888 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.888 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.888 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-05-09T06:01:10.888 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 478, 2024-05-09T06:01:10.888 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 29.381663257, 2024-05-09T06:01:10.888 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.061467914 2024-05-09T06:01:10.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.889 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-05-09T06:01:10.889 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.889 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.889 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.889 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-05-09T06:01:10.889 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.889 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.890 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.890 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-05-09T06:01:10.890 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.890 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.890 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.890 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-05-09T06:01:10.891 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.891 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.891 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.891 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-05-09T06:01:10.891 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.891 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.891 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.892 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-05-09T06:01:10.892 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:10 smithi115 ceph-mon[30349]: osdmap e126: 12 total, 12 up, 12 in 2024-05-09T06:01:10.893 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:10 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:01:10.893 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.893 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.893 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-05-09T06:01:10.893 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.893 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.894 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 180, 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 12.383791602, 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.068798842 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 6, 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.004864023, 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000810670 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 18, 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.052509602, 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002917200 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.895 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2, 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.014330935, 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.007165467 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.896 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1, 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000976695, 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000976695 2024-05-09T06:01:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2, 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000609275, 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000304637 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-05-09T06:01:10.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.899 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.900 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.900 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-05-09T06:01:10.900 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 30, 2024-05-09T06:01:10.900 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.046759202, 2024-05-09T06:01:10.900 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001558640 2024-05-09T06:01:10.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 1, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 2, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 1, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 1, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-05-09T06:01:10.901 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 85, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 85, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 1379, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:01:10.902 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:01:10.903 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:01:10.904 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 336455, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 119, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9320, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 178, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-05-09T06:01:10.905 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 2107397, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 8065, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 3207, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 14021486, 2024-05-09T06:01:10.906 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 28, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 3207, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3207, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 193.430577004, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.060315115 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3207, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 3597 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 3207, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 558, 2024-05-09T06:01:10.907 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 2649, 2024-05-09T06:01:10.908 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-05-09T06:01:10.908 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-05-09T06:01:10.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 0, 2024-05-09T06:01:10.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 10, 2024-05-09T06:01:10.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 8, 2024-05-09T06:01:10.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 1377, 2024-05-09T06:01:10.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 81, 2024-05-09T06:01:10.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-05-09T06:01:10.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-05-09T06:01:10.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-05-09T06:01:10.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 1, 2024-05-09T06:01:10.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 831, 2024-05-09T06:01:10.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-05-09T06:01:10.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-05-09T06:01:10.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-05-09T06:01:10.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 546, 2024-05-09T06:01:10.909 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 20, 2024-05-09T06:01:10.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:01:10.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-05-09T06:01:10.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:01:10.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-05-09T06:01:10.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-05-09T06:01:10.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-05-09T06:01:10.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:01:10.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-05-09T06:01:10.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:01:10.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 1, 2024-05-09T06:01:10.911 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-05-09T06:01:10.911 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-05-09T06:01:10.911 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-05-09T06:01:10.911 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-05-09T06:01:10.911 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-05-09T06:01:10.911 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-05-09T06:01:10.911 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-05-09T06:01:10.912 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-05-09T06:01:10.912 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-05-09T06:01:10.912 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-05-09T06:01:10.912 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-05-09T06:01:10.912 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-05-09T06:01:10.912 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-05-09T06:01:10.913 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-05-09T06:01:10.913 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-05-09T06:01:10.913 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-05-09T06:01:10.913 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 126, 2024-05-09T06:01:10.913 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-05-09T06:01:10.913 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 39, 2024-05-09T06:01:10.913 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 12, 2024-05-09T06:01:10.914 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 12, 2024-05-09T06:01:10.914 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-05-09T06:01:10.914 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-05-09T06:01:10.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 549, 2024-05-09T06:01:10.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 8, 2024-05-09T06:01:10.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 165 2024-05-09T06:01:10.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.915 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-05-09T06:01:10.915 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 1, 2024-05-09T06:01:10.915 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:01:10.915 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-05-09T06:01:10.915 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-05-09T06:01:10.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.915 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-05-09T06:01:10.915 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 1085 2024-05-09T06:01:10.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.916 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-05-09T06:01:10.916 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-05-09T06:01:10.916 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 26, 2024-05-09T06:01:10.916 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-05-09T06:01:10.916 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 5, 2024-05-09T06:01:10.916 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 816, 2024-05-09T06:01:10.916 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 48, 2024-05-09T06:01:10.917 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-05-09T06:01:10.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.917 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:01:10.917 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:10.917 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-09T06:01:10.917 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:10.917 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 12781, 2024-05-09T06:01:10.918 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 6442728, 2024-05-09T06:01:10.918 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:10.918 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 12781, 2024-05-09T06:01:10.918 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:10.918 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:10.918 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 12781, 2024-05-09T06:01:10.919 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 6442728, 2024-05-09T06:01:10.919 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:10.919 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.919 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.919 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.919 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:10.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.920 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-05-09T06:01:10.920 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:10.920 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-09T06:01:10.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:10.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:10.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:10.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:10.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:10.920 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 3207, 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 14028094, 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 3207, 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 14028094, 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-05-09T06:01:10.921 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 3207, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 3207, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 3207, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 3207, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.922 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 48, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 6672, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 48, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 8, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 6672, 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:10.923 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x562d286a6410": { 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 2101, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 13839547, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 2101, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:10.924 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 1369, 2024-05-09T06:01:10.925 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 13839547, 2024-05-09T06:01:10.925 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:10.925 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:10.925 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:10.925 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:10.925 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:10.925 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:10.925 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-09T06:01:10.925 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:10 smithi161 ceph-mon[33805]: osdmap e126: 12 total, 12 up, 12 in 2024-05-09T06:01:10.926 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:10 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T06:01:11.789 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:11 smithi195 ceph-mon[33844]: pgmap v730: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-09T06:01:11.853 INFO:tasks.check_counter:Daemon mds.c mds.exported=15 2024-05-09T06:01:11.853 INFO:tasks.check_counter:Daemon mds.c mds.imported=38 2024-05-09T06:01:11.853 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-09T06:01:11.853 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-09T06:01:11.853 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T06:01:11.853 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=1 2024-05-09T06:01:11.853 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T06:01:11.853 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T06:01:11.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:11 smithi115 ceph-mon[30349]: pgmap v730: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-09T06:01:11.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:11 smithi161 ceph-mon[33805]: pgmap v730: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-09T06:01:12.406 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T06:01:12.807 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:12 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:01:12.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:12 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:01:12.808 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:12 smithi195 ceph-mon[33844]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T06:01:12.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:12 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:01:12.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:12 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:01:12.890 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:12 smithi115 ceph-mon[30349]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T06:01:12.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:12 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T06:01:12.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:12 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T06:01:12.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:12 smithi161 ceph-mon[33805]: from='mgr.15327 172.21.15.115:0/1240650922' entity='mgr.x' 2024-05-09T06:01:13.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:13 smithi195 ceph-mon[33844]: pgmap v731: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-09T06:01:13.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:13 smithi115 ceph-mon[30349]: pgmap v731: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-09T06:01:13.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:13 smithi161 ceph-mon[33805]: pgmap v731: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-09T06:01:15.181 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-09T06:01:15.181 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:01:15.181 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 3351, 2024-05-09T06:01:15.181 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 4122, 2024-05-09T06:01:15.181 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1560736, 2024-05-09T06:01:15.181 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 4599604, 2024-05-09T06:01:15.181 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 17, 2024-05-09T06:01:15.181 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-05-09T06:01:15.181 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.317758931, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.096845123, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.181054751, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.042359269, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4122, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.275326873, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000066794 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4767, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001399943, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000293 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.182 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1905, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 2311, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1043313, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 2244212, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 30, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 5, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.215129707, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.093585609, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.100596271, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.025601573, 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:15.183 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2311, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.208540855, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000090238 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2333, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000554342, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000237 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 4320, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 833024 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 3197, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 3339, 2024-05-09T06:01:15.184 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 3267230, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 1911133, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 14, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 9, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.358252864, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.152793832, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.200266758, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.028204289, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3334, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.154034947, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000046201 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4567, 2024-05-09T06:01:15.185 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001477618, 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000323 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 2095920, 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 329648 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1291, 2024-05-09T06:01:15.186 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.441809609, 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000342222 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1058, 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.046116986, 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000043588 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 537, 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 511, 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-05-09T06:01:15.187 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 511, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 42.939123925, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.084029596 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 26, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 2, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 27, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 2, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 13, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 1, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 1, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 11, 2024-05-09T06:01:15.188 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 11, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 466, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 1, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 1, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 4, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 878, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 708, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 26, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 6, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 20, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 447, 2024-05-09T06:01:15.189 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 6, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 208, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 149, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 269, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 7409369197, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 6, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 149, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 19, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 19, 2024-05-09T06:01:15.190 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 130, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 6, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 336, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 374, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 907, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 500, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 3, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 91 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-05-09T06:01:15.191 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 14, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 30, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 39, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 39, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 15, 2024-05-09T06:01:15.192 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 1146, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 21, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 2, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 1305, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 1305, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-05-09T06:01:15.193 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 3, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 3, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 2, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 6472848, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 0, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 496, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 8.648772847, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.017437042 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.194 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 0 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 19, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 444, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 425, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 17, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 59, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 42, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 13, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 518, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 505, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 1, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 182, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 181, 2024-05-09T06:01:15.195 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 131292, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 223632 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 670, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 175, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 537, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 41, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 608, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 14, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.042099622, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.003007115 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1, 2024-05-09T06:01:15.196 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000478691, 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000478691 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 220, 2024-05-09T06:01:15.197 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 35.841726296, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.162916937 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.198 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.199 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.200 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.200 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-05-09T06:01:15.200 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.200 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.200 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 209, 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 6.990574908, 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.033447726 2024-05-09T06:01:15.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 28, 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.010125127, 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000361611 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.205 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 39, 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.054119281, 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001387673 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.206 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 3, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 1, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 2, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 123, 2024-05-09T06:01:15.207 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 61, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 1355, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:01:15.208 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-05-09T06:01:15.209 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 288887, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 104, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-05-09T06:01:15.210 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9560, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 184, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 80440, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 302, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-05-09T06:01:15.211 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 1941, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 2439973, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 33, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 1941, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1941, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 180.428836862, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.092956639 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-05-09T06:01:15.212 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1941, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2173 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 1941, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 152, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 1789, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 0, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 10, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 1, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 497, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 58, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-05-09T06:01:15.213 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 1028, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 149, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 20, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:01:15.214 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-05-09T06:01:15.215 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 126, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 39, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 12, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 12, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 326, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 4, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 80 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-05-09T06:01:15.216 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 1, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 1334 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 26, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 1, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 1014, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 39, 2024-05-09T06:01:15.217 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 8449, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 5076661, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 8449, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 8449, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 5076661, 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:15.218 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 1941, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 2446064, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 1941, 2024-05-09T06:01:15.219 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 2446064, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:15.220 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 1941, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 1941, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 1941, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1941, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 39, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 6123, 2024-05-09T06:01:15.221 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 39, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 1, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 6123, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x55d977978340": { 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-09T06:01:15.222 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 1305, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 2278544, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 1305, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 496, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 2278544, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:15.223 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-09T06:01:15.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:15 smithi195 ceph-mon[33844]: pgmap v732: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-09T06:01:15.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:15 smithi115 ceph-mon[30349]: pgmap v732: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-09T06:01:15.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:15 smithi161 ceph-mon[33805]: pgmap v732: 129 pgs: 20 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 57 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-09T06:01:16.061 INFO:tasks.check_counter:Daemon mds.f mds.exported=20 2024-05-09T06:01:16.061 INFO:tasks.check_counter:Daemon mds.f mds.imported=6 2024-05-09T06:01:16.061 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-09T06:01:16.061 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-09T06:01:16.061 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T06:01:16.061 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=2 2024-05-09T06:01:16.061 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T06:01:16.061 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T06:01:16.498 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T06:01:17.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:17 smithi195 ceph-mon[33844]: pgmap v733: 129 pgs: 18 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 70 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-09T06:01:17.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:17 smithi115 ceph-mon[30349]: pgmap v733: 129 pgs: 18 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 70 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-09T06:01:17.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:17 smithi161 ceph-mon[33805]: pgmap v733: 129 pgs: 18 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 70 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-09T06:01:19.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:19 smithi195 ceph-mon[33844]: pgmap v734: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-05-09T06:01:19.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:19 smithi115 ceph-mon[30349]: pgmap v734: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-05-09T06:01:19.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:19 smithi161 ceph-mon[33805]: pgmap v734: 129 pgs: 14 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 82 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-05-09T06:01:21.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:21 smithi195 ceph-mon[33844]: pgmap v735: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 187 B/s wr, 10 op/s 2024-05-09T06:01:21.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:21 smithi115 ceph-mon[30349]: pgmap v735: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 187 B/s wr, 10 op/s 2024-05-09T06:01:21.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:21 smithi161 ceph-mon[33805]: pgmap v735: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 187 B/s wr, 10 op/s 2024-05-09T06:01:23.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:23 smithi195 ceph-mon[33844]: pgmap v736: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T06:01:23.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:23 smithi115 ceph-mon[30349]: pgmap v736: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T06:01:23.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:23 smithi161 ceph-mon[33805]: pgmap v736: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-09T06:01:24.308 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 7124, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 7136, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 3155585, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 12600878, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 11, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.626880259, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.263213453, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.341202209, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.057706954, 2024-05-09T06:01:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 7140, 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.787029713, 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000110228 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9446, 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.002266920, 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000239 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 593616 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:01:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 6943, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 5461, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 4996788, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 7612711, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 17, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 7, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.715214330, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.245591133, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.351417109, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.155774016, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5464, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.685169477, 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000308413 2024-05-09T06:01:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 6679, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001754588, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000262 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 2844432, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 555728 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 12727, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 13671, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 4784250, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 14333988, 2024-05-09T06:01:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 341, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 12, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 1.099639247, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.317847825, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.665668373, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.168780793, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 13659, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.859142399, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000062899 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 21711, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.004396239, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000202 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 142016, 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 212544 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3746, 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2.511630104, 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000670483 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1607, 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.110292690, 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000068632 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 3715, 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 3472, 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3472, 2024-05-09T06:01:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 147.362275653, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.042443051 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 243, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 7, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 2, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 619, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 443, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 236, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 180, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 27, 2024-05-09T06:01:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 27, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 488, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 3, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 3, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 3, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 5617, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 5122, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 233, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 20, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 53, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 24, 2024-05-09T06:01:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 472, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 13, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 435, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 825, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 4, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 234, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 6, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 825, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 192, 2024-05-09T06:01:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 10, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 182, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 633, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 1299, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 1450, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 5, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 3012, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 858, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 22, 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 2149 2024-05-09T06:01:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 52, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 39, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 14, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 658, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 583, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 61, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-05-09T06:01:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 15, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 1209, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 10, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 25, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-05-09T06:01:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 4366, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 4353, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 4353, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 13, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 70, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 100, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 99, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 99, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 1, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 1, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 1, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 18575056, 2024-05-09T06:01:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 18585891, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 0, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 967, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 10.276857853, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.010627567 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 0 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 161, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 1176, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 1015, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 32, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 157, 2024-05-09T06:01:24.323 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 125, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 443, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 1758, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 1315, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 3, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 929, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 926, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 136664, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 231824 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 4532, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 592, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 3715, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 71, 2024-05-09T06:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 1358, 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 620, 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.778819041, 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001256159 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1153, 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.332103715, 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000288034 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 61, 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.136377299, 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002235693 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 389, 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 99.320571054, 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.255322804 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.328 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-05-09T06:01:24.328 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.328 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.328 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.328 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 23, 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.004386313, 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000190709 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 105, 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 15.739051858, 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.149895731 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.329 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 23, 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 8.977601926, 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.390330518 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 147, 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 6.930550065, 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.047146599 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 169, 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.598828957, 2024-05-09T06:01:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.003543366 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 95, 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 3.962982166, 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.041715601 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 30, 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.962200825, 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.032073360 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 17, 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 8.558782416, 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.503457789 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4, 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.003408304, 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000852076 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2, 2024-05-09T06:01:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000521534, 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000260767 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 8, 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.022712309, 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002839038 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-05-09T06:01:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 13, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.012176737, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000936672 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 613, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.021201134, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001665907 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 6, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 4, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 2, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 239, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 119, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 1369, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-05-09T06:01:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 360, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 360, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:01:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:01:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 408087, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 137, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-05-09T06:01:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9320, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 178, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 755071, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 5901, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 5863, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 16958346, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 37, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 5863, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5863, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 263.934242276, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.045016926 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5863, 2024-05-09T06:01:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 8240 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 5863, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 892, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 4971, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 520, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 641, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 48, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 1005, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 344, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-05-09T06:01:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 2111, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 844, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 1023, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-05-09T06:01:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 5, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-05-09T06:01:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 127, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 49, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 12, 2024-05-09T06:01:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 12, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 1530, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 18, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 151 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 1, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 1690 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 34, 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 7, 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 1593, 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 583, 2024-05-09T06:01:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 26790, 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 10398690, 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 26790, 2024-05-09T06:01:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 26790, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 10398690, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 5863, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 17027877, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 5863, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 17027877, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-05-09T06:01:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 5863, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 5863, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 5863, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 5863, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 583, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 69651, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 583, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 38, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 69651, 2024-05-09T06:01:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x55d6cc9e24e0": { 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 4366, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 14391587, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 4366, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 967, 2024-05-09T06:01:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 14391587, 2024-05-09T06:01:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:24.354 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-09T06:01:25.055 INFO:tasks.check_counter:Daemon mds.i mds.exported=24 2024-05-09T06:01:25.056 INFO:tasks.check_counter:Daemon mds.i mds.imported=13 2024-05-09T06:01:25.056 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-09T06:01:25.056 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=23 2024-05-09T06:01:25.056 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=17 2024-05-09T06:01:25.056 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-05-09T06:01:25.056 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T06:01:25.057 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T06:01:25.299 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T06:01:25.359 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:25 smithi195 ceph-mon[33844]: osdmap e127: 12 total, 12 up, 12 in 2024-05-09T06:01:25.359 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:25 smithi195 ceph-mon[33844]: pgmap v738: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-09T06:01:25.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:25 smithi115 ceph-mon[30349]: osdmap e127: 12 total, 12 up, 12 in 2024-05-09T06:01:25.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:25 smithi115 ceph-mon[30349]: pgmap v738: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-09T06:01:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:25 smithi161 ceph-mon[33805]: osdmap e127: 12 total, 12 up, 12 in 2024-05-09T06:01:25.403 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:25 smithi161 ceph-mon[33805]: pgmap v738: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-09T06:01:27.127 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-05-09T06:01:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T06:01:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 373, 2024-05-09T06:01:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 375, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 112365, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 118624, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 6, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 2, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.037920214, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.010166499, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.010247690, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.013730492, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 373, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.014407548, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000038626 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 223, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 261, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1585264, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 226238, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 5, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-05-09T06:01:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.027291935, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.007024761, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.011212185, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.006832458, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 260, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.027919056, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000107380 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 156848 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 79, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 67, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 265439, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 34399, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 3, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 2, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.011582917, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.003412931, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.008918773, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.001276949, 2024-05-09T06:01:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 66, 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.022378457, 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000339067 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 264624, 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 34080 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-05-09T06:01:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-05-09T06:01:27.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.133 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-05-09T06:01:27.133 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-09T06:01:27.133 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-09T06:01:27.133 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 981, 2024-05-09T06:01:27.133 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.035998810, 2024-05-09T06:01:27.133 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000036696 2024-05-09T06:01:27.133 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:27.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.134 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-05-09T06:01:27.134 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-05-09T06:01:27.134 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-05-09T06:01:27.134 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.134 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.134 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.134 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:27.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.134 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 0, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 0, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 0, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 0, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-05-09T06:01:27.135 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 565, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 565, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 18, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 2, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 0, 2024-05-09T06:01:27.136 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 4, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 234, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 6, 2024-05-09T06:01:27.137 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T06:01:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 0 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 0, 2024-05-09T06:01:27.139 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-05-09T06:01:27.140 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 0, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 365, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 0, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 2, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 9, 2024-05-09T06:01:27.141 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 18575056, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 18585891, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 18585755, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 365 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 150, 2024-05-09T06:01:27.142 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 304, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 154, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 20, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 24, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 4, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 565, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 606, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 41, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 0, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 0, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 121040, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 215440 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.143 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 0, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 0, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 0, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 0, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 0, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.144 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.145 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-05-09T06:01:27.146 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-05-09T06:01:27.147 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.148 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.148 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.148 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-05-09T06:01:27.148 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.148 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.149 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.150 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.151 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.152 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.153 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-05-09T06:01:27.153 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.153 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.153 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.153 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-05-09T06:01:27.153 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.153 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-05-09T06:01:27.154 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 2, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 0, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 0, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 0, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 0, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 2640, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-05-09T06:01:27.156 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-05-09T06:01:27.157 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-05-09T06:01:27.158 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 189239, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 48, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-05-09T06:01:27.159 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9320, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 178, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 788159, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 4575, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-05-09T06:01:27.160 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 592, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 26, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 4, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 592, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 592, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.887966996, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.003189133 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-05-09T06:01:27.161 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 592, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 593 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 592, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 592, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 0, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 370, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 0, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 219, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 0, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 0, 2024-05-09T06:01:27.162 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 2, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-05-09T06:01:27.163 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-05-09T06:01:27.164 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 127, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 6, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 5, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 5, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 0, 2024-05-09T06:01:27.165 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 2, 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 0 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 0, 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T06:01:27.166 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 0, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 0, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 0, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 671, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 1895833, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 671, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:27.167 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 671, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1895833, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:27.168 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 592, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 1524030, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 592, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1524030, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:27.169 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 592, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 592, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 592, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 592, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.170 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-05-09T06:01:27.171 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x5618f604cb60": { 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-05-09T06:01:27.172 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-05-09T06:01:27.173 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-05-09T06:01:27.173 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-05-09T06:01:27.173 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-05-09T06:01:27.173 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-05-09T06:01:27.173 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-05-09T06:01:27.173 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-05-09T06:01:27.173 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:27.173 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-05-09T06:01:27.173 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-05-09T06:01:27.858 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:27 smithi195 ceph-mon[33844]: pgmap v739: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:27.886 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-09T06:01:27.886 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-09T06:01:27.886 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-05-09T06:01:27.886 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-09T06:01:27.886 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T06:01:27.887 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-05-09T06:01:27.887 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-09T06:01:27.899 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-09T06:01:27.899 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T06:01:27.899 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:01:27.900 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:27 smithi115 ceph-mon[30349]: pgmap v739: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:27.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:27 smithi161 ceph-mon[33805]: pgmap v739: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:27.933 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T06:01:27.933 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:01:27.994 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T06:01:28.287 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T06:01:29.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:29 smithi115 ceph-mon[30349]: pgmap v740: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:29.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:29 smithi195 ceph-mon[33844]: pgmap v740: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:29.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:29 smithi161 ceph-mon[33805]: pgmap v740: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:30.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.814+0000 7f678864d700 1 -- 172.21.15.115:0/37099564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 msgr2=0x7f67801038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:30.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.814+0000 7f678864d700 1 --2- 172.21.15.115:0/37099564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 0x7f67801038f0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f677c0099c0 tx=0x7f677c009cd0 comp rx=0 tx=0).stop 2024-05-09T06:01:30.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 -- 172.21.15.115:0/37099564 shutdown_connections 2024-05-09T06:01:30.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 --2- 172.21.15.115:0/37099564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6780108760 0x7f67801100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:30.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 --2- 172.21.15.115:0/37099564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 0x7f67801038f0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:30.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 --2- 172.21.15.115:0/37099564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780107e10 0x7f6780108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:30.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 -- 172.21.15.115:0/37099564 >> 172.21.15.115:0/37099564 conn(0x7f67800fef30 msgr2=0x7f6780101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:01:30.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 -- 172.21.15.115:0/37099564 shutdown_connections 2024-05-09T06:01:30.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 -- 172.21.15.115:0/37099564 wait complete. 2024-05-09T06:01:30.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 Processor -- start 2024-05-09T06:01:30.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 -- start start 2024-05-09T06:01:30.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 0x7f678019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:01:30.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6780107e10 0x7f678019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:01:30.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780108760 0x7f67801a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:01:30.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6780113260 con 0x7f67801034e0 2024-05-09T06:01:30.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67801130e0 con 0x7f6780108760 2024-05-09T06:01:30.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f678864d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f67801133e0 con 0x7f6780107e10 2024-05-09T06:01:30.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f67863e9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 0x7f678019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f6785be8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6780107e10 0x7f678019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f67863e9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 0x7f678019d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49754/0 (socket says 172.21.15.115:49754) 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f67863e9700 1 -- 172.21.15.115:0/545267437 learned_addr learned my addr 172.21.15.115:0/545267437 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f6786bea700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780108760 0x7f67801a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f67863e9700 1 -- 172.21.15.115:0/545267437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6780107e10 msgr2=0x7f678019d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f67863e9700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6780107e10 0x7f678019d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f67863e9700 1 -- 172.21.15.115:0/545267437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780108760 msgr2=0x7f67801a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f67863e9700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780108760 0x7f67801a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f67863e9700 1 -- 172.21.15.115:0/545267437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67801a1ee0 con 0x7f67801034e0 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.816+0000 7f6785be8700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6780107e10 0x7f678019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.817+0000 7f67863e9700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 0x7f678019d010 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f677c000c00 tx=0x7f677c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.817+0000 7f67777fe700 1 -- 172.21.15.115:0/545267437 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c004830 con 0x7f67801034e0 2024-05-09T06:01:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.817+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67801a2170 con 0x7f67801034e0 2024-05-09T06:01:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.817+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67801a2640 con 0x7f67801034e0 2024-05-09T06:01:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.817+0000 7f67777fe700 1 -- 172.21.15.115:0/545267437 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f677c00f7f0 con 0x7f67801034e0 2024-05-09T06:01:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.817+0000 7f67777fe700 1 -- 172.21.15.115:0/545267437 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0053a0 con 0x7f67801034e0 2024-05-09T06:01:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.818+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6764005330 con 0x7f67801034e0 2024-05-09T06:01:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.820+0000 7f67777fe700 1 -- 172.21.15.115:0/545267437 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f677c029060 con 0x7f67801034e0 2024-05-09T06:01:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.821+0000 7f67777fe700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f676c0b29f0 0x7f676c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:01:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.821+0000 7f6785be8700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f676c0b29f0 0x7f676c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:01:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.821+0000 7f67777fe700 1 -- 172.21.15.115:0/545267437 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f677c0e3550 con 0x7f67801034e0 2024-05-09T06:01:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.821+0000 7f6785be8700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f676c0b29f0 0x7f676c0b4f00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6780109700 tx=0x7f67700073d0 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:01:30.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:30.825+0000 7f67777fe700 1 -- 172.21.15.115:0/545267437 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f677c013070 con 0x7f67801034e0 2024-05-09T06:01:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.042+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6764005620 con 0x7f67801034e0 2024-05-09T06:01:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.042+0000 7f67777fe700 1 -- 172.21.15.115:0/545267437 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v127) ==== 81+0+959 (secure 0 0 0) 0x7f677c0aac30 con 0x7f67801034e0 2024-05-09T06:01:31.030 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1352570099 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:31.030 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/792641138 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:31.030 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/2041699318 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:31.030 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/617068092 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:31.030 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1072371912 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:31.030 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3822912073 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:31.030 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1329653367 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/2041699318 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/4237548909 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1853670702 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/1212404088 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/1212404088 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/792641138 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3376666830 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1620364593 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2102093122 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f676c0b29f0 msgr2=0x7f676c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f676c0b29f0 0x7f676c0b4f00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6780109700 tx=0x7f67700073d0 comp rx=0 tx=0).stop 2024-05-09T06:01:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 msgr2=0x7f678019d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 0x7f678019d010 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f677c000c00 tx=0x7f677c0047b0 comp rx=0 tx=0).stop 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 shutdown_connections 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f676c0b29f0 0x7f676c0b4f00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6780108760 0x7f67801a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67801034e0 0x7f678019d010 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 --2- 172.21.15.115:0/545267437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6780107e10 0x7f678019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 >> 172.21.15.115:0/545267437 conn(0x7f67800fef30 msgr2=0x7f67801065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 shutdown_connections 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:31.045+0000 7f678864d700 1 -- 172.21.15.115:0/545267437 wait complete. 2024-05-09T06:01:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:listed 16 entries 2024-05-09T06:01:31.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:31 smithi115 ceph-mon[30349]: pgmap v741: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:31.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:31 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/545267437' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:01:31.746 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-09T06:01:31.746 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T06:01:31.764 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T06:01:31.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:31 smithi195 ceph-mon[33844]: pgmap v741: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:31.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:31 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/545267437' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:01:31.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:31 smithi161 ceph-mon[33805]: pgmap v741: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:31.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:31 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/545267437' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:01:32.063 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T06:01:33.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:33 smithi115 ceph-mon[30349]: pgmap v742: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:33.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:33 smithi195 ceph-mon[33844]: pgmap v742: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:33.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:33 smithi161 ceph-mon[33805]: pgmap v742: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-09T06:01:33.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.920+0000 7f9536edb700 1 -- 172.21.15.115:0/2473386379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 msgr2=0x7f9530108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:33.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.920+0000 7f9536edb700 1 --2- 172.21.15.115:0/2473386379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 0x7f9530108610 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f952c009990 tx=0x7f952c009ca0 comp rx=0 tx=0).stop 2024-05-09T06:01:33.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.920+0000 7f9536edb700 1 -- 172.21.15.115:0/2473386379 shutdown_connections 2024-05-09T06:01:33.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.920+0000 7f9536edb700 1 --2- 172.21.15.115:0/2473386379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95301042f0 0x7f9530110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:33.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.920+0000 7f9536edb700 1 --2- 172.21.15.115:0/2473386379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 0x7f9530108610 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:33.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.920+0000 7f9536edb700 1 --2- 172.21.15.115:0/2473386379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95301039a0 0x7f9530103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:33.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.920+0000 7f9536edb700 1 -- 172.21.15.115:0/2473386379 >> 172.21.15.115:0/2473386379 conn(0x7f95300fe520 msgr2=0x7f9530100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:01:33.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.920+0000 7f9536edb700 1 -- 172.21.15.115:0/2473386379 shutdown_connections 2024-05-09T06:01:33.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.920+0000 7f9536edb700 1 -- 172.21.15.115:0/2473386379 wait complete. 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9536edb700 1 Processor -- start 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9536edb700 1 -- start start 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9536edb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95301039a0 0x7f953019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9536edb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95301042f0 0x7f953019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9536edb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 0x7f95301a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9536edb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9530113220 con 0x7f9530108200 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9536edb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95301130a0 con 0x7f95301039a0 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9536edb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95301133a0 con 0x7f95301042f0 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9535478700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 0x7f95301a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9534c77700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95301039a0 0x7f953019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9534c77700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95301039a0 0x7f953019bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.115:33422/0 (socket says 172.21.15.115:33422) 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9535478700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 0x7f95301a6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35692/0 (socket says 172.21.15.115:35692) 2024-05-09T06:01:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9534c77700 1 -- 172.21.15.115:0/1133579979 learned_addr learned my addr 172.21.15.115:0/1133579979 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.921+0000 7f9527fff700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95301042f0 0x7f953019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9535478700 1 -- 172.21.15.115:0/1133579979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95301042f0 msgr2=0x7f953019c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9535478700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95301042f0 0x7f953019c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9535478700 1 -- 172.21.15.115:0/1133579979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95301039a0 msgr2=0x7f953019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9535478700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95301039a0 0x7f953019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9535478700 1 -- 172.21.15.115:0/1133579979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953019c910 con 0x7f9530108200 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9534c77700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95301039a0 0x7f953019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9535478700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 0x7f95301a6ba0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f952800ab40 tx=0x7f952800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9525ffb700 1 -- 172.21.15.115:0/1133579979 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528017020 con 0x7f9530108200 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95301a7200 con 0x7f9530108200 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9525ffb700 1 -- 172.21.15.115:0/1133579979 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9528004bb0 con 0x7f9530108200 2024-05-09T06:01:33.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.922+0000 7f9525ffb700 1 -- 172.21.15.115:0/1133579979 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528005280 con 0x7f9530108200 2024-05-09T06:01:33.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.923+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95301a76c0 con 0x7f9530108200 2024-05-09T06:01:33.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.924+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95300512c0 con 0x7f9530108200 2024-05-09T06:01:33.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.925+0000 7f9525ffb700 1 -- 172.21.15.115:0/1133579979 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 25) ==== 149784+0+0 (secure 0 0 0) 0x7f952800c580 con 0x7f9530108200 2024-05-09T06:01:33.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.926+0000 7f9525ffb700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95180b29f0 0x7f95180b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T06:01:33.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.926+0000 7f9534c77700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95180b29f0 0x7f95180b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T06:01:33.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.926+0000 7f9525ffb700 1 -- 172.21.15.115:0/1133579979 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f95280e1cc0 con 0x7f9530108200 2024-05-09T06:01:33.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.926+0000 7f9534c77700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95180b29f0 0x7f95180b4f00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f952c00b5c0 tx=0x7f952c005960 comp rx=0 tx=0).ready entity=mgr.15327 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T06:01:33.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:33.930+0000 7f9525ffb700 1 -- 172.21.15.115:0/1133579979 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9528021060 con 0x7f9530108200 2024-05-09T06:01:34.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.149+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f953010b7a0 con 0x7f9530108200 2024-05-09T06:01:34.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.150+0000 7f9525ffb700 1 -- 172.21.15.115:0/1133579979 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v127) ==== 81+0+959 (secure 0 0 0) 0x7f95280a93a0 con 0x7f9530108200 2024-05-09T06:01:34.136 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1352570099 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:34.136 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/792641138 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:34.136 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/2041699318 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:34.136 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/617068092 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1072371912 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3822912073 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1329653367 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/2041699318 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/4237548909 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1853670702 2024-05-10T05:38:11.918655+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6800/1212404088 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/1212404088 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6801/792641138 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3376666830 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1620364593 2024-05-10T05:26:06.415053+0000 2024-05-09T06:01:34.137 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2102093122 2024-05-10T05:26:32.548125+0000 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95180b29f0 msgr2=0x7f95180b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95180b29f0 0x7f95180b4f00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f952c00b5c0 tx=0x7f952c005960 comp rx=0 tx=0).stop 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 msgr2=0x7f95301a6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 0x7f95301a6ba0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f952800ab40 tx=0x7f952800ae50 comp rx=0 tx=0).stop 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 shutdown_connections 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:6800/1477729447,v1:172.21.15.115:6801/1477729447] conn(0x7f95180b29f0 0x7f95180b4f00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95301039a0 0x7f953019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9530108200 0x7f95301a6ba0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 --2- 172.21.15.115:0/1133579979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95301042f0 0x7f953019c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 >> 172.21.15.115:0/1133579979 conn(0x7f95300fe520 msgr2=0x7f9530100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.152+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 shutdown_connections 2024-05-09T06:01:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-09T06:01:34.153+0000 7f9536edb700 1 -- 172.21.15.115:0/1133579979 wait complete. 2024-05-09T06:01:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:listed 16 entries 2024-05-09T06:01:34.762 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:34 smithi115 ceph-mon[30349]: from='client.? 172.21.15.115:0/1133579979' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:01:34.790 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi115.front.sepia.ceph.com... 2024-05-09T06:01:34.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:01:34.790 DEBUG:teuthology.orchestra.run.smithi115:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T06:01:34.840 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T06:01:34.843 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi115.stderr:ceph-fuse[87222]: fuse finished with error 0 and tester_r 0 2024-05-09T06:01:34.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:34 smithi195 ceph-mon[33844]: from='client.? 172.21.15.115:0/1133579979' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:01:34.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:34 smithi161 ceph-mon[33805]: from='client.? 172.21.15.115:0/1133579979' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T06:01:35.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:35 smithi195 ceph-mon[33844]: pgmap v743: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T06:01:35.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:35 smithi115 ceph-mon[30349]: pgmap v743: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T06:01:35.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:35 smithi161 ceph-mon[33805]: pgmap v743: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-05-09T06:01:37.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:37 smithi195 ceph-mon[33844]: pgmap v744: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T06:01:37.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:37 smithi195 ceph-mon[33844]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T06:01:37.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:37 smithi115 ceph-mon[30349]: pgmap v744: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T06:01:37.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:37 smithi115 ceph-mon[30349]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T06:01:37.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:37 smithi161 ceph-mon[33805]: pgmap v744: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-09T06:01:37.903 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:37 smithi161 ceph-mon[33805]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-09T06:01:39.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:39 smithi195 ceph-mon[33844]: pgmap v745: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T06:01:39.889 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:39 smithi115 ceph-mon[30349]: pgmap v745: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T06:01:39.904 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:39 smithi161 ceph-mon[33805]: pgmap v745: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-09T06:01:40.942 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T06:01:40.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:01:40.943 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T06:01:40.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:01:40.983 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2024-05-09T06:01:41.058 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T06:01:41.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:01:41.059 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T06:01:41.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T06:01:41.130 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link delete ceph-brx 2024-05-09T06:01:41.222 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T06:01:41.234 INFO:tasks.cephadm:Teardown begin 2024-05-09T06:01:41.234 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/cephadm.py", line 2273, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-09T06:01:41.235 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:01:41.271 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:01:41.382 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:01:41.426 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T06:01:41.426 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T06:01:41.443 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T06:01:41.462 DEBUG:teuthology.orchestra.run.smithi195:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T06:01:41.485 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T06:01:41.485 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T06:01:41.485 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.a 2024-05-09T06:01:41.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:41 smithi161 ceph-mon[33805]: pgmap v746: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T06:01:41.791 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:41 smithi115 ceph-mon[30349]: pgmap v746: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T06:01:41.791 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:41 smithi115 systemd[1]: Stopping Ceph mon.a for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:01:41.859 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:41 smithi195 ceph-mon[33844]: pgmap v746: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-09T06:01:42.139 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:41 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a[30345]: 2024-05-09T06:01:41.821+0000 7f5c18a3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:01:42.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:41 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a[30345]: 2024-05-09T06:01:41.822+0000 7f5c18a3f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T06:01:42.140 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:41 smithi115 podman[140821]: 2024-05-09 06:01:41.914107502 +0000 UTC m=+0.218285067 container died 8cbc383f681f2c98febd5d219163594e98e58ef270fde82cf6cf750efb543c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:01:42.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:42 smithi115 podman[140821]: 2024-05-09 06:01:42.456767607 +0000 UTC m=+0.760945180 container cleanup 8cbc383f681f2c98febd5d219163594e98e58ef270fde82cf6cf750efb543c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T06:01:42.640 INFO:journalctl@ceph.mon.a.smithi115.stdout:May 09 06:01:42 smithi115 bash[140821]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-a 2024-05-09T06:01:42.908 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.a.service' 2024-05-09T06:01:42.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:01:42.969 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T06:01:42.969 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T06:01:42.969 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.b 2024-05-09T06:01:43.337 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:43 smithi161 systemd[1]: Stopping Ceph mon.b for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:01:43.653 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:43 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-b[33801]: 2024-05-09T06:01:43.349+0000 7fce34e04700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:01:43.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:43 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-b[33801]: 2024-05-09T06:01:43.349+0000 7fce34e04700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T06:01:43.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:43 smithi161 podman[72515]: 2024-05-09 06:01:43.411227662 +0000 UTC m=+0.183467507 container died 72441f6aa892b453a4dde7f34d7757a2d62a9ae3682484a71d9b3ea79c27b3e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T06:01:44.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:43 smithi161 podman[72515]: 2024-05-09 06:01:43.825433797 +0000 UTC m=+0.597673644 container cleanup 72441f6aa892b453a4dde7f34d7757a2d62a9ae3682484a71d9b3ea79c27b3e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T06:01:44.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:43 smithi161 bash[72515]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-b 2024-05-09T06:01:44.371 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 09 06:01:44 smithi161 podman[72529]: 2024-05-09 06:01:44.102270795 +0000 UTC m=+0.687500222 container remove 72441f6aa892b453a4dde7f34d7757a2d62a9ae3682484a71d9b3ea79c27b3e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:01:44.384 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.b.service' 2024-05-09T06:01:44.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:01:44.433 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T06:01:44.434 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T06:01:44.434 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.c 2024-05-09T06:01:44.760 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:44 smithi195 systemd[1]: Stopping Ceph mon.c for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:01:45.112 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:44 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-c[33818]: 2024-05-09T06:01:44.772+0000 7f123dac6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:01:45.112 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:44 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-c[33818]: 2024-05-09T06:01:44.772+0000 7f123dac6700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T06:01:45.112 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:45 smithi195 podman[73216]: 2024-05-09 06:01:45.008122675 +0000 UTC m=+0.357242732 container died 2e0df25d136516649ed1a9fa3bb759a33248b7494339a65ed49baf0696203b0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=) 2024-05-09T06:01:45.609 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:45 smithi195 podman[73216]: 2024-05-09 06:01:45.390190967 +0000 UTC m=+0.739311024 container cleanup 2e0df25d136516649ed1a9fa3bb759a33248b7494339a65ed49baf0696203b0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T06:01:45.609 INFO:journalctl@ceph.mon.c.smithi195.stdout:May 09 06:01:45 smithi195 bash[73216]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mon-c 2024-05-09T06:01:45.811 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mon.c.service' 2024-05-09T06:01:45.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:01:45.863 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T06:01:45.863 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T06:01:45.863 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.x 2024-05-09T06:01:46.214 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 06:01:45 smithi115 systemd[1]: Stopping Ceph mgr.x for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:01:46.516 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 06:01:46 smithi115 podman[140944]: 2024-05-09 06:01:46.236984009 +0000 UTC m=+0.165099886 container died 1b4a7b30461ef3ce29b3304688a2a1e9e59662e97197482f7bebf82b1c1e7cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-05-09T06:01:46.867 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 06:01:46 smithi115 podman[140944]: 2024-05-09 06:01:46.637666851 +0000 UTC m=+0.565782728 container cleanup 1b4a7b30461ef3ce29b3304688a2a1e9e59662e97197482f7bebf82b1c1e7cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T06:01:46.867 INFO:journalctl@ceph.mgr.x.smithi115.stdout:May 09 06:01:46 smithi115 bash[140944]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-x 2024-05-09T06:01:46.978 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-09T06:01:47.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:01:47.014 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T06:01:47.014 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T06:01:47.015 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.y 2024-05-09T06:01:47.365 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 06:01:47 smithi161 systemd[1]: Stopping Ceph mgr.y for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:01:47.653 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 06:01:47 smithi161 podman[72638]: 2024-05-09 06:01:47.378224978 +0000 UTC m=+0.150721038 container died 550fab3ac83c87596e06f66c9c0a6f6dfc5680c5c015c96e4d57dbe12004400d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-09T06:01:48.153 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 06:01:47 smithi161 podman[72638]: 2024-05-09 06:01:47.922781787 +0000 UTC m=+0.695277845 container cleanup 550fab3ac83c87596e06f66c9c0a6f6dfc5680c5c015c96e4d57dbe12004400d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T06:01:48.153 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 09 06:01:47 smithi161 bash[72638]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-y 2024-05-09T06:01:48.339 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-09T06:01:48.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:01:48.374 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T06:01:48.374 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T06:01:48.374 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.z 2024-05-09T06:01:48.704 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 06:01:48 smithi195 systemd[1]: Stopping Ceph mgr.z for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:01:49.035 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 06:01:48 smithi195 podman[73338]: 2024-05-09 06:01:48.752673262 +0000 UTC m=+0.161601845 container died 25037a9333afef3ca7188aef960f10e4b1bd0eff300054a0c34a7eab8a13a269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T06:01:49.359 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 06:01:49 smithi195 podman[73338]: 2024-05-09 06:01:49.164540279 +0000 UTC m=+0.573468874 container cleanup 25037a9333afef3ca7188aef960f10e4b1bd0eff300054a0c34a7eab8a13a269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T06:01:49.359 INFO:journalctl@ceph.mgr.z.smithi195.stdout:May 09 06:01:49 smithi195 bash[73338]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mgr-z 2024-05-09T06:01:49.596 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-09T06:01:49.635 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:01:49.635 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T06:01:49.635 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T06:01:49.635 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.0 2024-05-09T06:01:49.953 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:49 smithi115 systemd[1]: Stopping Ceph osd.0 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:01:50.389 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:49 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0[42417]: 2024-05-09T06:01:49.966+0000 7f36ff757700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:01:50.390 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:49 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0[42417]: 2024-05-09T06:01:49.966+0000 7f36ff757700 -1 osd.0 127 *** Got signal Terminated *** 2024-05-09T06:01:50.390 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:49 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0[42417]: 2024-05-09T06:01:49.966+0000 7f36ff757700 -1 osd.0 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:01:55.889 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:55 smithi115 podman[141065]: 2024-05-09 06:01:55.611338104 +0000 UTC m=+5.762767833 container died 7b821fadca5e06fbfdae5aeb1acfe25234c79780fc716aac4991cdb1ad6102ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-05-09T06:01:56.271 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:56 smithi115 podman[141065]: 2024-05-09 06:01:56.033498811 +0000 UTC m=+6.184928539 container cleanup 7b821fadca5e06fbfdae5aeb1acfe25234c79780fc716aac4991cdb1ad6102ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T06:01:56.271 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:56 smithi115 bash[141065]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0 2024-05-09T06:01:56.631 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:56 smithi115 podman[141085]: 2024-05-09 06:01:56.285461367 +0000 UTC m=+0.666182979 container remove 7b821fadca5e06fbfdae5aeb1acfe25234c79780fc716aac4991cdb1ad6102ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:01:56.889 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:56 smithi115 podman[141178]: 2024-05-09 06:01:56.54783955 +0000 UTC m=+0.018093774 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:01:56.889 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:56 smithi115 podman[141178]: 2024-05-09 06:01:56.695303953 +0000 UTC m=+0.165558150 container create 0abbb98550e48864c06b84ffcaf8b1ac93e761262804816e4715317095726c94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-09T06:01:57.390 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:56 smithi115 podman[141178]: 2024-05-09 06:01:56.973175172 +0000 UTC m=+0.443429378 container init 0abbb98550e48864c06b84ffcaf8b1ac93e761262804816e4715317095726c94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:01:57.390 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:56 smithi115 podman[141178]: 2024-05-09 06:01:56.976675993 +0000 UTC m=+0.446930197 container start 0abbb98550e48864c06b84ffcaf8b1ac93e761262804816e4715317095726c94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T06:01:57.390 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:57 smithi115 podman[141178]: 2024-05-09 06:01:57.089845085 +0000 UTC m=+0.560099378 container attach 0abbb98550e48864c06b84ffcaf8b1ac93e761262804816e4715317095726c94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T06:01:57.803 INFO:journalctl@ceph.osd.0.smithi115.stdout:May 09 06:01:57 smithi115 podman[141178]: 2024-05-09 06:01:57.520494384 +0000 UTC m=+0.990748586 container died 0abbb98550e48864c06b84ffcaf8b1ac93e761262804816e4715317095726c94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:01:58.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.0.service' 2024-05-09T06:01:58.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:01:58.170 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T06:01:58.170 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T06:01:58.170 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.1 2024-05-09T06:01:58.403 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:01:58 smithi161 systemd[1]: Stopping Ceph osd.1 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:01:58.903 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:01:58 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1[38402]: 2024-05-09T06:01:58.485+0000 7f4664713700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:01:58.903 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:01:58 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1[38402]: 2024-05-09T06:01:58.485+0000 7f4664713700 -1 osd.1 127 *** Got signal Terminated *** 2024-05-09T06:01:58.903 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:01:58 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1[38402]: 2024-05-09T06:01:58.485+0000 7f4664713700 -1 osd.1 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:02:04.312 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:04 smithi161 podman[72775]: 2024-05-09 06:02:04.028896582 +0000 UTC m=+5.655341885 container died 3189ba1ef780fa6a75e26750ff3ae669403af26699064b0975b83b31fc7180f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T06:02:04.653 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:04 smithi161 podman[72775]: 2024-05-09 06:02:04.444822652 +0000 UTC m=+6.071267908 container cleanup 3189ba1ef780fa6a75e26750ff3ae669403af26699064b0975b83b31fc7180f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-09T06:02:04.653 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:04 smithi161 bash[72775]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1 2024-05-09T06:02:04.957 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:04 smithi161 podman[72792]: 2024-05-09 06:02:04.671158683 +0000 UTC m=+0.628850993 container remove 3189ba1ef780fa6a75e26750ff3ae669403af26699064b0975b83b31fc7180f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-09T06:02:05.236 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:04 smithi161 podman[72882]: 2024-05-09 06:02:04.874814294 +0000 UTC m=+0.018112943 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:02:05.237 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:04 smithi161 podman[72882]: 2024-05-09 06:02:04.997735936 +0000 UTC m=+0.141034616 container create 84b31c90da9fc2c6403924e15b88a40754617a20228853db77ee3cd8b84e50d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T06:02:05.653 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:05 smithi161 podman[72882]: 2024-05-09 06:02:05.25068671 +0000 UTC m=+0.393985353 container init 84b31c90da9fc2c6403924e15b88a40754617a20228853db77ee3cd8b84e50d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T06:02:05.653 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:05 smithi161 podman[72882]: 2024-05-09 06:02:05.254323231 +0000 UTC m=+0.397621872 container start 84b31c90da9fc2c6403924e15b88a40754617a20228853db77ee3cd8b84e50d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-09T06:02:05.653 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:05 smithi161 podman[72882]: 2024-05-09 06:02:05.358730605 +0000 UTC m=+0.502029259 container attach 84b31c90da9fc2c6403924e15b88a40754617a20228853db77ee3cd8b84e50d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-09T06:02:06.048 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 09 06:02:05 smithi161 podman[72882]: 2024-05-09 06:02:05.758664721 +0000 UTC m=+0.901963363 container died 84b31c90da9fc2c6403924e15b88a40754617a20228853db77ee3cd8b84e50d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T06:02:06.320 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.1.service' 2024-05-09T06:02:06.362 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:02:06.362 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T06:02:06.362 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T06:02:06.362 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.2 2024-05-09T06:02:06.681 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:06 smithi195 systemd[1]: Stopping Ceph osd.2 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:02:07.108 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:06 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2[38992]: 2024-05-09T06:02:06.693+0000 7f7ee9a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:02:07.109 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:06 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2[38992]: 2024-05-09T06:02:06.693+0000 7f7ee9a00700 -1 osd.2 127 *** Got signal Terminated *** 2024-05-09T06:02:07.109 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:06 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2[38992]: 2024-05-09T06:02:06.693+0000 7f7ee9a00700 -1 osd.2 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:02:12.707 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:12 smithi195 podman[73477]: 2024-05-09 06:02:12.423800413 +0000 UTC m=+5.850719640 container died 27bcae66ce23b6710b2dd66b9ed1c2697adbcc95194630fede6dba630a7c6426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:02:13.082 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:12 smithi195 podman[73477]: 2024-05-09 06:02:12.81972849 +0000 UTC m=+6.246647761 container cleanup 27bcae66ce23b6710b2dd66b9ed1c2697adbcc95194630fede6dba630a7c6426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:02:13.082 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:12 smithi195 bash[73477]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2 2024-05-09T06:02:13.359 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:13 smithi195 podman[73494]: 2024-05-09 06:02:13.096418065 +0000 UTC m=+0.667313444 container remove 27bcae66ce23b6710b2dd66b9ed1c2697adbcc95194630fede6dba630a7c6426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-05-09T06:02:13.686 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:13 smithi195 podman[73589]: 2024-05-09 06:02:13.312844348 +0000 UTC m=+0.018446113 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:02:13.686 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:13 smithi195 podman[73589]: 2024-05-09 06:02:13.439144464 +0000 UTC m=+0.144746267 container create e5961a154c5ff40aac2bf6621dee3c5123ac6af9641c96aa677674d695a85d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T06:02:14.054 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:13 smithi195 podman[73589]: 2024-05-09 06:02:13.700122919 +0000 UTC m=+0.405724685 container init e5961a154c5ff40aac2bf6621dee3c5123ac6af9641c96aa677674d695a85d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-09T06:02:14.054 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:13 smithi195 podman[73589]: 2024-05-09 06:02:13.704117064 +0000 UTC m=+0.409718825 container start e5961a154c5ff40aac2bf6621dee3c5123ac6af9641c96aa677674d695a85d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux ) 2024-05-09T06:02:14.054 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:13 smithi195 podman[73589]: 2024-05-09 06:02:13.810774484 +0000 UTC m=+0.516376248 container attach e5961a154c5ff40aac2bf6621dee3c5123ac6af9641c96aa677674d695a85d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0) 2024-05-09T06:02:14.331 INFO:journalctl@ceph.osd.2.smithi195.stdout:May 09 06:02:14 smithi195 podman[73589]: 2024-05-09 06:02:14.0678894 +0000 UTC m=+0.773491155 container died e5961a154c5ff40aac2bf6621dee3c5123ac6af9641c96aa677674d695a85d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T06:02:14.781 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.2.service' 2024-05-09T06:02:14.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:02:14.820 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T06:02:14.820 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T06:02:14.821 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.3 2024-05-09T06:02:15.139 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:14 smithi115 systemd[1]: Stopping Ceph osd.3 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:02:15.412 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:15 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3[48887]: 2024-05-09T06:02:15.161+0000 7faf216f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:02:15.413 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:15 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3[48887]: 2024-05-09T06:02:15.161+0000 7faf216f1700 -1 osd.3 127 *** Got signal Terminated *** 2024-05-09T06:02:15.413 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:15 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3[48887]: 2024-05-09T06:02:15.161+0000 7faf216f1700 -1 osd.3 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:02:21.098 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:20 smithi115 podman[141283]: 2024-05-09 06:02:20.801306142 +0000 UTC m=+5.770070197 container died 222c8a4187878c6badd480fd60792186307333696c85ea17ae765af62fdbc9f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:02:21.389 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:21 smithi115 podman[141283]: 2024-05-09 06:02:21.312075658 +0000 UTC m=+6.280839752 container cleanup 222c8a4187878c6badd480fd60792186307333696c85ea17ae765af62fdbc9f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507) 2024-05-09T06:02:21.389 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:21 smithi115 bash[141283]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3 2024-05-09T06:02:21.889 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:21 smithi115 podman[141314]: 2024-05-09 06:02:21.547203887 +0000 UTC m=+0.743496211 container remove 222c8a4187878c6badd480fd60792186307333696c85ea17ae765af62fdbc9f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T06:02:22.229 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:21 smithi115 podman[141406]: 2024-05-09 06:02:21.879950896 +0000 UTC m=+0.017757627 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:02:22.229 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:22 smithi115 podman[141406]: 2024-05-09 06:02:22.006736428 +0000 UTC m=+0.144543188 container create 6a0429395324045ea5f21143fdce88e3758b71afcf5389655f115058a2bff22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T06:02:22.523 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:22 smithi115 podman[141406]: 2024-05-09 06:02:22.243417726 +0000 UTC m=+0.381224448 container init 6a0429395324045ea5f21143fdce88e3758b71afcf5389655f115058a2bff22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T06:02:22.523 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:22 smithi115 podman[141406]: 2024-05-09 06:02:22.247092901 +0000 UTC m=+0.384899626 container start 6a0429395324045ea5f21143fdce88e3758b71afcf5389655f115058a2bff22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5) 2024-05-09T06:02:22.523 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:22 smithi115 podman[141406]: 2024-05-09 06:02:22.3343442 +0000 UTC m=+0.472150928 container attach 6a0429395324045ea5f21143fdce88e3758b71afcf5389655f115058a2bff22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:02:22.524 INFO:journalctl@ceph.osd.3.smithi115.stdout:May 09 06:02:22 smithi115 podman[141406]: 2024-05-09 06:02:22.53778548 +0000 UTC m=+0.675592202 container died 6a0429395324045ea5f21143fdce88e3758b71afcf5389655f115058a2bff22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0) 2024-05-09T06:02:23.238 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.3.service' 2024-05-09T06:02:23.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:02:23.279 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T06:02:23.280 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T06:02:23.280 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.4 2024-05-09T06:02:23.593 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:23 smithi161 systemd[1]: Stopping Ceph osd.4 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:02:23.903 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:23 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4[44874]: 2024-05-09T06:02:23.606+0000 7f2b1e1cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:02:23.903 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:23 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4[44874]: 2024-05-09T06:02:23.606+0000 7f2b1e1cd700 -1 osd.4 127 *** Got signal Terminated *** 2024-05-09T06:02:23.903 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:23 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4[44874]: 2024-05-09T06:02:23.606+0000 7f2b1e1cd700 -1 osd.4 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:02:29.586 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:29 smithi161 podman[72989]: 2024-05-09 06:02:29.29167629 +0000 UTC m=+5.800214830 container died 0389c2e2d85c4c4cf948a53c4a05f79317ad2c757fce1f7b91ceff27203db670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T06:02:29.903 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:29 smithi161 podman[72989]: 2024-05-09 06:02:29.707602323 +0000 UTC m=+6.216140863 container cleanup 0389c2e2d85c4c4cf948a53c4a05f79317ad2c757fce1f7b91ceff27203db670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T06:02:29.903 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:29 smithi161 bash[72989]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4 2024-05-09T06:02:30.305 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:29 smithi161 podman[73006]: 2024-05-09 06:02:29.974075181 +0000 UTC m=+0.677113043 container remove 0389c2e2d85c4c4cf948a53c4a05f79317ad2c757fce1f7b91ceff27203db670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T06:02:30.582 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:30 smithi161 podman[73098]: 2024-05-09 06:02:30.223159132 +0000 UTC m=+0.018330552 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:02:30.582 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:30 smithi161 podman[73098]: 2024-05-09 06:02:30.355483197 +0000 UTC m=+0.150654608 container create ed4f63436767a217b1f13324788588d4346995bcc796f64c422158115069a78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T06:02:30.878 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:30 smithi161 podman[73098]: 2024-05-09 06:02:30.596529473 +0000 UTC m=+0.391700883 container init ed4f63436767a217b1f13324788588d4346995bcc796f64c422158115069a78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:02:30.878 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:30 smithi161 podman[73098]: 2024-05-09 06:02:30.599929057 +0000 UTC m=+0.395100476 container start ed4f63436767a217b1f13324788588d4346995bcc796f64c422158115069a78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:02:30.878 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:30 smithi161 podman[73098]: 2024-05-09 06:02:30.671268829 +0000 UTC m=+0.466440244 container attach ed4f63436767a217b1f13324788588d4346995bcc796f64c422158115069a78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-09T06:02:31.153 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 09 06:02:30 smithi161 podman[73098]: 2024-05-09 06:02:30.892718706 +0000 UTC m=+0.687890111 container died ed4f63436767a217b1f13324788588d4346995bcc796f64c422158115069a78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:02:31.500 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.4.service' 2024-05-09T06:02:31.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:02:31.540 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T06:02:31.540 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T06:02:31.540 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.5 2024-05-09T06:02:31.858 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:31 smithi195 systemd[1]: Stopping Ceph osd.5 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:02:32.358 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:31 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5[45486]: 2024-05-09T06:02:31.885+0000 7fc900e58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:02:32.359 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:31 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5[45486]: 2024-05-09T06:02:31.885+0000 7fc900e58700 -1 osd.5 127 *** Got signal Terminated *** 2024-05-09T06:02:32.359 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:31 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5[45486]: 2024-05-09T06:02:31.885+0000 7fc900e58700 -1 osd.5 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:02:37.789 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:37 smithi195 podman[73706]: 2024-05-09 06:02:37.494261523 +0000 UTC m=+5.736931530 container died 19406b5b572745bde005e0d03dbd814481d37f3092eb33802931d44db1999581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:02:38.108 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:37 smithi195 podman[73706]: 2024-05-09 06:02:37.89405662 +0000 UTC m=+6.136726636 container cleanup 19406b5b572745bde005e0d03dbd814481d37f3092eb33802931d44db1999581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:02:38.109 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:37 smithi195 bash[73706]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5 2024-05-09T06:02:38.466 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:38 smithi195 podman[73725]: 2024-05-09 06:02:38.178873624 +0000 UTC m=+0.683684921 container remove 19406b5b572745bde005e0d03dbd814481d37f3092eb33802931d44db1999581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T06:02:38.760 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:38 smithi195 podman[73814]: 2024-05-09 06:02:38.383383181 +0000 UTC m=+0.018054200 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:02:38.761 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:38 smithi195 podman[73814]: 2024-05-09 06:02:38.50489183 +0000 UTC m=+0.139562849 container create fa50bc50dbdc754cd8e9cc128d685b43abc5ef16b11b568ce3f2a417d566d23a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-09T06:02:39.053 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:38 smithi195 podman[73814]: 2024-05-09 06:02:38.774496774 +0000 UTC m=+0.409167792 container init fa50bc50dbdc754cd8e9cc128d685b43abc5ef16b11b568ce3f2a417d566d23a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:02:39.053 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:38 smithi195 podman[73814]: 2024-05-09 06:02:38.777942231 +0000 UTC m=+0.412613250 container start fa50bc50dbdc754cd8e9cc128d685b43abc5ef16b11b568ce3f2a417d566d23a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T06:02:39.053 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:38 smithi195 podman[73814]: 2024-05-09 06:02:38.882797831 +0000 UTC m=+0.517468865 container attach fa50bc50dbdc754cd8e9cc128d685b43abc5ef16b11b568ce3f2a417d566d23a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-09T06:02:39.358 INFO:journalctl@ceph.osd.5.smithi195.stdout:May 09 06:02:39 smithi195 podman[73814]: 2024-05-09 06:02:39.067502714 +0000 UTC m=+0.702173736 container died fa50bc50dbdc754cd8e9cc128d685b43abc5ef16b11b568ce3f2a417d566d23a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T06:02:39.768 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.5.service' 2024-05-09T06:02:39.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:02:39.807 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T06:02:39.807 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T06:02:39.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.6 2024-05-09T06:02:40.139 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:39 smithi115 systemd[1]: Stopping Ceph osd.6 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:02:40.639 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:40 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6[55419]: 2024-05-09T06:02:40.156+0000 7f660513d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:02:40.639 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:40 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6[55419]: 2024-05-09T06:02:40.156+0000 7f660513d700 -1 osd.6 127 *** Got signal Terminated *** 2024-05-09T06:02:40.640 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:40 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6[55419]: 2024-05-09T06:02:40.156+0000 7f660513d700 -1 osd.6 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:02:46.117 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:45 smithi115 podman[141511]: 2024-05-09 06:02:45.842408337 +0000 UTC m=+5.827038487 container died 3c1b714fe5476e4ae6976d5358cba8687c2d2c539a7cb1bd55f69aecec20f5c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:02:46.389 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:46 smithi115 podman[141511]: 2024-05-09 06:02:46.331218488 +0000 UTC m=+6.315848559 container cleanup 3c1b714fe5476e4ae6976d5358cba8687c2d2c539a7cb1bd55f69aecec20f5c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T06:02:46.389 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:46 smithi115 bash[141511]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6 2024-05-09T06:02:46.884 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:46 smithi115 podman[141528]: 2024-05-09 06:02:46.593241674 +0000 UTC m=+0.744431229 container remove 3c1b714fe5476e4ae6976d5358cba8687c2d2c539a7cb1bd55f69aecec20f5c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=) 2024-05-09T06:02:47.139 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:46 smithi115 podman[141616]: 2024-05-09 06:02:46.802012942 +0000 UTC m=+0.017536988 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:02:47.139 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:46 smithi115 podman[141616]: 2024-05-09 06:02:46.927652518 +0000 UTC m=+0.143176555 container create 11c3ece38f51a721640646e034a036f89839e18e710a7df09e36c95fd61760be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T06:02:47.489 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:47 smithi115 podman[141616]: 2024-05-09 06:02:47.205570338 +0000 UTC m=+0.421094374 container init 11c3ece38f51a721640646e034a036f89839e18e710a7df09e36c95fd61760be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T06:02:47.489 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:47 smithi115 podman[141616]: 2024-05-09 06:02:47.209651999 +0000 UTC m=+0.425176033 container start 11c3ece38f51a721640646e034a036f89839e18e710a7df09e36c95fd61760be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T06:02:47.489 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:47 smithi115 podman[141616]: 2024-05-09 06:02:47.300097535 +0000 UTC m=+0.515621577 container attach 11c3ece38f51a721640646e034a036f89839e18e710a7df09e36c95fd61760be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:02:47.785 INFO:journalctl@ceph.osd.6.smithi115.stdout:May 09 06:02:47 smithi115 podman[141616]: 2024-05-09 06:02:47.503338142 +0000 UTC m=+0.718862172 container died 11c3ece38f51a721640646e034a036f89839e18e710a7df09e36c95fd61760be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T06:02:48.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.6.service' 2024-05-09T06:02:48.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:02:48.191 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T06:02:48.191 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T06:02:48.192 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.7 2024-05-09T06:02:48.513 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:48 smithi161 systemd[1]: Stopping Ceph osd.7 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:02:48.903 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:48 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7[51684]: 2024-05-09T06:02:48.526+0000 7f496b07f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:02:48.903 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:48 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7[51684]: 2024-05-09T06:02:48.526+0000 7f496b07f700 -1 osd.7 127 *** Got signal Terminated *** 2024-05-09T06:02:48.903 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:48 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7[51684]: 2024-05-09T06:02:48.526+0000 7f496b07f700 -1 osd.7 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:02:54.389 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:54 smithi161 podman[73204]: 2024-05-09 06:02:54.106552349 +0000 UTC m=+5.703748012 container died 9ff86490d7c2e3d1121498827154a2d5fbf3073371b374964b1249bab72e5fee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:02:54.653 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:54 smithi161 podman[73204]: 2024-05-09 06:02:54.493540009 +0000 UTC m=+6.090735659 container cleanup 9ff86490d7c2e3d1121498827154a2d5fbf3073371b374964b1249bab72e5fee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T06:02:54.653 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:54 smithi161 bash[73204]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7 2024-05-09T06:02:55.147 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:54 smithi161 podman[73219]: 2024-05-09 06:02:54.728399369 +0000 UTC m=+0.620962200 container remove 9ff86490d7c2e3d1121498827154a2d5fbf3073371b374964b1249bab72e5fee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-09T06:02:55.403 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:55 smithi161 podman[73313]: 2024-05-09 06:02:55.064568309 +0000 UTC m=+0.018111715 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:02:55.403 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:55 smithi161 podman[73313]: 2024-05-09 06:02:55.18756669 +0000 UTC m=+0.141110085 container create 393be4f8579b9e5ff6fe396e3aa291d3cf70d6befab27cafb4cb709fe5d964f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T06:02:55.724 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:55 smithi161 podman[73313]: 2024-05-09 06:02:55.440554987 +0000 UTC m=+0.394098390 container init 393be4f8579b9e5ff6fe396e3aa291d3cf70d6befab27cafb4cb709fe5d964f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T06:02:55.724 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:55 smithi161 podman[73313]: 2024-05-09 06:02:55.444143824 +0000 UTC m=+0.397687217 container start 393be4f8579b9e5ff6fe396e3aa291d3cf70d6befab27cafb4cb709fe5d964f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T06:02:55.724 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:55 smithi161 podman[73313]: 2024-05-09 06:02:55.515475475 +0000 UTC m=+0.469018890 container attach 393be4f8579b9e5ff6fe396e3aa291d3cf70d6befab27cafb4cb709fe5d964f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:02:55.985 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 09 06:02:55 smithi161 podman[73313]: 2024-05-09 06:02:55.738559358 +0000 UTC m=+0.692102760 container died 393be4f8579b9e5ff6fe396e3aa291d3cf70d6befab27cafb4cb709fe5d964f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T06:02:56.304 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:02:56 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:02:55.999+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:02:56.329 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.7.service' 2024-05-09T06:02:56.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:02:56.370 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T06:02:56.370 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T06:02:56.370 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.8 2024-05-09T06:02:56.675 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:02:56 smithi195 systemd[1]: Stopping Ceph osd.8 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:02:56.938 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:02:56 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8[52188]: 2024-05-09T06:02:56.688+0000 7f841f96d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:02:56.938 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:02:56 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8[52188]: 2024-05-09T06:02:56.688+0000 7f841f96d700 -1 osd.8 127 *** Got signal Terminated *** 2024-05-09T06:02:56.938 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:02:56 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8[52188]: 2024-05-09T06:02:56.688+0000 7f841f96d700 -1 osd.8 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:02:57.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:02:56 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:02:56.974+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:02:58.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:02:57 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:02:57.941+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:02:59.304 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:02:58 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:02:58.952+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:02:59.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:02:59 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:02:59.121+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:00.389 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:00 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:00.135+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:00.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:02:59 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:02:59.997+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:00.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:00 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:00.145+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:01.358 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:00 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8[52188]: 2024-05-09T06:03:00.899+0000 7f841bf23700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:50.130394+0000 front 2024-05-09T06:01:50.130298+0000 (oldest deadline 2024-05-09T06:03:00.131075+0000) 2024-05-09T06:03:01.359 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:01 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:01.141+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:01.389 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:01 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:01.152+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:01.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:01 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:01.047+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:02.359 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:02 smithi195 podman[73921]: 2024-05-09 06:03:02.313688948 +0000 UTC m=+5.738551007 container died d380bb6ecde0b7ed3acd3933c356fd9bebfb69103b6ab560547da0e431dfcb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:03:02.359 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:02 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:02.115+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:02.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:02 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:02.064+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:02.639 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:02 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:02.191+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:02.943 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:02 smithi195 podman[73921]: 2024-05-09 06:03:02.722884153 +0000 UTC m=+6.147746211 container cleanup d380bb6ecde0b7ed3acd3933c356fd9bebfb69103b6ab560547da0e431dfcb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T06:03:02.943 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:02 smithi195 bash[73921]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8 2024-05-09T06:03:03.260 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:03 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:03.079+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:03.261 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:02 smithi195 podman[73950]: 2024-05-09 06:03:02.957704443 +0000 UTC m=+0.628010224 container remove d380bb6ecde0b7ed3acd3933c356fd9bebfb69103b6ab560547da0e431dfcb2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T06:03:03.389 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:03 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:03.146+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:03.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:03 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:03.103+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:03.588 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:03 smithi195 podman[74036]: 2024-05-09 06:03:03.275135596 +0000 UTC m=+0.114984995 container create 63940a37e447e9bc82fee4e0ea18ce17ce5c6bad9facc025f3b69c9cad907d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T06:03:03.588 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:03 smithi195 podman[74036]: 2024-05-09 06:03:03.178516355 +0000 UTC m=+0.018365765 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:03:03.588 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:03 smithi195 podman[74036]: 2024-05-09 06:03:03.502559215 +0000 UTC m=+0.342408622 container init 63940a37e447e9bc82fee4e0ea18ce17ce5c6bad9facc025f3b69c9cad907d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T06:03:03.588 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:03 smithi195 podman[74036]: 2024-05-09 06:03:03.505824662 +0000 UTC m=+0.345674065 container start 63940a37e447e9bc82fee4e0ea18ce17ce5c6bad9facc025f3b69c9cad907d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T06:03:03.588 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:03 smithi195 podman[74036]: 2024-05-09 06:03:03.602525655 +0000 UTC m=+0.442375069 container attach 63940a37e447e9bc82fee4e0ea18ce17ce5c6bad9facc025f3b69c9cad907d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T06:03:03.858 INFO:journalctl@ceph.osd.8.smithi195.stdout:May 09 06:03:03 smithi195 podman[74036]: 2024-05-09 06:03:03.804894247 +0000 UTC m=+0.644743646 container died 63940a37e447e9bc82fee4e0ea18ce17ce5c6bad9facc025f3b69c9cad907d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-09T06:03:04.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:04 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:04.101+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:04.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:04 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:04.073+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:04.499 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.8.service' 2024-05-09T06:03:04.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:04.540 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T06:03:04.541 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T06:03:04.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.9 2024-05-09T06:03:04.568 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:04.166+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:04.846 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:04 smithi115 systemd[1]: Stopping Ceph osd.9 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:05.139 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:04.859+0000 7f3c87ea6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:05.139 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:04.859+0000 7f3c87ea6700 -1 osd.9 127 *** Got signal Terminated *** 2024-05-09T06:03:05.139 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:04 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:04.859+0000 7f3c87ea6700 -1 osd.9 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:03:05.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:05 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:05.060+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:05.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:05 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:05.126+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:05.639 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:05 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:05.183+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:06.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:06 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:06.107+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:06.359 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:06 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:06.107+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:06.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:06 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:06.043+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:06.639 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:06 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:06.185+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:07.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:07 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:07.110+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:07.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:07 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:07.110+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:07.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:07 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:07.084+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:07.639 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:07 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:07.226+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:07.639 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:07 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:07.226+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:03.297125+0000 front 2024-05-09T06:02:03.297103+0000 (oldest deadline 2024-05-09T06:03:06.197083+0000) 2024-05-09T06:03:08.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:08 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:08.154+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:08.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:08 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:08.154+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:08.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:08 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:08.079+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:08.639 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:08.212+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:08.639 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:08 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:08.212+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:03.297125+0000 front 2024-05-09T06:02:03.297103+0000 (oldest deadline 2024-05-09T06:03:06.197083+0000) 2024-05-09T06:03:09.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:09 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:09.041+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:09.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:09 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:09.177+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:09.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:09 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:09.177+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:09.639 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:09 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:09.211+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.295227+0000 front 2024-05-09T06:01:53.295255+0000 (oldest deadline 2024-05-09T06:02:59.195463+0000) 2024-05-09T06:03:09.639 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:09 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9[62438]: 2024-05-09T06:03:09.211+0000 7f3c80454700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:03.297125+0000 front 2024-05-09T06:02:03.297103+0000 (oldest deadline 2024-05-09T06:03:06.197083+0000) 2024-05-09T06:03:10.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:10 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:10.036+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:10.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:10 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:10.036+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:02.003387+0000 front 2024-05-09T06:02:02.003514+0000 (oldest deadline 2024-05-09T06:03:09.604269+0000) 2024-05-09T06:03:10.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:10 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:10.206+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:10.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:10 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:10.206+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:10.889 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:10 smithi115 podman[141734]: 2024-05-09 06:03:10.625248032 +0000 UTC m=+5.871680122 container died d7fd055542de2b0caaaa3741537742a0c07c45497f9de6660cd3f27a7243e641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:03:11.389 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:11 smithi115 podman[141734]: 2024-05-09 06:03:11.184637861 +0000 UTC m=+6.431069947 container cleanup d7fd055542de2b0caaaa3741537742a0c07c45497f9de6660cd3f27a7243e641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-09T06:03:11.389 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:11 smithi115 bash[141734]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9 2024-05-09T06:03:11.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:11 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:11.042+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:11.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:11 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:11.042+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:02.003387+0000 front 2024-05-09T06:02:02.003514+0000 (oldest deadline 2024-05-09T06:03:09.604269+0000) 2024-05-09T06:03:11.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:11 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:11.190+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:11.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:11 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:11.190+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:11.808 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:11 smithi115 podman[141751]: 2024-05-09 06:03:11.519669623 +0000 UTC m=+0.884123181 container remove d7fd055542de2b0caaaa3741537742a0c07c45497f9de6660cd3f27a7243e641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T06:03:12.134 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:11 smithi115 podman[141847]: 2024-05-09 06:03:11.725933199 +0000 UTC m=+0.018313270 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:03:12.134 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:11 smithi115 podman[141847]: 2024-05-09 06:03:11.845449489 +0000 UTC m=+0.137829616 container create 5341729b5e7982611365f84ac0cee4b8979ed98dd1005dfd2b7e31ceb580ca46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T06:03:12.134 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:12 smithi115 podman[141847]: 2024-05-09 06:03:12.064774765 +0000 UTC m=+0.357154837 container init 5341729b5e7982611365f84ac0cee4b8979ed98dd1005dfd2b7e31ceb580ca46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T06:03:12.134 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:12 smithi115 podman[141847]: 2024-05-09 06:03:12.068209591 +0000 UTC m=+0.360589670 container start 5341729b5e7982611365f84ac0cee4b8979ed98dd1005dfd2b7e31ceb580ca46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T06:03:12.389 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:12 smithi115 podman[141847]: 2024-05-09 06:03:12.148109327 +0000 UTC m=+0.440489413 container attach 5341729b5e7982611365f84ac0cee4b8979ed98dd1005dfd2b7e31ceb580ca46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T06:03:12.389 INFO:journalctl@ceph.osd.9.smithi115.stdout:May 09 06:03:12 smithi115 podman[141847]: 2024-05-09 06:03:12.37723262 +0000 UTC m=+0.669612692 container died 5341729b5e7982611365f84ac0cee4b8979ed98dd1005dfd2b7e31ceb580ca46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T06:03:12.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:12 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:12.020+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:12.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:12 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:12.020+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:02.003387+0000 front 2024-05-09T06:02:02.003514+0000 (oldest deadline 2024-05-09T06:03:09.604269+0000) 2024-05-09T06:03:12.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:12 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:12.228+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:12.609 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:12 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:12.228+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:12.999 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.9.service' 2024-05-09T06:03:13.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:13.079 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T06:03:13.079 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T06:03:13.080 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.10 2024-05-09T06:03:13.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:13 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:13.004+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:13.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:13 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:13.004+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:02.003387+0000 front 2024-05-09T06:02:02.003514+0000 (oldest deadline 2024-05-09T06:03:09.604269+0000) 2024-05-09T06:03:13.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:13 smithi161 systemd[1]: Stopping Ceph osd.10 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:13.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:13 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:13.236+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:13.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:13 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:13.236+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:13.903 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:13 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:13.420+0000 7fbaf0259700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:13.903 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:13 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:13.420+0000 7fbaf0259700 -1 osd.10 127 *** Got signal Terminated *** 2024-05-09T06:03:13.903 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:13 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:13.420+0000 7fbaf0259700 -1 osd.10 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:03:14.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:13 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:13.986+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:14.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:13 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:13.986+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:02.003387+0000 front 2024-05-09T06:02:02.003514+0000 (oldest deadline 2024-05-09T06:03:09.604269+0000) 2024-05-09T06:03:14.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:13 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:13.986+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.605084+0000 front 2024-05-09T06:02:09.604946+0000 (oldest deadline 2024-05-09T06:03:13.705104+0000) 2024-05-09T06:03:14.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:14 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:14.230+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:14.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:14 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:14.230+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:14.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:14 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:14.230+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:15.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:14 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:14.961+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:15.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:14 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:14.961+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:02.003387+0000 front 2024-05-09T06:02:02.003514+0000 (oldest deadline 2024-05-09T06:03:09.604269+0000) 2024-05-09T06:03:15.403 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:14 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:14.961+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.605084+0000 front 2024-05-09T06:02:09.604946+0000 (oldest deadline 2024-05-09T06:03:13.705104+0000) 2024-05-09T06:03:15.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:15 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:15.250+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:15.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:15 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:15.250+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:15.609 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:15 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:15.250+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:16.153 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:15 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:15.916+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:16.153 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:15 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:15.916+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:02.003387+0000 front 2024-05-09T06:02:02.003514+0000 (oldest deadline 2024-05-09T06:03:09.604269+0000) 2024-05-09T06:03:16.153 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:15 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:15.916+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.605084+0000 front 2024-05-09T06:02:09.604946+0000 (oldest deadline 2024-05-09T06:03:13.705104+0000) 2024-05-09T06:03:16.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:16 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:16.237+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:16.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:16 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:16.237+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:16.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:16 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:16.237+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:17.153 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:16 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:16.870+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:17.153 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:16 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:16.870+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:02.003387+0000 front 2024-05-09T06:02:02.003514+0000 (oldest deadline 2024-05-09T06:03:09.604269+0000) 2024-05-09T06:03:17.153 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:16 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:16.870+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.605084+0000 front 2024-05-09T06:02:09.604946+0000 (oldest deadline 2024-05-09T06:03:13.705104+0000) 2024-05-09T06:03:17.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:17 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:17.274+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:17.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:17 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:17.274+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:17.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:17 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:17.274+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:18.153 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:17 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:17.868+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:53.500619+0000 front 2024-05-09T06:01:53.500516+0000 (oldest deadline 2024-05-09T06:02:55.701028+0000) 2024-05-09T06:03:18.153 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:17 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:17.868+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:02:02.003387+0000 front 2024-05-09T06:02:02.003514+0000 (oldest deadline 2024-05-09T06:03:09.604269+0000) 2024-05-09T06:03:18.153 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:17 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10[58411]: 2024-05-09T06:03:17.868+0000 7fbae8807700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.605084+0000 front 2024-05-09T06:02:09.604946+0000 (oldest deadline 2024-05-09T06:03:13.705104+0000) 2024-05-09T06:03:18.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:18 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:18.271+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:18.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:18 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:18.271+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:18.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:18 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:18.271+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:19.266 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:19 smithi161 podman[73434]: 2024-05-09 06:03:19.002063477 +0000 UTC m=+5.709293607 container died f53fbaf2a392965b7f49fe0662a2293722e379dbe8de2d01603349a2627a60b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:03:19.609 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:19 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:19.321+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:19.609 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:19 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:19.321+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:19.609 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:19 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:19.321+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:19.632 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:19 smithi161 podman[73434]: 2024-05-09 06:03:19.362372098 +0000 UTC m=+6.069602218 container cleanup f53fbaf2a392965b7f49fe0662a2293722e379dbe8de2d01603349a2627a60b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T06:03:19.632 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:19 smithi161 bash[73434]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10 2024-05-09T06:03:19.903 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:19 smithi161 podman[73452]: 2024-05-09 06:03:19.646986476 +0000 UTC m=+0.645420248 container remove f53fbaf2a392965b7f49fe0662a2293722e379dbe8de2d01603349a2627a60b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T06:03:20.269 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:19 smithi161 podman[73542]: 2024-05-09 06:03:19.886048766 +0000 UTC m=+0.017996879 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:03:20.269 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:19 smithi161 podman[73542]: 2024-05-09 06:03:19.989322732 +0000 UTC m=+0.121270833 container create 5525ff94bfcf540d05cae842685db25102c7a7a8b79b048c611662446893f7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T06:03:20.269 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:20 smithi161 podman[73542]: 2024-05-09 06:03:20.200055336 +0000 UTC m=+0.332003436 container init 5525ff94bfcf540d05cae842685db25102c7a7a8b79b048c611662446893f7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T06:03:20.269 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:20 smithi161 podman[73542]: 2024-05-09 06:03:20.203497816 +0000 UTC m=+0.335445917 container start 5525ff94bfcf540d05cae842685db25102c7a7a8b79b048c611662446893f7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T06:03:20.269 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:20 smithi161 podman[73542]: 2024-05-09 06:03:20.283148175 +0000 UTC m=+0.415096275 container attach 5525ff94bfcf540d05cae842685db25102c7a7a8b79b048c611662446893f7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T06:03:20.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:20 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:20.334+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:20.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:20 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:20.334+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:20.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:20 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:20.334+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:20.653 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:20 smithi161 podman[73542]: 2024-05-09 06:03:20.499451858 +0000 UTC m=+0.631399957 container died 5525ff94bfcf540d05cae842685db25102c7a7a8b79b048c611662446893f7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T06:03:21.018 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.10.service' 2024-05-09T06:03:21.045 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:21 smithi161 podman[73568]: 2024-05-09 06:03:21.011277833 +0000 UTC m=+0.506363906 container remove 5525ff94bfcf540d05cae842685db25102c7a7a8b79b048c611662446893f7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:03:21.045 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:21 smithi161 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.10.service: Deactivated successfully. 2024-05-09T06:03:21.045 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:21 smithi161 systemd[1]: Stopped Ceph osd.10 for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T06:03:21.045 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 09 06:03:21 smithi161 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.10.service: Consumed 5min 16.529s CPU time. 2024-05-09T06:03:21.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:21.061 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T06:03:21.061 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T06:03:21.061 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.11 2024-05-09T06:03:21.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:21 smithi195 systemd[1]: Stopping Ceph osd.11 for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:21.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:21 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:21.292+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:21.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:21 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:21.292+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:21.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:21 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:21.292+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:21.858 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:21 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:21.375+0000 7fe61dd11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:21.858 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:21 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:21.375+0000 7fe61dd11700 -1 osd.11 127 *** Got signal Terminated *** 2024-05-09T06:03:21.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:21 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:21.375+0000 7fe61dd11700 -1 osd.11 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T06:03:22.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:22 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:22.251+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:22.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:22 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:22.251+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:22.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:22 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:22.251+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:23.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:23 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:23.298+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:23.609 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:23 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:23.298+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:23.609 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:23 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:23.298+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:24.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:24 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:24.313+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:24.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:24 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:24.313+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:24.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:24 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:24.313+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:25.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:25 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:25.343+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:25.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:25 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:25.343+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:25.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:25 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:25.343+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:26.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:26 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:26.332+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.115:6806 osd.0 since back 2024-05-09T06:01:54.072017+0000 front 2024-05-09T06:01:54.072088+0000 (oldest deadline 2024-05-09T06:02:58.771964+0000) 2024-05-09T06:03:26.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:26 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:26.332+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-09T06:01:59.873166+0000 front 2024-05-09T06:01:59.873206+0000 (oldest deadline 2024-05-09T06:03:05.173221+0000) 2024-05-09T06:03:26.608 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:26 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11[58740]: 2024-05-09T06:03:26.332+0000 7fe61a2c7700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-05-09T06:02:09.874488+0000 front 2024-05-09T06:02:09.874512+0000 (oldest deadline 2024-05-09T06:03:13.974688+0000) 2024-05-09T06:03:27.358 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:27 smithi195 podman[74141]: 2024-05-09 06:03:27.109597498 +0000 UTC m=+5.843281729 container died 5e0565e2fe3b9755ff406ca22d34de4121bd0451a966f1e33ca0bdc7b68d0e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T06:03:27.858 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:27 smithi195 podman[74141]: 2024-05-09 06:03:27.534083589 +0000 UTC m=+6.267767819 container cleanup 5e0565e2fe3b9755ff406ca22d34de4121bd0451a966f1e33ca0bdc7b68d0e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:03:27.858 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:27 smithi195 bash[74141]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11 2024-05-09T06:03:27.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:27 smithi195 podman[74158]: 2024-05-09 06:03:27.752283159 +0000 UTC m=+0.647256635 container remove 5e0565e2fe3b9755ff406ca22d34de4121bd0451a966f1e33ca0bdc7b68d0e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-09T06:03:28.504 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:28 smithi195 podman[74252]: 2024-05-09 06:03:28.098244422 +0000 UTC m=+0.018546839 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T06:03:28.504 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:28 smithi195 podman[74252]: 2024-05-09 06:03:28.21137885 +0000 UTC m=+0.131681256 container create 5c9e60fd5b0b4a7a53880248027563cbaccf0d31a16d4753f24c05bae036cbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T06:03:28.505 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:28 smithi195 podman[74252]: 2024-05-09 06:03:28.43026086 +0000 UTC m=+0.350563260 container init 5c9e60fd5b0b4a7a53880248027563cbaccf0d31a16d4753f24c05bae036cbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_CLEAN=True) 2024-05-09T06:03:28.505 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:28 smithi195 podman[74252]: 2024-05-09 06:03:28.433683097 +0000 UTC m=+0.353985508 container start 5c9e60fd5b0b4a7a53880248027563cbaccf0d31a16d4753f24c05bae036cbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:03:28.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:28 smithi195 podman[74252]: 2024-05-09 06:03:28.522029573 +0000 UTC m=+0.442331974 container attach 5c9e60fd5b0b4a7a53880248027563cbaccf0d31a16d4753f24c05bae036cbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T06:03:28.859 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:28 smithi195 podman[74252]: 2024-05-09 06:03:28.730866541 +0000 UTC m=+0.651168933 container died 5c9e60fd5b0b4a7a53880248027563cbaccf0d31a16d4753f24c05bae036cbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T06:03:29.272 INFO:journalctl@ceph.osd.11.smithi195.stdout:May 09 06:03:29 smithi195 podman[74279]: 2024-05-09 06:03:29.259045213 +0000 UTC m=+0.522883893 container remove 5c9e60fd5b0b4a7a53880248027563cbaccf0d31a16d4753f24c05bae036cbec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5) 2024-05-09T06:03:29.278 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@osd.11.service' 2024-05-09T06:03:29.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:29.319 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T06:03:29.320 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T06:03:29.320 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.a 2024-05-09T06:03:29.639 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 06:03:29 smithi115 systemd[1]: Stopping Ceph mds.a for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:30.139 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 06:03:29 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-a[69603]: 2024-05-09T06:03:29.660+0000 7f5d6ac0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:30.139 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 06:03:29 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-a[69603]: 2024-05-09T06:03:29.660+0000 7f5d6ac0d700 -1 mds.a *** got signal Terminated *** 2024-05-09T06:03:33.430 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 06:03:33 smithi115 podman[141964]: 2024-05-09 06:03:33.165258402 +0000 UTC m=+3.638923377 container died 215309d10b0c8d22b58bdb9e64d4585b672aa8cdbca0e9c08ab3396efb22cb76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T06:03:33.789 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 06:03:33 smithi115 podman[141964]: 2024-05-09 06:03:33.57725766 +0000 UTC m=+4.050922672 container cleanup 215309d10b0c8d22b58bdb9e64d4585b672aa8cdbca0e9c08ab3396efb22cb76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-a, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T06:03:33.789 INFO:journalctl@ceph.mds.a.smithi115.stdout:May 09 06:03:33 smithi115 bash[141964]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-a 2024-05-09T06:03:34.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.a.service' 2024-05-09T06:03:34.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:34.036 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T06:03:34.036 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T06:03:34.036 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.d 2024-05-09T06:03:34.372 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:34 smithi115 systemd[1]: Stopping Ceph mds.d for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:34.639 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:34 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-d[68329]: 2024-05-09T06:03:34.386+0000 7f54bed97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:34.639 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:34 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-d[68329]: 2024-05-09T06:03:34.386+0000 7f54bed97700 -1 mds.d *** got signal Terminated *** 2024-05-09T06:03:38.639 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:38 smithi115 podman[142085]: 2024-05-09 06:03:38.393743052 +0000 UTC m=+4.128005107 container died 3299ca8f441b22158220397117655a45244f0579097635957c5ed7c602868ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-d, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-09T06:03:39.001 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:38 smithi115 podman[142085]: 2024-05-09 06:03:38.830866824 +0000 UTC m=+4.565128889 container cleanup 3299ca8f441b22158220397117655a45244f0579097635957c5ed7c602868ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:03:39.001 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:38 smithi115 bash[142085]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-d 2024-05-09T06:03:39.245 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.d.service' 2024-05-09T06:03:39.266 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:39 smithi115 podman[142099]: 2024-05-09 06:03:39.015519061 +0000 UTC m=+0.615532026 container remove 3299ca8f441b22158220397117655a45244f0579097635957c5ed7c602868ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-09T06:03:39.266 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:39 smithi115 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.d.service: Deactivated successfully. 2024-05-09T06:03:39.266 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:39 smithi115 systemd[1]: Stopped Ceph mds.d for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T06:03:39.266 INFO:journalctl@ceph.mds.d.smithi115.stdout:May 09 06:03:39 smithi115 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.d.service: Consumed 1.251s CPU time. 2024-05-09T06:03:39.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:39.279 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T06:03:39.279 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T06:03:39.279 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.g 2024-05-09T06:03:39.613 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:39 smithi115 systemd[1]: Stopping Ceph mds.g for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:39.889 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:39 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g[69157]: 2024-05-09T06:03:39.627+0000 7f1931579700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:39.889 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:39 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g[69157]: 2024-05-09T06:03:39.627+0000 7f1931579700 -1 mds.g *** got signal Terminated *** 2024-05-09T06:03:41.104 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:40 smithi115 podman[142204]: 2024-05-09 06:03:40.864346307 +0000 UTC m=+1.348212874 container died 94da5797eb467d8923b44e06dbca7f3ee27fdad8216f1b03b38517a89507a064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T06:03:41.371 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:41 smithi115 podman[142204]: 2024-05-09 06:03:41.201863311 +0000 UTC m=+1.685729885 container cleanup 94da5797eb467d8923b44e06dbca7f3ee27fdad8216f1b03b38517a89507a064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T06:03:41.372 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:41 smithi115 bash[142204]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g 2024-05-09T06:03:41.606 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.g.service' 2024-05-09T06:03:41.627 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:41 smithi115 podman[142218]: 2024-05-09 06:03:41.386368908 +0000 UTC m=+0.523637299 container remove 94da5797eb467d8923b44e06dbca7f3ee27fdad8216f1b03b38517a89507a064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:03:41.627 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:41 smithi115 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-09T06:03:41.627 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:41 smithi115 systemd[1]: Stopped Ceph mds.g for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T06:03:41.627 INFO:journalctl@ceph.mds.g.smithi115.stdout:May 09 06:03:41 smithi115 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.g.service: Consumed 3.364s CPU time. 2024-05-09T06:03:41.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:41.638 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T06:03:41.638 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T06:03:41.639 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.j 2024-05-09T06:03:41.889 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:41 smithi115 systemd[1]: Stopping Ceph mds.j for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:42.389 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:41 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-j[67882]: 2024-05-09T06:03:41.988+0000 7fbcb9d69700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:42.389 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:41 smithi115 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-j[67882]: 2024-05-09T06:03:41.988+0000 7fbcb9d69700 -1 mds.j *** got signal Terminated *** 2024-05-09T06:03:46.283 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:46 smithi115 podman[142325]: 2024-05-09 06:03:46.003830299 +0000 UTC m=+4.138555458 container died d8fa90235cf3ea298b45766395fc2ea0f495803408c3d05b5f6f247a77e187d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T06:03:46.550 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:46 smithi115 podman[142325]: 2024-05-09 06:03:46.380243873 +0000 UTC m=+4.514969025 container cleanup d8fa90235cf3ea298b45766395fc2ea0f495803408c3d05b5f6f247a77e187d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T06:03:46.550 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:46 smithi115 bash[142325]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-j 2024-05-09T06:03:46.788 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.j.service' 2024-05-09T06:03:46.809 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:46 smithi115 podman[142337]: 2024-05-09 06:03:46.564941705 +0000 UTC m=+0.562887611 container remove d8fa90235cf3ea298b45766395fc2ea0f495803408c3d05b5f6f247a77e187d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T06:03:46.810 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:46 smithi115 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.j.service: Deactivated successfully. 2024-05-09T06:03:46.810 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:46 smithi115 systemd[1]: Stopped Ceph mds.j for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T06:03:46.810 INFO:journalctl@ceph.mds.j.smithi115.stdout:May 09 06:03:46 smithi115 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.j.service: Consumed 1.270s CPU time. 2024-05-09T06:03:46.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:46.820 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T06:03:46.820 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T06:03:46.821 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.b 2024-05-09T06:03:47.123 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 06:03:46 smithi161 systemd[1]: Stopping Ceph mds.b for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:47.394 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 06:03:47 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-b[63602]: 2024-05-09T06:03:47.136+0000 7f0d377aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:47.394 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 06:03:47 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-b[63602]: 2024-05-09T06:03:47.136+0000 7f0d377aa700 -1 mds.b *** got signal Terminated *** 2024-05-09T06:03:50.840 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 06:03:50 smithi161 podman[73648]: 2024-05-09 06:03:50.593230787 +0000 UTC m=+3.560056750 container died 937d4c8c6dd8a6dcb914f0916a248c16ccca391ae36e891ef7cfabadc527cf81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T06:03:51.153 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 06:03:50 smithi161 podman[73648]: 2024-05-09 06:03:50.953113582 +0000 UTC m=+3.919939548 container cleanup 937d4c8c6dd8a6dcb914f0916a248c16ccca391ae36e891ef7cfabadc527cf81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-b, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:03:51.153 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 09 06:03:50 smithi161 bash[73648]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-b 2024-05-09T06:03:51.366 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.b.service' 2024-05-09T06:03:51.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:51.402 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T06:03:51.403 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T06:03:51.403 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.e 2024-05-09T06:03:51.653 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 06:03:51 smithi161 systemd[1]: Stopping Ceph mds.e for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:52.153 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 06:03:51 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-e[62984]: 2024-05-09T06:03:51.773+0000 7f2b02b41700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:52.153 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 06:03:51 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-e[62984]: 2024-05-09T06:03:51.773+0000 7f2b02b41700 -1 mds.e *** got signal Terminated *** 2024-05-09T06:03:54.403 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 06:03:54 smithi161 podman[73784]: 2024-05-09 06:03:54.129886587 +0000 UTC m=+2.493334720 container died 5bd2dbf594bf8c0e7d7c603573ac88e8f2620fcd4de454af26d114d9aeae8973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:03:54.802 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 06:03:54 smithi161 podman[73784]: 2024-05-09 06:03:54.548947211 +0000 UTC m=+2.912395336 container cleanup 5bd2dbf594bf8c0e7d7c603573ac88e8f2620fcd4de454af26d114d9aeae8973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0) 2024-05-09T06:03:54.802 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 09 06:03:54 smithi161 bash[73784]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-e 2024-05-09T06:03:55.027 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.e.service' 2024-05-09T06:03:55.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:03:55.061 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T06:03:55.061 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T06:03:55.062 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.h 2024-05-09T06:03:55.389 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 06:03:55 smithi161 systemd[1]: Stopping Ceph mds.h for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:03:55.652 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 06:03:55 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-h[63292]: 2024-05-09T06:03:55.402+0000 7f9acb72c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:03:55.653 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 06:03:55 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-h[63292]: 2024-05-09T06:03:55.402+0000 7f9acb72c700 -1 mds.h *** got signal Terminated *** 2024-05-09T06:03:59.938 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 06:03:59 smithi161 podman[73906]: 2024-05-09 06:03:59.680418077 +0000 UTC m=+4.385671496 container died 460ce5a078fce9e23c7db156c89f5504c72bc608abbbc9bc349347f08dd285ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:04:00.305 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 06:04:00 smithi161 podman[73906]: 2024-05-09 06:04:00.035517754 +0000 UTC m=+4.740771186 container cleanup 460ce5a078fce9e23c7db156c89f5504c72bc608abbbc9bc349347f08dd285ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T06:04:00.305 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 09 06:04:00 smithi161 bash[73906]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-h 2024-05-09T06:04:00.439 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.h.service' 2024-05-09T06:04:00.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:04:00.472 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T06:04:00.472 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T06:04:00.472 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.k 2024-05-09T06:04:00.652 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:00 smithi161 systemd[1]: Stopping Ceph mds.k for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:04:01.153 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:00 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k[62664]: 2024-05-09T06:04:00.805+0000 7f7e34d6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:04:01.153 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:00 smithi161 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k[62664]: 2024-05-09T06:04:00.805+0000 7f7e34d6b700 -1 mds.k *** got signal Terminated *** 2024-05-09T06:04:03.116 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:02 smithi161 podman[74027]: 2024-05-09 06:04:02.844453713 +0000 UTC m=+2.136464818 container died f22c6e34e52f68223b288f4d46afdb3a07c8603894cbc77f75e20719f4d31e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:04:03.379 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:03 smithi161 podman[74027]: 2024-05-09 06:04:03.23083327 +0000 UTC m=+2.522844382 container cleanup f22c6e34e52f68223b288f4d46afdb3a07c8603894cbc77f75e20719f4d31e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:04:03.380 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:03 smithi161 bash[74027]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k 2024-05-09T06:04:03.607 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.k.service' 2024-05-09T06:04:03.631 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:03 smithi161 podman[74041]: 2024-05-09 06:04:03.515188922 +0000 UTC m=+0.658217742 container remove f22c6e34e52f68223b288f4d46afdb3a07c8603894cbc77f75e20719f4d31e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T06:04:03.631 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:03 smithi161 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.k.service: Deactivated successfully. 2024-05-09T06:04:03.631 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:03 smithi161 systemd[1]: Stopped Ceph mds.k for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T06:04:03.631 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 09 06:04:03 smithi161 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.k.service: Consumed 3.938s CPU time. 2024-05-09T06:04:03.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:04:03.641 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T06:04:03.641 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T06:04:03.641 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.c 2024-05-09T06:04:03.945 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 06:04:03 smithi195 systemd[1]: Stopping Ceph mds.c for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:04:04.207 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 06:04:03 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-c[63894]: 2024-05-09T06:04:03.958+0000 7f19f25b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:04:04.207 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 06:04:03 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-c[63894]: 2024-05-09T06:04:03.958+0000 7f19f25b2700 -1 mds.c *** got signal Terminated *** 2024-05-09T06:04:08.858 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 06:04:08 smithi195 podman[74359]: 2024-05-09 06:04:08.601176549 +0000 UTC m=+4.746539715 container died c411b27886fdb37ef89b542d831a0e7920ca08c93af85616e7e965ec2e568081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:04:09.228 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 06:04:09 smithi195 podman[74359]: 2024-05-09 06:04:09.016987516 +0000 UTC m=+5.162350678 container cleanup c411b27886fdb37ef89b542d831a0e7920ca08c93af85616e7e965ec2e568081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:04:09.229 INFO:journalctl@ceph.mds.c.smithi195.stdout:May 09 06:04:09 smithi195 bash[74359]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-c 2024-05-09T06:04:09.436 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.c.service' 2024-05-09T06:04:09.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:04:09.472 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T06:04:09.472 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T06:04:09.472 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.f 2024-05-09T06:04:09.790 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 06:04:09 smithi195 systemd[1]: Stopping Ceph mds.f for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:04:10.108 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 06:04:09 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T06:04:09.803+0000 7f7f01754700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:04:10.108 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 06:04:09 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f[63584]: 2024-05-09T06:04:09.803+0000 7f7f01754700 -1 mds.f *** got signal Terminated *** 2024-05-09T06:04:14.040 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 06:04:13 smithi195 podman[74492]: 2024-05-09 06:04:13.668644199 +0000 UTC m=+3.963007672 container died 993232dbb6ea64f3487ca8248c873f75db3aa8927d4dcc89f1c7872d3f86507e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T06:04:14.358 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 06:04:14 smithi195 podman[74492]: 2024-05-09 06:04:14.145952315 +0000 UTC m=+4.440315799 container cleanup 993232dbb6ea64f3487ca8248c873f75db3aa8927d4dcc89f1c7872d3f86507e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T06:04:14.358 INFO:journalctl@ceph.mds.f.smithi195.stdout:May 09 06:04:14 smithi195 bash[74492]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-f 2024-05-09T06:04:14.513 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.f.service' 2024-05-09T06:04:14.548 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:04:14.548 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T06:04:14.548 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T06:04:14.549 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.i 2024-05-09T06:04:14.858 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 06:04:14 smithi195 systemd[1]: Stopping Ceph mds.i for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:04:15.199 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 06:04:14 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T06:04:14.873+0000 7f9d6ad23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:04:15.199 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 06:04:14 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i[64207]: 2024-05-09T06:04:14.873+0000 7f9d6ad23700 -1 mds.i *** got signal Terminated *** 2024-05-09T06:04:21.276 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 06:04:21 smithi195 podman[74612]: 2024-05-09 06:04:21.040790436 +0000 UTC m=+6.258482221 container died c7a2a7b858744d3b3ef27a253e48f1b24e5117ae4e8dd22a60df79ce1e0e6dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T06:04:21.608 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 06:04:21 smithi195 podman[74612]: 2024-05-09 06:04:21.390840207 +0000 UTC m=+6.608532001 container cleanup c7a2a7b858744d3b3ef27a253e48f1b24e5117ae4e8dd22a60df79ce1e0e6dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T06:04:21.608 INFO:journalctl@ceph.mds.i.smithi195.stdout:May 09 06:04:21 smithi195 bash[74612]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-i 2024-05-09T06:04:21.808 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.i.service' 2024-05-09T06:04:21.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:04:21.841 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T06:04:21.841 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T06:04:21.841 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l 2024-05-09T06:04:22.108 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:21 smithi195 systemd[1]: Stopping Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968... 2024-05-09T06:04:22.608 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:22 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[71696]: 2024-05-09T06:04:22.220+0000 7f1f32a69700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T06:04:22.608 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:22 smithi195 ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l[71696]: 2024-05-09T06:04:22.220+0000 7f1f32a69700 -1 mds.l *** got signal Terminated *** 2024-05-09T06:04:24.713 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:24 smithi195 podman[74732]: 2024-05-09 06:04:24.474357775 +0000 UTC m=+2.402071369 container died 7e9e6957cbf73ee7d9633237d138dc3606d4a1e108df23e1bd111bdd5561c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T06:04:24.993 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:24 smithi195 podman[74732]: 2024-05-09 06:04:24.827905587 +0000 UTC m=+2.755619165 container cleanup 7e9e6957cbf73ee7d9633237d138dc3606d4a1e108df23e1bd111bdd5561c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T06:04:24.993 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:24 smithi195 bash[74732]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l 2024-05-09T06:04:25.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service' 2024-05-09T06:04:25.246 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:25 smithi195 podman[74746]: 2024-05-09 06:04:25.115106146 +0000 UTC m=+0.642770697 container remove 7e9e6957cbf73ee7d9633237d138dc3606d4a1e108df23e1bd111bdd5561c8c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-69de8a38-0dc4-11ef-bc97-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T06:04:25.246 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:25 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Deactivated successfully. 2024-05-09T06:04:25.246 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:25 smithi195 systemd[1]: Stopped Ceph mds.l for 69de8a38-0dc4-11ef-bc97-c7b262605968. 2024-05-09T06:04:25.246 INFO:journalctl@ceph.mds.l.smithi195.stdout:May 09 06:04:25 smithi195 systemd[1]: ceph-69de8a38-0dc4-11ef-bc97-c7b262605968@mds.l.service: Consumed 1.562s CPU time. 2024-05-09T06:04:25.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T06:04:25.256 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T06:04:25.257 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T06:04:25.448 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T06:04:27.946 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T06:04:28.138 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T06:04:30.708 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T06:04:30.902 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T06:04:33.351 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:04:33.385 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:04:33.419 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T06:04:33.446 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T06:04:33.448 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/remote/smithi115/crash 2024-05-09T06:04:33.449 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/crash -- . 2024-05-09T06:04:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T06:04:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2024-05-09T06:04:33.495 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/remote/smithi161/crash 2024-05-09T06:04:33.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/crash -- . 2024-05-09T06:04:33.545 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T06:04:33.545 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-05-09T06:04:33.547 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/remote/smithi195/crash 2024-05-09T06:04:33.548 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/crash -- . 2024-05-09T06:04:33.604 INFO:teuthology.orchestra.run.smithi195.stderr:tar: /var/lib/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T06:04:33.604 INFO:teuthology.orchestra.run.smithi195.stderr:tar: Error is not recoverable: exiting now 2024-05-09T06:04:33.605 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T06:04:33.606 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T06:04:33.685 INFO:teuthology.orchestra.run.smithi115.stdout:2024-05-09T05:41:15.622797+0000 mds.i (mds.0) 24 : cluster [WRN] Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/0735efc5-9f8c-4467-8b69-740c3189ba00/client.0/tmp/ffsb/.deps) see mds.i log and `damage ls` output for details 2024-05-09T06:04:33.686 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-09T06:04:33.686 INFO:tasks.cephadm:Compressing logs... 2024-05-09T06:04:33.686 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T06:04:33.688 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T06:04:33.691 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-09T06:04:33.724 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T06:04:33.725 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-09T06:04:33.725 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-09T06:04:33.725 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T06:04:33.725 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-09T06:04:33.725 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T06:04:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T06:04:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log 2024-05-09T06:04:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T06:04:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-09T06:04:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-09T06:04:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-09T06:04:33.732 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-09T06:04:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-09T06:04:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-09T06:04:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-09T06:04:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T06:04:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-09T06:04:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T06:04:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-09T06:04:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-09T06:04:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T06:04:33.736 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T06:04:33.736 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-09T06:04:33.736 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log 2024-05-09T06:04:33.736 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T06:04:33.737 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T06:04:33.740 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-09T06:04:33.741 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-09T06:04:33.742 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T06:04:33.742 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.d.log: 87.9% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T06:04:33.742 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-09T06:04:33.758 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 84.2% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T06:04:33.806 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-09T06:04:33.817 INFO:teuthology.orchestra.run.smithi115.stderr: 94.4% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-09T06:04:33.828 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 84.3% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T06:04:33.847 INFO:teuthology.orchestra.run.smithi115.stderr: 94.3% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-09T06:04:33.886 INFO:teuthology.orchestra.run.smithi161.stderr: 90.5% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T06:04:33.960 INFO:teuthology.orchestra.run.smithi161.stderr: 91.0% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T06:04:33.968 INFO:teuthology.orchestra.run.smithi115.stderr: 90.4% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T06:04:33.969 INFO:teuthology.orchestra.run.smithi195.stderr: 91.0% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T06:04:33.997 INFO:teuthology.orchestra.run.smithi161.stderr: 92.6% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T06:04:34.021 INFO:teuthology.orchestra.run.smithi161.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T06:04:34.051 INFO:teuthology.orchestra.run.smithi115.stderr: 84.5% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T06:04:34.062 INFO:teuthology.orchestra.run.smithi195.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T06:04:34.065 INFO:teuthology.orchestra.run.smithi115.stderr: 91.0% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T06:04:34.070 INFO:teuthology.orchestra.run.smithi115.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T06:04:34.094 INFO:teuthology.orchestra.run.smithi161.stderr: 87.9% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T06:04:34.133 INFO:teuthology.orchestra.run.smithi195.stderr: 90.5% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T06:04:34.147 INFO:teuthology.orchestra.run.smithi195.stderr: 87.9% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T06:04:34.224 INFO:teuthology.orchestra.run.smithi115.stderr: 94.6% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-09T06:04:34.226 INFO:teuthology.orchestra.run.smithi195.stderr: 92.6% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T06:04:34.242 INFO:teuthology.orchestra.run.smithi115.stderr: 94.6% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-09T06:04:35.236 INFO:teuthology.orchestra.run.smithi195.stderr: 94.2% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-09T06:04:35.870 INFO:teuthology.orchestra.run.smithi115.stderr: 90.2% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T06:04:35.918 INFO:teuthology.orchestra.run.smithi161.stderr: 94.0% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-09T06:04:36.322 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-09T06:04:37.706 INFO:teuthology.orchestra.run.smithi161.stderr: 91.8% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-09T06:04:39.338 INFO:teuthology.orchestra.run.smithi195.stderr: 93.8% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-09T06:04:39.625 INFO:teuthology.orchestra.run.smithi161.stderr: 93.5% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-09T06:04:45.383 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-09T06:04:46.471 INFO:teuthology.orchestra.run.smithi195.stderr: 93.8% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-09T06:04:53.449 INFO:teuthology.orchestra.run.smithi161.stderr: 94.8% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-09T06:04:53.533 INFO:teuthology.orchestra.run.smithi115.stderr: 90.2% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-09T06:05:00.282 INFO:teuthology.orchestra.run.smithi195.stderr: 93.7% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-09T06:05:32.132 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-09T06:06:34.301 INFO:teuthology.orchestra.run.smithi195.stderr: 91.9% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-09T06:07:46.260 INFO:teuthology.orchestra.run.smithi161.stderr: 91.6% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-09T06:08:06.037 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-09T06:09:37.779 INFO:teuthology.orchestra.run.smithi161.stderr: 91.8% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-09T06:09:58.672 INFO:teuthology.orchestra.run.smithi161.stderr: 91.7% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-09T06:09:58.674 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-05-09T06:09:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:real 5m24.969s 2024-05-09T06:09:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:user 5m57.308s 2024-05-09T06:09:58.674 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m13.817s 2024-05-09T06:10:01.774 INFO:teuthology.orchestra.run.smithi195.stderr: 91.8% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-09T06:11:13.886 INFO:teuthology.orchestra.run.smithi195.stderr: 91.6% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-09T06:11:13.887 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-05-09T06:11:13.887 INFO:teuthology.orchestra.run.smithi195.stderr:real 6m40.183s 2024-05-09T06:11:13.887 INFO:teuthology.orchestra.run.smithi195.stderr:user 6m11.383s 2024-05-09T06:11:13.887 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m14.402s 2024-05-09T06:14:31.921 INFO:teuthology.orchestra.run.smithi115.stderr: 91.7% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-09T06:17:08.461 INFO:teuthology.orchestra.run.smithi115.stderr: 91.6% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-09T06:18:23.251 INFO:teuthology.orchestra.run.smithi115.stderr: 91.7% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-09T06:25:43.073 INFO:teuthology.orchestra.run.smithi115.stderr: 91.5% -- replaced with /var/log/ceph/69de8a38-0dc4-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-09T06:26:33.629 INFO:teuthology.orchestra.run.smithi115.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-09T06:26:33.631 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-05-09T06:26:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:real 21m59.933s 2024-05-09T06:26:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:user 9m19.626s 2024-05-09T06:26:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m26.677s 2024-05-09T06:26:33.631 INFO:tasks.cephadm:Archiving logs... 2024-05-09T06:26:33.632 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/remote/smithi115/log 2024-05-09T06:26:33.633 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T06:28:03.150 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/remote/smithi161/log 2024-05-09T06:28:03.151 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T06:29:04.110 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/remote/smithi195/log 2024-05-09T06:29:04.111 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T06:30:08.459 INFO:tasks.cephadm:Removing cluster... 2024-05-09T06:30:08.460 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 --force 2024-05-09T06:30:08.659 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T06:30:09.401 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 --force 2024-05-09T06:30:09.602 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T06:30:10.297 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69de8a38-0dc4-11ef-bc97-c7b262605968 --force 2024-05-09T06:30:10.493 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: 69de8a38-0dc4-11ef-bc97-c7b262605968 2024-05-09T06:30:11.204 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T06:30:11.204 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T06:30:11.224 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T06:30:11.241 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T06:30:11.258 INFO:tasks.cephadm:Teardown complete 2024-05-09T06:30:11.259 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T06:30:11.269 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-09T06:30:11.270 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T06:30:11.270 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T06:30:11.273 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T06:30:11.283 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T06:30:11.408 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T06:30:11.420 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T06:30:11.420 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:30:11.423 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:30:11.425 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T06:30:11.438 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:30:11.439 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:30:11.439 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-05-09T06:30:11.452 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:30:11.452 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-05-09T06:30:11.452 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +1623us[+1629us] +/- 56ms 2024-05-09T06:30:11.452 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -1127us[-1121us] +/- 79ms 2024-05-09T06:30:11.452 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -1179us[-1173us] +/- 66ms 2024-05-09T06:30:11.453 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:30:11.455 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:30:11.455 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-05-09T06:30:11.455 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +1650us[+1654us] +/- 56ms 2024-05-09T06:30:11.455 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 -1104us[-1104us] +/- 79ms 2024-05-09T06:30:11.455 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -1166us[-1162us] +/- 66ms 2024-05-09T06:30:11.455 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:30:11.464 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T06:30:11.464 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-05-09T06:30:11.464 INFO:teuthology.orchestra.run.smithi195.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +1613us[+1624us] +/- 56ms 2024-05-09T06:30:11.464 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 -1120us[-1108us] +/- 79ms 2024-05-09T06:30:11.464 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -1188us[-1188us] +/- 66ms 2024-05-09T06:30:11.465 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T06:30:11.465 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T06:30:11.477 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T06:30:11.477 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T06:30:11.488 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T06:30:11.678 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T06:30:11.792 DEBUG:teuthology.orchestra.run.smithi195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T06:30:11.873 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:30:11.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:30:11.904 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:30:11.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:30:11.936 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T06:30:11.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:30:11.964 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T06:30:11.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T06:30:11.988 INFO:teuthology.task.internal:Duration was 4306.564150 seconds 2024-05-09T06:30:11.988 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T06:30:12.002 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T06:30:12.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T06:30:12.004 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T06:30:12.007 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T06:30:12.067 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:30:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:30:12.082 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T06:30:12.457 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T06:30:12.457 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-05-09T06:30:12.457 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T06:30:12.478 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-05-09T06:30:12.479 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T06:30:12.503 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-05-09T06:30:12.503 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T06:30:12.534 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T06:30:12.534 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T06:30:12.537 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T06:30:12.546 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T06:30:12.601 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T06:30:12.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T06:30:12.608 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T06:30:12.617 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T06:30:13.918 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T06:30:13.930 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T06:30:13.930 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T06:30:13.956 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T06:30:13.986 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T06:30:14.027 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T06:30:14.039 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:30:14.041 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:30:14.044 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:30:14.075 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-05-09T06:30:14.101 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-05-09T06:30:14.214 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-05-09T06:30:14.237 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:30:14.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:30:14.252 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:30:14.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:30:14.269 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T06:30:14.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T06:30:14.288 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T06:30:14.300 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T06:30:14.301 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/remote/smithi115 2024-05-09T06:30:14.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T06:30:14.375 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/remote/smithi161 2024-05-09T06:30:14.375 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T06:30:14.416 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120/remote/smithi195 2024-05-09T06:30:14.416 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T06:30:14.458 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T06:30:14.459 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T06:30:14.461 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T06:30:14.464 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T06:30:14.511 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T06:30:14.523 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T06:30:14.524 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T06:30:14.536 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T06:30:14.536 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T06:30:14.538 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T06:30:14.541 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T06:30:14.551 INFO:teuthology.orchestra.run.smithi115.stdout: 265307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 06:30 /home/ubuntu/cephtest 2024-05-09T06:30:14.556 INFO:teuthology.orchestra.run.smithi161.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 06:30 /home/ubuntu/cephtest 2024-05-09T06:30:14.569 INFO:teuthology.orchestra.run.smithi195.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 06:30 /home/ubuntu/cephtest 2024-05-09T06:30:14.570 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T06:30:14.582 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T06:30:14.596 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-05-09T06:30:14.596 INFO:teuthology.nuke:Checking targets against current locks 2024-05-09T06:30:14.622 DEBUG:teuthology.nuke:shortname: smithi115 2024-05-09T06:30:14.622 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T06:30:14.644 DEBUG:teuthology.nuke:shortname: smithi161 2024-05-09T06:30:14.644 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T06:30:14.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 05:09:43.333263', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwLRy3NJu0QMR2Wrvxm2Ky4rCTroTs+v6A6dIq8ejOghgc2RO1q4cepqkmnVWhKWBTGFdIZVygQClYIjKB8Io4='} 2024-05-09T06:30:14.670 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 05:09:43.331219', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJmAfhVVJZsHL0RPD8MpyMI6DMOLMj4lsfWG9XVvbDFiRL68GpNLWjpCsRb0e5M9Q6xdVb3oDsnNWq2E3mNEX6A='} 2024-05-09T06:30:14.671 DEBUG:teuthology.nuke:shortname: smithi195 2024-05-09T06:30:14.672 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T06:30:14.698 INFO:teuthology.orchestra.console.smithi115:Power off 2024-05-09T06:30:14.698 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T06:30:14.710 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699120', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 05:09:43.332511', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGYmgDvZ4wD6omeOjOM/UDHcKW+kBeWAQjQOII2hHZxS3AvuiuUyzfR8Ex3IsV3pCRmdEZXVcgc+njbCh7f23BQ='} 2024-05-09T06:30:14.723 DEBUG:teuthology.orchestra.console.smithi115:power off output: Chassis Power Control: Down/Off 2024-05-09T06:30:14.723 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:14.744 INFO:teuthology.orchestra.console.smithi161:Power off 2024-05-09T06:30:14.744 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T06:30:14.754 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-05-09T06:30:14.767 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2024-05-09T06:30:14.767 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:14.780 INFO:teuthology.orchestra.console.smithi195:Power off 2024-05-09T06:30:14.780 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T06:30:14.790 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-05-09T06:30:14.806 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2024-05-09T06:30:14.806 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:14.830 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-05-09T06:30:18.755 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:18.791 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:18.831 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:18.867 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-05-09T06:30:18.902 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-05-09T06:30:18.942 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-05-09T06:30:22.871 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:22.903 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:22.943 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:22.985 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-05-09T06:30:23.013 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-05-09T06:30:23.055 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2024-05-09T06:30:23.155 INFO:teuthology.orchestra.console.smithi195:Power off completed 2024-05-09T06:30:26.988 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:27.014 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T06:30:27.099 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off 2024-05-09T06:30:27.125 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-05-09T06:30:27.201 INFO:teuthology.orchestra.console.smithi115:Power off completed 2024-05-09T06:30:27.226 INFO:teuthology.orchestra.console.smithi161:Power off completed 2024-05-09T06:30:27.331 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 4306.564150094986 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-05-09T06:30:27.331 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T06:30:27.415 INFO:teuthology.run:FAIL